# 2021-11-15 Stabilized FE

## Last time

* Element restriction
* Finite element assembly
* Nonlinear problems
* LibCEED

## Today

* Community presentations
* Petrov-Galerkin stabilization
* Mass lumping for time dependent problems
* Examples of finite element interfaces

# Community presentations

## [Corey Murphey: SU2](https://github.com/cu-numpde/2021-community-clmurphey/raw/d8b8e0c1a3949ec32babe1cffd92eb851979a69a/2021-11-15-community-analysis-prezo/2021-community_analysis_prezo_CLM.pdf)

## Matthew Kolbe: QuantLib

## Tim Aiken: libCEED

```julia
using Plots
using LinearAlgebra
using SparseArrays
using FastGaussQuadrature

default(linewidth=4)

function my_spy(A)
    cmax = norm(vec(A), Inf)
    s = max(1, ceil(120 / size(A, 1)))
    spy(A, marker=(:square, s), c=:diverging_rainbow_bgymr_45_85_c67_n256, clims=(-cmax, cmax))
end

function vander_legendre_deriv(x, k=nothing)
    if isnothing(k)
        k = length(x) # Square by default
    end
    m = length(x)
    Q = ones(m, k)
    dQ = zeros(m, k)
    Q[:, 2] = x
    dQ[:, 2] .= 1
    for n in 1:k-2
        Q[:, n+2] = ((2*n + 1) * x .* Q[:, n+1] - n * Q[:, n]) / (n + 1)
        dQ[:, n+2] = (2*n + 1) * Q[:,n+1] + dQ[:,n]
    end
    Q, dQ
end

function febasis(P, Q, quadrature)
    x, _ = gausslobatto(P)
    q, w = quadrature(Q)
    Pk, _ = vander_legendre_deriv(x)
    Bp, Dp = vander_legendre_deriv(q, P)
    B = Bp / Pk
    D = Dp / Pk
    x, q, w, B, D
end

function fe1_mesh(P, nelem)
    x = LinRange(-1, 1, nelem+1)
    rows = Int[]
    cols = Int[]
    for i in 1:nelem
        append!(rows, (i-1)*P+1:i*P)
        append!(cols, (i-1)*(P-1)+1:i*(P-1)+1)
    end
    x, sparse(cols, rows, ones(nelem*P))'
end

function xnodal(x, P)
    nelem = length(x) - 1
    xn = Float64[]
    xref, _ = gausslobatto(P)
    for i in 1:nelem
        xL, xR = x[i:i+1]
        append!(xn, (xL+xR)/2 .+ (xR-xL)/2 * xref[1+(i>1):end])
    end
    xn
end

struct FESpace
    P::Int
    Q::Int
    nelem::Int
    x::Vector
    xn::Vector
    Et::Adjoint{Float64, SparseMatrixCSC{Float64, Int64}}
    q::Vector
    w::Vector
    B::Matrix
    D::Matrix
    function FESpace(P, Q, nelem; quadrature=gausslegendre)
        x, Et = fe1_mesh(P, nelem)
        xn = xnodal(x, P)
        _, q, w, B, D = febasis(P, Q, quadrature)
        new(P, Q, nelem, x, xn, Et, q, w, B, D)
    end
end

function fe_element(fe, e)
    xL, xR = fe.x[e:e+1]
    q = (xL+xR)/2 .+ (xR-xL)/2*fe.q
    w = (xR - xL)/2 * fe.w
    E = fe.Et'[:, (e-1)*fe.P+1:e*fe.P]'
    dXdx = ones(fe.Q) * 2 / (xR - xL)
    q, w, E, dXdx
end
```

    fe_element (generic function with 1 method)

# Finite element residual assembly
$$ v^T F(u) \sim \int_\Omega v\cdot f_0(u, \nabla u) + \nabla v\cdot f_1(u, \nabla u) = 0, \quad \forall v$$
$$ \sum_e \mathcal E_e^T \Big( B^T W \left\lvert \frac{\partial x}{\partial X} \right\rvert f_0(\tilde u, \nabla \tilde u) + D^T \left(\frac{\partial X}{\partial x}\right)^{T} W \left\lvert \frac{\partial x}{\partial X} \right\rvert f_1(\tilde u, \nabla\tilde u) \Big) = 0 $$
where $\tilde u = B \mathcal E_e u$ and $\nabla \tilde u = \frac{\partial X}{\partial x} D \mathcal E_e u$ are the values and gradients evaluated at quadrature points.

```julia
kappa(x) = 0.6 .+ 0.4*sin(pi*x/2)
fq(q, u, Du) = 0*u .- 1, kappa.(q) .* Du
dfq(q, u, du, Du, Ddu) = 0*du, kappa.(q) .* Ddu

function fe_residual(u_in, fe, fq; bci=[1], bcv=[1.])
    u = copy(u_in); v = zero(u)
    u[bci] = bcv
    for e in 1:fe.nelem
        q, w, E, dXdx = fe_element(fe, e)
        B, D = fe.B, fe.D
        ue = E * u
        uq = B * ue
        Duq = dXdx .* (D * ue)
        f0, f1 = fq(q, uq, Duq)
        ve = B' * (w .* f0) + D' * (dXdx .* w .* f1)
        v += E' * ve
    end
    v[bci] = u_in[bci] - u[bci]
    v
end
```

    fe_residual (generic function with 1 method)

```julia
import NLsolve: nlsolve

fe = FESpace(3, 3, 5)
u0 = zero(fe.xn)
sol = nlsolve(u -> fe_residual(u, fe, fq), u0; method=:newton)
plot(fe.xn, sol.zero, marker=:auto)
```

# Advection-diffusion (time independent)

\begin{align}
\mathbf w \cdot \nabla u - \nabla\cdot\big[ \kappa \nabla u \big] &= s & \int_\Omega v \mathbf w \cdot \nabla u + \int_\Omega \nabla v \cdot \kappa \nabla u  &= \int_\Omega v s, \forall v
\end{align}

```julia
wind = 100
fq(q, u, Du) = wind .* Du -one.(u), 1 * Du

fe = FESpace(5, 5, 10)
u0 = zero(fe.xn)
N = length(fe.xn)
sol = nlsolve(u -> fe_residual(u, fe, fq; bci=[1, N], bcv=[0, 0]), zero(fe.xn), method=:newton)
plot(fe.xn, sol.zero, marker=:auto, legend=:none)
```

# Artificial diffusion and Streamline Upwinding

Observation: the residual is large where diffusion is needed.

$$\int_\Omega v \mathbf w \cdot \nabla u + \int_\Omega \nabla v \cdot \kappa \nabla u - \int_\Omega v s + \int_\Omega \tau^e (\mathbf w \cdot \nabla v) \big(\mathbf w \cdot \nabla u - \nabla\cdot(\kappa\nabla u) - s\big)$$

## Examine what this does to advection

$$(\mathbf w \cdot \nabla v)(\mathbf w \cdot \nabla u) = \nabla v (\mathbf w \otimes \mathbf w) \nabla u$$

This is "pencil" shaped diffusion, only along the streamline. If $\tau^e$ is chosen appropriately, this will be enough diffusion to get a Peclet number of about 1 when it needs it.

## Optimal stabilization
A nodally exact solution for 1D advection.
$$ \tau^e = \frac{h}{2 \lVert \mathbf w \rVert} \Big( \coth Pe - \frac{1}{Pe} \Big)$$

```julia
wind = 10; k = 1
fq(q, u, Du) = wind .* Du -one.(u), k * Du + tau * wind.^2 .* Du

n = 30; h = 2 / n; Pe = abs(wind) * h / (2 * k)
tau = h / (2 * abs(wind)) * (coth(Pe) - 1 / Pe)
fe = FESpace(2, 2, n)
u0 = zero(fe.xn)
N = length(fe.xn)
sol = nlsolve(u -> fe_residual(u, fe, fq; bci=[1, N], bcv=[0, 0]), zero(fe.xn), method=:newton)
plot(fe.xn, sol.zero, marker=:auto, legend=:none)
```

# Time dependent problems

1. Start with a time dependent problem in strong form
$$ u_t + \nabla\cdot (\mathbf w u - \kappa \nabla u) = 0 .$$
2. Multiply by a test function $v$ and integrate
$$ \int_\Omega \Big[ v u_t + \nabla v\cdot \big(\kappa \nabla u - \mathbf w u \big) - vs \Big] = 0, \forall v.$$
3. Discretize and assemble
$$ M u_t + A u - s = 0$$
4. Convert to explicit ODE form
$$ u_t = M^{-1} (-A u + s). $$

* Mass matrix $M$ has the same sparsity pattern as the physics $A$ -- direct solve costs the same.
  * Finite element methods must be explicit?
* $M$ is usually much better conditioned than $A$; solve in less than 10 CG iterations with Jacobi preconditioning.
* Replace $M$ with a diagonal approximation
  * $\operatorname{diag}(M)$ is inconsistent
  * row sums = "lumping"
  * use collocated Lobatto quadrature

# Collocated quadrature

```julia
dfq(q, u, du, Du, Ddu) = du, 0*Ddu

function fe_jacobian(u_in, fe, dfq; bci=[1], bcv=[1.])
    u = copy(u_in); u[bci] = bcv
    rows, cols, vals = Int[], Int[], Float64[]
    for e in 1:fe.nelem
        q, w, E, dXdx = fe_element(fe, e)
        B, D, P = fe.B, fe.D, fe.P
        ue = E * u
        uq = B * ue; Duq = dXdx .* (D * ue)
        K = zeros(P, P)
        for j in 1:fe.P
            du = B[:,j]
            Ddu = dXdx .* D[:,j]
            df0, df1 = dfq(q, uq, du, Duq, Ddu)
            K[:,j] = B' * (w .* df0) + D' * (dXdx .* w .* df1)
        end
        inds = rowvals(E')
        append!(rows, kron(ones(P), inds))
        append!(cols, kron(inds, ones(P)))
        append!(vals, vec(K))
    end
    A = sparse(rows, cols, vals)
    A[bci, :] .= 0; A[:, bci] .= 0
    A[bci,bci] = diagm(ones(length(bci)))
    A
end
```

    fe_jacobian (generic function with 1 method)

```julia
fe = FESpace(6, 6, 4, gausslobatto)
u0 = zero(fe.xn)
J1 = fe_jacobian(u0, fe, dfq, bci=[], bcv=[])
fe = FESpace(6, 6, 4, gausslegendre)
J2 = fe_jacobian(u0, fe, dfq, bci=[], bcv=[])
@show norm(J1 - diagm(diag(J1)))
@show norm(J2 - diagm(diag(J2)))
@show norm(sum(J2, dims=2) - diag(J1))
my_spy(J2)
```

    norm(J1 - diagm(diag(J1))) = 1.0314997307155373e-16
    norm(J2 - diagm(diag(J2))) = 0.07988743228160039
    norm(sum(J2, dims = 2) - diag(J1)) = 4.638301540976035e-16

# SUPG for time-dependent problems

$$\int_\Omega v \big( u_t + \mathbf w \cdot \nabla u \big) + \int_\Omega \nabla v \cdot \kappa \nabla u - \int_\Omega v s + \int_\Omega \tau^e (\mathbf w \cdot \nabla v) \big(u_t + \mathbf w \cdot \nabla u - \nabla\cdot(\kappa\nabla u) - s\big)$$

* There is a $u_t$ term in the stabilization, tested by a gradient of the test function. This means we can't create a simple explicit form
$$u_t = M^{-1} F(u)$$
* Some ad-hoc methods treat this term explicitly, which effectively lags this term. It can work, but limits the choice of time integrator and affects order of accuracy in time.
* One can use fully implicit methods with this formulation, usually written as $G(\dot u, u, t) = 0$. Generalized alpha (a second order scheme that can move continuously between midpoint and BDF2, which is L-stable) methods are popular.

* There is a strong  form $$\nabla\cdot(\kappa\nabla u)$$ appearing in stabilization.
* For linear elements, this is typically zero on each element.
  * Ignore the term (popular) or reconstruct it using neighbor cells (implementation complexity) and/or projection (extra work/communication).
* High order elements
  * If $\kappa$ is constant, $\kappa \nabla\cdot\nabla u$ can be implemented using second derivatives of the basis functions.
  * Or via projection
  * Should $\tau^e$ be constant or variable over the element?


### Example 1b: Linear convection in 2D, revisited

We will now revisit the first example of this tutorial with an example that is better suited to the numerical scheme used in Devito. As a reminder, the governing equation is:

$$\frac{\partial u}{\partial t}+c\frac{\partial u}{\partial x} + c\frac{\partial u}{\partial y} = 0$$

We then discretized this using forward differences in time and backward differences in space:

$$u_{i,j}^{n+1} = u_{i,j}^n-c \frac{\Delta t}{\Delta x}(u_{i,j}^n-u_{i-1,j}^n)-c \frac{\Delta t}{\Delta y}(u_{i,j}^n-u_{i,j-1}^n)$$

In the previous example, the system was initialised with a hat function. As easy as this example seems, it actually highlights a few limitations of finite differences and related methods:
 - The governing equation above contains spatial derivatives ($\frac{\partial u}{\partial x}$ and $\frac{\partial u}{\partial y}$). The hat, with its sharp corners, is discontinuous and therefore non-smooth, meaning that the derivatives do not exist at the corners of the hat. This means that the governing equation has no solution in the strict sense for this problem. Mathematically, this problem can be overcome by introducing weak solutions, which still exist in the presence of discontinuities, as long as the problem is smooth almost everywhere. The Finite Volume (FV), Finite Element (FEM) and related schemes are based on this weak form.
 - The finite differences method only works well if finite differences are a good approximation of the derivatives. With the chosen discretization above, this requires that $\frac{u_{i,j}^n-u_{i-1,j}^n}{\Delta x} \approx \frac{\partial u}{\partial x}$ and $\frac{u_{i,j}^n-u_{i,j-1}^n}{\Delta y} \approx \frac{\partial u}{\partial y}$. This is the case for systems with a smooth solution if $\Delta x$ and $\Delta y$ are sufficiently small. But if the solution is non-smooth, as in this example, then we can't expect much regardless of the grid size.
 - First-order methods, such as the backward differences that we have used in this example, are known to create artificial diffusion. Higher-order schemes, such as central differences, avoid this problem. However, in the presence of discontinuities these methods introduce so-called spurious oscillations. These oscillations may even build up (grow infinitely) and cause the computation to diverge.
 - Discontinuities can appear by themselves for some equations (such as the Burgers equation that we discuss next), even if the initial condition is smooth. In CFD, discontinuities appear for example as shocks in the simulation of transonic flow. For this reason, numerical schemes that behave well in the presence of discontinuities have been a research subject for a long time. A thorough discussion is beyond the scope of this tutorial, but can be found in [R. LeVeque (1992): Numerical Methods for Conservation Laws, 2nd ed., Birkhäuser Verlag, pp. 8-13].

In the remainder of this example, we will reproduce the results from the previous example, only this time with a smooth initial condition. This lets us observe Devito in a setting for which it is better equipped.

```python
from examples.cfd import plot_field, init_hat, init_smooth
import numpy as np
%matplotlib inline

# Some variable declarations
nx = 81
ny = 81
nt = 100
c = 1.
dx = 2. / (nx - 1)
dy = 2. / (ny - 1)
sigma = .2
dt = sigma * dx
```

Let us now initialise the field with an infinitely smooth bump, as given by [J.A. Krakos (2012): Unsteady Adjoint Analysis for Output Sensitivity
and Mesh Adaptation, PhD thesis, p. 68] as $$
f(r)=
\begin{cases}
\frac{1}{A}e^{-1/(r-r^2)} &\text{ for } 0 < r < 1,\\
0 &\text{ else.}
\end{cases}
$$
We use this with $A=100$, and define the initial condition in two dimensions as $$u^0(x,y)=1+f\left(\frac{2}{3}x\right)*f\left(\frac{2}{3}y\right).$$

```python
#NBVAL_IGNORE_OUTPUT

# Create field and assign initial conditions
u = np.empty((nx, ny))
init_smooth(field=u, dx=dx, dy=dy)

# Plot initial condition
plot_field(u, zmax=4)
```

Solving this will move the bump again.

```python
# Repeat initialisation, so we can re-run the cell
init_smooth(field=u, dx=dx, dy=dy)

for n in range(nt + 1):
    # Copy previous result into a new buffer
    un = u.copy()

    # Update the new result with a 3-point stencil
    u[1:, 1:] = (un[1:, 1:] - (c * dt / dy * (un[1:, 1:] - un[1:, :-1])) -
                              (c * dt / dx * (un[1:, 1:] - un[:-1, 1:])))

    # Apply boundary conditions.
    # Note: -1 here is the last index in the array, not the one at x=-1 or y=-1.
    u[0, :] = 1.  # left
    u[-1, :] = 1. # right
    u[:, 0] = 1.  # bottom
    u[:, -1] = 1. # top
```

```python
#NBVAL_IGNORE_OUTPUT

# A small sanity check for auto-testing
assert (u[45:55, 45:55] > 1.8).all()
u_ref = u.copy()

plot_field(u, zmax=4.)
```

Hooray, the wave moved! It looks like the solver works much better for this example: The wave has not noticeably changed its shape.

#### Devito implementation
Again, we can re-create this via a Devito operator. Let's fill the initial buffer with smooth data and look at it:

```python
#NBVAL_IGNORE_OUTPUT
from devito import Grid, TimeFunction

grid = Grid(shape=(nx, ny), extent=(2., 2.))
u = TimeFunction(name='u', grid=grid)
init_smooth(field=u.data[0], dx=dx, dy=dy)

plot_field(u.data[0])
```

We create again the discretized equation as shown below. Note that the equation is still the same, only the initial condition has changed.

```python
from devito import Eq

eq = Eq(u.dt + c*u.dxl + c*u.dyl)

print(eq)
```

    Eq(1.0*Derivative(u(t, x, y), x) + 1.0*Derivative(u(t, x, y), y) + Derivative(u(t, x, y), t), 0)

SymPy can re-organise this equation just like in the previous example.

```python
from devito import solve
from sympy import nsimplify

stencil = solve(eq, u.forward)

print(nsimplify(stencil))
```

    dt*(-u(t, x, y)/h_y + u(t, x, y - h_y)/h_y - u(t, x, y)/h_x + u(t, x - h_x, y)/h_x + u(t, x, y)/dt)

We can now use this stencil expression to create an operator to apply to our data object:

```python
#NBVAL_IGNORE_OUTPUT
from devito import Operator

# Reset our initial condition in both buffers.
# This is required to avoid 0s propagating into
# our solution, which has a background value of 1.
init_smooth(field=u.data[0], dx=dx, dy=dy)
init_smooth(field=u.data[1], dx=dx, dy=dy)

# Apply boundary conditions.
# Note that as the u.data method is from numpy, we can use the
#  -1 syntax to represent the last item in the array.
u.data[:, 0, :] = 1.
u.data[:, -1, :] = 1.
u.data[:, :, 0] = 1.
u.data[:, :, -1] = 1.

# Create an Operator that updates the forward stencil
# point in the interior subdomain only.
op = Operator(Eq(u.forward, stencil, subdomain=grid.interior))

# Apply the operator for a number of timesteps
op(time=nt, dt=dt)

plot_field(u.data[0])

# Some small sanity checks for the testing framework
assert (u.data[0, 45:55, 45:55] > 1.8).all()
assert np.allclose(u.data[0], u_ref, rtol=3.e-2)
```

Again, this looks just like the result from NumPy. Since this example is just like the one before, the low-level treatment of boundaries is also unchanged.

```python
#NBVAL_IGNORE_OUTPUT

# Reset our data field and ICs in both buffers
init_smooth(field=u.data[0], dx=dx, dy=dy)
init_smooth(field=u.data[1], dx=dx, dy=dy)

# For defining BCs, we generally to explicitly set rows/columns
# in our field using an expression. We can use Devito's "indexed"
# notation to do this. A u in this instance is a sympy function
# we cannot use the numpy shortcut u[-1] to refer to the last location:
x, y = grid.dimensions
t = grid.stepping_dim
bc_left = Eq(u[t + 1, 0, y], 1.)
bc_right = Eq(u[t + 1, nx-1, y], 1.)
bc_top = Eq(u[t + 1, x, ny-1], 1.)
bc_bottom = Eq(u[t + 1, x, 0], 1.)

# Now combine the BC expressions with the stencil to form operator.
expressions = [Eq(u.forward, stencil, subdomain=grid.interior)]
expressions += [bc_left, bc_right, bc_top, bc_bottom]
op = Operator(expressions=expressions, opt=None, openmp=False)  # <-- Turn off performance optimisations
op(time=nt, dt=dt)

plot_field(u.data[0])

# Some small sanity checks for the testing framework
assert (u.data[0, 45:55, 45:55] > 1.8).all()
assert np.allclose(u.data[0], u_ref, rtol=3.e-2)
```

The C code of the Kernel is also still the same.

```python
print(op.ccode)
```

    #define _POSIX_C_SOURCE 200809L
    #include "stdlib.h"
    #include "math.h"
    #include "sys/time.h"

    struct dataobj
    {
      void *restrict data;
      int * size;
      int * npsize;
      int * dsize;
      int * hsize;
      int * hofs;
      int * oofs;
    } ;

    struct profiler
    {
      double section0;
      double section1;
      double section2;
    } ;

    int Kernel(const float dt, const float h_x, const float h_y, struct dataobj *restrict u_vec, const int i0x_ltkn, const int i0x_rtkn, const int i0y_ltkn, const int i0y_rtkn, const int time_M, const int time_m, struct profiler * timers, const int x_M, const int x_m, const int y_M, const int y_m)
    {
      float (*restrict u)[u_vec->size[1]][u_vec->size[2]] __attribute__ ((aligned (64))) = (float (*)[u_vec->size[1]][u_vec->size[2]]) u_vec->data;

      for (int time = time_m, t0 = (time)%(2), t1 = (time + 1)%(2); time <= time_M; time += 1, t0 = (time)%(2), t1 = (time + 1)%(2))
      {
        struct timeval start_section0, end_section0;
        gettimeofday(&start_section0, NULL);
        /* Begin section0 */
        for (int i0x = i0x_ltkn + x_m; i0x <= -i0x_rtkn + x_M; i0x += 1)
        {
          for (int i0y = i0y_ltkn + y_m; i0y <= -i0y_rtkn + y_M; i0y += 1)
          {
            u[t1][i0x + 1][i0y + 1] = dt*(1.0F*u[t0][i0x + 1][i0y]/h_y - 1.0F*u[t0][i0x + 1][i0y + 1]/h_y + 1.0F*u[t0][i0x][i0y + 1]/h_x - 1.0F*u[t0][i0x + 1][i0y + 1]/h_x + u[t0][i0x + 1][i0y + 1]/dt);
          }
        }
        /* End section0 */
        gettimeofday(&end_section0, NULL);
        timers->section0 += (double)(end_section0.tv_sec-start_section0.tv_sec)+(double)(end_section0.tv_usec-start_section0.tv_usec)/1000000;
        struct timeval start_section1, end_section1;
        gettimeofday(&start_section1, NULL);
        /* Begin section1 */
        for (int y = y_m; y <= y_M; y += 1)
        {
          u[t1][1][y + 1] = 1.00000000000000F;
          u[t1][81][y + 1] = 1.00000000000000F;
        }
        /* End section1 */
        gettimeofday(&end_section1, NULL);
        timers->section1 += (double)(end_section1.tv_sec-start_section1.tv_sec)+(double)(end_section1.tv_usec-start_section1.tv_usec)/1000000;
        struct timeval start_section2, end_section2;
        gettimeofday(&start_section2, NULL);
        /* Begin section2 */
        for (int x = x_m; x <= x_M; x += 1)
        {
          u[t1][x + 1][81] = 1.00000000000000F;
          u[t1][x + 1][1] = 1.00000000000000F;
        }
        /* End section2 */
        gettimeofday(&end_section2, NULL);
        timers->section2 += (double)(end_section2.tv_sec-start_section2.tv_sec)+(double)(end_section2.tv_usec-start_section2.tv_usec)/1000000;
      }
      return 0;
    }

```python

```


# Computer lab 2 - Automatic control 2

$ \newcommand{\mexp}[1]{\mathrm{e}^{#1}} $
$ \newcommand{\transp}{ ^{\mathrm{T}} }$

## Preparations
### Exercise 1 - the spectral factorization theorem
Determine a filter
\begin{equation}
H(z) = \frac{b}{z+a}
\end{equation}
That generates a signal with spectral density
\begin{equation}
\phi(\omega) = \frac{0.75}{1.25 - cos\omega}
\end{equation}

Hint: Set up state space model and use the Lyaponov equation to find the variance.

\begin{align}
x(k+1) &= -a x(k) + bu(k)\\
y(k) &= x(k)
\end{align}

The covariance
\begin{equation}
P(k) = E\tilde{x}(k)\tilde{x}\transp(k) = E \big(x(k) - m(k)\big)\big(x^{\mathrm{T}}(k) - m^{\mathrm{T}}(k)\big)
\end{equation}
is governed by the difference equation
\begin{equation}
P(k+1) = \Phi P(k) \Phi^{\mathrm{T}} + b^2R_1 = a^2P(k) + b^2,
\end{equation}
with initial condition
\begin{equation}
P(0) = r_0.
\end{equation}
The solution is
\begin{align}
P(1) &= a^2r_0 + b^2\\
P(2) &= a^2P(1) + b^2 = a^4r_0 + a^2b^2 + b^2\\
P(3) &= a^2P(2) + b^2 = a^6r_0 + a^4b^2 + a^2b^2 + b^2 = a^6r_0 + (1 + a^2 + a^4)b^2\\
    & \vdots\\
P(k) &= a^{2k}r_0 + \big(1 + a^2 + \cdots + a^{2(k-1)} \big)b^2 = a^{2k}r_0 + \frac{1-a^{2k}}{1-a^2}b^2,
\end{align}
where we have made use of the properites of finite geomtric sums in the last equality.

For stable system ($|a| < 1$) we get
\begin{equation}
P(k) \to \frac{b^2}{1-a^2}
\end{equation}

The covariance function for $x$ is
\begin{equation}
r_x(k+\tau,k) = E\tilde{x}(k+\tau)\tilde{x}\transp(k) = a^\tau P(k),
\end{equation}
hence,
\begin{equation}
r_x(k+\tau,k) \to \frac{b^2a^{|\tau|}}{1-a^2}
\end{equation}
Hence, the variance is
\begin{equation}
\mathrm{Var} x = r(0) = \frac{b^2}{1-a^2}.
\end{equation}

Identify the parameters $a$ and $b$:

Note that the spectal density is
\begin{equation}
\phi(\omega) 2\pi = H(\mexp{i\omega})H(\mexp{-i\omega}) \phi_u(\omega) = \frac{b^2}{\big(\mexp{i\omega}+a\big)\big(\mexp{-i\omega}+a\big)} = \frac{b^2}{1 + a\mexp{i\omega}+a\mexp{-i\omega} + a^2} = \frac{b^2}{1+a^2+2acos\omega} = \frac{\frac{b^2}{-2a}}{\frac{1+a^2}{-2a} - cos\omega}.
\end{equation}
Setting coefficients equal in this expression and the given spectral density gives
\begin{align}
\frac{b^2}{-2\pi2a} &= \frac{3}{4} \quad \Rightarrow \quad b^2 = -3a\pi\\
\frac{1+a^2}{-2a} &= \frac{5}{4} \quad \Rightarrow \quad a^2 + \frac{5}{2}a + 1 = 0
\end{align}
so,
\begin{align}
a &= \begin{cases} -\frac{5}{4} + \frac{1}{2}\sqrt{\frac{25}{4}-4} = -\frac{1}{2} & \text{stable}\\
                   -\frac{5}{4} - \frac{1}{2}\sqrt{\frac{25}{4}-4} = -2 & \text{unstable}\\
                   \end{cases}\\
                   b &= \sqrt{\frac{3\pi}{2}}.
\end{align}

```python
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import control.matlab as cm
import sympy as sy
from scipy import signal
```

```python
def d_spectrum(A, B, sigma2):
    lgw1 = -2
    w = np.logspace(lgw1,np.log10(np.pi), 300)
    (wp,Hp) = signal.freqz(B,A,w)
    wm = -w
    (wm,Hm) = signal.freqz(B,A,wm)
    return (w, sigma2*Hp*Hm/(2*np.pi))
def lab2a(a,b):
    A = [1, a]
    B = b
    # Compute spectrum
    (w_ab, fi_ab)=d_spectrum(A,B,1)
    # True spectrum
    fi_0 = 0.75 / (1.25-np.cos(w_ab))

    # Plot spectra
    plt.loglog(w_ab, fi_ab, 'r')
    plt.loglog(w_ab, fi_0, '--k')
    plt.title('Output spectrum')
    plt.xlabel('\omega')
    plt.legend('Model', 'Correct solution')
a = -0.5
b = np.sqrt(3*np.pi/2)

lab2a(a,b)
```

## Preparation exercise 3
Covariance function of MA(1) process
\begin{align}
x(k+1) &= \begin{bmatrix} 0 & 0 \\ c & 0 \end{bmatrix} x(k) + \begin{bmatrix}1\\1\end{bmatrix} \epsilon(k)\\
y(k) &= \begin{bmatrix} 0 & 1 \end{bmatrix} x(k) = x_2(k)
\end{align}
with $\epsilon(k)$ being a sequence of white noise with unit variance and zero mean.

The state covariance $r_x(k) = P(k) = \mathrm{E}x(k)x\transp(k)$ is governed by the equation
\begin{equation}
P(k+1) = \Phi P(k) \Phi\transp + R_1.
\end{equation}
In steady state we have $P(k+1) = P(k)=P$, so $P$ can be found by solving the Lyaponov equation
\begin{equation}
P = \Phi P \Phi\transp + R_1,
\end{equation}
where, in the case here
\begin{equation}
R_1 = \begin{bmatrix}1\\1\end{bmatrix} \begin{bmatrix}1 & 1\end{bmatrix} = \begin{bmatrix} 1 & 1\\1 & 1\end{bmatrix}
\end{equation}

According to [wikipedia on the Lyaponov equation](https://en.wikipedia.org/wiki/Lyapunov_equation), the solution can be written as an infinite sum
\begin{equation}
P = \sum_{k=0}^{\infty} \Phi^k R_1 \big(A\transp\big)^k.
\end{equation}
Here we have
\begin{align}
\Phi &= \begin{bmatrix} 0 & 0 \\ c & 0 \end{bmatrix}\\
\Phi^2 &= \begin{bmatrix} 0 & 0 \\ c & 0 \end{bmatrix}\begin{bmatrix} 0 & 0 \\ c & 0 \end{bmatrix} = \begin{bmatrix} 0 & 0\\0 & 0\end{bmatrix}\\
\Phi^3 &= 0\\
&\vdots
\end{align}
This gives a very short series:
\begin{equation}
P = R_1 + \begin{bmatrix} 0 & 0 \\ c & 0 \end{bmatrix}R_1\begin{bmatrix} 0 & c \\ 0 & 0 \end{bmatrix} = R_1 + \begin{bmatrix} 0 & 0\\0 & c^2\end{bmatrix}
\end{equation}
See verification below

```python
c = sy.symbols('c')
Phi = sy.Matrix([[0,0],[c,0]])
Phi2 = Phi*Phi
print Phi2
R1 = sy.Matrix([[1,1],[1,1]])
P = R1 + Phi*R1*Phi.T
P
```

    Matrix([[0, 0], [0, 0]])

    Matrix([
    [1,        1],
    [1, c**2 + 1]])

The variance of $y$ is the variance of $x_2$, so
\begin{equation}
\mathrm{var}\; y = 1 + c^2.
\end{equation}

To get the covariance function, note that
\begin{equation}
r_x(k+\tau,k) = \mathrm{E} x(k+\tau)x\transp = \mathrm{E}\big(\Phi^\tau x(k) + \Gamma \epsilon(k+\tau) + \Phi B \epsilon(k+\tau-1) + \Phi^2B \epsilon(k+\tau-2) + \cdots + \Phi^\tau B \epsilon(k) \big)x\transp(k) = \Phi^\tau P(k),
\end{equation}
since $x(k)$ is independent of future noise terms.
Since $\Phi^k = 0$ for $k>1$, we have

\begin{align}
r_x(0) &= P = \begin{bmatrix} 1 & 1\\1 & 1+c^2\end{bmatrix}\\
r_x(1) &= \Phi P = \begin{bmatrix} 0 & 0\\c & c \end{bmatrix}\\
r_x(\tau) &= 0, \; \tau>1
\end{align}

## Exercise 2
Simulate ARMA(1) process

```python
a = np.arange(24)
print len(a[2:])
print len(a[:-2])
print np.cov(a[2:], a[:-2])
```

    22
    22
    [[ 42.16666667  42.16666667]
     [ 42.16666667  42.16666667]]

```python
import pdb
def xcov(a,b,nlags):
    """ Cross correlation function for given number of lags. Works for 1-dimensional vectors a and b
        OBS: Unlike matlabs xcov function, this does not compute values for negative lags (assumes) symmetric
        function.
    """
    aa = a.ravel()
    bb = b.ravel()
    return np.array( [np.cov(aa,aa)] + [np.cov(aa[:-tau],bb[tau:]) for tau in range(1,nlags) ] )

def lab2b(C,A,N=100,tau_max=50, nr=1):
    lam = 1.0
    a = A[1]
    c = C[1]
    tau = np.arange(tau_max)
    e = np.random.normal(0,1,(N,nr))
    sys = (C, A, 1)
    r_e = []
    for i in range(nr):
        (t_out, yn) = signal.dlsim(sys, e[:,i])
        cv = xcov(yn,yn, tau_max)
        r_e.append(cv[:,1,0])
    r_e = np.array(r_e)
    NN = min(N,50)

    r_t = lam/(1-a**2) * np.hstack( (1+c**2-2*a*c, (c-a)*(1-a*c)*(-a)**(tau[1:]-1) ) )

    m = np.mean(r_e, axis=0)
    st = np.std(r_e, axis=0)

    #1/0
    plt.plot(tau, r_t, 'r-')
    plt.plot(tau, r_e.T, 'b--')
    plt.legend(['True', 'Estimated'])
    plt.xlabel('\tau')

#lab2b([1, 0], [1, -0.4], N=400, nr=4)
lab2b([1, 0.5], [1, 0], N=400, nr=4)

```

```python
pdb.pm()
```

    > <ipython-input-30-e7a3b6b244f2>(31)lab2b()
    -> 1/0
    (Pdb) r_e.shape
    (1, 50)
    (Pdb) cv.shape
    (50, 2, 2)
    (Pdb) cv[:4,:,:]
    array([[[ 5.15203502,  5.15203502],
            [ 5.15203502,  5.15203502]],

           [[ 5.15240258,  4.6261038 ],
            [ 4.6261038 ,  5.15243297]],

           [[ 5.15229716,  4.15816722],
            [ 4.15816722,  5.15292142]],

           [[ 5.15215686,  3.71376601],
            [ 3.71376601,  5.15336661]]])
    (Pdb) q

## Exercise 3
Pole-placement for observer

Discrete time system
\begin{align}
x(t+1) &= Fx(t) + Gu(t) + Nv_1(t)\\
y(t) &= Hx(t) + v_2(t),
\end{align}
with
\begin{align}
F &= \begin{bmatrix} 0.3 & -0.5\\ 1 & 0\end{bmatrix},\\
G &= \begin{bmatrix} 1\\0\end{bmatrix},\\
H &= \begin{bmatrix} 0 & 1\end{bmatrix}, \\
N &= \begin{bmatrix} 1\\ 1 \end{bmatrix}
\end{align}

Consider observer
\begin{equation}
 \hat{x}(t+1) = F\hat{x}(t) + Gu(t) + K\big(y - H\hat{x}(t)\big),
\end{equation}
with state estimation error $\tilde{x}(t) = x(t) - \hat(x)(t)$ governed by
\begin{equation}
 \tilde{x}(t+1) = \big(F-KH\big)\tilde{x}(t) + Nv_1(t) - Kv_2(t).
\end{equation}

Find observer gain $K$ so that the poles of the observer are placed in
\begin{equation}
p = -0.4 \pm 0.3j
\end{equation}

```python
F = np.array([[0.3, -0.5], [1, 0]])
G = np.array([[1],[0]])
H = np.array([[0, 1]])
N = np.array([[1],[1]])
K = cm.place(F.T, H.T, [-0.4+0.3*1j, -0.4-0.3*1j]).T
print K
np.linalg.eig(F-np.dot(K,H))
```

    [[ 0.08]
     [ 1.1 ]]

    (array([-0.4+0.3j, -0.4-0.3j]),
     array([[ 0.55689010+0.23866719j,  0.55689010-0.23866719j],
            [ 0.79555728+0.j        ,  0.79555728-0.j        ]]))

The covariance matrix of the state estimation error is given by the solution to the Lyaponov equation
\begin{equation}
P = APA\transp + NN\transp \sigma_1^2 + KK\transp \sigma_2^2,
\end{equation}
which can be written
\begin{equation}
APA\transp - P + Q = 0
\end{equation}
where
\begin{equation}
Q = NN\transp \sigma_1^2 + KK\transp \sigma_2^2
\end{equation}

```python
sigma_1 = 1
sigma_2 = 0.5

Ao = F - np.dot(K,H)
Qo = np.dot(N,N.T)*sigma_1**2 + np.dot(K,K.T)*sigma_2**2
Xo = cm.dlyap(Ao, Qo)
print Xo
```

    [[ 1.25764228  1.33617886]
     [ 1.33617886  1.80691057]]

Find oserver gain for a steady-state Kalman filter instead.

Note that in the Kalman filter we have the recursions
\begin{align}
\bar{P}_k &= FP_{k-1}F\transp + Q\\
P_k &= (1-K_kH)\bar{P}_k,
\end{align}
with Kalman gain
\begin{equation}
K_k = \bar{P}_kH\transp\big(H\bar{P}_kH\transp + R\big)^{-1}.
\end{equation}
Here $Q=NN\transp \sigma_1^2$ is the covariance matrix of the process noise.

In steady state we have $\bar{P}_{k+1}= \bar{P}_k = \bar{P}$, so the two recursions can be combined to yield the algebraic Riccatti equation

\begin{equation}
\bar{P} = F\bar{P}F\transp + Q - F\bar{P}H\transp\big(H\bar{P}H\transp + R\big)^{-1}H\bar{P}F\transp.
\end{equation}

The solution to this can then be used to solve for the steady state Kalman gain

Then compute Kalman gain as
\begin{equation}
K = F \bar{P}H\transp\big(H\bar{P}H\transp + R\big)^{-1},
\end{equation}
where $R=\sigma_2^2$ is the covariance matrix of the measurement noise.

Now the Kalman update on observer form is given by
\begin{equation}
\hat{x}(t+1) = F\hat{x}(t) + Gu(t) + K\big(y(t) - H\hat{x}(t)\big)
\end{equation}

```python
A = F.T
B = H.T
(X,L,G) = cm.dare(F.T, H.T, sigma_1**2*np.dot(N,N.T), sigma_2**2)
K = np.dot(F,G.T)
Kk = np.dot(X, np.dot(H.T, np.linalg.inv(np.dot(H, np.dot(X, H.T)) + sigma_2**2) ))
Gtest = np.dot( np.linalg.inv( np.dot(B.T, np.dot(X, B) ) + sigma_2**2 ), np.dot(B.T, X) )
print Kk
print G
print Gtest
print np.dot(F,Kk)
print np.linalg.eig(F-np.dot(Kk, H))
print np.linalg.eig(A-np.dot(B,G))
print L

Ak = F - np.dot(G.T,H)
Qk = np.dot(N,N.T)*sigma_1**2 + np.dot(Kk,Kk.T)*sigma_2**2
Xk = cm.dlyap(Ak, Qk)
print Xk
print Xo
print X

```

    [[ 0.63417447]
     [ 0.8469135 ]]
    [[-0.23320441  0.63417447]]
    [[ 0.63417447  0.8469135 ]]
    [[-0.23320441]
     [ 0.63417447]]
    (array([-0.27345675+0.89739725j, -0.27345675-0.89739725j]), array([[ 0.72899571+0.j       ,  0.72899571-0.j       ],
           [ 0.36859189-0.5768061j,  0.36859189+0.5768061j]]))
    (array([-0.16708724+0.22051101j, -0.16708724-0.22051101j]), matrix([[ 0.88847810+0.j       ,  0.88847810-0.j       ],
            [-0.41499678+0.1959192j, -0.41499678-0.1959192j]]))
    [-0.16708724+0.22051101j -0.16708724-0.22051101j]
    [[ 1.11076086  1.16498821]
     [ 1.16498821  1.35903982]]
    [[ 1.25764228  1.33617886]
     [ 1.33617886  1.80691057]]
    [[ 1.03984474  1.03564729]
     [ 1.03564729  1.38306366]]

```python
L
```

    array([-0.16708724+0.22051101j, -0.16708724-0.22051101j])

```python

```


# Resolución del problema de Laplace 1D con condiciones Dirichlet

Calcular $u:[a,b] \to R$ solución de
$$
\begin{align}
  \label{pb1d}
  -\frac{d^2 u(x) }{dx^2} = f(x) \mbox{ en } [a,b], \\
  u(a)=u_a, \ u(b)=u_b,
\end{align}
$$

## Sistema de ecuaciones MDF 1D

```python
from numpy import *
```

```python
# Datos:
a, b = 0, 1 #Extremos del intervalo
n = 10 # Números de puntos interiores en la partición, x_1,..., x_n
h = 1./(n+1) # Tamaño de la partición
ua, ub = 0, 1  # Datos de contorno
f = lambda x: pi**2/4 * sin(x*pi/2)
```

### Matriz

```python
# Caso n=3
A=array(
    [
    [2,-1,0], # Fila1
    [-1,2,-1], # Fila 2
    [0,-1,2]  # Fila 3
    ]
)
print("n=3:\n", A)
# Caso general:
A_h = (1./h**2) * (
    2*diag( ones(n) )
    - diag( ones(n-1), +1 )
    - diag( ones(n-1), -1)
)
print("Caso general:\n", A_h)
```

    n=3:
     [[ 2 -1  0]
     [-1  2 -1]
     [ 0 -1  2]]
    Caso general:
     [[ 242. -121.    0.    0.    0.    0.    0.    0.    0.    0.]
     [-121.  242. -121.    0.    0.    0.    0.    0.    0.    0.]
     [   0. -121.  242. -121.    0.    0.    0.    0.    0.    0.]
     [   0.    0. -121.  242. -121.    0.    0.    0.    0.    0.]
     [   0.    0.    0. -121.  242. -121.    0.    0.    0.    0.]
     [   0.    0.    0.    0. -121.  242. -121.    0.    0.    0.]
     [   0.    0.    0.    0.    0. -121.  242. -121.    0.    0.]
     [   0.    0.    0.    0.    0.    0. -121.  242. -121.    0.]
     [   0.    0.    0.    0.    0.    0.    0. -121.  242. -121.]
     [   0.    0.    0.    0.    0.    0.    0.    0. -121.  242.]]

### Segundo miembro

```python
f_h = []
x = linspace(0, 1, num=n) # x_0, ..., x_{n-1}
#for i in range(n): # i 0,...n-1
#    f_h.append( f(x[i]) )
f_h = f(x) # f_h es el array resultante de aplicar f a cada elmento del array x
f_h[0]  += ua/h**2
f_h[-1] += ub/h**2
f_h
```

    array([  0.        ,   0.4284597 ,   0.84390088,   1.23370055,
             1.58601486,   1.8901389 ,   2.13683203,   2.31859861,
             2.42991573, 123.4674011 ])

### Resolver sistema

```python
from numpy.linalg import solve
u_h = solve(A_h, f_h)

from matplotlib.pylab import plot, show, grid, legend
plot(x, u_h, label="Solución aproximada", linewidth=3, color="green")
grid()
legend()
show()

```

## Papel sucio....

```python

```

```python
# Trabajando...
size = 24
print("El tamaño es %i" %size)

# Mucho más abajo:
from numpy import *
x=array([1,2,3])
size(x) # No me he dado cuenta de que el import ha pisado la variable size

# De nuevo más abajo
print("El tamaño es %i", size)

```

    El tamaño es 24
    El tamaño es %i <function size at 0x0000000004F2A048>

```python
import numpy as np
x = np.array([1,2,3])
y = np.sin(x)
y
```

    array([0.84147098, 0.90929743, 0.14112001])

```python
2*identity(3)
```

    array([[2., 0., 0.],
           [0., 2., 0.],
           [0., 0., 2.]])

```python

```

```python
ones(3)
```

    array([1., 1., 1.])

```python
diag([1,3,88],-2)
```

    array([[ 0,  0,  0,  0,  0],
           [ 0,  0,  0,  0,  0],
           [ 1,  0,  0,  0,  0],
           [ 0,  3,  0,  0,  0],
           [ 0,  0, 88,  0,  0]])

```python
x = linspace(0,1, n) # x_0, ..., x_{n-1}
x[9]
```

    1.0

```python
lista = [1,2,3,4,5,6]
lista[:5:2]
```

    [1, 3, 5]

```python
%pylab
```

    Using matplotlib backend: Qt5Agg
    Populating the interactive namespace from numpy and matplotlib

    C:\Users\Usuario\Anaconda3\lib\site-packages\IPython\core\magics\pylab.py:160: UserWarning: pylab import has clobbered these variables: ['f']
    `%matplotlib` prevents importing * from pylab and numpy
      "\n`%matplotlib` prevents importing * from pylab and numpy"

```python
plo
```


```python
import numpy
import math
import matplotlib.pyplot as plt
import scipy.special
```

Maclaurin series for $\tan(x)$ is:
\begin{align}
\tan(x)
&= \sum_{k=1}^{\infty} \frac{ (-1)^{k-1} 2^{2k} ( 2^{2k} -1 ) B_{2k} }{ (2k)! } x^{2k-1} \\
&= x + \frac{1}{3} x^3 + \frac{1}{15} x^5 + \frac{17}{315} x^7 + \frac{62}{2835} x^9 + \ldots
\end{align}
The roundoff error is associated with the addition/subtraction involving the largest term which (for $|x|<6$) will be the first term, so of order $|x|\epsilon$.

```python
# Bernoulli numbers
def bernoulli_numerator(n):
    N = [1.,1.,1.,0,-1.,0,1.,0,-1.,0,5.,0,-691.,0,7.,0,-3617.,0,43867.,0,-174611.]
    return N[n]
def bernoulli_denominator(n):
    D = [1,2,6,0,30,0,42,0,30,0,66,0,2730,0,6,0,510,0,798,0,330]
    return D[n]
def bernoulli(n):
    B = [1,-.5,1./6,0,-1./30,0,1./42,0,-1./30,0,5./66,0,-691./2730,0,7./6,0,-3617./510,0,43867./798,0,-174611./330]
    return bernoulli_numerator(n) / bernoulli_denominator(n)
    return B[n]
def tan_numerator(n):
    # http://oeis.org/A002430
    N = [1, 1, 2, 17, 62, 1382, 21844, 929569, 6404582, 443861162, 18888466084, 113927491862, 58870668456604, 8374643517010684, 689005380505609448, 129848163681107301953, 1736640792209901647222, 418781231495293038913922]
    return N[n]
def tan_denominator(n):
    # http://oeis.org/A036279
    D = [1, 3, 15, 315, 2835, 155925, 6081075, 638512875, 10854718875, 1856156927625, 194896477400625, 2900518163668125, 3698160658676859375, 1298054391195577640625, 263505041412702261046875, 122529844256906551386796875, 4043484860477916195764296875]
    return D[n]
print('%2s'%'k', '%23s'%'B[k]', '%25s'%'2**2k * (2**2k - 1)', '%48s'%'(2k)!')
for k in range(1,11):
    print( '%2i'%k, '%23.16e'%(bernoulli(2*k)), '%25i'%( 2**(2*k) * ( 2**(2*k) - 1 ) ), '%48i'%(math.factorial(2*k)) )
F,C = [0.]*(18), [0.]*(18)
for k in range(1,18):
    F[k] = 2**(2*k) * ( 2**(2*k) - 1 ) / math.factorial(2*k)
    #C[k] = F[k] * abs(bernoulli(2*k))
    C[k] = tan_numerator(k-1)/tan_denominator(k-1)
    #C[k] = ( 2**(2*k) * ( 2**(2*k) - 1 ) * bernoulli_numerator(2*k) ) / ( math.factorial(2*k) * bernoulli_denominator(2*k) )
    print( '%2i'%k, '%23.16e'%F[k], '%23.16e'%C[k], '%23.16e'%(tan_numerator(k-1)/tan_denominator(k-1)) )
print(1/3,2/15,17/315,62/2835)
```

     k                    B[k]       2**2k * (2**2k - 1)                                            (2k)!
     1  1.6666666666666666e-01                        12                                                2
     2 -3.3333333333333333e-02                       240                                               24
     3  2.3809523809523808e-02                      4032                                              720
     4 -3.3333333333333333e-02                     65280                                            40320
     5  7.5757575757575760e-02                   1047552                                          3628800
     6 -2.5311355311355310e-01                  16773120                                        479001600
     7  1.1666666666666667e+00                 268419072                                      87178291200
     8 -7.0921568627450977e+00                4294901760                                   20922789888000
     9  5.4971177944862156e+01               68719214592                                 6402373705728000
    10 -5.2912424242424242e+02             1099510579200                              2432902008176640000
     1  6.0000000000000000e+00  1.0000000000000000e+00  1.0000000000000000e+00
     2  1.0000000000000000e+01  3.3333333333333331e-01  3.3333333333333331e-01
     3  5.5999999999999996e+00  1.3333333333333333e-01  1.3333333333333333e-01
     4  1.6190476190476191e+00  5.3968253968253971e-02  5.3968253968253971e-02
     5  2.8867724867724870e-01  2.1869488536155203e-02  2.1869488536155203e-02
     6  3.5016835016835016e-02  8.8632355299021973e-03  8.8632355299021973e-03
     7  3.0789668884906979e-03  3.5921280365724811e-03  3.5921280365724811e-03
     8  2.0527385606750685e-04  1.4558343870513183e-03  1.4558343870513183e-03
     9  1.0733396354311387e-05  5.9002744094558595e-04  5.9002744094558595e-04
    10  4.5193377107039259e-07  2.3912911424355248e-04  2.3912911424355248e-04
    11  1.5651397205850165e-08  9.6915379569294509e-05  9.6915379569294509e-05
    12  4.5366376824749980e-10  3.9278323883316833e-05  3.9278323883316833e-05
    13  1.1167108640685585e-11  1.5918905069328964e-05  1.5918905069328964e-05
    14  2.3634092625370900e-13  6.4516892156554306e-06  6.4516892156554306e-06
    15  4.3464998053156398e-15  2.6147711512907546e-06  2.6147711512907546e-06
    16  7.0104835618574752e-17  1.0597268320104654e-06  1.0597268320104654e-06
    17  9.9971245106333945e-19  4.2949110782738057e-07  4.2949110782738057e-07
    0.3333333333333333 0.13333333333333333 0.05396825396825397 0.021869488536155203

```python
# Significance of each term to leading term
k, eps = numpy.arange(2,18), numpy.finfo(float).eps
print('epsilon = %.2e'%eps, "= 2**%i"%int(math.log(eps)/math.log(2)))
plt.semilogy(k, eps * (1+0*k), 'k--', label=r'$\epsilon$' )
plt.semilogy(k, (numpy.pi/4)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/4$ (45$^\circ$)' );
plt.semilogy(k, (numpy.pi/6)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/6$ (30$^\circ$)' );
plt.semilogy(k, (numpy.pi/8)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/8$ (22.5$^\circ$)' );
plt.semilogy(k, (numpy.pi/12)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/12$ (15$^\circ$)' );
plt.semilogy(k, (numpy.pi/18)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/18$ (10$^\circ$)' );
plt.semilogy(k, (numpy.pi/180)**(2*k-2) * numpy.array(C)[k], '.-', label=r'$x=\pi/180$ (1$^\circ$)' );
plt.gca().set_xticks(numpy.arange(1,18)); plt.legend(); plt.xlabel('Terms, n = (k+1)/2'); plt.ylim(1e-17,3);
plt.title(r'$C_kx^{2k-2}$');
print(180/8)
```

```python
def tan_by_series(x):
    # http://oeis.org/A002430
    N = [1, 1, 2, 17, 62, 1382, 21844, 929569, 6404582, 443861162, 18888466084, 113927491862, 58870668456604, 8374643517010684, 689005380505609448, 129848163681107301953, 1736640792209901647222]
    # http://oeis.org/A036279
    D = [1, 3, 15, 315, 2835, 155925, 6081075, 638512875, 10854718875, 1856156927625, 194896477400625, 2900518163668125, 3698160658676859375, 1298054391195577640625, 263505041412702261046875, 122529844256906551386796875, 4043484860477916195764296875]
    C = numpy.array(N) / numpy.array(D)
    n, x2, xx, r = len(C), x*x, 1., 0.
    term = [1.] * (n)
    for k in range(1,n):
        xx = xx * x2
        term[k] = xx * C[k]
    for j in range(k,-1,-1):
        r = r + term[j]
    return r * x
angle = numpy.array([numpy.pi/80, numpy.pi/8])
print( tan_by_series( angle ), numpy.tan( angle ) )
```

    [0.03929011 0.41421356] [0.03929011 0.41421356]

```python
x = numpy.linspace(0, numpy.pi/3,1000)
plt.plot( x*180/numpy.pi, numpy.tan(x), label='numpy');
plt.plot( x*180/numpy.pi, tan_by_series(x), label='series');
plt.plot( x*180/numpy.pi, numpy.sin(x)/numpy.cos(x), '--', label='sin(x)/cos(x)');
plt.legend()
```

```python
x = numpy.linspace(numpy.finfo(1.).eps, numpy.pi/4,1000)
plt.semilogy( x*180/numpy.pi, numpy.abs( tan_by_series(x) - numpy.tan(x) ) / numpy.tan(x));
```


```python
import sympy as sp
from sympy.physics.mechanics import dynamicsymbols
```

```python
m0, m1, l0, l1, k0, k1, t, g = sp.symbols(r'm_0 m_1 l_0 l_1 k_0 k_1 t g')
```

```python
theta0 = sp.Function(r'\theta_0')(t)
theta1 = sp.Function(r'\theta_1')(t)
r0 = sp.Function(r'r_0')(t)
r1 = sp.Function(r'r_1')(t)
```

```python
dtheta0 = theta0.diff(t)
dtheta1 = theta1.diff(t)
dr0 = r0.diff(t)
dr1 = r1.diff(t)
```

```python
I0 = m0 * r0 ** 2 / 12
I1 = m1 * r1 ** 2 / 12
```

```python
x0 =  (r0 / 2) * sp.sin(theta0)
y0 = -(r0 / 2) * sp.cos(theta0)
x1 =  r0 * sp.sin(theta0) + (r1 / 2) * sp.sin(theta1)
y1 = -r0 * sp.cos(theta0) - (r1 / 2) * sp.cos(theta1)
```

```python
spring_potential = k0 * (r0 - l0) ** 2 / 2 + k1 * (r1 - l1) ** 2 / 2
```

```python
gravitational_potential = (m0 * y0 + m1 * y1) * g
```

```python
kinetic = m0 * (x0.diff(t) ** 2 + y0.diff(t) ** 2) / 2 + m1 * (x1.diff(t) ** 2 + y1.diff(t) ** 2) / 2 + (I0 / 2) * theta0.diff(t) ** 2 + (I1 / 2) * theta1.diff(t) ** 2
```

```python
L = kinetic - (spring_potential + gravitational_potential)
```

```python
EL_r0     = sp.Eq(L.diff(    dr0).diff(t),L.diff(    r0)).simplify()
EL_r1     = sp.Eq(L.diff(    dr1).diff(t),L.diff(    r1)).simplify()
EL_theta0 = sp.Eq(L.diff(dtheta0).diff(t),L.diff(theta0)).simplify()
EL_theta1 = sp.Eq(L.diff(dtheta1).diff(t),L.diff(theta1)).simplify()
```

```python
soln = sp.solve(
    [EL_r0, EL_r1, EL_theta0, EL_theta1],
    [r0.diff(t, 2), r1.diff(t, 2), theta0.diff(t, 2), theta1.diff(t, 2)]
    )
```

```python
keys = list(soln.keys())

soln_list = [sp.Eq(key,soln[key]) for key in keys]
```

```python
ddr0 = soln_list[0].simplify()
ddr1 = soln_list[1].simplify()
ddtheta0 = soln_list[2].simplify()
ddtheta1 = soln_list[3].simplify()
```

```python
subs_dict = {
        r0.diff(t   ):sp.Function(      r'\dot{r}_0')(t),
        r1.diff(t   ):sp.Function(      r'\dot{r}_1')(t),
    theta0.diff(t   ):sp.Function( r'\dot{\theta}_0')(t),
    theta1.diff(t   ):sp.Function( r'\dot{\theta}_1')(t),
        r0.diff(t, 2):sp.Function(     r'\ddot{r}_0')(t),
        r1.diff(t, 2):sp.Function(     r'\ddot{r}_1')(t),
    theta0.diff(t, 2):sp.Function(r'\ddot{\theta}_0')(t),
    theta1.diff(t, 2):sp.Function(r'\ddot{\theta}_1')(t)
    }
```

```python
def convert(_):
    return str(_.subs(subs_dict).rhs).replace('(t)','').replace('\\left','').replace('\\right','').replace('\\theta','theta').replace('\\dot{theta}','dtheta').replace('\\dot{r}','dr').replace('_','').replace(' - ','-').replace(' + ','+')
```

```python
convert(ddr0)
```

    '2*(-6*m0*m1*(3*g*sin(theta1)-3*dtheta0**2*r0*sin(theta0-theta1)+6*dtheta0*dr0*cos(theta0-theta1)+4*dtheta1*dr1)*sin(theta0-theta1)-3*m1*(3*g*m0*sin(theta0)+6*g*m1*sin(theta0)+4*m0*dtheta0*dr0+12*m1*dtheta0*dr0+3*m1*dtheta1**2*r1*sin(theta0-theta1)+6*m1*dtheta1*dr1*cos(theta0-theta1))*sin(theta0-theta1)*cos(theta0-theta1)+(8*m0+6*m1)*(3*g*m1*cos(theta1)+6*k1*l1-6*k1*r1+3*m1*dtheta0**2*r0*cos(theta0-theta1)+6*m1*dtheta0*dr0*sin(theta0-theta1)+2*m1*dtheta1**2*r1)*cos(theta0-theta1)-(4*m0-12*m1*sin(theta0-theta1)**2-9*m1*cos(theta0-theta1)**2+12*m1)*(3*g*m0*cos(theta0)+6*g*m1*cos(theta0)+6*k0*l0-6*k0*r0+2*m0*dtheta0**2*r0+6*m1*dtheta0**2*r0+3*m1*dtheta1**2*r1*cos(theta0-theta1)-6*m1*dtheta1*dr1*sin(theta0-theta1)))/(3*m0*(-4*m0+2*m1*sin(theta0)*sin(theta1)*cos(theta0-theta1)+m1*cos(theta0)**2+m1*cos(theta1)**2-5*m1))'

```python
convert(ddr1)
```

    '2*(3*m0*m1**2*(3*g*sin(theta1)-3*dtheta0**2*r0*sin(theta0-theta1)+6*dtheta0*dr0*cos(theta0-theta1)+4*dtheta1*dr1)*sin(theta0-theta1)*cos(theta0-theta1)+6*m1*(m0+m1)*(3*g*m0*sin(theta0)+6*g*m1*sin(theta0)+4*m0*dtheta0*dr0+12*m1*dtheta0*dr0+3*m1*dtheta1**2*r1*sin(theta0-theta1)+6*m1*dtheta1*dr1*cos(theta0-theta1))*sin(theta0-theta1)+2*m1*(4*m0+3*m1)*(3*g*m0*cos(theta0)+6*g*m1*cos(theta0)+6*k0*l0-6*k0*r0+2*m0*dtheta0**2*r0+6*m1*dtheta0**2*r0+3*m1*dtheta1**2*r1*cos(theta0-theta1)-6*m1*dtheta1*dr1*sin(theta0-theta1))*cos(theta0-theta1)+(12*m1*(m0+3*m1)*sin(theta0-theta1)**2+9*m1*(m0+4*m1)*cos(theta0-theta1)**2-4*(m0+3*m1)*(m0+4*m1))*(3*g*m1*cos(theta1)+6*k1*l1-6*k1*r1+3*m1*dtheta0**2*r0*cos(theta0-theta1)+6*m1*dtheta0*dr0*sin(theta0-theta1)+2*m1*dtheta1**2*r1))/(3*m0*m1*(-4*m0+2*m1*sin(theta0)*sin(theta1)*cos(theta0-theta1)+m1*cos(theta0)**2+m1*cos(theta1)**2-5*m1))'

```python
convert(ddtheta0)
```

    '(-3*m0*m1*(3*g*sin(theta1)-3*dtheta0**2*r0*sin(theta0-theta1)+6*dtheta0*dr0*cos(theta0-theta1)+4*dtheta1*dr1)*cos(theta0-theta1)+2*m1*(3*g*m0*cos(theta0)+6*g*m1*cos(theta0)+6*k0*l0-6*k0*r0+2*m0*dtheta0**2*r0+6*m1*dtheta0**2*r0+3*m1*dtheta1**2*r1*cos(theta0-theta1)-6*m1*dtheta1*dr1*sin(theta0-theta1))*sin(theta0-theta1)*cos(theta0-theta1)-4*(m0+m1)*(3*g*m1*cos(theta1)+6*k1*l1-6*k1*r1+3*m1*dtheta0**2*r0*cos(theta0-theta1)+6*m1*dtheta0*dr0*sin(theta0-theta1)+2*m1*dtheta1**2*r1)*sin(theta0-theta1)+2*(m0-3*m1*sin(theta0-theta1)**2-4*m1*cos(theta0-theta1)**2+4*m1)*(3*g*m0*sin(theta0)+6*g*m1*sin(theta0)+4*m0*dtheta0*dr0+12*m1*dtheta0*dr0+3*m1*dtheta1**2*r1*sin(theta0-theta1)+6*m1*dtheta1*dr1*cos(theta0-theta1)))/(m0*(-4*m0+2*m1*sin(theta0)*sin(theta1)*cos(theta0-theta1)+m1*cos(theta0)**2+m1*cos(theta1)**2-5*m1)*r0)'

```python
convert(ddtheta1)
```

    '(-3*m0*(3*g*m0*sin(theta0)+6*g*m1*sin(theta0)+4*m0*dtheta0*dr0+12*m1*dtheta0*dr0+3*m1*dtheta1**2*r1*sin(theta0-theta1)+6*m1*dtheta1*dr1*cos(theta0-theta1))*cos(theta0-theta1)-2*m0*(3*g*m1*cos(theta1)+6*k1*l1-6*k1*r1+3*m1*dtheta0**2*r0*cos(theta0-theta1)+6*m1*dtheta0*dr0*sin(theta0-theta1)+2*m1*dtheta1**2*r1)*sin(theta0-theta1)*cos(theta0-theta1)+4*m0*(3*g*m0*cos(theta0)+6*g*m1*cos(theta0)+6*k0*l0-6*k0*r0+2*m0*dtheta0**2*r0+6*m1*dtheta0**2*r0+3*m1*dtheta1**2*r1*cos(theta0-theta1)-6*m1*dtheta1*dr1*sin(theta0-theta1))*sin(theta0-theta1)-2*(4*m1*(m0+3*m1)*cos(theta0-theta1)**2+3*m1*(m0+4*m1)*sin(theta0-theta1)**2-(m0+3*m1)*(m0+4*m1))*(3*g*sin(theta1)-3*dtheta0**2*r0*sin(theta0-theta1)+6*dtheta0*dr0*cos(theta0-theta1)+4*dtheta1*dr1))/(m0*(-4*m0+2*m1*sin(theta0)*sin(theta1)*cos(theta0-theta1)+m1*cos(theta0)**2+m1*cos(theta1)**2-5*m1)*r1)'

```js
ddr0 = (2*(-6*m0*m1*(3*g*sin(theta1)-3*dtheta0^2*r0*sin(theta0-theta1)+6*dtheta0*dr0*cos(theta0-theta1)+4*dtheta1*dr1)*sin(theta0-theta1)-3*m1*(3*g*m0*sin(theta0)+6*g*m1*sin(theta0)+4*m0*dtheta0*dr0+12*m1*dtheta0*dr0+3*m1*dtheta1^2*r1*sin(theta0-theta1)+6*m1*dtheta1*dr1*cos(theta0-theta1))*sin(theta0-theta1)*cos(theta0-theta1)+(8*m0+6*m1)*(3*g*m1*cos(theta1)+6*k1*l1-6*k1*r1+3*m1*dtheta0^2*r0*cos(theta0-theta1)+6*m1*dtheta0*dr0*sin(theta0-theta1)+2*m1*dtheta1^2*r1)*cos(theta0-theta1)-(4*m0-12*m1*sin^2(theta0-theta1)-9*m1*cos^2(theta0-theta1)+12*m1)*(3*g*m0*cos(theta0)+6*g*m1*cos(theta0)+6*k0*l0-6*k0*r0+2*m0*dtheta0^2*r0+6*m1*dtheta0^2*r0+3*m1*dtheta1^2*r1*cos(theta0-theta1)-6*m1*dtheta1*dr1*sin(theta0-theta1))))/(3*m0*(-4*m0+2*m1*sin(theta0)*sin(theta1)*cos(theta0-theta1)+m1*cos^2(theta0)+m1*cos^2(theta1)-5*m1));

ddr1 = \frac{2*(3*m0*m1^2*(3 g\sin{(\theta1)}-3 dtheta0^2 r0\sin{(\theta0-\theta1)}+6 dtheta0 dr0\cos{(\theta0-\theta1)}+4 dtheta1 dr1)\sin{(\theta0-\theta1)}\cos{(\theta0-\theta1)}+6 m1 (m0+m1) (3 g m0\sin{(\theta0)}+6 g m1\sin{(\theta0)}+4 m0 dtheta0 dr0+12 m1 dtheta0 dr0+3 m1 dtheta1^2 r1\sin{(\theta0-\theta1)}+6 m1 dtheta1 dr1\cos{(\theta0-\theta1)})\sin{(\theta0-\theta1)}+2 m1 (4 m0+3 m1) (3 g m0\cos{(\theta0)}+6 g m1\cos{(\theta0)}+6 k0 l0-6 k0 r0+2 m0 dtheta0^2 r0+6 m1 dtheta0^2 r0+3 m1 dtheta1^2 r1\cos{(\theta0-\theta1)}-6 m1 dtheta1 dr1\sin{(\theta0-\theta1)})\cos{(\theta0-\theta1)}+(12 m1 (m0+3 m1)\sin^2{(\theta0-\theta1)}+9 m1 (m0+4 m1)\cos^2{(\theta0-\theta1)}-4 (m0+3 m1) (m0+4 m1)) (3 g m1\cos{(\theta1)}+6 k1 l1-6 k1 r1+3 m1 dtheta0^2 r0\cos{(\theta0-\theta1)}+6 m1 dtheta0 dr0\sin{(\theta0-\theta1)}+2 m1 dtheta1^2 r1))}{3 m0 m1 (-4 m0+2 m1\sin{(\theta0)}\sin{(\theta1)}\cos{(\theta0-\theta1)}+m1\cos^2{(\theta0)}+m1\cos^2{(\theta1)}-5 m1)};

ddtheta0 = \frac{-3 m0 m1 (3 g\sin{(\theta1)}-3 dtheta0^2 r0\sin{(\theta0-\theta1)}+6 dtheta0 dr0\cos{(\theta0-\theta1)}+4 dtheta1 dr1)\cos{(\theta0-\theta1)}+2 m1 (3 g m0\cos{(\theta0)}+6 g m1\cos{(\theta0)}+6 k0 l0-6 k0 r0+2 m0 dtheta0^2 r0+6 m1 dtheta0^2 r0+3 m1 dtheta1^2 r1\cos{(\theta0-\theta1)}-6 m1 dtheta1 dr1\sin{(\theta0-\theta1)})\sin{(\theta0-\theta1)}\cos{(\theta0-\theta1)}-4 (m0+m1) (3 g m1\cos{(\theta1)}+6 k1 l1-6 k1 r1+3 m1 dtheta0^2 r0\cos{(\theta0-\theta1)}+6 m1 dtheta0 dr0\sin{(\theta0-\theta1)}+2 m1 dtheta1^2 r1)\sin{(\theta0-\theta1)}+2 (m0-3 m1\sin^2{(\theta0-\theta1)}-4 m1\cos^2{(\theta0-\theta1)}+4 m1) (3 g m0\sin{(\theta0)}+6 g m1\sin{(\theta0)}+4 m0 dtheta0 dr0+12 m1 dtheta0 dr0+3 m1 dtheta1^2 r1\sin{(\theta0-\theta1)}+6 m1 dtheta1 dr1\cos{(\theta0-\theta1)})}{m0 (-4 m0+2 m1\sin{(\theta0)}\sin{(\theta1)}\cos{(\theta0-\theta1)}+m1\cos^2{(\theta0)}+m1\cos^2{(\theta1)}-5 m1) r0};

ddtheta1 = \frac{-3 m0 (3 g m0\sin{(\theta0)}+6 g m1\sin{(\theta0)}+4 m0 dtheta0 dr0+12 m1 dtheta0 dr0+3 m1 dtheta1^2 r1\sin{(\theta0-\theta1)}+6 m1 dtheta1 dr1\cos{(\theta0-\theta1)})\cos{(\theta0-\theta1)}-2 m0 (3 g m1\cos{(\theta1)}+6 k1 l1-6 k1 r1+3 m1 dtheta0^2 r0\cos{(\theta0-\theta1)}+6 m1 dtheta0 dr0\sin{(\theta0-\theta1)}+2 m1 dtheta1^2 r1)\sin{(\theta0-\theta1)}\cos{(\theta0-\theta1)}+4 m0 (3 g m0\cos{(\theta0)}+6 g m1\cos{(\theta0)}+6 k0 l0-6 k0 r0+2 m0 dtheta0^2 r0+6 m1 dtheta0^2 r0+3 m1 dtheta1^2 r1\cos{(\theta0-\theta1)}-6 m1 dtheta1 dr1\sin{(\theta0-\theta1)})\sin{(\theta0-\theta1)}-2 (4 m1 (m0+3 m1)\cos^2{(\theta0-\theta1)}+3 m1 (m0+4 m1)\sin^2{(\theta0-\theta1)}-(m0+3 m1) (m0+4 m1)) (3 g\sin{(\theta1)}-3 dtheta0^2 r0\sin{(\theta0-\theta1)}+6 dtheta0 dr0\cos{(\theta0-\theta1)}+4 dtheta1 dr1)}{m0 (-4 m0+2 m1\sin{(\theta0)}\sin{(\theta1)}\cos{(\theta0-\theta1)}+m1\cos^2{(\theta0)}+m1\cos^2{(\theta1)}-5 m1) r1};
```

```python

```


# Lecture 4

This notebook presents some extensions of material handled in lectures. It presents

- A general method for non-homogeneous second-order ordinary differential equations; and
- The formulation of the differential equation for an elastic rod in one dimension via an integral approach.

## An ordinary differential equation extension

We present an extension to what was covered in the lecture, namely the *variation of parameters method* for non-homogeneous equations.

### Method of undetermined coefficients

In the previous lecture, we covered the solution of non-homogeneous second-order differential equations. Although not explicitly stated, the method we used is known as *method of undetermined coefficients*. It relied on computing a solution to the homogeneous problem, and then having a suitable 'guess' for for the particular solution $y_{P}$.

But what if we do not know what $y_{P}$ should look like? In this case we can use the *variation of parameters* method.

### Variation of parameters method

We wish to find a solution to the problem

$$
\frac{d^{2}y}{dx^{2}} + p(x) \frac{dy}{dx} + q(x) y = f(x)
$$

By linearity, if $y_{P}$ is a solution to the above equation and $y_{CF}$ is a solution to the homogeneous equation,

$$
\frac{d^{2}y}{dx^{2}} + p(x) \frac{dy}{dx} + q(x) y = 0
$$

then the general solution is $y = y_{CF} + y_{P}$.

For the case of constant coefficients, we know how to find the solution to the homogeneous problem, which is most cases is of the form

$$
y_{CF} = \underbrace{A_{1} e^{\lambda_{1} x}}_{y_{1}} + \underbrace{A_{2} e^{\lambda_{2} x}}_{y_{2}}
$$

where $\lambda$ are the roots of $\lambda + p\lambda + q$.

Consider a particular solution $y_{P}$ of the form

$$
y_{P} = v_{1}(x) y_{1} + v_{2}(x)y_{2}
$$

with the objective of finding the functions $v_{1}$ and $v_{2}$. We will insist of the constraint

$$
\frac{d v_{1}}{dx} y_{1} + \frac{d v_{2}}{dx} y_{2} = 0.
$$

Computing derivatives of $y_{P}$:

$$
\begin{align}
\frac{d y_{P}}{d x} &= v_{1} \frac{d y_{1}}{dx} + v_{2} \frac{d y_{2}}{dx}
\\
\frac{d^{2} y_{P}}{d x^{2}} &= \frac{d v_{1}}{dx} \frac{d y_{1}}{dx} + \frac{d v_{2}}{dx} \frac{d y_{2}}{dx} +
v_{1} \frac{d^{2} y_{1}}{dx^{2}} + v_{2} \frac{d^{2} y_{2}}{dx^{2}}
\end{align}
$$

Inserting the expressions for $y_{P}$ is its derivatives into the original equations (keeping in mind that $y_{1}$ and $y_{2}$ are solutions to the homogeneous problem), we find

$$
\frac{d v_{1}}{dx} \frac{d y_{1}}{dx} + \frac{d v_{2}}{dx} \frac{d y_{2}}{dx} = f(x)
$$

Combining the above equations with the conditon $v_{1}^{\prime} y_{1} + v_{2}^{\prime} y_{2} = 0$,

$$
\begin{align}
\frac{d v_{1}}{dx} &= -\frac{y_{2} f(x)}{W(x)}
\\
\frac{d v_{2}}{dx} &= \frac{y_{1} f(x)}{W(x)}
\end{align}
$$

where $W(x) = y_{1} y^{\prime}_{2} - y^{\prime}_{1} y_{2}$ is known as the *Wronskian*. Integrating both sides of the above equations,

$$
\begin{align}
v_{1} &= -\int \frac{y_{2} f(x)}{W(x)} dx
\\
v_{2} &= \int \frac{y_{1} f(x)}{W(x)} dx
\end{align}
$$

#### Solving a given non-homogeneous equation symbolically

We now solve and ODE symbolically with a RHS side $f(x)$ of unknown form. We first import SymPy:

```
from sympy import *
init_printing()
from IPython.display import display
```

We first solve the homogeneous equation:

```
x = Symbol("x")
y = Function("y")
f = Function("f")
eqn = Eq(Derivative(y(x), x, x) + 2*Derivative(y(x), x) + y(x), 0)
display(eqn)
dsolve(eqn)
```

Now solve the non-homogeneous case for some $f(x)$:

```
eqn = Eq(Derivative(y(x), x, x) + 2*Derivative(y(x), x) + y(x), f(x))
dsolve(eqn)
```

Note the integrand involving the product of $f(x)$ and terms from solution to the homogeneous problem, as expected from the variation of parameters method.

## From balance laws to differential equations

It is common in science and engineering to derive differential equations from integral balance laws. The example of an elastic one-dimensional rod was covered in Lecture 4 (but it not in the printed lecture notes). That example repeated and expanded here.

Before presenting the example, we consider a theorem that we will use in the derivation.

#### Localisation theorem

For a continuous function $\phi(x)$ on $\Omega$ statisfies

$$
\int_{B} \phi \, dx = 0 \quad \forall B \subset \Omega
$$

then

$$
\phi(x) = 0 \quad \forall x \in \Omega.
$$

*Proof:*
The proof is by contradiction. Assume at some point $x_{0}$ that $\phi(x_{0}) = \delta > 0$. Since $\phi$ is continuous, there is a neighborhood $D \in \Omega$ around $x_{0}$ on which $\phi > 0$. On $D$, $\int_{D} \phi \, dx > 0$, which contradicts the first part of the theorem.

Note that the notation '$\forall B \subset \Omega$' means all subdomains of $\Omega$.

### An elastic rod in one dimension

An elastic rod in one dimension can transmit forces along its axis only (there is not shear or bending - it is not a beam). The normal force in the rod is $N$, and by definition $h = N n_{x}$, where $h$ is a force and $n_{x}$ is the outward unit normal vector. We assume the the normal force in the rod is smooth, i.e. it is continuous as its first derivative.

Consider the below bar with forces acting at the ends and a distributed load $f$ (force per unit length) acting along the rod.

Summing the forces acting on the rod,

$$
\int_{0}^{L} f \, dx + h_{L} + h_{0} = 0.
$$

Replacing $h$ by $N n_{x}$ (in one dimension $n_{x} = \pm 1$),

$$
\int_{0}^{L} f \, dx + N_{L} - N_{0} = 0.
$$

We can convert the $N_{L} - N_{0}$ term to an integral from $0$ to $L$:

$$
\int_{0}^{L} f \, dx + \int_{0}^{L} \frac{dN}{dx} \, dx = 0.
$$

Since $f$ and $dN$/dx are continuous, from the localisation theorem implies that

$$
\frac{dN}{dx} + f = 0.
$$

This is the equilibrium equation. For an elastic bar, the normal force in terms of deformation is given by Hooke's law:

$$
N = EA\frac{du}{dx},
$$

where $E$ is Young's modulus, $A(x)$ is the cross-sectional area of the rod, $u$ is the displacement and $du/dx$ is the strain.  Inserting the expression for $N$ into the equilibrium equation:

$$
\frac{d}{dx}\left(EA \frac{du}{dx}\right) + f = 0.
$$

This is a second-order differential equation, with $u$ being the dependent variable.


# Lambdify in symbolic module
### Importing required modules

```python
import sympy
from sympy.abc import x, y
from sympy import symbols
from einsteinpy.symbolic import BaseRelativityTensor

sympy.init_printing()
```

### Calculating a Base Relativity Tensor

```python
syms = symbols("x y")
x, y = syms
T = BaseRelativityTensor([[x, 1],[0, x+y]], syms, config="ll")
```

### Calling the lambdify function

```python
args, func = T.tensor_lambdify()
args
```

`args` indicates the order in which arguments should be passed to the returned function `func`

### Executing the returned function for some value

```python
func(2, 1)
```


## 2.3 Multi-Way Tables and Simpson's Paradox

In the previous lesson, we summarized two categorical variables by cross-tabulating their frequencies.

```python
import pandas as pd
data_dir = "http://dlsun.github.io/pods/data/"
df_titanic = pd.read_csv(data_dir + "titanic.csv")

def class_to_type(c):
  if c in ["1st", "2nd", "3rd"]:
    return "passenger"
  else:
    return "crew"
df_titanic["type"] = df_titanic["class"].map(class_to_type)

joint_type_survived = pd.crosstab(
    df_titanic["type"],
    df_titanic["survived"],
    normalize=True
)
joint_type_survived
```

      survived
      0
      1

      type

      crew
      0.307657
      0.095605

      passenger
      0.370186
      0.226552

Each number in this table represents a joint probability. For example:
$$ P(\text{crew}, \text{survived}) = 0.095605. $$

We might want to know whether crew members or passengers survived at higher rates. To do this, we have to compare the conditional probabilities
$$\begin{align}
P(\text{survived} | \text{crew}) & & \text{vs.} & & P(\text{survived} | \text{passenger}).
\end{align}$$

In the last lesson, we learned to calculate conditional distributions using broadcasting.

```python
survived_given_type = joint_type_survived.divide(
    joint_type_survived.sum(axis=1),
    axis=0
)
survived_given_type
```

      survived
      0
      1

      type

      crew
      0.762921
      0.237079

      passenger
      0.620349
      0.379651

From the table, it is apparent that passengers survived at much higher rates than crew members:
$$ P(\text{survived}|\text{crew}) = 0.237079 < 0.379651 = P(\text{survived}|\text{passenger}). $$

### 2.3.1 Communication Corner: Reporting Differences in Probabilities

How do we report the difference between the two probabilities $23.71\%$ and $37.97\%$ above? There are a number of ways:

1. As an _additive change_: "Passengers were
$$ 37.97\% - 23.71\% = 14.26 \text{ percentage points} $$
more likely to survive than crew members."
2. As a _relative change_ (or _relative risk_): "Passengers were
$$ 37.97\% \big/ 23.71\% = 1.60 \text{ times} $$
as likely to survive as crew members."
3. We can translate relative changes to _percent changes_ by subtracting $1$ and multiplying by $100\%$. So we can rephrase the above as: "Passengers were
$$ 100\% \times (1.60 - 1.00) = 60\% $$
more likely to survive than crew members."
4. As an _odds ratio_: "The odds of a passenger surviving was
$$ \frac{37.97\% \big/ (100\% - 37.97\%)}{23.71\% \big/ (100\% - 23.71\%)} = 1.97 \text{ times} $$
greater than the odds of a crew member surviving."

Note that additive changes and percent changes should be compared to a baseline of 0.0 (and can be negative), while relative changes and odds ratios should be compared to a baseline of 1.0 (and cannot be negative).

Watch out: it is incorrect to say that passengers are $14.26\%$ more likely to survive than crew members, since the percent change is $60\%$. An additive change should always be reported in units of "percentage points".

### 2.3.2 Controlling for a Variable

But is this the whole story? We know that survival rates for males and females were very different. Will the trend between the survival rates for crew and passengers still hold after we _control_ for **gender**?

To do this, let's determine the joint distribution of these two variables and a third variable, **gender**. In principle, the frequencies could be represented using a three-dimensional table, but it is difficult to visualize more than two dimensions on paper or on a screen. So we put two of the variables along one dimension and one variable along the other, creating a _three-way table_.

```python
joint_gender_type_survived = pd.crosstab(
    [df_titanic["gender"], df_titanic["type"]],
    df_titanic["survived"],
    normalize=True
)
joint_gender_type_survived
```

      survived
      0
      1

      gender
      type

      female
      crew
      0.001359
      0.009062

      passenger
      0.057544
      0.153602

      male
      crew
      0.306298
      0.086543

      passenger
      0.312642
      0.072950

Of course, we would have chosen any two of the variables to place along the rows, or had the two variables along the columns instead of the rows. The particular representation above was chosen because it makes it easy to survival rates for each gender and type, i.e.,
$$ P(\text{survived} | \textbf{gender}, \textbf{type}), $$
where **gender** is either "male" or "female" and **type** is either "crew" or "passenger". Recall that the conditional probability is calculated as
$$ P(\text{survived} | \textbf{gender}, \textbf{type}) = \frac{P(\text{survived}, \textbf{gender}, \textbf{type})}{P(\textbf{gender}, \textbf{type})}. $$
The numerator is the joint distribution above. The denominator can be calculated by summing over the possible values of **survived**---in other words, across each row, over the columns.

```python
joint_gender_type = joint_gender_type_survived.sum(axis=1)
joint_gender_type
```

    gender  type
    female  crew         0.010421
            passenger    0.211146
    male    crew         0.392841
            passenger    0.385591
    dtype: float64

To obtain the conditional probabilities, we simply divide the joint distribution by the marginal.

```python
survived_given_gender_type = joint_gender_type_survived.divide(
    joint_gender_type,
    axis=0
)
survived_given_gender_type
```

      survived
      0
      1

      gender
      type

      female
      crew
      0.130435
      0.869565

      passenger
      0.272532
      0.727468

      male
      crew
      0.779700
      0.220300

      passenger
      0.810811
      0.189189

Now, let's compare the survival rates of passengers and crew members for females and males separately.

- For females, crew members survived at a higher rate:
$$ P(\text{survived} | \text{female}, \text{crew}) = 0.869565 > 0.727468 = P(\text{survived} | \text{female}, \text{passenger}) $$
- For males, crew members survived at a higher rate:
$$ P(\text{survived} | \text{male}, \text{crew}) = 0.220300 > 0.189189 = P(\text{survived} | \text{male}, \text{passenger}) $$

But remember, we found earlier that passengers survived at a higher rate overall:
$$ P(\text{survived} | \text{crew}) < P(\text{survived} | \text{passenger}). $$

How is it possible that both male and female crew members survived at a higher rate, yet crew members survived at a lower rate overall? This surprising phenomenon is known as _Simpson's paradox_.

### 2.3.3 Simpson's Paradox

Simpson's paradox is a phenomenon where a trend disappears or reverses when the data is aggregated. In the Titanic data set, both male and female crew members survived at higher rates, but when we aggregated over gender, the trend reversed.

In order to investigate Simpson's paradox, we first reorganize the probabilities. First, we keep only the survival rate, dropping the death rate (since it is redundant; it is just one minus the survival rate).

```python
survived_given_gender_type[1]
```

    gender  type
    female  crew         0.869565
            passenger    0.727468
    male    crew         0.220300
            passenger    0.189189
    Name: 1, dtype: float64

Next, we rearrange these probabilities into a two-way table, with gender along one dimension and type along the other. This can be achieved in code by "unstacking" a level of the index. (There are two "levels": **gender** and **type**.)

```python
survival_rates_by_gender_type = (survived_given_gender_type[1].
                                 unstack(level="type"))
survival_rates_by_gender_type
```

      type
      crew
      passenger

      gender

      female
      0.869565
      0.727468

      male
      0.220300
      0.189189

Caution: the probabilities in this table do not represent a distribution. They do not add up to 1.0. These probabilities originally came from the conditional distribution of **survived** given **gender** and **type**, but we dropped the death rates from the data.

Finally, we append the overall survival rates (which we calculated at the beginning of this lesson) to the last row of this `DataFrame`.

```python
survival_rates_by_gender_type.append(survived_given_type[1])
```

      type
      crew
      passenger

      gender

      female
      0.869565
      0.727468

      male
      0.220300
      0.189189

      1
      0.237079
      0.379651

The overall survival rates are weighted averages of the survival rates for each gender. If we look at the survival rates for crew members:

- The survival rate for female crew is 87.0%.
- The survival rate for male crew is 22.0%.
- The overall survival rate for all crew is 23.7%, which is between the gender-specific survival rates, but much closer to the survival rate for male crew.

Likewise, if we look at the survival rates for passengers:

- The survival rate for female crew is 72.7%.
- The survival rate for male crew is 18.9%.
- The overall survival rate for all crew is 38.0%, which is closer to the middle of the gender-specific survival rates.

Why would the survival rate for crew members be so close to the survival rate for male crew? To answer this question, let's examine the weights that go into this weighted average.

In mathematical notation, the overall survival rate can be decomposed as:
$$ \underbrace{P(\text{survived} | \textbf{type})}_{\text{overall survival rate}} = \sum_{\textbf{gender}} \underbrace{P(\textbf{gender} | \textbf{type})}_{\text{weight}} \underbrace{P(\text{survived} | \textbf{gender}, \textbf{type})}_{\text{gender-specific survival rate}}. $$
So we see that the weights are $P(\textbf{gender} | \textbf{type})$.

First, we calculate this conditional distribution from the joint distribution of **gender** and **type**.

```python
joint_gender_type = pd.crosstab(
    df_titanic["gender"],
    df_titanic["type"],
    normalize=True
)

gender_given_type = joint_gender_type.divide(
    joint_gender_type.sum(axis=0),
    axis=1
)
gender_given_type
```

      type
      crew
      passenger

      gender

      female
      0.025843
      0.353834

      male
      0.974157
      0.646166

Notice that 97.4% of crew members were male! So the lower male survival rate is going to dominate the weighted average when we calculate the overall survival rate for crew members. On the other hand, the gender ratio for passengers was more balanced, so their overall survival rate will end up being closer to the middle of the male and female survival rates.

Now, we calculate the weighted average, using the conditional distribution of gender as "weights" that we multiply by the survival rates. Then, we sum over the genders to get the weighted averages---i.e., the overall survival rates.

```python
(gender_given_type * survival_rates_by_gender_type).sum(axis=0)
```

    type
    crew         0.237079
    passenger    0.379651
    dtype: float64

Check that these match the overall survival rates that we calculated above.

So the secret of Simpson's Paradox lies in two facts:

1. Survival rates were generally much lower for men than for women.
2. Because crew members were predominantly male, their survival rate was weighted towards the lower male survival rate, that their overall survival rate ended up being lower than the survival rate for passengers.

Simpson's Paradox means that we have to be careful when comparing proportions from a two-way table, such as survival rates for crew and passengers. When we control for a third variable, such as **gender**, the direction of the effect could change!

### 2.3.4 Exercises

1\. Calculate the _percent change_ in survival rates between passengers and crew members, controlling for where they embarked. Does there appear to be a Simpson's paradox effect here?

Exercise 2 asks you to work with the Florida Death Penalty data set, which is available at  `https://dlsun.github.io/pods/data/death_penalty.csv`. This data set contains information about the races of the defendant and the victim, as well as whether a death penalty verdict was rendered, in 674 homicide trials in Florida between 1976-1987.

2\. Calculate the _relative risk_ of a death penalty verdict for black defendants (relative to white defendants), adjusting for the race of the victim. How does this compare to what you found at the end of the last chapter? Can you explain intuitively what is going on?

TesterBook/02_Categorical_Data/.ipynb_checkpoints/2.3 Multi Way Tables and Simpson's Paradox-checkpoint.ipynb

TesterBook/02_Categorical_Data/2.3 Multi Way Tables and Simpson's Paradox.ipynb


# Nonlinear Equations

We want to find a root of the nonlinear function $f$ using different methods.

1. Bisection method
2. Newton method
3. Chord method
4. Secant method
5. Fixed point iterations

```python
%matplotlib inline
from numpy import *
from matplotlib.pyplot import *
import sympy as sym

```

```python
t = sym.symbols('t')

f_sym = t/8. * (63.*t**4 - 70.*t**2. +15.) # Legendre polynomial of order 5

f_prime_sym = sym.diff(f_sym,t)

f = sym.lambdify(t, f_sym, 'numpy')
f_prime = sym.lambdify(t,f_prime_sym, 'numpy')

phi = lambda x : 63./70.*x**3 + 15./(70.*x)
#phi = lambda x : 70.0/15.0*x**3 - 63.0/15.0*x**5
#phi = lambda x : sqrt((63.*x**4 + 15.0)/70.)

# Let's plot
n = 1025

x = linspace(-1,1,n)
c = zeros_like(x)

_ = plot(x,f(x))
_ = plot(x,c)
_ = grid()

```

```python
# Initial data for the variuos algorithms

# interval in which we seek the solution
a = 0.7
b = 1.

# initial points
x0 = (a+b)/2.0
x00 = b

```

```python
# stopping criteria
eps = 1e-10
n_max = 1000
```

## Bisection method

$$
x^k = \frac{a^k+b^k}{2}
$$
```
                   if (f(a_k) * f(x_k)) < 0:
                      b_k1 = x_k
                      a_k1 = a_k
                   else:
                      a_k1 = x_k
                      b_k1 = b_k
```

```python
def bisect(f,a,b,eps,n_max):
    assert f(a)*f(b) < 0
    a_new = a
    b_new = b
    x = mean([a,b])
    err = eps + 1.
    errors = [err]
    it = 0
    while (err > eps and it < n_max):
        if ( f(a_new) * f(x) < 0 ):
            # root in (a_new,x)
            b_new = x
        else:
            # root in (x,b_new)
            a_new = x

        x_new = mean([a_new,b_new])

        #err = 0.5 *(b_new -a_new)
        err = abs(f(x_new))
        #err = abs(x-x_new)

        errors.append(err)
        x = x_new
        it += 1

    semilogy(errors)
    print(it)
    print(x)
    print(err)
    return errors

errors_bisect = bisect(f,a,b,eps,n_max)

```

```python
# is the number of iterations coherent with the theoretical estimation?
```

In order to find out other methods for solving non-linear equations, let's compute the Taylor's series of $f(x^k)$ up to the first order

$$
f(x^k) \simeq f(x^k) + (x-x^k)f^{\prime}(x^k)
$$
which suggests the following iterative scheme
$$
x^{k+1} = x^k - \frac{f(x^k)}{f^{\prime}(x^k)}
$$

The following methods are obtained applying the above scheme where

$$
f^{\prime}(x^k) \approx q^k
$$

## Newton's method
$$
q^k = f^{\prime}(x^k)
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q^k}
$$

```python
def newton(f,f_prime,x0,eps,n_max):
    err = abs(f(x0))
    errors = [err]
    it = 0
    x = x0
    while (err > eps and it < n_max):
        qk = f_prime(x)
        if abs(qk) < 1e-12:
            raise RuntimeError("f_prime(x) is close to zero")
        x_new = x - f(x)/qk
        err = abs(f(x_new))
        x = x_new
        errors.append(err)
        it += 1

    #print x
    #print err
    #print it
    semilogy(errors)
    return errors

%time errors_newton = newton(f,f_prime,1.0,eps,n_max)
```

## Chord method

$$
q^k \equiv q = \frac{f(b)-f(a)}{b-a}
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q}
$$

```python
def chord(f,a,b,x0,eps,n_max):
    q = (f(b) - f(a))/(b-a)
    err = eps +1.
    errors = [err]
    it = 0
    x = x0
    while (err > eps and it <n_max):
        x_new = x - f(x)/q
        err = abs(x_new - x)
        errors.append(err)
        x = x_new
        it += 1

    print(x)
    print(err)
    print(it)
    semilogy(errors)
    return errors

errors_chord = chord (f,a,b,x0,eps,n_max)
```

## Secant method

$$
q^k = \frac{f(x^k)-f(x^{k-1})}{x^k - x^{k-1}}
$$

$$
x^{k+1} = x^k - \frac{f(x^k)}{q^k}
$$

Note that this algorithm requirs **two** initial points

```python
def secant(f,x0,x00,eps,n_max):
    err = eps +1.
    errors = [err]

    it = 0

    xk = x0
    xkk = x00
    while (err > eps and it < n_max):
        qk = (f(xk) - f(xkk))/(xk - xkk)
        x_new = xk - f(xk)/qk
        err = abs (x_new - xk)
        xkk = xk
        xk = x_new
        it += 1
        errors.append(err)

    print(xk)
    print(err)
    print(it)
    semilogy(errors)
    return errors

errors_secant = secant(f,x0,x00,eps,n_max)
```

## Fixed point iterations

$$
f(x)=0 \to x-\phi(x)=0
$$

$$
x^{k+1} = \phi(x^k)
$$

```python
def fixed_point(phi,x0,eps,n_max):
    x = x0
    err = eps + 1.
    errors = [err]
    it = 0
    while (err > eps and it < n_max):
        x_new = phi (x)
        err = abs (x_new -x)
        x = x_new
        it += 1
        errors.append(err)

    print(x)
    print(err)
    print(it)
    semilogy(errors)
    return errors

errors_fixed = fixed_point(phi,0.3,eps,n_max)

```

## Comparison

```python
# plot the error convergence for the methods
loglog(errors_bisect, label='bisect')
loglog(errors_chord, label='chord')
loglog(errors_secant, label='secant')
loglog(errors_newton, label ='newton')
loglog(errors_fixed, label ='fixed')
_ = legend()
```

```python
# Let's compare the scipy implmentation of Newton's method with our..
```

```python
import scipy.optimize as opt
%time opt.newton(f, 1.0, f_prime, tol = eps)
```

    CPU times: user 27 µs, sys: 2 µs, total: 29 µs
    Wall time: 30.8 µs

    0.906179845938664

```python

```


```python
import mahotas as mh
import numpy as np
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt
from tpot import TPOT
from sklearn.cross_validation import train_test_split
import dautil as dl
```

```python
context = dl.nb.Context('extracting_texture')
lr = dl.nb.LatexRenderer(chapter=11, start=5, context=context)
lr.render(r' C_{\Delta x, \Delta y}(i,j)=\sum_{p=1}^n\sum_{q=1}^m\begin{cases} 1, & \text{if }I(p,q)=i\text{ and }I(p+\Delta x,q+\Delta y)=j \\ 0, & \text{otherwise}\end{cases}')
lr.render(r'''\begin{align}
 Angular \text{ } 2nd \text{ } Moment &= \sum_{i} \sum_{j} p[i,j]^{2}\\
 Contrast &= \sum_{n=0}^{Ng-1} n^{2} \left \{ \sum_{i=1}^{Ng} \sum_{j=1}^{Ng} p[i,j] \right \} \text{, where } |i-j|=n\\
 Correlation &= \frac{\sum_{i=1}^{Ng} \sum_{j=1}^{Ng}(ij)p[i,j] - \mu_x \mu_y}{\sigma_x \sigma_y} \\
 Entropy &= -\sum_{i}\sum_{j} p[i,j] log(p[i,j])\\
\end{align}''')
```

```python
digits = load_digits()
X = digits.data.copy()

for i, img in enumerate(digits.images):
    np.append(X[i], mh.features.haralick(
        img.astype(np.uint8)).ravel())

X_train, X_test, y_train, y_test = train_test_split(
    X, digits.target, train_size=0.75)
```

```python
tpot = TPOT(generations=6, population_size=101,
            random_state=46, verbosity=2)
tpot.fit(X_train, y_train)
```

```python
%matplotlib inline
context = dl.nb.Context('extracting_texture')
dl.nb.RcWidget(context)
```

```python
print('Score {:.2f}'.format(tpot.score(X_train, y_train, X_test, y_test)))
dl.plotting.img_show(plt.gca(), digits.images[0])
plt.title('Original Image')

plt.figure()
dl.plotting.img_show(plt.gca(), digits.data[0].reshape((8, 8)))
plt.title('Core Features')
plt.figure()
dl.plotting.img_show(plt.gca(), mh.features.haralick(
    digits.images[0].astype(np.uint8)))
plt.title('Haralick Features')
```

```python

```


# **Propagação de erros em Python3: Jupyter Notebook - Data analytics**

## *Subprojeto "Protótipo de Magnetômetro Portátil com Internet das Coisas" (Computação Física) da UFES/Alegre*

### Eduardo Destefani Stefanato, IC FAPES
### Professor : Roberto Colistete Jr., em 05/02/2021.

__________________________________

### O que é propagação de erros/incerteza?

Em estatística, propagação de incerteza ou propagação de erro (ambas diferem na forma de apresentar seus valores) é uma forma de verificar a confiabilidade dos dados de uma certa amostra ou medida, quando esta é submetida a diferentes operações matemáticas. Ela define como as incertezas ou erros das variáveis estão relacionadas e fornece a melhor estimativa para aquele conjunto de dados.

Incerteza é uma quantidade (dimensional ou adimensional) que expressa a confiabilidade de um conjunto de dados, dada a sua dispersão, independentemente do valor verdadeiro. A entidade máxima para os padrões de medidas de incerteza é o Escritório Internacional de Pesos e Medidas (BIPM).

Erro é a diferença entre o valor de uma certa medida e o seu valor verdadeiro.

#### Importação dos módulos Python para o programa

```python
# Data sat
from sympy import *
```

#### Desenvolvimento

Classe de calculos das derivadas parciais e listas de erros (incertezas):

```python
class uncertz :

    def __init__(self, name) :
        self.name = name

    def grad(*vartuple) :
        diff_func = []
        func = input('\nFunção: ')
        for i in vartuple :
            i = Symbol('{}' .format(i))
            diff_func.append(diff(func, i))

        return diff_func[1:]

    def error(*errtuple) :
        err_func = []
        for i in errtuple :
            err_func.append(i)

        return err_func[1:]
```

A celula a seguir possui o formulário que deve ser preenchido pelo usuário no cálculo da incerteza da função desejada:

```python
f = uncertz(input('Nome da função: '))

loop = True
while loop :
    try:
        qt_gran = int(input('\nNúmero de grandezas: '))
        loop = False
    except ValueError:
        loop = True
        print('Você não digitou um número inteiro.')

count = 0
lst_var = []
for i in range(qt_gran) :
    count += 1
    i = input('Variável {}: '. format(count))
    lst_var.append(i)

print('\n')
count = 0
lst_err = []
for i in range(qt_gran) :
    count += 1
    i = float(input('Incerteza da variável {}: '. format(count)))
    lst_err.append(i)

count = 0
lst_val = []
for i in range(qt_gran) :
    count += 1
    i = float(input('Valor da variável {}: '. format(count)))
    lst_val.append(i)

vartuple = tuple(lst_var)
errtuple = tuple(lst_err)
valtuple = tuple(lst_val)

dict_data = {}
for var, val in list(zip(vartuple, valtuple)):
    dict_data[var] = val

grad = f.grad(*vartuple)
error = f.error(*errtuple)

BIMP = prod_int = sqrt(sum(tuple(map(lambda a,b: (a*b)**2, grad, error))))
BIMP = BIMP.subs(dict_data); print('\n+/-', BIMP.round(2))
```

    Nome da função:  Momento

    Número de grandezas:  2
    Variável 1:  m
    Variável 2:  v

    Incerteza da variável 1:  0.01
    Incerteza da variável 2:  0.2
    Valor da variável 1:  89
    Valor da variável 2:  0.8

    Função:  m*v

    +/- 17.80

Fonte: [Propagação de erros](https://pt.wikipedia.org/wiki/Propaga%C3%A7%C3%A3o_de_erros)
End.