# Numerické řešení Stokesova problému pomocí MKO

Stokesův problém modeluje stacionární velmi pomalé proudění vazké nestlačitelné tekutiny. Je posán následujícími rovnicemi

\begin{eqnarray}
\nabla\cdot U &=& 0, \\
\nabla p &=& \nabla \cdot(\mu \nabla U).
\end{eqnarray}

kde $p$ je tlak, $U$ je rychlost (vektor) a $\mu$ je dynamická viskozita.

Pozn. pro $\mu=konst$ lze ve druhé rovnici nahradit tlak součinem $\mu p$ a rovnice se pak zjednoduší na $\nabla p = \Delta U$.

```julia
using PyPlot;
```

```julia
include("mesh.jl");
include("fields.jl");
include("operators.jl");
```

## Definice sítě, počátečních a okrajových podmínek

```julia
n = 25
m = cartesian_mesh(n,n);
```

```julia
plot_mesh(m)
axis("equal");
```

```julia
p = ScalarField(m);
for name ∈ ["left", "right", "bottom", "top"]
    set_neumann_patch!(p, name)
end
```

```julia
u = ScalarField(m)
set_dirichlet_patch!(u, "left", 0.0);
set_dirichlet_patch!(u, "right", 0.0);
set_dirichlet_patch!(u, "bottom", 0.0);
set_dirichlet_patch!(u, "top", 1.0);

v = ScalarField(m)
set_dirichlet_patch!(v, "left", 0.0);
set_dirichlet_patch!(v, "right", 0.0);
set_dirichlet_patch!(v, "bottom", 0.0);
set_dirichlet_patch!(v, "top", 0.0);
```

### Sdružený algoritmus

Soustava rovnic pro řešení Stokesova problému je
$$
    \begin{bmatrix}
    L & 0 & G_x \\
    0 & L & G_y \\
    D_x & D_y & 0
    \end{bmatrix}
    \begin{bmatrix} u \\ v \\ p \end{bmatrix}
    =
    \begin{bmatrix} b_u \\ b_v \\ b_0 \end{bmatrix}.
$$
kde $L$ je matice vzniklá aproximací $-\Delta$, $G_x$ je aproximace $\partial_x$ a $G_y$ je aproximace $\partial_y$. Podobně i $D_x$ a $D_y$.

```julia
μ = 1.e-4;
```

```julia
eq1 = -Δ(μ, u);
eq2 = -Δ(μ, v);
eq3 = ∂x(p);
eq4 = ∂y(p);
eq5 = ∂x(u);
eq6 = ∂y(v);
```

```julia
L  = eq1.A;
Gx = eq3.A;
Gy = eq4.A;
Dx = eq5.A;
Dy = eq6.A;
O  = spzeros(length(cells(m)),length(cells(m)));

M = [L  O  Gx;
     O  L  Gy;
     Dx Dy O];

bu = -eq1.b - eq3.b;
bv = -eq2.b - eq4.b;
b0 = -eq5.b - eq6.b;
b = [bu; bv; b0];
```

Matice $M$ je ale singulární!

Lze to ověřit například výpočtem čísla podmíněnosti
$$
\kappa_A = ||A|| ||A^{-1}||.
$$

To by sice pro singulární matice nemělo být definované (viz $A^{-1}$ ve vzorci), nicméně pro téměř singulární matice je $\kappa$ veliké. Při odhadu $\kappa$ tak může dojít buď k chybe výpočtu z důvodu singularity matice a nebo díky zaokrouhlovací chybě výpočet proběhne a vypočtený odhad $\kappa$ bude veliký.

```julia
# Odhad κ
cond(Array(M))
```

    8.029834350703744e16

Přidám proto k soustavě 1 rovnici p[end]=0. Touto rovnicí fixuji hodnotu tlaku v jednom bodě sítě. Singularita soustavy je totiž způsobena tím, že tlak je ve Stokesově problému určen až na konstantu.

```julia
M[end,end] = n*n;
cond(Array(M))
```

    120001.79818025605

```julia
sol = M \ b;
```

Řešení v poli `sol` teď obsahuje složky `[u, v, p]`. Tyto složky načteme z příslušných částí `sol`.

```julia
uu = sol[1:n*n];
vv = sol[n*n+1:2*n*n];
pp = reshape(sol[2*n*n+1:end], (n,n))';
```

```julia
cx = [v[1] for v in m.centre]
cy = [v[2] for v in m.centre]

contourf(reshape(cx,(n,n))', reshape(cy,(n,n))', pp)
colorbar();  axis("equal")
quiver(cx,cy,uu,vv);
```

## Algoritmus SIMPLE pro Stokesuv problem

Na jednotkovém čtevrci [0,1]x[0,1] řešme soustavu rovnic
\begin{align*}
- \Delta U &= - \nabla p, \\
\nabla \cdot U &= 0.
\end{align*}

s okrajovými podmínkami
$$
    U(x,y) = \left\{
    \begin{array}{ll}
        (1,0) & \text{pro } x \in [0,1] \land y=1,\\
        (0,0) & \text{pro } x=0 \lor x=1 \lor y=0.
    \end{array}
    \right.
$$
a
$$
    \frac{\partial p}{\partial n} = 0 \,\text{pro } x=0 \lor x=1 \lor y=0 \lor y=1.
$$

Diskretizace:
$$
    a_C U_C = \sum_{f} a_F U_F + Q_C - \nabla p_C = H(U)_C - \nabla p_C,
$$

a tedy

$$
    U_C = \frac{1}{a_C} H(U)_C - \frac{1}{a_C} \nabla p_C.
$$

Definujme
$$
    \hat{U}_C = \frac{1}{a_C} H(U)_C.
$$

Rovnice kontinuity

$$
    0 = \nabla \cdot U_C = \nabla \cdot (\hat{U}_C - \frac{1}{a_C} \nabla p_C)
$$

a tedy

$$
    \nabla\cdot(\frac{1}{a_C} \nabla p_C) = \nabla \cdot \hat{U}_C.
$$

Výsledná rovnice pro tlak však stále obsahuje neznámé rychlosti. Algoritmus SIMPLE řeší danou soustavu rovnic iteračně s tím, že se řeší zvlášť rovnice pro rychlosti a zvlášť rovnice pro tlak.

### Algortimus SIMPLE ve variantě s korekcemi tlaku a rychlosti

V této variantě algoritmu SIMPLE vycházíme ze stavu $U^n$ a $p^n$ a pokusíme se vypočítat novou aproximaci $U^{n+1}$ a $p^{n+1}$.

1. Řešíme rovnici pro rychlost s tlakem $p^n$, tj. $U^*_C = \frac{1}{a_c} H(U^*)_C - \frac{1}{a_C} \nabla p_C^n$.
Tato nová rychlost $U^*$ však nesplňuje rovnici kontinuity a navíc byla vypočtena pomocí $p^n$ a nikoliv zatím neznámého $p^{n+1}$.
2. Označme $U' = U^{n+1} - U^*$ a $p'=p^{n+1}-p^n$ a odečtěme od rovnice pro rychlost v iteraci $n+1$ rovnici pro $U^*$, tedy (za předpokladu linearity $H$)
$$
\begin{array}{rll}
U^{n+1}_C &= \frac{1}{a_C} H(U^{n+1})_C - \frac{1}{a_C} \nabla p_C^{n+1}, & \\
U^*_C &= \frac{1}{a_C} H(U^*)_C - \frac{1}{a_C} \nabla p_C^n, & \cdot(-1) \\
\hline \\
U'_C &= \frac{1}{a_C} H(U')_C - \frac{1}{a_C} \nabla p'_C. &
\end{array}
$$
3. Rovnice pro $U'$ však opět svazuje $U'$ a $p'$. Proto v rovnici **zanedbáme** člen s $H(U')$ a dostáváme vztah pro korekci rychlosti
\begin{equation}
    U'_C \approx - \frac{1}{a_C}\nabla p_C.
\end{equation}
4. Z rovnice kontinuity pro $U^{n+1}$ dostáváme $0 = \nabla\cdot U^{n+1} = \nabla\cdot U^* + \nabla\cdot U'$ a tedy
\begin{equation}
    \nabla\cdot(\frac{1}{a_C}\nabla p') = \nabla\cdot U^*.
\end{equation}
Tuto rovnici rovnici doplníme vhodnými okrajovými podmínkami pro $p'$ (tj. tak, aby $p^{n+1} = p^n + p'$ splňovalo zadané podmínky pro tlak) a určíme $p'$ a $p^{n+1}$.
5. Z rovnice pro korekci rychlosti určíme $U' = - \frac{1}{a_C}\nabla p'_C$ a $U^{n+1}=U^* + U'$.
6. Celý postup opakujeme dokud není $||U'||$ a $||p'||$ dostatečně malá.

### Algoritmus SIMPLE ve variantě s tlakem

Tato varianta je velmi podobná předchozímu algoritmu. Narozdíl od něj však nepočítá s korekcemi tlaku ale přímo s tlakem.

1. Řešíme rovnici pro rychlost s tlakem $p^n$, tj. $U^*_C = \frac{1}{a_c} H(U^*)_C - \frac{1}{a_C} \nabla p_C^n$.
2. Označme $\hat U = \frac{1}{a_c} H(U^*)_C$. Potom
\begin{equation}
U^{n+1}_C = \left[\frac{1}{a_c} H(U^{n+1})_C - \hat{U}\right] + \hat{U} - \frac{1}{a_C} \nabla p_C^{n+1}.
\end{equation}
Stejně jako v předchozí variantě **zanedbáme** výraz v hranaté závorce na pravé straně a dostáváme
\begin{equation}
U^{n+1}_C \approx \hat{U} - \frac{1}{a_C} \nabla p_C^{n+1}.
\end{equation}
3. Z rovnice kontinuity pro $U^{n+1}$ dostáváme $0 = \nabla\cdot U^{n+1} = \nabla\cdot \left(\hat{U} - \frac{1}{a_C}\nabla p^{n+1}_C\right)$ a tedy
\begin{equation}
    \nabla(\frac{1}{a_C}\nabla p^{n+1}) = \nabla\cdot \hat{U}.
\end{equation}
4. Z rovnice pro rychlost určíme $U^{n+1} = \hat{U} - \frac{1}{a_C}\nabla p_C^{n+1}$.
5. Celý postup opakujeme dokude není $||U^{n+1}-U^n||$ a $||p^{n+1}-p^n||$ dostatečně malá.

```julia
U = VectorField(m)
set_dirichlet_patch!(U, "left", Vector(0,0));
set_dirichlet_patch!(U, "right", Vector(0,0));
set_dirichlet_patch!(U, "bottom", Vector(0,0));
set_dirichlet_patch!(U, "top", Vector(1,0));
p ← 0.0;
```

Ručně provedeme 1 iteraci algoritmu SIMPLE:

```julia
Ueqn = - Δ(μ, U);
```

```julia
solve!(Ueqn + grad(p));
```

```julia
ra = 1 ./ Ac(Ueqn);
```

```julia
Ubar = VectorField(ra .* H(Ueqn), U.mesh, U.boundaries);
```

```julia
ra .* H(Ueqn);
```

```julia
pEqn = Δ(ra, p) - div(Ubar);
```

Diskretizovaná rovnice pro tlak s Neumannovskými podmínkami má však singulární matici. Proto přidáme k rovnici v bodě 1 $K$-násobek rovnice $p_1=0$. To se projeví přidáním $K$ k prvku $A[1,1]$. Hodnotu $K$ přitom volíme tak, aby (zhruba) odpovídala velikosti $A[1,1]$.

Z rovnice pro rychlost vidíme, že na kartézské síti ve 2D je ve vnitřním bodě sítě
$$
  ra = \frac{h^2}{4\mu}
$$

Dosazením do rovnice pro tlak je v tomto bodě
$$
  A[1,1] = - \frac{h^2}{4\mu} \frac{4}{h^2} = - \frac{1}{\mu}.
$$

```julia
# diagonalni prvek pro bunku odpovidajici cca stredu oblasti
pEqn.A[312,312]
```

    -10000.0

```julia
-1/μ
```

    -10000.0

```julia
# diagonalni prvek v bode 1 (ten je ovlivnen okrajovymi podminkami)
pEqn.A[1,1]
```

    -3666.6666666666656

```julia
solve!(pEqn)
```

```julia
plot_contourf(p)
axis("equal"); colorbar();
```

```julia
U ← Ubar - ra .* grad(p);
```

```julia
component(f::VectorField, c) = [ v[c] for v in f.values ]
```

    component (generic function with 1 method)

```julia
plot_contourf(p); colorbar();
plot_arrows(U);
axis("equal")
```

Výše uvedený postup bychom měli opakovat dokud řešení nezkonverguje ke stacionárnímu stavu.

Zkusíme provést několik iterací:

```julia
p ← 0.0
U ← Vector(0,0);

for iter = 0:50

    U_old, p_old = copy(U.values), copy(p.values)

    Ueqn = -Δ(μ,U)

    solve!(Ueqn + grad(p))

    ra = 1 ./ Ac(Ueqn);

    Ubar = VectorField(ra .* H(Ueqn), U.mesh, U.boundaries);

    pEqn = Δ(ra, p) - div(Ubar);
    pEqn.A[1,1] -= 1/μ

    solve!(pEqn)

    U ← Ubar - ra .* grad(p)

    if rem(iter,5)==0
        nxny = n*n
        pRez = norm(p_old - p.values) / nxny
        URez = norm(U_old - U.values) / nxny
        println(iter, "\t", pRez, "\t", URez)
    end
end
```

    0	0.0010822601278975242	0.0074786027331693004
    5	1.9852261989526276e7	5.418067661052335e7
    10	3.214472815822504e17	8.752312047871419e17
    15	5.19328308820385e27	1.413995865370508e28
    20	8.390105446347867e37	2.284407172606671e38
    25	1.3554790042036535e48	3.6906162587209583e48
    30	2.1898691753735859e58	5.962443356187659e58
    35	3.5378836487736185e68	9.632735641842778e68
    40	5.715693363337064e78	1.5562344227444036e79
    45	9.234094127154024e88	2.514203304837386e89
    50	1.4918311555356117e99	4.061867650317e99

Je vidět, že algoritmus **DIVERGUJE**

### Relaxace v algoritmu SIMPLE

Významnou roli v algoritmu SIMPLE hraje řešení soustav rovnic pro $U$ a $p$. Zatímco soustava rovnic pro $p$ vznikne diskretizací Poissonovy rovnice (a je tedy při vhodné diskretizaci symetrická a pozitivně definitní), soustava pro $U$ už symetrická být nemusí (díky konvektivním členům). Navíc v rovnici pro $U'$ zanedbáváme člen $H(U')/a_C$ a chtěli bychom, aby tento člen byl vzhledem k $U'$ malý. Proto je nutné v rovnici pro rychlost použí **relaxaci**.

Mějme soustavu lineárních rovnic $Ax=b$. Označme $x^*$ přesné řešení této soustavy a $x^n$ $n$-tou iteraci níže popsané relaxační metody. Tu získáme tak, že k řešíme místo původní soustavy soustavu vzniklou součtem $A x^{n+1} = b$ s vhodným násobek $Dx^{n+1}=Dx^n$, kde $D=diag(A)$, tedy
\begin{equation}
  A x^{n+1} + \frac{1-\alpha}{\alpha} D x^{n+1} = b + \frac{1-\alpha}{\alpha} D x^n,
\end{equation}
neboli
\begin{equation}
  \left(\frac{D}{\alpha} + (A-D)\right) x^{n+1} = b + \frac{1-\alpha}{\alpha} D x^n.
\end{equation}
Při vhodné volbě $\alpha \in (0,1)$ je matice relaxované soustavy ODD.

To má význam i pro odhad velikosti $H(U')$. Je-li $U' \approx const.$, pak $H(U')_C/a_C \approx \alpha U'_C$ a tedy lze tento člen v algoritmu SIMPLE zanedbat.

Dalším místem, kde v algoritmu SIMPLE zanedbáváme člen $H(U')$, je vztah pro $U'_C$. Provedeme-li relaxaci rovnice pro rychlost, pak lepší odhad korekce rychlosti bude
$$
    U'_C = \frac{1}{a_C} H(U')_C - \frac{1}{a_C} \nabla p'_C \approx \alpha U'_C - \frac{1}{a_C} \nabla p'_C,
$$
tedy pro $\beta = 1 - \alpha$
$$
    U'_C = - \frac{1}{\beta a_C} \nabla p'_C.
$$
Označme $p" = p'_C / \beta$. Potom $U'_C = - \frac{1}{a_C} \nabla p"_C$ a $p"$ najdeme řešením původní rovnice
\begin{equation}
    \nabla\cdot(\frac{1}{a_C}\nabla p") = \nabla\cdot U^*.
\end{equation}
Tlak $p^{n+1}$ nakonec upravíme dle vztahu $p^{n+1} = p^n + p' = p^n + \beta p"$.

Ve variantě s korekcí rychlosti tedy
1. Relaxujeme rovnici pro rychlost s relaxačním parameterem $\alpha \in (0,1)$.
2. Postupujeme dle standardního algoritmu SIMPLE s tím, že k tlaku přidáváme pouze $\beta$-násobek korekce.

Vhodné relxační parametry jsou $\alpha + \beta \approx 1$, často $\alpha=0.7$ a $\beta = 0.3$.

Celý algoritmus SIMPLE včetně relaxace bude tedy ve variantě s výpočtem tlaku vypadat následovně:

```julia
p ← 0.0
U ← Vector(0,0);

α = 0.7
β = 0.3

for iter = 0:50

    U_old, p_old = copy(U.values), copy(p.values)

    Ueqn = -Δ(μ,U)

    relax!(Ueqn, α)

    solve!(Ueqn + grad(p))

    ra = 1 ./ Ac(Ueqn);

    Ubar = VectorField(ra .* H(Ueqn), U.mesh, U.boundaries);

    pEqn = Δ(ra, p) - div(Ubar);
    pEqn.A[1,1] -= α/μ

    solve!(pEqn)

    p ← β*p + (1-β)*p_old
    U ← Ubar - ra .* grad(p)

    if rem(iter,5)==0
        nxny = n*n
        pRez = norm(p_old - p.values) / nxny
        URez = norm(U_old - U.values) / nxny
        println(iter, "\t", pRez, "\t", URez)
    end
end
```

    0	3.954386650172496e-5	0.003163301495748927
    5	8.056242457412622e-6	0.0006441976603973864
    10	1.745121478795265e-6	0.00026851516499101004
    15	1.6278778503933052e-6	0.0001556614053905297
    20	7.789914849081416e-7	0.00010629255136366686
    25	2.663636409866318e-7	7.61390113312426e-5
    30	6.477468568537342e-8	5.6156610040748246e-5
    35	1.2403007978091426e-8	4.22509911638249e-5
    40	6.28225480228712e-9	3.221046544787941e-5
    45	6.019827881791119e-9	2.4782509432768204e-5
    50	6.034119432285648e-9	1.919523383747767e-5

```julia
plot_contourf(p); colorbar();
plot_arrows(U);
axis("equal")
```

```julia
uu = reshape(component(U,1), (n,n));
vv = reshape(component(U,2), (n,n));
pp = reshape(p.values, (n,n));
```

```julia
# Ψ_x = v, Ψ_y = - u

function streamfunction(u, v, Δx=1, Δy=1)
    Ψ1 = zero(u)

    Ψ1[1,1] = 0.0
    for i=2:size(u,1)
        Ψ1[i,1] = Ψ1[i-1,1] + Δx * (v[i,1]+v[i-1,1])/2
    end
    for j=2:size(u,2)
        Ψ1[:,j] = Ψ1[:,j-1] - Δy * (u[:,j]+u[:,j-1])/2
    end

    Ψ2 = zero(Ψ1)

    Ψ2[1,1] = 0.0
    for j=2:size(u,2)
        Ψ2[1,j] = Ψ2[1,j-1] - Δy * (u[1,j]+u[1,j-1])/2
    end
    for i=2:size(u,1)
        Ψ2[i,:] = Ψ2[i-1,:] + Δx * (v[i,:]+v[i-1,:])/2
    end

    Ψ = (Ψ1 + Ψ2) / 2
    return Ψ
end
```

    streamfunction (generic function with 3 methods)

```julia
Ψ = streamfunction(uu,vv,1/25,1/25);
```

```julia
contour(reshape(cx,(n,n)), reshape(cy,(n,n)), reshape(Ψ,(n,n)), range(0,maximum(Ψ),length=10), colors="black");
contour(reshape(cx,(n,n)), reshape(cy,(n,n)), reshape(Ψ,(n,n)), range(minimum(Ψ),0,length=10), colors="red");
plot_contourf(p); colorbar();
axis("equal");
```

Totéž na jemnější síti

```julia
nf=50;
mf = cartesian_mesh(nf,nf);
```

```julia
pf = ScalarField(mf);
for name ∈ ["left", "right", "bottom", "top"]
    set_neumann_patch!(pf, name)
end
```

```julia
Uf = VectorField(mf)
set_dirichlet_patch!(Uf, "left", Vector(0,0));
set_dirichlet_patch!(Uf, "right", Vector(0,0));
set_dirichlet_patch!(Uf, "bottom", Vector(0,0));
set_dirichlet_patch!(Uf, "top", Vector(1,0));
```

```julia
α = 0.7
β = 0.3

for iter = 0:200

    U_old, p_old = copy(Uf.values), copy(pf.values)

    Ueqn = -Δ(μ,Uf)

    relax!(Ueqn, α)

    solve!(Ueqn + grad(pf))

    ra = 1 ./ Ac(Ueqn);

    Ubar = VectorField(ra .* H(Ueqn), Uf.mesh, Uf.boundaries);

    pEqn = Δ(ra, pf) - div(Ubar);
    pEqn.A[1,1] -= α/μ
    solve!(pEqn)

    pf ← β*pf + (1-β)*p_old
    Uf ← Ubar - ra .* grad(pf)

    if rem(iter,10)==0
        nxny = nf*nf
        pRez = norm(p_old - pf.values) / nxny
        URez = norm(U_old - Uf.values) / nxny
        println(iter, "\t", pRez, "\t", URez)
    end
end
```

    0	3.9379457172874146e-5	0.0011611564177153332
    10	1.0489250342791863e-6	0.00012036686861596944
    20	1.0324717758408478e-6	6.069723874741013e-5
    30	1.4015841293510713e-7	4.086872842453529e-5
    40	2.4189332807793004e-8	3.0301242146274417e-5
    50	2.1747870263312317e-8	2.3646813695293e-5
    60	1.7376601645836254e-8	1.9040678875467713e-5
    70	1.2903774175731714e-8	1.5655501914211847e-5
    80	9.80898968481186e-9	1.3063456830094027e-5
    90	7.65825947400577e-9	1.101993189421841e-5
    100	6.082844692047844e-9	9.373985530473466e-6
    110	4.888089617151349e-9	8.026648855959152e-6
    120	3.9605475141033595e-9	6.909785316313276e-6
    130	3.227785871163617e-9	5.974612165703855e-6
    140	2.6414801333923937e-9	5.185093308008976e-6
    150	2.168162954057708e-9	4.5139356045962825e-6
    160	1.7837631891174593e-9	3.940049012088611e-6
    170	1.4703557791562558e-9	3.4468684717288596e-6
    180	1.2141961536385876e-9	3.021205162228287e-6
    190	1.004500649384674e-9	2.652436053714866e-6
    200	8.326650723099685e-10	2.3319176377559002e-6

```julia
uuf = reshape(component(Uf,1), (nf,nf));
vvf = reshape(component(Uf,2), (nf,nf));
ppf = reshape(pf.values, (nf,nf));
```

```julia
Ψf = streamfunction(uuf,vvf,1/nf,1/nf);
```

```julia
cxf = [v[1] for v in mf.centre];
cyf = [v[2] for v in mf.centre];
```

```julia
contour(reshape(cxf,(nf,nf)), reshape(cyf,(nf,nf)), reshape(Ψf,(nf,nf)), range(0,maximum(Ψf),length=10), colors="black");
contour(reshape(cxf,(nf,nf)), reshape(cyf,(nf,nf)), reshape(Ψf,(nf,nf)), range(minimum(Ψf),0,length=10), colors="red");
plot_contourf(pf); colorbar();
axis("equal");
```

```julia

```


```python
import utils
%load_ext autoreload
%autoreload 2

from utils import build_transf, full_homo_transf, prop_velo, prop_force_torque, comp_jacobian
import utils
from sympy import sqrt
import sympy as sy
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

# Retake 19/20 Problem 2

Denavit-Hartenberg Parameters are from problem 2a).

```python
dh_params = [[0, 0, "n_1", "theta_1"],
             [-90, "a_1", "n_2", "theta_2"],
             [0, "a_2", "n_3", "theta_3"],
             [-90, "a_3", "n_4", "theta_4"],
             [90, 0, 0, "theta_5"],
             [-90, 0, 0, "theta_6"],
            ]
```

```python
transforms = build_transf(dh_params[:4], verbose=True)
full_transform = full_homo_transf(transforms)
```

$\displaystyle {}^0_1T = \left[\begin{matrix}\cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} & 0 & 0\\\sin{\left(\theta_{1} \right)} & \cos{\left(\theta_{1} \right)} & 0 & 0\\0 & 0 & 1 & n_{1}\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^1_2T = \left[\begin{matrix}\cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{2} \right)} & 0 & a_{1}\\0 & 0 & 1 & n_{2}\\- \sin{\left(\theta_{2} \right)} & - \cos{\left(\theta_{2} \right)} & 0 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^2_3T = \left[\begin{matrix}\cos{\left(\theta_{3} \right)} & - \sin{\left(\theta_{3} \right)} & 0 & a_{2}\\\sin{\left(\theta_{3} \right)} & \cos{\left(\theta_{3} \right)} & 0 & 0\\0 & 0 & 1 & n_{3}\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^3_4T = \left[\begin{matrix}\cos{\left(\theta_{4} \right)} & - \sin{\left(\theta_{4} \right)} & 0 & a_{3}\\0 & 0 & 1 & n_{4}\\- \sin{\left(\theta_{4} \right)} & - \cos{\left(\theta_{4} \right)} & 0 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^0_2T = \left[\begin{matrix}\cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} & a_{1} \cos{\left(\theta_{1} \right)} - n_{2} \sin{\left(\theta_{1} \right)}\\\sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} \right)} & \cos{\left(\theta_{1} \right)} & a_{1} \sin{\left(\theta_{1} \right)} + n_{2} \cos{\left(\theta_{1} \right)}\\- \sin{\left(\theta_{2} \right)} & - \cos{\left(\theta_{2} \right)} & 0 & n_{1}\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^0_3T = \left[\begin{matrix}\cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} & - \sin{\left(\theta_{2} + \theta_{3} \right)} \cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} & a_{1} \cos{\left(\theta_{1} \right)} + a_{2} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} - n_{2} \sin{\left(\theta_{1} \right)} - n_{3} \sin{\left(\theta_{1} \right)}\\\sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} & - \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} + \theta_{3} \right)} & \cos{\left(\theta_{1} \right)} & a_{1} \sin{\left(\theta_{1} \right)} + a_{2} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + n_{2} \cos{\left(\theta_{1} \right)} + n_{3} \cos{\left(\theta_{1} \right)}\\- \sin{\left(\theta_{2} + \theta_{3} \right)} & - \cos{\left(\theta_{2} + \theta_{3} \right)} & 0 & - a_{2} \sin{\left(\theta_{2} \right)} + n_{1}\\0 & 0 & 0 & 1\end{matrix}\right]$

$\displaystyle {}^0_4T = \left[\begin{matrix}\sin{\left(\theta_{1} \right)} \sin{\left(\theta_{4} \right)} + \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{4} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} & \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{4} \right)} - \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} & - \sin{\left(\theta_{2} + \theta_{3} \right)} \cos{\left(\theta_{1} \right)} & a_{1} \cos{\left(\theta_{1} \right)} + a_{2} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + a_{3} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} - n_{2} \sin{\left(\theta_{1} \right)} - n_{3} \sin{\left(\theta_{1} \right)} - n_{4} \sin{\left(\theta_{2} + \theta_{3} \right)} \cos{\left(\theta_{1} \right)}\\\sin{\left(\theta_{1} \right)} \cos{\left(\theta_{4} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} - \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{4} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} - \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{4} \right)} & - \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} + \theta_{3} \right)} & a_{1} \sin{\left(\theta_{1} \right)} + a_{2} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + a_{3} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} + n_{2} \cos{\left(\theta_{1} \right)} + n_{3} \cos{\left(\theta_{1} \right)} - n_{4} \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} + \theta_{3} \right)}\\- \sin{\left(\theta_{2} + \theta_{3} \right)} \cos{\left(\theta_{4} \right)} & \sin{\left(\theta_{4} \right)} \sin{\left(\theta_{2} + \theta_{3} \right)} & - \cos{\left(\theta_{2} + \theta_{3} \right)} & - a_{2} \sin{\left(\theta_{2} \right)} - a_{3} \sin{\left(\theta_{2} + \theta_{3} \right)} + n_{1} - n_{4} \cos{\left(\theta_{2} + \theta_{3} \right)}\\0 & 0 & 0 & 1\end{matrix}\right]$

### 2e) Calculate the position of frame 4 in {0}

```python
full_transform @ sy.Matrix([0, 0, 0, 1])
```

$\displaystyle \left[\begin{matrix}a_{1} \cos{\left(\theta_{1} \right)} + a_{2} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + a_{3} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} - n_{2} \sin{\left(\theta_{1} \right)} - n_{3} \sin{\left(\theta_{1} \right)} - n_{4} \sin{\left(\theta_{2} + \theta_{3} \right)} \cos{\left(\theta_{1} \right)}\\a_{1} \sin{\left(\theta_{1} \right)} + a_{2} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + a_{3} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} + \theta_{3} \right)} + n_{2} \cos{\left(\theta_{1} \right)} + n_{3} \cos{\left(\theta_{1} \right)} - n_{4} \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} + \theta_{3} \right)}\\- a_{2} \sin{\left(\theta_{2} \right)} - a_{3} \sin{\left(\theta_{2} + \theta_{3} \right)} + n_{1} - n_{4} \cos{\left(\theta_{2} + \theta_{3} \right)}\\1\end{matrix}\right]$

### 2f) Propagate angular and linear velocities for frames i=1,2,3

```python
joint_points = [
    sy.Matrix([0, 0, 0]),
    sy.Matrix([sy.Symbol("a_1"), sy.Symbol("n_2"), 0]),
    sy.Matrix([sy.Symbol("a_2"), 0, sy.Symbol("n_3")]),
]
v, omega, joint_params = prop_velo(dh_params[:3], joint_points)
```

$\displaystyle {}^1\omega_1 = \left[\begin{matrix}0\\0\\\dot\theta_{1}\end{matrix}\right]$

$\displaystyle {}^1v_1 = \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

$\displaystyle {}^2\omega_2 = \left[\begin{matrix}- \dot\theta_{1} \sin{\left(\theta_{2} \right)}\\- \dot\theta_{1} \cos{\left(\theta_{2} \right)}\\\dot\theta_{2}\end{matrix}\right]$

$\displaystyle {}^2v_2 = \left[\begin{matrix}- \dot\theta_{1} n_{2} \cos{\left(\theta_{2} \right)}\\\dot\theta_{1} n_{2} \sin{\left(\theta_{2} \right)}\\\dot\theta_{1} a_{1}\end{matrix}\right]$

$\displaystyle {}^3\omega_3 = \left[\begin{matrix}- \dot\theta_{1} \sin{\left(\theta_{2} + \theta_{3} \right)}\\- \dot\theta_{1} \cos{\left(\theta_{2} + \theta_{3} \right)}\\\dot\theta_{2} + \dot\theta_{3}\end{matrix}\right]$

$\displaystyle {}^3v_3 = \left[\begin{matrix}- \dot\theta_{1} n_{2} \cos{\left(\theta_{2} + \theta_{3} \right)} - \dot\theta_{1} n_{3} \cos{\left(\theta_{2} + \theta_{3} \right)} + \dot\theta_{2} a_{2} \sin{\left(\theta_{3} \right)}\\\dot\theta_{1} n_{2} \sin{\left(\theta_{2} + \theta_{3} \right)} + \dot\theta_{1} n_{3} \sin{\left(\theta_{2} + \theta_{3} \right)} + \dot\theta_{2} a_{2} \cos{\left(\theta_{3} \right)}\\\dot\theta_{1} \left(a_{1} + a_{2} \cos{\left(\theta_{2} \right)}\right)\end{matrix}\right]$

```python
J = comp_jacobian(dh_params[:3], joint_points, verbose=False)
```

$\displaystyle {}^3J = \left[\begin{matrix}- \left(n_{2} + n_{3}\right) \cos{\left(\theta_{2} + \theta_{3} \right)} & a_{2} \sin{\left(\theta_{3} \right)} & 0\\\left(n_{2} + n_{3}\right) \sin{\left(\theta_{2} + \theta_{3} \right)} & a_{2} \cos{\left(\theta_{3} \right)} & 0\\a_{1} + a_{2} \cos{\left(\theta_{2} \right)} & 0 & 0\\- \sin{\left(\theta_{2} + \theta_{3} \right)} & 0 & 0\\- \cos{\left(\theta_{2} + \theta_{3} \right)} & 0 & 0\\0 & 1 & 1\end{matrix}\right]$

# Problem 3

```python
dh_params_3 = [
    [0, 0, "L_1", "theta_1"],
    [90, 0, 0, "theta_2"],
    [0, 0, "L_2", 0],
    [90, 0, "d_3", 0],
]
```

```python
joint_points_3 = [
    sy.Matrix([0, 0, sy.Symbol("L_1")]),
    sy.Matrix([0, 0, 0]),
    sy.Matrix([0, -sy.Symbol("L_2"), 0]),
    sy.Matrix([0, -sy.Symbol("d_3"), 0])
]

v, omega, joint_params = prop_velo(dh_params_3, joint_points_3)
```

$\displaystyle {}^1\omega_1 = \left[\begin{matrix}0\\0\\\dot\theta_{1}\end{matrix}\right]$

$\displaystyle {}^1v_1 = \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

$\displaystyle {}^2\omega_2 = \left[\begin{matrix}\dot\theta_{1} \sin{\left(\theta_{2} \right)}\\\dot\theta_{1} \cos{\left(\theta_{2} \right)}\\\dot\theta_{2}\end{matrix}\right]$

$\displaystyle {}^2v_2 = \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

$\displaystyle {}^3\omega_3 = \left[\begin{matrix}\dot\theta_{1} \sin{\left(\theta_{2} \right)}\\\dot\theta_{1} \cos{\left(\theta_{2} \right)}\\\dot\theta_{2}\end{matrix}\right]$

$\displaystyle {}^3v_3 = \left[\begin{matrix}L_{2} \dot\theta_{2}\\0\\- L_{2} \dot\theta_{1} \sin{\left(\theta_{2} \right)} + \dot d_{3}\end{matrix}\right]$

$\displaystyle {}^4\omega_4 = \left[\begin{matrix}\dot\theta_{1} \sin{\left(\theta_{2} \right)}\\\dot\theta_{2}\\- \dot\theta_{1} \cos{\left(\theta_{2} \right)}\end{matrix}\right]$

$\displaystyle {}^4v_4 = \left[\begin{matrix}\dot\theta_{2} \left(L_{2} + d_{3}\right)\\- L_{2} \dot\theta_{1} \sin{\left(\theta_{2} \right)} + \dot d_{3} - \dot\theta_{1} d_{3} \sin{\left(\theta_{2} \right)}\\0\end{matrix}\right]$

```python

```


[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Übung / Exercise 6

```python
import sympy as sp
import matplotlib.pyplot as plt
from sympy.plotting.plot import MatplotlibBackend
sp.init_printing()

def logx_plot(Hw, wrange, **kwargs):
    # modified function from original at
    # Sascha Spors, 2020, Continuous- and Discrete-Time Signals and Systems - Theory and Computational Examples.
    # https://github.com/spatialaudio/signals-and-systems-lecture/blob/master/systems_spectral_domain/bode_plot.ipynb
    # 7d1df23
    'create plot with logarithmic x-axis'
    p = sp.plot(Hw, wrange, show=False, **kwargs)
    backend = MatplotlibBackend(p)
    backend.process_series()
    backend.ax[0].spines['left'].set_position(('axes', 0))
    backend.ax[0].spines['bottom'].set_position(('axes', 0))
    plt.xscale('log')
    plt.grid(True, which='both')

def db(x):
    'compute dB value'
    return 20 * sp.log(sp.Abs(x), 10)

def plot_bode(H):
    logx_plot(db(H.subs(s, sp.I * w)),
              (w, wl, wh),
              xlabel='$\omega$',
              ylabel='Level $20 \log_{10} | H(j \omega) |$ in dB',
              adaptive=False, nb_of_points=Nw)
    logx_plot(sp.arg(H.subs(s, sp.I * w))*180/sp.pi,
              (w, wl, wh),
              xlabel='$\omega$',
              ylabel=r'Phase $\angle H(j \omega)$ in deg',
              adaptive=False, nb_of_points=Nw)
```

```python
s = sp.symbols('s')
w = sp.symbols('omega', real=True)
wl, wh, Nw = 0.01, 100, 2**11
```

# Maximum Phase System

```python
# Max Phase
Hmax = 2*(s-2)/(s+1/2)
Hmax
```

```python
sp.apart(Hmax)
```

```python
plot_bode(Hmax)
```

# Minimum Phase System

```python
# Min Phase
Hmin = 2*(s+2)/(s+1/2)
Hmin
```

```python
sp.apart(Hmin)
```

```python
plot_bode(Hmin)
```

# Allpass System

```python
# Allpass
Hall = (s-2)/(s+2)
Hall
```

```python
sp.apart(Hall)
```

```python
plot_bode(Hall)
```

check if $H(s)_\mathrm{max} = H(s)_\mathrm{min} \cdot H(s)_\mathrm{all}$ by typical sympy $a-b=0$ check routine

```python
print(sp.apart(Hall * Hmin) - sp.apart(Hmax))
print(sp.simplify(Hall * Hmin - Hmax))
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


```python
from devito import *
from examples.seismic.source import DGaussSource, TimeAxis
from examples.seismic import plot_image
import numpy as np

from sympy import init_printing, latex
init_printing(use_latex='mathjax')
```

```python
# Initial grid: 1km x 1km, with spacing 100m
extent = (2000., 2000.)
shape = (81, 81)
x = SpaceDimension(name='x', spacing=Constant(name='h_x', value=extent[0]/(shape[0]-1)))
z = SpaceDimension(name='z', spacing=Constant(name='h_z', value=extent[1]/(shape[1]-1)))
grid = Grid(extent=extent, shape=shape, dimensions=(x, z))
```

```python
# Timestep size from Eq. 7 with V_p=6000. and dx=100
t0, tn = 0., 200.
dt = 1e2*(1. / np.sqrt(2.)) / 60.
time_range = TimeAxis(start=t0, stop=tn, step=dt)

src = DGaussSource(name='src', grid=grid, f0=0.01, time_range=time_range, a=0.004)
src.coordinates.data[:] = [1000., 1000.]
```

```python
#NBVAL_SKIP

src.show()
```

```python
# Now we create the velocity and pressure fields
p = TimeFunction(name='p', grid=grid, staggered=NODE, space_order=2, time_order=1)
v = VectorTimeFunction(name='v', grid=grid, space_order=2, time_order=1)
```

```python
from devito.finite_differences.operators import div, grad
t = grid.stepping_dim
time = grid.time_dim

# We need some initial conditions
V_p = 4.0
density = 1.

ro = 1/density * dt
l2m = V_p*V_p*density * dt

# The source injection term
src_p = src.inject(field=p.forward, expr=src)

# 2nd order acoustic according to fdelmoc
u_v_2 = Eq(v.forward, v + ro * grad(p))
u_p_2 = Eq(p.forward, p + l2m * div(v.forward))
```

```python
u_v_2
```

$\displaystyle \left[\begin{matrix}\operatorname{v_{x}}{\left(t + dt,x + \frac{h_{x}}{2},z \right)}\\\operatorname{v_{z}}{\left(t + dt,x,z + \frac{h_{z}}{2} \right)}\end{matrix}\right] = \left[\begin{matrix}\operatorname{v_{x}}{\left(t,x + \frac{h_{x}}{2},z \right)} + 1.17851130197758 \frac{\partial}{\partial x} p{\left(t,x,z \right)}\\\operatorname{v_{z}}{\left(t,x,z + \frac{h_{z}}{2} \right)} + 1.17851130197758 \frac{\partial}{\partial z} p{\left(t,x,z \right)}\end{matrix}\right]$

```python
u_p_2
```

$\displaystyle p{\left(t + dt,x,z \right)} = p{\left(t,x,z \right)} + 18.8561808316413 \frac{\partial}{\partial x} \operatorname{v_{x}}{\left(t + dt,x + \frac{h_{x}}{2},z \right)} + 18.8561808316413 \frac{\partial}{\partial z} \operatorname{v_{z}}{\left(t + dt,x,z + \frac{h_{z}}{2} \right)}$

```python
op_2 = Operator([u_v_2, u_p_2] + src_p)
```

```python
#NBVAL_IGNORE_OUTPUT

# Propagate the source
op_2(time=src.time_range.num-1)
```

    Operator `Kernel` ran in 0.01 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
      PerfEntry(time=0.0036389999999999942, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[])),
     (PerfKey(name='section1', rank=None),
      PerfEntry(time=0.0006309999999999991, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

```python
#NBVAL_SKIP

# Let's see what we got....
plot_image(v[0].data[0])
plot_image(v[1].data[0])
plot_image(p.data[0])
```

```python
norm_p = norm(p)
assert np.isclose(norm_p, .35098, atol=1e-4, rtol=0)
```

```python
# # 4th order acoustic according to fdelmoc
p4 = TimeFunction(name='p', grid=grid, staggered=NODE, space_order=4, time_order=1)
v4 = VectorTimeFunction(name='v', grid=grid, space_order=4, time_order=1)
src_p = src.inject(field=p4.forward, expr=src)
u_v_4 = Eq(v4.forward, v4 + ro * grad(p4))
u_p_4 = Eq(p4.forward, p4 + l2m * div(v4.forward))
```

```python
#NBVAL_IGNORE_OUTPUT
op_4 = Operator([u_v_4, u_p_4] + src_p)
# Propagate the source
op_4(time=src.time_range.num-1)
```

    Operator `Kernel` ran in 0.01 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
      PerfEntry(time=0.005507999999999999, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[])),
     (PerfKey(name='section1', rank=None),
      PerfEntry(time=0.0007870000000000003, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

```python
#NBVAL_SKIP

# Let's see what we got....
plot_image(v4[0].data[-1])
plot_image(v4[1].data[-1])
plot_image(p4.data[-1])
```

```python
norm_p = norm(p)
assert np.isclose(norm_p, .35098, atol=1e-4, rtol=0)
```


# Ten Sigma Event Extension
Statistical Consequences of Fat Tails (Page 53): [Link to ebook PDF](https://researchers.one/articles/statistical-consequences-of-fat-tails-real-world-preasymptotics-epistemology-and-applications/5f52699d36a3e45f17ae7e36)

```python
import numpy as np
import pandas as pd
from scipy.stats import kurtosis as scipy_kurtosis
from sympy.stats import P, E, variance, std, Die, Normal, StudentT
from sympy import Eq, simplify
```

```python
Z = Normal('Z', 0, 1) # Declare a Normal random variable with mean 0, std 1
T = StudentT('T', 2)
```

# Replicate Page 53

We want to find P(Gaussian|Event):

\begin{equation*}
\frac{P(Gaussian)*P(Event|Gaussian)}{\Bigl(1-P(Gaussian)\Bigr)*P(Event|NonGaussian)+P(Gaussian)*P(Event|Gaussian)}
\end{equation*}

```python
p_gaussian_list = [0.5, 0.999, 0.9999, 0.99999, 0.999999, 1] # P(Gaussian) values to check for

p_if_gauss = P(Z>10).evalf()
p_if_nongauss = P(T>10).evalf()
```

```python
1/p_if_gauss # Should be 1.31x10^23
```

$\displaystyle 1.3123612710498 \cdot 10^{23}$

```python
1/p_if_nongauss # # Should be 203
```

$\displaystyle 202.995049383621$

```python
# Evaluate the equation for each value in p_gaussian_list
p_gauss_if_event_list = []
for p_gauss in p_gaussian_list:

    numerator = p_gauss * p_if_gauss
    denominator = (1-p_gauss)*p_if_nongauss+p_gauss*p_if_gauss

    p_gauss_if_event = numerator/denominator

    p_gauss_if_event_list.append(p_gauss_if_event)
```

```python
p_gaussian_list
```

    [0.5, 0.999, 0.9999, 0.99999, 0.999999, 1]

```python
p_gauss_if_event_list
```

    [1.54679244093540e-21,
     1.54524564849446e-18,
     1.54663776169147e-17,
     1.54677697301803e-16,
     1.54679089409848e-15,
     1.00000000000000]

```python
# Create DataFrame
d = {'P(Gaussian)':p_gaussian_list, 'P(Gaussian|Event)':p_gauss_if_event_list}
page_53_table = pd.DataFrame(d)
```

```python
page_53_table
```

      P(Gaussian)
      P(Gaussian|Event)

      0
      0.500000
      1.54679244093540e-21

      1
      0.999000
      1.54524564849446e-18

      2
      0.999900
      1.54663776169147e-17

      3
      0.999990
      1.54677697301803e-16

      4
      0.999999
      1.54679089409848e-15

      5
      1.000000
      1.00000000000000

# Extension:

What if you fit a new normal distribution after observing the 10 sigma event. I'm not saying it makes sense but let's see what happens.

Let's suppose the event is from one day in 50 years. Then it's a 1 in 365*50 event.

```python
n = 365*50 # Our dataset holds this many points before the 10 sigma event happens
n
```

    18250

```python
normal_array = np.random.normal(size=n)

df = pd.DataFrame(normal_array, columns = ['normal_sample'])

df.head()
```

      normal_sample

      0
      -1.685426

      1
      -0.614605

      2
      -0.654180

      3
      -0.631316

      4
      -1.486185

```python
df.describe().T
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      normal_sample
      18250.0
      0.003066
      1.000099
      -3.618038
      -0.666777
      0.000853
      0.676138
      3.88181

```python
scipy_kurtosis(df.normal_sample, fisher=False)
```

    2.9903270004741063

So before the tail event:
* std=1
* kurtosis=3
* max observation ~= 4

### Add tail event

```python
new_df = df.append({'normal_sample':10},ignore_index=True)
```

```python
new_df.describe().T
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      normal_sample
      18251.0
      0.003613
      1.002805
      -3.618038
      -0.666752
      0.000918
      0.676495
      10.0

```python
scipy_kurtosis(new_df.normal_sample, fisher=False)
```

    3.4990530638632

After adding the tail event:
* std=1
* kurtosis=3.5
* max=10

Rough conclusion, If we add a 10 sigma event to 50 years of daily gaussian data, ...
* The standard deviation doesn't go up much
* The kurtosis goes up from 3 to 3.5.

Which leaves us with:
* New dataset has mu and sigma of a standard normal, but its kurtosis gives it away as being non-gaussian

notebooks/Notebook-05 - Sympy 10 Sigma Event Extension.ipynb


```python
import pandas as pd
import numpy as np
import os
import scipy.stats as st

import iqplot
import bokeh.io
import bokeh.plotting
from bokeh.models import Legend
import numba

import bebi103

bokeh.io.output_notebook()
```

    Loading BokehJS ...

# Aim 1: Data Validation for Microtubule Experiments #
### Comparing Labeled and Unlabeled tubulin performance ###
In the experiment conducted by Gardner et al., microtubules were labeled with fluorescent markers. We investigate whether or not these fluorescent markers influence tubulin performance, determined by time to catastrophe (s). We look at data gathered from unlabeled and labeled tubulin, and focus on three different comparisons:
1. ECDF of labeled vs unlabeled tubulin
2. Mean time to catastrophe of labeled vs unlabeled tubulin
3. Hypothesis testing assuming identical distributions

Each of these strategies checks whether or not the labeled or unlabeled tubulin datasets are different in some way. If a significant difference does exist, this means that the fluorescent markers have some sort of impact on microtubule performance. This would also mean that the labeled tubulin used in the subsequent experiments do not accurately depict microtubule behavior. In this way, we hope to validate the data collected by confirming that the fluorescent markers do not influence microtubule performance.

To start our investigation, we read in the dataset and save the values in a tidy data frame.

```python
data_path = '../datasets'
file_path = os.path.join(data_path, 'gardner_time_to_catastrophe_dic_tidy.csv')

# get rid of the index column when loading data
df = pd.read_csv(file_path).iloc[:, 1:]
# replace True/False with labeled vs unlabeleed
df['labeled'] = df['labeled'].apply(lambda x: 'labeled tubulin' if x else 'unlabeled tubulin')

df
```

      time to catastrophe (s)
      labeled

      0
      470.0
      labeled tubulin

      1
      1415.0
      labeled tubulin

      2
      130.0
      labeled tubulin

      3
      280.0
      labeled tubulin

      4
      550.0
      labeled tubulin

      ...
      ...
      ...

      301
      180.0
      unlabeled tubulin

      302
      145.0
      unlabeled tubulin

      303
      745.0
      unlabeled tubulin

      304
      390.0
      unlabeled tubulin

      305
      1410.0
      unlabeled tubulin

306 rows × 2 columns

### 1. ECDF comparison ###
To determine whether or not microtubule performance is different between labeled and unlabeled tubulin, we first look at the cumulative distributions of the empirical data. If the cumulative distributions occupy the same areas, then the fluorescent markers probably do not have a strong effect on microtubule performance since unlabeled/labeled times to catastrophe are indistinguishable from each other. We use `iqplot` to display the respective ECDFs below and observe whether or not the unlabeled and labeled datasets are identically distributed.

```python
p = iqplot.ecdf(
    data=df,
    q='time to catastrophe (s)',
    cats='labeled',
    style='staircase',
    conf_int = True
)

bokeh.io.show(p)
```

By a quick, visual inspection of the plot, it looks like the catastrophe times for microtubules with labeled and unlabeled tubulin could be identically distributed. The confidence interval for the unlabeled tubulin almost always overlaps with the labeled tubulin confidence intervals.

Since we are using the confidence intervals to check whether or not the datasets overlap, further investigation of confidence interval generation is worth exploring. The confidence intervals above were calculated with bootstrapping, but we can also use Dvoretzky-Kiefer-Wolfowitz Inequality (DKW) to compute confidence intervals for the ECDF. To start, we define an ecdf function that can compute the ecdf at an arbitrary x value.

```python
def ecdf(x, data):
    """
    This function computes the value of the ECDF built from a 1D array, data,
    at arbitrary points, x, which can also be an array.

    x can be an integer or float, an array of ints, or a list of ints
    """
    data_sorted = np.sort(data)

    ecdf_li = []

    if type(x) == int or type(x) == float:
        index_tup = np.where(data_sorted <= x)[0]
        if index_tup.size == 0:
            ecdf = 0
        else:
            ecdf = (index_tup[-1] + 1) / len(data)
        ecdf_li.append(ecdf)
    else:
        for value in x:
            index_tup = np.where(data_sorted <= value)[0]
            if index_tup.size == 0:
                ecdf = 0
            else:
                ecdf = (index_tup[-1] + 1) / len(data)
            ecdf_li.append(ecdf)

    return np.array(ecdf_li)
```

The DKW inequality states that for any $\epsilon > 0$,

\begin{align}
P\left(\mathrm{sup}_x \left|F(x) - \hat{F}(x)\right| > \epsilon\right) \le 2\mathrm{e}^{-2 n \epsilon^2},
\end{align}

To calculate the DKW inequality for the microtubule catastrophe data then, we first calculate $\alpha$ which while be used to calculate $\epsilon$. For the 95% confidence interval:

\begin{align}
100*(1-\alpha) & = 95 \\
1-\alpha & = 0.95 \\
0.05 &= \alpha \\
\end{align}

Now we create a function to get $\epsilon$ for a given dataset (since $n$ might vary), which I will later use to calculate the upper and lower bounds. I use the calculated $\alpha$ value and the expression:

\begin{align}
\epsilon &= \sqrt{\frac{1}{2n} \log{\frac{2}{\alpha}}} \\
\end{align}

```python
alpha = 0.05

def calc_epsilon(data):
    n = len(data)
    x = 1/(2*n)
    l = np.log(2/alpha)
    return np.sqrt(x*l)
```

Next we create a function that returns the lower bound, given by the expression:

\begin{align}
L(x) = \max\left(0, \hat{F}(x) - \epsilon\right),
\end{align}

```python
def lower_bound(data):
    """
    for a given array of experimental data,
    this function returns a list of the DKW lower bound values
    corresponding to the given data points
    """
    ep = calc_epsilon(data)
    l_bounds = []
    for x in data:
        lower = np.maximum(0, x-ep)
        l_bounds.append(lower)
    return l_bounds
```

Now we create a function that returns the upper bound, given by the expression:

\begin{align}
U(x) = \min\left(1, \hat{F}(x) + \epsilon\right).
\end{align}

```python
def upper_bound(data):
    """
    for a given array of experimental data,
    this function returns a list of the DKW upper bound values
    corresponding to the given data points
    """
    ep = calc_epsilon(data)
    u_bounds = []
    for x in data:
        upper = np.minimum(1, x+ep)
        u_bounds.append(upper)
    return u_bounds
```

Now I want to plot the confidence intervals for the ECDFs of the labeled and unlabeled tubulin times to catastrophe. I use the previously made `unlabeled_tubulin` and `labeled tubulin` arrays, along with the `ecdf` method made in part e to obtain the ecdf values of the two arrays. I use the `lower_bound` and `upper_bound` methods just created to also store the lower and upper bounds for each dataset.

For the **unlabeled** tubulin dataset I create a dataframe that holds all the values needed for plotting. I sort the dataset by the value so I can plot in order.

```python
values_u = []
ecdf_u = []
for x in unlabeled_tubulin:
    values_u.append(x)
    a = ecdf([x], unlabeled_tubulin)
    ecdf_u.append(a[0])
df_unlabeled = pd.DataFrame(data = {"value":values_u,
                                    "ecdf":ecdf_u,
                                   }
                           )
df_unlabeled = df_unlabeled.sort_values(by = ["value"])
e = df_unlabeled.loc[:, "ecdf"]
lower_u = lower_bound(e)
upper_u = upper_bound(e)

df_unlabeled["lower_bound"] = lower_u
df_unlabeled["upper_bound"] = upper_u

df_unlabeled.head()
```

      value
      ecdf
      lower_bound
      upper_bound

      29
      40.0
      0.010526
      0.0
      0.149865

      47
      60.0
      0.021053
      0.0
      0.160391

      28
      75.0
      0.031579
      0.0
      0.170917

      85
      80.0
      0.042105
      0.0
      0.181444

      61
      85.0
      0.063158
      0.0
      0.202496

Now for the **labeled** tubulin dataset I repeat this procedure of creating a dataframe and then plotting.

```python
values_l = []
ecdf_l = []
for x in labeled_tubulin:
    values_l.append(x)
    a = ecdf([x], labeled_tubulin)
    ecdf_l.append(a[0])
df_labeled = pd.DataFrame(data = {"value":values_l,
                                  "ecdf":ecdf_l,
                                 }
                         )
df_labeled = df_labeled.sort_values(by = ["value"])
e_l = df_labeled.loc[:, "ecdf"]
lower_l = lower_bound(e_l)
upper_l = upper_bound(e_l)

df_labeled["lower_bound"] = lower_l
df_labeled["upper_bound"] = upper_l

df_labeled.head()
```

      value
      ecdf
      lower_bound
      upper_bound

      10
      55.0
      0.004739
      0.0
      0.098235

      15
      60.0
      0.009479
      0.0
      0.102974

      34
      65.0
      0.018957
      0.0
      0.112453

      5
      65.0
      0.018957
      0.0
      0.112453

      41
      75.0
      0.023697
      0.0
      0.117192

Finally, I create the final plot which overlays the two ecdf's with their respective confidence intervals, calculated with the DKW lower/upper bound expressions.

```python
p2 = bokeh.plotting.figure(
    width=800,
    height=400,
    x_axis_label="time to catastrophe",
    y_axis_label="ecdf",
    title = "unlabeled vs. labeled tubulin",
)

e = p2.line(source = df_unlabeled, x = "value", y = "ecdf",
             color = "#b2abd2", line_width = 3, alpha = 0.7)

l = p2.circle(source = df_unlabeled, x = "value", y = "lower_bound",
             color = "#5e3c99", alpha = 0.5)

u = p2.circle(source = df_unlabeled, x = "value", y = "upper_bound",
             color = "#5e3c99", alpha = 0.5)

e_l = p2.line(source = df_labeled, x = "value", y = "ecdf",
               color = "#fdb863", line_width = 3, alpha = 0.7)

l_l = p2.circle(source = df_labeled, x = "value", y = "lower_bound",
               color = "#e66101", alpha = 0.3)

u_l = p2.circle(source = df_labeled, x = "value", y = "upper_bound",
               color = "#e66101", alpha = 0.3)

legend = Legend(items=[("unlabeled ecdf"   , [e]),
                       ("unlabeled lower bound" , [l]),
                       ("unlabeled upper bound" , [u]),
                       ("labeled ecdf"   , [e_l]),
                       ("labeled lower bound" , [l_l]),
                       ("labeled upper bound" , [u_l]),], location="center")

p2.add_layout(legend, 'right')
p2.legend.click_policy = "hide"

bokeh.io.show(p2)
```

The purple dots here show the unlabeled tubulin bounds while the orange dots show the labeled tubulin bounds, and the lines show the ecdf values.

Comparing the lower bounds, the orange and purple dots seem to follow the same trajectory. This means that the unlabeled and labeled tubulin lower bound values are very similar.

Comparing the upper bounds, the labeled values the purple dots are noticeably above the orange dots. This means that the unlabeled upper bound values are slightly higher than the labeled upper bound values, though the shape of the bounds is the same.

Though the upper bound values are not as aligned as the lower bound values, it is still reasonable to conclude that these confidence intervals are very similar. Therefore, this quick visual check supports the hypothesis that microtubule times to catastrophe are identically distributed between unlabeled and labeled tubulin.

This conclusion matches what we found from the iqplot calculations.

### 2. Mean time to catastrophe comparison ###
Next, we compare the mean times to catastrophe between labeled and unlabeled tubulin to detect any possible differences in performance. If the mean times are close to each other, there is more reason to believe that the fluorescent markers do not affect microtubule performance. To check this, we use nonparametric bootstrapping to compute confidence intervals for the plug-in estimate for the mean time to catastrophe for each of the two conditions. First we define some functions to calculate our bootstrap replicate for each bootstrap sample.

```python
rg = np.random.default_rng()

# set up numpy arrays with values for the labeled and unlabeled tubulin
unlabeled_tubulin = df.loc[df['labeled'] == 'unlabeled tubulin', 'time to catastrophe (s)'].values
labeled_tubulin = df.loc[df['labeled'] == 'labeled tubulin', 'time to catastrophe (s)'].values

def generate_bootstrap_samples(data):
    """Draw N bootstrap samples from a 1D data set."""
    return rg.choice(data, size=len(data))

def bootstrap_reps_mean(data, N=1):
    """Draw boostrap replicates of the mean from 1D data set."""
    means = np.empty(N)
    for i in range(N):
        means[i] = np.mean(generate_bootstrap_samples(data))
    return means
```

Now we can generate 100,000 bootstrap samples for both the labeled and unlabeled and calculate the plug-in estimate for the mean.

```python
unlabeled_means = bootstrap_reps_mean(unlabeled_tubulin, N=100000)
labeled_means = bootstrap_reps_mean(labeled_tubulin, N=100000)

unlabeled_mean_conf_int = np.percentile(unlabeled_means, [2.5, 97.5])
labeled_mean_conf_int = np.percentile(labeled_means, [2.5, 97.5])

print(f"Unlabeled tubulin time to catastrophe(s) confidence interval: [{unlabeled_mean_conf_int[0]:.2f}, {unlabeled_mean_conf_int[1]:.2f}]")
print(f"Labeled tubulin time to catastrophe(s) confidence interval: [{labeled_mean_conf_int[0]:.2f}, {labeled_mean_conf_int[1]:.2f}]")
```

    Unlabeled tubulin time to catastrophe(s) confidence interval: [353.68, 476.89]
    Labeled tubulin time to catastrophe(s) confidence interval: [401.90, 481.54]

We use the `bebi103` package to visually display these confidence intervals.

```python
labeled_mean = labeled_tubulin.mean()
unlabeled_mean = unlabeled_tubulin.mean()

summaries = [
    dict(label = "unlabeled tubulin", estimate = unlabeled_mean,
         conf_int = unlabeled_mean_conf_int),
    dict(label = "labeled tubulin", estimate = labeled_mean,
         conf_int = labeled_mean_conf_int)
]
bokeh.io.show(
    bebi103.viz.confints(summaries)
)
```

The confidence intervals of the two categories have significant overlap. This calculation supports the previous conclusion from the ecdf since there is not a clear difference in microtubule performance between labeled and unlabeled samples.

Again, since we are using the confidence intervals to check value overlaps it is worth double checking that our confidence interval generation is appropriate. In this case, we can double check our confidence intervals with a theoretical distribution rather than the empirical distribution. Specifically, we can use the normal distribution by the central limit theorem:
\begin{align}
&\mu = \bar{x},\\[1em]
&\sigma^2 = \frac{1}{n(n-1)}\sum_{i=1}^n (x_i - \bar{x})^2,
\end{align}

We define a function to calculate the variance of the data set using this theoretical equation.

```python
def calc_variance(data_array):
    """This function calculates the variance of a 1D data array"""
    n = data_array.size
    mean = data_array.mean()
    numer = 0

    for i in range(n):
        numer += (data_array[i] - mean) ** 2

    denom = n * (n-1)

    return numer/denom
```

Now we perform the calculation and visualize the confidence intervals.

```python
unlabeled_variance = calc_variance(unlabeled_tubulin)
labeled_variance = calc_variance(labeled_tubulin)

labeled_conf1 = st.norm.ppf(0.025, loc=labeled_mean, scale=np.sqrt(labeled_variance))
labeled_conf2 = st.norm.ppf(0.975, loc=labeled_mean, scale=np.sqrt(labeled_variance))

unlabeled_conf1 = st.norm.ppf(0.025, loc=unlabeled_mean, scale=np.sqrt(unlabeled_variance))
unlabeled_conf2 = st.norm.ppf(0.975, loc=unlabeled_mean, scale=np.sqrt(unlabeled_variance))

print(f"Unlabeled tubulin time to catastrophe(s) confidence interval: [{unlabeled_conf1:.2f}, {unlabeled_conf2:.2f}]")
print(f"Labeled tubulin time to catastrophe(s) confidence interval: [{labeled_conf1:.2f}, {labeled_conf2:.2f}]")

summaries = [
    dict(label = "unlabeled tubulin", estimate = unlabeled_mean,
         conf_int = [unlabeled_conf1, unlabeled_conf2]),
    dict(label = "labeled tubulin", estimate = labeled_mean,
         conf_int = [labeled_conf1, labeled_conf2])
]
bokeh.io.show(
    bebi103.viz.confints(summaries)
)
```

    Unlabeled tubulin time to catastrophe(s) confidence interval: [350.84, 474.21]
    Labeled tubulin time to catastrophe(s) confidence interval: [400.74, 480.68]

When comparing the confidence interval calculation from the theoretical distribution to the confidence interval derived from the empirical distribution, we can see that the confidence intervals are very similar. Again, there does not seem to be a significant difference between the times to catastrophe between the unlabeled and labeled tubulin.

### 3. Hypothesis testing assuming identical distributions ###
Next, we use a permutation hypothesis test to test the hypothesis that the distribution of catastrophe times for microtubules with labeled tubulin is the same as that for unlabeled tubulin.

#### Step 1: State the null hypothesis.
> The null hypothesis is that the time to catastrophe for labeled and unlabeled tubulin are identically distributed.

#### Step 2: Define a test statistic.
> For our first NHST experiment, the test statistic that will be used is difference in means. This test statistic will offer a good comparison to results acquired in part a and b, when we compared the confidence intervals of the means of the two categories.

> For our second NHST experiment, the test statistic that will be used is difference in variance. Time to catastrophe can be modeled by a combination of exponential processes (as shown in HW6), and we know that we can get a normal approximation of the distribution using the equating moments or taylor expansion method. Since a normal distribution can be described by the mean and variance, we decided to also compare the difference of variance as a test statistic since we are interested in the question of if the labeled and unlabeled tubulin time to catastrophe come from the same distribution.

> For our third NHST experiment, the test statistic that will be used is difference in medians. We decided to conduct this test to compare a different parameter that can describe a distribution to offer more information about if the two categories come from the same distribution. This was mainly done out of curiosity.

#### Step 3: Simulate data acquisition for the scenario where the null hypothesis is true, many many times.

> We will concatenate the two data sets, randomly shuffle them, designate the first entries in the shuffled array to be a “labeled” data set and the rest to be a “unlabeled” data set. Our null hypothesis posits that both the labeled and unlabeled tubulin catastrophe times come from the same distribution, so our concatenated data set will include all points from both categories.

```python
@numba.njit
def generate_perm_sample(x, y):
    """Generate a permutation sample."""
    combined_data = np.concatenate((x, y))
    np.random.shuffle(combined_data)
    half_index = int(combined_data.size / 2)

    return combined_data[:half_index], combined_data[half_index:]
```

#### Step 4a: (NHST Experiment 1) Compute the p-value (the fraction of simulations for which the test statistic (diff in mean) is at least as extreme as the test statistic computed from the measured data). Do this 10 million times.

```python
@numba.njit
def generate_perm_reps_diff_mean(x, y, N):
    """Generate array of permuation replicates."""
    out = np.empty(N)
    for i in range(N):
        x_perm, y_perm = generate_perm_sample(x, y)
        out[i] = np.mean(x_perm) - np.mean(y_perm)

    return out

# Compute test statistic for original data set
diff_mean = np.mean(labeled_tubulin) - np.mean(unlabeled_tubulin)

# Draw replicates
perm_reps = generate_perm_reps_diff_mean(labeled_tubulin, unlabeled_tubulin, 10000000)

# Compute p-value
p_val = np.sum(perm_reps >= diff_mean) / len(perm_reps)

print('NHST experiment 1: Difference in mean p-value =', p_val)
```

    NHST experiment 1: Difference in mean p-value = 0.2058407

#### Step 4b: (NHST Experiment 2) Compute the p-value (the fraction of simulations for which the test statistic (diff in variance) is at least as extreme as the test statistic computed from the measured data). Do this 10 million times.

```python
@numba.njit
def generate_perm_reps_diff_variance(x, y, N):
    """Generate array of permuation replicates."""
    out = np.empty(N)
    for i in range(N):
        x_perm, y_perm = generate_perm_sample(x, y)
        out[i] = np.var(x_perm) - np.var(y_perm)

    return out

# Compute test statistic for original data set
diff_variance = np.var(labeled_tubulin) - np.var(unlabeled_tubulin)

# Draw replicates
perm_reps = generate_perm_reps_diff_variance(labeled_tubulin, unlabeled_tubulin, 10000000)

# Compute p-value
p_val = np.sum(perm_reps >= diff_variance) / len(perm_reps)

print('NHST experiment 2: Difference in variance p-value =', p_val)
```

    NHST experiment 2: Difference in variance p-value = 0.5942287

#### Step 4c: (NHST Experiment 3) Compute the p-value (the fraction of simulations for which the test statistic (diff in median) is at least as extreme as the test statistic computed from the measured data). Do this 10 million times.

```python
@numba.njit
def generate_perm_reps_diff_median(x, y, N):
    """Generate array of permuation replicates."""
    out = np.empty(N)
    for i in range(N):
        x_perm, y_perm = generate_perm_sample(x, y)
        out[i] = np.median(x_perm) - np.median(y_perm)

    return out

# Compute test statistic for original data set
diff_median = np.median(labeled_tubulin) - np.median(unlabeled_tubulin)

# Draw replicates
perm_reps = generate_perm_reps_diff_median(labeled_tubulin, unlabeled_tubulin, 10000000)

# Compute p-value
p_val = np.sum(perm_reps >= diff_median) / len(perm_reps)

print('NHST experiment 3: Difference in median p-value =', p_val)
```

    NHST experiment 3: Difference in median p-value = 0.4767821

The p-value is 0.21 where the test statistic is the difference in means.

The p-value is 0.59 where the test statistic is the difference in variance.

The p-value is 0.48 where the test statistic is the difference in medians.

This means that the probability of getting a difference of means, variances, and medians, respectively, as extreme as was observed under the null hypothesis is relatively high (the null hypothesis being that the control and test samples were drawn from identical distribution). Although this result does not confirm that we can reject the null hypothesis (p-value does not represent the probability that a given hypothesis is "true"), it supports the findings in the previous sections where we do not observe a strong influence of fluorescent markers on microtubule performance.

```python

```


# Tema 4: Aproximación de funciones

```python
!pip install -r https://raw.githubusercontent.com/alexmascension/ANMI/main/requirements.txt
```

```python
from sympy import *
from sympy.matrices import Matrix as mat
from sympy.matrices import randMatrix
from sympy import symbols
import sympy

import numpy as np

from scipy.linalg import orth

from matplotlib import pyplot as plt
import matplotlib as mpl
mpl.rcParams['figure.dpi'] = 150
```

```python
from anmi.genericas import norma_p_func, norma_inf_func

from anmi.T4 import metodo_ruffini
from anmi.T4 import producto_deriv, producto_asecas, producto_escalar_peso, metodo_gram, gram_schmidt_f, polinomios_orto_peso
from anmi.T4 import producto_escalar_trigono, coefs_fourier, coefs_fourier_discr, serie_fourier
from anmi.T4 import polinomios_bernstein
```

```python
x, y, z, a, lambda_ = symbols('x'), symbols('y'), symbols('z'), symbols('a'), symbols('lambda')
```

El objetivo de la aproximación de funciones es encontrar una función de caracteristicas menores o con términos más fáciles para computar, que se aproxime a una función dada en un intervalo o conjunto de puntos determinado. A nivel computacional, también es más eficiente y estable poder emplear cierto tipo de funciones de aproximación.

## Representación anidada (algoritmo de Horner) / Método de Ruffini
Si tenemos un polinomio $p(x) = a_0 + a_1x + a_2x^2+\cdots+a_nx^n$, entonces podemos hacer el polinomio más compacto, en la forma $p(x) = q_1(q_2(\cdots(q_n(x)))$, tal que $q_i(x) = a_{i-1} + a_i(x)$.

```python
help(metodo_ruffini)
```

```python
poli_1 = Poly(x**3 - x**2 + 2*x - 5)
```

```python
metodo_ruffini(poli_1, 1)
```

## Aproximación de funciones
Si $f$ es una función en un espacio vectorial $V$ podemos definir una norma $||\;\;||$, que es una aplicación que cumple:
* $||f|| = 0  \iff f = 0$
* $||\lambda f|| = |\lambda|\,||f||$
* $||f + g|| \le ||f|| + ||g||$

Si fijamos un subespacio $U \subset V$, podemos buscar una función $g\in U$ tal que sea la más similar.

Si queremos medir la distancia entre 2 funciones, hay múltiples opciones:
* Norma 2: $||f||_2 = \left( \int_a^bf(x)^2 dx \right)^\frac{1}{2}$
* Norma p: $||f||_p = \left( \int_a^b |f(x)|^p dx \right)^\frac{1}{p}$
* Norma inf: $||f||_{\inf} = \max_{x \in [a,b]} |f(x)|$

```python
f = x
g1 = sin(x)
g2 = 1 - cos(x)
```

```python
norma_p_func(f - g1, p=2, a=0, b=2*pi)
```

```python
norma_inf_func(f - g1, a=0, b=2*pi)
```

```python
norma_p_func(f - g2, p=2, a=0, b=2*pi)
```

```python
norma_inf_func(f - g2, a=0, b=2*pi)
```

### Aproximación por mínimos cuadrados (continua y discreta)

Para la aproximación definimos el producto escalar como una aplicación $\langle\,,\rangle \; :\,V \times V \rightarrow \mathbb{R}^+$ que verifica:
* $\langle f, f\rangle = \iff f = 0$
* $\langle f, g\rangle =\langle g, f\rangle$
* $\langle\alpha f+\mu g, h\rangle = \alpha\langle f, h\rangle + \mu \langle g, h\rangle$

La norma se define como $||f|| = \langle f, f\rangle^{\frac{1}{2}}$.

Una forma de producto escalar es, por ejemplo:
$$\langle f, g \rangle = \int_a^b f(x)g(x) dx$$

Esa forma puede aplicarse de forma discreta (para un conjunto discreto $I = \{x_1, x_2, \cdots, x_m\}$ como :
$$\langle f, g \rangle = \sum_{i \in I} f(i)g(i)$$

En la aproximación de mínimos cuadrados deseamos encontrar, para una base $\{\phi_0, \cdots, \phi_n\}$ de $U$, un vector $\alpha$ que minimice una función de coste $J(\alpha) = ||f - \sum_0^n \alpha_i\phi_i||^2$. Para ello se emplea el método de Gram, que consiste en encontrar la solución a
$$G\alpha = \bar{f}$$
donde $G_{ij} = \langle\phi_i, \phi_j\rangle$ y $\bar{f}_i = \langle f, \phi_i\rangle$ en un producto escalar definido arbitrariamente.

**IMPORTANTE**

Si la base $U$ es ortonormal, la matriz de Gram es la identidad y, en este caso, la mejor aproximación a $f$ es

$$g = \sum_{i=0}^n \langle f, \phi_i \rangle \phi_i$$

```python
help(producto_deriv)
help(producto_asecas)
```

```python
help(metodo_gram)
```

#### Ejercicio 33

```python
U = [S(1), x, x ** 2]  # S(1) lo hacemos porque hacer 1.diff da error por ser int.
f = E ** x
dict_prod_deriv = metodo_gram(f, U, var=x, func_producto=producto_deriv, a=0, b=1)
```

```python
dict_prod_deriv['poly']
```

```python
dict_prod_deriv['G']
```

```python
dict_prod_deriv['f_bar']
```

```python
dict_prod_deriv['alpha']
```

Ahora con una versión discreta en los puntos 0, 0.2, 0.4, 0.6, 0.8, 1:

```python
U = [S(1), x, x ** 2]  # S(1) lo hacemos porque hacer 1.diff da error por ser int.
f = E ** x
dict_prod_deriv_discreto = metodo_gram(f, U, var=x, func_producto=producto_deriv, I = [0, 0.2, 0.4, 0.6, 0.8, 1])
```

```python
dict_prod_deriv_discreto['poly']
```

```python
dict_prod_deriv_discreto['G']
```

```python
dict_prod_deriv_discreto['f_bar']
```

```python
dict_prod_deriv_discreto['alpha']
```

Ahora la versión continua pero con la función $\langle f, g \rangle = \int_a^b f(x)g(x) dx$

```python
U = [S(1), x, x ** 2]  # S(1) lo hacemos porque hacer 1.diff da error por ser int.
f = E ** x
dict_prod_asecas = metodo_gram(f, U, var=x, func_producto=producto_asecas, a=0, b=1)
```

```python
dict_prod_asecas['poly']
```

```python
x_range = np.linspace(0, 1, 100)
y_real = [(E ** x).subs(x, i) for i in x_range]
y_prod_deriv = [dict_prod_deriv['poly'].subs(x, i) for i in x_range]
y_prod_discreto = [dict_prod_deriv_discreto['poly'].subs(x, i) for i in x_range]
y_prod_asecas = [dict_prod_asecas['poly'].subs(x, i) for i in x_range]

plt.plot(x_range, y_real, label='real')
plt.plot(x_range, y_prod_deriv, label="fg + f'g' continuo")
plt.plot(x_range, y_prod_discreto, label="fg + f'g' discreto")
plt.plot(x_range, y_prod_asecas, label="fg")
plt.legend()
```

#### Ejercicio 34
En el ejercicio piden usar la base $\{1, x\}$. Nosotros lo hacemos con $\{1, x, x^2, x^3, x^4, x^5\}$, que da una representación más real de la aproximación.

Calculamos la versión continua

```python
U = [S(1), x, x ** 2, x ** 3, x ** 4]  # S(1) lo hacemos porque hacer 1.diff da error por ser int.
f = cos(pi * x)/2 + sin(pi/2 * x)/3
dict_prod_continuo = metodo_gram(f, U, var=x, func_producto=producto_asecas, a=-1, b=1)
```

```python
dict_prod_continuo['poly']
```

```python
dict_prod_continuo['G']
```

```python
dict_prod_continuo['f_bar']
```

```python
dict_prod_continuo['alpha']
```

Y ahora la discreta

```python
dict_prod_discreto = metodo_gram(f, U, var=x, func_producto=producto_asecas, I=[-1, 0, 1])
```

```python
dict_prod_discreto['poly']
```

```python
dict_prod_discreto['G']
```

```python
dict_prod_discreto['f_bar']
```

```python
dict_prod_discreto['alpha']
```

Repetimos el proceso, por curiosidad, pero con el producto escalar fg + f'g'

```python
dict_prod_continuo_deriv = metodo_gram(f, U, var=x, func_producto=producto_deriv, a=-1, b=1)
dict_prod_discreto_deriv = metodo_gram(f, U, var=x, func_producto=producto_deriv, I=[-1, 0, 1])
```

```python
dict_prod_continuo_deriv['poly']
```

```python
dict_prod_discreto_deriv['poly']
```

```python
x_range = np.linspace(-1, 1, 100)
y_real = [f.subs(x, i) for i in x_range]
y_prod_continuo = [dict_prod_continuo['poly'].subs(x, i) for i in x_range]
y_prod_discreto = [dict_prod_discreto['poly'].subs(x, i) for i in x_range]
y_prod_continuo_deriv = [dict_prod_continuo_deriv['poly'].subs(x, i) for i in x_range]
y_prod_discreto_deriv = [dict_prod_discreto_deriv['poly'].subs(x, i) for i in x_range]

plt.plot(x_range, y_real, label='real')
plt.plot(x_range, y_prod_continuo, label="fg continuo")
plt.plot(x_range, y_prod_discreto, label="fg discreto")
plt.plot(x_range, y_prod_continuo_deriv, label="fg + f'g' continuo")
plt.plot(x_range, y_prod_discreto_deriv, label="fg + f'g' discreto")

plt.legend()
```

Si nos fijamos, la aproximación con el método discreto es bastante mala, al no encontrar una inversa adecuada.

```python
help(gram_schmidt_f)
```

#### Ejemplo 23

```python
gram_schmidt_f([S(1), x, x ** 2, x ** 3, x ** 4, x ** 5, x ** 6], x, prod_esc=producto_asecas)
```

#### Ejercicio 35

```python
I = [-1, 0, 1, 2]
U_GS = gram_schmidt_f([S(1), x, x ** 2], x, prod_esc=producto_asecas, I=I)
```

```python
for i in U_GS:
    print(i)
```

```python
f = sin(pi/2 * x)
dict_prod_discreto = metodo_gram(f, U_GS, var=x, func_producto=producto_asecas, I=I)
```

```python
dict_prod_discreto['poly']
```

### Aproximación con peso

Para las anteriores definiciones podemos tomar un producto escalar
$$\langle f, g \rangle = \int_a^b f(x)g(x)\omega(x) dx$$

Donde $\omega(x)$ es una función positiva en $(a,b)$

Se puede construir la sucesión de polinomios
$$p_0(x) = 1$$
$$p_1(x) = x - a_1$$
$$p_2(x) = (x - a_2)p_1(x) - b_2p_0(x)$$
$$\cdots$$
$$p_n(x) = (x - a_n)p_{n-1}(x) - b_np_{n-2}(x)$$

Con
$$a_n = \frac{\langle xp_{n-1},p_{n-1}\rangle}{\langle p_{n-1},p_{n-1}\rangle}$$
$$b_n = \frac{\langle xp_{n-1},p_{n-2}\rangle}{\langle p_{n-2},p_{n-2}\rangle}$$

```python
help(polinomios_orto_peso)
```

#### Polinomios de Legendre
Los polinomios de Legendre surgen de la aplicación del método descrito anteriormente para la base $\mathcal{P}(n) = \{1, x, x^2, \cdots, x^n\}$.

```python
# La base para los polinomios tiene que ser ortogonal porque si no los polinomios no son ortogonales entre si!
base = [S(1), x, x**2, x**3, x**4, x**5, x**6]
base_ortogonal = gram_schmidt_f(base, var=x, prod_esc=producto_asecas, a=-1, b=1)
base_legendre = polinomios_orto_peso(base_ortogonal, w=S(1), var=x, a=-1, b=1)
```

```python
base_legendre
```

```python
x_range = np.linspace(-1, 1, 100)

for p in base_legendre['p']:
    plt.plot(x_range, [p.subs(x, i)/abs(p.subs(x, -1)) for i in x_range], label=S(1)/abs(p.subs(x, -1)))

plt.legend(bbox_to_anchor=(1, 0.5))
```

Ahora probamos a resolver algun problema asociado. Por ejemplo, vamos a aproximar sin(x) en [0, 2 * pi]

```python
base = [S(1), x, x**2, x**3]
base_ortogonal = gram_schmidt_f(base, var=x, prod_esc=producto_asecas, a=0, b=2*pi)
base_legendre = polinomios_orto_peso(base_ortogonal, w=S(1), var=x, a=0, b=2*pi)
```

```python
f = sin(x)

U_GS_base = metodo_gram(f, base, var=x, func_producto=producto_escalar_peso, a=0, b=2*pi, w=S(1))

U_GS_ortogonal = metodo_gram(f, base_ortogonal, var=x, func_producto=producto_escalar_peso, a=0, b=2*pi, w=S(1))

U_GS_legendre = metodo_gram(f, base_legendre['p'], var=x, func_producto=producto_escalar_peso, a=0, b=2*pi, w=S(1))
```

```python
expand(U_GS_base['poly'])
```

```python
expand(U_GS_ortogonal['poly'])
```

```python
expand(U_GS_legendre['poly'])
```

Los polinomios son técnicamente los mismos. Sin embargo, las matrices de Gram son mucho más convenientes para la base ortogonal / de legendre.

```python
U_GS_base['G']
```

```python
U_GS_ortogonal['G']
```

```python
U_GS_legendre['G']
```

Ahora vamos a hacer la aproximación gráfica para las funciones $\sin(x)$ y $\ln(x)$

```python
x_range = np.linspace(0, 2 * np.pi, 100)

base = [S(1), x, x**2, x**3, x**4, x**5, x**6, x**7]
f = sin(x)

plt.plot(x_range, [sin(x).subs(x,i) for i in x_range], label='real')
for i in range(3, 8, 2):
    base_sub = base[:i]
    U_GS_base = metodo_gram(f, base_sub, var=x, func_producto=producto_escalar_peso, a=0, b=2*pi, w=S(1))
    plt.plot(x_range, [U_GS_base['poly'].subs(x,i) for i in x_range], label=f'poly_{i}')

plt.legend(bbox_to_anchor=(1, 0.5))
```

```python
x_range = np.linspace(0.01, 2, 100)

base = [S(1), x, x**2, x**3, x**4, x**5, x**6, x**7,]
f = ln(x)/ln(2)

plt.plot(x_range, [f.subs(x,i) for i in x_range], label='real')
for i in range(3, 8, 2):
    base_sub = base[:i]
    U_GS_base = metodo_gram(f, base_sub, var=x, func_producto=producto_escalar_peso, a=0.01, b=2, w=S(1))
    plt.plot(x_range, [U_GS_base['poly'].subs(x,i) for i in x_range], label=f'poly_{i}')

plt.legend(bbox_to_anchor=(1.5, 0.3))
```

Vemos que una aproximación polinomial de log es bastante buena también!

### Polinomios de Chebyshev
Los polinomios de Chevysheb responden a la ecuación $T_n = \cos(n \arccos(x))$. En base a las propiedades trigonométricas, estos polinomios también responden a la forma:
$$T_0 = 1, \;\; T_1 = x$$
$$T_n = 2xT_{n-1} - T_{n-2}$$
En el intervalo [-1, 1]. Fuera de él la expresión T_n da valores complejos, aunque la parte real de esos valores se aproxima mucho a la forma polinomial.

Estos polinomios son ortogonales en el intervalo [-1, 1] estableciendo $\omega(x) = \frac{1}{\sqrt{1-x^2}}$:
$$\int_{-1}^{1} T_nT_m \frac{1}{\sqrt{1-x^2}} dx = \begin{cases}
0 & n\neq m \\
\pi & n = m = 0 \\
\pi/2 & n = m \neq 0
\end {cases}$$

```python
# Aqui los generamos iterativamente
t0 = S(1)
t1 = x
t2 = expand(2 * x * t1 - t0)
t3 = expand(2 * x * t2 - t1)
t4 = expand(2 * x * t3 - t2)
t5 = expand(2 * x * t4 - t3)
t5
```

```python
# También podemos usar la función de sympy
chebyshevt_poly(5)
```

```python
T5 = cos(5 * acos(x))
```

```python
x_range = np.linspace(-3, 3, 1000)
y_t5 = [t5.subs(x, i) for i in x_range]
y_T5 = [re(T5.subs(x, i)) for i in x_range]

plt.plot(x_range, y_t5, label='t5')
plt.plot(x_range, y_T5, label="T5")
plt.legend()
```

Vemos que ambos polinomios son iguales

Ahora vamos a comprobar la ortogonalidad de los polinomios usando el peso adecuado, u otro peso

```python
producto_escalar_peso(chebyshevt_poly(3, x), chebyshevt_poly(5, x), x, w=S(1),a=-1, b=1)
```

```python
producto_escalar_peso(chebyshevt_poly(3, x), chebyshevt_poly(5, x), x, w=1/sqrt(1 - x**2),a=-1, b=1)
```

Vemos que si no añadimos el peso $\omega(x)$ el resultado de la integral no es 0.

```python
producto_escalar_peso(chebyshevt_poly(0, x), chebyshevt_poly(0, x), x, w=1/sqrt(1 - x**2),a=-1, b=1)
```

```python
producto_escalar_peso(chebyshevt_poly(1, x), chebyshevt_poly(1, x), x, w=1/sqrt(1 - x**2),a=-1, b=1)
```

```python
producto_escalar_peso(chebyshevt_poly(3, x), chebyshevt_poly(3, x), x, w=1/sqrt(1 - x**2),a=-1, b=1)
```

```python
producto_escalar_peso(chebyshevt_poly(5, x), chebyshevt_poly(5, x), x, w=1/sqrt(1 - x**2),a=-1, b=1)
```

Para dos polinomios iguales, los resultados son los esperados

### Aproximación trigonométrica (series de Fourier)
La aproximación trigonométrica considera el siguiente producto escalar:
$$\langle f, g\rangle = \frac{1}{2\pi}\int_{-\pi}^{\pi}f(x)g(x)dx$$

Si en el método de Gram consideramos los polinomios trigonométricos $\{1, \cos(x), \sin(x), \cdots, \cos(nx), \sin(nx)\}$ tenemos que $\langle f, g\rangle$ para la base $\mathcal{U}$ es:
$$\int_{-\pi}^{\pi} \cos(kx)\sin(jx)dx = 0$$
$$\int_{-\pi}^{\pi} \cos(kx)\cos(jx)dx = \int_{-\pi}^{\pi} \cos(kx)\cos(jx)dx =
\begin{cases}0 & k\neq j \\ \pi & k=j>1 \\ 2\pi & k = j = 0\end{cases}$$

Así, la matriz a resolver es:
$$\begin{bmatrix}
2\pi & 0 & 0 & \cdots & 0 & 0 \\
0  & \pi & 0 & \cdots & 0 & 0 \\
0 & 0 & \pi  & \cdots & 0 & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\
0  & 0 & 0 & \cdots & \pi & 0 \\
0 & 0 & 0  & \cdots & 0 & \pi \\
\end{bmatrix}
\begin{bmatrix}
 a_0  \\
 a_1  \\
 b_1 \\
\vdots \\
 a_n \\
 b_n  \\
\end{bmatrix}
=
\begin{bmatrix}
\langle f, 1 \rangle \\
\langle f, \cos(x) \rangle \\
\langle f, \sin(x) \rangle \\
\vdots \\
\langle f, \cos(nx) \rangle \\
\langle f, \sin(nx) \rangle \\
\end{bmatrix}
$$

Luego
$$
\begin{bmatrix}
 a_0  \\
 a_1  \\
 b_1 \\
\vdots \\
 a_n \\
 b_n  \\
\end{bmatrix}
=
\begin{bmatrix}
\frac{1}{2\pi} & 0 & 0 & \cdots & 0 & 0 \\
0  & \frac{1}{\pi} & 0 & \cdots & 0 & 0 \\
0 & 0 & \frac{1}{\pi}  & \cdots & 0 & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\
0  & 0 & 0 & \cdots & \frac{1}{\pi} & 0 \\
0 & 0 & 0  & \cdots & 0 & \frac{1}{\pi} \\
\end{bmatrix}
\begin{bmatrix}
\langle f, 1 \rangle \\
\langle f, \cos(x) \rangle \\
\langle f, \sin(x) \rangle \\
\vdots \\
\langle f, \cos(nx) \rangle \\
\langle f, \sin(nx) \rangle \\
\end{bmatrix} =
\begin{bmatrix}
\frac{1}{2\pi}\langle f, 1 \rangle \\
\frac{1}{\pi}\langle f, \cos(x) \rangle \\
\frac{1}{\pi}\langle f, \sin(x) \rangle \\
\vdots \\
\frac{1}{\pi}\langle f, \cos(nx) \rangle \\
\frac{1}{\pi}\langle f, \sin(nx) \rangle \\
\end{bmatrix}
$$

Así, los coeficientes quedan definidos como:
$$a_k = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \cos(kx) dx$$
$$b_k = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \sin(kx) dx$$

Si combinamos los coeficientes con los elementos de $\mathcal{U}$ tenemos que el polinomio óptimo es, hasta grado $n$:
$$g_n(x) = \frac{a_0}{2} + \sum_{k=1}^n a_k\cos(kx) + b_k\sin(kx)$$

```python
help(producto_escalar_trigono)
```

#### EJERCICIO 36

```python
base = [S(1), cos(x), sin(x)]

m_gram = zeros(3, 3)
f_base = zeros(3, 1)

for i in range(3):
    for j in range(3):
        m_gram[i, j] = producto_escalar_trigono(base[i], base[j], x)

    f_base[i, 0] = producto_escalar_trigono(x, base[i], x)
```

```python
m_gram  # Se ve que la base es ortogonal
```

```python
f_base
```

```python
# Resolvemos la matriz de Gram
g = ((m_gram ** -1 * f_base).T * mat([base]).T)[0]
g
```

```python
help(coefs_fourier)
help(coefs_fourier_discr)
help(serie_fourier)
```

#### EJEMPLO 25 ($f(x) = e^x$)
En este ejercicio vamos a calcular la expansión de Fourier para órdenes 3, 6 y 15. Comprobamos que en orden 6 la aproximación es mejor, pero en los extremos del intervalo la aproximación empeora, tal y como se comenta en el texto base.

```python
f = E ** x
```

```python
coefs_fourier(f, var=x, I=[-pi, pi], n_coefs=6)
```

```python
serie_f_3 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=3))
serie_f_6 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=6))
serie_f_15 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=15))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f3 = [serie_f_3.subs(x, i) for i in x_range]
f6 = [serie_f_6.subs(x, i) for i in x_range]
f15 = [serie_f_15.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="E**x")
plt.plot(x_range, f3, label='f3')
plt.plot(x_range, f6, label="f6")
plt.plot(x_range, f15, label="f15")

plt.legend()
```

Las series de Fourier también admiten un caso discreto. En este caso, si dividimos el intervalo $[a,b]$ en $2m$ (asumimos $m$ hasta $[a, \frac{a+b}{2}]$ y otros $m$ hasta $[\frac{a+b}{2}, b]$), para el caso discreto los coeficientes $a_k$ y $b_k$ se redefinen como:
$$a_k = \frac{1}{m}\sum_{i=0}^{2m-1}f(x_i)\cos(kx_i)$$
$$b_k = \frac{1}{m}\sum_{i=0}^{2m-1}f(x_i)\sin(kx_i)$$

que, por simplificar, sería una media de $m$ valores en el intervalo, en lugar de la integral. Cuanto más grande sea $m$, más se van a parecer los coeficientes del caso discreto al continuo.

Ahora hacemos el caso discreto para m=6

```python
serie_f_6_m_3 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=3))
```

```python
serie_f_6_m_10 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=10))
```

```python
serie_f_6_m_20 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=20))
```

```python
serie_f_6_m_50 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=50))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f6m3 = [serie_f_6_m_3.subs(x, i) for i in x_range]
f6m10 = [serie_f_6_m_10.subs(x, i) for i in x_range]
f6m20 = [serie_f_6_m_20.subs(x, i) for i in x_range]
f6m50 = [serie_f_6_m_50.subs(x, i) for i in x_range]

f6 = [serie_f_6.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="E**x")
plt.plot(x_range, f6, label='f6')
plt.plot(x_range, f6m3, label="f6disc-m3")
plt.plot(x_range, f6m10, label="f6disc-m10")
plt.plot(x_range, f6m20, label="f6disc-m20")
plt.plot(x_range, f6m50, label="f6disc-m50")

plt.legend()
```

#### EJEMPLO 25 ($f(x) = x$)

```python
f = x
```

```python
coefs_fourier(f, x, [-pi, pi], n_coefs=6)
```

```python
serie_f_3 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=3))
serie_f_6 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=6))
serie_f_15 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=15))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f3 = [serie_f_3.subs(x, i) for i in x_range]
f6 = [serie_f_6.subs(x, i) for i in x_range]
f15 = [serie_f_15.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="x")
plt.plot(x_range, f3, label='f3')
plt.plot(x_range, f6, label="f6")
plt.plot(x_range, f15, label="f15")

plt.legend()
```

Ahora hacemos el caso discreto para m=6

```python
serie_f_6_m_5 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=5))
```

```python
serie_f_6_m_10 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=10))
```

```python
serie_f_6_m_20 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=20))
```

```python
serie_f_6_m_50 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=50))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f6m5 = [serie_f_6_m_5.subs(x, i) for i in x_range]
f6m10 = [serie_f_6_m_10.subs(x, i) for i in x_range]
f6m20 = [serie_f_6_m_20.subs(x, i) for i in x_range]
f6m50 = [serie_f_6_m_50.subs(x, i) for i in x_range]

f6 = [serie_f_6.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="x")
plt.plot(x_range, f6, label='f6')
plt.plot(x_range, f6m5, label="f6disc-m3")
plt.plot(x_range, f6m10, label="f6disc-m10")
plt.plot(x_range, f6m20, label="f6disc-m20")
plt.plot(x_range, f6m50, label="f6disc-m50")

plt.legend()
```

#### EJEMPLO 25 ($f(x) = x^2$)

```python
f = x ** 2
```

```python
coefs_fourier(f, x, [-pi, pi], n_coefs=6)
```

```python
N(2*pi**2/3)
```

```python
coefs_fourier_discr(f, x, [-np.pi, np.pi], n_coefs=6, m=100)
```

```python
serie_f_3 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=3))
serie_f_6 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=6))
serie_f_15 = expand(serie_fourier(f, x, [-pi, pi], n_coefs=15))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f3 = [serie_f_3.subs(x, i) for i in x_range]
f6 = [serie_f_6.subs(x, i) for i in x_range]
f15 = [serie_f_15.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="x^2")
plt.plot(x_range, f3, label='f3')
plt.plot(x_range, f6, label="f6")
plt.plot(x_range, f15, label="f15")

plt.legend()
```

Ahora hacemos el caso discreto para m=6

```python
serie_f_6_m_3 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=3))
```

```python
serie_f_6_m_10 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=10))
```

```python
serie_f_6_m_20 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=20))
```

```python
serie_f_6_m_50 = expand(serie_fourier(f, x, [-np.pi, np.pi], n_coefs=6, discreto=True, m=50))
```

```python
x_range = np.linspace(-np.pi, np.pi, 300)
real = [(f).subs(x, i) for i in x_range]
f6m3 = [serie_f_6_m_3.subs(x, i) for i in x_range]
f6m10 = [serie_f_6_m_10.subs(x, i) for i in x_range]
f6m20 = [serie_f_6_m_20.subs(x, i) for i in x_range]
f6m50 = [serie_f_6_m_50.subs(x, i) for i in x_range]

f6 = [serie_f_6.subs(x, i) for i in x_range]

plt.plot(x_range, real, label="x^2")
plt.plot(x_range, f6, label='f6')
plt.plot(x_range, f6m3, label="f6disc-m3")
plt.plot(x_range, f6m10, label="f6disc-m10")
plt.plot(x_range, f6m20, label="f6disc-m20")
plt.plot(x_range, f6m50, label="f6disc-m50")

plt.legend()
```

### Aproximación uniforme (Polinomios de Bernstein)
Otro modo de realizar aproximaciones a funciones es empleando la aproximación uniforme. El tma de Weierstrass indica que para $f$ continua en un intervalo cerrado y acotado $I$, para todo $\epsilon > 0$ existe un polinomio $p$ tal que $||f-p||_\infty < \epsilon$.

Para $I = [0, 1]$, el conjunto de polinomios de grado $n$ que satisface esa condición son los polinomios de Bernstein:
$$B_{n,f}(x) = \sum_{i = 0}^n f\left(\frac{i}{n}\right) {n \choose i} x^i(1-x)^{n-i}$$

Para $I = [a, b]$, hay que realizar el cambio de variable $x = (b-a)t + a$ para $t \in [0,1]$. Con ello, definimos $g(t)$ y tenemos el polinomio
$$B_{n,f}(t) = \sum_{i = 0}^n g\left(\frac{i}{n}\right) {n \choose i} t^i(1-t)^{n-i}$$
Que después reconvertimos a $x$ aplicando el cambio $t = \frac{x - a}{b - a}$

```python
help(polinomios_bernstein)
```

#### EJERCICIO 38

```python
f = abs(x)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=1)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=2)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=3)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=4)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=5)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=6)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=7)
```

```python
polinomios_bernstein(f, x, I=[-1, 1], grado=8)
```

Vemos que los polinomios se repiten por paridad. Esto tiene sentido porque los polinomios solo pueden tener coeficientes pares, ya que abs(x) siempre es positivo.
Vamos a plotear los polinomios hasta un grado alto.

```python
x_range = np.linspace(-1, 1, 300)
real = [(f).subs(x, i) for i in x_range]
B2 = [polinomios_bernstein(f, x, I=[-1, 1], grado=2).subs(x, i) for i in x_range]
B6 = [polinomios_bernstein(f, x, I=[-1, 1], grado=6).subs(x, i) for i in x_range]
B10 = [polinomios_bernstein(f, x, I=[-1, 1], grado=10).subs(x, i) for i in x_range]
B100 = [polinomios_bernstein(f, x, I=[-1, 1], grado=100).subs(x, i) for i in x_range]

plt.plot(x_range, real, label="abs(x)")
plt.plot(x_range, B2, label='B2')
plt.plot(x_range, B6, label='B6')
plt.plot(x_range, B10, label='B10')
plt.plot(x_range, B100, label='B100')

plt.legend()
```

Vemos que la curva se acerca cada vez más a abs(x).

#### EJEMPLO 26

```python
f = E**(3*x) * sin(pi * x)
f
```

```python
polinomios_bernstein(f, x, I=[0, 1], grado=1)
```

```python
polinomios_bernstein(f, x, I=[0, 1], grado=2)
```

```python
polinomios_bernstein(f, x, I=[0, 1], grado=3)
```

```python
polinomios_bernstein(f, x, I=[0, 1], grado=4)
```

Vemos que los polinomios se repiten por paridad. Esto tiene sentido porque los polinomios solo pueden tener coeficientes pares, ya que abs(x) siempre es positivo.
Vamos a plotear los polinomios hasta un grado alto.

```python
x_range = np.linspace(0, 1, 300)
real = [(f).subs(x, i) for i in x_range]
B2 = [polinomios_bernstein(f, x, I=[0, 1], grado=2).subs(x, i) for i in x_range]
B4 = [polinomios_bernstein(f, x, I=[0, 1], grado=4).subs(x, i) for i in x_range]
B8 = [polinomios_bernstein(f, x, I=[0, 1], grado=8).subs(x, i) for i in x_range]
B16 = [polinomios_bernstein(f, x, I=[0, 1], grado=16).subs(x, i) for i in x_range]
B32 = [polinomios_bernstein(f, x, I=[0, 1], grado=32).subs(x, i) for i in x_range]

plt.plot(x_range, real, label="f")
plt.plot(x_range, B2, label='B2')
plt.plot(x_range, B4, label='B4')
plt.plot(x_range, B8, label='B8')
plt.plot(x_range, B16, label='B16')
plt.plot(x_range, B32, label='B32')

plt.legend()
```

Ahora vamos a plotear el error entre la función real y la aproximación. Vemos que conforme el polinomio es más complejo, más se acerca a la función, pero también salen artefactos en la computación.

```python
x_range = np.linspace(0, 1, 300)
real = [(f).subs(x, i) for i in x_range]
B2 = [polinomios_bernstein(f, x, I=[0, 1], grado=2).subs(x, i) for i in x_range]
B4 = [polinomios_bernstein(f, x, I=[0, 1], grado=4).subs(x, i) for i in x_range]
B8 = [polinomios_bernstein(f, x, I=[0, 1], grado=8).subs(x, i) for i in x_range]
B16 = [polinomios_bernstein(f, x, I=[0, 1], grado=16).subs(x, i) for i in x_range]
B32 = [polinomios_bernstein(f, x, I=[0, 1], grado=32).subs(x, i) for i in x_range]

plt.plot(x_range, np.array(real) - np.array(B2), label="2")
plt.plot(x_range, np.array(real) - np.array(B4), label="4")
plt.plot(x_range, np.array(real) - np.array(B8), label="8")
plt.plot(x_range, np.array(real) - np.array(B32), label="32")

plt.legend()
```

## Ejercicios

### Ejercicio 40
Determinar la mejor aproximación de $x^3$ en $\mathcal{P}_2$ usando el producto escalar de Chebyshev.

Recordemos el producto escalar de Chebyshev:
$$\langle f, g\rangle = \int_{-1}^{1} f(x)g(x) \frac{1}{\sqrt{1-x^2}} dx$$

```python
help(metodo_gram)
```

```python
U = [sqrt(1/pi) * chebyshevt_poly(0, x), sqrt(2/pi) * chebyshevt_poly(1, x), sqrt(2/pi) * chebyshevt_poly(2, x)]  # S(1) lo hacemos porque hacer 1.diff da error por ser int.
f = x ** 3
dict_prod_deriv = metodo_gram(f, U, var=x, func_producto=producto_escalar_peso, a=-1, b=1, w=1/sqrt(1 - x**2))
```

```python
dict_prod_deriv['G']
```

```python
dict_prod_deriv['alpha']
```

```python
dict_prod_deriv['f_bar']
```

```python
dict_prod_deriv['poly']
```

### Ejercicio 41

Aplicar el procedimiento de orto-normalización de Gram-Schmidt a la base $\{1, x, x^2\}$ respecto el producto escalar.

$$\langle f, g \rangle = \frac{1}{2}\int_0^1(1+3x^2)f(x)g(x)dx$$

Aplicamos el procedimiento. En este caso los pasos a seguir serían:
$$p_0(x) = 1$$
$$p_1(x) = x - \frac{\langle p_0, x\rangle}{\langle p_0, p_0\rangle}p_0$$
$$p_2(x) = x^2 - \frac{\langle p_0, x^2\rangle}{\langle p_0, p_0\rangle}p_0 -
           \frac{\langle p_1, x^2\rangle}{\langle p_1, p_1\rangle}p_1$$

Después habría que, para ortonormalizar, aplicar:
$$p_i := \frac{p_i}{\langle p_i, p_i\rangle}$$

```python
p0 = S(1)
p0
```

```python
p1 = x - producto_escalar_peso(p0, x, w=(1 + 3*x**2)/2, a=0, b=1) / producto_escalar_peso(p0, p0, w=(1 + 3*x**2)/2, a=0, b=1) * p0
p1
```

```python
p1_norm = p1 / sqrt(producto_escalar_peso(p1, p1, w=(1 + 3*x**2)/2, a=0, b=1))
p1_norm
```

```python
p2 = x**2 - producto_escalar_peso(p0, x**2, w=(1 + 3*x**2)/2, a=0, b=1) / producto_escalar_peso(p0, p0, w=(1 + 3*x**2)/2, a=0, b=1) * p0 - \
            producto_escalar_peso(p1, x**2, w=(1 + 3*x**2)/2, a=0, b=1) / producto_escalar_peso(p1, p1, w=(1 + 3*x**2)/2, a=0, b=1) * p1
p2
```

```python
p2_norm = p2 / sqrt(producto_escalar_peso(p2, p2, w=(1 + 3*x**2)/2, a=0, b=1))
p2_norm
```

### Ejercicio 42
Hallar la recta que mejor aproxima la gráfica de la función $y = \frac{x}{1+x^2}$ con la norma inducida por el producto escalar
$$\langle f, g \rangle = \int_0^5f(x)g(x)dx$$

Usando una base de polinomios ortonormales.

El primer punto es transformar la base $\{1, x\}$, que conforma las posibles rectas, en una base ortonormal para ese producto escalar. Para ello empleamos el procedimiento de Gram Schmidt.

```python
f = x/(1 + x**2)
```

```python
U_ortogonal = gram_schmidt_f(base=[S(1), x], var=x, prod_esc=producto_asecas, a=0, b=5)
U_ortogonal
```

```python
p0 = U_ortogonal[0] / sqrt(producto_asecas(U_ortogonal[0], U_ortogonal[0], a=0, b=5))
p0
```

```python
producto_asecas(p0, p0, a=0, b=5)
```

```python
p1 = U_ortogonal[1] / sqrt(producto_asecas(U_ortogonal[1], U_ortogonal[1], a=0, b=5))
p1
```

```python
producto_asecas(p1, p1, a=0, b=5)
```

Ahora que tenemos la base de polinomios ortonormales vamos a desarrollar por el método de Gram, por comprobar la ortonormalidad.

```python
dict_gram = metodo_gram(f=f, U = [p0, p1], var=x, func_producto=producto_asecas, a=0, b=5)
dict_gram['G']
```

```python
dict_gram['alpha']
```

```python
dict_gram['f_bar']
```

```python
dict_gram['poly']
```

```python
x_vals = np.linspace(0, 5, 100)
y_vals = [dict_gram['poly'].subs(x, i) for i in x_vals]

plt.plot(x_vals, [f.subs(x, i) for i in x_vals])
plt.plot(x_vals, y_vals)
```

### Ejercicio 43
Hallar la recta que mejor aproxima la función $y = \frac{x}{1+x^2}$ con el producto discreto $\langle f, g \rangle = f(0)g(0)+f(2)g(2) + f(3)g(3) + f(5)g(5)$ usando una base de polinomios ortogonales.

Los polinomios ortonormales de la seccion anterior no nos son válidos porque en este caso el producto ha cambiado. Por ello tenemos que generar una nueva base.

```python
U_ortogonal_disc = gram_schmidt_f(base=[S(1), x], var=x, prod_esc=producto_asecas, a=0, b=5, I=[0, 2, 3, 5])
U_ortogonal_disc
```

```python
p0_disc = U_ortogonal_disc[0] / sqrt(producto_asecas(U_ortogonal_disc[0], U_ortogonal_disc[0], a=0, b=5, I=[0, 2, 3, 5]))
p0_disc
```

```python
producto_asecas(p0_disc, p0_disc, a=0, b=5, I=[0, 2, 3, 5])
```

```python
p1_disc = U_ortogonal_disc[1] / sqrt(producto_asecas(U_ortogonal_disc[1], U_ortogonal_disc[1], a=0, b=5, I=[0, 2, 3, 5]))
p1_disc
```

```python
producto_asecas(p1_disc, p1_disc, a=0, b=5, I=[0, 2, 3, 5])
```

```python
dict_gram_disc = metodo_gram(f=f, U = [p0_disc, p1_disc], var=x, func_producto=producto_asecas, a=0, b=5, I=[0, 2, 3, 5])
dict_gram_disc['G']
```

```python
dict_gram_disc['alpha']
```

```python
dict_gram_disc['f_bar']
```

```python
dict_gram_disc['poly']
```

```python
x_vals = np.linspace(0, 5, 100)
y_vals = [dict_gram['poly'].subs(x, i) for i in x_vals]
y_vals_disc = [dict_gram_disc['poly'].subs(x, i) for i in x_vals]

plt.plot(x_vals, [f.subs(x, i) for i in x_vals])
plt.plot(x_vals, y_vals, label='cont')
plt.plot(x_vals, y_vals_disc, label='disc')
plt.legend()
```

```python
x_vals = np.linspace(0, 5, 100)
y_vals = [(f - dict_gram['poly']).subs(x, i) for i in x_vals]
y_vals_disc = [(f - dict_gram_disc['poly']).subs(x, i) for i in x_vals]

plt.plot(x_vals, y_vals, label='cont')
plt.plot(x_vals, y_vals_disc, label='disc')
plt.legend()
```

Vemos que en el caso continuo la mayor diferencia está en 0, y es cercana a 0.4, y en el caso discreto está en 1, y es cercana a 0.3. Vamos a hallar esas diferencias explicitamente.

Ahora vamos a hallar las normas $\vert\vert f - g \vert\vert_\infty$ para la versión discreta y continua

```python
# Continua
max_x = [N(i) for i in solve((dict_gram['poly'] - (x/(1+x**2))).diff(x), x)]
max_x
```

```python
N(abs((dict_gram['poly'] - (x/(1+x**2))).subs(x, max_x[1])))
```

```python
N(abs((dict_gram['poly'] - (x/(1+x**2))).subs(x, max_x[3])))
```

```python
N((dict_gram['poly'] - (x/(1+x**2))).subs(x, 0))
```

El valor más cercano dentro de $[0, 5]$ es 1.09, pero es menor que el valor en 0. Así que el valor máximo es en 0.

```python
# Discreta
max_x = [N(i) for i in solve((dict_gram_disc['poly'] - (x/(1+x**2))).diff(x), x)]
max_x
```

```python
N(abs((dict_gram_disc['poly'] - (x/(1+x**2))).subs(x, max_x[1])))
```

En el caso discreto el valor máximo está en $x\sim0.94$ y es $0.32$.

Así, la mayor diferencia está para el caso continuo (en base a la norma uniforme).

Sin embargo, si tomamos la diferencia de áreas, el área de diferencia es mayor para la discreta, que *falla* más a lo largo de la función.

```python
Integral(abs(dict_gram['poly'] - (x/(1+x**2))), (x, 0, 5)).evalf()
```

```python
Integral(abs(dict_gram_disc['poly'] - (x/(1+x**2))), (x, 0, 5)).evalf()
```

### Ejercicio 44
Construir los cuatro primeros polinomios de grado creciente, de coeficiente principal igual a 1, que sean ortogonales respecto al producto escalar

$$\langle f, g\rangle = \int_{-1}^{1} \vert x\vert f(x)g(x) dx$$

Para esto recurrimos al Tma 14, que indica que si $\omega$ es una función positiva en el intervalo (en este caso lo es), se cumple que se pueden generar los polinomios por recurrencia.

```python
U = [S(1), x, x**2, x**3]
```

```python
help(polinomios_orto_peso)
```

```python
base_legendre = polinomios_orto_peso(U, w=abs(x), var=x, a=-1, b=1)
```

```python
base_legendre
```

```python
U_ortogonal_disc = gram_schmidt_f(base=U, var=x, prod_esc=producto_escalar_peso, w=abs(x), a=-1, b=1)
U_ortogonal_disc
```

```python
Vemos que la base de polinomios es la misma.
```

```python
producto_escalar_peso(f=U_ortogonal_disc[1], g=U_ortogonal_disc[2], w=abs(x), a=-1, b=1)
```

```python
producto_escalar_peso(f=U_ortogonal_disc[3], g=U_ortogonal_disc[2], w=abs(x), a=-1, b=1)
```

### Ejercicio 45
Determinar la mejor aproximación a la función $f(x) = e^x$ en $\mathcal{P}_2$ usando la norma asociada al siguiente producto escalar

$$\langle f, g \rangle = \int_0^1 (f(x)g(x) + f'(x)g'(x)) dx$$

Vamos a usar el método de Gram directamente

```python
U = [S(1), x, x**2]
f = E ** x
```

```python
dict_prod_continuo = metodo_gram(f, U, var=x, func_producto=producto_deriv,a=0, b=1)
```

```python
expand(dict_prod_continuo['poly'])
```

```python
dict_prod_continuo['G']
```

```python
dict_prod_continuo['f_bar']
```

```python
dict_prod_continuo['alpha']
```

Y ahora vamos a buscar una base ortonormal y repetir el método

```python
base_ortogonal = gram_schmidt_f(U, var=x, prod_esc=producto_deriv,a=0, b=1)
base_ortogonal
```

```python
base_ortonormal = [i / sqrt(producto_deriv(i, i, a=0, b = 1)) for i in base_ortogonal]
base_ortonormal
```

```python
dict_prod_continuo_ortonormal = metodo_gram(f, base_ortonormal, var=x, func_producto=producto_deriv,a=0, b=1)
```

```python
dict_prod_continuo_ortonormal['poly']
```

```python
dict_prod_continuo_ortonormal['G']
```

```python
dict_prod_continuo_ortonormal['f_bar']
```

```python
dict_prod_continuo_ortonormal['alpha']
```

```python
N(dict_prod_continuo['poly'])
```

```python
x_vals = np.linspace(0, 1, 100)
y_vals = [dict_prod_continuo['poly'].subs(x, i) for i in x_vals]

plt.plot(x_vals, [f.subs(x, i) for i in x_vals])
plt.plot(x_vals, y_vals)
```