```python
from sympy import *
from sympy.physics.mechanics import *

mass, R, c1, c2, grav = var("mass, R, c1, c2, grav", real=True, positive=True)
omega = var("omega", real=True, positive=True)

x, phi = dynamicsymbols('x, phi')
xp, phip = dynamicsymbols('x, phi', 1)

J = mass*R**2/2

T = J*phip**2/2 + mass*xp**2/2
U = - mass*grav*x + c1/2 * (x - R*phi)**2 + c2/2 * (x + R*phi)**2
L = T - U

pprint("\nL:")
pprint(L)
LM = LagrangesMethod(L, [x,phi])

# pprint(L.expand().collect([p1,p2,p3,p4]))

LM.form_lagranges_equations()
M = LM.mass_matrix
f = LM.forcing

pprint("\nMass matrix:")
pprint(M)
pprint("\nForce matrix (not stiffness matrix)")
pprint(f)
for i in range(2):
    print("--- Row "+str(i+1)+":")
    term = -1*f[i].expand()
    pprint(term.collect([x,phi]))

from sympy.abc import x, y
system = Matrix([
    (c1 + c2, R*(-c1 + c2), mass*grav), (R*(-c1 + c2), R**2*(c1 + c2), 0)
    ])
sol = solve_linear_system(system, x,y)
x0 = sol[x]
p0 = sol[y]

pprint("\nx0:")
pprint(x0)
pprint("\np0:")
pprint(p0)

```


# Problem Set I.3

```python
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import sympy
import networkx as nx
```

## \# 1

$Bx=0,AB=C \implies ABx=Cx, A(Bx)=Cx, A \cdot 0 = Cx, Cx = 0$ or $ABx=0$

## \# 2

Example of $rank(A^2) < rank(A)$

$$A=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix},
A^2=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
=\begin{bmatrix}
 0 & 0 \\
 0 & 0
\end{bmatrix}$$

$rank(A^TA)=rank(A)$

$$A^TA=\begin{bmatrix}
 0 & 0 \\
 1 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
$$

$rank(AA^T)=rank(A)$

$$AA^T=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
 0 & 0 \\
 1 & 0
\end{bmatrix}
=\begin{bmatrix}
 1 & 0 \\
 0 & 0
\end{bmatrix}
$$

## \# 3

$$
C=\begin{bmatrix}
 A \\
 B
\end{bmatrix}
$$

$$Cx=0
\implies
\begin{bmatrix}
 A \\
 B
\end{bmatrix}x=0,
\begin{bmatrix}
 Ax \\
 Bx
\end{bmatrix}=0
\implies Ax=0,Bx=0
$$

$N(C)=N(A) \cap N(B)$

## \# 4

$C(A)=C(A^T)$ and $N(A)=N(A^T)$

$Ax=A^Tx=0$ because $Ax=0$ and $A^Ty=0$ and $A^T=A \implies x = y$

$$\begin{bmatrix}
 u & v
\end{bmatrix}=\begin{bmatrix}
 u & v
\end{bmatrix}^T,
\begin{bmatrix}
 u & v
\end{bmatrix}=\begin{bmatrix}
 u^* \\
 v^*
\end{bmatrix}
$$

I suppose than $A = A^T \implies S = S^T$ is symmetric matrix

## \# 5

1) $r = m = n$, $A_1x=b$ has 1 solution for every b

$A_1$ is any full-rank matrix

2) $r = m < n$, $A_2x=b$ has 1 or $\infty$ solutions

$$A_2x=
\begin{bmatrix}
 1 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
 x_1 \\
 x_2
\end{bmatrix}
=b
$$

$A_2$ has an extra column

3) $r=n < m, A_3x=b$ has 0 or 1 solutions

$A_3=A_2^T$

4) $r < m, r < n; A_4x = b$ has 0 or $\infty$ solutions

$$
A_4=
\begin{bmatrix}
 1 & 0 \\
 0 & 0
\end{bmatrix}
$$

or dependent columns

$$
\begin{bmatrix}
 1 & 2 \\
 1 & 2
\end{bmatrix}
$$

$$
\begin{bmatrix}
 1 & 2 & 3 \\
 1 & 2 & 3 \\
 1 & 2 & 3
\end{bmatrix}
$$

And etc.

## \# 6

$Ax=0, A^TAx=A^T(Ax)=A^T0=0$

$N(A) \subset N(A^TA)$

$A^TAx=0$, then $x^TA^TAx=x^T0 \implies (Ax)^T(Ax)=0 \implies ||Ax||^2=0$

$N(A^TA)=N(A)$

## \# 7

$$A=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix};
A^2=\begin{bmatrix}
 0 & 0 \\
 0 & 0
\end{bmatrix}
$$

$A$ and $A^2$ have different $N(A)$

## \# 8

$$A=\begin{bmatrix}
 0 & 1 \\
 0 & 0
\end{bmatrix}
$$

$C(A)=N(A)=\begin{bmatrix}
 x_1 \\
 0
\end{bmatrix};$

But $C(A) \neq N(A^T)$ because those two subspaces are orthogonal

## \# 9

```python
g = nx.DiGraph()
g.add_nodes_from([1,2,3,4,5])
g.add_edge(1,2)
g.add_edge(1,3)
g.add_edge(1,4)
g.add_edge(2,5)
g.add_edge(2,3)
g.add_edge(3,4)
g.add_edge(3,5)
g.add_edge(4,5)

pos = { 1: (0, 20), 2: (20, 20), 3: (10, 10), 4: (0, 0), 5: (20, 0)}

nx.draw_networkx(g, pos=pos, with_labels=True)

plt.show()
```

$$A=
\begin{bmatrix}
 -1 &  1 &  0 &  0 & 0 \\
 -1 &  0 &  1 &  0 & 0 \\
 -1 &  0 &  0 &  1 & 0 \\
  0 & -1 &  1 &  0 & 0 \\
  0 & -1 &  0 &  0 & 1 \\
  0 &  0 & -1 &  1 & 0 \\
  0 &  0 & -1 &  0 & 1 \\
  0 &  0 &  0 & -1 & 1
\end{bmatrix}
$$

```python
A = sympy.Matrix([
    [-1,  1,  0,  0, 0],
    [-1,  0,  1,  0, 0],
    [-1,  0,  0,  1, 0],
    [ 0, -1,  1,  0, 0],
    [ 0, -1,  0,  0, 1],
    [ 0,  0, -1,  1, 0],
    [ 0,  0, -1,  0, 1],
    [ 0,  0,  0, -1, 1],
])
A
```

$\displaystyle \left[\begin{matrix}-1 & 1 & 0 & 0 & 0\\-1 & 0 & 1 & 0 & 0\\-1 & 0 & 0 & 1 & 0\\0 & -1 & 1 & 0 & 0\\0 & -1 & 0 & 0 & 1\\0 & 0 & -1 & 1 & 0\\0 & 0 & -1 & 0 & 1\\0 & 0 & 0 & -1 & 1\end{matrix}\right]$

```python
A.nullspace()
```

    [Matrix([
     [1],
     [1],
     [1],
     [1],
     [1]])]

```python
x_1, x_2, x_3, x_4, x_5 = sympy.symbols("x_1, x_2, x_3, x_4, x_5")

eq1 = -x_1 + x_2
eq2 = -x_1 + x_3
eq3 = -x_1 + x_4
eq4 = -x_2 + x_3
eq5 = -x_2 + x_5
eq6 = -x_3 + x_4
eq7 = -x_3 + x_5
eq8 = -x_4 + x_5

sympy.solve([eq1, eq2, eq3, eq4, eq5, eq6, eq7, eq8], [x_1, x_2, x_3, x_4, x_5], dict=True)
```

    [{x_1: x_5, x_2: x_5, x_3: x_5, x_4: x_5}]

$x_1=x_2=x_3=x_4=x_5$

```python
c = sympy.Symbol("c")

x = sympy.Matrix([c, c, c, c, c])

A*x
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\\0\\0\\0\end{matrix}\right]$

$rank(A)=n-1=4, N(A^T)=m-r=8-4=4$ also means 4 loops in the graph

```python
A.T.nullspace()
```

    [Matrix([
     [ 1],
     [-1],
     [ 0],
     [ 1],
     [ 0],
     [ 0],
     [ 0],
     [ 0]]),
     Matrix([
     [ 0],
     [ 1],
     [-1],
     [ 0],
     [ 0],
     [ 1],
     [ 0],
     [ 0]]),
     Matrix([
     [-1],
     [ 1],
     [ 0],
     [ 0],
     [-1],
     [ 0],
     [ 1],
     [ 0]]),
     Matrix([
     [-1],
     [ 0],
     [ 1],
     [ 0],
     [-1],
     [ 0],
     [ 0],
     [ 1]])]

Loop 1:

```python
y_1 = sympy.Matrix([1, 0, -1, 0, 1, 0, 0, -1])

y_1
```

$\displaystyle \left[\begin{matrix}1\\0\\-1\\0\\1\\0\\0\\-1\end{matrix}\right]$

```python
A.T*y_1
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\end{matrix}\right]$

Loop 2:

```python
y_2 = sympy.Matrix([1, -1, 0, 0, 1, 0, -1, 0])

y_2
```

$\displaystyle \left[\begin{matrix}1\\-1\\0\\0\\1\\0\\-1\\0\end{matrix}\right]$

```python
A.T*y_2
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\end{matrix}\right]$

Loop 3:

```python
y_3 = sympy.Matrix([1, 0, -1, 1, 0, 1, 0, 0])

y_3
```

$\displaystyle \left[\begin{matrix}1\\0\\-1\\1\\0\\1\\0\\0\end{matrix}\right]$

```python
A.T*y_3
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\end{matrix}\right]$

Loop 4:

```python
y_4 = sympy.Matrix([1, -1, 0, 1, 0, 0, 0, 0])

y_4
```

$\displaystyle \left[\begin{matrix}1\\-1\\0\\1\\0\\0\\0\\0\end{matrix}\right]$

```python
A.T*y_4
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\end{matrix}\right]$

## \# 10

If $N(A)=x=\begin{bmatrix}
 0      \\
 0      \\
 \vdots \\
 0
\end{bmatrix}$, then what are the nullspace of $B=\begin{bmatrix}
 A & A & A
\end{bmatrix}$

$Ax=0;
B\cdot
\begin{bmatrix}
 x \\
 x \\
 x
\end{bmatrix}
=\begin{bmatrix}
 A & A & A
\end{bmatrix}
\cdot
\begin{bmatrix}
 x \\
 x \\
 x
\end{bmatrix}=0$

$N(B)=\begin{bmatrix}
 x \\
 x \\
 x
\end{bmatrix}$

## \# 11

$dim(S)=2$ and $dim(T)=7$ of $R^{10}$

1) $dim(S \bigcap T) \leq dim(S) \leq 2$

2) $dim(S+T) \leq dim(T) + dim(S)$

Or

$7 \leq dim(S+T) \leq 9$

3) $S^{\perp}=dim(R^{10})-dim(S^{\perp})=10-8=2$

1. Highlights of Linear Algebra/3. The Four Fundamental Subspaces/Problem Set I.3.ipynb


# *simbODE* - Simbolic Differential Equations

*simbODE* is the function implemented to determine the differential equation system related to a biological system. These equations are used to compute the system output. The funcion determine the simbolic form of the equations, so their variables have to be replace for their actual values at the time when the differential equation system is used to compute the system output.

The implementation of this function was done mainly using sympy, which it is the main Python library for symbolic mathematic.

```
#packages required
import numpy as np
import sympy as sp
```

**Define biological system**
To define a biological system is necessary to set the names of the molecular species that are considered. Also, it should be defined a variable meant to represent the system input "u". The name of each species together with the names of the kinetic variables are converted to a sympy class. This class allows to manipulate each variable in a simbolic way. Also, the way how the reactions take place and the species interact with each other is represented by the reagent and product matrices. The first one represents the count for every species while it is performing as a reagent. In the other hand, the product matrix represents the count for every species while it is performing as a product. In both matrices, each reaction is reprenseted as a column, while each species is a row from the matrices.

```
#molecular species
species = ['x1', 'x2']
species = sp.var(species)

#system input
inp = ['u']
uh = sp.var(inp)
ruh = 0  #reaction that it is affected by the input 0 = first reaction

#reagent and product matrices
#"1" represents that a specific species is changing in a certain reactin
                      #r1,r2,r3,r4
reactants = np.array([[0, 1, 1, 0],
                      [0, 0, 0, 1]])
products = np.array([[1, 0, 1, 0],
                     [0, 0, 1, 0]])

#kinetic parameters
pars = ['c1', 'c2', 'c3', 'c4']
parsValues = sp.var(pars)
#parsValues = [4.0, 0.010, 1.0, 0.006] #to use numeric parameters uncomment this
#line and comment the two previous ones
```

To identify the differential equations system from the previous defined data, first it has to be calculated a general soichiometric matrix. This matrix is equal to the difference between the product and the reagent matrices. The equations of the system are found using a propensity function. This function has the expressions corresponding to each interaction between the species in a certain reaction. Each value of the propensity function is multiplied by its respective value in the stoichiometric matrix.

```
#stoichiometric matrix
V = products - reactants

#pre-propensity function
aPro = parsValues
aPro[ruh] = aPro[ruh]*uh[0]  # adds input to the corresponding reaction

#reaction network dimentions
Sn, Rm = V.shape

#propensity function vector
for r in range(0,Rm):
    for s in range(0, Sn):
        #determine species interactions simbolically
        for a in range(0, reactants[s,r]):
            aPro[r] *= species[s]
        #end for a
    #end for s
#end for r

#differential equations system
odeX = []
for s in range(0,Sn):
    temp = 0
    #define each equation of the system
    for r in range(0, Rm):
        temp += V[s,r]*aPro[r]
    #end for r

    #set of equations
    odeX.append(temp)
#end for s
```

**Differential Equations System** Each differential equation represents a different species and how it interacts with the other species. Usually, the last equation serves as the equation to compute the gene expression in a certain biological system.

```
#Shows the differential equations system
for s in range(0, Sn):
    print(f'd{species[s]}/dt:', odeX[s])
```

    dx1/dt: c1*u - c2*x1
    dx2/dt: c3*x1 - c4*x2


Ответ на вопрос https://ru.stackoverflow.com/questions/1293488/

Пусть дана точка $P(x_p,y_p,z_p)$ и эллипсоид, у которого известны полуоси и центр, его уравнение: $x^2/a^2+y^2/a^2+z^2/b^2=1$.

Точка $P$ лежит за пределами эллипсоида в некотором отдалении. Множество касательных к эллипсоиду, проведенных из $P$ образует коническую поверхность. Надо найти общий вид точек касания этого конуса и эллипсоида (понятно, что эти точки будут принадлежать эллипсу, то есть задача по сути сводится к отысканию уравнения эллипса касания конуса и эллипсоида). Можно ли здесь упростить задачу, перейдя к сферическим координатам? Если да, то как действовать дальше? Если нет, то вопрос аналогичный.

```python
import sympy as sp
import sympy.abc as abc
import sympy.vector as spv
```

$a$ и $b$ - параметры эллипсоида $\frac{x^2}{a^2} + \frac{y^2}{a^2} + \frac{z^2}{b^2}$

```python
a,b = abc.a, abc.b

```

Три системы координат:
- $A$ - исходная система координат, в которой задан эллипсоид
- $B$ - система координат, в которой эллипсоид превращается в единичную сферу
- $C$ - система координат, в которой эллипсоид превращен в сферу, а точка $P$ находится на оси $Z$

Матрицы:
- $A2B$ переводит координаты из $A$ в $B$, а матрица $B2A$ переводит обратно
- $B2C$ переводит координаты из $B$ в $C$, а матрица $C2B$ переводит обратно

```python
A2B = sp.Matrix([[1/a, 0,   0],
                 [0,   1/a, 0],
                 [0,   0,   1/b]])
```

```python
B2A = sp.Matrix([[a, 0, 0],
                 [0, a, 0],
                 [0, 0, b]])
```

Проверка того, что матрицы $A2B$ и $B2A$ обратны

```python
B2A*A2B
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

В системе координат $B$ вектор $OP$ составляет с осью $x$ угол $\phi$, а с осью $\z$ угол $\theta$

Длина вектора $OP$ в этой системе координат равно $R = \sqrt{\frac{x^2}{a^2} + \frac{y^2}{a^2} + \frac{z^2}{b^2}}$, где $x,y,z$ - координаты точки $P$ в исходной системе координат.

Углы $\theta$ и $\phi$ в системе $B$ определяются так:
- $\cos\theta = \frac{z}{bR}$
- $\cos\phi = \frac{x}{aRsin\theta}$
- $\sin\phi = \frac{y}{aRsin\theta}$

```python
phi, theta = abc.phi, abc.theta
```

Поворот системы координат от $B$ к $C$, в которой точка $P$ окажется на оси $Z$, выполняется в два приёма:
- сначала на угол $\phi$ вокруг оси $Z$. В результате точка $P$ окажется в плоскости $ZOX$
- затем на угол $\theta$ вокруг оси $Y$. В результате точка $P$ окажется на оси $Z$ в новой системе координат.

```python
B2C = spv.BodyOrienter(phi, theta, 0, 'ZYZ').rotation_matrix()
B2C
```

$\displaystyle \left[\begin{matrix}\cos{\left(\phi \right)} \cos{\left(\theta \right)} & \sin{\left(\phi \right)} \cos{\left(\theta \right)} & - \sin{\left(\theta \right)}\\- \sin{\left(\phi \right)} & \cos{\left(\phi \right)} & 0\\\sin{\left(\theta \right)} \cos{\left(\phi \right)} & \sin{\left(\phi \right)} \sin{\left(\theta \right)} & \cos{\left(\theta \right)}\end{matrix}\right]$

Матрица обратного преобразования строится в обратном порядке:
- сначала на угол $-\theta$ вокруг оси $Y$,
- затем на угол $-\phi$ вокруг оси $Z$.

```python
C2B = spv.BodyOrienter(-theta, -phi, 0, 'YZX').rotation_matrix()
C2B
```

$\displaystyle \left[\begin{matrix}\cos{\left(\phi \right)} \cos{\left(\theta \right)} & - \sin{\left(\phi \right)} & \sin{\left(\theta \right)} \cos{\left(\phi \right)}\\\sin{\left(\phi \right)} \cos{\left(\theta \right)} & \cos{\left(\phi \right)} & \sin{\left(\phi \right)} \sin{\left(\theta \right)}\\- \sin{\left(\theta \right)} & 0 & \cos{\left(\theta \right)}\end{matrix}\right]$

Проверка того, что матрицы $B2C$ и $C2B$ обратны друг другу

```python
sp.trigsimp(C2B*B2C) # упрощение тригонометрических выражений
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

```python
R = abc.R
```

Точка $P$ в системе координат $C$ находится на оси $Z$ на расстоянии $R$ от начала координат

```python
P_C = sp.Matrix([[0],[0],[R]])
```

Координаты точки $P$ в исходной системе координат

```python
P_A = B2A*C2B*P_C
```

```python
P_A
```

$\displaystyle \left[\begin{matrix}R a \sin{\left(\theta \right)} \cos{\left(\phi \right)}\\R a \sin{\left(\phi \right)} \sin{\left(\theta \right)}\\R b \cos{\left(\theta \right)}\end{matrix}\right]$

В системе координат $C$ эллипс представлен единичной сферой. Касательные из точки $P$ в этой системе координат образуют круг радиусом $r$ в плоскости, отстоящей на расстояние $h$ от начала координат.

Подробнее о касательной к кругу можно прочитать здесь: https://www.wikiwand.com/en/Tangent_lines_to_circles

```python
r = sp.sqrt(1 - 1/(R*R))
r
```

$\displaystyle \sqrt{1 - \frac{1}{R^{2}}}$

```python
h = sp.sqrt(1 - r*r)
```

```python
h
```

$\displaystyle \sqrt{\frac{1}{R^{2}}}$

```python

```

Построим уравнение касательного круга в параметрическом виде, как вращение точки вокруг оси $Z$ с круговой частотой $\omega$

```python
omega = abc.omega
t = abc.t
```

```python
base = sp.Matrix([[0],[0], [1/R]]) # центр круга
e1 = sp.Matrix([[r],[0],[0]]) # проекция точки на ось x
e2 = sp.Matrix([[0], [r],[0]]) # проекция точки на ось y
```

Радиус-вектор точек, в которых конус касается эллипсоида, в системе координат $C$

```python
tangent = base + e1*sp.cos(t) + e2*sp.sin(t)
tangent
```

$\displaystyle \left[\begin{matrix}\sqrt{1 - \frac{1}{R^{2}}} \cos{\left(t \right)}\\\sqrt{1 - \frac{1}{R^{2}}} \sin{\left(t \right)}\\\frac{1}{R}\end{matrix}\right]$

Проверка: касательная плоскость к единичной сфере в точке $(x_0, y_0, z_0)$ задаётся уравнением $2x_0(x-x_0) + 2y_0(y-y_0) + 2z_0(z-z_0) = 0$

Подставим в это уравнение в качестве $(x_0, y_0, z_0)$ координаты из $tangent$, а вместо $(x, y, z)$ координаты точки $P = (0, 0, R)$

Для справки уравнение касательной плоскости к поверхности, заданной уравнением $F(x,y,z)=0$ в точке $(x_0, y_0, z_0)$:

$F'_x(x_0)\cdot(x-x_0) + F'_y(y_0)\cdot(y-y_0) + F'_z(z_0)\cdot(z-z_0) = 0$

```python
_t = tangent
_u = _t[0]*(-_t[0]) + _t[1]*(-_t[1]) + _t[2]*(R-_t[2])
sp.trigsimp(_u)
```

$\displaystyle 0$

Радиус вектор точек , в которых конус касается эллипсоида, в системе координат $A$

```python
tangent_A = B2A*C2B*tangent
```

```python
tangent_A
```

$\displaystyle \left[\begin{matrix}- a \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\phi \right)} \sin{\left(t \right)} + a \sqrt{1 - \frac{1}{R^{2}}} \cos{\left(\phi \right)} \cos{\left(t \right)} \cos{\left(\theta \right)} + \frac{a \sin{\left(\theta \right)} \cos{\left(\phi \right)}}{R}\\a \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\phi \right)} \cos{\left(t \right)} \cos{\left(\theta \right)} + a \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(t \right)} \cos{\left(\phi \right)} + \frac{a \sin{\left(\phi \right)} \sin{\left(\theta \right)}}{R}\\- b \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\theta \right)} \cos{\left(t \right)} + \frac{b \cos{\left(\theta \right)}}{R}\end{matrix}\right]$

Проверим, что найденные точки лежат на эллипсоиде

```python
_t = tangent_A
_u = _t[0]**2/a**2 + _t[1]**2/a**2 + _t[2]**2/b**2
sp.trigsimp(_u)
```

$\displaystyle 1$

Проверка 2: касательная плоскость к эллипсоиду в точке $(x_0, y_0, z_0)$ задаётся уравнением $2x_0/a^2(x-x_0) + 2y_0/a^2(y-y_0) + 2z_0/b^2(z-z_0) = 0$

```python
_t = tangent_A
_p = P_A
_u = _t[0]/a**2*(_p[0]-_t[0]) + _t[1]/a**2*(_p[1]-_t[1]) + _t[2]/b**2*(_p[2]-_t[2])
sp.trigsimp(_u)
```

$\displaystyle 0$

Подставим формулы для $R$ и углов $\theta$ и $\phi$ в выражение для точек касания конуса к эллипсу

```python
x,y,z = sp.symbols("x y z")
```

```python
t_a = tangent_A.subs({
                sp.cos(phi) : x/a/sp.sin(theta)/R,
                sp.sin(phi) : y/a/sp.sin(theta)/R
               })                                        \
    .subs({sp.sin(theta) : sp.sqrt(1-sp.cos(theta)**2)}) \
    .subs({sp.cos(theta) : z/b/R})                       \
    .subs({R:sp.sqrt((x/a)**2 + (y/a)**2 + (z/b)**2)})
t_a
```

$\displaystyle \left[\begin{matrix}\frac{x}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}} - \frac{y \sqrt{1 - \frac{1}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} \sin{\left(t \right)}}{\sqrt{1 - \frac{z^{2}}{b^{2} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}} \sqrt{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} + \frac{x z \sqrt{1 - \frac{1}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} \cos{\left(t \right)}}{b \sqrt{1 - \frac{z^{2}}{b^{2} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}\\\frac{x \sqrt{1 - \frac{1}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} \sin{\left(t \right)}}{\sqrt{1 - \frac{z^{2}}{b^{2} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}} \sqrt{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} + \frac{y}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}} + \frac{y z \sqrt{1 - \frac{1}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} \cos{\left(t \right)}}{b \sqrt{1 - \frac{z^{2}}{b^{2} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}\\- b \sqrt{1 - \frac{z^{2}}{b^{2} \left(\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}\right)}} \sqrt{1 - \frac{1}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}} \cos{\left(t \right)} + \frac{z}{\frac{z^{2}}{b^{2}} + \frac{x^{2}}{a^{2}} + \frac{y^{2}}{a^{2}}}\end{matrix}\right]$

```python
center = B2A*C2B*base
v1 = B2A*C2B*e1
v2 = B2A*C2B*e2
```

```python
center
```

$\displaystyle \left[\begin{matrix}\frac{a \sin{\left(\theta \right)} \cos{\left(\phi \right)}}{R}\\\frac{a \sin{\left(\phi \right)} \sin{\left(\theta \right)}}{R}\\\frac{b \cos{\left(\theta \right)}}{R}\end{matrix}\right]$

```python
v1
```

$\displaystyle \left[\begin{matrix}a \sqrt{1 - \frac{1}{R^{2}}} \cos{\left(\phi \right)} \cos{\left(\theta \right)}\\a \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\phi \right)} \cos{\left(\theta \right)}\\- b \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\theta \right)}\end{matrix}\right]$

```python
v2
```

$\displaystyle \left[\begin{matrix}- a \sqrt{1 - \frac{1}{R^{2}}} \sin{\left(\phi \right)}\\a \sqrt{1 - \frac{1}{R^{2}}} \cos{\left(\phi \right)}\\0\end{matrix}\right]$

```python
import numpy as np
```

```python
def tangent_cone(a,b, P):
    """
    Возвращает три вектора `c,v1,v2`, определяющие эллипс, по точкам которого эллипсоид касается конус с вершиной `P`.

    Точки r_tan эллипса вычисляются так:
    ```
    c, v1, v2 = tangent_cone(a,b, P)
    t = np.linspace(0, 2*np.pi, 100)
    r_tan = c.reshape(3,1) + v1.reshape(3,1)*np.cos(t) + v2.reshape(3,1)*np.sin(t)
    ```
    В этом случае r_tan - матрица из 3 строк и 100 столбцов. Первая строка - координата `x` точек,
    вторая строка - координата `y`, третья строка - координата `z`.
    """
    assert P.ndim == 1
    assert len(P) == 3

    x,y,z = P
    R = np.sqrt(x**2/a**2 + y**2/a**2 + z**2/b**2)
    cos_theta = (z/b)/R
    # theta меняется от 0 до pi, поэтому sin(theta) >= 0
    sin_theta = np.sqrt(1 - cos_theta**2)

    sin_phi = (y/a)/(sin_theta*R)
    cos_phi = (x/a)/(sin_theta*R)

    center = np.array([a*cos_phi*sin_theta/R, a*sin_phi*sin_theta/R, b*cos_theta/R])

    r = np.sqrt(1 - 1/R**2)
    v1 = np.array([a*r*cos_phi*cos_theta, a*r*sin_phi*cos_theta, -b*r*sin_theta])
    v2 = np.array([-a*r*sin_phi, a*r*cos_phi, 0])

    return center, v1, v2
```

```python
# Эллипсоид вытянут вдоль Z
a = 1
b = 4
```

```python
# Вершина конуса на бисектрисе первого квадранта
P = np.array([5,5,5])
```

```python
c, v1, v2 = tangent_cone(a,b, P)

```

```python
t = np.linspace(0, 2*np.pi, 100)
r_tan = c.reshape(3,1)+v1.reshape(3,1)*np.cos(t) + v2.reshape(3,1)*np.sin(t)
```

```python
el_X, el_Y = np.meshgrid(np.linspace(-a, a, 200), np.linspace(-a, a, 200))
```

```python
el_Z = b*np.sqrt(1 - el_X**2/a**2 - el_Y**2/a**2, )

```

    <ipython-input-104-80c6e10389cd>:1: RuntimeWarning: invalid value encountered in sqrt
      el_Z = b*np.sqrt(1 - el_X**2/a**2 - el_Y**2/a**2, )

```python
import matplotlib.pyplot as plt
```

```python
fig = plt.figure(figsize=(10, 10))
ax = fig.add_subplot(projection='3d')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.view_init(30, 120)

ax.plot_wireframe(el_X, el_Y, el_Z, cstride=10, rstride = 10, alpha=0.3)
ax.plot_wireframe(el_X, el_Y, -el_Z, cstride=10, rstride = 10, alpha=0.3)
ax.plot(r_tan[0], r_tan[1], r_tan[2], color="black")
ax.scatter(*P)

```

```python

```


## Document-image binarization using modified Sauvola's algorithm

This binarization technique is an improvement over Sauvola's binarization technique. In this work, we improve the existing Sauvola's binarization technique by preserving more foreground information in the binarized document-images. In order to achieve this, we introduce a confidence score for the background pixels.

In Sauvola's algorithm, a threshold is computed for each pixel using the Eq $\ref{eq:sauvola}$, where, for an input image $I$,  $R=\frac{\textrm{max}(I) - \textrm{min}(I)}{2}$.
\begin{equation}
T_W (p)= m_{W}^{p} \times [1+k\times (\frac{s_{W}^{p}}{R} - 1)]
\label{eq:sauvola} \tag{1}
\end{equation}
The threshold $T$ is computed for each pixel ($p$) based on a window $W$ of size $n \times n$ surrounding it, where $m_{W}^{p}, s_{W}^{p}$ respectively represent mean and standard deviation of $W$ around pixel $p$, and  $k$ lies between $0 \le k \le 1$.

We define a confidence value, $C$, for each pixel $p$ as Eq \ref{eq:conf}, where max($I$) represent maximum value of any pixel of an input image $I$.
\begin{equation}
  C_W(p)  = \left\{
  \begin{array}{l l}
    \frac{I(p) - T_W(p)}{\textrm{max}(I) - T_W(p)} & \quad \textrm{if \textit{$I(p)$} $>$ \textit{$T_W(p)$}}\\
    0 & \quad  \textrm{otherwise}\\
  \end{array} \right.
\label{eq:conf} \tag{2}
\end{equation}

A new image $I_{c}$ corresponding to an input image $I$ is generated using the confidence value given by the  Eq $\ref{eq:conf}$. Later, Eq $\ref{eq:sauvola}$ is used to compute thresholds for the image $I_c$. These thresholds are further used to generate the new binary image.

For details refer to this [paper](https://drive.google.com/file/d/1D3CyI5vtodPJeZaD2UV5wdcaIMtkBbdZ/view?usp=sharing)

This notebook provides an example of how to utilize modified sauvola algorithm for image binarization task.

```python
import numpy as np
import cv2
from matplotlib import pyplot as plt
from skimage.filters import threshold_sauvola
import ModifiedSauvola_Binarization as MSB
```

# Sample Usage

```python

filename = "test_images/2.jpeg"
in_color = cv2.imread(filename,1)

plt.imshow(cv2.cvtColor(in_color, cv2.COLOR_BGR2RGB))
plt.title('input')
plt.show()

##########################################################################################
# Executing Sauvola for binarization

win_size = int(0.05*min(in_color.shape[0],in_color.shape[1]))
if (win_size%2==0):
    win_size = win_size + 1
k = 0.5
gray = cv2.cvtColor(in_color, cv2.COLOR_BGR2GRAY)
T = threshold_sauvola(gray, window_size=win_size,k=k)
binary = np.copy(gray)
binary[gray <= T] = 0
binary[gray > T] = 255

plt.imshow(gray,cmap = 'gray')
plt.title('gray')
plt.show()

plt.imshow(binary,cmap = 'gray')
plt.title('sauvola')
plt.show()
##########################################################################################
# Executing modified sauvola for binarization using deafult parameters

binary_default = MSB.SauvolaModBinarization(in_color)

plt.imshow(binary_default,cmap = 'gray')
plt.title('sauvola_mod_default')
plt.show()

##########################################################################################
# Executing modified sauvola for binarization using custom parameters

k1=0.5
k2=0.5
n1 = int(0.04*min(in_color.shape[0],in_color.shape[1]))
if (n1%2==0):
    n1 = n1+1
n2 = int(0.09*min(in_color.shape[0],in_color.shape[1]))
if (n2%2==0):
    n2 = n2+1
binary_nondefault = MSB.SauvolaModBinarization(in_color,n1,n2,k1,k2,default=False)

plt.imshow(binary_nondefault,cmap = 'gray')
plt.title('sauvola_mod_nondefault')
plt.show()

##########################################################################################
```

```python

```


```python
import sympy as sm
import matplotlib.pyplot as plt
import numpy as np
from chempy import ReactionSystem
from chempy.units import to_unitless, SI_base_registry as si, default_units as u, default_constants as const
from chempy.kinetics.ode import get_odesys
from chempy.kinetics.rates import RampedTemp
sm.init_printing()
%matplotlib inline
```

```python
t, t0, A, B, C1 = sm.symbols('t t0 A B C1')
```

```python
y = C1/sm.E**((A*(((t + t0)*(-B + t + t0))/sm.E**(B/(t + t0)) - B**2*sm.Ei(-(B/(t + t0)))))/2)
```

```python
y
```

```python
(y.diff(t)/y).simplify().expand().simplify().factor().powsimp(force=True)
```

```python
y.subs(t, 0)
```

```python
yunit0 = y.subs(C1, C1/y.subs(t, 0)).simplify()
yunit0
```

```python
from scipy.special import expi
f = sm.lambdify([t, t0, A, B], yunit0, modules=['numpy', {'Ei': expi}])
```

```python
R = 8.314472
T_K = 290
kB = 1.3806504e-23
h = 6.62606896e-34
dH = 80e3
dS = 10
rsys1 = ReactionSystem.from_string("""
NOBr -> NO + Br; EyringParam(dH={dH}*J/mol, dS={dS}*J/K/mol)
""".format(dH=dH, dS=dS))
kref = 20836643994.118652*T_K*np.exp(-(dH - T_K*dS)/(R*T_K))
kref
```

```python
_A = kB/h*np.exp(dS/R)
_B = dH/R
```

```python
f(np.array([0, 1, 5, 20]), 290, _A, _B)
```

```python
NOBr0_M = 0.7
init_cond = dict(
    NOBr=NOBr0_M*u.M,
    NO=0*u.M,
    Br=0*u.M
)
t = 20*u.second
```

```python
def integrate_and_plot(rsys):
    odes, extra = get_odesys(rsys, unit_registry=si, constants=const, substitutions={
        'temperature': RampedTemp([T_K*u.K, 1*u.K/u.s])})
    fig, all_axes = plt.subplots(2, 3, figsize=(14, 6))
    for axes, odesys in zip(all_axes, [odes, odes.as_autonomous()]):
        res = odesys.integrate(t, init_cond, integrator='cvode')
        t_sec = to_unitless(res.xout, u.second)
        NOBr_ref = NOBr0_M*f(t_sec, T_K, _A, _B)
        cmp = to_unitless(res.yout, u.M)
        ref = np.empty_like(cmp)
        ref[:, odesys.names.index('NOBr')] = NOBr_ref
        ref[:, odesys.names.index('Br')] = NOBr0_M - NOBr_ref
        ref[:, odesys.names.index('NO')] = NOBr0_M - NOBr_ref
        axes[0].plot(t_sec, cmp)
        axes[1].plot(t_sec, cmp - ref)
        res.plot_invariant_violations(ax=axes[2])
        assert np.allclose(cmp, ref)
        print({k: v for k, v in res.info.items() if not k.startswith('internal')})
```

```python
integrate_and_plot(rsys1)
```

```python
rsys2 = ReactionSystem.from_string("""
NOBr -> NO + Br; MassAction(EyringHS([{dH}*J/mol, {dS}*J/K/mol]))
""".format(dH=dH, dS=dS))
```

```python
integrate_and_plot(rsys2)
```


# Reconstructing an *off-axis* hologram by Fresnel Approximation

Reference: Digital holography and wavefront sensing by Ulf Schnars, Claas Falldorf, John Watson, and Werner Jüptner, Springer-verlag Berlin an, 2016. (Section 3.2)

## Info about the digital hologram:
'ulf7.BMP' is a digital hologram created by recording an object at about 1 meter distance with HeNe laser (632.8 nm) and an image sensor with 6.8 µm pixel size.

```python
#Import libraries realted to matplotlib and mathematical operations
# %matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from PIL import Image
import numpy as np
import ipywidgets as widgets
from IPython.display import display
```

```python
# Read the hologram image file
hologram = Image.open('ulf7.BMP')
hologram = np.array(hologram).astype(np.float64) #Convert into float type. Crucial for non integer based mathematical operations
# plot the hologram
imgplot = plt.imshow(hologram, cmap="viridis")
```

## Some equations from the book!

The *Fresnel-Kirchhoff* integral describing diffraction field beyond an aperture is given by the coherent superposition of the secondary waves (section 2.4)
\begin{equation}
\Gamma\left(\xi^{\prime}, \eta^{\prime}\right)=\frac{i}{\lambda} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} A(x, y) \frac{\exp \left(-i \frac{2 \pi}{\lambda} \rho^{\prime}\right)}{\rho^{\prime}} Q d x d y
\end{equation}
where, $A(x, y)$ is the complex amplitude in the plane of the diffracting aperture, $\rho^{\prime}$ is the distance between a point in the aperture plane and a point in the observation plane, and $Q$ is the inclination factor defined to take care of no backward propagation of the diffracted optical field. For holograms, $Q$ is approximately equal to 1.

A hologram $h(x,y)$ recorded by a reference light wave $E_{R}(x, y)$ can be reconstructed by a conjugate reference wave $E_{R}^{*}(x, y)$ as described by the following *Fresnel-Kirchhoff* integral
\begin{equation}
\Gamma(\xi, \eta)=\frac{i}{\lambda} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} h(x, y) E_{R}^{*}(x, y) \frac{\exp \left(-i \frac{2 \pi}{\lambda} \rho\right)}{\rho} d x d y
\end{equation}
with $\rho = \sqrt{ (x-\xi)^2 + (y-\eta)^2 + d^2 }$. Here $d$ is the distance between the object and hologram planes. Substituting the apprximated *Taylor* expansion of $\rho$ in above equation leads to the Fresnel reonstruction field relation (see section 3.2 of the book)
\begin{aligned} \Gamma(\xi, \eta)=& \frac{i}{\lambda d} \exp \left(-i \frac{2 \pi}{\lambda} d\right) \exp \left[-i \frac{\pi}{\lambda d}\left(\xi^{2}+\eta^{2}\right)\right] \times \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} E_{R}^{*}(x, y) h(x, y) \exp \left[-i \frac{\pi}{\lambda d}\left(x^{2}+y^{2}\right)\right] \exp \left[i \frac{2 \pi}{\lambda d}(x \xi+y \eta)\right] d x d y \end{aligned}
Or, in a digital form by
\begin{aligned} \Gamma(m, n)=& \frac{i}{\lambda d} \exp \left(-i \frac{2 \pi}{\lambda} d\right) \exp \left[-i \pi \lambda d\left(\frac{m^{2}}{N^{2} \Delta x^{2}}+\frac{n^{2}}{N^{2} \Delta y^{2}}\right)\right] \times \sum_{k=0}^{N-1} \sum_{l=0}^{N-1} E_{R}^{*}(k, l) h(k, l) \exp \left[-i \frac{\pi}{\lambda d}\left(k^{2} \Delta x^{2}+l^{2} \Delta y^{2}\right)\right] \exp \left[i 2 \pi\left(\frac{k m}{N}+\frac{l n}{N}\right)\right] \\ =& C \times \sum_{k=0}^{N-1} \sum_{l=0}^{N-1} E_{R}^{*}(k, l) h(k, l) \exp \left[-i \frac{\pi}{\lambda d}\left(k^{2} \Delta x^{2}+l^{2} \Delta y^{2}\right)\right] \exp \left[i 2 \pi\left(\frac{k m}{N}+\frac{l n}{N}\right)\right] \end{aligned}
where, $h(k,l)$ is the hologram, $N$ is number of pixels in camera sensor (assumed number of rows = number of columns, if not, convert the hologram in such a way prior to the operations), $\lambda$ is wavelength, $\Delta x$ and $\Delta y$ are horizontal and vertical distance of neighboring sensor pixels, and $d$ is the distance of reconstruction. It is easy to see that the last term under discrete integral is actually an IFT (inverse Fourier transform) of a multiple of hologram function and an exponential factor term. C is just a complex constant which does not affect the reconstruction process and $E_{R}^{*}(k, l)$ gets simplified to unity for a plane wave as reconstruction/recording wave.

```python
# User defined reconstruction distance
w = widgets.FloatSlider(value=-1.054,min=-2.0,max=2.0,step=0.001,
description='d (in meters):',orientation='horizontal',readout=True,readout_format='.3f',)
display(w)
```

```python
# User define parameters
Nr,Nc = np.shape(hologram) #number of rows and columns in the hologram
wavelength = 632.8e-9 #HeNe laser wavelength in SI units i.e. meters
dx = 6.8e-6 #sensor pixel size in meters
d = w.value #-1.054 #reconstruction distance in meters
# prepare the Fresnel operand for the hologram
Nr = np.linspace(0, Nr-1, Nr)-Nr/2
Nc = np.linspace(0, Nc-1, Nc)-Nc/2
k, l = np.meshgrid(Nc,Nr)
factor = np.multiply(hologram, np.exp(-1j*np.pi/(wavelength*d)*(np.multiply(k, k)*dx**2 + np.multiply(l, l)*dx**2)))
reconstructed_field = np.fft.ifftshift(np.fft.ifft2(np.fft.ifftshift(factor))) # Take inverse Fourier transform of the factor
# plot
I = np.abs(reconstructed_field)/np.max(np.abs(reconstructed_field)) #normalized intensity profile
fig = plt.figure(figsize=(10,10)) #setup a blank figure
plt.imshow(I, cmap="hot", clim=(0.0, 0.3))
plt.colorbar()
```


# Random Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing.

## Ensemble Averages

Ensemble averages characterize the average properties of a random process across the population of all possible sample functions in the ensemble. We distinguish between first- and higher-order ensemble averages. The former consider the average properties of the sample functions of one random process for one particular time-instant $k$, while the latter take more than one random process at different time-instants into account.

### First-Order Ensemble Averages

#### Definition

The first-order ensemble average of a continuous-amplitude real-valued random signal $x[k]$ is defined as

\begin{equation}
E\{ f(x[k]) \} = \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} f(x_n[k])
\end{equation}

where $E\{ \cdot \}$ denotes the expectation operator, $x_n[k]$ the $n$-th sample function and $f(\cdot)$ an arbitrary mapping function. It is evident from the definition that the ensemble average can only be given exactly for random processes where the internal structure is known. For practical random processes, like e.g. speech, the ensemble average can only be approximated by a finite but sufficiently large number $N$ of sample functions.

The ensemble average can also be given in terms of the univariate probability density function (PDF)

\begin{equation}
E\{ f(x[k]) \} = \int\limits_{-\infty}^{\infty} f(\theta) \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

#### Properties

The following properties can be concluded from the definition of the ensemble average:

1. The ensemble averages of two different time-instants $k_1$ and $k_2$ differ in general

    \begin{equation}
    E\{ f(x[k_1]) \} \neq E\{ f(x[k_2]) \}
    \end{equation}

2. For a linear mapping $f(x[k]) = C \cdot x[k]$ with $C \in \mathbb{R} \setminus \{0\}$, the ensemble average is a linear operation

    \begin{equation}
    E\{ A \cdot x[k] + B \cdot y[k] \} = A \cdot E\{ x[k] \} + B \cdot E\{ y[k] \}
    \end{equation}

3. For a deterministic signal $x_n[k] = s[k]$ $\forall n$ the ensemble average is

    \begin{equation}
    E\{ f(s[k]) \} = f(s[k])
    \end{equation}

The choice of the mapping function $f(\cdot)$ determines the particular property of the random process which is characterized by the ensemble average. Common choices are discussed in the following.

#### Linear mean

The linear [mean](https://en.wikipedia.org/wiki/Mean), which is given for the linear mapping $f(x[k]) = x[k]$, is the arithmetic mean value across all sample functions $x_n[k]$ for a given time instant $k$. It is also known as the first (raw) [moment](https://en.wikipedia.org/wiki/Moment_%28mathematics%29).

Introducing $f(x[k]) = x[k]$ into the definition of the ensemble average yields

\begin{equation}
\mu_x[k] = E\{ x[k] \} = \int\limits_{-\infty}^{\infty} \theta \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

where $\mu_x[k]$ commonly denotes the linear mean. A process with $\mu_x[k] = 0$ is termed as *zero-mean* or *mean-free*. Note that $\mu_x$ should not be confused with the discrete frequency index of the DFT.

#### Quadratic mean

The quadratic mean is given by the mapping $f(x[k]) = x^2[k]$

\begin{equation}
E\{ x^2[k] \} = \int\limits_{-\infty}^{\infty} \theta^2 \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

It quantifies the average instantaneous power of a sample function for a given time index $k$. It is also known as second (raw) moment.

#### Variance

The [variance](https://en.wikipedia.org/wiki/Variance) is defined as the quadratic mean of a zero-mean random process. For a general random process with linear mean $\mu_x$ it is given as

\begin{equation}
\sigma_x^2[k] = E\{ (x[k] - \mu_x[k])^2 \}
\end{equation}

where $\sigma_x^2[k]$ commonly denotes the variance, $\sigma_x[k]$ is known as the [standard deviation](https://en.wikipedia.org/wiki/Standard_deviation). Note that the subtraction of the linear mean $\mu_x$ from the samples $x_n[k]$ makes a random signal mean-free. The variance characterizes how far the amplitude values of a random signal are spread out from its mean value. It is also known as second centralized moment.

The variance can be given in terms of the linear and quadratic mean

\begin{equation}
\sigma_x^2[k] = E\{ x^2[k] \} - \mu_x^2[k]
\end{equation}

**Excercise**

* Derive above relation from the definitions and properties of the first order ensemble average.

Solution: Taking the linearity of the expectation operator and the fact that the linear mean is a deterministic value into account yields

\begin{equation}
\begin{split}
\sigma_x^2[k] &= E\{ (x[k] - \mu_x[k])^2 \} \\
&= E\{ x^2[k] - 2 \mu_x[k] x[k] + \mu_x^2[k]  \} \\
&= E\{ x^2[k] \} - 2 \mu_x[k] E\{ x[k] \} + \mu_x^2[k] \\
& = E\{ x^2[k] \} - \mu_x^2[k]
\end{split}
\end{equation}

#### Example - Linear and quadratic mean, and variance of a random process

The following example computes and plots the linear $\mu_x$ and quadratic mean, and variance $\sigma_x^2$ of a random process. Since in practice only a limited number $N$ of sample functions can be evaluated numerically the true values of these quantities are only approximated/estimated. Estimated quantities are denoted by a 'hat' over the respective quantity, e.g. $\hat{\mu}_x[k]$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

K = 64  # number of random samples
N = 1000  # number of sample functions

# generate the sample functions
np.random.seed(5)
x = np.random.normal(size=(N, K))
x += np.tile(np.cos(2*np.pi/K*np.arange(K)), [N, 1])

# estimate the linear mean as ensemble average
mu = 1/N * np.sum(x, 0)
# estimate the quadratic mean
qu = 1/N * np.sum(x**2, 0)
# estimate the variance
sigma = 1/N * np.sum((x-mu)**2, 0)

# plot results
plt.rc('figure', figsize=(10, 3))

plt.figure()
plt.stem(x[0, :])
plt.title(r'Sample function $x_0[k]$')
plt.xlabel(r'$k$')
plt.ylabel(r'$x_0[k]$')
plt.axis([0, K, -3, 3])

plt.figure()
plt.stem(mu)
plt.title(r'Estimate of linear mean')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{\mu}_x[k]$')
plt.axis([0, K, -1.5, 1.5])

plt.figure()
plt.stem(qu)
plt.title(r'Estimate of quadratic mean')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{E}\{x^2[k]\}$')
plt.axis([0, K, 0, 2.5])

plt.figure()
plt.stem(sigma)
plt.title(r'Estimate of variance')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{\sigma}^2_x[k]$')
plt.axis([0, K, 0, 1.5]);
```

**Exercise**

* What does the linear and quadratic mean, and the variance tell you about the average behavior of the sample functions?
* Change the number `N` of sample functions and rerun the example. What influence has a decrease/increase of the sample functions on the estimated ensemble averages?

Inspection of the estimated linear mean reveals that in average the sample functions follow a cosine function with respect to the sample index $k$. The variance shows that the amount of which the samples of the sample functions for one particular time-instant $k$ are spread around the linear mean is constant. The estimate of the quadratic mean is given as $\hat{E}\{ x^2[k] \} = \hat{\mu}_x^2[k] + \hat{\sigma}_x^2[k]$. The higher the number $N$ of sample functions used for the estimate of the ensemble averages, the lower the uncertainty in comparison to the true values becomes.

### Second-Order Ensemble Averages

#### Definition

The second-order ensemble average of two continuous-amplitude real-valued random signals $x[k]$ and $y[k]$ is defined as

\begin{equation}
E\{ f(x[k_x], y[k_y]) \} := \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} f(x_n[k_x], y_n[k_y])
\end{equation}

It is given in terms of the bivariate PDF as

\begin{equation}
E\{ f(x[k_x], y[k_y]) \} = \iint\limits_{-\infty}^{\infty} f(\theta_x, \theta_y) \, p_{xy}(\theta_x, \theta_y, k_x, k_y) \, \mathrm{d}\theta_x\, \mathrm{d}\theta_y
\end{equation}

The definition of the second-order ensemble average can be extended straightforward to the case of more than two random variables. The resulting ensemble average is then termed as higher-order ensemble average.

By setting $y[k] = x[k]$, the second-order ensemble average can also be used to characterize the average properties of the sample functions between two different time-instants $k_1$ and $k_2$ of one random process

\begin{equation}
\begin{split}
E\{ f(x[k_1], x[k_2]) \} &= \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} f(x_n[k_1], x_n[k_2]) \\
&= \iint\limits_{-\infty}^{\infty} f(\theta_1, \theta_2) \, p_{xx}(\theta_1, \theta_2, k_1, k_2) \, \mathrm{d}\theta_1\, \mathrm{d}\theta_2
\end{split}
\end{equation}

The choice of the mapping function $f(\cdot)$ determines the particular property of the random process which is characterized by the ensemble average. The case of a linear mapping is discussed in the following.

#### Cross-correlation function

The [cross-correlation function](https://en.wikipedia.org/wiki/Cross-correlation) (CCF) of two random signals $x[k]$ and $y[k]$ is defined as the second-order ensemble average for a linear mapping $f(x[k_x], y[k_y]) = x[k_x] \cdot y[k_y]$

\begin{equation}
\varphi_{xy}[k_x, k_y] = E\{ x[k_x] \cdot y[k_y] \}
\end{equation}

It characterizes the statistical dependencies of two random signals $x[k]$ and $y[k]$ at two different time instants $k_x$ and $k_y$.

#### Auto-correlation function

The [auto-correlation function](https://en.wikipedia.org/wiki/Autocorrelation) (ACF) of a random signal $x[k]$ is defined as the second-order ensemble average for a linear mapping $f(x[k_1], y[k_2]) = x[k_1] \cdot x[k_2]$

\begin{equation}
\varphi_{xx}[k_1, k_2] = E\{ x[k_1] \cdot x[k_2] \}
\end{equation}

It characterizes the average statistical dependencies between the samples of a random signal $x[k]$ at two different time instants $k_1$ and $k_2$.

#### Example - Auto-correlation function of a random signal

The following example estimates the ACF of a random signal $x[k]$ as ensemble average of a finite number $N$ of sample functions. The ACF is plotted as image where the colors denote the level of the ACF $\varphi_{xx}[k_1, k_2]$ for given time-instants $k_1$ and $k_2$.

```python
L = 64  # number of random samples
N = 1000  # number of sample functions

# generate sample functions
np.random.seed(1)
r = np.random.normal(size=(N, L))
h = np.random.normal(size=(N, 10))
x = np.asarray([np.convolve(r[n,:], h[n,:], mode='same') for n in range(N)]) \
    + np.tile(np.cos(2*np.pi/L*np.arange(L)), [N,1])

# estimate the auto-correlation function (ACF)
acf = np.zeros((L, L))
for n in range(L):
    for m in range(L):
        acf[n, m] = 1/N * np.sum(x[:, n]*x[:, m], axis=0)

# plot ACF
plt.figure(figsize = (7, 5))
plt.pcolor(np.arange(L), np.arange(L), acf, vmin=-2, vmax=2)
plt.title(r'Estimate of ACF $\hat{\varphi}_{xx}[k_1, k_2]$')
plt.xlabel(r'$k_1$')
plt.ylabel(r'$k_2$')
plt.colorbar()
plt.axis('tight');
```

**Exercise**

* Can you explain the specific symmetry of the ACF?

Solution: Inspection of the definition of the ACF $\varphi_{xx}[k_1, k_2] = \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} x_n[k_1] \cdot x_n[k_2]$ reveals that the sample indexes $k_1$ and $k_2$ can be interchanged without changing the value of the ACF. Hence $\varphi_{xx}[k_1, k_2] = \varphi_{xx}[k_2, k_1]$. This symmetry can be observed in above example.


# Занятие 4
# Прикладная алгебра и численные методы
## Сингулярное разложение (SVD), линейная регрессия
https://numpy.org/doc/stable/reference/generated/numpy.linalg.svd.html#numpy.linalg.svd

```python
import numpy as np
import scipy.linalg
import sympy
import matplotlib.pyplot as plt
from copy import deepcopy
%matplotlib inline
```

## Сингулярное разложение (SVD)
$$
A = Q\Sigma P^*, \quad A_{m\times n},\ Q_{m\times m}, \ \Sigma_{m\times n}, \ P_{n\times n},
$$
$Q$, $P$ - ортогональные матрицы, $\Sigma$ - диагональная, на диагонали сингулярные числа.

## Пример 1
Найти SVD
$$
\left(
\begin{matrix}
1 & 0 & 0 & 1\\
0 & 1 & 0 & 1\\
0 & 0 & 1 & 1
\end{matrix}
\right)
$$
Вначале вычислим $A^*A$:

```python
A = sympy.Matrix([[1, 0, 0, 1], [0, 1, 0, 1], [0, 0, 1, 1]])
A_star_A = A.T*A
```

Получим собственные числа и собственные векторы с помощью eigenvects(), нормализуем векторы (чтобы норма была равна единице) методом normalized()

```python
A_star_A_sympy_ev = sympy.Matrix(A_star_A).eigenvects()
A_star_A_sympy_ev = [(item[0], item[1], [elem.normalized() for elem in item[2]]) for item in A_star_A_sympy_ev]
display(*A_star_A_sympy_ev)
```

    (0, 1, [Matrix([
      [-1/2],
      [-1/2],
      [-1/2],
      [ 1/2]])])

    (1, 2, [Matrix([
      [-sqrt(2)/2],
      [ sqrt(2)/2],
      [         0],
      [         0]]), Matrix([
      [-sqrt(2)/2],
      [         0],
      [ sqrt(2)/2],
      [         0]])])

    (4, 1, [Matrix([
      [sqrt(3)/6],
      [sqrt(3)/6],
      [sqrt(3)/6],
      [sqrt(3)/2]])])

Выделим собственные векторы, обозначим их e0, e11, e12, e4, они соответствуют собственным значеним 0, 1, 1 и 4. К двум векторам, соответствующим собственному значению 1 применим процесс ортогонализации Грамма-Шмидта
$$
\begin{matrix}
e_1^{new} = e_1\\
e_2^{new} = e_2 - \frac{(e_1, e_2)}{(e_1, e_1)}e_1
\end{matrix}
$$
Полученный ортогональный вектор нормализуем, проверим ортогональность с помощью скалярного произведения:

```python
e0, e11, e12, e4 = A_star_A_sympy_ev[0][2] + A_star_A_sympy_ev[1][2] + A_star_A_sympy_ev[2][2]
e12 = (e12 - e11.dot(e12)*e11).normalized()
display(e11.dot(e12))
P = e4.row_join(e11).row_join(e12).row_join(e0)
display(P)
```

$\displaystyle 0$

$\displaystyle \left[\begin{matrix}\frac{\sqrt{3}}{6} & - \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6} & - \frac{1}{2}\\\frac{\sqrt{3}}{6} & \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6} & - \frac{1}{2}\\\frac{\sqrt{3}}{6} & 0 & \frac{\sqrt{6}}{3} & - \frac{1}{2}\\\frac{\sqrt{3}}{2} & 0 & 0 & \frac{1}{2}\end{matrix}\right]$

Построим векторы-столбцы матрицы $Q$ и проверим, что найдено разложение SVD для исходной матрицы:

```python
sigma = (2, 1, 1)
f1, f2, f3 = [A*ei/sigma[i] for i, ei in enumerate((e4, e11, e12))]
Q = f1.row_join(f2).row_join(f3)
Sig = sympy.Matrix([[2, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0]])
display(Q, Sig, P, Q*Sig*P.T)
```

$\displaystyle \left[\begin{matrix}\frac{\sqrt{3}}{3} & - \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6}\\\frac{\sqrt{3}}{3} & \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6}\\\frac{\sqrt{3}}{3} & 0 & \frac{\sqrt{6}}{3}\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}2 & 0 & 0 & 0\\0 & 1 & 0 & 0\\0 & 0 & 1 & 0\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}\frac{\sqrt{3}}{6} & - \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6} & - \frac{1}{2}\\\frac{\sqrt{3}}{6} & \frac{\sqrt{2}}{2} & - \frac{\sqrt{6}}{6} & - \frac{1}{2}\\\frac{\sqrt{3}}{6} & 0 & \frac{\sqrt{6}}{3} & - \frac{1}{2}\\\frac{\sqrt{3}}{2} & 0 & 0 & \frac{1}{2}\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 1\\0 & 1 & 0 & 1\\0 & 0 & 1 & 1\end{matrix}\right]$

Теперь то же самое, но с numpy, вычислим $A^*A$:

```python
A = np.array([[1, 0, 0, 1],
              [0, 1, 0, 1],
              [0, 0, 1, 1]])
A_star_A = np.matmul(A.T, A)
display(A_star_A)
```

    array([[1, 0, 0, 1],
           [0, 1, 0, 1],
           [0, 0, 1, 1],
           [1, 1, 1, 3]])

Найдем собственные числа и собственные векторы полученной матрицы:

```python
A_star_A_eigen_vals, A_star_A_eigen_vects = np.linalg.eig(A_star_A)
display('CЧ', A_star_A_eigen_vals, 'СВ', A_star_A_eigen_vects)
```

    'CЧ'

    array([ 4.000000e+00, -2.493665e-16,  1.000000e+00,  1.000000e+00])

    'СВ'

    array([[ 2.88675135e-01,  5.00000000e-01, -8.16496581e-01,
            -4.08248290e-01],
           [ 2.88675135e-01,  5.00000000e-01,  4.08248290e-01,
            -4.08248290e-01],
           [ 2.88675135e-01,  5.00000000e-01,  4.08248290e-01,
             8.16496581e-01],
           [ 8.66025404e-01, -5.00000000e-01,  3.34893759e-16,
             2.58096210e-16]])

Расположим сингуляные числа (квадратные корни из полученных собственных чисел) по убыванию, для этого сначала отсортируем их с помощью sort() по возрастанию, а затем запишем array в обратном порядке с помощью flip():

```python
A_star_A_eigen_vals.sort()
A_star_A_eigen_vals_reversed = np.flip(A_star_A_eigen_vals)
display(A_star_A_eigen_vals, A_star_A_eigen_vals_reversed)
```

    array([-2.493665e-16,  1.000000e+00,  1.000000e+00,  4.000000e+00])

    array([ 4.000000e+00,  1.000000e+00,  1.000000e+00, -2.493665e-16])

Обратите внимание, что .sort() изменяет array на месте, а flip() возвращает view записанного в обратном порядке array, не изменяя его.
## !!!
По сути, мы получаем указатель на конец нашего array, так что все действия, которые мы проделаем с элементами A_star_A_eigen_vals_reversed автоматически распространятся на A_star_A_eigen_vals, поскольку это не два разных array, а один, только номера элементов считаются по-разному:

```python
arr1 = np.array([1, 2, 3, 4])
arr1_reversed = np.flip(arr1)
arr1[0] = 9
display(arr1, arr1_reversed)
arr1_reversed[-2] = 8
display(arr1, arr1_reversed)
```

    array([9, 2, 3, 4])

    array([4, 3, 2, 9])

    array([9, 8, 3, 4])

    array([4, 3, 8, 9])

Поскольку нам достаточно работать с A_star_A_eigen_vals_reversed, не будем делать deepcopy(), чтобы сохратить в неприкосновенности A_star_A_eigen_vals.

Осталось извлечь квадратные корни из положительных элементов A_star_A_eigen_vals_reversed, и получим невозрастающую последовательность сингулярных значений.

```python
sigmas = [round(np.sqrt(item), 1) for item in A_star_A_eigen_vals_reversed if item > 0]
sigmas
```

    [2.0, 1.0, 1.0]

Составим матрицу $\Sigma$:

```python
Sigma = np.hstack((np.diag(sigmas), np.zeros((3, 1))))
Sigma
```

    array([[2., 0., 0., 0.],
           [0., 1., 0., 0.],
           [0., 0., 1., 0.]])

Обратимся к полученным вместе с собственными числами собственным векторам:

```python
e4, e0, e11, e12 = [item.reshape((4, 1)) for item in  A_star_A_eigen_vects.T]
A_star_A_eigen_vects = [e4, e0, e11, e12]
display(A_star_A_eigen_vects)
```

    [array([[0.28867513],
            [0.28867513],
            [0.28867513],
            [0.8660254 ]]), array([[ 0.5],
            [ 0.5],
            [ 0.5],
            [-0.5]]), array([[-8.16496581e-01],
            [ 4.08248290e-01],
            [ 4.08248290e-01],
            [ 3.34893759e-16]]), array([[-4.08248290e-01],
            [-4.08248290e-01],
            [ 8.16496581e-01],
            [ 2.58096210e-16]])]

Вычислим нормы полученных векторов и скалярное произведение

```python
display(*[np.linalg.norm(item) for item in A_star_A_eigen_vects], 'скалярное произведение', e11[0].dot(e12[0]))
```

    0.9999999999999999

    1.0

    1.0

    1.0

    'скалярное произведение'

    0.3333333333333335

Сначала заменим e12 на вектор, ортогональный e11, затем нормализуем векторы и составим из них матрицу $P$:

```python
A_star_A_eigen_vects = [item/np.linalg.norm(item) for item in A_star_A_eigen_vects]
e4, e0, e11, e12 = A_star_A_eigen_vects
e12 = e12 - (e11[0].dot(e12[0])/(e11[0].dot(e11[0])))*e11
print('(e11, e12new) =', e11[0].dot(e12[0]))
A_star_A_eigen_vects[-1] = e12
e4, e0, e11, e12 = [item/np.linalg.norm(item) for item in A_star_A_eigen_vects]
P = np.hstack((e4, e11, e12, e0))
P
```

    (e11, e12new) = -0.0

    array([[ 2.88675135e-01, -8.16496581e-01,  0.00000000e+00,
             5.00000000e-01],
           [ 2.88675135e-01,  4.08248290e-01, -7.07106781e-01,
             5.00000000e-01],
           [ 2.88675135e-01,  4.08248290e-01,  7.07106781e-01,
             5.00000000e-01],
           [ 8.66025404e-01,  3.34893759e-16,  1.04672831e-16,
            -5.00000000e-01]])

Составим матрицу $Q$:

```python
sigma = (2, 1, 1)
f1, f2, f3 = [np.matmul(A, ei)/sigma[i] for i, ei in enumerate((e4, e11, e12))]
Q = np.hstack((f1, f2, f3))
Sig = np.hstack((np.diag(sigma), np.zeros((3, 1))))
display(Q, Sig, P, np.matmul(np.matmul(Q, Sig), P.T))
```

    array([[ 5.77350269e-01, -8.16496581e-01,  1.04672831e-16],
           [ 5.77350269e-01,  4.08248290e-01, -7.07106781e-01],
           [ 5.77350269e-01,  4.08248290e-01,  7.07106781e-01]])

    array([[2., 0., 0., 0.],
           [0., 1., 0., 0.],
           [0., 0., 1., 0.]])

    array([[ 2.88675135e-01, -8.16496581e-01,  0.00000000e+00,
             5.00000000e-01],
           [ 2.88675135e-01,  4.08248290e-01, -7.07106781e-01,
             5.00000000e-01],
           [ 2.88675135e-01,  4.08248290e-01,  7.07106781e-01,
             5.00000000e-01],
           [ 8.66025404e-01,  3.34893759e-16,  1.04672831e-16,
            -5.00000000e-01]])

    array([[ 1.00000000e+00,  2.64089394e-16,  2.60350884e-16,
             1.00000000e+00],
           [ 9.67157293e-17,  1.00000000e+00, -1.88904597e-16,
             1.00000000e+00],
           [ 1.66992088e-16, -6.83580866e-17,  1.00000000e+00,
             1.00000000e+00]])

## Построение псевдообратной матрицы при помощи SVD
$$
A^+ = P\Sigma^+Q^*,\quad
\Sigma^+ =
\left(
\begin{matrix}
\sigma_1^{-1} & ... & ... & ... & ... & 0\\
0 & \sigma_1^{-1} & ... & ... & ... & 0\\
0 & ... & ... & ... & ... & 0\\
0 & ... & ... & \sigma_r^{-1}  & ... & 0\\
0 & ... & ... & ... & ... & 0\\
\end{matrix}
\right)
$$

```python
Sigma_plus = np.vstack((np.diag([1/item for item in sigma]), np.zeros((1, 3))))
A_pinv_my = np.matmul(np.matmul(P, Sigma_plus), Q.transpose())
display(A_pinv_my, np.linalg.pinv(A))
```

    array([[ 0.75, -0.25, -0.25],
           [-0.25,  0.75, -0.25],
           [-0.25, -0.25,  0.75],
           [ 0.25,  0.25,  0.25]])

    array([[ 0.75, -0.25, -0.25],
           [-0.25,  0.75, -0.25],
           [-0.25, -0.25,  0.75],
           [ 0.25,  0.25,  0.25]])

## И наконец SVD от numpy:

```python
Q, sigma, P = np.linalg.svd(A, full_matrices=True)
Sig = np.hstack((np.diag(sigma), np.zeros((3, 1))))
display('P.T', P, 'sigma', sigma, 'Q', Q, 'Sig', Sig, 'QSigP.T', np.matmul(np.matmul(Q, Sig), P))
```

## Линейная регрессия
В некотором эксперименте измерялись значения величин $g_1$, $g_2$, $g_3$ и $H$:
$$
\begin{matrix}
g_1 & 0.12 & 0.15 & 0.9 & 0.8\\
g_2 & 2.4  & 1.8  & 3.2 & 3.6\\
g_3 & 1.1  & 1.2  & 1.3 & 1.4\\
H   & 5.1  & 6.2  & 5.5 & 4.1
\end{matrix}
$$

Найти коэффициенты $a$, $b$, $c$ линейной регрессии $H = ag_1 + bg_2 +cg_3$.

Составим матрицу $A$ столбцы которой образуют значения $g_1$, $g_2$, $g_3$.
Также составим матрицу-столбец $H$ из значений $H$,
тогда
$$
\left[\begin{matrix}a\\b\\c\end{matrix}\right] = A^+H
$$

```python
A = np.array([[ 0.12, 0.15, 0.9, 0.8],
               [ 2.4, 1.8, 3.2, 3.6],
               [1.1, 1.2, 1.3, 1.4]]).transpose()
H1 = np.array([[5.1], [6.2], [5.5],  [4.1]])
res = np.matmul(np.linalg.pinv(A), H1)
a, b, c = res[:, 0]
display('A', A, 'H', H1, 'a', a, 'b', b, 'c', c)
```

Вычислим относительные отклонения экспериментальных данных от функции $H = ag_1 + bg_2 + cg_3$

```python
def Hfunc(g1, g2, g3):
    return a*g1 + b*g2 + c*g3
print(*[abs((Hfunc(*g) - H1[i][0])/H1[i][0]) for i, g in enumerate(A)])
```


# [HW10] Simple Linear Regression

## 1. Linear regression

Linear regression은 종속 변수 $y$와 한개 이상의 독립 변수 $X$와의 선형 관계를 모델링하는 방법론입니다. 여기서 독립 변수는 입력 값이나 원인을 나타내고, 종속 변수는 독립 변수에 의해 영향을 받는 변수입니다. 종속 변수는 보통 결과물을 나타냅니다.

선형 관계를 모델링한다는 것은 1차로 이루어진 직선을 구하는 것입니다. 우리의 데이터를 가장 잘 설명하는 최적의 직선을 찾아냄으로써 독립 변수와 종속 변수 사이의 관계를 도출해 내는 과정입니다.

이번 실습에서는 독립 변수가 1개인 simple linear regression을 진행하겠습니다. 변수가 하나인 직선을 정의하겠습니다.
$$f(x_i) = wx_i + b$$

우리의 데이터를 가장 잘 설명하는 직선은 우리가 직선을 통해 예측한 값이 실제 데이터의 값과 가장 비슷해야 합니다. 우리의 모델이 예측한 값은 위에서 알 수 있듯 $f(x_i)$입니다. 그리고 실제 데이터는 $y$ 입니다.

실제 데이터(위 그림에서 빨간 점) 과 직선 사이의 차이를 줄이는 것이 우리의 목적입니다. 그것을 바탕으로 cost function을 다음과 같이 정의해보겠습니다.

$$\text{cost function} = \frac{1}{N}\sum_{i=1}^n (y_i - f(x_i))^2$$

우리는 cost function을 최소로 하는 $w$와 $b$를 찾아야 합니다.

우리의 cost function은 이차함수입니다. 우리는 고등학교 수학시간에 이차함수의 최솟값을 구하는 방법을 배웠습니다! 고등학교 때 배웠던 방법을 다시 한번 알아보고, 새로운 gradient descent 방법도 알아보겠습니다.

### 1.1 Analytically

다음 식의 최솟값을 어떻게 찾을 수 있을까요?
$$f(w) = w^2 + 3w -5$$

고등학교 때 배운 방법은 미분한 값이 0이 되는 지점을 찾는 것입니다.

손으로 푸는 방법은 익숙하겠지만 sympy와 numpy 패키지를 사용하여 코드를 통해서 알아보도록 하겠습니다.

```python
import sympy
import numpy

from matplotlib import pyplot
%matplotlib inline
sympy.init_printing()
```

```python
w = sympy.Symbol('w', real=True)

f = w**2 + 3*w - 5
f
```

```python
sympy.plotting.plot(f);
```

1차 미분한 식은 다음과 같이 알아볼 수 있습니다.

```python
fprime = f.diff(w)
fprime

```

그리고 해당 식의 해는 다음과 같이 구할 수 있습니다.

```python
sympy.solve(fprime, w)
```

### 1.2 Gradient Descent

두번째 방법은 오늘 배운 Gradient Descent 방법으로 한번에 정답에 접근하는 것이 아닌 반복적으로 정답에 가까워지는 방법입니다.

이것도 코드를 통해서 이해해보도록 하겠습니다.

먼저 기울기값을 구하는 함수를 먼저 만들겠습니다.

```python
fpnum = sympy.lambdify(w, fprime)
type(fpnum)
```

    function

그 다음 처음 $w$ 값을 설정한 뒤, 반복적으로 최솟값을 향해서 접근해보겠습니다.

```python
w = 10.0   # starting guess for the min

for i in range(1000):
    w = w - fpnum(w)*0.01 # with 0.01 the step size

print(w)
```

    -1.4999999806458753

이처럼 첫번째 방법과 두번째 방법에서 같은 값이 나온 것을 알 수 있습니다.

Gradient descent 방법을 직접 데이터를 만들어서 적용해보겠습니다.

### 1.3 Linear regression

실제로 linear 한 관계를 가진 데이터 셋을 사용하기 위해서 직접 데이터를 만들어보도록 하겠습니다.

Numpy 패키지 안에 Normal distribution 함수를 통해서 조금의 noise 를 추가해서 생성하도록 하겠습니다.

```python
x_data = numpy.linspace(-5, 5, 100)
w_true = 2
b_true = 20

y_data = w_true*x_data + b_true + numpy.random.normal(size=len(x_data))

pyplot.scatter(x_data,y_data);
```

```python
x_data.shape
```

```python
y_data.shape
```

총 100개의 데이터를 생성하였습니다. 이제 코드를 통해 접근해보도록 하겠습니다.

먼저 cost function을 나타내보겠습니다.

```python
w, b, x, y = sympy.symbols('w b x y')

cost_function = (w*x + b - y)**2
cost_function
```

위의 gradient descent 예시에서 한 것처럼 기울기 함수를 정의합니다.

```python
grad_b = sympy.lambdify([w,b,x,y], cost_function.diff(b), 'numpy')
grad_w = sympy.lambdify([w,b,x,y], cost_function.diff(w), 'numpy')
```

이제 $w$와 $b$의 초기값을 정의하고 gradient descent 방법을 적용하여 cost function을 최소로 하는 $w$와 $b$ 값을 찾아보겠습니다.

```python
w = 0
b = 0

for i in range(1000):
    descent_b = numpy.sum(grad_b(w,b,x_data,y_data))/len(x_data)
    descent_w = numpy.sum(grad_w(w,b,x_data,y_data))/len(x_data)
    w = w - descent_w*0.01 # with 0.01 the step size
    b = b - descent_b*0.01

print(w)
print(b)
```

    2.0303170198038307
    19.90701393429327

처음에 데이터를 생성할 때 정의한 $w, b$ 값과 매우 유사한 값을 구할 수 있었습니다.

```python
pyplot.scatter(x_data,y_data)
pyplot.plot(x_data, w*x_data + b, '-r');
```

우리가 구한 직선이 데이터와 잘 맞는 것을 볼 수 있습니다. 이번에는 실제 데이터에서 linear regression을 진행해보겠습니다.

## 2. Earth temperature over time

오늘 배운 linear regression 방법을 사용해서 시간 흐름에 따른 지구의 온도 변화를 분석해보겠습니다.

Global temperature anomaly라는 지표를 통해서 분석을 해볼 것입니다.

여기서 temperature anomaly는 어떠한 기준 온도 값을 정해놓고 그것과의 차이를 나타낸 것입니다. 예를 들어서 temperature anomaly가 양수의 높은 값을 가진다면 그것은 평소보다 따듯한 기온을 가졌다는 말이고, 음수의 작은 값을 가진다면 그것은 평소보다 차가운 기온을 가졌다는 말입니다.

세계 여러 지역의 온도가 각각 다 다르기 때문에 global temperature anomaly를 사용해서 분석을 하도록 하겠습니다. 자세한 내용은 아래 링크에서 확인하실 수 있습니다.

https://www.ncdc.noaa.gov/monitoring-references/faq/anomalies.php

```python
from IPython.display import YouTubeVideo
YouTubeVideo('gGOzHVUQCw0')
```

위 영상으로 기온이 점점 상승하고 있다는 것을 알 수 있습니다.

이제부터는 실제 데이터를 가져와서 분석해보도록 하겠습니다.

### Step 1 : Read a data file

NOAA(National Oceanic and Atmospheric Administration) 홈페이지에서 데이터를 가져오겠습니다.

아래 명령어로 데이터를 다운받겠습니다.

```python
from urllib.request import urlretrieve
URL = 'http://go.gwu.edu/engcomp1data5?accessType=DOWNLOAD'
urlretrieve(URL, 'land_global_temperature_anomaly-1880-2016.csv')
```

    ('land_global_temperature_anomaly-1880-2016.csv',
     <http.client.HTTPMessage at 0x7f158b3dca50>)

다운로드한 데이터를 numpy 패키지를 이용해 불러오겠습니다.

```python
import numpy
```

```python
fname = '/content/land_global_temperature_anomaly-1880-2016.csv'

year, temp_anomaly = numpy.loadtxt(fname, delimiter=',', skiprows=5, unpack=True)
```

### Step 2 : Plot the data

Matplotlib 패키지의 pyplot을 이용해서 2D plot을 찍어보도록 하겠습니다.

```python
from matplotlib import pyplot
%matplotlib inline
```

```python
pyplot.plot(year, temp_anomaly);
```

Plot 에 여러 정보를 추가해서 더 보기 좋게 출력해보겠습니다.

```python
pyplot.rc('font', family='serif', size='18')

#You can set the size of the figure by doing:
pyplot.figure(figsize=(10,5))

#Plotting
pyplot.plot(year, temp_anomaly, color='#2929a3', linestyle='-', linewidth=1)
pyplot.title('Land global temperature anomalies. \n')
pyplot.xlabel('Year')
pyplot.ylabel('Land temperature anomaly [°C]')
pyplot.grid();
```

### Step 3 : Analytically

Linear regression을 하기 위해서 먼저 직선을 정의하겠습니다.
$$f(x_i) = wx + b$$

그 다음 수업 시간에 배운 cost function을 정의하도록 하겠습니다. 우리가 최소화 해야 할 cost function은 다음과 같습니다.

$$\frac{1}{n} \sum_{i=1}^n (y_i - f(x_i))^2 = \frac{1}{n} \sum_{i=1}^n (y_i - (wx_i + b))^2$$

이제 cost function 을 구하고자 하는 변수로 미분한 뒤 0이 되도록 하는 값을 찾으면 됩니다.

먼저 $b$에 대해서 미분을 하겠습니다.

$$\frac{\partial{J(w,b)}}{\partial{b}} = \frac{1}{n}\sum_{i=1}^n -2(y_i - (wx_i+b)) = \frac{2}{n}\left(b + w\sum_{i=1}^n x_i -\sum_{i=1}^n y_i\right) = 0$$
위 식을 만족하는 $b$에 대해서 정리하면

$$b = \bar{y} - w\bar{x}$$
여기서  $\bar{x} = \frac{\sum_{i=1}^n x_i}{n}$ , $\bar{y} = \frac{\sum_{i=1}^n y_i}{n}$ 입니다.

이제 $w$에 대해서 미분을 하겠습니다.

$$\frac{\partial{J(w,b)}}{\partial{w}} =  \frac{1}{n}\sum_{i=1}^n -2(y_i - (wx_i+b))x_i = \frac{2}{n}\left(b\sum_{i=1}^nx_i + w\sum_{i=1}^n x_i^2 - \sum_{i=1}^n x_iy_i\right)$$

여기에 아까 구한 $b$를 대입한 후 0이 되는 $w$값을 구하하면

$$w = \frac{\sum_{i=1}^ny_i(x_i-\bar{x_i})}{\sum_{i=1}^nx_i(x_i-\bar{x_i})}$$

가 됩니다.

우리는 계산을 통해서 $w$와 $b$ 값을 구했습니다.

이제 코드를 통해서 적용해보도록 하겠습니다.

```python
w = numpy.sum(temp_anomaly*(year - year.mean())) / numpy.sum(year*(year - year.mean()))
b = a_0 = temp_anomaly.mean() - w*year.mean()

print(w)
print(b)
```

    0.01037028394347266
    -20.148685384658464

이제 그래프로 그려서 확인해보도록 하겠습니다.

```python
reg = b + w * year
```

```python
pyplot.figure(figsize=(10, 5))

pyplot.plot(year, temp_anomaly, color='#2929a3', linestyle='-', linewidth=1, alpha=0.5)
pyplot.plot(year, reg, 'k--', linewidth=2, label='Linear regression')
pyplot.xlabel('Year')
pyplot.ylabel('Land temperature anomaly [°C]')
pyplot.legend(loc='best', fontsize=15)
pyplot.grid();
```

오늘은 linear regression을 직접 만든 데이터와 실제로 있는 데이터로 진행해보았습니다.

년도에 따른 기온의 변화를 gradient descent 로 하는 방법은 내일 실습에서 추가로 알아보도록 하겠습니다.

질문 있으면 편하게 해주세요~~


# Introduction

In this notebook we demonstrate the solution of simple linear transport equations such as

$$
\begin{align}
    -\epsilon \Delta u + b \cdot \nabla u &= f \qquad x \in \Omega \\
    u &= 0 \qquad x \in \partial \Omega
\end{align}
$$

where $\epsilon > 0$ is a (small) parameter, $b$ a given vector field and $f$ a given
function. For this problem to be well posed it is required that $\nabla \cdot b = 0$.

There are two operators, the diffusion operator $-\epsilon \Delta$ which acts to smooth
out the source proportionally to $\epsilon$, and the transport operator
$b \cdot \nabla u$ which acts to transport infinitesimal volumes of $u$ in the direction
of the vector field $b$. These two operators model the physical processes of diffusion
and convection respectively.

```python
import tenfem
import tensorflow as tf
import matplotlib.pyplot as plt
from matplotlib.tri import Triangulation
from matplotlib.collections import LineCollection

tf_dtype = tf.float32

eps = 0.1 # the small constant multiplying the diffusion term
function_space_deg = 2
```

```python
mesh = tenfem.mesh.examples.circle(0.1, dtype=tf_dtype)

element = tenfem.reference_elements.TriangleElement(degree=function_space_deg, dtype=tf_dtype)

if function_space_deg == 2:
    mesh = tenfem.mesh.triangle.convert_linear_to_quadratic(mesh)
    mesh.add_matplotlib_tri()
```

```python
fig, ax = plt.subplots()
ax.triplot(mesh._triang)
ax.set_aspect('equal'); ax.axis('off')
plt.show()
```

```python
def transport_vector_field(x):
    """ Returns a vector [1, 1] at each spatial coordinate of x.

    Args:
        x: A float `Tensor` of shape `[..., 2]`.

    Returns:
        tvf: A float `Tensor` of shape `[..., 2]` giving the
          vector field at each coordinate.
    """
    return tf.ones_like(x)[..., tf.newaxis] * tf.ones([2])

# create the PDE model
poisson_layer = tenfem.layers.LinearEllipticOperator(
    diffusion_coefficient = lambda x: 0.1*tf.ones_like(x[..., 0]),
    source = lambda x: tf.ones_like(x[..., 0]),
    boundary_condition='dirichlet',
    boundary_values = lambda x: tf.zeros_like(x[..., 0])[..., tf.newaxis],
    reference_element=element)

pde_layer = tenfem.layers.LinearEllipticOperator(
    diffusion_coefficient = lambda x: 0.1*tf.ones_like(x[..., 0]),
    source = lambda x: tf.ones_like(x[..., 0]),
    transport_vector_field = transport_vector_field,
    boundary_condition='dirichlet',
    boundary_values = lambda x: tf.zeros_like(x[..., 0])[..., tf.newaxis],
    reference_element=element)

```

The solution is plotted below, next to the same solution for the model with no transport.
We can clearly see that the mass injected by the source is "pushed" in the direction of
the transport vector. We can also observe the *boundary* layer, where the relatively
large value of the of the transported mass must also rapidly decrease to zero in order
to satisfy the boundary conditions.

```python
u = pde_layer(mesh.get_tensor_repr())
u_poisson = poisson_layer(mesh.get_tensor_repr())

mtri = Triangulation(*mesh.nodes.numpy().T)

fig, axes = plt.subplots(ncols=2, figsize=(8, 4))

axes[0].tricontourf(mtri, u_poisson[:, 0], levels=20, cmap='inferno')
axes[1].tricontourf(mtri, u[:, 0], levels=20, cmap='inferno')

for ax in axes.flatten():
    ax.set_aspect('equal'); ax.axis('off')
    ax.add_collection(
        LineCollection(tf.gather(mesh.nodes, mesh.boundary_elements),
                       colors='k', label='$\partial \Omega$'))
    ax.set_xlim((-1.05, 1.05))
    ax.set_ylim((-1.05, 1.05))

plt.show()
```

```python

```


```python
# Required to load webpages
from IPython.display import IFrame
```

[Table of contents](../toc.ipynb)

# SciPy

* Scipy extends numpy with powerful modules in
  * optimization,
  * interpolation,
  * linear algebra,
  * fourier transformation,
  * signal processing,
  * image processing,
  * file input output, and many more.
* Please find here the scipy reference for a complete feature list [https://docs.scipy.org/doc/scipy/reference/](https://docs.scipy.org/doc/scipy/reference/).

We will take a look at some features of scipy in the latter. Please explore the rich content of this package later on.

## Optimization

* Scipy's optimization module provides many optimization methods like least squares, gradient methods, BFGS, global optimization, and many more.
* Please find a detailed tutorial here [https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html](https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html).
* Next, we will apply one of the optimization algorithms in a simple example.

A common function to test optimization algorithms is the Rosenbrock function for $N$ variables:

$f(\boldsymbol{x}) = \sum\limits_{i=2}^N 100 \left(x_{i+1} - x_i^2\right)^2 + \left(1 - x_i^2 \right)^2$.

The optimum is at $x_i=1$, where $f(\boldsymbol{x})=0$

```python
import numpy as np
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
from matplotlib import cm
%matplotlib inline
```

```python
def rosen(x):
    """The Rosenbrock function"""
    return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)
```

We need to generate some data in a mesh grid.

```python
X = np.arange(-2, 2, 0.2)
Y = np.arange(-2, 2, 0.2)
X, Y = np.meshgrid(X, Y)

data  = np.vstack([X.reshape(X.size), Y.reshape(Y.size)])
```

Let's evaluate the Rosenbrock function at the grid points.

```python
Z = rosen(data)
```

And we will plot the function in a 3D plot.

```python
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.plot_surface(X, Y, Z.reshape(X.shape), cmap='bwr')
ax.view_init(40, 230)
```

Now, let us check that the true minimum value is at (1, 1).

```python
rosen(np.array([1, 1]))
```

    0.0

Finally, we will call scipy optimize and find the minimum with Nelder Mead algorithm.

```python
from scipy.optimize import minimize

x0 = np.array([1.3, 0.7])
res = minimize(rosen, x0, method='nelder-mead',
               options={'xatol': 1e-8, 'disp': True})

print(res.x)
```

    Optimization terminated successfully.
             Current function value: 0.000000
             Iterations: 79
             Function evaluations: 150
    [1. 1.]

Many more optimization examples are to find in scipy optimize tutorial [https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html](https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html).

```python
IFrame(src='https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html', width=1000, height=600)
```

## Interpolation

* Interpolation of data is very often required, for instance to replace NaNs or to fill missing values in data records.
* Scipy comes with
  * 1D interpolation,
  * multivariate data interpolation
  * spline, and
  * radial basis function interpolation.
* Please find here the link to interpolation tutorials [https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html](https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html).

```python
from scipy.interpolate import interp1d

x = np.linspace(10, 20, 15)
y = np.sin(x) + np.cos(x**2 / 10)

f = interp1d(x, y, kind="linear")
f1 = interp1d(x, y, kind="cubic")
```

```python
x_fine = np.linspace(10, 20, 200)

plt.plot(x, y, 'ko',
         x_fine, f(x_fine), 'b--',
         x_fine, f1(x_fine), 'r--')
plt.legend(["Data", "Linear", "Cubic"])
plt.show()
```

## Signal processing

The signal processing module is very powerful and we will have a look at its tutorial [https://docs.scipy.org/doc/scipy/reference/tutorial/signal.html](https://docs.scipy.org/doc/scipy/reference/tutorial/signal.html) for a quick overview.

```python
IFrame(src='https://docs.scipy.org/doc/scipy/reference/tutorial/signal.html', width=1000, height=600)
```

## Linear algebra

* In addition to numpy, scipy has its own linear algebra module.
* It offers more functionality than numpy's linear algebra module and is based on BLAS/LAPACK support, which makes it faster.
* The respective tutorial is here located [https://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html](https://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html).

```python
IFrame(src='https://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html', width=1000, height=600)
```

### Total least squares as linear algebra application

We will now implement a total least squares estimator [[Markovsky2007]](./references.bib) with help of scipy's sigular value decomposition (svd). The total least squares estimator provides a solution for the errors in variables problem, where model inputs and outputs are corrupted by noise.

The model becomes
$A X \approx B$, where $A \in \mathbb{R}^{m \times n}$ and $B \in \mathbb{R}^{m \times d}$ are input and output data, and $X$ is the unknown parameter vector.

More specifically, the total least squares regression becomes
$\widehat{A}X = \widehat{B}$, $\widehat{A} := A + \Delta A$, $\widehat{B} := B + \Delta B$.

The estimator can be written as pseudo code as follows.

$C = [A B] = U \Sigma V^\top$, where $U \Sigma V^\top$ is the svd of $C$.

$V:= \left[\begin{align}V_{11} &V_{12} \\
V_{21} & V_{22}\end{align}\right]$,

$\widehat{X} = -V_{12} V_{22}^{-1}$.

In Python, the implementation could be like this function.

```python
from scipy import linalg

def tls(A, B):
    m, n = A.shape
    C = np.hstack((A, B))
    U, S, V = linalg.svd(C)
    V12 = V.T[0:n,n:]
    V22 = V.T[n:,n:]
    X = -V12 / V22
    return X
```

Now we create some data where input and output are appended with noise.

```python
A = np.random.rand(100, 2)
X = np.array([[3], [-7]])
B = A @ X

A += np.random.randn(100, 2) * 0.1
B += np.random.randn(100, 1) * 0.1
```

The total least squares solution becomes

```python
tls(A , B)
```

    array([[ 2.95496063],
           [-6.7920108 ]])

And this solution is closer to correct value $X = [3 , -7]^\top$ than ordinary least squares.

```python
linalg.solve((A.T @ A), (A.T @ B))
```

    array([[ 2.5035526 ],
           [-6.28316014]])

Finally, next function shows a "self" written least squares estimator, which uses QR decomposition and back substitution. This implementation is numerically robust in contrast to normal equations

$A ^\top A X = A^\top B$.

Please find more explanation in [[Golub2013]](./references.bib) and in section 3.11 of [[Burg2012]](./references.bib).

```python
def ls(A, B):
    Q, R = linalg.qr(A, mode="economic")
    z = Q.T @ B

    return linalg.solve_triangular(R, z)
```

```python
ls(A, B)
```

    array([[ 2.5035526 ],
           [-6.28316014]])

## Integration

* Scipy's integration can be used for general equations as well as for ordinary differential equations.
* The integration tutorial is linked here [https://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html](https://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html).

### Solving a differential equation

Here, we want to use an ode solver to simulate the differential equation (ode)

$y'' + y' + 4 y = 0$.

To evaluate this second order ode, we need to convert it into a set of first order ode. The trick is to use this substitution: $x_2 = y'$, $x_1 = y$, which yields

$\begin{align}
x'_1 &= x_2 \\
x'_2 &= -4 x_1 - x_2
\end{align}$

The implementation in Python becomes.

```python
def equation(t, x):
    """Index is -1 because Python is zero based."""
    return [x[1], -4 * x[0] - x[1]]
```

```python
from scipy.integrate import solve_ivp
```

```python
time_span = [0, 20]; init = [1, 0]
time = np.arange(0, 20, 0.01)
sol = solve_ivp(equation, time_span, init, t_eval=time)
```

```python
plt.plot(time,sol.y[0, :])
plt.plot(time, sol.y[1, :])
plt.legend(["$y'$", "$y''$"])
plt.xlabel("Time")
plt.show()
```


# 02 Notebook exercise:

Consider the following boundary value problem for the 1D wave equation:

\begin{align}
  \frac{1}{c^2}\frac{\partial^2 u}{\partial t^2}(x,t)-\frac{\partial^2 u}{\partial x^2}(x,t)&=0, \\
  u(0,t)=u(10,t)&= 0, \\
  \frac{\partial u}{\partial t}(x,0)&=0, \\
  u(x,0)&=
    \begin{cases}
      0, & 0 \leq x < 4,\\
      1, & 4 \leq x \leq 6,\\
      0, & 6 < x \leq 10.
    \end{cases}
\end{align}

The initial state is illustrated in the figure below:

The exact solution can be written as:
\begin{equation}
 u(x,t)=\sum_{n=1}^{\infty}A_{n}\sin\left(\frac{n\pi x}{10}\right)\cos\left(\frac{n\pi ct}{10}\right),
\end{equation}
where
\begin{equation}
  A_n=\frac{2}{n\pi}\left[\cos\left(\frac{4n\pi}{10}\right)-\cos\left(\frac{6n\pi}{10}\right)\right].
\end{equation}

A potential Devito implementation is outlined below:

```python
from numpy import sin, cos, pi, linspace, shape, zeros, mod

import matplotlib.pyplot as plt

# Global constants
L = 10. # Define length of domain as a global variable
N = 2000 # Number of terms in the Fourier sine series
l = 1001 # Define number of points in domain
c = 1.0 # Speed of sound

x = linspace(0, L, l) # x axis for calling u(x, t) at given t

# Calculate exact solution

def u_e(x, t):
    s = zeros(x.size)
    for i in range(1,N+1):
        s+=A_n(i)*sin(i*pi*x/L)*cos(i*pi*c*t/L)
    return s

def A_n(n):
    const = 2/n/pi*(cos(4*n*pi/L)-cos(6*n*pi/L))
    return const
```

```python
%matplotlib inline
# Plot solution at a point T in time
T = 0
plt.plot(x, u_e(x, T))
plt.xlabel('x')
plt.ylabel('u(x,0)')
plt.show()
```

```python
# Devito implimentation

time_order = 2
space_order = 2
```

```python

# Required imports
from devito import Grid, TimeFunction, Eq, solve, Operator

# Define the grid/computational domain
grid = Grid(shape=(l), extent=(L))
t_s = grid.stepping_dim

# Wavefield
u = TimeFunction(name='u', grid=grid, time_order=time_order, space_order=space_order)

dx = abs(x[1]-x[0])
# Critical time-step
dt = 0.004 # (dt < 0.42*dx/c)

# Define the equation and boundary conditions
pde = 1/c**2*u.dt2-u.dx2
eq = Eq(u.forward, solve(pde, u.forward))
bc = [Eq(u[t_s+1, 0], 0)]
bc += [Eq(u[t_s+1, -1], 0)]

# Set the initial conditions
u0 = u_e(x, 0)
u1 = u_e(x, dt)
u.data[0,:] = u0
u.data[1,:] = u1

# Create the Devito Operator
op = Operator([eq]+bc)
```

```python
# Now lets run the operator
op.apply(time_m=1, time_M=4999, dt=dt)
```

    Operator `Kernel` run in 0.01 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
                         PerfEntry(time=0.0009600000000000168, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

```python
%matplotlib inline
# Lets plot some Devito results:
plt.plot(x, u.data[mod(5000,3), :])
plt.xlabel('x')
plt.ylabel('u(x,t)')
plt.show()
```

- **How can we improve on this result?**


# CHEM 1000 - Spring 2022
Ryan Wheat, University of Pittsburgh

## Recitation

For this recitation, we'll focus on:
- Graphical Interpolation for Optimization
- Lagrange Multipliers Practice
---

The goal of Chapter 6 is to give a cursory overview of the vastly expanding field that is "optimization algorithms". All fields, in some form, deal with optimizing something with known constraints. It is to your advantage to hold onto the conceptual aspects of this content.

Please follow along with recitation and complete the following questions to gain credit for this assignment.

### Part A: Simionescu Function

Sometimes determining the analytical solution to a function is not practical. In this case, one might wish to better understand the function graphically to then interpolate the extrema. Today we will be working with another function, the Simionescu function, which is unique not for the function itself, but how the constraint is defined. Here, f(x, y) is the function and g(x, y) is the constraint.

$$
f(x, y)=0.1xy
$$

$$
g(x, y)=x^{2}+y^{2}-(1+0.2cos(8tan^{-1}(x/y)))^{2}
$$

#### Run the code in each cell and answer the three questions below to get credit for this recitation.

1. Calculate all partial derivatives.
2. Describe the constraint on the basis of symmetry.
3. Where are the maxima and minima?

```python
from sympy import init_session
init_session()
```

```python
# Calculate derivatives here with sympy. Note that arctan(x) = atan(x).

dfdx =
dfdy =
dgdx =
dgdy =
```

Use this website to plot the constraint function. Describe what you see? What symmetries do you notice? Why do you think this constraint makes optimization difficult? Think critically about how a computer might search for minima and maxima...

https://www.desmos.com/calculator

```python
# Write your description of the constraint function below.

```

We can extend our original function in three dimensions by setting it equal to z.

$$
f(x,y) = 0.1xy = z
$$

Follow along with recitation to learn how to graphically determine the maxima and minima. In the cell below, write (approximately) what values correspond to the maxima and minima.

```python
maxima =
minima =
```

### Part B: Company Startup

Let's say that you are starting a company and need to hire staff and order materials. The materials (m) you need can be ordered in bulk at 50 dollars for each crate. You need to hire staff (s) to convert the materials into some product (think assembly line) at 20 dollars per hour. After consulting an entrepeneur and an engineer, you find that similar companies have used this model:

$$
f(m,s) = 5m^{2}s+2s^{2}m
$$

#### Run the code in each cell and answer the three questions below to get credit for this recitation.

1. Your company has a startup fund of 600 dollars. Optimize your initial spending given the model above.
2. Your friend is also starting a company and has been instructed to use the same model. However, you decided to give your friend two crates of material and to transfer one of your workers to their company to start production. Modify the model with these initial conditions.
3. Your friend has applied for a startup fund and was awarded 480 dollars. Optimize spending to maximize their efficiency.

```python
# Calculate your optimal spending.

f = 5*s*m**2 + 2*m*s**2
g =
h =
```

```python
# Write your new model here...

f_new =
```

```python
# Calculate the optimal spending for your friend.

f_new =
g =
h =
```


```python
from sympy import *
from sympy.plotting import plot
import numpy as np
import matplotlib.pyplot as plt
```

```python
r, A, B = symbols('r A B')
V_LJ_AB, V_LJ_se, V_LJ_re, V = symbols('V_LJ_AB V_LJ_se V_LJ_re V', cls=Function)
```

```python
V_LJ_AB = A/r**12 - B/r**6
```

```python
V_LJ_AB
```

$\displaystyle \frac{A}{r^{12}} - \frac{B}{r^{6}}$

```python
epsilon, sigma, r_min= symbols('epsilon sigma r_min')
V_LJ_se = V_LJ_AB.subs(A,4*epsilon*sigma**12).subs(B,4*epsilon*sigma**6)
simplify(V_LJ_se)
```

$\displaystyle \frac{4 \epsilon \sigma^{6} \left(- r^{6} + \sigma^{6}\right)}{r^{12}}$

```python
V_LJ_re = V_LJ_se.subs(sigma,2**(-1/6)*r_min)
simplify(V_LJ_re)
```

$\displaystyle \frac{\epsilon r_{min}^{6} \left(- 2.0 r^{6} + 1.0 r_{min}^{6}\right)}{r^{12}}$

```python
p1 = plot(V_LJ_re.subs(epsilon,0.5).subs(r_min,2), (r,1.75,4.5), show=False)
p1.show()
```

```python
dat = np.loadtxt('rerun-XXX/rerun-XXX.dat', dtype={'names': ('step', 'Epot', 'dist'), 'formats': ('i4', 'f8', 'f8')})
V = V_LJ_re.subs(epsilon,0.5).subs(r_min,2)
Vy = lambdify(r, V)(dat['dist'])
plt.plot(dat['dist'], dat['Epot'], 'k*')
plt.plot(dat['dist'], Vy)
plt.show()
```

```python

```


\begin{align}
\alpha & = \frac{1}{\sqrt{1+2M/r}} & \beta^i & = \frac{2Mx^i}{r+2M} & \gamma_{ij} & = \delta_{ij} + \frac{2Mx_{i}x_{j}}{r^3} & H & = M/r
\end{align}

\begin{align}
K & = 2\alpha^3(1+H)x_{i}\partial_{i}H + 2\alpha H \partial_{i}(x^{i}/r) & \Gamma^k_{ij} & = 0.5 * ( \gamma^{kl}(\partial_{i}\gamma_{lj})+\partial_{j}\gamma_{li}-\partial_{l}\gamma_{ij} )
\end{align}

\begin{align}
\partial_{t}\varphi & = \alpha\Pi + \beta^i\partial_{i}\varphi \\
\partial_{t}\Pi & = \alpha\partial_{i}\partial^{i}\varphi + \alpha (K\Pi - \gamma^{ij}\Gamma^k_{ij}\partial_{k}\varphi+\frac{dV}{d\varphi}) + \partial_{i}\varphi \partial^{i}\alpha +\beta^i\partial_{i}\Pi
\end{align}

## That is

\begin{align}
\partial_{t}\varphi & = \alpha\Pi + \beta^x\partial_{x}\varphi + \beta^y\partial_{y}\varphi + \beta^z\partial_{z}\varphi\\
& = \alpha\Pi + \frac{2Mx}{r+2M}\partial_{x}\varphi + \frac{2My}{r+2M}\partial_{y}\varphi + \frac{2Mz}{r+2M}\partial_{z}\varphi
\end{align}

x component of $\partial_{t}\Pi$ :
\begin{align}
\partial_{t}\Pi & = \alpha\partial_{i}\partial^{i}\varphi
+ \alpha (K\Pi - \gamma^{ij}\Gamma^k_{ij}\partial_{k}\varphi+\frac{dV}{d\varphi})
+ \partial_{i}\varphi \partial^{i}\alpha +\beta^i\partial_{i}\Pi \\
& = {\alpha\partial_{x}\partial^{x}\varphi
+ \alpha [(2\alpha^3(1+H)x\partial_{x}H + 2\alpha H \partial_{x}(x/r))\Pi - \gamma^{xj}\Gamma^k_{xj}\partial_{k}\varphi+\frac{dV}{d\varphi}]
+ \partial_{x}\varphi \partial^{x}\alpha +\beta^x\partial_{x}\Pi}
\end{align}

where $\gamma^{xj}\Gamma^k_{xj}$ is :
\begin{align}
\gamma^{xj}\Gamma^k_{xj} & = \gamma^{xx} \Gamma^k_{xx} + \gamma^{xy} \Gamma^k_{xy} + \gamma^{xz} \Gamma^k_{xz}\\
& = \gamma^{xx} \Gamma^x_{xx} + \gamma^{xx} \Gamma^y_{xx} + \gamma^{xx} \Gamma^z_{xx} + \gamma^{xy} \Gamma^x_{xy} + \gamma^{xy} \Gamma^y_{xy} + \gamma^{xy} \Gamma^z_{xy} + \gamma^{xz} \Gamma^x_{xz} + \gamma^{xz} \Gamma^y_{xz} + \gamma^{xz} \Gamma^z_{xz}
\end{align}

Since $$\Gamma^k_{ij}  = 0.5 * ( \gamma^{kl}(\partial_{i}\gamma_{lj})+\partial_{j}\gamma_{li}-\partial_{l}\gamma_{ij} )$$
Then that is:
\begin{align}
\Gamma^x_{xx} = 0.5 * ( \gamma^{xl}(\partial_{x}\gamma_{lx})+\partial_{x}\gamma_{lx}-\partial_{l}\gamma_{xx} )
\end{align}

## Scalar Wave

The Scalar Wave Equations are:
\begin{align}
\partial_t u &= v \\
\partial_t v &= c^2 \nabla^2 u.
\end{align}

The second order derivative term is:
\begin{equation}
\left[\partial_x^2 u(t,x)\right]_j = \frac{1}{(\Delta x)^2}
\left(
-\frac{1}{12} \left(u_{j+2} + u_{j-2}\right)
+ \frac{4}{3}  \left(u_{j+1} + u_{j-1}\right)
- \frac{5}{2} u_j \right)
+ \mathcal{O}\left((\Delta x)^4\right).
\end{equation}

## In our case

$$(\partial_x u)_0 \approx \frac{1}{(\Delta x)} \times \left(\frac{1}{12}( u_{-2} - u_{2}) + \frac{2}{3}( -u_{-1} + u_{1})\right)$$
\begin{equation}
\left[\partial_x\partial_y u(t,x,y)\right]_j = \frac{1}{(\Delta x)(\Delta y)}
\left(
-\frac{1}{12} \left(u_{j+2} + u_{j-2}\right)
+ \frac{4}{3}  \left(u_{j+1} + u_{j-1}\right)
- \frac{5}{2} u_j \right)
+ \mathcal{O}\left((\Delta x\Delta y)^2\right).
\end{equation} (wrong!!)

The inverse of $\gamma$
\begin{align}
\gamma_{ij} & = \delta_{ij} + \frac{2Mx_{i}x_{j}}{r^3}
\end{align}

$$ \gamma = \begin{pmatrix}
1 + \frac{2Mxx}{r^3} &  \frac{2Mxy}{r^3} &  \frac{2Mxz}{r^3} \\
\frac{2Myx}{r^3} & 1 + \frac{2Myy}{r^3} & \frac{2Myz}{r^3} \\
\frac{2Mzx}{r^3} & \frac{2Mzy}{r^3} & 1 + \frac{2Mzz}{r^3}
\end{pmatrix} $$

So the inverse is:
$$ \gamma^{-1} = \frac{r}{r + 2M} \begin{pmatrix}
1 + \frac{2Myy}{r^3} + \frac{2Mzz}{r^3} &  -\frac{2Myx}{r^3} &  -\frac{2Mzx}{r^3} \\
-\frac{2Mxy}{r^3} & 1 + \frac{2Mxx}{r^3} + \frac{2Mzz}{r^3} & \frac{2Mzy}{r^3} \\
-\frac{2Mxz}{r^3} & -\frac{2Myz}{r^3} & 1 + \frac{2Mxx}{r^3} + \frac{2Myy}{r^3}
\end{pmatrix} $$

$$ \gamma^{-1}\gamma = \frac{r}{r + 2M} \begin{pmatrix}
1 + \frac{2M}{r} & 0 & 0\\
0 & 1 + \frac{2M}{r} & 0\\
0 & 0 & 1 + \frac{2M}{r}
\end{pmatrix} = 1$$

$$K  = 2\alpha^3(1+H)x_{i}\partial_{i}H + 2\alpha H \partial_{i}(x^{i}/r) $$

\begin{align}
\partial_{x} H & = -\frac{Mx}{(x^2+y^2+z^2)^{3/2}} = -\frac{Mx}{r^3} & \partial_{y} H & = -\frac{My}{(x^2+y^2+z^2)^{3/2}} = -\frac{My}{r^3} & \partial_{z} H & = -\frac{Mz}{(x^2+y^2+z^2)^{3/2}} = -\frac{Mz}{r^3}
\end{align}

\begin{align}
\partial_{x}(\frac{x}{r}) & = \dfrac{z^2+y^2}{\left(x^2+z^2+y^2\right)^\frac{3}{2}} = \frac{z^2+y^2}{r^3} & \partial_{y}(\frac{y}{r}) & = \dfrac{z^2+x^2}{\left(x^2+z^2+y^2\right)^\frac{3}{2}} = \frac{z^2+x^2}{r^3} & \partial_{z}(\frac{z}{r}) & = \dfrac{x^2+y^2}{\left(x^2+z^2+y^2\right)^\frac{3}{2}} = \frac{x^2+y^2}{r^3}
\end{align}

$$ K = -2\alpha^3(1+H)H + \frac{4\alpha H}{r}$$

## First component
$$\alpha\partial_{i}\partial^{i}\varphi = \alpha\gamma^{-1}\partial_{i}\partial_{j}\varphi $$

## Second component
$$\alpha (K\Pi - \gamma^{ij}\Gamma^k_{ij}\partial_{k}\varphi)$$

Where
$$ \partial_{x}\gamma = \begin{pmatrix}
\dfrac{4mx}{r^3}-\dfrac{6mx^3}{r^5} & \dfrac{2my}{r^3}-\dfrac{6myx^2}{r^5} & \dfrac{2mz}{r^3}-\dfrac{6mzx^2}{r^5}\\
\dfrac{2my}{r^3}-\dfrac{6myx^2}{r^5} & -\dfrac{6my^2x}{r^5} & -\dfrac{6myzx}{r^5} \\
\dfrac{2mz}{r^3}-\dfrac{6mzx^2}{r^5} & -\dfrac{6myzx}{r^5} & -\dfrac{6mz^2x}{r^5}
\end{pmatrix} $$

Simplify
$$ \partial_{x}\gamma = \frac{2M}{r^5}\begin{pmatrix}
2xr^2-3x^3 & yr^2-3yx^2 & zr^2-3zx^2\\
yr^2-3yx^2 & -3y^2x & -3yzx \\
zr^2-3zx^2 & -3yzx & -3z^2x
\end{pmatrix} $$

And
$$ \partial_{y}\gamma = \frac{2M}{r^5}\begin{pmatrix}
-3x^2y & xr^2-3xy^2 & -3xyz\\
xr^2-3xy^2 & 2yr^2 - 3y^3 & zr^2 - 3zy^2 \\
-3xyz & zr^2 - 3zy^2 & -3z^2y
\end{pmatrix} $$

$$ \partial_{z}\gamma = \frac{2M}{r^5}\begin{pmatrix}
-3x^2z & -3xyz & xr^2-3xz^2\\
-3xyz & -3y^2z & yr^2-3yz^2 \\
xr^2-3xz^2 & yr^2-3yz^2 & 2zr^2-3z^3
\end{pmatrix} $$

## Third component
$$ \partial_{i}\varphi \partial^{i}\alpha = \gamma^{-1} \partial_{i}\varphi \partial_{j}\alpha $$

$$\partial_{x}\alpha = \dfrac{mx}{\left(x^2+z^2+y^2\right)^\frac{3}{4}\left(\sqrt{x^2+z^2+y^2}+2m\right)^\frac{3}{2}} = \frac{mx}{r^\frac{3}{2} (r+2m)^\frac{3}{2}} $$

```python
%matplotlib inline
import matplotlib.pyplot as plt
import mpmath as mp
import csv

def file_reader(filename):
    with open(filename) as file:
        reader = csv.reader(file, delimiter=" ")
        data  = list(zip(*reader))
        data0 = []
        data1 = []
        data2 = []
        data3 = []
        for i in range(len(data[0])):
            data0.append(float(data[0][i]))
            data1.append(float(data[1][i]))
            data2.append(float(data[2][i]))
            data3.append(float(data[3][i]))
        return data0,data1,data2,data3
```

```python
from mpl_toolkits import mplot3d
from matplotlib.pyplot import savefig
from matplotlib import cm
import numpy as np

itermax = 10
i = 0
while i <= itermax:
    filename = 'ScalarHair_resolution_100x100x100__iter_'+str(i)
    x_iter_0,y_iter_0,num_0,exact_0 = file_reader(filename+".txt")
    fig = plt.figure()
    ax = plt.axes(projection='3d')
    ax.set_xlabel('x_coord')
    ax.set_ylabel('y_coord')
    ax.set_zlabel('Numerical Soln.(log)')
    ax.set_title('ScalarHair_resolution_100x100x100')
    ax.scatter3D(x_iter_0,y_iter_0,np.log10(num_0), c=num_0, cmap='viridis',s=1)
#    ax.plot_trisurf(x_iter_0,y_iter_0,np.log10(num_0),cmap=cm.jet, linewidth=0)
    savefig(filename+".png",dpi=450)
    plt.close(fig)
    i+=1
```

    D:\Anaconda3\lib\site-packages\ipykernel_launcher.py:17: RuntimeWarning: invalid value encountered in log10

```python
import matplotlib.image as mgimg
from matplotlib import animation

fig = plt.figure(frameon=False)
ax = fig.add_axes([0, 0, 1, 1])
ax.axis('off')

myimages = []

i = 0
while i <= itermax:
    filename = 'ScalarHair_resolution_100x100x100__iter_'+str(i)
    img = mgimg.imread(filename+".png")
    imgplot = plt.imshow(img)
    myimages.append([imgplot])
    i += 1

ani = animation.ArtistAnimation(fig, myimages, interval=100,  repeat_delay=1000)
plt.close()
ani.save("ScalarHair_resolution_100x100x100_Gauss.mp4", fps=5,dpi=450)
```

```python
from IPython.display import HTML
#the filename needed to be changed manually

HTML("""

""")
```

```python
from IPython.display import HTML
#the filename needed to be changed manually

HTML("""

""")
```

$\partial_t B^i = \beta^j \bar{D}_j B^i
                       + \frac{3}{4} ( \partial_t \bar{\Lambda}^{i} - \beta^j \bar{D}_j \bar{\Lambda}^{i} )
                       - \eta B^{i}
                         = \beta^j B^i_{,j} + \beta^j \bar{\Gamma}^i_{mj} B^m
                       + \frac{3}{4}[ \partial_t \bar{\Lambda}^{i}
                                    - \beta^j (\bar{\Lambda}^i_{,j} + \bar{\Gamma}^i_{mj} \bar{\Lambda}^m)]
                       - \eta B^{i}$

```python

```


# Pré-Introdução à teoria das vibrações

## Aula 2 - Elementos de mola

### *Daniel Barbosa Mapurunga Matos (Aluno PPGEC/UFRGS)*

```python
import numpy as np
import matplotlib.pyplot as plt
```

## 1. Equação da mola
A mola é responsável por representar as forças restitutivas do sistema, sendo estas causadas por **deformações** em seu comprimento

Também chamada de força
restitutiva , a força
elástica atua contra a direção do movimento e tem
seu módulo calculado por:

\begin{align}
F_{el} = Kx
\end{align}

Onde K é a **rigidez** da mola ( constante elástica) e x a deformação. É válido lembrar que a energia potencial elástica para uma mola linear pode ser calculada por:

\begin{align}
U =\frac{ Kx^2}{2}
\end{align}

## 2. Associação de molas em série
Neste tópico, criaremos uma função para calcular a **rigidez equivalente** para molas em série. Relembrando a teoria, molas em série são aquelas que estão sujeitas a mesma força, podendo apresentar deformações iguais ou diferentes. A figura abaixo ilustra esta associação.

Chamando de $K_1$ e $K_2$ a rigidez de cada mola, é possível deduzir a equação da mola equivalente.

\begin{align}
F_1 &= K_1x_1\\
F_2 &= K_2x_2 \\
F &= K_{eq} x_{eq}\\
F &= F_1 = F_2
\end{align}
Igualando as forças a força total que atua no sistema, é possível escreve:

\begin{align}
F &= F_1 = F_2 \\
x_{eq} &= x_1+x_2 \\
x_{eq} &= \frac{F_1}{K_1} + \frac{F_2}{K_2}
\end{align}
Agora, substituindo as forças por $F$, pode-se escrever que:

\begin{align}
\frac{F}{K_{eq}} = \frac{F}{K_1} + \frac{F}{K_2}
\end{align}
Sendo assim, chega-se a seguinte expressão genérica:

\begin{align}
\frac{1}{K_{eq}} = \frac{1}{K_1} + \frac{1}{K_2}+... + \frac{1}{K_n}
\end{align}

Agora, monta-se o script:

```python
def serie(x):                       #x será uma lista com a rigidez de cada mola
    keq_inv = 0
    for i in range(len(x)):        # loop para fazer a soma dos inversos
        keq_inv += 1/x[i]
    keq = 1/keq_inv                # invertendo para resultar na rigidez equivalente.

    return keq

x = [2,6,16]
keq = serie(x)

print(keq)
```

    1.3714285714285714

## 3. Associação de molas em paralelo
Para molas em paralelo, será feito o mesmo procedimento.

De acordo com o exposto, é possível visualisar agora que $ F = F_1 + F_2$ e $x_{eq} = x_1 = x_2$, sendo fácil deduzir que:

\begin{align}
K_{eq}x_{eq} &= K_{1}x_{1} + K_{2}x_{2}\\
\end{align}
cancelando as deformações, é possível escrever:

\begin{align}
K_{eq} &= K_{1}+ K_{2}+...+ K_N
\end{align}

```python
def paralelo(x):                       #x será uma lista com a rigidez de cada mola
    keq = np.sum(x)             # invertendo para resultar na rigidez equivalente.

    return keq

x = [2,6,16]
keq = paralelo(x)

print(keq)
```

    24

Existem outros tipos de associação de molas, porém os mais importantes estão expostos neste notebook. A associação de amortecedores é exatamente igual à de molas, sendo possível utilizar as mesmas expressões.

## 4. Exercício
considerando $ K1 = 5 \frac{N}{m}$ e $K2= 10 \frac{N}{m}$, calcular a rigidez equivalente.

Nota-se que a associação é mista, sendo necessário resolver por etapas.

Primeiramente, faz-se a associação dos sistemas que estão em paralelo ( cada um em um "andar")

```python
keq1 = paralelo([10,10,10])
keq2 = paralelo([5,5])
```

E agora associa-se em série as duas resultantes já encontradas:

```python
keq = serie([keq1,keq2])
print("a rigidez equivalente vale {}".format(keq))
```

    a rigidez equivalente vale 7.5

```python

```