## Task 1

Вектор – это частный случай матрицы 1хN и Nх1. Повторите материал для векторов, уделяя особое внимание умножению A∙B.
Вычислите, по возможности не используя программирование: $(5Е)^{–1}$, где Е – единичная матрица размера 5х5

```python
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
```

```python
E = np.identity(5)
E
```

```python
A = 5*E
A
```

```python
#Определитель диагональной матрицы равен произведению элементов стоящих на главной диагонали
D = 5**5
D
```

    3125

```python
A11 = 5**4
A11
```

    625

A11 = Ann = 625

```python
A_1 = np.identity(5)*(A11/D)
A_1
```

    array([[0.2, 0. , 0. , 0. , 0. ],
           [0. , 0.2, 0. , 0. , 0. ],
           [0. , 0. , 0.2, 0. , 0. ],
           [0. , 0. , 0. , 0.2, 0. ],
           [0. , 0. , 0. , 0. , 0.2]])

Проверим: $A \cdot A^{-1} = E$

```python
np.dot(A_1, A)
```

    array([[1., 0., 0., 0., 0.],
           [0., 1., 0., 0., 0.],
           [0., 0., 1., 0., 0.],
           [0., 0., 0., 1., 0.],
           [0., 0., 0., 0., 1.]])

## Task 2

Вычислите определитель:

$
\begin{equation}
\begin{vmatrix}
1 & 2 & 3 \\
4 & 0 & 6 \\
7 & 8 & 9
\end{vmatrix}
\end{equation}
$

$
\Delta = 1 \cdot 0 \cdot 9 + 2 \cdot 6 \cdot 7 + 3 \cdot 8 \cdot 4 - 7 \cdot 0 \cdot 3 - 6 \cdot 8 \cdot 1 - 9 \cdot 4 \cdot 2
$

$\Delta = 60$

## Task 3

Вычислите матрицу, обратную данной:

$
\begin{equation}
\begin{vmatrix}
1 & 2 & 3 \\
4 & 0 & 6 \\
7 & 8 & 9
\end{vmatrix}
\end{equation}
$

```python
A = np.matrix([[1, 2,3], [4,0,6],[7,8,9]])
A
```

    matrix([[1, 2, 3],
            [4, 0, 6],
            [7, 8, 9]])

```python
A11 = -6*8
A12 = -(4*9-6*7)
A13 = 4*8
A21 = -(2*9 - 3*8)
A22 = 1*9 - 3*7
A23 = -(1*8 - 2*7)
A31 = 2*6
A32 = -(1*6 - 3*4)
A33 = -2*4
```

```python
A_1 = np.matrix([[A11, A12, A13], [A21, A22, A23], [A31, A32, A33]])/60
A_inv = A_1.T
A_inv
```

    matrix([[-0.8       ,  0.1       ,  0.2       ],
            [ 0.1       , -0.2       ,  0.1       ],
            [ 0.53333333,  0.1       , -0.13333333]])

```python
np.linalg.inv(A)
```

    matrix([[-0.8       ,  0.1       ,  0.2       ],
            [ 0.1       , -0.2       ,  0.1       ],
            [ 0.53333333,  0.1       , -0.13333333]])

Проверим:

```python
np.dot(A, A_inv)
```

    matrix([[ 1.00000000e+00, -2.77555756e-17,  2.77555756e-17],
            [-2.22044605e-16,  1.00000000e+00,  5.55111512e-17],
            [-7.77156117e-16,  2.77555756e-17,  1.00000000e+00]])

```python
np.dot(A_inv, A)
```

    matrix([[ 1.00000000e+00,  0.00000000e+00, -1.66533454e-16],
            [-2.77555756e-17,  1.00000000e+00, -8.32667268e-17],
            [ 2.77555756e-17,  0.00000000e+00,  1.00000000e+00]])

## Task 4

Приведите пример матрицы 4х4, ранг которой равен 1

```python
a = np.matrix([[1,2,3,4], [2,4,6,8], [3,6,9,12], [4,8,12,16]])
a
```

    matrix([[ 1,  2,  3,  4],
            [ 2,  4,  6,  8],
            [ 3,  6,  9, 12],
            [ 4,  8, 12, 16]])

```python
#np.ndim(a)
np.linalg.matrix_rank(a)
```

    1

## Task 5

Вычислите скалярное произведение двух векторов:
(1, 5)  и  (2, 8)

```python
a = np.array([1,5])
b = np.array([2,8])
X, Y = np.array([0, 0]), np.array([0, 0])
U, V = np.array([a[0], b[0]]), np.array([a[1], b[1]])
plt.quiver(X, Y, U, V, angles='xy', scale_units = 'xy', scale=1)
plt.xlim(-1, 3)
plt.ylim(-1, 9)
plt.grid()
plt.show()
```

```python
s = 2 + 5*8
s
```

    42

## Task 6

Вычислите смешанное произведение трех векторов:
(1, 5, 0), (2, 8, 7) и (7, 1.5, 3)

```python
a = np.array([1,5,0])
b = np.array([2,8,7])
c = np.array([7,1.5,3])
```

$\vec a \,x \, \vec b=
\begin{vmatrix}
i & j & k \\
1 & 5 & 0 \\
2 & 8 & 7
\end{vmatrix}
$

```python
ab = np.array([(5*7), -7, 8-10])
ab
```

    array([35, -7, -2])

```python
v = np.cross(a, b)
v
```

    array([35, -7, -2])

```python
vc = 35*7 - 7*1.5 - 2*3
vc
```

    228.5

```python
np.inner(v, c)
```

    228.5

## Task 7

Решите линейную систему:

$
\begin{bmatrix}
1 & 2 & 3 \\
4 & 0 & 6 \\
7 & 8 & 9
\end{bmatrix} \cdot X =
\begin{bmatrix}
12 \\ 2 \\ 1
\end{bmatrix}
$

```python
A = np.matrix([[1, 2, 3], [4, 0, 6], [7, 8, 9]])
A
```

    matrix([[1, 2, 3],
            [4, 0, 6],
            [7, 8, 9]])

```python
B = np.matrix([[12], [2], [1]])
B
```

    matrix([[12],
            [ 2],
            [ 1]])

```python
#np.linalg.solve(A, B)
X = np.dot(np.linalg.inv(A), B)
X
```

    matrix([[-9.2       ],
            [ 0.9       ],
            [ 6.46666667]])

## Task 8

Найдите псевдорешение:

$
x + 2y - z = 1 \\
3x - 4y + 0z = 7 \\
8x - 5y + 2z = 12 \\
2x + 0y - 5z = 7 \\
11x + 4y - 7z = 15
$

```python
A = np.matrix([[1, 2, -1], [3, -4, 0], [8, -5, 2], [2, 0, -5], [11, 4, -7]])
A
```

    matrix([[ 1,  2, -1],
            [ 3, -4,  0],
            [ 8, -5,  2],
            [ 2,  0, -5],
            [11,  4, -7]])

```python
B = np.matrix([1, 7, 12, 7, 15]).T
B
```

    matrix([[ 1],
            [ 7],
            [12],
            [ 7],
            [15]])

```python
X, residuals, rnk, s =  np.linalg.lstsq(A, B, rcond=None)
```

```python
X
```

    matrix([[ 1.13919353],
            [-0.90498444],
            [-0.9009803 ]])

```python
np.dot(A, X)
```

    matrix([[ 0.23020495],
            [ 7.03751834],
            [11.83650981],
            [ 6.78328855],
            [15.21805313]])

## Task 9

Сколько решений имеет линейная система:

$
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9
\end{bmatrix} \cdot X =
\begin{bmatrix}
12 \\ 2 \\ 1
\end{bmatrix}
$

```python
A = np.matrix([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
B = np.matrix([[12], [2], [1]])
```

```python
np.linalg.det(A)
```

    -9.51619735392994e-16

Определитель равен нулю, решений не имеет

```python
from pylab import *
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
```

```python
fig = figure()
ax = Axes3D(fig)
X = np.arange(-5, 5, 0.5)
Y = np.arange(-5, 5, 0.5)
X, Y = np.meshgrid(X, Y)
Z1 = 4 - 2/3*Y - 1/3*X
Z2 = 2/6 - 2/3*X - 5/6*Y
Z3 = 1/9 - 7/9*X - 8/9*Y
ax.plot_wireframe(X, Y, Z1, color='red')
ax.plot_wireframe(X, Y, Z2, color='green')
ax.plot_wireframe(X, Y, Z3, color='blue')
show()
```

```python
fig = figure()
ax = Axes3D(fig)
X = np.arange(-5, 5, 0.5)
Y = np.arange(-5, 5, 0.5)
X, Y = np.meshgrid(X, Y)
Z1 = 0 - 2/3*Y - 1/3*X
Z2 = 0/6 - 2/3*X - 5/6*Y
Z3 = 0/9 - 7/9*X - 8/9*Y
ax.plot_surface(X, Y, Z1, color='red')
ax.plot_surface(X, Y, Z2, color='green')
ax.plot_surface(X, Y, Z3, color='blue')
show()
```

```python
A = np.matrix([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
B = np.matrix([[0], [0], [0]])
```

Чтобы система стала совместной изменим вектор B на [0,0,0]. В таком случае система будет иметь тривиальное решение [0,0,0]

## Task 10

Вычислите LU-разложение матрицы:

$
\begin{vmatrix}
1 & 2 & 3 \\
2 & 16 & 21 \\
4 & 28 & 73
\end{vmatrix}
$

После этого придумайте вектор правых частей и решите полученную линейную систему трех уравнений с данной матрицей.

```python
import scipy
import scipy.linalg
```

```python
A = np.matrix([[1, 2, 3], [2, 16, 21], [4, 28, 73]])
```

```python
P, L, U = scipy.linalg.lu(A)
```

```python
print('P\n',P,'\nL\n', L, '\nU\n', U)
```

    P
     [[0. 1. 0.]
     [0. 0. 1.]
     [1. 0. 0.]]
    L
     [[ 1.    0.    0.  ]
     [ 0.25  1.    0.  ]
     [ 0.5  -0.4   1.  ]]
    U
     [[  4.    28.    73.  ]
     [  0.    -5.   -15.25]
     [  0.     0.   -21.6 ]]

```python
np.dot(P.T,A)-np.dot(L,U)
```

    matrix([[0., 0., 0.],
            [0., 0., 0.],
            [0., 0., 0.]])

```python
B = np.matrix([1,2,3]).T
B
```

    matrix([[1],
            [2],
            [3]])

```python
Y = np.dot(np.linalg.inv(L), B)
Y
```

    matrix([[1.  ],
            [1.75],
            [3.2 ]])

```python
X = np.dot(np.linalg.inv(U), Y)
X
```

    matrix([[ 2.24074074],
            [ 0.10185185],
            [-0.14814815]])

Проверим:

```python
np.dot(A, X)
```

    matrix([[2.],
            [3.],
            [1.]])

Значения совпали но строки почему-то сместились... `¯\_(ツ)_/¯`

## Task 11

 Найдите нормальное псевдорешение недоопределенной системы:

$x + 2y - z = 1 \\ 8x - 5y + 2z = 12$

```python
fig = figure()
ax = Axes3D(fig)
X = np.arange(-5, 5, 0.5)
Y = np.arange(-5, 5, 0.5)
X, Y = np.meshgrid(X, Y)
Z1 = X + 2*Y - 1
Z2 = 6 - 4*X + 5/2*Y
ax.plot_surface(X, Y, Z1, color='blue')
ax.plot_surface(X, Y, Z2, color='green')
show()
```

```python
A = np.matrix([[1, 2, -1], [8, -5, 2]])
B = np.matrix([1, 12]).T
```

```python
X, res, r, s = np.linalg.lstsq(A,B, rcond=None)
```

```python
np.dot(A,X)
```

    matrix([[ 1.],
            [12.]])

```python
# минимум в точке
X
```

    matrix([[ 1.38191882],
            [-0.18081181],
            [ 0.0202952 ]])

## Task 12

Найдите одно из псевдорешений вырожденной системы:

$
\begin{bmatrix}
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9
\end{bmatrix} \cdot X =
\begin{bmatrix}
2 \\ 5 \\ 11
\end{bmatrix}
$

```python
A = np.matrix([[1,2,3],[4,5,6],[7,8,9]])
B = np.matrix([2,5,11]).T
```

```python
np.linalg.det(A)
```

    -9.51619735392994e-16

```python
X, res, r, s = np.linalg.lstsq(A,B, rcond=None)
```

```python
X
```

    matrix([[ 1.25],
            [ 0.5 ],
            [-0.25]])


# Control based on León paper

A. E. Leon and J. A. Solsona, "Performance Improvement of Full-Converter Wind Turbines Under Distorted Conditions," in IEEE Transactions on Sustainable Energy, vol. 4, no. 3, pp. 652-660, July 2013, doi: 10.1109/TSTE.2013.2239317.

```python
%matplotlib widget
```

```python
import numpy as np
import sympy as sym
import matplotlib.pyplot as plt
import scipy.signal as sctrl
import pydae.ssa as ssa
from sympy.physics.quantum import TensorProduct
```

### Plant model

```python
Δt = 50e-6
R_t =  0.039269908169872414
L_t =  0.00125
C_m =  4e-06
G_d =  1.0
R_s =  0.039269908169872414
L_s =  0.00125

A = np.array([
    [-R_t/L_t,       0,   -1/L_t,        0,        0,        0],
    [       0,-R_t/L_t,        0,   -1/L_t,        0,        0],
    [   1/C_m,       0, -G_d/C_m,        0,   -1/C_m,        0],
    [       0,   1/C_m,        0, -G_d/C_m,        0,   -1/C_m],
    [       0,       0,    1/L_s,        0, -R_s/L_s,        0],
    [       0,       0,        0,    1/L_s,        0, -R_s/L_s],
    ])

B = np.array([
    [ 1/L_t,        0],
    [       0,  1/L_t],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    ])

B_g = np.array([
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [-1/L_s,        0],
    [       0, -1/L_s],
    ])

C_c = np.array([
    [ 0, 0, 0, 0, 1, 0],
    [ 0, 0, 0, 0, 0, 1],
    ])

D_c = np.array([
    [ 0, 0],
    [ 0, 0],
    ])

C_o = np.array([
    [ 0, 0, 1, 0, 0, 0],
    [ 0, 0, 0, 1, 0, 0],
    [ 0, 0, 0, 0, 1, 0],
    [ 0, 0, 0, 0, 0, 1],
    ])

D_o = np.array([
    [ 0, 0],
    [ 0, 0],
    ])

# plant discretization
A_d,B_d,C_d,D_d,Dt = sctrl.cont2discrete((A,B,C_c,D_c),Δt,method='zoh')
A_,B_gd,C_,D_,Dt   = sctrl.cont2discrete((A,B_g,C_c,D_c),Δt,method='zoh')
A_,B_,C_o,D_o,Dt   = sctrl.cont2discrete((A,B,C_o,D_o),Δt,method='zoh')

```

## Park aplication

```python
omega_g,t_k,Δt_sym = sym.symbols('omega_g,t_k,Δt_sym')
theta_k = omega_g*t_k
theta_kp1 = omega_g*(t_k + Δt_sym)

P_k =  sym.Matrix([
                [   sym.cos(theta_k), -sym.sin(theta_k)],
                [  -sym.sin(theta_k), -sym.cos(theta_k)],
               ])

P_kp1 =  sym.Matrix([
                [   sym.cos(theta_kp1), -sym.sin(theta_kp1)],
                [  -sym.sin(theta_kp1), -sym.cos(theta_kp1)],
               ])

W_sym = sym.simplify(P_kp1 @ P_k.inv())

m2 = sym.Matrix([[1,0,0],[0,1,0],[0,0,1]])
P_kp1_3 = TensorProduct(m2, P_kp1)
P_ki_3  = TensorProduct(m2, P_k.inv())

A_b_sym   = sym.simplify(P_kp1_3 @ A_d  @ P_ki_3 )
B_b_sym   = sym.simplify(P_kp1_3 @ B_d  @ P_k.inv() )
B_g_bsym =  sym.simplify(P_kp1_3 @ B_gd @ P_k.inv() )
```

```python
A_b_eval = sym.lambdify([omega_g,Δt_sym], A_b_sym)
B_b_eval = sym.lambdify([omega_g,Δt_sym], B_b_sym)
B_g_b_eval = sym.lambdify([omega_g,Δt_sym], B_g_bsym)
W_eval  = sym.lambdify([omega_g,Δt_sym], W_sym)
```

```python
omega_b = 2*np.pi*50
A_b = A_b_eval(omega_b,Δt)
B_b = B_b_eval(omega_b,Δt)
B_g_b = B_g_b_eval(omega_b,Δt)
W =  W_eval(omega_b,Δ𝑡)
```

## Controller

```python
# Controller ##################################################################################
N_x_c,N_u_d = B_b.shape
N_z_c,N_x_c = C_c.shape

O_ux = np.zeros((N_u_d,N_x_c))
O_xu = np.zeros((N_x_c,N_u_d))
O_uu = np.zeros((N_u_d,N_u_d))
I_uu = np.eye(N_u_d)

# discretized plant:
# Δx_d = A_d*Δx_d + B_d*Δu_d
# Δz_c = C_c*Δx_d + D_c*Δu_d

# dinamic extension:
# Δx_d = A_d*Δx_d + B_d*Δu_d
# Δx_i = Δx_i + Δt*(Δz_c-Δz_c_ref) = Δx_i + Δt*C_c*Δx_d - Dt*Δz_c_ref
# Δz_c = z_c - z_c_0
# Δz_c_ref = z_c_ref - z_c_0
# (Δz_c-Δz_c_ref) = z_c - z_c_ref

A_e = np.block([
                [    A_b,    B_b, O_xu],    # Δx_d
                [   O_ux,   O_uu, O_uu],    # Δx_r
                [ Δt*C_d,   O_uu, I_uu],    # Δx_i
               ])

B_e = np.block([
                [   O_xu],
                [      W],
                [   O_uu],
               ])

# weighting matrices
Q_c = np.eye(A_e.shape[0])
Q_c[-1,-1] = 1e8
Q_c[-2,-2] = 1e8

R_c = np.eye(B_e.shape[1])*10

K_c,S_c,E_c = ssa.dlqr(A_e,B_e,Q_c,R_c)

E_cont = np.log(E_c)/Δt
```

## Observer

```python
N_z_o = C_o.shape[0]

Q_o = np.eye(A_d.shape[0])

R_o = np.diag([1]*N_z_o)
K_o_T,S_o,E_o = ssa.dlqr(A_d.T,C_o.T,Q_o,R_o)
K_o = K_o_T.T

print('damp_ctrl',-E_c.real/np.abs(E_c))
print('damp_obs',-E_o.real/np.abs(E_o))
```

    damp_ctrl [-0.99989883 -0.99989883 -0.99881325 -0.99881325 -0.9995029  -0.9995029
     -0.99987663 -0.99987663 -1.         -1.        ]
    damp_obs [-1. -1. -1. -1. -1. -1.]

## Simulink

```python
# Control without observer Du_r = -K_c*Dx_e
x_d_1,x_d_2,x_d_3,x_d_4,x_d_5,x_d_6 = sym.symbols('Dx_d_1,Dx_d_2,Dx_d_3,Dx_d_4,Dx_d_5,Dx_d_6')
x_r_1,x_r_2 = sym.symbols('Dx_r_1,Dx_r_2')
x_i_1,x_i_2 = sym.symbols('Dx_i_1,Dx_i_2')

x_e = sym.Matrix([x_d_1,x_d_2,x_d_3,x_d_4,x_d_5,x_d_6,x_r_1,x_r_2,x_i_1,x_i_2])
u_r = -K_c * x_e

u_r_d = str(sym.N(u_r[0],8))
u_r_q = str(sym.N(u_r[1],8))

print(f'Du_r_1 = {u_r_d};')
print(f'Du_r_2 = {u_r_q};')

print('\nWarning: Control output is v_t_dq!!')
```

    Du_r_1 = -1.8262175*Dx_d_1 + 0.019754652*Dx_d_2 - 0.0048070784*Dx_d_3 + 0.0010494948*Dx_d_4 - 3.4543535*Dx_d_5 + 0.39761572*Dx_d_6 - 2715.7977*Dx_i_1 + 1040.9568*Dx_i_2 - 0.071871552*Dx_r_1 + 0.00052148736*Dx_r_2;
    Du_r_2 = -0.019754652*Dx_d_1 - 1.8262175*Dx_d_2 - 0.0010494948*Dx_d_3 - 0.0048070784*Dx_d_4 - 0.39761572*Dx_d_5 - 3.4543535*Dx_d_6 - 1040.9568*Dx_i_1 - 2715.7977*Dx_i_2 - 0.00052148736*Dx_r_1 - 0.071871552*Dx_r_2;

    Warning: Control output is v_t_dq!!

```python
it_ini = 4
Δx_o = sym.Matrix([sym.Symbol(f'xD[{it+it_ini}]') for it in range(6)])
Δz_o = sym.Matrix([sym.Symbol(item) - sym.Symbol(item+'_0') for item in ['v_md', 'v_mq', 'i_sd', 'i_sq']])
Δx_r = sym.Matrix([sym.Symbol(f'Dx_r_{it+1}') for it in range(2)])
Δu_pert = sym.Matrix([sym.Symbol(f'Du_pert_{it+1}') for it in range(2)])

Δx_o_kp1 = A_d @ Δx_o + B_b@(Δx_r) +  K_o @ (Δz_o - C_o @ Δx_o) # + B_pert@Δu_pert  +  K_o @ (Δz_o - C_o @ Δx_o - D_o @ (Δx_r))

for it in range(6):
    print(f'xD[{it+it_ini}] = {Δx_o_kp1[it]};')
```

    xD[4] = 0.0392952872921423*Dx_r_1 + 0.000617299701089028*Dx_r_2 + 0.167670572903802*i_sd - 0.167670572903802*i_sd_0 + 0.427599429013122*v_md - 0.427599429013122*v_md_0 + 0.962811636697318*xD[4] - 0.430586049216921*xD[6] - 0.132051772873075*xD[8];
    xD[5] = -0.000617299701089028*Dx_r_1 + 0.0392952872921423*Dx_r_2 + 0.167670572903802*i_sq - 0.167670572903802*i_sq_0 + 0.427599429013122*v_mq - 0.427599429013122*v_mq_0 + 0.962811636697318*xD[5] - 0.430586049216921*xD[7] - 0.132051772873075*xD[9];
    xD[6] = 0.0356406531431186*Dx_r_1 + 0.000559888120127431*Dx_r_2 - 0.425448653071038*i_sd + 0.425448653071038*i_sd_0 + 0.443158057967852*v_md - 0.443158057967852*v_md_0 + 0.933318813687106*xD[4] - 0.449166750687225*xD[6] - 0.507870160616068*xD[8];
    xD[7] = -0.000559888120127431*Dx_r_1 + 0.0356406531431186*Dx_r_2 - 0.425448653071038*i_sq + 0.425448653071038*i_sq_0 + 0.443158057967853*v_mq - 0.443158057967853*v_mq_0 + 0.933318813687106*xD[5] - 0.449166750687225*xD[7] - 0.507870160616068*xD[9];
    xD[8] = 0.000668382397147673*Dx_r_1 + 1.0499789730636e-5*Dx_r_2 + 0.590326602628365*i_sd - 0.590326602628365*i_sd_0 - 0.0126468749218836*v_md + 0.0126468749218836*v_md_0 + 0.0356188000307263*xD[4] + 0.0156334951256824*xD[6] + 0.372485034068953*xD[8];
    xD[9] = -1.0499789730636e-5*Dx_r_1 + 0.000668382397147673*Dx_r_2 + 0.590326602628365*i_sq - 0.590326602628365*i_sq_0 - 0.0126468749218836*v_mq + 0.0126468749218836*v_mq_0 + 0.0356188000307263*xD[5] + 0.0156334951256824*xD[7] + 0.372485034068953*xD[9];

```python
#eta_dq[0] = Du_r_1*2/v_dc[0] + (v_sd)*2/v_dc[0] ;
#eta_dq[1] = Du_r_2*2/v_dc[0] + (v_sq)*2/v_dc[0] ;
#Du_r_1 = eta_dq[0]*v_dc[0]/2
#Du_r_2 = eta_dq[1]*v_dc[0]/2

Du_r_1,Du_r_2 = sym.symbols('Du_r_1,Du_r_2')
Du_r = sym.Matrix([Du_r_1,Du_r_2 ])

Dx_r = W@Du_r

Dx_r_1 = str(sym.N(Dx_r[0],8))
Dx_r_1 = str(sym.N(Dx_r[1],8))

print(f'Dx_r_1 = {Dx_r_1};')
print(f'Dx_r_2 = {Dx_r_1};')
```

    Dx_r_1 = -0.015707317*Du_r_1 + 0.99987663*Du_r_2;
    Dx_r_2 = -0.015707317*Du_r_1 + 0.99987663*Du_r_2;

```python
# Control with observer Du_r = -K_c*Dx_o
x_d_1,x_d_2,x_d_3,x_d_4,x_d_5,x_d_6 = sym.symbols('xD[4],xD[5],xD[6],xD[7],xD[8],xD[9]')
x_r_1,x_r_2 = sym.symbols('Dx_r_1,Dx_r_2')
x_i_1,x_i_2 = sym.symbols('Dx_i_1,Dx_i_2')

x_e = sym.Matrix([x_d_1,x_d_2,x_d_3,x_d_4,x_d_5,x_d_6,x_r_1,x_r_2,x_i_1,x_i_2])
u_r = -K_c * x_e

u_r_d = str(sym.N(u_r[0],8))
u_r_q = str(sym.N(u_r[1],8))

print(f'Du_r_1 = {u_r_d};')
print(f'Du_r_2 = {u_r_q};')

print('\nWarning: Control output is eta_dq!!')
```

    Du_r_1 = -2715.7977*Dx_i_1 + 1040.9568*Dx_i_2 - 0.071871552*Dx_r_1 + 0.00052148736*Dx_r_2 - 1.8262175*xD[4] + 0.019754652*xD[5] - 0.0048070784*xD[6] + 0.0010494948*xD[7] - 3.4543535*xD[8] + 0.39761572*xD[9];
    Du_r_2 = -1040.9568*Dx_i_1 - 2715.7977*Dx_i_2 - 0.00052148736*Dx_r_1 - 0.071871552*Dx_r_2 - 0.019754652*xD[4] - 1.8262175*xD[5] - 0.0010494948*xD[6] - 0.0048070784*xD[7] - 0.39761572*xD[8] - 3.4543535*xD[9];

    Warning: Control output is eta_dq!!

## Symbolic obtention of the plant model
# as in pydae example (synchronous dq):
di_tD = 1/L_t*(eta_D/2*v_dc - R_t*i_tD + omega*L_t*i_tQ - v_mD)
di_tQ = 1/L_t*(eta_Q/2*v_dc - R_t*i_tQ - omega*L_t*i_tD - v_mQ)
dv_mD = 1/C_m*(i_tD + C_m*omega*v_mQ - G_d*v_mD - i_sD)
dv_mQ = 1/C_m*(i_tQ - C_m*omega*v_mD - G_d*v_mQ - i_sQ)
di_sD = 1/L_s*(v_mD - R_s*i_sD + omega*L_s*i_sQ - v_sD)
di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - omega*L_s*i_sD - v_sQ)

# equivalent to pydae example (stationary dq):
di_tD = 1/L_t*(v_tD - R_t*i_tD - v_mD)
di_tQ = 1/L_t*(v_tQ - R_t*i_tQ - v_mQ)
dv_mD = 1/C_m*(i_tD - G_d*v_mD - i_sD)
dv_mQ = 1/C_m*(i_tQ - G_d*v_mQ - i_sQ)
di_sD = 1/L_s*(v_mD - R_s*i_sD - v_sD)
di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - v_sQ)

```python
R_t,L_t,C_m,G_d,R_s,L_s = sym.symbols('R_t,L_t,C_m,G_d,R_s,L_s', real=True)
i_tD,i_tQ,v_mD,v_mQ,i_sD,i_sQ = sym.symbols('i_tD,i_tQ,v_mD,v_mQ,i_sD,i_sQ', real=True)
v_tD,v_tQ = sym.symbols('v_tD,v_tQ', real=True)
v_sD,v_sQ = sym.symbols('v_sD,v_sQ', real=True)

A = sym.Matrix([
    [-R_t/L_t,       0,   -1/L_t,        0,        0,        0],
    [       0,-R_t/L_t,        0,   -1/L_t,        0,        0],
    [   1/C_m,       0, -G_d/C_m,        0,   -1/C_m,        0],
    [       0,   1/C_m,        0, -G_d/C_m,        0,   -1/C_m],
    [       0,       0,    1/L_s,        0, -R_s/L_s,        0],
    [       0,       0,        0,    1/L_s,        0, -R_s/L_s],
    ])

B = sym.Matrix([
    [ 1/L_t,        0],
    [       0,  1/L_t],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    ])

B_g = sym.Matrix([
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [       0,      0],
    [-1/L_s,        0],
    [       0, -1/L_s],
    ])

x = sym.Matrix([i_tD,i_tQ,v_mD,v_mQ,i_sD,i_sQ])
u = sym.Matrix([v_tD,v_tQ])
u_g = sym.Matrix([v_sD,v_sQ])

dx_new = A@x + B@u + B_g@u_g

di_tD = 1/L_t*(v_tD - R_t*i_tD - v_mD)
di_tQ = 1/L_t*(v_tQ - R_t*i_tQ - v_mQ)
dv_mD = 1/C_m*(i_tD - G_d*v_mD - i_sD)
dv_mQ = 1/C_m*(i_tQ - G_d*v_mQ - i_sQ)
di_sD = 1/L_s*(v_mD - R_s*i_sD - v_sD)
di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - v_sQ)
dx = sym.Matrix([di_tD,di_tQ,dv_mD,dv_mQ,di_sD,di_sQ])

dx = sym.Matrix([di_tD,di_tQ,dv_mD,dv_mQ,di_sD,di_sQ])
sym.simplify(dx_new - dx)  # just to check the model is ok
```

$\displaystyle \left[\begin{matrix}0\\0\\0\\0\\0\\0\end{matrix}\right]$


```python
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from sympy import *
from sympy.abc import *

init_printing()
```

```python
# 単位ベクトルとか角度の定義
angles = symbols("phi_0 phi_1 phi_2")# 原点から肩部への方向を表す角度
thetas = symbols("theta_0 theta_1 theta_2")# モーターの角度(±90deg)
unit_vectors = [Matrix([cos(angles[i]),sin(angles[i]),0]) for i in range(3)]
ez = Matrix([0,0,1])
```

```python
# パラメータ(実数)
params = [(A,130.0),(B,200.0),(C,400.0),(D,130.0)]
params.append((angles[0],2.0*np.pi/3.0*0))
params.append((angles[1],2.0*np.pi/3.0*1))
params.append((angles[2],2.0*np.pi/3.0*2))
params.append((x,0))
params.append((y,0))
params.append((z,400))
```

```python
# 各座標の計算
A_vectors = map(lambda x: A*x,unit_vectors)
B_vectors = [A_vectors[i] + B*(unit_vectors[i] * cos(thetas[i])-ez*sin(thetas[i]) ) for i in range(3)]
D_vector = Matrix([x,y,z])
C_vectors = [D_vector + D * unit_vectors[i] for i in range(3)]
```

```python
# モーターのある位置の座標．肩っぽい部分の座標．
A_vectors
```

$$\left [ \left[\begin{matrix}A \cos{\left (\phi_{0} \right )}\\A \sin{\left (\phi_{0} \right )}\\0\end{matrix}\right], \quad \left[\begin{matrix}A \cos{\left (\phi_{1} \right )}\\A \sin{\left (\phi_{1} \right )}\\0\end{matrix}\right], \quad \left[\begin{matrix}A \cos{\left (\phi_{2} \right )}\\A \sin{\left (\phi_{2} \right )}\\0\end{matrix}\right]\right ]$$

```python
# 腕の中央の関節．肘っぽい部分の座標
B_vectors
```

$$\left [ \left[\begin{matrix}A \cos{\left (\phi_{0} \right )} + B \cos{\left (\phi_{0} \right )} \cos{\left (\theta_{0} \right )}\\A \sin{\left (\phi_{0} \right )} + B \sin{\left (\phi_{0} \right )} \cos{\left (\theta_{0} \right )}\\- B \sin{\left (\theta_{0} \right )}\end{matrix}\right], \quad \left[\begin{matrix}A \cos{\left (\phi_{1} \right )} + B \cos{\left (\phi_{1} \right )} \cos{\left (\theta_{1} \right )}\\A \sin{\left (\phi_{1} \right )} + B \sin{\left (\phi_{1} \right )} \cos{\left (\theta_{1} \right )}\\- B \sin{\left (\theta_{1} \right )}\end{matrix}\right], \quad \left[\begin{matrix}A \cos{\left (\phi_{2} \right )} + B \cos{\left (\phi_{2} \right )} \cos{\left (\theta_{2} \right )}\\A \sin{\left (\phi_{2} \right )} + B \sin{\left (\phi_{2} \right )} \cos{\left (\theta_{2} \right )}\\- B \sin{\left (\theta_{2} \right )}\end{matrix}\right]\right ]$$

```python
# 下部の接続部分の座標．手首みたいな場所，
C_vectors
```

$$\left [ \left[\begin{matrix}D \cos{\left (\phi_{0} \right )} + x\\D \sin{\left (\phi_{0} \right )} + y\\z\end{matrix}\right], \quad \left[\begin{matrix}D \cos{\left (\phi_{1} \right )} + x\\D \sin{\left (\phi_{1} \right )} + y\\z\end{matrix}\right], \quad \left[\begin{matrix}D \cos{\left (\phi_{2} \right )} + x\\D \sin{\left (\phi_{2} \right )} + y\\z\end{matrix}\right]\right ]$$

```python
# 下部の中央部分の座標．
D_vector
```

$$\left[\begin{matrix}x\\y\\z\end{matrix}\right]$$

```python
def norm2(x):
    """ 3次元ベクトルxのノルムの2乗を計算する関数 """
    return x[0]*x[0]+x[1]*x[1]+x[2]*x[2]
```

```python
# 肘部分と手首の距離はCとして定義していることから
# (C_vectors[0] - Bvectors[0]).norm() - C == 0 となります
# ただ標準のnormではうまく計算ができなかったので自作した関数を用いてノルム(の2乗)を計算します
eq0 = C**2-norm2((C_vectors[0] - B_vectors[0]))
eq0 = expand(eq0)
eq0 = simplify(eq0)
eq0
```

```python
# 先に P + Q * sin(theta_0) + R * cos(theta_0) = 0 の方程式を考える
formula = Eq(P + Q * sin(thetas[0]) + R * cos(thetas[0]),0)
formula
```

```python
# thetaについての解はこうなる
theta_ans = solve(formula,thetas[0])
```

```python
# P + Q*sin(theta_0) + R*cos(theta_0) = 0 の形を取り出す
Q_dash = eq0.coeff(sin(thetas[0]),1)
R_dash = eq0.coeff(cos(thetas[0]),1)
P_dash = eq0.coeff(cos(thetas[0]),0) - Q_dash * sin(thetas[0])

# P + Q*sin(thetas[0])+R*cos(thetas[0])が上のeq0と同じものであるか確認
simplify(expand(P_dash + Q_dash*sin(thetas[0])+R_dash*cos(thetas[0])))
```

```python
P_dash
```

```python
Q_dash
```

```python
R_dash
```

```python
# Pythonのコードを生成
from sympy.printing.lambdarepr import lambdarepr
print("P = " + lambdarepr(P_dash))
print("Q = " + lambdarepr(Q_dash))
print("R = " + lambdarepr(R_dash))
print("theta_0 = " + lambdarepr(theta_ans[0]))
print("theta_1 = " + lambdarepr(theta_ans[1]))
```

    P = -A**2 + 2*A*D + 2*A*x*cos(phi_0) + 2*A*y*sin(phi_0) - B**2 + C**2 - D**2 - 2*D*x*cos(phi_0) - 2*D*y*sin(phi_0) - x**2 - y**2 - z**2
    Q = -2*B*z
    R = -2*A*B + 2*B*D + 2*B*x*cos(phi_0) + 2*B*y*sin(phi_0)
    theta_0 = -2*atan((Q - sqrt(-P**2 + Q**2 + R**2))/(P - R))
    theta_1 = -2*atan((Q + sqrt(-P**2 + Q**2 + R**2))/(P - R))

```python
# Cのコードを生成
float_type = "double"
print(float_type + " P = " + ccode(P_dash) + ";")
print(float_type + " Q = " + ccode(Q_dash) + ";")
print(float_type + " R = " + ccode(R_dash) + ";")
print(float_type + " theta_0 = " + ccode(theta_ans[0]) + ";")
print(float_type + " theta_1 = " + ccode(theta_ans[1]) + ";")
```

    double P = -pow(A, 2) + 2*A*D + 2*A*x*cos(phi_0) + 2*A*y*sin(phi_0) - pow(B, 2) + pow(C, 2) - pow(D, 2) - 2*D*x*cos(phi_0) - 2*D*y*sin(phi_0) - pow(x, 2) - pow(y, 2) - pow(z, 2);
    double Q = -2*B*z;
    double R = -2*A*B + 2*B*D + 2*B*x*cos(phi_0) + 2*B*y*sin(phi_0);
    double theta_0 = -2*atan((Q - sqrt(-pow(P, 2) + pow(Q, 2) + pow(R, 2)))/(P - R));
    double theta_1 = -2*atan((Q + sqrt(-pow(P, 2) + pow(Q, 2) + pow(R, 2)))/(P - R));


```python
from __future__ import print_function
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
import matplotlib.pyplot as plt
from PIL import Image
import sympy as spp
imgx=800
imgy=800
imgz=800
image=Image.new("RGB",(imgx,imgy))
image.putpixel((100,100),(255,255,255))
xa=-2
xb=2
ya=-2
yb=2
maxit=30
h=1e-6
eps=1e-3
@interact(k=[2,3,4,5,6,7,8,9],R=[1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100],G=[1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100],B=[1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100])
def Q(k,R,G,B):
    k=int(k)
    def f(z):
        return z**k-k-k
    for y in range (imgy):
        zy=y*(yb-ya)/(imgy-1)+ya
        for x in range (imgx):
            zx=x*(xb-xa)/(imgx-1)+xa
            z=complex(zx,zy)
            for i in range (maxit):
                dz=(f(z+complex(h,h))-f(z))/complex(h,h)
                z0=z-f(z)/dz
                if abs (z0-z)<eps:
                    break
                R=int(R)
                G=int(G)
                B=int(B)
                z=z0
                r=i*R
                g=i*G
                b=i*B
                image.putpixel((x,y),(r,g,b))
    return image
```


# Find distribution of local maxima in a Gaussian Random Field

In this notebook, I try to find the distribution of local maxima in a Gaussian Random Field.  I followed several steps to (a) look at the distribution through simulations, (b) apply different possible approaches to find the distribution.

It is a draft and a bit a mess of things that might seem logical stuff, but I also wanted to summarise (and see) more or less what is known.

# 1. Simulate random fields (without activation) and extract local maxima

In a first step, I
- Simulate a guassian random field using nipype.  This field is huge (500x500x500) and therefore memory-intensive to ensure we have lots of local maxima.
- Look at the GRF.
- Export the GRF to a nifti file.  Before saving the data, I make sure all values are positive, because step (d) only extracts local maxima that are above 0.
- Extract all local maxima using nipype and fsl cluster.
- Look at the table of local maxima and print the total number of peaks.
- Look at the distribution of these maxima.

#### Import libraries

```python
% matplotlib inline
import os
import numpy as np
import nibabel as nib
from nipy.labs.utils.simul_multisubject_fmri_dataset import surrogate_3d_dataset
import nipy.algorithms.statistics.rft as rft
from __future__ import print_function, division
import math
import matplotlib.pyplot as plt
import palettable.colorbrewer as cb
from nipype.interfaces import fsl
import pandas as pd
import nipy.algorithms.statistics.intvol as intvol
from matplotlib import colors
import scipy.stats as stats

```

#### Simulate very large RF

```python
smooth_FWHM = 3
smooth_sd = smooth_FWHM/(2*math.sqrt(2*math.log(2)))
data = surrogate_3d_dataset(n_subj=1,sk=smooth_sd,shape=(500,500,500),noise_level=1)
```

#### Show part of the RF (20x20x1)

```python
plt.figure(figsize=(6,4))
plt.imshow(data[1:20,1:20,1])
plt.colorbar()
plt.show()
```

#### Save RF

```python
minimum = data.min()
newdata = data - minimum #little trick because fsl.model.Cluster ignores negative values
img=nib.Nifti1Image(newdata,np.eye(4))
img.to_filename("files/RF.nii.gz")
```

#### Run fsl cluster to extract local maxima

```python
cl=fsl.model.Cluster()
cl.inputs.threshold = 0
cl.inputs.in_file="files/RF.nii.gz"
cl.inputs.out_localmax_txt_file="files/locmax.txt"
cl.inputs.num_maxima=1000000
cl.inputs.connectivity=26
cl.inputs.terminal_output='none'
cl.run()
```

    <nipype.interfaces.base.InterfaceResult at 0x10deddc90>

#### Read and print top of file with peaks ,  print total number of peaks.

```python
peaks = pd.read_csv("files/locmax.txt",sep="\t").drop('Unnamed: 5',1)
peaks.Value = peaks.Value + minimum
peaks[:5]
```

      Cluster Index
      Value
      x
      y
      z

      0
       1
       6.668672
        66
         0
       243

      1
       1
       6.468672
       468
         0
       255

      2
       1
       6.268672
       227
       499
       401

      3
       1
       6.168672
       291
       499
       191

      4
       1
       6.168672
       312
       499
       330

```python
len(peaks)
```

    615036

#### Plot histogram local maxima

```python
col=cb.qualitative.Set1_8.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(peaks.Value,40,normed=1,facecolor=col[0],alpha=0.75,lw=0)
ax.set_xlim([-1,5])
plt.show()
```

# 2. First approach: analytically derived distribution of local maxima above u

#a. (Cheng & Schwartzman, 2015)
#b. RFT

[Cheng and Schwartzman](https://www.dropbox.com/s/goo6bj5vw82gd8b/Cheng%2C%20Schwartzman%20-%202015%20-%20Extremes%20-%20Distribution%20of%20the%20height%20of%20local%20maxima%20of%20Gaussian%20random%20fields.pdf?dl=0) recently published a paper in which they derive a distribution of local maxima over a certain threshold.  They make (like RFT) the assumption that the field is a GRF whose interior is non-empty... A consequence is that we can only compute this when the threshold is high enough to ensure there are only blobs and no holes.  We'll take a look how the theoretical distribution performs for lower thresholds.  This is their derived distribution:

For all local maxima above threshold $v$, extending $u$ above this threshold,

For each $t_0 \in T$ and each fixed $u>0$, as $v \rightarrow \infty$,

\begin{equation*}
F_t(u,v) = \frac{(u+v)^{N-1}e^{-(u+v)^2/2}}{v^{N-1}e^{-v^2/2}}
\end{equation*}

Below you can see that the approximation indeed only works well on very high thresholds, and therefore cannot be used to obtain the full distribution of all peaks.

We also compare with the random field theory approximation, with u the threshold:

\begin{equation}
F_t(u,t_0) = u e^{-u(t_0 - u)}
\end{equation}

#### Function for pdf of peaks given a certain threshold

```python
def nulprobdens(exc,peaks):
	v = exc
	u = peaks - v
	f0 = (2+(u+v)**2)*(u+v)*np.exp(-(u+v)**2/2)/(v**2*np.exp(-v**2/2))
	return f0
```

#### Function for pdf of peaks given a certain threshold, RFT

```python
def nulprobdensRFT(exc,peaks):
    f0 = exc*np.exp(-exc*(peaks-exc))
    return f0
```

#### Compute density function over a range with different excursion thresholds

```python
fig,axs=plt.subplots(1,5,figsize=(13,3))
fig.subplots_adjust(hspace = .5, wspace=0.3)
axs=axs.ravel()
thresholds=[2,2.5,3,3.5,4]
bins=np.arange(2,5,0.5)
x=np.arange(2,10,0.0001)

twocol=cb.qualitative.Paired_10.mpl_colors
for i in range(5):
    thr=thresholds[i]
    axs[i].hist(peaks.Value[peaks.Value>thr],lw=0,facecolor=twocol[i*2-2],normed=True,bins=np.arange(thr,5,0.1))
    axs[i].set_xlim([thr,5])
    axs[i].set_ylim([0,3])
    xn = x[x>thr]
    yn = nulprobdens(thr,xn)
    ynb = nulprobdensRFT(thr,xn)
    axs[i].plot(xn,yn,color=twocol[i*2-1],lw=3,label="C&S")
    axs[i].plot(xn,ynb,color=twocol[i*2-1],lw=3,linestyle="--",label="RFT")
    axs[i].set_title("threshold:"+str(thr))
    axs[i].set_xticks(np.arange(thr,5,0.5))
    axs[i].set_yticks([1,2])
    axs[i].legend(loc="upper right",frameon=False)
    axs[i].set_xlabel("peak height")
    axs[i].set_ylabel("density")
plt.show()
```

In this figure, we see the observed tail distribution of the local maxima (light colored) in our simulated data.  The thick line represents the theoretical distribution of the local maxima above a certain threshold. It is only a good approximation for really high thresholds. RFT works better for lower thresholds.

# 3. Second approach: what can we use the Euler Characteristic for?

The [Euler Characteristic](https://en.wikipedia.org/wiki/Euler_characteristic) is a topological invariant that represents for a certain threshold applied to a certain random field \#peaks - \#holes.  Therefore, again, if the threshold is high enough (there are no holes), it computes the number of peaks.

If we compute the EC for a range of thresholds (above a certain threshold), we can construct a pdf of peak heights.  It should be noted that the number of peaks is dependent of the smoothness, but this dependency can be taken out by dividing the smoothness by the total volume as a correcting factor.

This principle has led to a theoretical approximation of the pdf of peaks, which is closely related to the approach of Cheng \& Schwartzmann.

#### Let's first look at the masks that result from our random field with certain thresholds

```python
fig,axs=plt.subplots(1,4,figsize=(13,7))
fig.subplots_adjust(hspace = .1, wspace=0.1)
axs=axs.ravel()
thresholds=np.arange(0,4,1)
cmap = colors.ListedColormap(['white', 'black'])
bounds=[0,0.5,1]
norm = colors.BoundaryNorm(bounds, cmap.N)

for t in range(len(thresholds)):
    mask = np.zeros(shape=data.shape,dtype=np.intp)
    mask[data>thresholds[t]]=1
    axs[t].imshow(mask[1:200,1:200,20],cmap=cmap,norm=norm)
    axs[t].set_title("threshold:"+str(thresholds[t]))
    axs[t].patch.set_visible(False)
    axs[t].axis('off')

```

#### Now we'll look at the number of peaks and the Euler Characteristic against the threshold.

```python
EulerDens = []
EulerDensInv = []
urange = np.arange(-4,4,0.3)
for t in urange:
    mask = np.zeros(shape=data.shape,dtype=np.intp)
    mask[data>t]=1
    EulerDens.append(intvol.EC3d(mask))
    mask2 = 1-mask
    EulerDensInv.append(intvol.EC3d(mask2))

```

```python
sumpeak = []
for t in urange:
    sumpeak.append(sum(peaks.Value>t))

```

```python
plt.figure(figsize=(7,5))
plt.plot(urange,EulerDens,color=col[1],lw=3,label="observed Euler Characteristic")
plt.plot(urange,EulerDensInv,color=col[2],lw=3,label="observed inverse Euler Characteristic")
plt.plot(urange,sumpeak,color=col[3],lw=3,label="Number of peaks")
plt.legend(loc="upper right",frameon=False)
plt.ylim([-600000,1200000])
plt.show()
```

In this plot we can indeed see that the Euler Characteristic gives the number of peaks, but only above a certain threshold that is high enough.  Below these higher thresholds, is gives \# peaks - \# holes.  Is there a way to estimate the number of holes in the presence of peaks using the EC?  I don't think so, it's the exact same problem as the number of paeks in the presence of holes?  Therefore the Euler Characteristic cannot give us information for lower thresholds.

# 4. Third approach: peaks as the maximum of something of which we know the distribution?

The above procedures use the fact that these fields are (should be) continuous fields.  However, what comes out of the scanner is **not** continuous at all.  We make it __more__ continuous by applying a smoothing kernel, which allows to use these processes.  However, I wonder, is it possible to look at the peaks simply as the maximum of a sample of a known distribution?

Can we look at the random fields as multivariate normal with a correlation structure dependent on the smoothness?

Here the smoothness comes into play!  How does smoothness affect the distribution of the peaks?

#### First simulate random fields for different smoothnesses.

```python
smoothnesses = [0,3,6,9]
minima = []
for sm in range(len(smoothnesses)):
    smooth_FWHM = smoothnesses[sm]
    smooth_sd = smooth_FWHM/(2*math.sqrt(2*math.log(2)))
    data = surrogate_3d_dataset(n_subj=1,sk=smooth_sd,shape=(500,500,500),noise_level=1)
    minimum = data.min()
    newdata = data - minimum #little trick because fsl.model.Cluster ignores negative values
    minima.append(minimum)
    img=nib.Nifti1Image(newdata,np.eye(4))
    img.to_filename(os.path.join("files/RF_"+str(sm)+".nii.gz"))
    cl=fsl.model.Cluster()
    cl.inputs.threshold = 0
    cl.inputs.in_file=os.path.join("files/RF_"+str(sm)+".nii.gz")
    cl.inputs.out_localmax_txt_file=os.path.join("files/locmax_"+str(sm)+".txt")
    cl.inputs.num_maxima=10000000
    cl.inputs.connectivity=26
    cl.inputs.terminal_output='none'
    cl.run()

```

#### Look at the distribution of the voxels?

We simulated the random fields to be a GRF, from a normal distribution.  Here you can see that indeed the distribution is normal independent from the smoothness.

```python
col=cb.qualitative.Set1_8.mpl_colors+cb.qualitative.Set2_8.mpl_colors
plt.figure(figsize=(10,5))
ax=plt.subplot(111)
for sm in range(len(smoothnesses)):
    file = os.path.join("files/RF_"+str(sm)+".nii.gz")
    tvals = nib.load(file).get_data().astype('float64')+minima[sm]
    values, base = np.histogram(tvals,100,normed=1)
    ax.plot(base[:-1],values,label="smoothness: "+str(smoothnesses[sm]),color=col[sm],lw=2)
ax.set_xlim([-4,4])
ax.set_ylim([0,0.5])
ax.legend(loc="lower right",frameon=False)
ax.set_title("distribution of peak heights for different smoothing kernels (FWHM)")
plt.show()
```

#### So how about the distribution of the maximum of a sample of these distributions?

```python
all = []
for sm in range(len(smoothnesses)):
    peaks = pd.read_csv(os.path.join("files/locmax_"+str(sm)+".txt"),sep="\t").drop('Unnamed: 5',1).Value
    peaks = peaks + minima[sm]
    all.append(peaks)

```

```python
col=cb.qualitative.Set1_8.mpl_colors+cb.qualitative.Set2_8.mpl_colors
plt.figure(figsize=(10,5))
ax=plt.subplot(111)
for sm in range(len(smoothnesses)):
    values, base = np.histogram(all[sm],30,normed=1)
    ax.plot(base[:-1],values,label="smoothness: "+str(smoothnesses[sm]),color=col[sm],lw=2)
ax.set_xlim([-1,5])
ax.set_ylim([0,1.2])
ax.legend(loc="lower right",frameon=False)
ax.set_title("distribution of peak heights for different smoothing kernels (FWHM)")
plt.show()

```

As expected, from a certain smoothness (3 x voxel size), the distribution remains the same.

## First: the distribution without smoothness!

The RF with smoothness 0 should be just normally distributed values without correlation.  The peaks with smoothness 0 should be just maxima of a sample of normally distributed values.  We know the distribution of a sample with normally distribution values from order statistics:

$f_{max}(x) = nF(x)^{n-1}f(x)$

#### This is to show that the distribution of the maximum of the sample is the right one.
* Simulate n normally distributed values
* Take the maximum
* Compute for a range of values the distribution of the maximum
* Show histogram of simulated maxima with the pdf on top

```python
# random sample

smplm = []

for i in range(100000):
    smpl = np.random.standard_normal((n,))
    smplm.append(max(smpl))

# distribution

xm = np.arange(-1,5,0.001)
ym = n*stats.norm.cdf(xm)**(n-1)*stats.norm.pdf(xm)

# histogram

twocol=cb.qualitative.Paired_10.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(smplm,100,normed=1,facecolor=twocol[0],alpha=0.75,lw=0)
ax.plot(xm,ym,color=twocol[1],lw=3)
ax.set_xlim([-1,5])
plt.show()
```

#### Now can we just state that the peaks in our unsmoothed random field is the maxima of a sample?

First: the sample size is variable!  Let's try it on the average sample size, i.e. the average number of voxels per peak.

```python
n = (500**3)/len(all[1])
n
```

    203.16118808662793

#### Show histogram of peaks of uncorrected field with the distribution of maximum of a sample of size n.

```python
# distribution of a maximum

xm = np.arange(-1,5,0.001)
ym = n*stats.norm.cdf(xm)**(n-1)*stats.norm.pdf(xm)

# histogram

twocol=cb.qualitative.Paired_10.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(all[1],100,normed=1,facecolor=twocol[0],alpha=0.75,lw=0)
ax.plot(xm,ym,color=twocol[1],lw=3)
ax.set_xlim([-1,5])
plt.show()
```

# Ok, I'm stuck.

The stuff below is the maximum of a sample with correlation.  We have the distribution, but if it doesn't work for uncorrelated values, it's not going to work for correlated values.  So I leave it here but it's non-informative.

```python
# random sample

smplmc = []
n = 2
mean = [0,0]
r = 0.2
cov = [[1,r],[r,1]]

for i in range(100000):
    smpl = np.random.multivariate_normal(mean,cov,int(n/n))
    smplmc.append(np.max(smpl))

# distribution

xmc = np.arange(-2,3,0.001)
corf = (1-r)/np.sqrt(1-r**2)
ymc = n*stats.norm.cdf(corf*xmc)**(n-1)*stats.norm.pdf(xmc)

# histogram

twocol=cb.qualitative.Paired_10.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(smplmc,100,normed=1,facecolor=twocol[2],alpha=0.75,lw=0)
ax.plot(xmc,ymc,color=twocol[3],lw=3)
ax.set_xlim([-1,5])
plt.show()

```

```python
# random sample

smplmc = []
n = 10
mean = np.array([0,0,0,0,0,0,0,0,0,0])
r = 0.5
cov = np.array([[1,r,r,r,r,r,r,r,r,r],
       [r,1,r,r,r,r,r,r,r,r],
       [r,r,1,r,r,r,r,r,r,r],
       [r,r,r,1,r,r,r,r,r,r],
       [r,r,r,r,1,r,r,r,r,r],
       [r,r,r,r,r,1,r,r,r,r],
       [r,r,r,r,r,r,1,r,r,r],
       [r,r,r,r,r,r,r,1,r,r],
       [r,r,r,r,r,r,r,r,1,r],
       [r,r,r,r,r,r,r,r,r,1]
      ])

for i in range(100):
    smpl = np.random.multivariate_normal(mean,cov,int(n/n))
    smplmc.append(np.max(smpl))

# distribution (just max of gaussian normal)

xm = np.arange(-1,5,0.001)
corf = (1-r)/np.sqrt(1-r**2)
ym = n*stats.norm.cdf(xm)**(n-1)*stats.norm.pdf(xm)

# histogram

twocol=cb.qualitative.Paired_10.mpl_colors
plt.figure(figsize=(6,3))
ax=plt.subplot(111)
ax.hist(smplm,100,normed=1,facecolor=twocol[0],alpha=0.75,lw=0)
ax.plot(xm,ym,color=twocol[1],lw=3)
ax.set_xlim([-1,5])
plt.show()
```

```python
newline
```

```python

```


# Simple fitness landscape

(c) 2019 Manuel Razo. This work is licensed under a [Creative Commons Attribution License CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/). All code contained herein is licensed under an [MIT license](https://opensource.org/licenses/MIT).

---

```python
import os

# Our numerical workhorses
import numpy as np
import scipy
import pandas as pd

# Import matplotlib stuff for plotting
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import matplotlib

# Seaborn, useful for graphics
import seaborn as sns

# Import the project utils
import sys
sys.path.insert(0, '../../')
import infofit_utils as utils

# Magic function to make matplotlib inline; other style specs must come AFTER
%matplotlib inline

# This enables SVG graphics inline (only use with static plots (non-Bokeh))
%config InlineBackend.figure_format = 'svg'

figdir = '../fig/simple_fitness_landscape/'
tmpdir = '../../tmp/'
```

```python
# Set PBoC plotting format
utils.viz.set_plotting_style()
```

## $\LaTeX$ macros
$\newcommand{\ee}[1]{\left\langle #1 \right\rangle}$
$\newcommand{\bb}[1]{\mathbf{#1}}$
$\newcommand{\foldchange}{\text{fold-change}}$
$\newcommand{\Nns}{N_{NS}}$
$\newcommand{\er}{\Delta\varepsilon_r}$
$\newcommand{\eAI}{\Delta\varepsilon_{AI}}$

## Implementing a simple fitness landscape

On an excellent and intriguing paper, [Dekel and Alon](https://www.nature.com/articles/nature03842) measured the cost and benefit function of producing *LacZ* protein. In this paper they measure the cost and benefit of producing the enzyme, finding that non-linearities in this cost-benefit function predicted an optimal expression expression level. With this they can create a fitness lanscape as a function of the lactose concentration in the environment and the enzyme expression level. This fitness landscape was then used by [Taylor, Tishby and Bialek](https://arxiv.org/abs/0712.4382) while analyzing the connection beteen information as defined by Shannon's information theory and biological fitness.

### Fitness cost of producing protein

Dekel and Alon propose two models for the cost function. The first model $\eta_1$ assumes that the cost has a quadratic form, i.e.
\begin{equation}
    \eta_1(p) \equiv \eta_0 p + \eta_0' p^2,
\end{equation}
where $\eta_0$ and $\eta_0'$ are phenomenological parameters for this model, and $p$ is the LacZ protein expression level.

For the second model they assume that given a resource $c$, for example lactose, the cellular growth rate $\mu$ can be modeled by the monod equation
\begin{equation}
    \mu = \mu_\max \frac{c}{K_s + c},
\end{equation}
where $\mu_\max$ is the maximum growth rate and $K_s$ is the "half growth rate" phenomenological constant.

If we assume that the production of a single enzyme decreases the resource $c$ by a factor $\varepsilon$, then the growth rate $\mu$ as a function of the enzyme concentration is given by
\begin{equation}
    \mu(p) = \mu_\max \frac{c - \varepsilon p}{K_s + c - \varepsilon p}.
\end{equation}

This gives a cost function $\eta_2$ of the form
\begin{equation}
    \eta_2(p) = \frac{\mu(0) - \mu(p)}{\mu(p)} = \frac{\eta_o p}{1 - \frac{p}{M}},
\end{equation}
where $\eta_o = \frac{\varepsilon K_s}{C (K_s + C)}$ and $M = \frac{K_s + C}{\varepsilon}$.

This cost function reaches its maximum value of 1 when $ \varepsilon p = C $

The authors give the values of these parameters with respect to the expression level of the wildtype strain when fully induced with saturating IPTG $p_{wt}$. These values are
1. $\eta_o \cdot p_{wt} = 0.02 \pm 0.003$
2. $M = (1.8 \pm 0.3) p_{wt}$

With these values in hand we can plot the cost function as a function of this relative expression level.
Let's first define a function to compute this cost.

```python
def cost_func(p_rel, eta_o=0.02, M=1.8):
    '''
    Returns the relative growth rate cost of producing LacZ protein according to
    Dekel and Alon's model:
        eta_2 = eta_o * p_rel / (1 - p_rel / M)

    Parameter
    ---------
    p_rel : array-like.
        Relative expression with respect to the wild type expression when
        fully induced with IPTG
    eta_o : float.
        Parameter of the cost function
    M : float.
        Parameter of the cost function

    Returns
    -------
    eta_2 : array-like
        relative reduction in growth rate with respect to wild type when not
        expressing the enzyme.
    '''
    p_rel = np.array(p_rel)
    return eta_o * p_rel / (1 - p_rel / M)
```

Now let's plot the resulting cost function

```python
# Define range of relative expressions with respect to the fully induced WT
p_rel = np.linspace(0, 1, 200)

# Plot cost function
plt.plot(p_rel, cost_func(p_rel))

# Label axis
plt.xlabel(r'Relative LacZ expression $p / p_{wt}$')
plt.ylabel(r'Cost function $\eta_2$')

plt.tight_layout()
```

### Fitness benefit of producing the enzyme

The benefit of producing the enzyme in the presence of lactose is capture by a monod-like expression. What this means is that just as we did to derive the cost function, we assume that the growth rate $\mu$ is given by
\begin{equation}
    \mu = \mu_\max \frac{c}{K_s + c},
\end{equation}

### Building a fitness landscape

Puting the cost and benefit funcitons together together we obtain an expression for the relative growth rate $r$
\begin{equation}
    r(C, p) = \delta \cdot p \frac{C}{K_s + C} - \frac{\eta_o p}{1 - \frac{p}{M}},
\end{equation}
where $ \delta $ is the fitness benefit per substrate cleaved per enzyme and $ K_s $ is the Monod constant that gives half the maximum growth rate possible.

The values for these constants reported by the authors are
1. $\delta$ = 0.17.
2. $K_s$ = 0.4 mM.

The maximum for this fitness funciton can be obtained by computing $\frac{dr}{dp} = 0$.
This results in the optimum protein expression level $ p^* $ given by
\begin{equation}
    p^* = M \left( 1 - \sqrt{\frac{\eta}{\delta} \frac{C + K_s}{C}} \right).
\end{equation}

Dekel and Alon specify that $p^* = 0$ for concentrations lower than
\begin{equation}
    C = K_s \left( \frac{\delta}{\eta} - 1 \right)^{-1} \approx 0.057 mM,
\end{equation}
since the cost exceeds the benefit.

With this we can compute and plot the fitness landscape as a function of the relative expression level and the substrate concentration. Let's define the necessary functions to compute this fitness landscape.

```python
def benefit_func(p_rel, C_array, delta=0.17, Ks=0.4):
    '''
    Returns the relative growth rate benefit of producing LacZ protein
    according to Dekel and Alon's model:
        r = delta * p_rel * C / (Ks + C)

    Parameter
    ---------
    p_rel : array-like.
        Relative expression with respect to the wild type expression when
        fully induced with IPTG.
    C_array : array-like.
        Substrate concentration.
    delta : float.
        growth benefit per substrate cleaved per enzyme.
    Ks : float.
        Monod constant for half maximum growth rate.

    Returns
    -------
    r : array-like
        relative increase in growth rate with respect to wild type when not
        expressing the enzyme.
    '''
    p_rel = np.array(p_rel)
    return delta * p_rel * C_array / (Ks + C_array)

def fitness(p_rel, C_array, delta=0.17, Ks=0.4, eta_o=0.02, M=1.8,
           logC=False):
    '''
    Returns the relative fitness according to Dekel and Alon's model.

    Parameter
    ---------
    p_rel : array-like.
        Relative expression with respect to the wild type expression when
        fully induced with IPTG.
    C_array : array-like.
        Substrate concentration. If logC==True this is defined as log10(C)
    delta : float.
        growth benefit per substrate cleaved per enzyme.
    Ks : float.
        Monod constant for half maximum growth rate.
    eta_o : float.
        Parameter of the cost function
    M : float.
        Parameter of the cost function
    logC : Bool.
        boolean indicating if the concentration is given in log scale

    Returns
    -------
    fitness : array-like
        relative fitness with respect to wild type when not
        expressing the enzyme.
    '''
    p_rel = np.array(p_rel)
    C_array = np.array(C_array)
    if logC:
        C_array = 10**C_array
    # Compute benefit - cost
    return benefit_func(p_rel, C_array, delta, Ks) - cost_func(p_rel, eta_o, M)

def p_opt(C_array, delta=0.17, Ks=0.4, eta_o=0.02, M=1.8, logC=False):
    '''
    Returns the optimal protein expression level p* as a function of
    substrate concentration.

    Parameters
    ----------
    C_array : array-like.
        Substrate concentration.
    delta : float.
        growth benefit per substrate cleaved per enzyme.
    Ks : float.
        Monod constant for half maximum growth rate.
    eta_o : float.
        Parameter of the cost function
    M : float.
        Parameter of the cost function
    logC : Bool.
        boolean indicating if the concentration is given in log scale

    Returns
    -------
    p* the optimal expression level for a given concentration.
    '''
    C_array = np.array(C_array)
    if logC:
        C_array = 10**C_array

    # Dekel and Alon specify that concentrations lower than a lower
    # threshold should be zero. Then let's build that array
    thresh = Ks * (delta / eta_o - 1)**-1

    popt = np.zeros_like(C_array)
    popt[C_array > thresh] = M * (1 - np.sqrt(eta_o / delta * \
        (C_array[C_array > thresh] + Ks) / C_array[C_array > thresh]))

    return popt
```

To test these function let's reproduce Dekel and Alon's plots for different environemntal lactose concentrations.

```python
# Define the relative expression levels and the lactose concentrations
p_rel = np.linspace(0, 1.5, 200)
C_array = [5.0, 0.6, 0.1]

# Compute the relative fitness and the optimal expression level
r_mat = np.zeros([len(C_array), len(p_rel)])
popt = np.zeros(len(C_array))
r_max = np.zeros(len(C_array))

# Compute the different curves
for i, c in enumerate(C_array):
    r_mat[i, :] = fitness(p_rel, c)
    popt[i] = p_opt(c)
    r_max[i] = fitness(popt[i], c)

# Define color palette
colors = sns.color_palette('Greens_r', n_colors=len(C_array) + 2)
# Plot curves along with optimal expression level
for i, r in enumerate(r_mat):
    plt.plot(p_rel, r, label=r'{0:0.1f}'.format(C_array[i]),
             color=colors[i])
    plt.scatter(popt[i], r_max[i], color=colors[i])

# Add legend
plt.legend(loc=0, title=r'[lactose] (mM)')

# Label axis
plt.xlabel(r'Relative LacZ expression $p / p_{wt}$')
plt.ylabel(r'Relative growth rate difference')

plt.tight_layout()
```

Great! Now we have the functions to define the whole fitness landscape. Let's just go and plot it.

## Plotting the fitness landscape and the optimal expression level

Having reproduced the cost-benefit function we can now extend the range and plot the entire fitness landscape by defining a grid of expession values and a grid of lactose concentrations.

```python
# Define the sugar concentrations grid
c_min = 0
c_max = 5
C_array = np.linspace(c_min, c_max, 500)

# Define the relative expressions to evaluate
p_rel_min = 0
p_rel_max = 1.5
p_rel = np.linspace(p_rel_min, p_rel_max, 501)

# Generate grid to evaluate the fitness landscape
x_mat, y_mat = np.meshgrid(C_array, p_rel)

# Initialize matrix to save growth rate
fitness_landscape = np.empty(np.shape(x_mat))

# Compute the fitness for every lactose concentration and relative expression
for i, x in enumerate(x_mat):
    fitness_landscape[i, :] = fitness(y_mat[i, :], x)

# Compute the optimal expression level for each concentration in S_array
p_rel_opt = p_opt(C_array)
```

Let's now plot the fitness lanscape.

```python
with sns.axes_style('white'):
    fig = plt.figure()
    ax = plt.subplot(111)
    cax = ax.matshow(fitness_landscape, cmap=cm.viridis,
                    origin='lower', extent=(c_min, c_max, p_rel_min, p_rel_max),
                    aspect='auto')
    ax.plot(C_array, p_rel_opt, label=r'optimal relative expression $p^*$',
           color='darkred')

    # Set plot limits
    ax.set_xlim([c_min, c_max])
    ax.set_ylim([p_rel_min, p_rel_max])

    # Add axis labels
    ax.set_xlabel(r'[lactose] (mM)')
    ax.set_ylabel(r'relative LacZ expression $p / p_{wt}$')

    # Add legend
    leg = ax.legend(loc=0, frameon=False)
    # Changing the color of the legend text
    for text in leg.get_texts():
        text.set_color("white")
    # Change default position of axis ticks
    ax.get_xaxis().set_ticks_position('bottom')

    # Set the ticks in both axis
    start, end = ax.get_xlim()
    ax.xaxis.set_ticks(np.arange(start, end + 0.1, 0.5))

    start, end = ax.get_ylim()
    ax.yaxis.set_ticks(np.arange(start, end + 0.1, 0.2))

    # Remove tick lines but still show labels
    ax.tick_params(axis='both', which='both',length=0, pad=5)

    # Add color-bar
    cbar = fig.colorbar(cax)
    cbar.ax.set_ylabel(r'relative growth rate')
    cbar.ax.tick_params(axis='both', which='both',length=2, pad=5)

plt.savefig(figdir + 'Dekel_fitness_landscape_relative.pdf',
            bbox_inches='tight')
```

Eureka! We now have our fitness landscape.

### Plotting absolute growth rate.

To convert it to an "absolute" fitness measurement we need to convert the relative growth rate measurements to absolute growth rate in $\text{hr}^{-1}$. For this we find in the SI of the Dekel and Alon paper that the average reference growth rate was $1.15 \pm 0.015 \; \text{hr}^{-1}$. This means that if $r(c,p)$ is the **relative growth rate** as a function of the gene expression level $p$ and the substrate concentration $c$, then $f(c,p)$ is the **absolute growth rate** given by
\begin{equation}
f(c, p) = 1.15 \cdot (1 + r(c, p)) \; \text{hr}^{-1}.
\end{equation}

Let's define a function to compute this.

```python
def abs_fitness(p_rel, C_array, delta=0.17,
                Ks=0.4, eta_o=0.02, M=1.8,
                mu=1.15, logC=False):
    '''
    Returns the absolute fitness according to Dekel and Alon's model.

    Parameter
    ---------
    p_rel : array-like.
        Relative expression with respect to the wild type expression when
        fully induced with IPTG.
    C_array : array-like.
        Substrate concentration. If logC==True this is defined as log10(C)
    delta : float.
        growth benefit per substrate cleaved per enzyme.
    Ks : float.
        Monod constant for half maximum growth rate.
    eta_o : float.
        Parameter of the cost function
    M : float.
        Parameter of the cost function
    mu : float.
        Absolute growth rate of reference strain
    logC : Bool.
        boolean indicating if the concentration is given in log scale

    Returns
    -------
    fitnes : array-like
        absolute fitness of strain.
    '''
    p_rel = np.array(p_rel)
    C_array = np.array(C_array)
    if logC:
        C_array = 10**C_array
    # Compute benefit - cost
    rel_fit = benefit_func(p_rel, C_array, delta, Ks) -\
              cost_func(p_rel, eta_o, M)

    return mu * (1 + rel_fit)
```

Let's reproduce the plot above using this absolute growth rate.

```python
# Define the sugar concentrations grid
c_min = 0
c_max = 5
C_array = np.linspace(c_min, c_max, 500)

# Define the relative expressions to evaluate
p_rel_min = 0
p_rel_max = 1.5
p_rel = np.linspace(p_rel_min, p_rel_max, 500)

# Generate grid to evaluate the fitness landscape
x_mat, y_mat = np.meshgrid(C_array, p_rel)

# Initialize matrix to save growth rate
fitness_landscape = np.empty(np.shape(x_mat))

# Compute the fitness for every lactose concentration and relative expression
for i, x in enumerate(x_mat):
    fitness_landscape[i, :] = abs_fitness(y_mat[i, :], x)

# Compute the optimal expression level for each concentration in S_array
p_rel_opt = p_opt(C_array)
```

```python
with sns.axes_style('white'):
    fig = plt.figure()
    ax = plt.subplot(111)
    cax = ax.matshow(fitness_landscape, cmap=cm.viridis,
                    origin='lower', extent=(c_min, c_max, p_rel_min, p_rel_max),
                    aspect='auto')
    ax.plot(C_array, p_rel_opt, label=r'optimal relative expression $p^*$',
           color='darkred')

    # Set plot limits
    ax.set_xlim([c_min, c_max])
    ax.set_ylim([p_rel_min, p_rel_max])

    # Add axis labels
    ax.set_xlabel(r'[lactose] (mM)')
    ax.set_ylabel(r'relative LacZ expression $p / p_{wt}$')

    # Add legend
    leg = ax.legend(loc=0, frameon=False)
    # Changing the color of the legend text
    for text in leg.get_texts():
        text.set_color("white")
    # Change default position of axis ticks
    ax.get_xaxis().set_ticks_position('bottom')

    # Set the ticks in both axis
    start, end = ax.get_xlim()
    ax.xaxis.set_ticks(np.arange(start, end + 0.1, 0.5))

    start, end = ax.get_ylim()
    ax.yaxis.set_ticks(np.arange(start, end + 0.1, 0.2))

    # Remove tick lines but still show labels
    ax.tick_params(axis='both', which='both',length=0, pad=5)

    # Add color-bar
    cbar = fig.colorbar(cax)
    cbar.ax.set_ylabel(r'growth rate (hr$^{-1})$')
    cbar.ax.tick_params(axis='both', which='both',length=2, pad=5)

plt.savefig(figdir + 'Dekel_fitness_landscape_absolute.pdf',
            bbox_inches='tight')
```

### Plotting the fitness landscape in log-scale

Given that we usually plot the $x$ axis of our induction curves on a log scale it would be useful to transform this axis of our fintess landscape into that scale.

Let's reproduce the plot above in log scale.

```python
# Define the sugar concentrations and the relative expressions to evaluate
c_min_log = -1.5 # The minimum concentration at which there's a benefit
c_max_log = np.log10(5)
C_array_log = np.logspace(c_min_log, c_max_log, 50)

# Define the the relative expressions to evaluate
p_rel_min_log = 0
p_rel_max_log = 1.4
p_rel_log = np.linspace(p_rel_min_log, p_rel_max_log, 51)

# Generate grid
x_mat_log, y_mat_log = np.meshgrid(C_array_log, p_rel_log)

# Initialize matrix to save growth rate
fitness_landscape_log = np.empty(np.shape(x_mat_log))

# Compute the fitness for every lactose concentration and relative expression
for i, x in enumerate(x_mat_log):
    fitness_landscape_log[i, :] = abs_fitness(y_mat_log[i, :], x)

# Compute the optimal expression level for each concentration in C_array
p_rel_opt_log = p_opt(C_array_log)
```

Having evaluated this, let's plot the fitness landscape.

```python
with sns.axes_style('white'):
    fig = plt.figure()
    ax = plt.subplot(111)
    cax = ax.pcolormesh(C_array_log, p_rel_log,
                        fitness_landscape_log[:-1, :-1],
                        cmap=cm.viridis)
    ax.plot(C_array_log, p_rel_opt_log,
            label=r'optimal relative expression $p^*$',
            color='darkred')

    # Set log scale for x axis
    ax.set_xscale('log')

    # Set x axis ticks
    ax.tick_params(axis='x', which='minor', length=2, pad=5)
    ax.tick_params(axis='x', which='major', length=4, pad=5)

    # Set y axis ticks
    ax.yaxis.set_major_locator(matplotlib.ticker.MaxNLocator(4))

    # Set axis labels
    ax.set_xlabel(r'[lactose] (mM)')
    ax.set_ylabel(r'relative LacZ expression $p / p_{wt}$')

    # Add legend
    leg = ax.legend(loc=0, frameon=False)
    # Changing the color of the legend text
    for text in leg.get_texts():
        text.set_color("white")
    # Change default position of axis ticks
    ax.get_xaxis().set_ticks_position('bottom')

    # Add color-bar
    cbar = fig.colorbar(cax)
    cbar.ax.set_ylabel(r'growth rate (hr$^{-1})$')
    cbar.ax.tick_params(axis='both', which='both',length=2, pad=5)

plt.savefig(figdir + 'Dekel_fitness_landscape_absolute_log.pdf',
            bbox_inches='tight')
```

## Comparison of optimal expression with mean expression profile of the simple repression motif

An interesting features to compare is how far is the optimal expression level of this fitness landscape compared to the expected mean expression level of our simple repression motif. We know that the fold-change in gene expression is given by
$$
\foldchange = \left(1 + {R \over \Nns}
              \left[ \left( 1 + {c \over K_A}\right)^2 \over
              \left( 1 + {c \over K_A}\right)^2 +
              e^{-\beta \eAI}\left( 1 + {c \over K_I}\right)^2\right]
              e^{-\beta \er}\right)^{-1},
$$
where $R$ is the number of transcriptional repressors per cell, $\Nns$ is the number of non-specific binding sites where the repressor can bind, $c$ is the concentration of the inducer, $K_A$ and $K_I$ are the dissociation constants of this inducer to the active and inactive state of the repressor, respectively, $\eAI$ is the free energy difference between the active and inactive state of the repressor, $\er$ is the repressor binding energy to the specific binding site, and $\beta \equiv (k_BT)^{-1}$.

In order to match our relative expression levels to the relative metric used by Dekel and Alon we will (incorrectly) assume that their relative expression of 1 is equivalent to our fold-change of 1. This means that we can directly compare the optimal expression to our fold-change metric.

Let's compare these two quantities.

```python
# Define inducer range to use
c = np.logspace(-4, np.log10(5), 200) # µM

# MWC induction parameters
ka = 139 / 1000 # mM
ki = 0.53 / 1000 # mM
epsilon = 4.5 # kBT

# Define repressor copy numbers to use
R_array = [22, 260, 1740]

# Define repressor/concentration matrices to evaluate fold-change
cc, RR = np.meshgrid(c, R_array)

# Define binding energy
er = -13.9  # kBT (O2 binding site)

# Compute fold-change for all these parameter combinations
fc = utils.model.fold_change_statmech(cc, RR, er, ka, ki,
                                      epsilon=epsilon)

# Compute the optimal expression level for each concentration in C_array
p_rel_opt = p_opt(c)
```

Now we can plot these expression profiles

```python
# Define colors
colors = sns.color_palette('Blues_r', n_colors=len(R_array)+2)

# Plot fold-change curves
for i, f in enumerate(fc):
    plt.plot(c, f, color=colors[i],
             label='{:d} rep./cell'.format(R_array[i]))

# Plot optimal expression profile
plt.plot(c, p_rel_opt, color='darkred',
         linestyle='--', label='optimal expression')

# Set x scale to log
plt.xscale('log')

# Label axis
plt.xlabel('[lactose] (mM)')
plt.ylabel('fold-change')

# set limits
plt.ylim([-0.05, 1.6])
# Add legend
plt.legend(loc='upper left', fontsize=6,ncol=2)
```

    <matplotlib.legend.Legend at 0x1a1f7c57f0>

# Alternative fitness landscape

Given the limitations that Uri's lanscape has, such as having a saturating relative expression set by parameter $M$ that is giving issues with the following scripts, in this section we will discuss an alternative version.

In his book Bialek discusses an alternative fitness landscape whith the following assumptions:
1. The enzyme LacZ acts in a Michaelis-Menten way, i.e.
\begin{equation}
    V = V_\max g \frac{C_{free}}{K_C + C_{free}},
\end{equation}
where $V$ and $V_\max$ are the enzyme rates and the maximum enzyme rate (in molecules per second) respectively, $K_C$ is the Michaelis constant, $C_{free}$ is the substrate concentration in souluion and $g$ is the enzyme copy number.
 2. The total substrate concentration is the sum of the free in solution and bound to the enzyme, i.e.
 \begin{equation}
     C = C_{free} + \frac{1}{\Omega} g \frac{C_{free}}{K_C + C_{free}},
 \end{equation}
 where $\Omega$ is the cell volume used to convert the enzyme copy number into concentration units.

3. Finally if the growth rate is proportional to the metabolic rate minus a linear term that represents the cost of making the enzyme we would have that our growth function $r$ is given by
\begin{equation}
    r(g, C) = \alpha g \frac{C_{free}}{K_C + C_{free}} - \beta g,
\end{equation}
where $\alpha$ is the benefit in growth per enzyme cleaved and $\beta$ is the metabolic cost of producing an enzyme.

If we solves for $C_{free}$ on the second assumption we find that
\begin{equation}
    C_{free} = \frac{1}{2} \left[ - \left( K_C + \frac{g}{\Omega} - C \right) \pm \sqrt{\left( K_C + \frac{g}{\Omega} - C \right)^2 + 4 K_C} \right],
\end{equation}
So substituting this into our function $r$ gives us the growth as a function of the enzyme and substrate concentration.

## Useful Bio-numbers.

In order to obtain reasonable numbers for this function we can always visit the beautiful bio-numbers website and other sources. The useful numbers for our case are
1. $K_C$ = 0.4 mM. Dekel and Alon paper. This is properly the constant for the LacY pump rather than LacZ, but they claim this is the relevant number for the physiology.
2. Beta-galactosidase tetramers per induced cell = 10000 [BNID:102019](http://bionumbers.hms.harvard.edu/bionumber.aspx?&id=102019&ver=3&trm=beta%20galactosidase)
3. 1000 protein / mRNA. Cell biology by the numbers.
4. $\Omega \approx$ 1 femto liter. PBoC.


(c) Juan Gomez 2019. Thanks to Universidad EAFIT for support. This material is part of the course Introduction to Finite Element Analysis

# Global assembly of the elemental FEM equilibrium equations

## Introduction

In this Notebook we describe computational details related to the final assembly of the global equilibrium equations in the finite element formulation of the theory of elasticity boundary value problem. This assembly process corresponds to the addition of the stiffness matrices corresponding to each element of the mesh considering the appropriate position of each coeeficient. **After completing this notebook you should be able to:**

* Understand the physical basis, in terms of the action-reaction principle, behind the process of assembly of global equilibrium equations.

* Recognize the main numerical operators involved in the process of assembly of global equilibrium equations.

* Implement algorithms to assemble global equilibrium equations for general elements.

## Finite element assembly

Consider the simple finite element model shown in the figure below. As discussed previously, and as a result of discretizing the PVW, the nodal forces associated to the $Q$-th degree of freedom satisfy the following equilibrium relationship:

$$
K^{QP}u^P=f_V^Q+f_t^Q.
$$

The term $K^{QP}u^P$ in this equilibrium equation corresponds to the nodal force $f_\sigma^Q$ resulting from internal forces associated to the element stresses. The total magnitude of these internal forces its due to the contribution from all the elements connecting to the node. This is exactly the same case when solving a simple spring-mass system, [see for instance Bathe(2006) Example 3.1].

The process of considering internal force terms like $f_\sigma^Q$, accounting for all the elements, and leading to the global equilibrium equations of the system is called the assembly process. The resulting internal forces for the complete system $\left\{F_\sigma\right\}$ can be written in organized form like:

$$
\left\{F_\sigma\right\}=\left[K^G\right]\left\{U^G\right\}
$$

and the equilibriun equations for the finite element model as:

$$
\left\{F_\sigma\right\}-\left\{F_V\right\}-\left\{F_t\right\}=0
$$

where $\left\{F_V\right\}$ and $\left\{F_t\right\}$ are global force vectors due to body forces and surface tractions. The assembly of the global stiffness matrix $\left[K^G\right]$ leading to the internal forces vector can be written like:

$$
\left[K^G\right]=\bigwedge_{i=1}^{Numel} k^i
$$

where $\bigwedge$ is called the **assembly operator** which loops through the $NUMEL$ elements in the mesh and adds each local coefficient matrix $k^i$. The assembly operator works like the standard summation operator $\Sigma$ but with the intrinsic inteligence of adding the terms at the right positions.

## Physical assembly

The process of assembly can be easily understood considering Newton's third law of action and reaction. This is ilustrated in the figure below where we have labeled $U_b$ those degrees of freedom along the common surface $S_b$ and $U_a$ and $U_c$ those pertaining to other regions of element $1$ and $2$ respectively.

Now, the nodal forces representing the internal stresses take the following forms in each element:

$$
\begin{Bmatrix}F_a\\F_b\end{Bmatrix} = \begin{bmatrix}K_{aa}^1&K_{ab}^1\\K_{ba}^1&K_{bb}^1\end{bmatrix}\begin{Bmatrix}U_a\\U_b\end{Bmatrix}
$$

and

$$
\begin{Bmatrix}-F_b\\F_c\end{Bmatrix}=\begin{bmatrix}K_{bb}^2&K_{bc}^2\\K_{cb}^2&K_{cc}^2\end{bmatrix}\begin{Bmatrix}U_b\\U_c\end{Bmatrix}.
$$

Using the equilibrium and compatibility conditions in terms of nodal forces and displacements given by:

\begin{align*}
& F_b^1+F_b^2=0
& U_b^1=U_b^2
\end{align*}

yields the equilibrium equations for the two element assemblage:

$$
\begin{bmatrix}K_{aa}^1&K_{ab}^1&0\\K_{ba}^1&K_{bb}^1+K_{bb}^2&K_{bc}^2\\0&K_{cb}^2&K_{cc}^2\end{bmatrix}\begin{Bmatrix}U_a\\U_b\\U_c\end{Bmatrix}=\begin{Bmatrix}F_a\\0\\F_c\end{Bmatrix}.
$$

The addition of more elements via mechanical interaction through the exposed surfaces implies the same process of canceling force terms and enforcing displacement compatibility. At the end of the process the only forces left are those introduced by surface tractions and body forces.

**Questions:**

***For the mesh shown in the figure, with internal surfaces between elements 1-3 and 3-2 labeled $S_b$ and $S_c$ respectively, write the form of the global stiffness matrix resulting from the physical assembly. Explicitly formulate the force and displacement compatibility equations along both boundaries.**

## Computational assembly

Computationally, the assembly process implies (i) identifying active and restrained degrees of freedom (dof) in the mesh (ii) assigning equation identifiers to the active degrees of freedom and (iii) identifying the contributtion from each element to the different degrees of freedom.

### Boundary conditions array IBC()

To identify active and restrained dofs the nodal data specifies a bounadry condition index to each node (see figure) with values $0$ and $-1$ specifying a free and restrained dof respectively. So the nodal data in the input file gives for each node its nodal id, the nodal coordinates in the global reference system and the boundary condition flag.

The boundary conditions data is then stored into an integer array **IBC()** which in a first instance contains only $0$s and $-1$s

$$
\begin{array}{c}0\\1\\2\\3\\4\\5\\6\\7\\8\end{array}\begin{bmatrix}0&-1\\-1&-1\\0&-1\\0&0\\0&0\\0&0\\0&0\\0&0\\0&0\end{bmatrix}
$$

and in a second instance is transformed into equation numbers:

$$
\begin{array}{c}0\\1\\2\\3\\4\\5\\6\\7\\8\end{array}\begin{bmatrix}0&-1\\-1&-1\\1&-1\\2&3\\4&5\\6&7\\8&9\\10&11\\12&13\end{bmatrix}
$$

The following two subroutines read the input (text) files (nodes, mats , elements and loads) and form the boundary conditions array **IBC()** in its two instances completing steps (i) and (ii) for the computational assembly process. This last step is performed by the subroutine **eqcounter()**.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import sympy as sym
```

**(Add comments to clarify the relevant steps ion the code below)**.

```python
def readin():
    nodes    = np.loadtxt('files/' + 'snodes.txt', ndmin=2)
    mats     = np.loadtxt('files/' + 'smater.txt', ndmin=2)
    elements = np.loadtxt('files/' + 'seles.txt', ndmin=2, dtype=np.int)
    loads    = np.loadtxt('files/' + 'sloads.txt', ndmin=2)

    return nodes, mats, elements, loads
nodes, mats, elements, loads = readin()
```

```python
def eqcounter(nodes):

    nnodes = nodes.shape[0]
    IBC = np.zeros([nnodes, 2], dtype=np.integer)
    for i in range(nnodes):
        for k in range(2):
            IBC[i , k] = int(nodes[i , k+3])
    neq = 0
    for i in range(nnodes):
        for j in range(2):
            if IBC[i, j] == 0:
                IBC[i, j] = neq
                neq = neq + 1

    return neq, IBC
neq, IBC = eqcounter(nodes)
```

### Element connectivites array IELCON()

Step (iii) in the process is completed after relating nodes in each element to the equation numbers specified in **IBC()**. The nodal points defining each element are input in a data file (see figure below). Note that each nodal identifier indicates the row in the **IBC()** array storing the equation numbers assigned to this node.

The nodal data for each element is stored in a connectivities array **IELCON()** where the row and element number coincide.

$$
\begin{array} {c}0\\1\\2\\3\end{array}\begin{bmatrix}0&1&4&3\\3&4&7&6\\4&5&8&7\\1&2&5&4\end{bmatrix}
$$

**Question:**

**Modify the node ordering in the definition of each elementand explain what would be the implications of this change in the local stiffness matrix.**

### The assembly operator DME() array

The final step in the construction of the assembly operator, termed here the **DME()** operator is just the translation of the **IELCON()** array storing nodal numbers into equation numbers stored in **IBC()**:

$$
\begin{array}{c}0\\1\\2\\3\end{array}\begin{bmatrix}0&-1&-1&-1&4&5&2&3\\2&3&4&5&10&11&8&9\\4&5&6&7&12&13&10&11\\-1&-1&1&-1&6&7&4&5\end{bmatrix}
$$

**Question:**

**(i) Use the IELCON() array together with the boundary conditions array IBC() to find the assembly operator.**

**(ii) Use a different numberig scheme for the sample mesh shown above and repeat the computation of the assebly operator.**

The **DME()** operator can now be used in a straight forward process relating local to global equations identifiers. For instance the first row of the stiffness matrix for element 2 is assembled as indicated next:

$$
\begin{align*}
K_{22}^G & \leftarrow K_{22}^G+k_{00}^2\\
K_{23}^G & \leftarrow K_{23}^G+k_{01}^2\\
K_{24}^G & \leftarrow K_{24}^G+k_{02}^2\\
K_{25}^G & \leftarrow K_{25}^G+k_{03}^2\\
K_{2,10}^G & \leftarrow K_{2,10}^G+k_{04}^2\\
K_{2,11}^G & \leftarrow K_{2,11}^G+k_{05}^2\\
K_{28}^G & \leftarrow K_{28}^G+k_{06}^2\\
K_{29}^G & \leftarrow K_{29}^G+k_{07}^2
\end{align*}
$$

The **DME()** operator is obtained by the following subroutine which takes as input arguments the nodes and elements arrays and returns the assembly operator.

**(Add comments to clarify the relevant steps in the code below)**.

```python
def DME(nodes, elements):

    nels = elements.shape[0]
    IELCON = np.zeros([nels, 4], dtype=np.integer)
    DME = np.zeros([nels, 8], dtype=np.integer)

    neq, IBC = eqcounter(nodes)
    ndof   = 8
    nnodes = 4
    ngpts  = 4
    for i in range(nels):
        for j in range(nnodes):
            IELCON[i, j] = elements[i, j+3]
            kk = IELCON[i, j]
            for l in range(2):
                DME[i, 2*j+l] = IBC[kk, l]

    return DME , IBC , neq
```

```python
DME , IBC , neq = DME(nodes, elements)
print(DME)
```

    [[ 0 -1 -1 -1  4  5  2  3]
     [ 2  3  4  5 10 11  8  9]
     [ 4  5  6  7 12 13 10 11]
     [-1 -1  1 -1  6  7  4  5]]

It was shown that the assembly involves a typical step like:

$$
K_{22}^G \leftarrow K_{22}^G+k_{00}^2
$$

which involves computation of local elemental matrices with terms $K_{ij}^q$. The following subroutine uses as input the **DME()** operator and loops through the elements of the mesh to compute the local matrix [see **UEL()**] and add its contribution into the global matrix.

**(Add comments to clarify the relevant steps ion the code below)**.

```python
def assembly(elements, mats, nodes, neq, DME, uel=None):

    IELCON = np.zeros([4], dtype=np.integer)
    KG = np.zeros((neq, neq))
    nels = elements.shape[0]
    nnodes = 4
    ndof = 8
    for el in range(nels):
        elcoor = np.zeros([nnodes, 2])
        im     = np.int(elements[el , 2])
        par0, par1 = mats[im , :]
        for j in range(nnodes):
            IELCON[j] = elements[el , j+3]
            elcoor[j, 0] = nodes[IELCON[j], 1]
            elcoor[j, 1] = nodes[IELCON[j], 2]
        kloc = uel4nquad(elcoor, par1, par0)
        dme = DME[el, :ndof]
        for row in range(ndof):
            glob_row = dme[row]
            if glob_row != -1:
                for col in range(ndof):
                    glob_col = dme[col]
                    if glob_col != -1:
                        KG[glob_row, glob_col] = KG[glob_row, glob_col] +\
                                                 kloc[row, col]

    return KG
```

In this case we have assumed that the elemental subroutine produces a stiffness matrix filled with $1$s.

**(Complete this suboroutine with the implementation performed in NB 8)**.

```python
def uel4nquad(coord, enu, Emod):

    kl = np.ones([8, 8])
    return kl
```

```python
KG = assembly(elements, mats, nodes, neq, DME)
print(KG)
```

    [[1. 0. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0.]
     [0. 1. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
     [1. 0. 2. 2. 2. 2. 0. 0. 1. 1. 1. 1. 0. 0.]
     [1. 0. 2. 2. 2. 2. 0. 0. 1. 1. 1. 1. 0. 0.]
     [1. 1. 2. 2. 4. 4. 2. 2. 1. 1. 2. 2. 1. 1.]
     [1. 1. 2. 2. 4. 4. 2. 2. 1. 1. 2. 2. 1. 1.]
     [0. 1. 0. 0. 2. 2. 2. 2. 0. 0. 1. 1. 1. 1.]
     [0. 1. 0. 0. 2. 2. 2. 2. 0. 0. 1. 1. 1. 1.]
     [0. 0. 1. 1. 1. 1. 0. 0. 1. 1. 1. 1. 0. 0.]
     [0. 0. 1. 1. 1. 1. 0. 0. 1. 1. 1. 1. 0. 0.]
     [0. 0. 1. 1. 2. 2. 1. 1. 1. 1. 2. 2. 1. 1.]
     [0. 0. 1. 1. 2. 2. 1. 1. 1. 1. 2. 2. 1. 1.]
     [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 1. 1. 1. 1.]
     [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 1. 1. 1. 1.]]

**Question:**

**For the mesh shown in the figure propose different node numbering schemes and identify the resulting changes in the size of the half-band in the stiffness matrix. Assume that each element subroutine is full of $1$s.**

### Glossary of terms.

**Boundary conditions array IBC():** Integer type array storing equation numbers assigned to each nodal point in the mesh.

**Connectivity array IELCON():** Integer type array storing identifiers for the nodal points defining each element in the mesh.

**Assembly:** Computational procedure by which the elemental stiffness matrix are properly added together to form the global stiffness matrix.

**Assembly operator DME():** Integer type array storing the nodal connectivities from each element but translated into equation numbers through the boudnary conditions array **IBC()**.

## Class activity.

* (i) Use the subroutines developed previously to compute the stiffness matrix of bi-linear and cuadratic finite elements to compute the global stiffness matrix for the sample problem discussed in this notebook and with the input files **Snodes.txt and Selements.txt** provided.

* (ii) Assume nodal values for the active displacemnts and use the global matrix found in step (i) to find the internal forces vector $\left\{F_\sigma\right\}$ consistent with the element stresses.

* (iii) Repeat step (ii) but instead of assuming known nodal displacements find them after applying point forces along degrees of freedom $9$, $11$ and $13$ and solving the system of equations:

$$
\left[K^G\right]\left\{U^G\right\} = \left\{F\right\}.
$$

* (iv) Verify that the nodal displacemnts $U^G$ found in step (iii) produce internal forces $\left\{F_\sigma\right\}$ in equilibrium with the external forces $\left\{F\right\}.$

### References

* Bathe, Klaus-Jürgen. (2006) Finite element procedures. Klaus-Jurgen Bathe. Prentice Hall International.

* Juan Gómez, Nicolás Guarín-Zapata (2018). SolidsPy: 2D-Finite Element Analysis with Python, <https://github.com/AppliedMechanics-EAFIT/SolidsPy>.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


# **Noise in Image Restoration**
## Muhammad Reza Fahlevi (NIM: 181401139)
----

### Definisi Image
Image the didefinisikan sebagai suatu intesitas I pixel citra pada posisi (x,y), yaitu $I(x,y)$. Secara umum, citra terbagi atas 3 kategori:

* Citra biner (monochrome)
* Citra abu-abu (grayscale)
* Citra berwarna

Citra biner memiliki intensitas pixel 0 (nilai yang paling gelap) dan 255 (nilai yang paling terang). Citra abu - abu merupakan citra yang nilai pixelnya pada posisi $(x, y)$ merupakan rata-rata dari nilai rgbnya. Sedangkan citra berwarna adalah citra yang setiap pixel pada $(x, y)$ memiliki 3 nilai, yaitu $(red, green, blue)$
$$
\begin{align}
f(x, y) &=
\begin{bmatrix}
r(x, y) \\
g(x, y) \\
b(x, y)
\end{bmatrix}
\end{align}
$$

### Model Degradasi Image
Degradasi pada image dapat dimodelkan sebagai berikut.

Jika $h(x, y)$ adalah fungsi degradasi, $\eta(x, y)$ adalah fungsi noise. Maka, image terdegradasi $g(x, y)$ untuk inputan image $\phi(x,y)$ di definisikan sebagai
$$
g(x, y) = h(x, y) \ast \phi(x, y) + \eta(x, y)
$$

Yang mana $\ast$ merupakan operator konvulusi. Restorasi merupakan proses perbaikan suatu citra yang pada pixelnya terdapat noise. Untuk merestorasi citra, maka kita harus menurunkan (mendegradasi) citra tersebut.

### Noise pada Image
Didefinisikan sebagai:
$$
n(x, y) = f(x, y) + \eta(x, y)
$$
Yang mana $n(x, y)$ adalah citra yang terdapat noise dan $f(x, y)$ adalah citra original (atau citra yang diharapkan, tanpa noise).

Pada notebook ini, noise yang akan dibahas adalah:

* Gaussian noise
* Impulse noise (salt and pepper)
* Poisson noise
* Speckel noise

### Gaussian Noise
Gaussian noise adalah noise pada citra $f(x, y)$ yang terdistribusi normal.

**Definition $\text{[Gauss Distribution]}$.** Jika $\mu$ adalah rata-rata dan $\sigma^2$ adalah variasi, maka distribusi untuk random variable X adalah
$$
G(x; \mu, \sigma) = \frac{1}{\sqrt{2 \pi}\sigma} exp\left\{-\frac{1}{2 \sigma^2}(x - \mu)^2 \right\}
$$
atau
$$
G(x, y; \mu, \sigma) = \frac{1}{2 \pi\sigma^2} exp\left\{-\frac{1}{2 \sigma^2}(x^2 + y^2) \right\}
$$
untuk noise pada citra $f(x, y)$

### Impulse Noise
Didefinisikan sebagai
$$
\begin{align}
P(z) &= \begin{cases}
P_a && \text{if } z = a \\
P_b && \text{if } z = b
\end{cases}
\end{align}
$$

### Poisson Noise
Didefinisikan sebagai
$$
p(x; \lambda t) = \frac{e^{\lambda t}(\lambda t)^x}{x!}
$$

### Speckel Noise
Speckle noise can be generated by multiplying random pixel values with different pixels of an image.

```python
#Prepare the tools
import numpy as np
import random
print("Initialize.... \nDone.")
```

    Initialize....
    Done.

```python
Pixel = np.matrix([
    [123, 321, 123],
    [123, 532, 123],
    [98, 12, 321]
])
Noise = np.random.normal(0, 2, size= (3, 3))
Noise
print(f"Pixel: \n{Pixel}\nNoise:\n {Noise} \nPixel with noise\n {Pixel + Noise}")
```

    Pixel:
    [[123 321 123]
     [123 532 123]
     [ 98  12 321]]
    Noise:
     [[ 0.49848639 -1.30062922  1.36659205]
     [-5.53448321 -0.19500632  0.69103171]
     [-1.41848636 -0.48238745  1.69381804]]
    Pixel with noise
     [[123.49848639 319.69937078 124.36659205]
     [117.46551679 531.80499368 123.69103171]
     [ 96.58151364  11.51761255 322.69381804]]

```python

```


# <center> ЛАБОРАТОРНАЯ  РАБОТА 3 </center>
# <center> РЕШЕНИЕ СИСТЕМ ЛИНЕЙНЫХ АЛГЕБРАИЧЕСКИХ УРАВНЕНИЙ ПРЯМЫМИ МЕТОДАМИ. ТЕОРИЯ ВОЗМУЩЕНИЙ
 </center>

Теоретический материал к данной теме содержится в [1, глава 5].

```python
from sympy.solvers import solve
from sympy import exp, sin, cos, sqrt, log, ln, pi
from sympy import Rational as syR
from sympy import Symbol, diff
from scipy.misc import derivative

import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline
```

## Задание 1
Дана система уравнений $Ax=b$ порядка n. Исследовать зависимость погрешности решения x от погрешностей  правой части системы b.
## Порядок решения задачи:
1. Задать матрицу системы A и вектор правой части b. Составить программу, реализующую метод Гаусса (схема частичного выбора) для произвольной системы $Ax=b$. Используя составленную программу, найти решение заданной системы $Ax=b$.
2. С помощью встроенной функции вычислить число обусловленности матрицы  A.
3. Принимая решение x, полученное в п. 1, за точное, вычислить вектор $d=(d_1,...,d_n)^T$, $$d_i=\frac{||x-x^i||_{\infty}}{||x||_{\infty}},\quad i=1, ..., n,$$
относительных погрешностей решений $x^i$ систем $Ax^i=b^i$ , где  компоненты векторов $b^i$ вычисляются по формулам:
\begin{equation*}
 b^{i}_{k}=
 \begin{cases}
   b_k+\Delta ,&\text{k=i}\\
   b_k ,&\text{$k\neq i$}
 \end{cases}
 ;k=1,...,n
\end{equation*}
( $\Delta$ - произвольная величина погрешности).
4.  На основе вычисленного вектора d построить гистограмму. По гистограмме определить компоненту $b_m$  вектора b, которая оказывает наибольшее влияние на погрешность решения.
5. Оценить теоретически погрешность решения $x^m$ по  формуле:$$\delta(x^m)<= cond(A)*\delta(b^m)$$
Сравнить значение $\delta(x^m)$ со значением практической погрешности $d_m$. Объяснить полученные результаты.

Компоненты вектора  b задаются формулой  $b_i=N$ ,  $\forall i=1...n$,  коэффициенты   $c=c_{ij}=0.1* N *i*j$ ,$\forall i,j=1,...,n$

Ваши компоненты вектора и матрицы:
$c(i,j) = {Task1[TASK_VARIANT][f_latex]}$
$a(i,j) = {Task1[TASK_VARIANT][f_latex]}$

$N={[Task1[TASK_VARIANT][N]};$

```python
epsilon1 = 10**(-5)
# Компоненты
c1 = {Task1[TASK_VARIANT][f_latex]}
a1 = {Task1[TASK_VARIANT][f_latex]}

N1 = {Task1[TASK_VARIANT][N]}
```

Решите систему с помощью встроенной функции np.linalg.solve

```python

```

Напишите функцию meth_Gauss, которая реализует метод Гаусса

```python
def meth_Gauss(matrix, string_b):

    return return np.array([b[i]/A[i, i] for i in range(n)])
```

Определите обусловленность матрицы с помощью встроенной функции np.linalg.cond

```python
##Ответ-число обсуловленности
```

Реализуйте функцию для вычисления вектора d

```python
##решение
```

Постройте гистограмму, используя пакет matplotlib. Укажите компоненту вектора b, которая больше всего влияет на погрешность решения

```python
##график
```

Оцените погреность решения по формуле:
$$\delta(x^m)<= cond(A)*\delta(b^m)$$

```python
##массив тру
```


To generate a presentation from these slides, run:

`jupyter nbconvert PRC-presentation.ipynb --to slides --post serve SlidesExporter.reveal_theme=serif SlidesExporter.reveal_scroll=True SlidesExporter.reveal_transition=none`

# PRC: Implementing a Novel Machine Learning Method in Julia
## John Waczak
## May 4 2021

# Outline
1. Problem Description
2. Implementation Details
3. Model Application
4. Next Steps
5. What Worked and What Didn't
6. Lessons Learned
7. Topics Suggestions

# 1. Problem Description

- **Abstract Problem**: Machine learning, particularly deep neural networks, tend to be black box solutions. It's hard to understand what exactly is happening to your data.
- **Specific Research Problem**: Using frequency decomposition of 64 electrode EEG measurements to predict blinking, specifically, eye aspect ratio (EAR)
- **Solution**: Develop a new neural network layer based on a simple binning procedure

Partition the domain of your function into $N$ bins with $N+1$ bin edges. Model a bin by a smooth step function:
\begin{equation}
    B(x) = \frac{1}{2}\left(\tanh(\alpha(x-b_l))-\tanh(\alpha(x-b_r)) \right)
\end{equation}
where $\alpha$ controlls the steepness (a hyperparameter) and $b_l$ and $b_r$ are the left and right bin bin edges, respectively.

Action of the layer is as follows:

# 2. Implementation Details

**Language of choice:** `Julia`
**Why?** The neural network library in Julia, `Flux.jl`, is implemented **100%** in Julia:

Popular libraries like tensorflow are built on top of lower level C++ implementations. This makes it harder to implement new models.

## Summary:
- Binning layer implemented in Julia by extending `Flux.jl`
- Code version controlled via `github` and `gitlab`
- Package envrionment managed via Julia, i.e. we version control the `Project.toml` and `Manifest.toml` files. This is similar to the Conda environments we talked about in class.
- Code organized into standard package structure via templates from `PkgTemplates.jl`
- Added fancy docstrings to allow for `help` lookup at the REPL
- Began writing tests that are stored in `/tests/`
- CI/CD implemented via Travis.ci by adding a simple `.travis.yaml` file

## Easy package install from github

## Environment Management

## Help from fancy dosctrings

## Running Tests

## CI/CD via Travis

# 3. Model Application

**Demo Problem**: bin optimization for maintaining integral.
- Sample sine wave from $x=0$ to $x=10$ with $1000$ points
- Use `DomainBinner` to bin the 1000 points down to 20
- Optimize bin edges to maintain the value of the integral

**Problem 2**: Predicting Eye Aspect Ratio (EAR) value from binned EEG frequency data.

**Model**: Binning layer to reduce 257 frequencies down to 100. Output of binning layer connected to a dense layer with 100 nodes. The dense nodes are then connected to estimate the Ear value

### Results so far
- The `DomainBinner` is able to function together with the `Dense` layer *out of the box* via the `Chain()` function
- `Flux` is able to track the gradients as expected and update the bin edges
- A single electrode is insufficient to predict the EAR value (as expected)

Output of evaluation scripts are captured and converted into nice human readable html/markdown documents using `Weave.jl`.

# 4. Next Steps
- Create slurm scritps to train models on Europa
- Train more reliable models by utilizing multiple electrodes simultaneously
- Perform a hyperparamter optimization (number of bins, number of nodes in hidden layer, etc...)
- Apply binning layer to frequency dimension of Hyper-Spectral Images
- Explore using tensorboard for live tracking of model convergence
- Optimize the code for performance (I think Julia is actually column-major, unlike Python so I should probably transpose my data for faster training...)
- If this actually proves usefull, submit a PR and add the layer to the `Flux` ecosystem.

# 5. How did you start? What worked? What didn't work?
- Started development of layer via notebooks
- The documentation for `Flux.jl` was super helpful for making a new layer that works with the package
- Had issues early on trying to get the auto-diff to work on batches of data.
- Had trouble coming up with a simple toy problem to test the model on. Eventually was able to settle on the integraal test.

# 6. Lessons Learned
The two big takeaways for me were:
- I'm finally starting to appreciate the value of writing tests for everything. This was super helpful when I was trying to get the auto-diff to work on batches of data. It also helped me catch when I made code-breaking changes (specifically array shape related issues).
- Using new environments for new projects. This was super helpful for verifying that the code works across multiple machines.

# 7. Topic Suggestions
- The version control tutorials were super helpful. It would be nice to add some additional content for the proper way to contribute to larger codes you don't own. Maybe you could add forking and submitting pull requests to the git homework assignment.
- I would personally be interested in a lecture on containers. I've had plenty of issues trying to work across multiple operating systems and I think this could help.

```julia

```