```python
import numpy as np
import scipy.stats as spst
from tqdm import tqdm
from sif.models.sigmoid import sigmoid
from sif.kernels import BrownianMotionKernel
from sif.samplers import multivariate_normal_sampler
```

```python
import matplotlib.pyplot as plt
from matplotlib import cm
%matplotlib inline
```

In this notebook I will apply Girsanov's Theorem to Langevin dynamics for a Bayesian logistic regression problem. Recall that Girsanov's Theorem says that if $W_t$ is a Brownian motion and $\Theta_t$ is a stochastic process adapted to that Brownian motion, then we may define the Radon-Rikodym derivative process
\begin{align}
Z(t) &= \exp\left(-\int_0^t \Theta_s dW_s - \frac{1}{2} \int_0^t \Theta_s^2 ds\right) \\
Z &\equiv Z(T)
\end{align}
and under the probability measure $\tilde{\mathbb{P}}(A)= \int_A Z(\omega) d\mathbb{P}(\omega)$ the process
\begin{align}
\tilde{W}_t &= W_t + \int_0^t \Theta_s ds
\end{align}
is a Brownian motion.

Consider the Langevin equation for sampling from a distribution $p(x)\propto \exp\left(-V(x)\right)$:
\begin{align}
dX_t = -\frac{1}{2}\nabla V(X_t) dt + dW_t
\end{align}
Integrating from differential notation to integral notation this says,
\begin{align}
X_t - X_0 = W_t + \int_0^t \left(-\frac{1}{2}\nabla V(X_s)\right) ds
\end{align}
which, when $X_0\equiv 0$ resembles the Girsanov Brownian motion with adapted process $\Theta_t \equiv -\frac{1}{2}\nabla V(X_t)$.

First I'll generate some random data from a logistic regression model using a two-dimensional known coefficient vector. I'll also define a function for computing the gradient of log-posterior.

```python
n = 100
loss_div = 1
beta_star = np.array([2, -1])
X = np.random.normal(scale=2, size=(n, len(beta_star)))
y = np.random.binomial(1, sigmoid(X.dot(beta_star)))
```

```python
def grad_V(beta, lambda_reg=1.):
    y_tile = np.tile(np.atleast_2d(y).T, beta.shape[0])
    ll_grad = -X.T.dot(y_tile - sigmoid(X.dot(beta.T)))
    ll_grad = ll_grad.T
    reg_grad = lambda_reg * beta
    return (ll_grad + reg_grad) / loss_div
```

I set the learning rate to some small number. I'll also choose a number of trials to perform that is inversely proportional to the selected learning rate. This has the effect of running the stochastic dynamics forward in time until one unit of time has elapsed.

```python
dt = 1e-3 / n
time_units = 3/n
n_iter = int(time_units * 1/dt)
n_trials = 5000
```

Create arrays that contain the Brownian differential and the Brownian motion. Initialize arrays that will contain the gradient process and the Langevin dynamics process.

```python
dB = np.random.normal(scale=np.sqrt(dt / loss_div), size=(n_trials, n_iter + 1, X.shape[1]))
B = np.cumsum(dB[:, :-1, :], axis=1)
Theta = np.zeros(B.shape)
Beta = np.zeros(B.shape)
# Beta[:, 0, :] = beta_star
beta_init = np.array([ 1.55452502, -0.78723773])
Beta[:, 0, :] = beta_init
```

```python
for i in tqdm(range(1, n_iter)):
    beta = Beta[:, i-1]
    Theta[:, i] = -0.5*grad_V(beta)
    Beta[:, i] = beta + Theta[:, i]*dt + dB[:, i]#/np.sqrt(loss_div)
```

    100%|██████████| 2998/2998 [00:47<00:00, 62.98it/s]

```python
Beta.mean(axis=0)
```

    array([[ 1.55452502, -0.78723773],
           [ 1.55456543, -0.78719167],
           [ 1.55459717, -0.78716402],
           ...,
           [ 1.53052865, -0.74073385],
           [ 1.53053893, -0.74070526],
           [ 1.53055269, -0.74070368]])

Compute the Radon-Nikodym derivative process.

```python
alpha = -np.cumsum((Theta*dB[:, :-1, :]).sum(axis=2), axis=1)
# alpha = -np.cumsum((Theta*np.random.normal(scale=np.sqrt(dt), size=Theta.shape)).sum(axis=2), axis=1)
beta = -0.5 * np.cumsum((Theta**2).sum(axis=2) * dt, axis=1)
delta = np.cumsum(Theta*dt, axis=1)
B_tilde = B + delta - dB[:, [0], :]
```

```python
Z = np.exp(alpha + beta)
```

```python
print(Z.mean(axis=0))
```

    [1.         0.99998973 1.00000492 ... 1.0034745  1.00330653 1.00328201]

```python
print(B_tilde + beta_init - Beta)
```

    [[[ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00 -1.11022302e-16]
      ...
      [ 2.66453526e-15  3.33066907e-15]
      [ 2.88657986e-15  3.44169138e-15]
      [ 2.88657986e-15  3.44169138e-15]]

     [[ 0.00000000e+00  0.00000000e+00]
      [-2.22044605e-16  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      ...
      [-4.66293670e-15  4.44089210e-16]
      [-4.66293670e-15  5.55111512e-16]
      [-4.44089210e-15  5.55111512e-16]]

     [[ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      [ 2.22044605e-16  0.00000000e+00]
      ...
      [-2.88657986e-15  2.66453526e-15]
      [-2.88657986e-15  2.66453526e-15]
      [-2.88657986e-15  2.77555756e-15]]

     ...

     [[ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      ...
      [-4.88498131e-15 -2.44249065e-15]
      [-5.10702591e-15 -2.55351296e-15]
      [-4.88498131e-15 -2.55351296e-15]]

     [[ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      ...
      [ 8.88178420e-16 -9.99200722e-16]
      [ 8.88178420e-16 -9.99200722e-16]
      [ 8.88178420e-16 -8.88178420e-16]]

     [[ 0.00000000e+00  0.00000000e+00]
      [ 0.00000000e+00  0.00000000e+00]
      [-2.22044605e-16 -1.11022302e-16]
      ...
      [ 8.88178420e-15  1.05471187e-15]
      [ 8.88178420e-15  1.05471187e-15]
      [ 9.10382880e-15  9.99200722e-16]]]

```python
plt.figure(figsize=(17, 6))
T = np.linspace(0, time_units, n_iter)

plt.subplot(1, 2, 1)
for i in range(min(500, n_trials)):
    plt.plot(T, B[i, :, 0], "b-", alpha=0.05)
plt.plot(T, B[:, :, 0].mean(axis=0), "r--")
plt.grid()
plt.xlabel("$t$", fontsize=18)
plt.ylabel("$B_t$", fontsize=18)
plt.title("Brownian Motion", fontsize=20)

plt.subplot(1, 2, 2)
for i in range(min(500, n_trials)):
    plt.plot(T, Z[i], "b-", alpha=0.05)
plt.plot(T, Z.mean(axis=0), "r--")
plt.grid()
plt.xlabel("$t$", fontsize=18)
plt.ylabel("$Z_t$", fontsize=18)
plt.title("Radon-Nikodym Derivative Process", fontsize=20)

plt.show()
```

```python
plt.figure(figsize=(17, 4))

plt.subplot(1, 2, 1)
plt.plot(T.ravel(), B[:, :, 0].mean(axis=0), label="$P$")
plt.plot(T.ravel(), (B_tilde[:, :, 0] * Z).mean(axis=0), label="$\\tilde{P}$")
plt.grid()
plt.gca().set_title("Expectation", fontsize=20)
plt.legend(fontsize=16)
plt.xlabel("$t$", fontsize=18)

plt.subplot(1, 2, 2)
plt.plot(T.ravel(), B[:, :, 0].var(axis=0))
plt.plot(T.ravel(), ((B_tilde[:, :, 0]**2) * Z).mean(axis=0) - (B_tilde[:, :, 0] * Z).mean(axis=0)**2)
plt.plot(T.ravel(), Beta[:, :, 0].var(axis=0))
plt.grid()
plt.xlabel("$t$", fontsize=18)
plt.gca().set_title("Variance", fontsize=20)

plt.show()
```

```python
W_tilde = B_tilde[:, :, 0]
R = np.outer(np.mean(W_tilde * Z, axis=0), np.mean(W_tilde * Z, axis=0))
S = W_tilde.T.dot(W_tilde * Z) / n_trials
i_lower = np.tril_indices(R.shape[0], -1)
S[i_lower] = S.T[i_lower]
K_tilde = S - R
```

```python
kernel = BrownianMotionKernel()
C = kernel.cov(np.atleast_2d(T).T)
K = np.cov(Beta[:, :, 0].T)
```

```python
plt.figure(figsize=(17, 6))
plt.subplot(1, 3, 1)
plt.imshow(K_tilde)
plt.xlabel("$t$", fontsize=16)
plt.ylabel("$s$", fontsize=16)
plt.title("Covariance of Girsanov Brownian Motion")
plt.colorbar()
plt.subplot(1, 3, 2)
plt.imshow(C - K_tilde)
plt.xlabel("$t$", fontsize=16)
plt.ylabel("$s$", fontsize=16)
plt.title("Covariance Difference for Girsanon Brownian Motion")
plt.colorbar()
plt.subplot(1, 3, 3)
plt.imshow(C - K)
plt.xlabel("$t$", fontsize=16)
plt.ylabel("$s$", fontsize=16)
plt.title("Covariance Difference for Langevin Dynamics")
plt.colorbar()
plt.show()
```

```python

```

```python

```

```python

```

```python

```

examples/sampling/Bayesian Logistic Regression with Girsanov's Theorem.ipynb


## Подготовка:

```python
import numpy as np
from numpy.linalg import *
rg = matrix_rank
from IPython.display import display, Math, Latex, Markdown

from sympy import *
from sympy import Matrix, solve_linear_system
```

```python
pr = lambda s: display(Markdown('$'+str(latex(s))+'$'))
def psym_matrix(a, intro='',ending='',row=False):
    try:
        if row:
                return(intro+str(latex(a))+ending)
        else:
                display(Latex("$$"+intro+str(latex(a))+ending+"$$"))
    except TypeError:
        display(latex(a)) #TODO MAY BY...
pr = lambda s: display(Markdown('$'+str(latex(s))+'$'))

def pmatrix(a, intro='',ending='',row=False):
    if len(a.shape) > 2:
        raise ValueError('pmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{pmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{pmatrix}']
    if row:
        return(intro+'\n'.join(rv)+ending)
    else:
        display(Latex('$$'+intro+'\n'.join(rv)+ending+'$$'))
def crearMatrix(name,shape=(2,2)):
    X = []
    for i in range(shape[0]):
        row = []
        for j in range(shape[1]):
            row.append(Symbol(name+'_{'+str(i*10+j+11)+'}'))
        X.append(row)
    return Matrix(X)
from re import sub
def matrix_to_word(A):
    replaced = sub(r"begin{matrix}", r"(■( ", psym_matrix(A,row=1))
    replaced = sub(r"{", r"", replaced)
    replaced = sub(r"}", r"", replaced)
    replaced = sub(r"\\\\\\\\", r" @ ", replaced)
    replaced = sub(r"\\\\", r" @ ", replaced)
    replaced = sub(r"[\[\]]", r"", replaced)
    replaced = sub(r"left", r"", replaced)
    replaced = sub(r".endmatrix.right", r" ))", replaced)
    replaced = sub(r"\\", r"", replaced)
    print(replaced )
```

# 3 пункт
## пример:
### Дано:

```python
X = np.array([[Symbol("x_{11}"), Symbol("x_{12}")],[Symbol("x_{21}"),Symbol("x_{22}")]])
pmatrix(X, intro="X=")
```

$$X=\begin{pmatrix}
  x_{11} & x_{12}\\
  x_{21} & x_{22}\\
\end{pmatrix}$$

```python
B = np.array([[5,1],
              [5,2]])

C1 = np.array([[1,1],
               [1,2]])
D1 = np.array([[2,1],
               [1,0]])

C2 = np.array([[1,-1],
               [0,1]])

D2 = np.array([[1,1],
               [0,1]])

```

```python
W = np.array([
    [0,2,1,6],
    [0,0,1,2],
    [0,0,0,3],
    [0,0,0,0]
])
print(rg(W))
```

    3

### Распишим  $$C^1XD^1+C^2XD^2$$

```python
A = (C1.dot(X)).dot(D1) + (C2.dot(X)).dot(D2)
pmatrix(A)
```

$$\begin{pmatrix}
  3*x_{11} & + & x_{12} & + & x_{21} & + & x_{22} & 2*x_{11} & + & x_{12} & - & x_{22}\\
  2*x_{11} & + & x_{12} & + & 5*x_{21} & + & 2*x_{22} & x_{11} & + & 3*x_{21} & + & x_{22}\\
\end{pmatrix}$$

### Приведём к виду Ax=b, где x,b вектора

```python
F = np.array([[3,1,1,1],
              [2,1,0,-1],
              [2,1,5,2],
              [1,0,3,1]])
pmatrix(F, ending=pmatrix(X.reshape((4, 1)),row=True)+"="+pmatrix(B.reshape((4, 1)),row=True))
```

$$\begin{pmatrix}
  3 & 1 & 1 & 1\\
  2 & 1 & 0 & -1\\
  2 & 1 & 5 & 2\\
  1 & 0 & 3 & 1\\
\end{pmatrix}\begin{pmatrix}
  x_{11}\\
  x_{12}\\
  x_{21}\\
  x_{22}\\
\end{pmatrix}=\begin{pmatrix}
  5\\
  1\\
  5\\
  2\\
\end{pmatrix}$$

```python
pmatrix(rg(F), intro=r'rg(F)=', ending="\Leftrightarrow\exists F^{-1}\sum") #ранг
```

$$rg(F)=\begin{pmatrix}
  4\\
\end{pmatrix}\Leftrightarrow\exists F^{-1}\sum$$

### Решаем это:

```python
from sympy.abc import a,b,c,d
system = Matrix(( (3,1,1,1,5), (2,1,0,-1,1), (2,1,5,2,5),(1,0,3,1,2) ))
x = solve_linear_system(system, a,b,c,d)
X =np.array([[x[a],x[b]],[x[c],x[d]] ])
pmatrix(X,intro="X=")
```

$$X=\begin{pmatrix}
  10/11 & 9/11\\
  -2/11 & 18/11\\
\end{pmatrix}$$

# 3 пункт обощение:

## Рассмотрим $$C^1XD^1+C^2XD^2=B$$

### Обозначения:

#### Примечание: $$С^n,D^n,\textbf{ где n не степень!} $$

```python
X = crearMatrix("x")
B = crearMatrix("b")

D1= crearMatrix("d^1")
C1= crearMatrix("c^1")
D2= crearMatrix("d^2")
C2= crearMatrix("c^2")

psym_matrix(X, intro="X=")
psym_matrix(B, intro="B=")

psym_matrix(D1, intro="D^1=")
psym_matrix(C1, intro="C^1=")
psym_matrix(D2, intro="D^2=")
psym_matrix(C2, intro="C^2=")
```

$$X=\left[\begin{matrix}x_{11} & x_{12}\\x_{21} & x_{22}\end{matrix}\right]$$

$$B=\left[\begin{matrix}b_{11} & b_{12}\\b_{21} & b_{22}\end{matrix}\right]$$

$$D^1=\left[\begin{matrix}d^1_{11} & d^1_{12}\\d^1_{21} & d^1_{22}\end{matrix}\right]$$

$$C^1=\left[\begin{matrix}c^1_{11} & c^1_{12}\\c^1_{21} & c^1_{22}\end{matrix}\right]$$

$$D^2=\left[\begin{matrix}d^2_{11} & d^2_{12}\\d^2_{21} & d^2_{22}\end{matrix}\right]$$

$$C^2=\left[\begin{matrix}c^2_{11} & c^2_{12}\\c^2_{21} & c^2_{22}\end{matrix}\right]$$

### Символьно посчитаем выражение
$$A = C^1XD^1+C^2XD^2$$

```python

A = C2*X*D1+C2*X*D2
psym_matrix(crearMatrix("a"), intro="A=")
```

$$A=\left[\begin{matrix}a_{11} & a_{12}\\a_{21} & a_{22}\end{matrix}\right]$$

```python
psym_matrix(A)
```

$$\left[\begin{matrix}d^1_{11} \left(c^2_{11} x_{11} + c^2_{12} x_{21}\right) + d^1_{21} \left(c^2_{11} x_{12} + c^2_{12} x_{22}\right) + d^2_{11} \left(c^2_{11} x_{11} + c^2_{12} x_{21}\right) + d^2_{21} \left(c^2_{11} x_{12} + c^2_{12} x_{22}\right) & d^1_{12} \left(c^2_{11} x_{11} + c^2_{12} x_{21}\right) + d^1_{22} \left(c^2_{11} x_{12} + c^2_{12} x_{22}\right) + d^2_{12} \left(c^2_{11} x_{11} + c^2_{12} x_{21}\right) + d^2_{22} \left(c^2_{11} x_{12} + c^2_{12} x_{22}\right)\\d^1_{11} \left(c^2_{21} x_{11} + c^2_{22} x_{21}\right) + d^1_{21} \left(c^2_{21} x_{12} + c^2_{22} x_{22}\right) + d^2_{11} \left(c^2_{21} x_{11} + c^2_{22} x_{21}\right) + d^2_{21} \left(c^2_{21} x_{12} + c^2_{22} x_{22}\right) & d^1_{12} \left(c^2_{21} x_{11} + c^2_{22} x_{21}\right) + d^1_{22} \left(c^2_{21} x_{12} + c^2_{22} x_{22}\right) + d^2_{12} \left(c^2_{21} x_{11} + c^2_{22} x_{21}\right) + d^2_{22} \left(c^2_{21} x_{12} + c^2_{22} x_{22}\right)\end{matrix}\right]$$

### Выражаем вектор x из A:

#### пришлось вручную...

```python
F = Matrix([ # How???
    [ D1[0,0]*C1[0,0]+D2[0,0]*C2[0,0], D1[1,0]*C1[0,0]+D2[1,0]*C2[0,0], D1[0,0]*C1[0,1]+D2[0,0]*C2[0,1], D1[1,0]*C1[0,1]+D2[1,0]*C2[0,1]],
    [ D1[0,0]*C1[0,0]+D2[0,1]*C2[0,0], D1[1,1]*C1[0,0]+D2[1,1]*C2[0,0], D1[0,1]*C1[0,1]+D2[0,1]*C2[0,1], D1[1,1]*C1[0,1]+D2[1,1]*C2[0,1]],
    [ D1[0,1]*C1[1,0]+D2[0,0]*C2[1,0], D1[1,0]*C1[1,0]+D2[1,0]*C2[1,0], D1[0,0]*C1[1,1]+D2[0,0]*C2[1,1], D1[1,0]*C1[1,1]+D2[1,0]*C2[1,1]],
    [ D1[0,1]*C1[1,0]+D2[0,1]*C2[1,0], D1[1,1]*C1[1,0]+D2[1,1]*C2[1,0], D1[0,1]*C1[1,1]+D2[0,1]*C2[1,1], D1[1,1]*C1[1,1]+D2[1,1]*C2[1,1]]
])
```

```python
psym_matrix(F,intro="F=")
```

$$F=\left[\begin{matrix}c^1_{11} d^1_{11} + c^2_{11} d^2_{11} & c^1_{11} d^1_{21} + c^2_{11} d^2_{21} & c^1_{12} d^1_{11} + c^2_{12} d^2_{11} & c^1_{12} d^1_{21} + c^2_{12} d^2_{21}\\c^1_{11} d^1_{11} + c^2_{11} d^2_{12} & c^1_{11} d^1_{22} + c^2_{11} d^2_{22} & c^1_{12} d^1_{12} + c^2_{12} d^2_{12} & c^1_{12} d^1_{22} + c^2_{12} d^2_{22}\\c^1_{21} d^1_{12} + c^2_{21} d^2_{11} & c^1_{21} d^1_{21} + c^2_{21} d^2_{21} & c^1_{22} d^1_{11} + c^2_{22} d^2_{11} & c^1_{22} d^1_{21} + c^2_{22} d^2_{21}\\c^1_{21} d^1_{12} + c^2_{21} d^2_{12} & c^1_{21} d^1_{22} + c^2_{21} d^2_{22} & c^1_{22} d^1_{12} + c^2_{22} d^2_{12} & c^1_{22} d^1_{22} + c^2_{22} d^2_{22}\end{matrix}\right]$$

```python
matrix_to_word(F)
```

    (■( c^1_11 d^1_11 + c^2_11 d^2_11 & c^1_11 d^1_21 + c^2_11 d^2_21 & c^1_12 d^1_11 + c^2_12 d^2_11 & c^1_12 d^1_21 + c^2_12 d^2_21 @ c^1_11 d^1_11 + c^2_11 d^2_12 & c^1_11 d^1_22 + c^2_11 d^2_22 & c^1_12 d^1_12 + c^2_12 d^2_12 & c^1_12 d^1_22 + c^2_12 d^2_22 @ c^1_21 d^1_12 + c^2_21 d^2_11 & c^1_21 d^1_21 + c^2_21 d^2_21 & c^1_22 d^1_11 + c^2_22 d^2_11 & c^1_22 d^1_21 + c^2_22 d^2_21 @ c^1_21 d^1_12 + c^2_21 d^2_12 & c^1_21 d^1_22 + c^2_21 d^2_22 & c^1_22 d^1_12 + c^2_22 d^2_12 & c^1_22 d^1_22 + c^2_22 d^2_22 ))

```python
psym_matrix(X.reshape(4, 1),
            ending="="+psym_matrix(B.reshape(4, 1),
                                   row=True),
            intro=psym_matrix(crearMatrix("f",shape=(4,4)),
                              row=True))
```

$$\left[\begin{matrix}f_{11} & f_{12} & f_{13} & f_{14}\\f_{21} & f_{22} & f_{23} & f_{24}\\f_{31} & f_{32} & f_{33} & f_{34}\\f_{41} & f_{42} & f_{43} & f_{44}\end{matrix}\right]\left[\begin{matrix}x_{11}\\x_{12}\\x_{21}\\x_{22}\end{matrix}\right]=\left[\begin{matrix}b_{11}\\b_{12}\\b_{21}\\b_{22}\end{matrix}\right]$$

## Разложение

### $$ \exists R,T: R_{4x4}T_{4x4}X_{4x1}=B_{4x1}$$

```python
R1 = Matrix([ # How???
    [ C1[0,0], 0     , C1[0,1], 0     ,],
    [ 0     , C1[0,0], 0     , C1[0,1],],
    [ C1[1,0], 0     , C1[1,1], 0     ,],
    [ 0     , C1[1,0], 0     , C1[1,1],]
])
T1 = Matrix([ # How???
    [D1[0,0],D1[1,0],   0  , 0    ],
    [D1[0,1],D1[1,1],   0  , 0    ],
    [ 0    , 0    ,D1[0,0],D1[1,0]],
    [ 0    , 0    ,D1[0,1],D1[1,1]]
])
R2 = Matrix([ # How???
    [ C2[0,0], 0     , C2[0,1], 0     ,],
    [ 0     , C2[0,0], 0     , C2[0,1],],
    [ C2[1,0], 0     , C2[1,1], 0     ,],
    [ 0     , C2[1,0], 0     , C2[1,1],]
])
T2 = Matrix([ # How???
    [D2[0,0],D2[1,0],   0  , 0    ],
    [D2[0,1],D2[1,1],   0  , 0    ],
    [ 0    , 0    ,D2[0,0],D2[1,0]],
    [ 0    , 0    ,D2[0,1],D2[1,1]]
])
```

```python
psym_matrix(T1*R1)
```

$$\left[\begin{matrix}c^1_{11} d^1_{11} & c^1_{11} d^1_{21} & c^1_{12} d^1_{11} & c^1_{12} d^1_{21}\\c^1_{11} d^1_{12} & c^1_{11} d^1_{22} & c^1_{12} d^1_{12} & c^1_{12} d^1_{22}\\c^1_{21} d^1_{11} & c^1_{21} d^1_{21} & c^1_{22} d^1_{11} & c^1_{22} d^1_{21}\\c^1_{21} d^1_{12} & c^1_{21} d^1_{22} & c^1_{22} d^1_{12} & c^1_{22} d^1_{22}\end{matrix}\right]$$

```python
psym_matrix(T1,
            intro=psym_matrix(R1,row=True),
            ending="+"+psym_matrix(T2, row=True,
                                   intro=psym_matrix(R2,row=True))
           )
matrix_to_word(T1)
```

```python
psym_matrix(R1*T1,
           ending="+"+psym_matrix(R2*T2,row=1))
```

```python
psym_matrix(R1*T1+R2*T2)
```

## А что если... $$ X_{1x4}R^{'} _{4x4}T^{'} _{4x4}=B_{1x4}$$

```python
X = crearMatrix("x")
B = crearMatrix("b")

D= crearMatrix("d")
C= crearMatrix("c")

psym_matrix(X, intro="X=")
psym_matrix(B, intro="B=")
psym_matrix(B, intro="D=")
psym_matrix(C, intro="C=")
```

```python
print("Для удобства транспонируем")
psym_matrix((C*X*D).reshape(4, 1),
            ending="="+ psym_matrix(B.reshape(4, 1),row=1))
```

```python
R_ = Matrix([ # How???
    [ C[0,0], 0     , C[1,0], 0     ,],
    [ 0     , C[0,0], 0     , C[1,0],],
    [ C[0,1], 0     , C[1,1], 0     ,],
    [ 0     , C[0,1], 0     , C[1,1],]
])
T_ = Matrix([ # How???
    [D[0,0],D[0,1],   0  , 0    ],
    [D[1,0],D[1,1],   0  , 0    ],
    [ 0    , 0    ,D[0,0],D[0,1]],
    [ 0    , 0    ,D[1,0],D[1,1]]
])
psym_matrix(T_,
           intro=psym_matrix(X.reshape(1,4),row=1)+psym_matrix(R_,row=1),
           ending="="+psym_matrix(B.reshape(1,4),row=1))
```

```python
psym_matrix(R_*T_, row=1)
```

## Заметим, что: $$ R^{'},T^{'} = R^TT^T $$

```python
P1 = R1*T1
P2 = R2*T2
```

```python
print(P2*P1 == P1*P2)
print(R1*T1 == T1*R1)
```

    False
    True

```python

```


# Software profesional en Acústica 2020-21 (M2i)

*This notebook was adapted from Chapter 1 of [The FEniCS Tutorial Volume I](https://fenicsproject.org/pub/tutorial/sphinx1/) by Hans Petter Langtangen and Anders Logg, released under CC Attribution 4.0 license. It has been created by Xiangmin Jiao (University of Stony Brook University) and it is available in the repository [Unifem/FEniCS-note](https://github.com/unifem/fenics-notes).*

# The equations of vibrations in linear elasticity

Vibrations in linear elasticity is the study of how solid objects are responding to a mechanical vibration and become
internally stressed due to prescribed time-harmonic loading conditions. It is an important problem
in modern engineering. Its corresponding PDE is a generalization of the
Helmholtz equation, and it is among one of the most popular PDEs in
engineering. We now study its variational formulation and how to solve
this problem using FEniCS in 2D.

## PDE problem

The time-harmonic equation governing elastic vibrations of a body $\Omega$ can be written as

\begin{equation}
\label{ftut-elast-varform-equilibrium}\tag{1}
-\omega^2\rho\boldsymbol{u} -\boldsymbol{\nabla}\cdot\boldsymbol{\sigma} = \boldsymbol{f}\hbox{ in }\Omega,
\end{equation}

where $\boldsymbol{\sigma}$ is the *stress tensor*, and $\boldsymbol{f}$ is the *body force per unit
volume*, $\rho$ is the *mass density*, and $\omega$ the angular frequency. For isotropic materials, the stress tensor is further related to the deformation by
the following two equations:
\begin{align}
\boldsymbol{\sigma} &= \lambda\,\hbox{tr}\,(\boldsymbol{\varepsilon}) \boldsymbol{I} + 2\mu\boldsymbol{\varepsilon},
\label{ftut-elast-varform-stresstrain}\tag{2}\\
\boldsymbol{\varepsilon} &= \frac{1}{2}\left(\boldsymbol{\nabla} \boldsymbol{u} + (\boldsymbol{\nabla} \boldsymbol{u})^{\top}\right),
\label{ftut-elast-varform-strainu}\tag{3}
\end{align}
where $\boldsymbol{\varepsilon}$ is the *symmetric strain-rate tensor* (symmetric gradient),
and $\boldsymbol{u}$ is the *displacement vector field*, $\boldsymbol{I}$ denotes the *identity tensor*,
$\mathrm{tr}$ denotes the *trace operator* on a tensor, and $\lambda$ and $\mu$
are material properties known as *Lamé's elasticity parameters*.

We can combine (\ref{ftut-elast-varform-stresstrain}) and
(\ref{ftut-elast-varform-strainu}) to obtain
\begin{equation}
\label{ftut-elast-varform-stressu}\tag{4}
\boldsymbol{\boldsymbol{\sigma}} = \lambda(\boldsymbol{\nabla}\cdot \boldsymbol{u})\boldsymbol{I} + \mu(\boldsymbol{\nabla} \boldsymbol{u} + (\boldsymbol{\nabla} \boldsymbol{u})^{\top})
\end{equation}

Note that
(\ref{ftut-elast-varform-equilibrium})--(\ref{ftut-elast-varform-strainu})
can easily be transformed to a single vector PDE for $\boldsymbol{u}$, which is the
governing PDE for the unknown $\boldsymbol{u}$ (Navier's equation).  In the
derivation of the variational formulation, however, it is convenient
to keep the equations split as above.

## Variational formulation

The variational formulation of
([1](#mjx-eqn-ftut-elast-varform-equilibrium)--[3](#mjx-eqn-ftut-elast-varform-strainu))
consists of forming the inner product of
[1](#mjx-eqn-ftut-elast-varform-equilibrium) and a *vector* test function
$\boldsymbol{v}\in \hat{V}$, where $\hat{V}$ is a vector-valued test function space, and
integrating over the domain $\Omega$:

\begin{equation}
-\omega^2\int_\Omega \rho\boldsymbol{u}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}-\int_\Omega (\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}) \cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} =
\int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x},\label{eq:elast:varform:variational}\tag{5}
\end{equation}
where $\mathrm{d}\boldsymbol{x}$ denotes the volume measure.

Since $\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}$ contains second-order derivatives of the primary
unknown $\boldsymbol{u}$, we integrate this term by parts:

\begin{equation}
-\omega^2\int_\Omega \rho\boldsymbol{u}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}-\int_\Omega (\boldsymbol{\nabla}\cdot\boldsymbol{\sigma}) \cdot \boldsymbol{v} \ \mathrm{d}\boldsymbol{x}
= \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\nabla} \boldsymbol{v} \ \mathrm{d}\boldsymbol{x} - \int_{\partial\Omega}
(\boldsymbol{\sigma}\cdot \boldsymbol{n})\cdot \boldsymbol{v} \ \mathrm{d}\boldsymbol{s},
\label{eq:elast:varform:integbypart}\tag{6}
\end{equation}

where the colon operator is the inner product between tensors (summed
pairwise product of all elements), $\boldsymbol{n}$ is the outward unit normal
at the boundary, and $\mathrm{d}\boldsymbol{s}$ is a measure in surface area.

The quantity $\boldsymbol{\sigma}\cdot \boldsymbol{n}$ is known as the
*traction* or stress vector at the boundary, and is often prescribed
as a boundary condition on a part $\partial_T\Omega$ of the boundary
as $\boldsymbol{\sigma}\cdot \boldsymbol{n} = \boldsymbol{T}$, whereas the
remaining part of the boundary would have the value of the displacements prescribed.

Substituting (\ref{eq:elast:varform:integbypart}) into (\ref{eq:elast:varform:variational}),
we thus obtain
\begin{equation}
-\omega^2\int_\Omega \rho\boldsymbol{u}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}
+ \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\nabla} \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} =
\int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}
+ \int_{\partial\Omega} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s}.
\label{eq:elast:varform:varform_original}\tag{7}
\end{equation}

Inserting the expression ([4](#mjx-eqn-ftut-elast-varform-stressu)) for
$\boldsymbol{\sigma}$ gives the variational form with $\boldsymbol{u}$ as unknown.

### Symmetrizing $\boldsymbol{\nabla} \boldsymbol{v}$

One can show that the inner product of a symmetric tensor $\boldsymbol{A}$ and an
anti-symmetric tensor $\boldsymbol{B}$ vanishes. Since $\boldsymbol{\sigma}$ is a
symmetric tensor, if we express $\boldsymbol{\nabla} \boldsymbol{v}$ as a sum
of its symmetric and anti-symmetric parts, only the symmetric part will
survive in the product $\boldsymbol{\sigma} :\boldsymbol{\nabla} \boldsymbol{v}$. Thus replacing $\boldsymbol{\nabla} \boldsymbol{u}$ by the symmetric gradient gives rise to the slightly different variational form

\begin{equation}
-\omega^2\int_\Omega \rho\boldsymbol{u}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}
+ \int_\Omega \boldsymbol{\sigma} : \boldsymbol{\epsilon}(\boldsymbol{v})\ \mathrm{d}\boldsymbol{x} =
\int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}
+ \int_{\partial\Omega} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s}
\label{ftut-elast-varform-sigma_inner_gradv}\tag{8}
\end{equation}

where $\boldsymbol{\epsilon}(\boldsymbol{v})$ is the symmetric part of $\boldsymbol{\nabla} \boldsymbol{v}$:
\begin{equation}
\boldsymbol{\epsilon}(\boldsymbol{v}) = \frac{1}{2}\left(\boldsymbol{\nabla} \boldsymbol{v} + (\boldsymbol{\nabla} \boldsymbol{v})^{\top}\right).
\label{ftut-elast-varform-sigma_inner_eps}\tag{9}
\end{equation}

The formulation (\ref{ftut-elast-varform-sigma_inner_gradv}) is what naturally
arises from the minimization of elastic potential energy, and is a more
popular formulation than ([7](#mjx-eqn-eq:elast:varform:varform_original})).

Here, $\boldsymbol{\epsilon}$ is a useful operator. The symmetric strain-rate tensor $\boldsymbol{\varepsilon}$ in ([3](#mjx-eqn-ftut-elast-varform-strainu)) is equal to $\boldsymbol{\epsilon}(\boldsymbol{u})$.

### Enforcing boundary conditions

Now let us consider how to enforce boundary conditions.
For Dirichlet boundaries, we will enforce boundary-conditions strongly.
For these points, no test functions are associated with the Dirichlet nodes.

For traction boundary conditions, we will enforce the boundary condition
weakly using the variational form ([8](#mjx-eqn-ftut-elast-varform-sigma_inner_gradv)).
Similar to the Helmholtz equation, we require their corresponding test
functions $\boldsymbol{v}$ vanish along $\partial \Omega$ for interior points.
Then, the boundary integral above has no effects for points on
$\partial\Omega\setminus\partial\Omega_T$.

### Summary of variational form
In summary, the variational problem is to find $\boldsymbol{u}$ in a vector function space $\hat{V}$ such that
\begin{equation}
a(\boldsymbol{u},\boldsymbol{v}) = L(\boldsymbol{v})\quad\forall \boldsymbol{v}\in\hat{V},\tag{10}
\end{equation}
where
\begin{align}
a(\boldsymbol{u},\boldsymbol{v}) &= -\omega^2\int_\Omega \rho\boldsymbol{u}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x}
+ \int_\Omega\sigma(\boldsymbol{u}) :\varepsilon(\boldsymbol{v})\ \mathrm{d}\boldsymbol{x},\tag{11}\\
L(\boldsymbol{v}) &= \int_\Omega \boldsymbol{f}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{x} + \int_{\partial\Omega_T} \boldsymbol{T}\cdot \boldsymbol{v}\ \mathrm{d}\boldsymbol{s}\tag{12},
\end{align}
and
\begin{equation}
\boldsymbol{\sigma}(\boldsymbol{u}) = \lambda(\boldsymbol{\nabla}\cdot \boldsymbol{u})\boldsymbol{I} + \mu(\boldsymbol{\nabla} \boldsymbol{u} + (\boldsymbol{\nabla} \boldsymbol{u})^{\top}).\tag{13}\\
\end{equation}

## FEniCS implementation

To demonstrate the implementation, we will model a clamped beam deformed under a time-harmonic surface force on the opposite free cross-section surface. This can be modeled by setting $\boldsymbol{T}=\boldsymbol{f}=(0,0,1)$ on that boundary part $x=L$. The beam is
box-shaped with length $L$ and has a square cross section of width $W$. We
set $\boldsymbol{u}=(0,0,0)$ at the clamped end, $x=0$. The rest of the lateral boundary is
traction free; that is, we set $\boldsymbol{T} = 0$. Therefore,
$$L(\boldsymbol{v}) = \int_{\{x=L\}} \boldsymbol{f}\cdot \boldsymbol{v} \mathrm{d}\boldsymbol{s}$$
for this problem.

### Import packages

We start by importing fenics and enforcing matplotlib. In addition, we import `mshr` for mesh generation.

```python
import numpy as np
from dolfin import *
from mshr import *
import matplotlib.pyplot as plt
%matplotlib inline
```

### Generate the mesh and function spaces

Our action startes by generating meshes and defining function spaces.

```python
# Create mesh and define function space
length = 1; width = 0.2

mesh = RectangleMesh(Point(0, 0), Point(length, width), 50, 10, "right/left")
V = VectorFunctionSpace(mesh, 'P', 1)
```

### Define the variational problem

The primary unknown is now a vector field $\boldsymbol{u}$ and not a scalar field,
so we need to work with a vector function space. We will use
piecewise-linear basis functions for all the components.

```python
u = TrialFunction(V)
v = TestFunction(V)
```

With `u = TrialFunction(V)` we get `u` as a vector-valued finite element
function with three components for this 2D problem.

Next, we define the stress tensor and $a$. The gradient and divergence operators
now have a prefix `nabla_`.
This is not strictly necessary in the present problem, but is
recommended in general for vector PDEs arising from continuum mechanics,
if you interpret $\boldsymbol{\nabla}$ as a vector in the PDE notation. See
the notes on `grad(u)` vs. `nabla_grad(u)` below.

```python
from ufl import nabla_div

# Define strain and stress
def epsilon(u):
    return 0.5*(nabla_grad(u) + nabla_grad(u).T)
    #return sym(nabla_grad(u))

# Physical constants
rho = 1.
omega = 2*np.pi*3
beta = 1.25
lambda_ = beta
mu = 1
d = u.geometric_dimension()  # space dimension

def sigma(u):
    return lambda_*nabla_div(u)*Identity(d) + 2*mu*epsilon(u)

# Define a
a = -omega**2*rho*inner(u, v)*dx + inner(sigma(u), epsilon(v))*dx
```

To define the partition of the boundary in the computational domain, each face of the finite element mesh is marked with a number flag or *marker*:

```python
# Initialize subdomain and boundary markers
tol = 1e-3
clamped_boundary = CompiledSubDomain('on_boundary and near(x[0],L,tol)', L=0., tol=tol)
traction_boundary = CompiledSubDomain('on_boundary and near(x[0],L,tol)', L=length, tol=tol)

# Initialize mesh function for boundary
boundary_markers = MeshFunction('size_t', mesh, mesh.topology().dim() - 1)
boundary_markers.set_all(0)
clamped_boundary.mark(boundary_markers, 1)
traction_boundary.mark(boundary_markers, 2)

# Define the boundary measure with the boundary markers
ds = Measure('ds', domain=mesh, subdomain_data=boundary_markers)
```

To define $L$, $\boldsymbol{f}=(0, 0, 1)$ is a constant vector, instead of a scalar.
Such a vector constant is specified as `Constant(0, 0, 1)` in FEniCS.

```python
# Define L
f = Constant((0, 1))
L = dot(f, v)*ds(2)
```

### Define boundary conditions

We only specify the Dirichlet boundary condition. For the boundary condition
$u=(0, 0, 0)$, we must set a vector value to zero, not just a scalar.
We specify the vector constant as `Constant((0, 0, 0))`.

```python
# Define boundary condition
bc = DirichletBC(V, Constant((0, 0)), boundary_markers, 1)
```

### Solve the variational problem

Finally, we can solve the problem.

```python
# Compute solution
u = Function(V)
solve(a == L, u, bc)
```

### Plot the solution

Any component of the solution can be computed or even the modulus of the vector-field solution

```python
# Plot the Finite Element approximation
def plot_solution(u):
    '''plot solution of FEM-based simulation'''
    fig = plt.figure(figsize=(10,10))
    fig = plot(u)
    plt.xlabel(r'$x$ / m')
    plt.ylabel(r'$y$ / m')
    plt.colorbar(fig, fraction=0.038, pad=0.04);

# Plot vector field
plot_solution(u)

# Compute magnitude of displacement
u_magnitude = sqrt(dot(u, u))
Q = FunctionSpace(mesh, 'P', 1)
u_magnitude = project(u_magnitude, Q)
plot_solution(u_magnitude)
```

```python
# Plot second component of the solution
plot_solution(u[0])
plot_solution(u[1])
```

## Note on `grad(u)` vs. `nabla_grad(u)`

For scalar functions, $\boldsymbol{\nabla} u$ has a clear meaning as the vector
$$\boldsymbol{\nabla} u =\left(\frac{\partial u}{\partial x}, \frac{\partial u}{\partial y},
\frac{\partial u}{\partial z}\right).$$

However, for a vector-valued function $\boldsymbol{u}$, the meaning is less clear.
Some sources define $\boldsymbol{\nabla}\boldsymbol{u}$ as the matrix with elements
$\partial u_j / \partial x_i$, especially in continuum mechanics.
Some other sources prefer $\partial u_i / \partial x_j$, such as in the
*Jacobian matrix*.

In our notes, $\boldsymbol{\nabla}$ always denotes the matrix
with elements $\partial u_j / \partial x_i$, to be consistent with most PDEs.
The corresponding operator in FEniCS is `nabla_grad(u)`.
For the Jacobian matrix, its corresponding operator in FEniCS is `grad(u)`.


#Torus
This python notebook will show the proof of the formula for volume of the torus using the Jacobian Method and the Triple Integrals Method computationally

###Import init_printing to display the integral result in a pretty way

```
from sympy import init_printing
init_printing()
```

###Import required libraries

```
import numpy as np
import sympy as sp
from sympy import integrate, Symbol, symbols, cos, sin, Matrix, pi, exp, plot, diff
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
```

###Create the torus graph and the formula for volume of torus

```
#Generate torus
angle = np.linspace(0, 2 * np.pi, 20)
theta, phi = np.meshgrid(angle, angle)

#toroidal coordinate system
r, R = 3, 8.5
X = (R + r * np.cos(phi)) * np.cos(theta)
Y = (R + r * np.cos(phi)) * np.sin(theta)
Z = r * np.sin(phi)

x = sp.Symbol('x')
y = sp.Symbol('y')

#Volume of torus
R, r, phi, theta = symbols('R r phi theta')

attributes1 = (r, 0, r)
attributes2 = (phi, 0, 2*pi)
attributes3 = (theta, 0, 2*pi)

#f variable value below was from the jacobian matrix calculation
#of the coordinate system
f = r*(R + cos(phi))
I1 = integrate(f, attributes2)
I2 = integrate(I1, attributes3)
I3 = integrate(I2, attributes1)
```

###Displaying the Toroidal shape and its volume formula

```
# Display the shape
fig = plt.figure()
ax = fig.gca(projection = '3d')
ax.set_xlim3d(-10, 10)
ax.set_ylim3d(-10, 10)
ax.set_zlim3d(-10, 10)
ax.plot_surface(X, Y, Z, color = 'c', edgecolors='w', rstride = 1, cstride = 1)
plt.show()

print('\n  The formula for volume of the torus: ')
display(I3)
```

###Jacobian Matrix Calculation

```
#toroidal coordinate system
R, r, phi, theta = symbols('R r phi theta', real=True)
X = (R + r * cos(theta)) * cos(phi)
Y = (R + r * cos(theta)) * sin(phi)
Z = r*sin(theta)

#Jacobian matrix
J = Matrix([
    [diff(X, r),diff(X, theta),diff(X,phi)],
    [diff(Y, r),diff(Y, theta),diff(Y,phi)],
    [diff(Z, r),diff(Z, theta),diff(Z,phi)]
])

display('A is: ', J)
display('Det(A) = ', J.det())
```

```

```


# Exercise session nº 5
---
# Furrow Constriction in Animal Cell Cytokinesis

__*Sacha Ichbiah, 21/02/22, ENS Paris*__

This subject is extracted from :
> Hervé Turlier et al., *Furrow Constriction in Animal Cell Cytokinesis*, Biophysical Journal, 2014. \
> https://doi.org/10.1016/j.bpj.2013.11.014

Cytokinesis is the process of physical cleavage at the end of cell division; it proceeds by ingression of an acto- myosin furrow at the equator of the cell. Its failure leads to multinucleated cells and is a possible cause of tumorigenesis. Despite its ubiquity in developmental biology, its precise description and understanding have challenged biologists and physicists.

In this paper, the authors propose a model based on a minimal geometry and scaling arguments that gives an interpretation of the process appearing during cytokinesis. It notably demonstrates that because of the cytoplasm incompressibility, the cytokinesis leads to a competition between the furrow line tension and the cell poles' surface tension. This competition sets a threshold for cytokinesis completion, and explains cytokinesis dynamics.

During these session, we will derive the equations of this scaling model of furrow constriction, and will integrate these equation to see constriction dynamics. We will show that it allows to have a cytokinesis duration independant of cell size, which has been observed in C-Elegans.

---

## I - The Scaling Model

The geometry of the dividing cell is described by the apposition of two spherical caps, parametrized by an angle $\theta$ as shown on the left sketch.

The volume of a spherical cap (in blue) is : $\mathcal{V}_{sc}(r,h) = \dfrac{\pi}{3} h^2 (3r - h)$, and its area : $\mathcal{A}_{sc}(r,h)=2\pi r h $ (right sketch).

#### **Question 1 :**
> Noting that the cytoplasmic is an incompressible fluid, establish that $R_0 = R F(\theta)$

We define a dimensionless parameter $\kappa$ to express the competition between the mean contractile surface tension at the furrow and the tension at the cell poles: $\kappa = \dfrac{\gamma}{2R_0N^a_0}$.

The polar contractility tends to reduce the surface $A_p = 2\pi R^2 (1+\text{cos}(\theta))$ of each cell poles, whereas the line tension tends to reduce the contractile ring circumference $r_f$. These effects are captured by a simple mechanical energy $\mathcal{E} = 2\pi r_f \gamma + 2 A_p N^a_0$.

#### **Question 2 :**
> Rescale the energy $\mathcal{E}$ by an energy $\mathcal{E}_0 = 4 \pi R_0^2 N^a_0$ to make it only depend on $\theta$ and $\kappa$.

## II - Mechanical Equilibrium

The local minimum of the energy gives the equilibrium configuration of the cell.

To find this minimum, we will use a library doing symbolic calculus in python called sympy. This will allow us to compute the derivatives effortlessly.

### Symbolic Computation with Sympy *(In french "Calcul Formel")*

We will use sympy, a library that allows to do symbolic computation. Analytical results are always the best, but sometimes the equations does not lead to beautiful simplifications. If we are interested in the numerical result of the equations, we can use sympy to work on the analytical expression directly, obtain derivatives, etc.. before evaluating them on real values. There are three main functions that we will use in sympy, that we will present briefly. If interested, the best symbolic calculus tools are present in Wolfram Mathematica, which is under license.

#### a) Defining expression with symbols and trigonometric functions, and obtain derivatives :

```python
!pip install sympy

import sympy
from sympy import symbols, diff,lambdify, simplify
from sympy import cos, sin
import numpy as np
from scipy.optimize import minimize
from scipy.optimize import fsolve
import matplotlib.pyplot as plt
```

```python
#Symbols are unknown variables, that we aim to replace with real values in the end. We define them with the function symbols :
a,b,c = symbols("a b c")

#We can then define a symbolic expression, containing or not trigonometric functions (among many other possibilities !)
E = a**2 + a*b + cos(c)

#And obtain its derivatives with respect to any variables, eg a :
First_derivative = E.diff(a)
Second_derivative = First_derivative.diff(a)
First_derivative, Second_derivative
```

    (2*a + b, 2)

#### b) Substituting variables and evaluating symbolic expressions :

```python
# We can replace symbols with real variables with the method subs :
print("c = pi gives :",E.subs([(c,np.pi)]))
print("Subs method : ",E.subs([(a,2),(b,1),(c,0)]) )

#We can also transform a symbolic expression into a lambda function
#This is a faster process than subs if we need to evaluate the function on many points :
f = lambdify((a,b,c),E,"numpy")
print("Lambify method : ",f(2,1,0))

#We can combine both to replace certain variables before creating a lambda function with the remaining variables :
g = lambdify(a,E.subs([(b,1),(c,0)]))
print("Subs and lambdify combined :",g(2))

#Short benchmarking
from time import time
values_evaluated = np.linspace(0,np.pi,1000)
t1 = time()
g = lambdify(a,E.subs([(b,2),(c,0.2)]))
g(values_evaluated)
t2 = time()
for value in values_evaluated :
    E.subs([(a,value),(b,2),(c,0.2)])
t3 = time()

print("Time with lambdify :",round((t2-t1),4))
print("Time with subs :",round((t3-t2),4))
```

    c = pi gives : a**2 + a*b - 1.0
    Subs method :  7
    Lambify method :  7.0
    Subs and lambdify combined : 7
    Time with lambdify : 0.0029
    Time with subs : 0.9684

### The equilibrium configuration during cytokinesis

Let's go back to the initial problem. Our goal is to study the properties of the normalized energy $\overline{\mathcal{E}} = \mathcal{E}/\mathcal{E}_0$.

```python
x, k = symbols("x k")
F = 1 +1.5*cos(x) - 0.5*(cos(x))**3
energy = k * sin(x)/(F**(1/3)) + (1+ cos(x))/(F**(2/3))

#We see that there is no simplification easily given by sympy :
print(simplify(energy),'\n')

#We can replace the values of k and x :
print(energy.subs([(x,np.pi/3),(k,.5)]))

energy
```

    (0.873580464736299*k*(-0.333333333333333*cos(x)**3 + cos(x) + 0.666666666666667)**0.666666666666667*sin(x) + 0.763142828368888*(cos(x) + 1)*(-0.333333333333333*cos(x)**3 + cos(x) + 0.666666666666667)**0.333333333333333)*(-0.333333333333333*cos(x)**3 + cos(x) + 0.666666666666667)**(-1.0)

    1.42197524663604

$\displaystyle \frac{0.873580464736299 k \sin{\left(x \right)}}{\left(- 0.333333333333333 \cos^{3}{\left(x \right)} + \cos{\left(x \right)} + 0.666666666666667\right)^{0.333333333333333}} + \frac{0.763142828368888 \left(\cos{\left(x \right)} + 1\right)}{\left(- 0.333333333333333 \cos^{3}{\left(x \right)} + \cos{\left(x \right)} + 0.666666666666667\right)^{0.666666666666667}}$

Now that we have implemented our energy in sympy, we can automatically obtain the derivatives with the diff engine. We see that the analytical formulas are quite long and obtaining by hand the derivatives would be both painful and prone to errors.

#### **Question 3 :**
> Obtain the expression of the first and second derivatives of the energy $\overline{\mathcal{E}}$ with respect to theta (i.e x) with the diff function :

```python

```

#### **Question 4 :**
> Plot the energy profile $\overline{\mathcal{E}}(\theta), \theta \in [0,\dfrac{\pi}{2}]$ for $\kappa \in \{0.0,0.1,0.2,0.3,0.4,0.5\}$. What do you observe ?

```python
plt.figure(figsize = (15,25))
vals_theta = np.linspace(0,np.pi/2,10000)

for k_val in np.linspace(0,0.5,6):
    print("k_value :",k_val)
    ...
    plt.plot(vals_theta,...,label = k_val.round(2))
plt.legend()
```

#### **Question 5 :**
> Starting with $\theta = \pi/4$, find the angle giving the local energy minimum for each k in values_k. Plot the equilibrium angle $\theta_{min}$, and the value of the derivatives of the energy $\left. \dfrac{\partial \mathcal{E}}{\partial \theta}\right\rvert_{\theta_{min}}$, $\left. \dfrac{\partial^2 \mathcal{E}}{\partial \theta^2}\right\rvert_{\theta_{min}}$, $\left (\left. \dfrac{\partial^2 \mathcal{E}}{\partial \theta^2}\right\rvert_{\theta_{min}}\right)^2$ at this angle for each k.

__Tip :__ : Lambdify the energy function into a function f and find its minimum with the function minimize from scipy (see bellow)

```python
e = lambdify((x,k), energy, "numpy")
npoints = 1001
values_k = np.linspace(0, 1, npoints)
...

for j,k_val in enumerate(values_k) :

    f = lambda x : e(x,k_val)

    ...

    sols = minimize(fun = f,x0 = (np.pi/4),method = "SLSQP",bounds=[(0,np.pi/2)])
    assert (sols.success)
    min_theta = sols['x']

    ...

fig,ax = plt.subplots(1,4,figsize = (21,3))
ax[0].plot(values_k, ...)
ax[0].set_xlabel("k")
ax[0].set_ylabel("minimum found")
ax[1].plot(values_k, ...)
ax[1].set_xlabel("k")
ax[1].set_ylabel("derivative value at minimum")
ax[2].plot(values_k, ...)
ax[2].set_xlabel("k")
ax[2].set_ylabel("second derivative value at minimum")
ax[3].plot(values_k, ...)
ax[3].set_xlabel("k")
ax[3].set_ylabel("square of second derivative value at minimum")
```

#### **Question 6 :**
> Estimate the value $k_c$ where this local minimum disappears, and its associated angle $\theta_c$. Compute the values of the first two derivatives $\left. \dfrac{\partial \mathcal{E}}{\partial \theta}\right\rvert_{\theta_c} $ and  $\left. \dfrac{\partial^2 \mathcal{E}}{\partial \theta^2}\right\rvert_{\theta_c} $. What do you deduce on the order of the phase transition ?

__Tip :__ Find the value where the second derivative evaluated at $\theta_c$ is equal to 0 (i.e the minimum of the square of the second derivative evaluated at $\theta_c$)

#### **Optional : Question 7 :**
> Plot the equilibrium angles and draw the shape of the cells for $\kappa \in \{0,0.2,0.4,0.6\}$

```python
fig,ax = plt.subplots(1,5,figsize =(25,5))
colors = ['tab:blue','tab:orange','tab:green','tab:red']
R0 = 1
for j,idx_val in enumerate([0,2000,4000,6000]) :

    theta_sol = Solutions[idx_val]
    k_value = values_k[idx_val]
    R = R0/((F.subs(x,theta_sol[0]))**(1/3))
    e = lambdify((x), energy.subs(k,k_value), "numpy")

    theta_values_k = np.linspace(theta_sol, 2*np.pi-theta_sol,100)
    circle_x = R*np.cos(theta_values_k)
    circle_y = R*np.sin(theta_values_k)

    ax[0].plot(vals_theta*180/np.pi,e(vals_theta))
    ax[0].scatter(theta_sol*180/np.pi,e(theta_sol),s = 180)
    ax[0].set_ylabel("Energy")
    ax[0].set_xlabel("Angle value")
    ax[0].set_xlim(-5,95)

    ax[j+1].plot(circle_x-R*np.cos(theta_sol),circle_y,color=colors[j],linewidth=5)
    ax[j+1].plot(R*np.cos(theta_sol)-circle_x,circle_y,color=colors[j],linewidth=5)
    ax[j+1].set_title("Equilibrium angle value :" + str((theta_sol[0]*180/np.pi).round(2)))
    ax[j+1].set_xlim(-2,2)
    ax[j+1].set_ylim(-2,2)
    ax[j+1].set_aspect('equal')#, adjustable='box')
```

## III - Dynamics

We now want to study the furrow constriction dynamics, i.e the temporal evolution of $\dfrac{r_f}{R_0}$. We will establish these dynamics by expressing the derivative of this quantity with respect to $\theta$. As before, we will use the symbolic computation library sympy to evaluate numerical quantities.

To establish the dynamic equation, we note that the power of active effects is exactly dissipated by viscous cell deformations. The viscous dissipation is made of two contributions, the stretching of the poles and the constriction of the ring, which we estimate in scaling. The volume of acto-myosin in the poles is $V_p = 2A_p e_p$ and in the ring $V_f = 2\pi r_f w e_f$, where $w$ and $e_f$ are the width and thickness of the contractile ring. (We remind that the surface of each cell pole writes : $A_p = 2\pi R^2 (1+\text{cos}(\theta))$). The value $e_p \approx e_0$ and the ring thickness $e_f$ reach a steady-state value that depends on turnover. This yields the viscous dissipated power :

$\begin{align}
P_d &= \dfrac{1}{2} \eta \left[ V_p \left(\dfrac{1}{R} \dfrac{dr_f}{dt} \right)^2 + V_f \left( \dfrac{1}{r_f} \dfrac{dr_f}{dt}\right)^2  \right] \newline
P_d &= \dfrac{1}{2} \eta \left[ e_f 4 \pi R^2 (1+\text{cos}\theta) \dfrac{1}{R^2}  \left(\dfrac{dr_f}{dt} \right)^2 + 2\pi w e_f r_f \dfrac{1}{r_f^2} \left(  \dfrac{dr_f}{dt}\right)^2  \right]  \newline
&\approx \dfrac{1}{2} \eta \left[ 4 e_0 \pi(1+\text{cos}\theta)  \left(\dfrac{dr_f}{dt} \right)^2 + 2\pi w e_f\dfrac{1}{r_f} \left( \dfrac{dr_f}{dt}\right)^2  \right]  \newline
&= \left(\dfrac{dr_f}{dt} \right)^2 \dfrac{1}{2} \eta \left[ 4 \pi e_0 (1+\text{cos}\theta)   + \dfrac{4\pi}{2} w e_f\dfrac{F(\theta)^{1/3}}{R_0 \sin \theta}  \right]  \newline
&= \left(\dfrac{dr_f}{dt} \right)^2 4 \pi e_0 \eta \left[ (1+\text{cos}\theta)   + \dfrac{1}{2 R_0} w \dfrac{e_f}{e_0} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right]  \newline
&= \left(\dfrac{dr_f}{dt} \right)^2 4 \pi e_0 \eta \left[ (1+\text{cos}\theta)   + \lambda \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right]  \newline
\end{align}
$

The balance of mechanical and dissipated powers yields :

$\dfrac{d \mathcal{E}}{dt} + P_d = 0$

Besides :

$
\dfrac{1}{\mathcal{E}_0} \dfrac{d\mathcal{E}}{dt} = \dfrac{\partial \mathcal{E}/\mathcal{E}_0}{\partial \theta} \dfrac{\partial \theta}{\partial r_f} \dfrac{d r_f}{d_t} = \dfrac{\partial \mathcal{E}/\mathcal{E}_0}{\partial \theta} \left(\dfrac{\partial r_f}{\partial \theta}\right)^{-1} \dfrac{d r_f}{d_t}
$

And, with $T_a=\frac{\eta e_0}{N^a_0}$ :

$\dfrac{1}{\mathcal{E}_0} P_d = \left(\dfrac{dr_f}{dt} \right)^2 \dfrac{4\pi e_0 \eta}{4 \pi R_0^2 N^a_0} \left[ (1+\text{cos}\theta)   + \dfrac{\lambda}{2} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right] = \left(\dfrac{dr_f}{dt} \right)^2 \dfrac{T_a}{ R_0^2} \left[ (1+\text{cos}\theta)   + \dfrac{\lambda}{2} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right] $

We have thus from $\dfrac{d \mathcal{E}/\mathcal{E}_0}{dt} = - P_d/\mathcal{E}_0$ :

$
\begin{align}
\dfrac{\partial \mathcal{E}/\mathcal{E}_0}{\partial \theta} \left(\dfrac{\partial r_f}{\partial \theta}\right)^{-1} \dfrac{d r_f}{d_t}  &= - \left(\dfrac{dr_f}{dt} \right)^2 \dfrac{T_a}{ R_0^2} \left[ (1+\text{cos}\theta)   + \dfrac{\lambda}{2} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right]
\newline
\dfrac{\partial \mathcal{E}/\mathcal{E}_0}{\partial \theta} \left(\dfrac{\partial r_f/R_0}{\partial \theta}\right)^{-1} &= - \dfrac{dr_f}{dt} \dfrac{T_a}{ R_0} \left[ (1+\text{cos}\theta)   + \dfrac{\lambda}{2} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right] \newline
\dfrac{dr_f}{dt} \dfrac{T_a}{ R_0} &= - \dfrac{\partial \mathcal{E}/\mathcal{E}_0}{\partial \theta} \left(\dfrac{\partial r_f/R_0}{\partial \theta}\right)^{-1} \left[ (1+\text{cos}\theta)   + \dfrac{\lambda}{2} \dfrac{F(\theta)^{1/3}}{\sin \theta}  \right]^{-1} = -\mathcal{H}(\theta,\kappa,\lambda)
\end{align}
$

We will compute numerical the values of this function $\mathcal{H}$ to obtain the evolution of the furrow radius $r_f$

#### **Question 8 :**
> From the last equation, express the angle temporal variation $\dot \theta$.

#### **Question 9 :**
> Compute numerically $\dot \theta$ with sympy, and integrate the evolution of $\theta$, $r_f$ in time with a forward-euler-scheme for $t \in [0,15]$, starting with $\theta(0)=\pi/2$, with $\lambda = 1$ and $\kappa \in \{0.1,0.25,0.4,0.5,0.75,1\}$. Check that it is compatible with the previous results obtained from the static analysis.

```python
x, k, l = symbols("x k l")
F = 1 +1.5*cos(x) - 0.5*(cos(x))**3
energy = k * sin(x)/(F**(1/3)) + (1+ cos(x))/(F**(2/3))

r_f = sin(x)*(F**(-1/3))

dr_f = diff(r_f,x)
first_derivative = diff(energy,x)

H = ((1+cos(x)) + l*(F**(1/3))/sin(x))**(-1) * first_derivative/dr_f

dtheta = - H/dr_f
dtheta = dtheta.subs(l,0.1)

#We can already lambdify the function r_f as it does not depend on k :
func_rf = lambdify(x,r_f,"numpy")
```

```python
theta0 = np.pi/2
R0 = 1

for k_value in [0.1,0.25,0.4,0.5,0.75,1]:

    ...

    npoints = 1000
    timepoints = np.linspace(0,15,npoints)
    dt = timepoints[1]-timepoints[0]

    ...

    for j,t in enumerate(timepoints) :

        ...

    plt.plot(timepoints,...,label = k_value)

plt.title("Constriction completion or failure")
plt.xlabel("time t/Ta")
plt.ylabel("furrow radius r_f/R_0")
plt.legend()
```

#### **Question 10 :**
> Determine the cytokinesis duration with $R_0 \in \{0.5,1,2,4\}$. Show that in case of cytokinesis completion $\lambda = 1, \kappa = 0.75$, the initial cell radius R0 has no impact on the cytokinesis time.

```python
theta0 = np.pi/2
k_value = 0.75

for R0 in [0.5,1,2,4]:

    ...

    npoints = 1000
    timepoints = np.linspace(0,6,npoints)
    dt = timepoints[1]-timepoints[0]

    ...

    for j,t in enumerate(timepoints) :

        ...

    plt.plot(timepoints,...,label ="R0: " +str(R0))

plt.title("Cytokinesis time is independant of the cell initial radius")
plt.xlabel("time t/Ta")
plt.ylabel("furrow radius r_f/R_0")
plt.legend()
```


# PRAKTIKUM 13
`Solusi Persamaan Differensial Biasa (PDB) 2`

1. Runge-Kutta Fehlberg (RKF45)
2. Runge-Kutta untuk Sistem Persamaan Differensial Biasa
3. Persamaan Differensial Biasa (PDB) ordo tinggi (lebih dari 1)
4. PDB dengan Masalah Nilai Batas
    1. Metode _Linear-Shooting_
    2. Metode Beda-Hingga (_Finite-Difference_)

# 1 Runge-Kutta Fehlberg (RKF45)
RKF45 merupakan modifikasi metode RK4 untuk memperoleh metode dengan ukuran langkah h yang adaptif, artinya ukuran langkah akan menyesuaikan bentuk fungsi (h kecil jika fungsi curam, h besar jika fungsi landai).
Solusi pada setiap iterasinya dihitung sebanyak dua kali, masing-masing menggunakan metode berorde 4 dan metode berorde 5.
Selisih kedua solusi pada setiap iterasinya digunakan untuk menentukan ukuran langkah h pada iterasi berikutnya. Ukuran langkah h dapat membesar atau mengecil sesuai kebutuhan.

```julia
#%%METODE RUNGE-KUTTA FEHLBERG 4/5
#%
#% Digunakan untuk mencari solusi persamaan differensial
#%   dy/dt = f(t,y) dengan masalah nilai awal y(a) = y0
#%
#% sol = rkf45(f,a,b,y0,M,delta)
#% Input  : f    -> fungsi f(t,y)
#%          a,b  -> batas bawah dan atas solusi MNA
#%          y0   -> nilai awal y(a)=y0
#%          M    -> banyaknya sub-interval awal
#%          delta-> toleransi per langkah yang diberikan
#% Output : sol  -> solusi PD, sol=[T,Y]
#%
#% Digunakan Sebagai Pedoman Praktikum Metode Numerik
#%
#% Lihat juga : heun, taylor, rungekutta
function rkf45(f,a,b,y0,M,delta)
  M = round(M)
  a2=1/4;b2=1/4;a3=3/8;b3=3/32;c3=9/32;
  a4=12/13;b4=1932/2197;c4=-7200/2197;d4=7296/2197;
  a5=1;b5=439/216;c5=-8;d5=3680/513;e5=-845/4104;
  a6=1/2;b6=-8/27;c6=2;d6=-3544/2565;e6=1859/4104;f6=-11/40;
  r1=1/360;r3=-128/4275;r4=-2197/75240;r5=1/50;r6=2/55;
  n1=25/216;n3=1408/2565;n4=2197/4104;n5=-1/5;
  big=1e15;
  h=(b-a)/M;
  hmin=h/64;
  hmax=h*64;
  maxi=200;
  j=1;
  Y = y0;
  T = a;
  br= b-0.001*abs(b);
  err=NaN
  while T[j]<b
    if (T[j]+h)>br;h=b-T[j];end
    #% Hitung koefisien
    k1=h*f(T[j],Y[j]);
    y2=Y[j]+b2*k1;
    k2=h*f(T[j]+a2*h,y2);
    y3=Y[j]+b3*k1+c3*k2;
    k3=h*f(T[j]+a3*h,y3);
    y4=Y[j]+b4*k1+c4*k2+d4*k3;
    k4=h*f(T[j]+a4*h,y4);
    y5=Y[j]+b5*k1+c5*k2+d5*k3+e5*k4;
    k5=h*f(T[j]+a5*h,y5);
    y6=Y[j]+b6*k1+c6*k2+d6*k3+e6*k4+f6*k5;
    k6=h*f(T[j]+a6*h,y6);
    err=abs(r1*k1+r3*k3+r4*k4+r5*k5+r6*k6);
    ynew=Y[j]+n1*k1+n3*k3+n4*k4+n5*k5;
    #% Perbarui ukuran langkah
    if (err<delta) || (h<2*hmin)
      Y = [Y; ynew];
      if (T[j]+h)>br
        T = [T; b];
      else
        T = [T; T[j]+h];
      end
      j=j+1;
    end
    if (err==0)
      s=0;
    else
      s=0.84*(delta*h/err)^(0.25);
    end
    if (s<0.75)&&(h>2*hmin);h = h/2;end
    if (s>1.50)&&(2*h<hmax);h = 2*h;end
    if abs(Y[j])>big || maxi==j;break;end
  end
  sol = [T Y];
  return sol
end
```

### Contoh 1
Diberikan persamaan differensial biasa $$y'=1+y^2$$ dengan masalah nilai awal $y(0)=0$. Berikut merupakan solusi numerik dari PDB tersebut menggunakan metode RKF45 pada $ t\in [0,1.4]$ dengan toleransi kesalahan $2\times10^{-5}$ dan ukuran langkah awal $h_0=0.2$.

```julia
# Hitung Solusi
f(t,y) = 1+y^2
a = 0
b = 1.4
y0 = 0
h0 = 0.2
M  = (b-a)/h0
delta = 2*10^-5
sol = rkf45(f,a,b,y0,M,delta)
```

```julia
using Plots
```

```julia
# Plot Solusi
t = sol[:,1];
y = sol[:,2];
plot(t,y,legend = :false)
scatter!(t,y)
```

# 2 Runge-Kutta untuk Sistem PDB
Sistem persamaan differensial merupakan masalah yang sering ditemui dalam pemodelan matematika. Sistem persamaan differensial biasanya digambarkan dalam suatu bidang fase dan bidang solusi yang dapat diselesaikan secara numerik menggunakan metode Runge-Kutta orde-4 seperti berikut.

```julia
#%%METODE RUNGE-KUTTA ORDE 4 untuk sistem PD
#%
#% Digunakan untuk mencari solusi sistem persamaan
#%  differensial dx/dt = f(t,x,y) dengan x(a) = x0
#%               dy/dt = g(t,x,y) dengan y(a) = y0
#%
#% sol = rungekuttasistem(f,a,b,y0,M)
#% Input  : f    -> fungsi berisi f(t,[x,y]) dan g(t,[x,y])
#%          a,b  -> batas bawah dan atas solusi MNA
#%          y0   -> nilai awal y(a)=[x0,y0]
#%          M    -> banyaknya sub-interval
#% Output : sol  -> solusi PD, sol=[T,X,Y]
#%
#% Digunakan Sebagai Pedoman Praktikum Metode Numerik
#%
#% Lihat juga : heun, taylor, rungekutta
function rungekuttasistem(f,a,b,y0,M)
  M = Int(M)
  h = (b-a)/M;
  T = a:h:b;
  Y = Array{Float64}(undef,M+1,length(y0))
  Y[1,:] = y0;
  for k = 1:M
    f1 = f(T[k]     ,Y[k,:]        );
    f2 = f(T[k]+h/2 ,Y[k,:]+f1*h/2 );
    f3 = f(T[k]+h/2 ,Y[k,:]+f2*h/2 );
    f4 = f(T[k]+h   ,Y[k,:]+f3*h   );
    Y[k+1,:] = Y[k,:] + h/6*(f1+2*f2+2*f3+f4);
  end
  sol = [T Y];
end
```

### Contoh 2
Diberikan sistem persamaan differensial seperti berikut.
\begin{align}
\dot{x}= x+2y \\
\dot{y}=3x+2y
\end{align}
dengan $ x(0)=6 $ dan $ y(0)=4 $.

Berikut merupakan langkah-langkah untuk menggambarkan plot bidang solusi dari sistem persamaan differensial di atas pada interval $ t\in[0, 0.2] $ dengan ukuran langkah $ h = 0.02 $.

```julia
# Hitung Solusi, misalkan x = z(1) dan y = z(2)
f(t,z) = [ z[1]+2*z[2] , 3*z[1]+2*z[2] ]
a = 0
b = 0.2
y0 = [6, 4]
h = 0.02
M = (b-a)/h
sol = rungekuttasistem(f,a,b,y0,M)
```

```julia
# Plot Fase
t = sol[:,1];
x = sol[:,2];
y = sol[:,3];
plt = plot(x,y,label="Bidang Fase",legend=:topleft,xlabel="x",ylabel="y")
```

```julia
# Plot Solusi
t = sol[:,1];
x = sol[:,2];
y = sol[:,3];
plt = plot(t,x,label="solusi x",legend=:topleft)
plot!(t,y,label="solusi y")
```

# 3. Persamaan Differensial Biasa (PDB) ordo tinggi (lebih dari 1)
Selain persamaan differensial orde pertama, sering kali dijumpai masalah matematika dalam bentuk persamaan differensial orde ke-2 atau lebih tinggi. Masalah persamaan differensial orde ke-2 atau lebih tinggi dapat diselesaikan dengan cara mentransformasi persamaan tersebut ke dalam bentuk sistem persamaan differensial orde pertama, kemudian diselesaikan menggunakan metode Runge-Kutta orde-4.

### Contoh 3
Diketahui suatu persamaan differensial orde-2 seperti berikut.
\begin{equation}\label{eq:13 orde2}
x''(t)+4x'(t)+5x(t)=0
\end{equation}
dengan nilai awal $ x(0)=3 $ dan $ x'(0)=-5 $. Berikut merupakan penyelesaian masalah tersebut menggunakan metode Runge-Kutta pada interval $ [0,5] $ dengan $ 50 $ sub-interval dan perbandingan solusi numerik terhadap solusi eksaknya, yaitu $$  x(t)=3e^{-2t}\cos(t)+e^{-2t}\sin(t)  $$

#### Transformasi persamaan differensial orde-2 menjadi sistem persamaan differensial orde-1

Misalkan, $ x'(t)=y(t) $ sehingga persamaan differensial memiliki bentuk
\begin{align*}
				&\  x''(t)+4x'(t)+5x(t)=0 \\
\Leftrightarrow &\ y'(t)+4y(t)+5x(t)=0 \\
\Leftrightarrow &\ y'(t)=-4y(t)-5x(t)
\end{align*}
dan nilai awal $ x'(0)=-5 $ menjadi $ y(0)=-5 $.

Secara lengkap, sistem baru yang equivalen dengan persamaan differensial di atas dapat dituliskan sebagai berikut.
\begin{align}
x'&=y \\
y'&=-4y-5x
\end{align}
dengan $x(0)=3$ dan $ y(0)=-5 $

```julia
# Hitung Solusi
f(t,z) = [ z[2] , -4*z[2]-5*z[1] ]
a = 0
b = 5
y0 = [3, -5]
M  = 50
sol = rungekuttasistem(f,a,b,y0,M)
```

```julia
# Plot Solusi
t = sol[:,1];
x = sol[:,2];
plt = plot(t,x,label="x(t)")
```

```julia
# Hitung Galat
xt(t) = 3*exp(-2*t)*cos(t)+exp(-2*t)*sin(t)
galat = abs.(x .- xt.(t))
```

```julia
# Plot Galat
plt = plot(t,galat)
```

# 4 PDB dengan Masalah Nilai Batas
Masalah nilai batas merupakan bentuk lain dari masalah persamaan differensial biasa.

Masalah nilai batas memiliki bentuk umum berupa persamaan differensial orde 2 yaitu

$$ x''=f(x',x,t) $$

dengan nilai batas pada selang $ [a,b] $, yaitu $ x(a)=\alpha $ dan $ x(b)=\beta $.

Pada materi ini, akan dipelajari dua metode untuk mencari solusi numerik masalah nilai batas, yaitu metode \textit{linear shooting} dan \textit{finite-difference}.

## A. Metode _Linear-Shooting_
Ide dasar dari metode _linear shooting_ adalah melakukan transformasi masalah nilai batas menjadi dua persamaan differensial orde-2 dengan masalah nilai awal.

Misalkan, diketahui masalah nilai batas $ x''= p(t)x'+q(t)x+r(t) $ dengan nilai batas $ x(a)=\alpha $ dan $ x(b)=\beta $. Transformasi dari masalah nilai batas tersebut adalah

$ u''=p(t)u'+q(t)u+r(t) $ dengan $ u(a)=\alpha $ dan $ u'(a)=0 $

$ v''=p(t)v'+q(t)v      $ dengan $ v(a)=0 $ dan $ v'(a)=1 $

Persamaan tersebut dapat diselesaikan menggunakan metode Runge-Kutta orde-4 dengan mengubahnya menjadi bentuk sistem persamaan differensial, sehingga akan didapatkan solusi $ u(t) $ dan $ v(t) $. Selanjutnya, solusi masalah nilai batas $ x(t) $ dapat dicari dengan Persamaan berikut.

\begin{equation}\label{eq:13 linshoot}
x(t) = u(t)+\dfrac{\beta-u(b)}{v(b)}v(t)
\end{equation}

```julia
#%%METODE LINEAR SHOOTING
#%
#% Digunakan untuk mencari solusi persamaan differensial
#%       x''=p(t)x'+q(t)x+r(t)
#% dengan masalah nilai batas x(a) = alpha , x(b)=beta
#%
#% fungsi ini membutuhkan adanya fungsi rungekuttasistem
#%
#% solusi = linearshooting(F1,F2,a,b,alpha,beta,M)
#% Input  : F1,F2     -> SPD hasil transformasi u dan v
#%          a,b       -> batas bawah dan atas solusi MNA
#%          alpha,beta-> nilai awal x(a)=alpha,x(b)=beta
#%          M         -> banyaknya sub-interval
#% Output : solusi    -> solusi PD, sol=[T,X]
#%
#% Digunakan Sebagai Pedoman Praktikum Metode Numerik
#%
#% Lihat juga : rkf45, findiff, rungekuttasistem
function linearshooting(F1,F2,a,b,alpha,beta,M)
  M = Int(M)
  Za = [alpha,0];
  sol = rungekuttasistem(F1,a,b,Za,M);
  U = sol[:,2];

  Za = [0,1];
  sol = rungekuttasistem(F2,a,b,Za,M);
  V = sol[:,2];

  T = sol[:,1];
  X = U + (beta-U[M+1])*V/V[M+1];
  solusi = [T X];
end
```

### Contoh 4
Diketahui persamaan masalah nilai batas seperti berikut.

\begin{equation}\label{eq:13 kasus 13.3}
x''(t)=\dfrac{2t}{1+t^2}x'(t)-\dfrac{2}{1+t^2}x(t)+1
\end{equation}

dengan nilai batas $ x(0)=1.25 $ dan $ x(4)=-0.95 $ pada interval $ [0,4] $.

Berikut merupakan langkah-langkah untuk menyelesaikan masalah nilai batas tersebut menggunakan metode _linear shooting_.

#### Transformasi masalah nilai batas menjadi masalah nilai awal orde-2.

Berdasarkan Persamaan transformasi linear shooting, dua masalah nilai awal orde-2 yang setara dengan masalah nilai batas di atas adalah

$u''=\dfrac{2t}{1+t^2}u'-\dfrac{2}{1+t^2}u+1$ dengan $ u(0)=1.25 $ dan $ u'(0)=0 $

Serta,

$v''=\dfrac{2t}{1+t^2}v'-\dfrac{2}{1+t^2}v$ dengan $ v(0)=0 $ dan $ v'(0)=1 $.

#### Transformasi masing-masing masalah nilai awal orde-2 menjadi sistem persamaan differensial orde-1.
Masing-masing masalah nilai awal orde-2 di atas dapat ditulis dalam bentuk sistem persamaan linear orde pertama, yaitu

$u'  =u_2 $

$u_2'=\dfrac{2t}{1+t^2}u_2-\dfrac{2}{1+t^2}u+1$

dengan $ u(0)=1.25 $ dan $ u_2(0)=0 $. Serta,

$v'	 =v_2$

$v_2' =\dfrac{2t}{1+t^2}v_2-\dfrac{2}{1+t^2}v$

dengan $ v(0)=0 $ dan $ v_2(0)=1 $.

```julia
# Hitung Solusi
F1(t,z) = [ z[2], 2*t/(1+t^2)*z[2]-2/(1+t^2)*z[1]+1 ]
F2(t,z) = [ z[2], 2*t/(1+t^2)*z[2]-2/(1+t^2)*z[1] ]
a = 0
b = 4
alpha = 1.25
beta = -0.95
M = 40
solusi = linearshooting(F1,F2,a,b,alpha,beta,M)
```

```julia
# Plot Solusi
t = solusi[:,1];
x = solusi[:,2];
plt = plot(t,x,legend=:false)
```

# B. Metode Beda-Hingga (Finite-Difference)
Ide dasar dari metode _finite-difference_ adalah mengubah persamaan differensial dengan masalah nilai batas menjadi formula beda-hingga. Bentuk SPL dari beda-hingga dapat dilihat pada bahan perkuliahan

```julia
#%%METODE FINITE-DIFFERENCE
#
#% Digunakan untuk mencari solusi persamaan differensial
#%       x''=p(t)x'+q(t)x+r(t)
#% dengan masalah nilai batas x(a) = alpha , x(b)=beta
#
#% solusi = findiff(p,q,r,a,b,alpha,beta,M)
# Input  : p,q,r     -> fungsi p(t) q(t) dan r(t)
#%          a,b       -> batas bawah dan atas solusi MNA
#%          alpha,beta-> nilai awal x(a)=alpha,x(b)=beta
#          M         -> banyaknya sub-interval
#% Output : solusi    -> solusi PD, sol=[T,X]
#%
#% Digunakan Sebagai Pedoman Praktikum Metode Numerik
#%
#% Lihat juga : rkf45, linearshooting, rungekuttasistem
using LinearAlgebra
function findiff(p,q,r,a,b,alpha,beta,M)
  h = (b-a)/M;
  T = a:h:b;
  T = T[2:end-1];
  #% Bangun Matriks B
  B = -h^2*r.(T);
  B[1] = B[1] + (1+h/2*p(T[1]))*alpha;
  B[end] = B[end] + (1-h/2*p(T[end]))*beta;
  # Bangun Matriks A - Bagian Diagonal
  Ad = 2 .+h^2*q.(T);
  #% Bangun Matriks A - Bagian Bawah Diagonal
  Tbawah = T[2:end];
  Abawah = -1 .-h/2*p.(Tbawah);
  #% Bangun Matriks A - Bagian Atas Diagonal
  Tatas = T[1:end-1];
  Aatas = -1 .+h/2*p.(Tatas);
  A = Tridiagonal(Abawah,Ad,Aatas)
  #% Selesaikan AX=B
  X = A\B;
  T = [a; T; b];
  X = [alpha; X ;beta];
  solusi = [T X];
end

```

### Contoh 5
Diketahui persamaan masalah nilai batas seperti berikut.

\begin{equation}
x''(t)=\dfrac{2t}{1+t^2}x'(t)-\dfrac{2}{1+t^2}x(t)+1
\end{equation}

dengan nilai batas $ x(0)=1.25 $ dan $ x(4)=-0.95 $ pada interval $ [0,4] $.

Berikut merupakan langkah-langkah untuk menyelesaikan masalah nilai batas tersebut menggunakan metode _finite-difference_.

```julia
# Hitung Solusi
p(t)= 2*t/(1+t^2);
q(t)= -2/(1+t^2);
r(t)= 1 + 0*t;
a = 0
b = 4
alpha = 1.25
beta = -0.95
M = 40
solusi = findiff(p,q,r,a,b,alpha,beta,M)
```

```julia
# Plot Solusi
t = solusi[:,1];
x = solusi[:,2];
plt = plot(t,x,legend=:false)
```

# Soal Latihan
Kerjakan soal berikut pada saat kegiatan praktikum berlangsung.

`Nama: ________`

`NIM: ________`

### Soal 1
Diberikan persamaan differensial

$$ \dfrac{dy}{dt} = -y-2t-1 ,\ \text{ dengan } y(0)=2 $$

Gunakan metode Runge-Kutta-Fehlberg 4/5  (RKF45) untuk menyelesaikan PD di atas dengan nilai toleransi $ \delta=10^{-6} $, $ 10^{-12} $ dan $ 10^{-16} $, kemudian bandingkan dengan solusi eksaknya yaitu $ y(t)=e^{-t}-2t+1 $ dan gambarkan solusi beserta titik-titik pembagian ukuran langkah seperti pada **Contoh 1**.

```julia

```

### Soal 2
Ulangi langkah-langkah pada **Contoh 2** untuk menggambarkan plot bidang fase dan bidang solusi dari sistem persamaan berikut.

$$
\begin{align}
	\dot{x}&= -2x-y \\
	\dot{y}&=   x-y \\
\end{align}
$$

dengan $ x(0)=6 $ dan $ y(0)=0 $ pada interval $ t\in[0,\ 5] $ dengan ukuran langkah $ h=0.1 $.

```julia

```

### Soal 3
Diketahui suatu persamaan differensial orde-2 seperti berikut.

$$ 2x''(t)-5x'(t)-3x(t)=45e^{2t} $$

dengan nilai awal $ x(0)=2 $ dan $ x'(0)=1 $. Selesaikan masalah tersebut menggunakan Runge-Kutta pada interval $ [0,\ 2] $ dengan $ h=0.05 $, kemudian bandingkan solusi numerik terhadap solusi eksaknya yaitu $ x(t)=4e^{-t/2}+7e^{3t}-9e^{2t} $ dengan langkah-langkah seperti pada **Contoh 3**.

```julia

```

### Soal 4
Diketahui persamaan masalah nilai batas seperti berikut.

$$ x''(t)=-\frac{2}{t}\ x'(t)+\frac{2}{t^2}\ x(t)+\frac{\sin(t)}{t^2} $$

dengan nilai batas $ x(1)=-0.02 $ dan $ x(6)=0.02 $ pada interval $ [1,6] $. Gunakan metode _linear shooting_ untuk menyelesaikan masalah nilai batas di atas dengan langkah-langkah pada **Contoh 4**.

```julia

```

### Soal 5
Diketahui persamaan masalah nilai batas seperti berikut.

$$ x''(t)=-\frac{2}{t}\ x'(t)+\frac{2}{t^2}\ x(t)+\frac{\sin(t)}{t^2} $$

dengan nilai batas $ x(1)=-0.02 $ dan $ x(6)=0.02 $ pada interval $ [1,6] $. Gunakan metode _finite difference_ untuk menyelesaikan masalah nilai batas di atas dengan langkah-langkah pada **Contoh 5**.

```julia

```


# Exercises For Session 10 (Scipy)

## Exercise 1: Doing Economics with Scipy

Solve the exercises below to become acquainted with scipy.

### Exercise 1.1: Pooled OLS
Consider again the function we use to draw data in Session 9.

```python
# Initialization
import numpy as np
import pandas as pd
from scipy.optimize import minimize

# Setup
np.random.seed(208)
ID            = 20
Periods       = 5
beta          = np.array([1, 0.5, 1.4, 3, 0.2, 5]) # True values

# Define function
def create_data(ID, Periods, beta):

    data_mu       = np.array([1, 0.7, -0.25, 0.6, 0.4, -0.1])
    data_var      = [ [ 1.0000,  -0.2962,    0.3144,    0.5061,   -0.0014,   0.0077],
                    [-0.2962,   1.0000,    0.3082,    0.0301,   -0.0101,   0.5034],
                    [ 0.3144,   0.3082,    1.0000,    0.7012,    0.6674,   0.6345],
                    [ 0.5061,   0.0301,    0.7012,    1.0000,    0.1950,   0.2173],
                    [-0.0014,  -0.0101,    0.6674,    0.1950,    1.0000,   0.1860],
                    [ 0.0077,   0.5034,    0.6345,    0.2173,    0.1860,   1.0000] ]
    year          = np.sum(np.kron(np.linspace(1,Periods,Periods),np.identity(ID)),0)
    idx           = np.sum(np.kron(np.identity(Periods),np.linspace(1,ID,ID)),0)
    X             = np.exp(np.array(np.random.multivariate_normal(data_mu, data_var, ID*Periods)))
    y             = X @ beta + np.random.normal(0,1,ID*Periods)
    data          = np.c_[year, idx, X, y]
    return data

# Call function
data = create_data(ID, Periods, beta)
#print(pd.DataFrame(data))
```

In the last session, we used the fact that the Pooled OLS estimator admits a closed form solution and we estimated the parameters of interests with matrix algebra. However, we can achieve the same result by minimizing the sum of squares residual

\begin{align}
RSS = \sum_{i=1}^{N}(y_i - X_i\beta)^2
\end{align}

Create a new function, call it "SSR", that takes a (6 x 1) vector of parameters as input and produces the RSS formula above as output (Hint: consider using [np.square](https://docs.scipy.org/doc/numpy/reference/generated/numpy.square.html). Else, $x^2$ is coded as "x**2"). Pay attention to the fact that the Pooled OLS objective function must return a scalar. Use [minimize](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize) to find the vector $\beta$ that minimizes the RSS. If you do this correctly, you should get exactly the same point estimates as in Session 9.

### Exercise 1.2: Bertand-Nash Equilibrium
Two firms, $i \in \{1,2\}$, compete on prices à la Bertand-Nash. Let $\mathbf{p} = [p_1 \ p_2]^T$ and let's assume that consumers' demand reads

\begin{align}
q_i(\mathbf{p}) = \frac{\exp(\alpha p_{i})}{\sum_{l \in \{1,2\}}\exp(\alpha p_{l})}, \ \ \forall i.
\end{align}

Firms have the same maginal costs, $c>0$. The first order conditions associated with their optimization problem read

\begin{align}
p_i = c - \frac{1}{\alpha(1-q_i(\mathbf{p}))}
\end{align}

Create a new function, call it "FOC", that takes a (2x1) vector of prices as input and produces the FOC above as output. Set $c=2$ and $\alpha = -0.5$. Then, use one of the routines in [scipy.optimize](https://docs.scipy.org/doc/scipy/reference/optimize.html) to find the vector of prices that constitues the unique Bertand-Nash equilibrium of this game.

A few hints:

- I recommend to use either [root](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.root.html#scipy.optimize.root), [least_squares](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html#scipy.optimize.least_squares) or [fixed_point](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fixed_point.html#scipy.optimize.fixed_point). In the first two cases, you should minimize the following implicit function

\begin{align}
F_i := p_i - c + \frac{1}{\alpha(1-q_i(\mathbf{p}))}
\end{align}

In the third case, you can directly supply the original FOC. Please see the documentation of the two solvers for an explanation.

- Make sure you explicitly define $q_i(\mathbf{p})$ as a function of *both* $p_1$ and $p_2$ inside the objective function. Remember that the solver has to search for the vector of prices s.t. $ FOC(\mathbf{p}) = \mathbf{p}$.

- Assume that both firms set the same initial prices. In other words, use the same numbers as starting values for the solver (e.g. [0,0]). Note: if you use optimize.least_squares or optimize.root, experiment with asymmetric starting values. The solution must always be symmetrical because firms have the same marginal costs.

- No matter which solver you use, you should get always the same result. The correct solution is $[p_1 \ p_2]^T = [6 \ 6]^T$.

## Exercise 2: Searching for Global Minima

Consider the following non-linear function
\begin{align}
 f(x) = 2 - 34\sin(x) + x^2.
\end{align}

It clearly admits one global minimum and multiple local minima.

```python
# Initialization
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Define function
def my_fun(x):
    return 2 - 34*np.sin(x) + x**2

# Plot
x = np.linspace(-20,20,100)
y = my_fun(x)
plt.plot(x, y, 'b-', linewidth=2)
plt.show()
```

Suppose that you use [minimize](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize) to find the value of x that minimizes this function. The solution clearly depends on the starting values that you set.

```python
from scipy.optimize import minimize
x = np.linspace(-20, 20, 10)
for i in x:
    print("The point of minimum attained starting from x = ", round(i,2), "is", minimize(my_fun, i).x)
```

    The point of minimum attained starting from x =  -20.0 is [-4.44768016]
    The point of minimum attained starting from x =  -15.56 is [-16.04478232]
    The point of minimum attained starting from x =  -11.11 is [-10.34161869]
    The point of minimum attained starting from x =  -6.67 is [-4.4476801]
    The point of minimum attained starting from x =  -2.22 is [-4.44768016]
    The point of minimum attained starting from x =  2.22 is [1.48342487]
    The point of minimum attained starting from x =  6.67 is [7.4033948]
    The point of minimum attained starting from x =  11.11 is [7.40339483]
    The point of minimum attained starting from x =  15.56 is [13.24400339]
    The point of minimum attained starting from x =  20.0 is [1.4834249]

Eye-detection suggests that the global minimum is attained at $x=1.4834249$. Use one of the global optimization routines in scipi.optimize, e.g. [brute](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brute.html#scipy.optimize.brute), to find the unique global minimum of the function.


Massimo Nocentini

February 4, 2018: exponential P

Abstract

Exponential $\mathcal{P}$, according to Paul Barry's book.

```python
from sympy import *
from sympy.abc import n, i, N, x, lamda, phi, z, j, r, k, a, alpha

from commons import *
from matrix_functions import *
from sequences import *
import functions_catalog

init_printing()
```

---

```python
m = 10
```

```python
eP = Matrix(m, m, lambda n,k: factorial(n)*binomial(n,k)/factorial(k))
eP
```

```python
inspect(eP)
```

    nature(is_ordinary=False, is_exponential=True)

```python
eP_pm = production_matrix(eP)
eP_epm = production_matrix(eP, exp=True)

eP_pm, eP_epm
```

```python
F = Matrix(m, m, diagonal_func_matrix(factorial))
F_inv = F**(-1)

F, F_inv
```

In order to factorize `eP` as `F U F^{-1}`, for some matrix `U`

```python
B = F_inv * eP * F
B
```

```python
U = Matrix(m, m, rows_shift_matrix(by=1))
U
```

```python
F_inv * U * F
```

```python
F_inv * U * F * B
```

```python
B**(-1) * F_inv * U * F * B
```

```python
F * B**(-1) * F_inv * U * F * B * F_inv
```

---

```python
P = Matrix(m, m, binomial)
```

```python
P_bar = Matrix(m, m, lambda i, j: binomial(i, j) if j < i else 0)
P_bar
```

```python
production_matrix(P_bar[1:,:-1], exp=False), production_matrix(P_bar[1:,:-1], exp=True)
```

```python
j=3
(P_bar**j).applyfunc(lambda i: i/factorial(j))
```

---

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.


# Portable cooling system

A portable cooling system uses canisters of volume 2 L charged with refrigerant R134a at 20°C and quality 0.05 (i.e., vapor mass fraction). Heat is transfered from a person to the canister. Saturated vapor escapes from the canister when the pressure in the canister reaches 30°C, controlled by a relief valve. The cooling system stops working and must be discarded when it is empty of liquid refrigerant.

**Problem:** Determine the cooling density of the system: the ratio of the amount of energy that can be absorbed before the canister is discarded to the initial system mass. Compare this to the cooling density of an ice pack (with latent heat of fusion of $\Delta h_{\text{fus}} = 333.6$ J/g).

```python
# load necessary modules

# Numpy adds some useful numerical types and functions
import numpy as np

# Cantera will handle thermodynamic properties
import cantera as ct

# Pint gives us some helpful unit conversion
from pint import UnitRegistry
ureg = UnitRegistry()
Q_ = ureg.Quantity
```

Specify initial state:

```python
volume = Q_(2, 'liter')
temp_charge = Q_(20, 'degC')
quality_charge = 0.05

initial = ct.Hfc134a()
initial.TX = temp_charge.to('K').magnitude, quality_charge
```

Final state, based on the temperature when the relief valve opens and that the fluid is saturated vapor:

```python
final = ct.Hfc134a()
temp_final = Q_(30, 'degC')
quality_final = 1.0
final.TX = temp_final.to('K').magnitude, quality_final
```

Do a mass balance on the system, over the process from state 1 to state 2:
\begin{equation}
0 = m_{\text{out}} + m_2 - m_1 \;,
\end{equation}
which we can use to find the mass that leaves the canister ($m_{\text{out}}$):

```python
mass_initial = volume / Q_(initial.v, 'm^3/kg')
mass_final = volume / Q_(final.v, 'm^3/kg')
mass_out = mass_initial - mass_final
print(f'mass that left the canister: {mass_out.to(ureg.kg): .2f}')
```

    mass that left the canister: 0.70 kilogram

Next, we can perform an energy balance of the system over the process:
\begin{equation}
Q_{\text{in}} =  m_{\text{out}}  h_{\text{out}} + m_2 u_2 - m_1 u_1 \;,
\end{equation}
where $h_{\text{out}}$ is the enthalpy of the refigerant leaving the system, which has the same properties as the final state in the tank (no liquid, and 30°C):

```python
heat_in = (
    mass_out*Q_(final.h, 'J/kg') +
    mass_final*Q_(final.u, 'J/kg') -
    mass_initial*Q_(initial.u, 'J/kg')
    )
```

THe cooling density (neglecting the canister mass) is
\begin{equation}
CD = \frac{Q_{\text{in}}}{m_1}
\end{equation}

```python
cooling_density = heat_in / mass_initial
cooling_density.ito('W hr/kg')
print(f'cooling density: {cooling_density: .2f}')

cooling_density_ice = Q_(333.6, 'J/g')
cooling_density_ice.ito('W hr/kg')
print(f'cooling density of ice: {cooling_density_ice: .2f}')
```

    cooling density: 49.37 hour * watt / kilogram
    cooling density of ice: 92.67 hour * watt / kilogram


# Predicting the phase transition of the two dimensional Ising model using neural networks

We will in this notebook repeat the process of predicting the phase transition of the two dimensional Ising model as we did with logistic regression, but now we will employ  a _feed-forwad neural network_.

## Predicting the phase of the two-dimensional Ising model using a feed-forward neural network

We now predict the phase transition using Scikit-learn's implementation of a _multilayer perceptron neural network_. This is one of the simpler versions of a neural network.

A neural network consists of layers of _neurons_ or perceptrons. A neuron can take a continuous value where as a perceptron will be either "on" or "off". Scikit-learn's `MLPClassifier` is called a multilayer perceptron model, but it uses neurons. A neuron accepts a vector of inputs $x$ and produces a scalar output $a_i$. A layer of neurons take in the a matrix $X$ and produces an activation vector $a$ consisting of each $a_i$ from each neuron in that layer.

\begin{align}
    z^{(l)} = a^{(l - 1)}w^{(l)} + b^{(l)},
\end{align}

where $w^{(l)}$ is that layers _weights_ and $b^{(l)}$ is that layers _biases_. The activation $a^{(l)}$ is now computed by

\begin{align}
    a^{(l)} = f(z^{(l)}),
\end{align}

where $f(z)$ is an _activation function_. The first activation is given by $a^{(0)} = X$, where $X$ is the input data.

It is common to use the cross-entropy as the cost function for a categorical neural network. This is the same the one used in logistic regression.

### Training a neural network

The training of a neural network is a tricky affair as each neuron in each layer must be updated to minimize a cost function. The minimization process resembles that of other classifiers and regressors in that we use an optimization algorithm, e.g., stochastic gradient descent, to compute the change in the weights and biases in order to find a minimum of the cost function. The tricky part comes in computing the gradients of the weights and biases in the neural network. Due to an ingenious technique called _backpropagation_ this can be done within reasonable time. The algorithm is listed below.

1. Do a feedforward pass, i.e., compute all activations $a^{(l)}$ as listed above.
2. Calculate the error of the weights and biases using
\begin{align}
    \delta_{j}^{(l)} = \frac{\partial C}{\partial a_{j}^{(l)}} \frac{\mathrm{d}f(z_j^{(l)})}{\mathrm{d}z}.
\end{align}
3. Propagate the error backwards (hence backpropagation) using
\begin{align}
    \delta_{j}^{(l)} = \left(
        \sum_{k}\delta_k^{(l + 1)}\omega_{kj}^{(l + 1)}
    \right)\frac{\mathrm{d}f(z_j^{(l)}}{\mathrm{d}z}.
\end{align}
4. Compute the change in the gradients of the weights and biases.
\begin{align}
    \omega^{(l)} &\to \omega^{(l)} - \frac{\eta}{m}\sum_{k}\delta^{(l)}_ka_k^{(l - 1)}, \\
    b^{(l)} &\to b^{(l)} - \frac{\eta}{m}\sum_{k}\delta^{(l)}_k,
\end{align}
    where $\eta$ is the learning rate and $m$ is the number of elements (batch size in the case of stochastic gradient descent).

We will do a large search over a two-dimensional parameter grid for varying hidden layer sizes and learning rate.

## DIY Neural net from Keras

An easier approach is to use libraries such as Tensorflow or Keras (which by default uses Tensorflow as a backend) to specify how the neural net should be. The libraries then build a neural net based on your specifications and lets you fit and predict on your very own construct. As a bonus, Tensorflow builds a graph from the executing code thus figuring out the most optimal way of executing the code and which parts of the code that can be done in parallel.

A perk of building a neural net this way is that we have a lot of freedom in how we build our neural net. For instance, we can set individual activation functions for each layer, decide the optimization technique to use, the cost function to minimize, which metric to evaluate our model on etc.

We now build a neural net from Keras with three hidden layers. The input and the hidden layer uses the activation function

\begin{align}
    f(z_j) = \tanh(z_j),
\end{align}

whereas the output layer uses the _softmax_ activation function.

\begin{align}
    g(z_j) = \frac{e^{z_j}}{\sum_{i = 1}^{n} e^{z_i}}.
\end{align}

We avoid using the tangent hyperbolicus for the output layer as $f(z_j) \in [-1, 1]$ as opposed to the softmax function with $g(z_j) \in [0, 1]$. The latter more fits the categorical classification. Another perk of using the softmax activation function is that it better weighs all the classes in a multiclassification setting much as the partition function. We use stochastic gradient descent as our optimizer and the cross-entropy (shown in the notebook on logistic regression) as our cost function.

```python
clf = km.Sequential()

clf.add(
    kl.Dense(50, activation="tanh", input_dim=X_train.shape[1])
)
clf.add(
    kl.Dense(100, activation="tanh")
)
clf.add(
    kl.Dense(200, activation="tanh")
)

clf.add(
    kl.Dense(2, activation="softmax")
)
clf.compile(
    loss="binary_crossentropy",
    optimizer=ko.SGD(
        lr=0.01
    ),
    metrics=["accuracy"]
)
```

We have to convert our integer labels to categorical values in a format that Keras accepts.

```python
y_train_k = to_categorical(y_train[:, np.newaxis])
y_test_k = to_categorical(y_test[:, np.newaxis])
y_critical_k = to_categorical(labels[critical][:, np.newaxis])
```

We now fit our model for $10$ epochs and validate on the test data.

```python
history = clf.fit(
    X_train, y_train_k,
    validation_data=(X_test, y_test_k),
    epochs=10,
    batch_size=200,
    verbose=True
)
```

    Train on 65000 samples, validate on 65000 samples
    Epoch 1/10
    65000/65000 [==============================] - 6s 88us/step - loss: 0.6629 - acc: 0.6971 - val_loss: 0.5873 - val_acc: 0.8317
    Epoch 2/10
    65000/65000 [==============================] - 5s 72us/step - loss: 0.4253 - acc: 0.9397 - val_loss: 0.2529 - val_acc: 0.9956
    Epoch 3/10
    65000/65000 [==============================] - 5s 76us/step - loss: 0.1287 - acc: 0.9950 - val_loss: 0.0639 - val_acc: 0.9989
    Epoch 4/10
    65000/65000 [==============================] - 3s 44us/step - loss: 0.0419 - acc: 0.9992 - val_loss: 0.0300 - val_acc: 0.9990
    Epoch 5/10
    65000/65000 [==============================] - 3s 46us/step - loss: 0.0222 - acc: 0.9994 - val_loss: 0.0189 - val_acc: 0.9991
    Epoch 6/10
    65000/65000 [==============================] - 3s 43us/step - loss: 0.0146 - acc: 0.9994 - val_loss: 0.0137 - val_acc: 0.9991
    Epoch 7/10
    65000/65000 [==============================] - 3s 48us/step - loss: 0.0106 - acc: 0.9994 - val_loss: 0.0107 - val_acc: 0.9992
    Epoch 8/10
    65000/65000 [==============================] - 4s 56us/step - loss: 0.0083 - acc: 0.9995 - val_loss: 0.0087 - val_acc: 0.9992
    Epoch 9/10
    65000/65000 [==============================] - 4s 59us/step - loss: 0.0068 - acc: 0.9995 - val_loss: 0.0074 - val_acc: 0.9992
    Epoch 10/10
    65000/65000 [==============================] - 3s 53us/step - loss: 0.0057 - acc: 0.9996 - val_loss: 0.0064 - val_acc: 0.9992

We now evaluate the model.

```python
train_accuracy = clf.evaluate(X_train, y_train_k, batch_size=200)[1]
test_accuracy = clf.evaluate(X_test, y_test_k, batch_size=200)[1]
critical_accuracy = clf.evaluate(data[critical], y_critical_k, batch_size=200)[1]

print ("Accuracy on train data: {0}".format(train_accuracy))
print ("Accuracy on test data: {0}".format(test_accuracy))
print ("Accuracy on critical data: {0}".format(critical_accuracy))
```

    65000/65000 [==============================] - 2s 34us/step
    65000/65000 [==============================] - 4s 54us/step
    30000/30000 [==============================] - 3s 103us/step
    Accuracy on train data: 0.999538461978619
    Accuracy on test data: 0.9992307699643649
    Accuracy on critical data: 0.9312000012397766

Then we plot the ROC curve for three datasets.

```python
fig = plt.figure(figsize=(20, 14))

for (_X, _y), label in zip(
    [
        (X_train, y_train_k),
        (X_test, y_test_k),
        (data[critical], y_critical_k)
    ],
    ["Train", "Test", "Critical"]
):
    proba = clf.predict(_X)
    fpr, tpr, _ = skm.roc_curve(_y[:, 1], proba[:, 1])
    roc_auc = skm.auc(fpr, tpr)

    print ("Keras AUC ({0}): {1}".format(label, roc_auc))

    plt.plot(fpr, tpr, label="{0} (AUC = {1})".format(label, roc_auc), linewidth=4.0)

plt.plot([0, 1], [0, 1], "--", label="Guessing (AUC = 0.5)", linewidth=4.0)

plt.title(r"The ROC curve for Keras", fontsize=18)
plt.xlabel(r"False positive rate", fontsize=18)
plt.ylabel(r"True positive rate", fontsize=18)
plt.axis([-0.01, 1.01, -0.01, 1.01])
plt.xticks(fontsize=18)
plt.yticks(fontsize=18)
plt.legend(loc="best", fontsize=18)
plt.show()
```

We see that we get much the same performance as for the `MLPClassifier` from Scikit-learn.

## Conclusion

We have successfully used  a neural net using Keras and predicted the phase transition of the two-dimensional Ising model. As opposed to the logistic regression we get a much better performance on all datasets. More importantly we successfully predict reasonable values on the critical dataset which is extrapolated from the training and testing data.