# Theory
The complex transverse water proton magnetization $M$
in the rotating frame satisfies the Bloch-Torrey PDE:
\begin{alignat}{4}
	  & \frac{\partial}{\partial t}{M(x,t)} &                       & = -\jmath\gamma f(t) \mathbf{g} \cdot x \,M(x,t)
	+ \nabla \cdot (\mathcal{D}_0 \nabla M(x,t)), &                                       & \quad x \in \Omega,
\end{alignat}
where $\gamma=267.513\, rad\,\mu s^{-1}T^{-1}$ is the
gyromagnetic ratio of the water proton, $\jmath$ is the imaginary unit,
$\mathcal{D}_0$ is the intrinsic diffusion coefficient in the neuron compartment $\Omega$.

We make the following assumptions:
- negligible membrane permeability
- PGSE sequence with $\delta \ll \Delta$
- Diffusion in a closed 2D region $\Omega$

We transform the Bloch-Torrey equation to a diffusion equation subjecting to zero initial conditions and non-zero Neumann boundary conditions.
Define
$$
\omega(x,t) \equiv M(x,t+\delta)
-\rho e^{-4\pi^2 \mathcal{D}_0 \|q\|^2 t} e^{-2\pi\jmath q \cdot x},  \quad x \in \Omega,
\quad t \in [0,\Delta-\delta]$$
where $q =\delta\gamma \mathbf{g}/2 \pi$
so that $\omega(x,t)$ satisfies the diffusion equation:
$$
\frac{\partial}{\partial t}{\omega(x,t)}
= \nabla \cdot (\mathcal{D}_0 \nabla \omega(x,t)), \quad x \in \Omega,
\quad t \in [0,\Delta-\delta],
$$
$$ \mathcal{D}_0 \nabla \omega(x,t) \cdot \mathbf{n}
	= \jmath\mathcal{D}_0\rho \mathbf{q} \cdot \mathbf{n}
	e^{--4\pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 t}e^{-2\pi \jmath \mathbf{q}\cdot x},
	\quad x \in \partial \Omega , \quad t \in [0,\Delta-\delta],
$$
$$
	\omega(x,0) = 0, \quad x \in \Omega.
$$

The diffusion MRI signal $\mathcal{S}$ can be formulated in terms of $\omega$:
$$
\mathcal{S} = |\Omega|\rho e^{-4\pi^2 \mathcal{D}_0  \|\mathbf{q}\|^2 (\Delta - \delta)} + \overline{\omega}(\mathbf{q}, \Delta-\delta),
$$
with
$$
\overline{\omega}(\mathbf{q}, t) = \mathcal{D}_0 \int_{0}^{t} \int_{\partial \Omega} -2\pi \jmath \mathbf{q} \cdot \mathbf{n}
e^{-4\pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 (t-\tau)} \omega(\mathbf{x}, \tau) e^{2\pi \jmath \mathbf{q} \cdot \mathbf{x}} ds_{\mathbf{x}} d\tau
$$
and
$$\rho = 1 A/m.$$

```
!pip install cupy-cuda101
!pip install scipy
!wget https://github.com/fachra/FourierPotential/raw/main/geo.mat
import warnings
import scipy.io as io
import matplotlib.pyplot as plt
import time
try:
    import cupy as xp
    import numpy as np
    xp.empty(1)
except:
    import numpy as np
    xp = np
    warnings.warn("GPU/cupy is not detected, Numpy is used. Modify Edit -> Notebook settings -> Hardware accelerator -> GPU to use cupy.")

ctype = xp.complex64  # complex float number
rtype = xp.float64
```

```
def get_q(sdelta, bdelta, b):
    # generate 3 q-vectors
    # (just an examplary function to give some q-vectors )
    abs_q = np.sqrt(b/(bdelta-sdelta/3))/(2*np.pi)
    qx = xp.linspace(0, abs_q, 10).reshape((1, -1))
    qy = xp.zeros(10).reshape((1, -1))
    q = xp.concatenate((qx, qy), axis=0)  # [2 x n_q] (1/um)
    q_square = (q**2).sum(axis=0).reshape((1, -1))  # [1 x n_q]
    return q, q_square, abs_q

# dMRI settings
D0 = 1e-3  # (um^2/us)
sdelta, bdelta = 2500, 5000  # (us)
b = 3000  # (us/um^2)
q, q_square, q_max = get_q(sdelta, bdelta, b)
print(q)
```

    [[0.         0.01500527 0.03001054 0.04501582 0.06002109 0.07502636
      0.09003163 0.1050369  0.12004218 0.13504745]
     [0.         0.         0.         0.         0.         0.
      0.         0.         0.         0.        ]]

```
def get_geo(mat, dtype=xp.float64):
    boundary_points = xp.array(mat['boundary_points'], dtype=dtype)  # [2 x n_points]
    boundary_normals = xp.array(mat['boundary_normals'], dtype=dtype)  # [2 x n_points]
    dl = xp.array(mat['dl'], dtype=dtype)  # [1 x n_points]
    curvature = xp.array(mat['curvature'], dtype=dtype)  # [1 x n_points]
    region_area = xp.array(mat['omega_area'], dtype=dtype)  # [1 x 1]
    return boundary_points, boundary_normals, dl, curvature, region_area

# load saved model
mat = io.loadmat('geo.mat')
boundary_points, boundary_normals, dl, curvature, region_area = get_geo(mat)

if xp is np:
    plt.plot(boundary_points[0,:], boundary_points[1,:], '*')
else:
    # convert cupy ndarray to numpy array
    plt.plot(xp.asnumpy(boundary_points[0,:]), xp.asnumpy(boundary_points[1,:]), '*')
plt.title('Omega')
```

# Single layer potential representation
We choose to represent the solution $\omega(x,t)$ as a single layer potential with a density function $\mu$ defined on $\partial \Omega$:
$$\omega(\mathbf{x},t) = S[\mu](\mathbf{x}, t) \equiv \mathcal{D}_0 \int_{0}^{t} \int_{\partial \Omega} G(\mathbf{x}-\mathbf{y},t-\tau) \mu(\mathbf{y},\tau) ds_{\mathbf{y}} d\tau$$
where $G(\mathbf{x},t)$ is the fundamental solution of the 2D diffusion equation in a box $[-L/2, L/2] \times [-L/2, L/2]$ with periodic boundary conditions.
The density $\mu$ is chosen to be a causal function and is determined by imposing the Neumann boundary conditions:
$$ \lim_{\mathbf{x}\rightarrow \mathbf{x}_0\in \partial \Omega} \mathcal{D}_0 \nabla S[\mu](\mathbf{x},t) \cdot \mathbf{n} = \mathcal{N}(\mathbf{x}_0,t)
\equiv  2\pi \jmath \rho \mathcal{D}_0 \mathbf{q}\cdot \mathbf{n}
e^{-4\pi^2 \mathcal{D}_0  \|\mathbf{q}\|^2 t}e^{-2\pi \jmath \mathbf{q}\cdot\mathbf{x}_0}, \mathbf{x}_0 \in \partial \Omega.$$

Due the singularity of the fundamental solution $G$ when $\tau \rightarrow t$, we split the single layer potential into a short time part and a long time part at time $t-\eta$, with $\eta$ a small quantity to be determined:
$$S[\mu](\mathbf{x},t) = \color{red}{\int_{t-\eta}^{t} ...} + \int_0^{t-\eta} ... = \color{red}{S_{short}[\mu](\mathbf{x},t)} + S_{long}[\mu](\mathbf{x},t)$$

According to [[Greengard1989]](https://www.math.nyu.edu/faculty/greengar/preprint_GS1.pdf), the asymptotic expressions in two dimensions for the short time part is:
\begin{equation}
	S_{short}[\mu](\mathbf{x}_0, t) = \sqrt{\frac{\mathcal{D}_0 \eta}{\pi}} \mu(\mathbf{x}_0, t) + O(\eta^{3/2})
\end{equation}

The smooth part of the single layer potential has a Fourier representation:
\begin{equation}
S_{long}[\mu](\mathbf{x}_0, t) = \mathcal{D}_0 \sum_{\nu = -\nu_{max}}^{\nu_{max}} \hat{f}(\nu, t) e^{2\pi \jmath \nu \cdot \mathbf{x}_0} \Delta \nu^2
\end{equation}
with
$$\hat{f}(\nu, t) =  \int_{\partial \Omega}\int_{0}^{t-\eta} e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2(t-\tau)}  e^{-2\pi \jmath \nu \cdot \mathbf{y}} \mu(\mathbf{y}, \tau)d\tau ds_{\mathbf{y}}, \quad \nu \in [-\nu_{max}, \nu_{max}]^2.$$

For more details, please read our paper.

```
def get_time(sdelta, bdelta, dt, dtype=xp.float64):
    return xp.linspace(0, bdelta-sdelta, round((bdelta-sdelta)/dt) +
                       1, dtype=dtype).reshape((1, -1))

def get_freq(freq_max, q_max, freq_resolution, dtype=xp.float64):
    n_temp = int(2*xp.ceil((freq_max+q_max)/freq_resolution) + 1)
    temp = xp.linspace(-freq_max-q_max, freq_max+q_max,
                       n_temp, dtype=dtype)
    freq_y, freq_x = xp.meshgrid(-temp, temp)
    # freq: [2 x n_freqs]
    freq = xp.concatenate(
        (freq_x.reshape((1, -1)), freq_y.reshape((1, -1))), axis=0)
    freq_square = (freq**2).sum(axis=0).reshape(1, -1)  # [1 x n_freqs]
    return freq, freq_square

# simulation setting
dt = 5  # (us)
n_eta = 2
eta = n_eta*dt  # (us)
freq_resolution = 0.01  # (um^-1)
freq_max = 1  # (um^-1)

time_val = get_time(sdelta, bdelta, dt)
freq, freq_square = get_freq(freq_max, 0, freq_resolution)

# some constants
n_points, n_time = boundary_points.shape[1], time_val.shape[1]
n_freqs, n_q = freq.shape[1], q.shape[1]
print('number of points/samples on the boundary: {}'.format(n_points))
print('number of time steps: {}'.format(n_time))
print('number of Fourier modes: {}'.format(n_freqs))
print('number of q-vectors: {}'.format(n_q))
# init
mu = xp.empty((n_q, n_points, n_time), dtype=ctype)
S_short = xp.empty((n_q, n_points, n_time), dtype=ctype)
K_long = xp.zeros((n_q, n_points, n_time), dtype=ctype)
fhat = xp.zeros((n_q, n_freqs, n_time), dtype=ctype)
omega_bar = xp.zeros((n_q, n_time), dtype=ctype)
```

    number of points/samples on the boundary: 128
    number of time steps: 501
    number of Fourier modes: 40401
    number of q-vectors: 10

The complex-valued Neumann data is defined as
$$\mathcal{N}(\mathbf{x}_0, t) = 2\pi \jmath\mathbf{q} \cdot \mathbf{n}(\mathbf{x}_0)
e^{-4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 t}e^{-2\pi \jmath\mathbf{q}\cdot\mathbf{x}_0},
\quad \mathbf{x}_0 \in \partial \Omega , \quad t \in [0,\Delta-\delta].$$

```
def neumann(points, normals, q, q_square, t_val, D0=1e-3):
    n_q = q_square.shape[1]
    xq = 2*xp.pi*1j*((normals.T @ q) *
                     xp.exp(-2*xp.pi*1j*points.T @ q)).T.reshape(n_q, -1, 1)
    qt = xp.exp(-4*xp.pi**2*D0*q_square.T @ t_val).reshape(n_q, 1, -1)
    return xq @ qt

# neumann data [n_q x n_points x n_time]
neu_data = neumann(boundary_points, boundary_normals, q, q_square, time_val, D0=D0)
```

## $0<= t <= \eta$
When $0<= t <= \eta$, the expression of the density funtion $\mu$ is
$$\mu(\mathbf{x}_0, t) = \frac{2\mathcal{N}(\mathbf{x}_0, t)}{1 - \sqrt{\frac{\mathcal{D}_0 t}{\pi}}\kappa(\mathbf{x}_0)},
	\quad \mathbf{x}_0 \in \partial \Omega , \quad t \in [0,\eta]$$
and
\begin{equation}
	S_{short}[\mu](\mathbf{x}_0, t) = \sqrt{\frac{\mathcal{D}_0 t}{\pi}} \mu(\mathbf{x}_0, t)
\end{equation}

```
# compute density mu for [0, eta=n_eta*dt] (us)
mu[:, :, :n_eta+1] = 2 * neu_data[:, :, :n_eta+1] / (1 - curvature.T @ xp.sqrt(D0*time_val[:, :n_eta+1]/xp.pi))
S_short[:, :, :n_eta + 1] = xp.sqrt(D0*time_val[:, :n_eta+1]/xp.pi) * mu[:, :, :n_eta+1]
```

## $\eta< t <= 2\eta$
1. We compute the Fourier coefficients $\hat{f}$ at time $t \in (\eta,\ 2\eta]$:
$$\hat{f}(\nu, t) =  e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2 \Delta t}\hat{f}(\nu, t - \Delta t) + \underbrace{\int_{\partial \Omega}\int_{t-\eta-\Delta t}^{t-\eta} e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2(t-\tau)}  e^{-2\pi \jmath \nu \cdot \mathbf{y}} \mu(\mathbf{y}, \tau)d\tau ds_{\mathbf{y}}}_{\hat{f}_{temp1}(\nu, t)}$$
with
$$\hat{f}_{temp1}(\nu, t) = \int_{\partial \Omega} 4\pi \jmath \mathbf{q}\cdot \mathbf{n} e^{-2\pi \jmath (\mathbf{q}+\nu)\cdot \mathbf{y}} \underbrace{\int_{t-\eta-\Delta t}^{t-\eta} \dfrac{e^{-4\pi^2 \mathcal{D}_0 \left[\|\nu\|^2(t-\tau)+\|\mathbf{q}\|^2 \tau\right]}}{1-\sqrt{\frac{\mathcal{D}_0 \tau}{\pi}}\kappa(\mathbf{y})} d\tau}_{p} ds_{\mathbf{y}}$$
and
\begin{equation}
	p =\begin{cases} -\dfrac{2\pi e^{-4\pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 t}}{\mathcal{D}_0 \kappa^2(\mathbf{y})} \Big[\kappa(\mathbf{y})\sqrt{\frac{\mathcal{D}_0}{\pi}}\left(\sqrt{t-\eta}-\sqrt{t-\eta-\Delta t}\right)   + \ln\left(\dfrac{1-\kappa(\mathbf{y})\sqrt{\frac{\mathcal{D}_0}{\pi}(t-\eta)}}{1 - \kappa(\mathbf{y})\sqrt{\frac{\mathcal{D}_0}{\pi}(t-\eta-\Delta t)}}\right) \Big] & \|\nu\| = \|\mathbf{q}\|     \\
		 e^{-4 \pi^2 \mathcal{D}_0 \left[\|\mathbf{q}\|^2(t-\eta) +\|\nu\|^2 \eta\right]} \Big[\dfrac{1+e^{4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2)\Delta t}(4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2)\Delta t- 1)}{\Delta t(4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2))^2 \left(1 - \kappa(\mathbf{y})\sqrt{\frac{\mathcal{D}_0}{\pi}(t-\eta-\Delta t)}\right)} + \dfrac{e^{4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2)\Delta t} - 4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2)\Delta t- 1}{\Delta t(4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2-\|\nu\|^2))^2 \left(1 - \kappa(\mathbf{y})\sqrt{\frac{\mathcal{D}_0}{\pi}(t-\eta)}\right)}\Big] & \|\nu\| \neq \|\mathbf{q}\|
	\end{cases}
\end{equation}

2. We compute the long time part $K_{long}[\mu]$ at time $t$ by applying the inverse discrete Fourier transform:
$$K_{long}[\mu](\mathbf{x}_0, t) = \mathcal{D}_0 \sum_{\nu=-\nu_{max}}^{\nu_{max}} 2\pi \jmath\nu \cdot \mathbf{n}\hat{f}(\nu, t) e^{2\pi \jmath \nu \cdot \mathbf{x}_0} \Delta\nu^2, 	\quad \mathbf{x}_0 \in \partial \Omega.$$

```
# compute fhat and K_long for [(n_eta+1)*dt, 2*eta=2*n_eta*dt] (us)
def func_p(t, eta, dt, p_weights, freq_square, q_square, zero_mask, curvature, D0):
    # intg: [n_q x n_freqs x n_points]
    D0_curva_coeff = xp.sqrt(D0/xp.pi)*curvature
    denom1 = 1/(1-D0_curva_coeff*xp.sqrt(t-eta-dt))
    denom2 = 1/(1-D0_curva_coeff*xp.sqrt(t-eta))
    denom = xp.concatenate([denom1, denom2], axis=0)  # [2 x n_points]
    # [n_q x n_freqs x n_points]
    intg = (xp.exp(-4*(xp.pi**2)*D0*(q_square.T * (t-eta) +
                                     freq_square*eta)).reshape(q_square.shape[1], -1, 1) *
            p_weights) @ denom
    # when a == 0
    for iq in range(zero_mask.shape[0]):
        mask_temp = zero_mask[iq, :]
        if mask_temp.any():
            zero_inds = mask_temp.nonzero()
            for ind in zero_inds:
                intg[iq, ind, :] = \
                    -(2*xp.pi/D0)*xp.exp(-4*(xp.pi**2)*D0*q_square[0, iq]*t) * \
                    (curvature*xp.sqrt(D0/xp.pi)*(xp.sqrt(t-eta)-xp.sqrt(t-eta-dt)) +
                     xp.log((1-curvature*xp.sqrt(D0*(t-eta)/xp.pi)) /
                            (1-curvature*xp.sqrt(D0*(t-eta-dt)/xp.pi)))) / \
                    curvature**2
    return intg

# reuseable intermediate results
## [n_q x n_freqs]
freq_zero_mask = (xp.abs(q_square.T-freq_square) <= 1e-20)
## [n_freqs x n_points]
fourier_bases = xp.exp(-2*xp.pi*1j*freq.T @ boundary_points)
## [1 x n_freqs]
exp_dt = xp.exp(-4*(xp.pi**2)*D0*dt*freq_square)
## [n_q x n_freqs x n_points]
exponential_term = xp.exp(-2*xp.pi*1j*(freq.T.reshape(1, -1, 2) + q.T.reshape(-1, 1, 2)) @ boundary_points)
## [n_q x n_points]
normal_derivative_coef = 4*xp.pi*1j*q.T @ boundary_normals
## [n_q x n_freqs x n_points]
coef_temp1 = normal_derivative_coef.reshape(n_q, 1, n_points)*exponential_term
## [n_q x n_freqs]
a = 4*(xp.pi**2)*D0*(q_square.T-freq_square)
## [1 x n_points]
Kshort_appro = 1-xp.sqrt(D0*eta/xp.pi)*curvature
## [n_points x n_freqs]
Klong_fourier_coef = ((2*xp.pi*1j*boundary_normals.T @ freq) * fourier_bases.T.conj())*(D0*freq_resolution**2)
## [n_q x n_freqs x 2]
p_weights = xp.empty((n_q, n_freqs, 2), dtype=rtype)
p_weights[:, :, 0] = (1+(a*dt-1)*xp.exp(a*dt))/((a**2)*dt)
p_weights[:, :, 1] = (xp.exp(a*dt)-a*dt-1)/((a**2)*dt)

# compute fhat and K_long for [(n_eta+1)*dt, 2*eta=2*n_eta*dt] (us)
T2 = np.array(range(n_eta+1, 2*n_eta+1))
for it in T2:
    # [n_q x n_freqs x n_points]
    fhat_integrand = coef_temp1 * func_p(it*dt, eta, dt, p_weights, freq_square, q_square, freq_zero_mask, curvature, D0)
    # [n_q x n_freqs]
    fhat_temp1 = xp.squeeze(fhat_integrand @ dl.T)
    # [n_q x n_freqs x n_time]
    fhat[:, :, it] = fhat_temp1 + fhat[:, :, it-1] * exp_dt

K_long[:, :, T2] = Klong_fourier_coef @ fhat[:, :, T2]
```

## $2\eta< t <= \Delta-\delta$
1. We compute the Fourier coefficients $\hat{f}$ at time $t \in (2\eta, \Delta-\delta]$:
$$\hat{f}(\nu, t) =  e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2 \Delta t}\hat{f}(\nu, t - \Delta t) + \underbrace{\int_{\partial \Omega}\int_{t-\eta-\Delta t}^{t-\eta} e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2(t-\tau)}  e^{-2\pi \jmath \nu \cdot \mathbf{y}} \mu(\mathbf{y}, \tau)d\tau ds_{\mathbf{y}}}_{\hat{f}_{temp2}(\nu, t)}$$
where
\begin{align}
	\hat{f}_{temp2}(\nu, t)  = & \int_{\partial \Omega} 2\left(1-\sqrt{\frac{\mathcal{D}_0 \eta}{\pi}}\kappa(\mathbf{y})\right)^{-1} e^{-2\pi \jmath \nu \cdot \mathbf{y}}\times           \\ &\bigl( 2\pi \jmath \mathbf{q}\cdot \mathbf{n} e^{-2\pi \jmath \mathbf{q}\cdot \mathbf{y}}\underbrace{\int_{t-\eta-\Delta t}^{t-\eta} e^{-4\pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2 \tau+\|\nu\|^2(t-\tau))} d\tau}_{h1}- \underbrace{\int_{t-\eta-\Delta t}^{t-\eta} K_{long}[\mu](\mathbf{y}, \tau)e^{-4\pi^2 \mathcal{D}_0 \|\nu\|^2(t-\tau)}d\tau}_{h2} \bigr) ds_{\mathbf{y}}.
\end{align}
with
\begin{equation}
	h1 =\begin{cases} \Delta t \cdot e^{-4 \pi^2 \mathcal{D}_0 \|\nu\|^2t}                                                                                & \|\mathbf{q}\| = \|\nu\|    \\
		\dfrac{e^{-4 \pi^2 \mathcal{D}_0 \left[\|\mathbf{q}\|^2 (t-\eta-\Delta t)+\|\nu\|^2 (\eta+\Delta t)\right]}-e^{-4 \pi^2 \mathcal{D}_0 \left[\|\mathbf{q}\|^2 (t-\eta)+\|\nu\|^2 \eta\right]}}{4 \pi^2 \mathcal{D}_0 (\|\mathbf{q}\|^2 - \|\nu\|^2)} & \|\mathbf{q}\| \neq \|\nu\|
	\end{cases}
\end{equation}
and
\begin{equation}
	h2 =\begin{cases} \frac{\Delta t}{2} \left[K_{long}[\mu](\mathbf{y},t-\eta)+K_{long}[\mu](\mathbf{y}, t-\eta-\Delta t)\right]    &\|\nu\| = 0     \\
		e^{-4 \pi^2 \mathcal{D}_0 \|\nu\|^2 \eta}\cdot  \Big[  \frac{1-e^{-4 \pi^2 \mathcal{D}_0 \|\nu\|^2\Delta t}(4 \pi^2 \mathcal{D}_0 \|\nu\|^2\Delta t+1)}{(4 \pi^2 \mathcal{D}_0 \|\nu\|^2)^2\Delta t}K_{long}[\mu](\mathbf{y}, t-\eta-\Delta t) + \frac{e^{-4 \pi^2 \mathcal{D}_0 \|\nu\|^2\Delta t}+ 4 \pi^2 \mathcal{D}_0 \|\nu\|^2\Delta t-1}{(4 \pi^2 \mathcal{D}_0 \|\nu\|^2)^2\Delta t}K_{long}[\mu](\mathbf{y}, t-\eta)\Big] &\|\nu\| \neq 0
	\end{cases}
\end{equation}

2. We compute the long time part $K_{long}[\mu]$ at time $t$ by applying the inverse discrete Fourier transform:
$$K_{long}[\mu](\mathbf{x}_0, t) = \mathcal{D}_0 \sum_{\nu=-\nu_{max}}^{\nu_{max}} 2\pi \jmath\nu \cdot \mathbf{n}\hat{f}(\nu, t) e^{2\pi \jmath \nu \cdot \mathbf{x}_0} \Delta\nu^2, 	\quad \mathbf{x}_0 \in \partial \Omega.$$

```
# compute fhat and K_long for [(2*n_eta+1)*dt, bdelta-sdelta] (us)
def func_h1(t, eta, dt, freq_square, q_square, zero_mask, D0):
    # output: [n_q x n_freqs x 1]
    # [n_q x n_freqs]
    h1 = (xp.exp(-4*(xp.pi**2)*D0*(q_square.T*(t-eta-dt) + freq_square*(eta+dt))) -
          xp.exp(-4*(xp.pi**2)*D0*(q_square.T*(t-eta) + freq_square*eta))) / \
        (4*(xp.pi**2)*D0*(q_square.T-freq_square))
    for iq in range(zero_mask.shape[0]):
        mask_temp = zero_mask[iq, :]
        if mask_temp.any():
            zero_inds = mask_temp.nonzero()
            for ind in zero_inds:
                h1[iq, ind] = dt * \
                    xp.exp(-4*(xp.pi**2)*D0*q_square[0, iq]*t)
    return h1.reshape(q_square.shape[1], -1, 1)

def func_h2(exp_eta, h2_weights, klong):
    # exp_eta: [1 x n_freqs], h2_weights: [n_freqs x 2], klong: [n_q, n_points, 2]
    # output: [n_q x n_freqs x n_points]
    return exp_eta.reshape(1, -1, 1)*(h2_weights @ klong.transpose(0, 2, 1))

# reuseable intermediate results
## [1 x n_freqs]
exp_eta = xp.exp(-4*(xp.pi**2)*D0*eta*freq_square)
## [n_q x 1 x n_points]
fhat_h1_coef = (normal_derivative_coef/Kshort_appro).reshape(n_q, 1, n_points)
## [n_freqs x n_points]
fhat_h2_coef = (2*fourier_bases/Kshort_appro).reshape(1, n_freqs, n_points)

## [n_freqs]
h2_a = 4*(xp.pi**2)*D0*freq_square.reshape(-1)
## [n_freqs x 2]
h2_weights = xp.zeros((n_freqs, 2), dtype=rtype)
h2_weights[:, 0] = (1 - xp.exp(-h2_a*dt)*(h2_a*dt + 1))/((h2_a**2)*dt)
h2_weights[:, 1] = (xp.exp(-h2_a*dt) + h2_a*dt - 1)/((h2_a**2)*dt)
### when h2_a == 0
h2_weights[xp.abs(h2_a) <= 1e-20, :] = dt/2

# compute fhat and K_long for [(2*n_eta+1)*dt, bdelta-sdelta] (us)
T3 = np.array(range(2*n_eta+1, n_time))
# timing the most expansive part
s = time.time()
for it in T3:
    # [n_q x n_freq]
    fhat_it = exp_dt*fhat[:, :, it-1] + \
        xp.squeeze(
            ((func_h1(it*dt, eta, dt, freq_square, q_square, freq_zero_mask, D0) @
              fhat_h1_coef) * exponential_term -
             func_h2(exp_eta, h2_weights, K_long[:, :, it-n_eta-1:it-n_eta+1])*fhat_h2_coef) @ dl.T)
    fhat[:, :, it] = fhat_it
    K_long[:, :, it] = fhat_it @ Klong_fourier_coef.T
if xp is not np:
  xp.cuda.stream.get_current_stream().synchronize()
e = time.time()
print(e-s)
print('CPU running time for (10 q-vectors, 40401 Fourier modes): 715.17s')
```

    41.01460528373718
    CPU running time for (10 q-vectors, 40401 Fourier modes): 715.17s

## Compute $\mu$, $S_{short}$ and $S_{long}$ for $t \in (\eta, \Delta-\delta]$
1. $$\mu(\mathbf{x}_0, t) = 2\left(1-\sqrt{\frac{\mathcal{D}_0 \eta}{\pi}}\kappa(\mathbf{x}_0)\right)^{-1}\left[\mathcal{N}(\mathbf{x}_0, t) - K_{long}[\mu](\mathbf{x}_0, \tau)\right]$$

2. $$S_{short}[\mu](\mathbf{x}_0, t) = \sqrt{\frac{\mathcal{D}_0 \eta}{\pi}} \mu(\mathbf{x}_0, t)$$

3. $$S_{long}[\mu](\mathbf{x}_0, t) = \mathcal{D}_0 \sum_{\nu = -\nu_{max}}^{\nu_{max}} \hat{f}(\nu, t) e^{2\pi \jmath \nu \cdot \mathbf{x}_0} \Delta \nu^2$$

4. $$\omega(\mathbf{x}_0, t) = S[\mu](\mathbf{x}_0, t) = S_{short}[\mu](\mathbf{x}_0, t) + S_{long}[\mu](\mathbf{x}_0, t)$$

```
mu[:, :, n_eta+1:] = 2*(neu_data[:, :, n_eta+1:] - K_long[:, :, n_eta+1:])/Kshort_appro.reshape(1, n_points, 1)
S_short[:, :, n_eta+1:] = xp.sqrt(D0*eta/xp.pi)*mu[:, :, n_eta+1:]
S_long = (fourier_bases.T.conj() @ fhat)*(D0*freq_resolution**2)
omega = S_long + S_short
```

# Compute the dMRI sigal $\mathcal{S}$
The diffusion MRI signal $\mathcal{S}$ can be formulated in terms of $\omega$:
$$
\mathcal{S} = |\Omega|\rho e^{-4\pi^2 \mathcal{D}_0  \|\mathbf{q}\|^2 (\Delta - \delta)} + \overline{\omega}(\mathbf{q}, \Delta-\delta),
$$
with
\begin{equation}
	\begin{split}
		\overline{\omega}(\mathbf{q}, t) =\ &e^{-4\pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 \Delta t}\overline{\omega}(\mathbf{q}, t-\Delta t)\\ &- \mathcal{D}_0  \underbrace{\int_{\partial \Omega} 2\pi \jmath \mathbf{q} \cdot \mathbf{n} e^{2\pi \jmath \mathbf{q} \cdot \mathbf{y}} \overbrace{\int_{t-\Delta t}^{t} e^{-4\pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2 (t-\tau)} \omega(\mathbf{y}, \tau)  d\tau}^{u} ds_{\mathbf{y}}}_{\overline{\omega}_{temp}(\mathbf{q}, t)}.
	\end{split}
\end{equation}
and
\begin{equation}
	u =\begin{cases} \frac{\Delta t}{2} \left[\omega(\mathbf{y}, t-\Delta t)+\omega(\mathbf{y}, t)\right] & \|\mathbf{q}\| = 0      \\
		\frac{1-e^{-4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2\Delta t}(4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2\Delta t+1)}{(4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2)^2\Delta t}\omega(\mathbf{y}, t-\Delta t)+\frac{e^{-4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2\Delta t}+ 4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2\Delta t-1}{(4 \pi^2 \mathcal{D}_0 \|\mathbf{q}\|^2)^2\Delta t}\omega(\mathbf{y}, t) & \|\mathbf{q}\| \neq 0
	\end{cases}
\end{equation}

```
# reuseable intermediate results
u = xp.zeros((n_q, n_points, n_time), dtype=ctype)
omega_bar_a = 4*(xp.pi**2)*D0*q_square.reshape((n_q, 1, 1))
weight1 = (1 - xp.exp(-omega_bar_a*dt) * (omega_bar_a*dt + 1))/(omega_bar_a**2*dt)
weight2 = (xp.exp(-omega_bar_a*dt) + omega_bar_a*dt - 1)/(omega_bar_a**2*dt)
## [n_q x n_points x n_time] matrix
u[:, :, 1:] = weight1*omega[:, :, 0:-1] + weight2*omega[:, :, 1:]
for iq in range(n_q):
    if np.abs(q_square[0, iq]) < 1e-20:
        u[iq, :, 1:] = (omega[iq, :, 0:-1] + omega[iq, :, 1:])*dt/2

# compute omega_bar for [0, bdelta-sdelta] (us)
omega_bar_temp = xp.squeeze(dl @ (u*(2*xp.pi*1j*(q.T @ boundary_normals) *
                                     xp.exp(2*xp.pi*1j*q.T@boundary_points)).reshape(n_q, n_points, 1)))
for it in range(1, n_time):
    omega_bar[:, it] = xp.exp(-4*(xp.pi**2)*D0*q_square.reshape(-1)*dt) * \
        omega_bar[:, it-1] - D0*omega_bar_temp[:, it]
# [n_q x n_time]
omega_bar = omega_bar/region_area

for iq in range(n_q):
  plt.figure()
  if xp is np:
      plt.plot(time_val.flatten(), xp.real(omega_bar[iq,:]).flatten())
      plt.title('omega_bar, q={}'.format(xp.sqrt(q_square[0, iq])))
  else:
      # convert cupy ndarray to numpy array
      plt.plot(xp.asnumpy(time_val).flatten(), xp.asnumpy(xp.real(omega_bar[iq,:])).flatten())
      plt.title('omega_bar, q={}'.format(xp.asnumpy(xp.sqrt(q_square[0, iq]))))

# dMRI signal
normalized_signal = xp.real(xp.exp(-4*xp.pi**2*D0*q_square[0,:]*(bdelta-sdelta)) + omega_bar[:, -1]).reshape(-1)
print(normalized_signal)
```

```

```


# 第二讲 - 矩阵消元及其与矩阵乘法的关系

- 消元法解方程组
- 矩阵简化
- 反向替代
- 矩阵乘法

## 消元法解方程组

$$x+2y+z=2\quad(1)\\3x+8y+z=12\quad(2)\\4y+z=2\quad(3)$$

提取出矩阵：

$$A=\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}$$

下面先做一些初始化工作：

```python
import numpy as np
from sympy import *
init_printing()
x, y, z = symbols('x y z')
lhs = (x + 2*y + z, 3*x + 8*y + z, 4*y + z)
rhs = (2, 12, 2)
A = np.array([[1, 2, 1], [3, 8, 1], [0, 4, 1]])
```

### 方程组消元方式

首先消去未知数x，我们用等式(2)减去3倍的等式(1)，我们得到等式(4)：

```python
eq4 = Eq(lhs[1]-3*lhs[0], rhs[1]-3*rhs[0])
eq4
```

由于等式(3)的x系数为0，因此，等式(3)不需要消除x，也可以认为等式(3)减去0倍的等式(1)，得到等式(5)：

```python
eq5 = Eq(lhs[2]-0*lhs[0], rhs[2]-0*rhs[0])
eq5
```

此时我们的方程组已经经过消去未知数x，变为：

$$x+2y+z=2\quad(1)\\2y-2z=6\quad(4)\\4y+z=2\quad(5)$$

下面我们利用等式(4)消去等式(5)的未知数y，得到等式(6)：

```python
eq6 = Eq(eq5.args[0]-2*eq4.args[0], eq5.args[1]-2*eq4.args[1])
eq6
```

这是我们得到了消元后的方程组：

$$x+2y+z=2\quad(1)\\2y-2z=6\quad(4)\\5z=-10\quad(6)$$

等式(6)很容易接触未知数z：

```python
solz = solve(eq6, dict=True)
print(solz[0])
```

    {z: -2}

然后依次将z带入等式(4)，求得y，将y和z带入等式(1)，求得x，最终解出全方程组：

```python
soly = solve(eq4.subs(z, -2), y, dict=True)
print(soly[0])
```

    {y: 1}

```python
solx = solve(Eq(x+2*y+z, 2).subs({y:1, z:-2}), x, dict=True)
print(solx[0])
```

    {x: 2}

```python
print({**solx[0], **soly[0], **solz[0]})
```

    {x: 2, y: 1, z: -2}

### 矩阵消元方式

矩阵消元的方式会显得更加自然和简单，从左上角到右下角沿对角线（称为矩阵的主对角线）依次消除对角线以下的数字（将对角线以下数字变成0）。我们将保留在对角线或以上用来进行消元的数字称为Pivot number。

首先将矩阵第二行减去三倍的第一行：

```python
A[1] -= 3 * A[0]
A
```

    array([[ 1,  2,  1],
           [ 0,  2, -2],
           [ 0,  4,  1]])

当然我们也可以认为下面还进行了将矩阵的第三行减去0倍的第一行：

```python
A[2] -= 0 * A[0]
A
```

    array([[ 1,  2,  1],
           [ 0,  2, -2],
           [ 0,  4,  1]])

因此，经过第一次消元，矩阵第一列除了pivot number之外，全部变为0，下面我们继续对角线第二个pivot number。

将矩阵第三行减去2倍的第二行：

```python
A[2] -= 2 * A[1]
A
```

    array([[ 1,  2,  1],
           [ 0,  2, -2],
           [ 0,  0,  5]])

此时，矩阵已经变为一个上三角矩阵，主对角线以下的元素全部为0.

对于等式的右边 RHS，进行同样的操作，我们得到如下的矩阵方程式：

$$\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}2\\6\\-10\end{bmatrix}$$

```python
b = np.array([2, 12, 2])
b[1] -= 3 * b[0]
b[2] -= 0 * b[0]
b[2] -= 2 * b[1]
b
```

    array([  2,   6, -10])

下面我们完成反向替代的过程，一下步骤等式左边的矩阵和等式右边的向量会一起进行操作：

```python
# 解出z
A[2] = A[2]/5
b[2] /= 5
print(A, b)
```

    [[ 1  2  1]
     [ 0  2 -2]
     [ 0  0  1]] [ 2  6 -2]

```python
# 解出y
A[1] += 2 * A[2]
b[1] += 2 * b[2]
A[0] -= A[2]
b[0] -= b[2]
A[1] = A[1]/2
b[1] /= 2
print(A, b)
```

    [[1 2 0]
     [0 1 0]
     [0 0 1]] [ 4  1 -2]

```python
# 解出x
A[0] -= 2 * A[1]
b[0] -= 2 * b[1]
print(A, b)
```

    [[1 0 0]
     [0 1 0]
     [0 0 1]] [ 2  1 -2]

上面的结果表示成矩阵方程组（或者更进一步列视角）为：

$$\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}2\\2\\-2\end{bmatrix}\quad\to\quad x\begin{bmatrix}1\\0\\0\end{bmatrix}+y\begin{bmatrix}0\\1\\0\end{bmatrix}+z\begin{bmatrix}0\\0\\1\end{bmatrix}=\begin{bmatrix}2\\1\\-2\end{bmatrix}$$

显而易见，等式右边就是方程组的解。

### 行交换

正如将等式(1)(2)(3)进行位置交换不会影响方程求解结果一样，在矩阵表达中，交换矩阵的行（同时也要交换等式右边RHS的向量位置）也不会影响结果。

我们也可以将上面方程组的矩阵表达方式写成：

$$A=\begin{bmatrix}1&2&1\\0&4&1\\3&8&1\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}=\begin{bmatrix}2\\2\\12\end{bmatrix}$$

## 矩阵乘法

假设有n行n列的矩阵A，$A=\begin{bmatrix}a_{11}&\cdots&a_{1n}\\ \vdots & \ddots & \vdots \\ a_{n1}&\cdots&a_{nn}\end{bmatrix}$，按照行视角，我们可以表示为$A=\begin{bmatrix}r_1 \\ \vdots \\ r_n \end{bmatrix}$，按照列视角，我们可以表示为$A=\begin{bmatrix}c_1 & \cdots & c_n \end{bmatrix}$。

如果矩阵A右边乘以一个n长度的向量$x=\begin{bmatrix}x_1\\ \vdots \\ x_n \end{bmatrix}$，按照列视角非常容易表达，结果得到一个n长度的向量：

$$ \begin{bmatrix}c_1 & \cdots & c_n \end{bmatrix}\begin{bmatrix}x_1\\ \vdots \\ x_n \end{bmatrix}=x_1c_1+\cdots+x_nc_n$$

如果矩阵A左边乘以一个$1\times n$的矩阵（向量的倒置）$x=\begin{bmatrix}x_1& \cdots & x_n \end{bmatrix}$，按照行视角非常容易表达，结果得到一个$1\times n$的矩阵（向量的倒置）：

$$ \begin{bmatrix}x_1& \cdots & x_n \end{bmatrix}\begin{bmatrix}r_1 \\ \vdots \\ r_n \end{bmatrix}=x_1r_1+\cdots+x_nr_n$$

**因此当进行矩阵乘法时，结果中的列是乘号左边的矩阵的列的线性组合，结果中的行是乘号右边的矩阵的行的线性组合。**

当我们进行矩阵消元时，我们完全可以依靠矩阵乘法来进行，例如对于矩阵$A=\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}$来说，第一步是需要消去第一列出pivot number外的所有数字，将其变为0。我们已经知道这需要将A的第二行减去3倍的第一行，将A的第三行减去0倍的第一行，结果中第一行保持不变。我们可以先写出乘号左边的第一行，让其是的乘号右边的行的线性组合等于第一行本身，于是可知：

$$\begin{bmatrix}1&0&0\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\end{bmatrix}$$

使用numpy进行验证：

```python
A = np.array([[1, 2, 1], [3, 8, 1], [0, 4, 1]])
np.array([1, 0, 0])@A
```

    array([1, 2, 1])

同理可知，结果第二行应该是矩阵第二行减去3倍第一行所得（或者可以认为是$-3\times r_1 + r_2 + 0\times r_3$）：

$$\begin{bmatrix}-3&1&0\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}0&2&-2\end{bmatrix}$$

结果的第三行应该是矩阵第三行减去0被第一行所得（或者可以认为是$0\times r_1 + 0\times r_2 + r_3$）：

$$\begin{bmatrix}0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}0&4&1\end{bmatrix}$$

使用numpy进行验证：

```python
np.array([-3, 1, 0])@A
```

    array([ 0,  2, -2])

```python
np.array([0, 0, 1])@A
```

    array([0, 4, 1])

将这三个水平向量组合成一个矩阵，可以在一次乘法中完成第一个pivot number的消元过程：

$$\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&4&1\end{bmatrix}$$

在numpy中进行验证：

```python
fac1 = np.eye(3)
fac1[1, 0] = -3
fac1@A
```

    array([[ 1.,  2.,  1.],
           [ 0.,  2., -2.],
           [ 0.,  4.,  1.]])

类推下去，可以在左边再乘以一个矩阵，完成第二个pivot number的消元过程：

$$\begin{bmatrix}1&0&0\\0&1&0\\0&-2&1\end{bmatrix}\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}$$

验证：

```python
fac2 = np.eye(3)
fac2[2, 1] = -2
fac2@fac1@A
```

    array([[ 1.,  2.,  1.],
           [ 0.,  2., -2.],
           [ 0.,  0.,  5.]])

如果继续在左边添加矩阵相乘，我们还能完成反向替代的过程：

$$\begin{bmatrix}1&-2&0\\0&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&0&0\\0&\frac{1}{2}&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&0&-1\\0&1&2\\0&0&1\end{bmatrix}\begin{bmatrix}1&0&0\\0&1&0\\0&0&\frac{1}{5}\end{bmatrix}\begin{bmatrix}1&0&0\\0&1&0\\0&-2&1\end{bmatrix}\begin{bmatrix}1&0&0\\-3&1&0\\0&0&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}$$

验证（因为涉及数值计算精度问题，所以最终结果不是完全为0或1，使用round函数目的是为了清晰）：

```python
fac3 = np.eye(3)
fac3[2, 2] = 1.0/5
fac4 = np.array([[1, 0 , -1],[0, 1, 2], [0, 0, 1]])
fac5 = np.eye(3)
fac5[1, 1] = 1.0/2
fac6 = np.eye(3)
fac6[0, 1] = -2
(fac6@fac5@fac4@fac3@fac2@fac1@A).round()
```

    array([[ 1.,  0., -0.],
           [ 0.,  1.,  0.],
           [ 0.,  0.,  1.]])

从上面我们可以看到，如果需要一个矩阵将A变为上三角矩阵U的话，我们需要的是fac2乘以fac1：

$$\begin{bmatrix}1&0&0\\-3&1&0\\6&-2&1\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&2&-2\\0&0&5\end{bmatrix}$$

```python
fac2@fac1
```

    array([[ 1.,  0.,  0.],
           [-3.,  1.,  0.],
           [ 6., -2.,  1.]])

### 矩阵逆

如果将fac6依次乘到fac1，得到的结果和A相乘就会变成一个单位矩阵I，根据定义$A^{-1}A=I$，我们可知：

$$fac6 \times fac5 \times fac4 \times fac3 \times fac2 \times fac1 = A^{-1}$$

```python
fac6@fac5@fac4@fac3@fac2@fac1
```

    array([[ 0.4,  0.2, -0.6],
           [-0.3,  0.1,  0.2],
           [ 1.2, -0.4,  0.2]])

### 使用矩阵乘法行交换

当乘法左边是单位矩阵I时，矩阵数据不会发生变化，因此如果我们重新排列单位矩阵的行之后，在与矩阵A相乘，就能对A的行进行交换，例如我们需要交换A的第二行和第三行：

$$\begin{bmatrix}1&0&0\\0&0&1\\0&1&0\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}1&2&1\\0&4&1\\3&8&1\end{bmatrix}$$

验证：

```python
np.array([[1, 0, 0],[0, 0, 1],[0, 1, 0]])@A
```

    array([[1, 2, 1],
           [0, 4, 1],
           [3, 8, 1]])

如果我们需要循环的将行下移一个位置的话：

$$\begin{bmatrix}0&0&1\\1&0&0\\0&1&0\end{bmatrix}\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}=\begin{bmatrix}0&4&1\\1&2&1\\3&8&1\end{bmatrix}$$

验证：

```python
np.array([[[0, 0, 1],[1, 0, 0],[0, 1, 0]]])@A
```

    array([[[0, 4, 1],
            [1, 2, 1],
            [3, 8, 1]]])

通过上面的列视角我们可以知道，如果需要交换列的位置的话，只需要在矩阵A的右边乘以单位矩阵的重新排列即可，如：

$$\begin{bmatrix}1&2&1\\3&8&1\\0&4&1\end{bmatrix}\begin{bmatrix}0&0&1\\1&0&0\\0&1&0\end{bmatrix}=\begin{bmatrix}2&1&1\\8&1&3\\4&1&0\end{bmatrix}$$

验证：

```python
A@np.array([[[0, 0, 1],[1, 0, 0],[0, 1, 0]]])
```

    array([[[2, 1, 1],
            [8, 1, 3],
            [4, 1, 0]]])

```python

```


# Gaussian Process Distribution of Relaxation Times

## In this tutorial we will reproduce Figure 7 of the article https://doi.org/10.1016/j.electacta.2019.135316

GP-DRT is our newly developed approach that can be used to obtain both the mean and covariance of the DRT from EIS data by assuming that the DRT is a Gaussian process (GP). The GP-DRP can predict the DRT and the imaginary part of the impedance at frequencies that were not previously measured.

To obtain the DRT from the impedance we take that $\gamma(\xi)$ is a GP where $f$ is the frequency and $\xi=\log f$. Under the DRT model and considering that GPs are closed linear transformations, it follows that $Z^{\rm DRT}_{\rm im}\left(\xi\right)$ is also a GP.

In other words, we can write

$$\begin{pmatrix}
\gamma(\xi) \\
Z^{\rm DRT}_{\rm im}\left(\xi\right)
\end{pmatrix}\sim \mathcal{GP}\left(\mathbf 0, \begin{pmatrix}
k(\xi, \xi^\prime) & \mathcal L^{\rm im}_{\xi^\prime} \left(k(\xi, \xi^\prime)\right)\\
\mathcal L^{\rm im}_{\xi} k(\xi, \xi^\prime) & \mathcal L^{\rm im}_{\xi^\prime}\left(\mathcal L^{\rm im}_{\xi} \left(k(\xi, \xi^\prime)\right)\right)
\end{pmatrix}\right)$$

where

$$\mathcal L^{\rm im}_\xi \left(\cdot\right) = -\displaystyle \int_{-\infty}^\infty \frac{2\pi \displaystyle e^{\xi-\hat \xi}}{1+\left(2\pi \displaystyle e^{\xi-\hat \xi}\right)^2}  \left(\cdot\right) d \hat \xi$$

is a linear functional. The latter functional, maps the DRT to the imaginary part of the impedance.

Assuming $N$ observations, we can set $\left(\mathbf Z^{\rm exp}_{\rm im}\right)_n = Z^{\rm exp}_{\rm im}(\xi_n)$ with $\xi_n =\log f_n$ and $n =1, 2, \ldots N $. The corresponding multivariate Gaussian random variable can be written as

$$\begin{pmatrix}
\boldsymbol{\gamma} \\
\mathbf Z^{\rm exp}_{\rm im}
\end{pmatrix}\sim \mathcal{N}\left(\mathbf 0, \begin{pmatrix}
\mathbf K & \mathcal L_{\rm im} \mathbf K\\
\mathcal L_{\rm im}^\sharp \mathbf K & \mathcal L^2_{\rm im} \mathbf K + \sigma_n^2 \mathbf I
\end{pmatrix}\right)$$

where

$$\begin{align}
(\mathbf K)_{nm} &= k(\xi_n, \xi_m)\\
(\mathcal L_{\rm im} \mathbf K)_{nm} &= \left. \mathcal L^{\rm im}_{\xi^\prime} \left(k(\xi, \xi^\prime)\right) \right |_{\xi_n, \xi_m}\\
(\mathcal L_{\rm im}^\sharp \mathbf K)_{nm} &= \left.\mathcal L^{\rm im}_{\xi} \left(k(\xi, \xi^\prime)\right) \right|_{\xi_n, \xi_m}\\
(\mathcal L^2_{\rm im} \mathbf K)_{nm} &= \left.\mathcal L^{\rm im}_{\xi^\prime}\left(\mathcal L^{\rm im}_{\xi} \left(k(\xi, \xi^\prime)\right)\right) \right|_{\xi_n, \xi_m}
\end{align}$$

and $\mathcal L_{\rm im} \mathbf K^\top = \mathcal L_{\rm im}^\sharp \mathbf K$.

To obtain the DRT from impedance, the distribution of $\mathbf{\gamma}$ conditioned on $\mathbf Z^{\rm exp}_{\rm im}$ can be written as

$$\boldsymbol{\gamma}|\mathbf Z^{\rm exp}_{\rm im}\sim \mathcal N\left( \mathbf \mu_{\gamma|Z^{\rm exp}_{\rm im}}, \mathbf\Sigma_{\gamma| Z^{\rm exp}_{\rm im}}\right)$$

with

$$\begin{align}
\mathbf \mu_{\gamma|Z^{\rm exp}_{\rm im}} &= \mathcal L_{\rm im} \mathbf K \left(\mathcal L^2_{\rm im} \mathbf K + \sigma_n^2 \mathbf I \right)^{-1} \mathbf Z^{\rm exp}_{\rm im} \\
\mathbf \Sigma_{\gamma| Z^{\rm exp}_{\rm im}} &= \mathbf K- \mathcal L_{\rm im} \mathbf K \left(\mathcal L^2_{\rm im} \mathbf K + \sigma_n^2 \mathbf I \right)^{-1}\mathcal L_{\rm im} \mathbf K^\top
\end{align}$$

The above formulas depend on 1) the kernel, $k(\xi, \xi^\prime)$; 2) the noise level, $\sigma_n$; and 3) the experimental data, $\mathbf Z^{\rm exp}_{\rm im}$ (at the log-frequencies $\mathbf \xi$).

```python
import numpy as np
import matplotlib.pyplot as plt
from math import sin, cos, pi
import GP_DRT
from scipy.optimize import minimize
%matplotlib inline
```

## 1) Define parameters of the ZARC circuit which will be used for the synthetic experiment generation

The impedance of a ZARC can be written as
$$
Z^{\rm exact}(f) = R_\infty + \displaystyle \frac{1}{\displaystyle \frac{1}{R_{\rm ct}}+C \left(i 2\pi f\right)^\phi}
$$

where $\displaystyle C = \frac{\tau_0^\phi}{R_{\rm ct}}$.

The corresponding DRT is given by

$$
\gamma(\log \tau) =  \displaystyle \frac{\displaystyle R_{\rm ct}}{\displaystyle 2\pi} \displaystyle \frac{\displaystyle \sin\left((1-\phi)\pi\right)}{\displaystyle \cosh(\phi \log(\tau/\tau_0))-\cos(\pi(1-\phi))}
$$

```python
# define the frequency range
N_freqs = 81
freq_vec = np.logspace(-4., 4., num=N_freqs, endpoint=True)
xi_vec = np.log(freq_vec)
tau  = 1/freq_vec

# define the frequency range used for prediction
# note: we could have used other values
freq_vec_star = np.logspace(-4., 4., num=81, endpoint=True)
xi_vec_star = np.log(freq_vec_star)

# parameters for ZARC model, the impedance and analytical DRT are calculated as the above equations
R_inf = 10
R_ct = 50
phi = 0.8
tau_0 = 1.

C = tau_0**phi/R_ct
Z_exact = R_inf+1./(1./R_ct+C*(1j*2.*pi*freq_vec)**phi)
gamma_fct = (R_ct)/(2.*pi)*sin((1.-phi)*pi)/(np.cosh(phi*np.log(tau/tau_0))-cos((1.-phi)*pi))

# we will use a finer mesh for plotting the results
freq_vec_plot  = np.logspace(-4., 4., num=10*(N_freqs-1), endpoint=True)
tau_plot  = 1/freq_vec_plot
# for plotting only
gamma_fct_plot = (R_ct)/(2.*pi)*sin((1.-phi)*pi)/(np.cosh(phi*np.log(tau_plot/tau_0))-cos((1.-phi)*pi))

# we will add noise to the impedance computed analytically
rng = np.random.seed(214975)
sigma_n_exp = 1.
Z_exp = Z_exact + sigma_n_exp*(np.random.normal(0, 1, N_freqs)+1j*np.random.normal(0, 1, N_freqs))
```

## 2) Show the synthetic impedance in the Nyquist plot - this is similar to Figure 7 (a)

```python
plt.rc('text', usetex=True)
plt.rc('font', family='serif', size=15)
plt.rc('xtick', labelsize=15)
plt.rc('ytick', labelsize=15)

# Nyquist plot of the impedance
plt.plot(np.real(Z_exact), -np.imag(Z_exact), linewidth=4, color="black", label="exact")
plt.plot(np.real(Z_exp), -np.imag(Z_exp), "o", markersize=10, color="red", label="synth exp")
plt.plot(np.real(Z_exp[20:60:10]), -np.imag(Z_exp[20:60:10]), 's', markersize=10, color="black")
plt.legend(frameon=False, fontsize = 15)
plt.axis('scaled')

plt.xticks(range(10, 70, 10))
plt.yticks(range(0, 60, 10))
plt.gca().set_aspect('equal', adjustable='box')
plt.xlabel(r'$Z_{\rm re}/\Omega$', fontsize = 20)
plt.ylabel(r'$-Z_{\rm im}/\Omega$', fontsize = 20)
# label the frequency points
plt.annotate(r'$10^{-2}$', xy=(np.real(Z_exp[20]), -np.imag(Z_exp[20])),
             xytext=(np.real(Z_exp[20])-2, 10-np.imag(Z_exp[20])),
             arrowprops=dict(arrowstyle="-",connectionstyle="arc"))
plt.annotate(r'$10^{-1}$', xy=(np.real(Z_exp[30]), -np.imag(Z_exp[30])),
             xytext=(np.real(Z_exp[30])-2, 6-np.imag(Z_exp[30])),
             arrowprops=dict(arrowstyle="-",connectionstyle="arc"))
plt.annotate(r'$1$', xy=(np.real(Z_exp[40]), -np.imag(Z_exp[40])),
             xytext=(np.real(Z_exp[40]), 10-np.imag(Z_exp[40])),
             arrowprops=dict(arrowstyle="-",connectionstyle="arc"))
plt.annotate(r'$10$', xy=(np.real(Z_exp[50]), -np.imag(Z_exp[50])),
             xytext=(np.real(Z_exp[50])-1, 10-np.imag(Z_exp[50])),
             arrowprops=dict(arrowstyle="-",connectionstyle="arc"))
plt.show()
```

## 3) Obtain the optimal hyperparameters of the GP-DRT model by minimizing the negative marginal log-likelihood (NMLL)

We constrain the kernel to be a squared exponential, _i.e._,

$$
k(\xi, \xi^\prime) = \sigma_f^2 \exp\left(-\frac{1}{2 \ell^2}\left(\xi-\xi^\prime\right)^2 \right)
$$

and optimize its two parameters, $\sigma_f$ and $\ell$ as well as the noise level $\sigma_n$. Therefore, the vector of GP-DRT hyperparameters is $\boldsymbol \theta = \begin{pmatrix} \sigma_n, \sigma_f, \ell  \end{pmatrix}^\top$.

Following the article, we can write that

$$
\log p(\mathbf Z^{\rm exp}_{\rm im}|\boldsymbol \xi, \boldsymbol \theta)= - \frac{1}{2} {\mathbf Z^{\rm exp}_{\rm im}}^\top \left(\mathcal L^2_{\rm im} \mathbf K +\sigma_n^2\mathbf I \right)^{-1} \mathbf Z^{\rm exp}_{\rm im} -\frac{1}{2} \log \left| \mathcal L^2_{\rm im} \mathbf K+\sigma_n^2\mathbf I  \right| - \frac{N}{2} \log 2\pi
$$

We will call $L(\boldsymbol \theta)$ the negative (and shifted) MLL (NMLL):
$$
L(\boldsymbol \theta) = - \log p(\mathbf Z^{\rm exp}_{\rm im}|\boldsymbol \xi, \boldsymbol \theta) - \frac{N}{2} \log 2\pi
$$

the experimental evidence is maximized for

$$
\boldsymbol \theta = \arg \min_{\boldsymbol \theta^\prime}L(\boldsymbol \theta^\prime)
$$

The above minimization problem is solved using the `optimize` function given in `scipy`

```python
# initialize the parameters for the minimization of the NMLL, see (31) in the manuscript
sigma_n = sigma_n_exp
sigma_f = 5.
ell = 1.

theta_0 = np.array([sigma_n, sigma_f, ell])
seq_theta = np.copy(theta_0)
def print_results(theta):
    global seq_theta
    seq_theta = np.vstack((seq_theta, theta))
    print('{0:.7f}  {1:.7f}  {2:.7f}'.format(theta[0], theta[1], theta[2]))

print('sigma_n,   sigma_f,   ell')

# minimize the NMLL L(\theta) w.r.t sigma_n, sigma_f, ell using the Newton-CG method as implemented in scipy
res = minimize(GP_DRT.NMLL_fct, theta_0, args=(Z_exp, xi_vec), method='Newton-CG', \
               jac=GP_DRT.grad_NMLL_fct,  callback=print_results, options={'disp': True})

# collect the optimized parameters
sigma_n, sigma_f, ell = res.x
```

    sigma_n,   sigma_f,   ell
    0.8903599  5.0014151  1.0120875
    0.8136354  5.0035337  1.0291912
    0.8291863  5.0357867  1.2588673
    0.8303934  5.0832372  1.2117784
    0.8304464  5.2060761  1.2283664
    0.8305219  5.3874435  1.2524151
    0.8305286  5.4068909  1.2546651
    0.8305276  5.4070863  1.2546870
    0.8305265  5.4070865  1.2546866
    Optimization terminated successfully.
             Current function value: 53.657989
             Iterations: 9
             Function evaluations: 11
             Gradient evaluations: 54
             Hessian evaluations: 0

## 4) Core of the GP-DRT

### 4a) Compute matrices
Once we have identified the optimized parameters we can compute $\mathbf K$, $\mathcal L_{\rm im} \mathbf K$, and $\mathcal L^2_{\rm im} \mathbf K$, which are given in equation (18) in the article

```python
K = GP_DRT.matrix_K(xi_vec, xi_vec, sigma_f, ell)
L_im_K = GP_DRT.matrix_L_im_K(xi_vec, xi_vec, sigma_f, ell)
L2_im_K = GP_DRT.matrix_L2_im_K(xi_vec, xi_vec, sigma_f, ell)
Sigma = (sigma_n**2)*np.eye(N_freqs)
```

### 4b) Factorize the matrices and solve the linear equations
We are computing
$$
\boldsymbol{\gamma}|\mathbf Z^{\rm exp}_{\rm im}\sim \mathcal N\left( \boldsymbol \mu_{\gamma|Z^{\rm exp}_{\rm im}}, \boldsymbol \Sigma_{\gamma| Z^{\rm exp}_{\rm im}}\right)
$$

using
$$
\begin{align}
\boldsymbol \mu_{\gamma|Z^{\rm exp}_{\rm im}} &= \mathcal L_{\rm im} \mathbf K\left(\mathcal L^2_{\rm im} \mathbf K+\sigma_n^2\mathbf I\right)^{-1}\mathbf Z^{\rm exp}_{\rm im} \\
\boldsymbol \Sigma_{\gamma| Z^{\rm exp}_{\rm im}} &= \mathbf K-\mathcal L_{\rm im} \mathbf K\left(\mathcal L^2_{\rm im} \mathbf K+\sigma_n^2\mathbf I\right)^{-1}\mathcal L_{\rm im} \mathbf K^\top
\end{align}
$$

The key step is to do Cholesky factorization of $\mathcal L^2_{\rm im} \mathbf K+\sigma_n^2\mathbf I$, _i.e._, K_im_full

```python
# the matrix $\mathcal L^2_{\rm im} \mathbf K + \sigma_n^2 \mathbf I$ whose inverse is needed
K_im_full = L2_im_K + Sigma

# check if the K_im_full is positive definite, otherwise, a nearest one would replace the K_im_full
if not GP_DRT.is_PD(K_im_full):
    K_im_full = GP_DRT.nearest_PD(K_im_full)

# Cholesky factorization, L is a lower-triangular matrix
L = np.linalg.cholesky(K_im_full)

# solve for alpha
alpha = np.linalg.solve(L, Z_exp.imag)
alpha = np.linalg.solve(L.T, alpha)

# estimate the gamma of eq (21a)
gamma_fct_est = np.dot(L_im_K, alpha)

# covariance matrix
inv_L = np.linalg.inv(L)
inv_K_im_full = np.dot(inv_L.T, inv_L)

# estimate the sigma of gamma for eq (21b)
cov_gamma_fct_est = K - np.dot(L_im_K, np.dot(inv_K_im_full, L_im_K.T))
sigma_gamma_fct_est = np.sqrt(np.diag(cov_gamma_fct_est))
```

### 4c) Plot the obtained DRT against the analytical DRT

```python
# plot the DRT and its confidence region
plt.semilogx(freq_vec_plot, gamma_fct_plot, linewidth=4, color="black", label="exact")
plt.semilogx(freq_vec, gamma_fct_est, linewidth=4, color="red", label="GP-DRT")
plt.fill_between(freq_vec, gamma_fct_est-3*sigma_gamma_fct_est, gamma_fct_est+3*sigma_gamma_fct_est, color="0.4", alpha=0.3)
plt.rc('text', usetex=True)
plt.rc('font', family='serif', size=15)
plt.rc('xtick', labelsize=15)
plt.rc('ytick', labelsize=15)
plt.axis([1E-4,1E4,-5,25])
plt.legend(frameon=False, fontsize = 15)
plt.xlabel(r'$f/{\rm Hz}$', fontsize = 20)
plt.ylabel(r'$\gamma/\Omega$', fontsize = 20)
plt.show()
```

### 4d) Predict the $\gamma$ and the imaginary part of the GP-DRT impedance

#### This part is explained in Section 2.3.3 of the main article

```python
# initialize the imaginary part of impedance vector
Z_im_vec_star = np.empty_like(xi_vec_star)
Sigma_Z_im_vec_star = np.empty_like(xi_vec_star)

gamma_vec_star = np.empty_like(xi_vec_star)
Sigma_gamma_vec_star = np.empty_like(xi_vec_star)

# calculate the imaginary part of impedance at each $\xi$ point for the plot
for index, val in enumerate(xi_vec_star):
    xi_star = np.array([val])

    # compute matrices shown in eq (23), xi_star corresponds to a new point
    k_star = GP_DRT.matrix_K(xi_vec, xi_star, sigma_f, ell)
    L_im_k_star_up = GP_DRT.matrix_L_im_K(xi_star, xi_vec, sigma_f, ell)
    L2_im_k_star = GP_DRT.matrix_L2_im_K(xi_vec, xi_star, sigma_f, ell)
    k_star_star = GP_DRT.matrix_K(xi_star, xi_star, sigma_f, ell)
    L_im_k_star_star = GP_DRT.matrix_L_im_K(xi_star, xi_star, sigma_f, ell)
    L2_im_k_star_star = GP_DRT.matrix_L2_im_K(xi_star, xi_star, sigma_f, ell)

    # compute Z_im_star mean and standard deviation using eq (26)
    Z_im_vec_star[index] = np.dot(L2_im_k_star.T, np.dot(inv_K_im_full, Z_exp.imag))
    Sigma_Z_im_vec_star[index] = L2_im_k_star_star-np.dot(L2_im_k_star.T, np.dot(inv_K_im_full, L2_im_k_star))

    # compute gamma_star mean and standard deviation using eq (29)
    gamma_vec_star[index] = np.dot(L_im_k_star_up, np.dot(inv_K_im_full, Z_exp.imag))
    Sigma_gamma_vec_star[index] = k_star_star-np.dot(L_im_k_star_up, np.dot(inv_K_im_full, L_im_k_star_up.T))
```

### 4e) Plot the imaginary part of the GP-DRT impedance together with the exact one and the synthetic experiment

```python
plt.semilogx(freq_vec_star, -np.imag(Z_exact), ":", linewidth=4, color="blue", label="exact")
plt.semilogx(freq_vec, -Z_exp.imag, "o", markersize=10, color="black", label="synth exp")
plt.semilogx(freq_vec_star, -Z_im_vec_star, linewidth=4, color="red", label="GP-DRT")
plt.fill_between(freq_vec_star, -Z_im_vec_star-3*np.sqrt(abs(Sigma_Z_im_vec_star)), -Z_im_vec_star+3*np.sqrt(abs(Sigma_Z_im_vec_star)), alpha=0.3)
plt.rc('text', usetex=True)
plt.rc('font', family='serif', size=15)
plt.rc('xtick', labelsize=15)
plt.rc('ytick', labelsize=15)
plt.axis([1E-4,1E4,-5,25])
plt.legend(frameon=False, fontsize = 15)
plt.xlabel(r'$f/{\rm Hz}$', fontsize = 20)
plt.ylabel(r'$-Z_{\rm im}/\Omega$', fontsize = 20)
plt.show()
```


```python
%matplotlib inline

import numpy as np
import pylab as pl
import math
from sympy import *
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from mpl_toolkits.mplot3d import Axes3D
```

```python
import numpy as np
import pylab as pl
import math
from sympy import *

class RNN:
    def __init__(self):
        pass

    def set_input_layer(self, x):
        self.input_layer = np.zeros(x)
        return self.input_layer

    def set_output_layer(self, x):
        self.output_layer = np.zeros(x)
        self.before_output_layer = np.zeros(x)
        self.supervised_data = np.zeros(x)
        return self.output_layer, self.before_output_layer, self.supervised_data

    def set_hidden_layer(self, x):
        self.hidden_layer = np.zeros(x)
        self.before_hidden_layer = np.zeros(x)
        return self.hidden_layer, self.before_hidden_layer

    def setup(self):
        w_k = np.zeros(len(self.output_layer))
        self.w_kj = np.array([w_k for i in range(len(self.hidden_layer))])
        w_j = np.zeros(len(self.hidden_layer))
        self.w_ji = np.array([w_j for i in range(len(self.input_layer))])
        return self.w_kj, self.w_ji

    def initialize(self, hidden=None):
        for i in range(len(self.hidden_layer)):
            for j in range(len(self.output_layer)):
                self.w_kj[i][j] = np.random.uniform(-1.0/math.sqrt(1.0/len(self.hidden_layer)), 1.0/math.sqrt(1.0/len(self.hidden_layer)))

        for i in range(len(self.input_layer)):
            for j in range(len(self.hidden_layer)):
                self.w_ji[i][j] = np.random.uniform(-1.0/math.sqrt(1.0/len(self.input_layer)), 1.0/math.sqrt(1.0/len(self.input_layer)))

        if hidden is None:
            u = Symbol('u')
            self.hfunction = 1/(1+exp(-u))
            self.diff_hf = diff(self.hfunction)
        else:
            self.hfunction = hidden
            self.diff_hf = diff(self.hfunction)

    def supervised_function(self, sdata):
        for i in range(len(self.supervised_data)):
            self.supervised_data[i] = sdata[i]

    def set_hidden_error(self, j):
        u = Symbol("u")
        diff_hf = self.diff_hf
        hidden_error = 0
        for k in range(len(self.output_layer)):
            delta_z = diff_hf.subs([(u, self.before_output_layer[k])])
            hidden_error += self.w_kj[j][k]*(self.supervised_data[k] - self.output_layer[k])*delta_z
        return hidden_error

    def calculation(self):
        u = Symbol("u")
        hfunction = self.hfunction
        diff_hf = self.diff_hf

        for i in range(len(self.input_layer)):
            self.before_hidden_layer = np.matrix(self.w_ji).T*np.matrix(self.input_layer).T

        for i in range(len(self.hidden_layer)):
            self.hidden_layer[i] = hfunction.subs([(u, self.before_hidden_layer[i])])

        for i in range(len(self.before_output_layer)):
            self.before_output_layer = np.matrix(self.w_kj).T*np.matrix(self.hidden_layer).T

        for i in range(len(self.output_layer)):
            self.output_layer[i] = hfunction.subs([(u, self.before_output_layer[i])])

    def output_ad(self):
        u = Symbol("u")
        hfunction = self.hfunction
        diff_hf = self.diff_hf

        eta = self.eta
        for j in range(len(self.hidden_layer)):
            for k in range(len(self.output_layer)):
                delta_J = self.supervised_data[k] - self.output_layer[k]
                delta_z = self.output_layer[k]*(1-self.output_layer[k])
                delta_v = self.hidden_layer[j]
                self.w_kj[j][k] += eta*delta_J*delta_z*delta_v

    def input_ad(self):
        u = Symbol("u")
        hfunction = self.hfunction
        diff_hf = self.diff_hf

        eta = self.eta
        for i in range(len(self.input_layer)):
            for j in range(len(self.hidden_layer)):
                hidden_error = self.set_hidden_error(j)
                delta_y = self.hidden_layer[j]*(1-self.hidden_layer[j])
                delta_u = self.input_layer[i]
                self.w_ji[i][j] += eta*hidden_error*delta_y*delta_u

    def simulate(self, idata, sdata, eta):
        self.eta = eta
        self.thidden = np.array([])
        self.toutput = np.array([])
        for i in range(len(idata)):
            self.supervised_function(sdata[i])
            for j in range(len(idata[i])):
                self.input_layer[j] = idata[-i-1][j]
            for j in range(len(self.hidden_layer)):
                self.input_layer[len(idata[-i-1])+j-1] = self.hidden_layer[j]
            self.calculation()
            self.output_ad()
            self.calculation()
            self.input_ad()
        return self.output_layer

    def main(self, idata, sdata, eta, N, i=2, h=2, o=1):
        self.set_input_layer(i+h)
        self.set_hidden_layer(h)
        self.set_output_layer(o)
        self.setup()
        self.initialize()
        return self.output_layer[0]

    def set_network(self, i=2, h=5, o=1):
        self.set_input_layer(i+h)
        self.set_hidden_layer(h)
        self.set_output_layer(o)
        self.setup()
        self.initialize()

    def main2(self, idata, sdata, eta):
        self.simulate(idata, sdata, eta)
        return self.output_layer[0]

    def realize(self, idata):
        for i in range(len(idata)):
            for j in range(len(idata[-i])):
                self.input_layer[j] = idata[i][j]
            for j in range(len(self.hidden_layer)):
                self.input_layer[len(idata[i])+j-1] = self.hidden_layer[j]
            self.calculation()
        return self.output_layer[0]
```

ゲーム理論で用いられるTit for Tatを再現してみる。二人のプレーヤーが互いにRNNで相手の行動を予測し、相手の行動に対してTit for Tatに基づいた行動を選択する。

最初の行動はRNNで指定できないので、所与となる。この初期値と裏切りに対する感応度で収束の仕方が決まる。

協調を1、裏切りを0としている。RNNの予測値は整数値でないが、p=（RNNの出力値）で次回に協調を行う。

例１：1期目に、プレーヤー１が協力、プレーヤー２が裏切り。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 0]]
idata2 = [[0, 1]]
sdata1 = [[0]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], sdata2[-1][0]])
    idata2.append([idata1[-1][1], idata1[-1][0]])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)
    sdata1.append([np.random.choice([1, 0], p=[n1r, 1-n1r])])

    sdata2.append([np.random.choice([1, 0], p=[n2r, 1-n2r])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

    0.020952236405 0.0540398955796 [[1, 0], [0, 1], [1, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]]

```python
下の図より、最初は交互に相手にしっぺ返しをしているが、やがて両者が裏切り合うこと状態に収束する。
```

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1)
plt.plot(p2)
```

例２：1期目に、プレーヤー１が協力、プレーヤー２が協力。ただし、プレーヤー２は相手の裏切りをかなり警戒している。

警戒を表すためにp=（RNNの出力値 - 0.2）とする。p<0の場合はp=0に直す。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 1]]
idata2 = [[1, 1]]
sdata1 = [[1]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], sdata2[-1][0]])
    idata2.append([idata1[-1][1], idata1[-1][0]])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)

    prob1 = n1r
    prob2 = n2r - 0.3

    if prob2 < 0:
        prob2 = 0

    sdata1.append([np.random.choice([1, 0], p=[prob1, 1-prob1])])

    sdata2.append([np.random.choice([1, 0], p=[prob2, 1-prob2])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

    0.615854475273 0.882223771712 [[1, 1], [1, 1], [1, 1], [1, 1], [1, 0], [1, 0], [1, 0], [1, 0], [0, 1], [0, 1], [1, 1], [0, 1], [1, 1], [1, 0], [1, 1], [1, 1], [1, 0], [1, 1], [0, 0], [1, 1], [1, 1]]

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1)
plt.plot(p2)
```

例３：次に相手の行動を完全には観測できない場合を考える。t期の相手の行動をt+1期にノイズが加わって知る。例えば、1期目に相手が協調したことを、確率90%で2期目に正しく知れるが、10%で裏切りと誤って伝わる場合である。

ノイズは20%の確率で加わるものとする。その他の条件は例1と同じにした。

```python
nn1 = RNN()
nn1.set_network()
nn2 = RNN()
nn2.set_network()

idata1 = [[1, 0]]
idata2 = [[0, 1]]
sdata1 = [[0]]
sdata2 = [[1]]

for t in range(20):

    for i in range(10):
        nn1.main2(idata1, sdata2, 0.9)
        nn2.main2(idata2, sdata1, 0.9)

    idata1.append([sdata1[-1][0], np.random.choice([sdata2[-1][0], 1-sdata2[-1][0]], p=[0.8, 0.2])])
    idata2.append([sdata2[-1][0], np.random.choice([sdata1[-1][0], 1-sdata1[-1][0]], p=[0.8, 0.2])])

    n1r = nn1.realize(idata1)
    n2r = nn2.realize(idata1)

    prob1 = n1r
    prob2 = n2r

    sdata1.append([np.random.choice([1, 0], p=[prob1, 1-prob1])])

    sdata2.append([np.random.choice([1, 0], p=[prob2, 1-prob2])])

idata.append([sdata1[-1][0], sdata2[-1][0]])
print nn1.realize(idata1), nn2.realize(idata), idata1
```

    0.18325162225 0.229439706952 [[1, 0], [0, 1], [1, 0], [1, 1], [0, 0], [0, 0], [0, 0], [0, 0], [1, 0], [0, 0], [1, 1], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [1, 0], [1, 0], [0, 1], [0, 0], [0, 0]]

```python
p1 = []
p2 = []
for i in range(len(idata1)):
    p1.append(idata1[i][0])
for i in range(len(idata2)):
    p2.append(idata2[i][0])
plt.plot(p1)
plt.plot(p2)
```

```python
print sdata1, sdata2
```

    [[1], [1], [1], [1]] [[1], [1], [1], [1]]

```python

```

```python
j = np.array([np.array([0])])
j = np.append(j, np.array([1]))
```

```python
j
```

    array([0, 1])

```python

```


```python
import numpy as np
from numpy.linalg import *
rg = matrix_rank
from IPython.display import display, Math, Latex, Markdown

from sympy import *
from sympy import Matrix, solve_linear_system
```

```python
pr = lambda s: display(Markdown('$'+str(latex(s))+'$'))
def psym_matrix(a, intro='',ending='',row=False):
    try:
        if row:
                return(intro+str(latex(a))+ending)
        else:
                display(Latex("$$"+intro+str(latex(a))+ending+"$$"))
    except TypeError:
        display(latex(a)) #TODO MAY BY...
pr = lambda s: display(Markdown('$'+str(latex(s))+'$'))

def pmatrix(a, intro='',ending='',row=False):
    if len(a.shape) > 2:
        raise ValueError('pmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{pmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{pmatrix}']
    if row:
        return(intro+'\n'.join(rv)+ending)
    else:
        display(Latex('$$'+intro+'\n'.join(rv)+ending+'$$'))
def crearMatrix(name,shape=(2,2)):
    X = []
    for i in range(shape[0]):
        row = []
        for j in range(shape[1]):
            row.append(Symbol(name+'_{'+str(i*10+j+11)+'}'))
        X.append(row)
    return Matrix(X)
```

```python
from re import sub
def matrix_to_word(A):
    replaced = sub(r"begin{matrix}", r"(■( ", psym_matrix(A,row=1))
    replaced = sub(r"{", r"", replaced)
    replaced = sub(r"}", r"", replaced)
    replaced = sub(r"\\\\\\\\", r" @ ", replaced)
    replaced = sub(r"\\\\", r" @ ", replaced)
    replaced = sub(r"[\[\]]", r"", replaced)
    replaced = sub(r"left", r"", replaced)
    replaced = sub(r".endmatrix.right", r" ))", replaced)
    replaced = sub(r"\\", r"", replaced)
    print(replaced )
```

```python
X = crearMatrix("x")
B = crearMatrix("b")

D= crearMatrix("d")
C= crearMatrix("c")

psym_matrix(X, intro="X=")
matrix_to_word(X)
#psym_matrix(B, intro="B=")
#psym_matrix(B, intro="D=")
#psym_matrix(C, intro="C=")
```

$$X=\left[\begin{matrix}x_{11} & x_{12}\\x_{21} & x_{22}\end{matrix}\right]$$

    (■( x_11 & x_12 @ x_21 & x_22 ))

### $$ CXD=B $$

```python
psym_matrix(D,
            intro=psym_matrix(C,row=1)+psym_matrix(X,row=1),
            ending="="+ psym_matrix(B,row=1))
```

$$\left[\begin{matrix}c_{11} & c_{12}\\c_{21} & c_{22}\end{matrix}\right]\left[\begin{matrix}x_{11} & x_{12}\\x_{21} & x_{22}\end{matrix}\right]\left[\begin{matrix}d_{11} & d_{12}\\d_{21} & d_{22}\end{matrix}\right]=\left[\begin{matrix}b_{11} & b_{12}\\b_{21} & b_{22}\end{matrix}\right]$$

```python
psym_matrix(C*X*D, ending="="+ psym_matrix(B,row=1))
```

$$\left[\begin{matrix}d_{11} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{21} \left(c_{11} x_{12} + c_{12} x_{22}\right) & d_{12} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{22} \left(c_{11} x_{12} + c_{12} x_{22}\right)\\d_{11} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{21} \left(c_{21} x_{12} + c_{22} x_{22}\right) & d_{12} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{22} \left(c_{21} x_{12} + c_{22} x_{22}\right)\end{matrix}\right]=\left[\begin{matrix}b_{11} & b_{12}\\b_{21} & b_{22}\end{matrix}\right]$$

```python
psym_matrix((C*X*D).reshape(4, 1),
            ending="="+ psym_matrix(B.reshape(4, 1),row=1))
```

$$\left[\begin{matrix}d_{11} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{21} \left(c_{11} x_{12} + c_{12} x_{22}\right)\\d_{12} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{22} \left(c_{11} x_{12} + c_{12} x_{22}\right)\\d_{11} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{21} \left(c_{21} x_{12} + c_{22} x_{22}\right)\\d_{12} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{22} \left(c_{21} x_{12} + c_{22} x_{22}\right)\end{matrix}\right]=\left[\begin{matrix}b_{11}\\b_{12}\\b_{21}\\b_{22}\end{matrix}\right]$$

```python
R = (Matrix([ # How???
    [ C[0,0], 0     , C[1,0], 0     ,],
    [ 0     , C[0,0], 0     , C[1,0],],
    [ C[0,1], 0     , C[1,1], 0     ,],
    [ 0     , C[0,1], 0     , C[1,1],]
])).T
T = Matrix([ # How???
    [D[0,0],D[1,0],   0  , 0    ],
    [D[0,1],D[1,1],   0  , 0    ],
    [ 0    , 0    ,D[0,0],D[1,0]],
    [ 0    , 0    ,D[0,1],D[1,1]]
])
#psym_matrix(T,row=1)
psym_matrix(T*R)

matrix_to_word(T*R)
```

$$\left[\begin{matrix}c_{11} d_{11} & c_{11} d_{21} & c_{12} d_{11} & c_{12} d_{21}\\c_{11} d_{12} & c_{11} d_{22} & c_{12} d_{12} & c_{12} d_{22}\\c_{21} d_{11} & c_{21} d_{21} & c_{22} d_{11} & c_{22} d_{21}\\c_{21} d_{12} & c_{21} d_{22} & c_{22} d_{12} & c_{22} d_{22}\end{matrix}\right]$$

    (■( c_11 d_11 & c_11 d_21 & c_12 d_11 & c_12 d_21 @ c_11 d_12 & c_11 d_22 & c_12 d_12 & c_12 d_22 @ c_21 d_11 & c_21 d_21 & c_22 d_11 & c_22 d_21 @ c_21 d_12 & c_21 d_22 & c_22 d_12 & c_22 d_22 ))

```python
psym_matrix(X.reshape(1, 4))
psym_matrix(X.reshape(1, 4)*R*T)
```

$$\left[\begin{matrix}x_{11} & x_{12} & x_{21} & x_{22}\end{matrix}\right]$$

$$\left[\begin{matrix}d_{11} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{21} \left(c_{11} x_{12} + c_{12} x_{22}\right) & d_{12} \left(c_{11} x_{11} + c_{12} x_{21}\right) + d_{22} \left(c_{11} x_{12} + c_{12} x_{22}\right) & d_{11} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{21} \left(c_{21} x_{12} + c_{22} x_{22}\right) & d_{12} \left(c_{21} x_{11} + c_{22} x_{21}\right) + d_{22} \left(c_{21} x_{12} + c_{22} x_{22}\right)\end{matrix}\right]$$

```python
psym_matrix(F, intro=r'rg(F)=', ending="\Leftrightarrow\exists F^{-1}") #ранг
```

$$\sum_a^bAD$$

```python
R = Matrix([ # How???
    [3,1,0,0],
    [1,0,0,0],
    [0,0,3,1],
    [0,0,1,0]
])
T = Matrix([ # How???
    [1,0,0,0],
    [0,1,0,0],
    [2,0,1,0],
    [0,2,0,1]
])
#psym_matrix(T,row=1)
psym_matrix(T*R)
```

$$\left[\begin{matrix}3 & 1 & 0 & 0\\1 & 0 & 0 & 0\\6 & 2 & 3 & 1\\2 & 0 & 1 & 0\end{matrix}\right]$$

```python

```


```python
import numpy as np
import keras
import pandas
from keras_tqdm import TQDMNotebookCallback
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler

data = np.array(pandas.read_csv("./comp_new_trainingdata.csv", header=0))
## Have to drop all the rows that have nan values because they will not help with net
## clean out rows with nan values
data = data[~np.isnan(data).any(axis=1)]
np.random.shuffle(data)

print(data.shape)

```

    (255, 9)

```python
from sympy import *
init_printing(use_latex=True)

import matplotlib.pyplot as plt
%matplotlib inline

## we will use scaled data
scaler = StandardScaler()
## when testing predicitions
## X = scaler.fit_transform( X )
## test = scaler.transform( test )

X = data[:,0:8]
X = scaler.fit_transform(X)
print(X.shape)
display(X)

labels = data[:,8]
print(labels.shape)
display(labels)

Y = keras.utils.to_categorical(labels, len(np.unique(labels)))
```

    (255, 8)

    array([[-1.35757558, -1.27522973, -1.52086612, ..., -1.55873791,
            -1.89660855,  1.46604656],
           [ 0.75380536,  0.86117985,  0.52723835, ..., -0.61404332,
             0.88202838,  0.91031442],
           [-1.02767231, -1.09963442, -1.49160748, ..., -1.49575827,
            -1.82949172,  1.29932692],
           ...,
           [ 0.46514   , -0.26067906, -1.37457294, ...,  0.73885264,
             0.82833491, -1.46080939],
           [ 0.46055801,  0.38038318, -1.00884   , ...,  0.90213319,
             1.06995551, -1.44228499],
           [ 0.83628118, -0.0899614 , -1.55743941, ...,  0.50559472,
             0.80148818, -1.40523618]])

    (255,)

    array([0., 1., 0., 2., 1., 1., 1., 0., 2., 1., 0., 1., 0., 1., 0., 0., 0.,
           0., 1., 0., 1., 0., 2., 2., 0., 1., 1., 1., 0., 0., 2., 1., 0., 1.,
           1., 0., 0., 2., 2., 0., 2., 2., 0., 0., 1., 1., 2., 1., 0., 0., 2.,
           1., 2., 1., 1., 0., 0., 2., 0., 2., 2., 0., 0., 0., 2., 0., 2., 2.,
           0., 2., 0., 2., 0., 2., 1., 0., 0., 2., 1., 1., 0., 1., 2., 1., 2.,
           0., 1., 2., 0., 0., 1., 2., 2., 0., 0., 2., 2., 1., 2., 1., 2., 2.,
           1., 2., 0., 0., 2., 0., 0., 1., 2., 2., 0., 1., 2., 1., 2., 2., 1.,
           0., 1., 1., 0., 2., 1., 0., 2., 0., 2., 2., 0., 1., 0., 0., 2., 2.,
           1., 2., 0., 1., 1., 1., 2., 2., 2., 2., 1., 2., 0., 0., 0., 1., 2.,
           1., 2., 1., 2., 0., 0., 0., 0., 0., 1., 2., 1., 0., 0., 0., 1., 1.,
           2., 0., 2., 0., 2., 2., 2., 0., 2., 1., 0., 0., 1., 2., 2., 0., 0.,
           2., 1., 0., 0., 0., 0., 1., 2., 1., 1., 0., 1., 2., 1., 1., 2., 0.,
           1., 1., 0., 1., 0., 1., 0., 2., 2., 1., 0., 2., 2., 0., 1., 0., 1.,
           2., 2., 2., 2., 1., 0., 2., 2., 1., 0., 0., 2., 1., 2., 0., 2., 0.,
           1., 1., 0., 2., 1., 1., 0., 1., 2., 1., 0., 0., 1., 0., 0., 0., 0.])

```python
input_size = X.shape[1]
output_size = Y.shape[1]
display(X.shape[1])
```

```python
model = keras.models.Sequential()

model.add(keras.layers.Dense(64,input_dim=8,activation='relu', bias_initializer=keras.initializers.Constant(value=0.01)))
model.add(keras.layers.Dense(64,input_dim=8,activation='sigmoid', bias_initializer=keras.initializers.Constant(value=0.01)))

model.add(keras.layers.Dense(3,activation='softmax'))
#categorical_crossentropy
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])

print(model.summary())
```

    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    dense_1 (Dense)              (None, 64)                576
    _________________________________________________________________
    dense_2 (Dense)              (None, 64)                4160
    _________________________________________________________________
    dense_3 (Dense)              (None, 3)                 195
    =================================================================
    Total params: 4,931
    Trainable params: 4,931
    Non-trainable params: 0
    _________________________________________________________________
    None

```python
history = model.fit(X, Y,
    batch_size=56,
    epochs=100,
    verbose=0,
    callbacks=[TQDMNotebookCallback()],
    validation_split = 0.2)

```

    HBox(children=(IntProgress(value=0, description='Training', style=ProgressStyle(description_width='initial')),…

    HBox(children=(IntProgress(value=0, description='Epoch 0', max=204, style=ProgressStyle(description_width='ini…

```python
plt.figure(1)

plt.subplot(211)
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')

plt.subplot(212)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.tight_layout()
plt.show()

score = model.evaluate(X, Y, verbose=1)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
```

```python
model.save_weights('MLN.weights')
model.save('MLN.model')
```

```python

```


```python
import numpy as np
from numba import jit
```

# nonuniform flow model

$$
\begin{align}
\frac{d}{dx} \left( \frac{\beta Q^2}{2gA^2} + H \right)= -i_e
\end{align}
$$

```python
@jit(nopython=True, parallel=False)
def NonUniformflow(sections, Q, Hdb):
    g = float(9.8)
    dhini = float(0.5)
    H = np.empty_like(Q)
    H[0] = Hdb
    arr = sections[0].calIeAlphaBetaRcUsubABS(Q[0], H[0])
    ied = arr[0]
    Ad = arr[-3]

    Hd = H[0]
    Qd = Q[0]

    for i in range(1, len(Q)):
        d = i - 1
        sc, sd = sections[i], sections[d]
        Qc = Q[i]
        Hc = sc.calHcABS( Qc )[0]
        arr = sc.calIeAlphaBetaRcUsubABS(Qc, Hc)
        iec = arr[0]
        Ac  = arr[-3]

        dx = sc.distance - sd.distance

        E1 = 0.5/g*Qc**2.0/Ac**2.0 + Hc
        E2 = 0.5/g*Qd**2.0/Ad**2.0 + Hd + 0.5*dx*(ied + iec)

        if E2 < E1 :
            H[i] = Hc
        else :
            Hc = Hc + float(0.001)
            dh = dhini
            for n in range(1000):
                arr = sc.calIeAlphaBetaRcUsubABS(Qc, Hc)
                iec = arr[0]
                Ac  = arr[-3]

                E1 = 0.5/g*Qc**2.0/Ac**2.0 + Hc
                E2 = 0.5/g*Qd**2.0/Ad**2.0 + Hd + 0.5*dx*(ied + iec)

                if np.abs(E1 - E2) < 0.00001 :
                    break
                elif E1 > E2 :
                    dh *= float(0.5)
                    Hc -= dh
                else:
                    Hc += dh

            H[i] = Hc

        Qd, Hd, ied, Ad = Qc, Hc, iec, Ac

    return H
```

# unsteady flow model

$$
\begin{align}
    &\frac{\partial A}{\partial t} + \frac{\partial Q}{\partial x} = 0 \\
    &\frac{\partial Q}{\partial t} + \frac{\partial }{\partial x}\left(\dfrac{\beta Q^2}{A}\right)
    + gA \frac{\partial H}{\partial x} + gAi_e = 0
\end{align}
$$

```python
@jit(nopython=True, parallel=False)
def UnSteadyflow(sections, A, Q, H, Abound, Qbound, dt):
    g = float(9.8)
    imax = len(A)
    Anew, Qnew, Hnew = np.zeros(imax), np.zeros(imax), np.zeros(imax)
    ie = np.zeros(imax)
    Beta = np.zeros(imax)

# continuous equation
    for i in range(1, imax-1) :
        dx = 0.5*(sections[i-1].distance - sections[i+1].distance)
        Anew[i] = A[i] - dt * ( Q[i] - Q[i-1] ) / dx

    Anew[imax-1] = Abound
    Anew[0] = Anew[1]
#     Anew[0] = (Anew[1] - A[1]) + A[0]

    for i in range(imax) :
        s = sections[i]
        Hnew[i], _, _ = s.A2HBS(Anew[i], H[i])
        arr = s.calIeAlphaBetaRcUsubABS(Q[i], H[i])
        ie[i] = arr[0]
        Beta[i] = arr[2]

# moumentum equation
    for i in range(1, imax-1):
        ic, im, ip = i, i-1, i+1
        dxp = sections[ic].distance - sections[ip].distance
        dxm = sections[im].distance - sections[ic].distance
        dxc = 0.5*(sections[im].distance - sections[ip].distance)

        Cr1 = 0.5*( Q[ic]/A[ic] + Q[ip]/A[ip] )*dt/dxp
        Cr2 = 0.5*( Q[ic]/A[ic] + Q[im]/A[im] )*dt/dxm
        dHdx1 = ( Hnew[ip] - Hnew[ic] ) / dxp
        dHdx2 = ( Hnew[ic] - Hnew[im] ) / dxm
        dHdx = (float(1.0) - Cr1) * dHdx1 + Cr2 * dHdx2

        Qnew[ic] = Q[ic] - dt * ( Beta[ic]*Q[ic]**2/A[ic] - Beta[im]*Q[im]**2/A[im] ) / dxc \
                         - dt * g * Anew[ic] * dHdx \
                         - dt * g * A[ic] * ie[ic]

    Qnew[imax-1] = Qnew[imax-2]
    Qnew[0] = Qbound

    return Anew, Qnew, Hnew
```


```python
print("Hello Universe")
```

    Hello Universe

```python
import sympy as smp
import math as m
```

```python
alpha,beta,gamma,p0,Y500,n,M0,M500,E,r500,r = smp.symbols('alpha beta gamma  p0 Y500 n M0 M500 E r500 r')
```

```python
h = 0.719 # km/Mpc s
```

```python
rho = 9.9 * 1e-27  # Critical density
M0 = (h * (4/3) * smp.pi * rho * 500 * r500 **3)/(3*1e14) # combining two reations of M500
```

```python
[p0, c500, gamma, alpha, beta] = [7.810,1.156,0.3292,1.0620,5.4807]
```

```python
M500 = (4/3) * smp.pi * rho * 500 * r500**3
```

```python
M500
```

$\displaystyle 6.6 \cdot 10^{-24} \pi r_{500}^{3}$

```python
p500 = (1.65 * 1e-3) * (E**smp.Rational(8,3) * (h/0.7)**2 * (M500 * (h/0.7)/((3 * 1e14) * M0))**smp.Rational(2,3))
p500
```

$\displaystyle 0.00220807397103211 E^{\frac{8}{3}}$

```python
p500.subs([(h,0.719)])
```

$\displaystyle 0.00220807397103211 E^{\frac{8}{3}}$

```python
r_s = r500/c500
x = r/r500 * 1.156
x = 1.138
```

```python
def p(r):
    return (p0 * p500) / ((x**gamma) * (1+ x**alpha)**((beta-gamma)/alpha))
```

```python
p(r)
```

$\displaystyle 0.000405875292732661 E^{\frac{8}{3}}$

```python
p(r).subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463243203779394),(h,0.719)])
```

$\displaystyle 0.000405875292732661 E^{\frac{8}{3}}$

```python
# Let's get the value of E
L500,Cm,delta =smp.symbols("L500 Cm delta")
L500 = 1e37 # 'J/s'
delta = 1.61  # alpha(m)
```

```python
Cm = (smp.exp(0.295) * 1e37) / (h/0.7)**2
Cm
```

$\displaystyle 1.27307846386144 \cdot 10^{37}$

```python
E = ((Cm/L500) * ((M500*h/0.7)/(3*1e14*M0))**delta)**(-3/7)
E
```

$\displaystyle 0.704984740946815$

```python
from sympy import integrate
```

```python
a = smp.integrate(p(r) * (4 * m.pi *r **2),(r,0,n * r500))
```

```python
a.subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(h,0.719)])
```

$\displaystyle 0.00170012645056338 E^{\frac{8}{3}} n^{3} r_{500}^{3}$

```python
b = smp.integrate(p(r)* (4 * m.pi * r**2),(r,0,r500))
```

```python
b.subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(h,0.719)])
```

$\displaystyle 0.00170012645056338 E^{\frac{8}{3}} r_{500}^{3}$

```python
def Y_nr500(a):
    return Y500 * (a/b)
```

```python
Y500 = 1.54 * 1e-3 * E**smp.Rational(2,3) *((h/0.719)**(-5/2)) * (M500 * h/ (3 * 1e14 * M0))
Y500.subs([(h,0.719)])
```

$\displaystyle 0.00121985215802838$

```python
Y_nr500(a = a) # generalization
```

$\displaystyle 0.00121985215802838 n^{3}$

## For n=1

```python
a1 = a.subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,1),(h,0.719)])
a1
```

$\displaystyle 0.00170012645056338 E^{\frac{8}{3}} r_{500}^{3}$

```python
Y_nr500(a = a1).subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,1),(h,0.719)])
```

$\displaystyle 0.00121985215802838$

```python
Y_nr500(a = a1)/Y500 #1
```

$\displaystyle 1.0$

## For n=2

```python
a2 = a.subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,2),(h,0.719)])
a2
```

$\displaystyle 0.013601011604507 E^{\frac{8}{3}} r_{500}^{3}$

```python
Y_nr500(a = a2).subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,2),(h,0.719)])
```

$\displaystyle 0.00975881726422702$

```python
Y_nr500(a = a2)/Y500 #2
```

$\displaystyle 8.0$

## For n = 3

```python
a3 = a.subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,3),(h,0.719)])
a3
```

$\displaystyle 0.0459034141652112 E^{\frac{8}{3}} r_{500}^{3}$

```python
Y_nr500(a = a3).subs([(alpha,1.0620), (beta,5.4807), (gamma,0.3292),(p0,8.463),(n,3),(h,0.719)])
```

$\displaystyle 0.0329360082667662$

```python
Y_nr500(a = a3)/Y500 #3
```

$\displaystyle 27.0$


# 微積分

## 微分

関数 $y=f(x)$ が与えられた時に $x=a$ から $x=b$ への値の変化に注目する

この間の $f(x)$ の増分の $x$ の増分に対する割合、

$$
\frac{f(b) - f(a)}{b - a}
$$

を $x$ が $a$ から $b$ に変化するときの **平均変化率** と呼ぶ

これはつまり、点 $(a, f(a))$ と点 $(b, f(b))$ を結ぶ直線の傾きである

$b$ を $a$ に近づけていくことで、点 $(a, f(a))$ と点 $(b, f(b))$ を通る直線は下図のように、点 $(a, f(a))$ における $y=f(x)$ の **接線** となる

このときの平均変化率の極限は接線の傾きになる

この接線の傾きのことを $x=a$ における $f(x)$ の **微分係数** と呼び $f'(a)$ で表す

$$
f'(a) = \lim_{b\to{a}}\frac{f(b) - f(a)}{b - a}
$$

ここで $b-a=h$ とおくと、$b\rightarrow a$ のとき $h\rightarrow0$ なので、次のように定義することが出来る

$$
f'(a) = \lim_{h\to0}\frac{f(a+h) - f(a)}{h}
$$

一般には上式を微分係数の定義として用いることが多い

この極限値が存在する時、関数 $f(x)$ は $x=a$ で微分可能である、という

逆に、次の関数は $x=0$ で右極限と左極限が異なるため、微分不可能である

$$
\begin{align}
f(x) = |x| \\
\\
\lim_{h\to+0}\frac{|x+h| - |x|}{h} = 1 \\
\lim_{h\to-0}\frac{|x+h| - |x|}{h} = -1 \\
\end{align}
$$

ここまで $a$ を固定して考えてきたが、$a$ を動かしていくと $f'(a)$ は各点での $y=f(x)$ の接線の傾きを表す関数と考えることが出来る

これを $x$ の関数と考えて $f'(x)$ と記述し、$f(x)$ の **導関数** と呼ぶ

導関数の表記にはいくつかあり、

$$
f'(x),\ \frac{df}{dx}(x),\ \frac{d}{dx}f(x),\ \frac{dy}{dx}
$$

は全て同じものである

特に $\frac{d}{dx}$ という表記は、$x$ の関数に対して左に作用させることで導関数を意味するため、記述上便利なことが多い

導関数を求めることを **微分** すると呼ぶ

### 導関数の性質
定数関数 $f(x) = k$ の導関数は 0 となる

実際 $f(x+h) = f(x) = k$ であるため、$\frac{k}{dx}k = \lim_{h\to0}\frac{k-k}{h} = 0$ となる

導関数には、次のような性質がある

$$
\begin{align}
\frac{d}{dx}(f(x) + g(x)) &= \frac{d}{dx}f(x) + \frac{d}{dx}g(x) \\
\frac{d}{dx}(kf(x)) &= k\frac{d}{dx}f(x)
\end{align}
$$

### 多項式の微分
多項式の定理として以下の **二項定理** が知られている

$$
(a+b)^n = {}_nC_0a^n + {}_nC_1a^{n-1}b + {}_nC_2a{n-2}b^2 + \cdots + {}_nC_nb^n
$$

ここで、

$$
{}_nC_k = \frac{n!}{k!(n - k)!}
$$

であり、特に ${}_nC_0=1$, ${}_nC1=n$ となる

これを使って多項式の微分を考える

一般の多項式の微分を考えるには、導関数の性質より、$x^n$ の微分を考えれば十分で、後は定数倍と和で導関数を求めることが出来る

そのため、以下のように $x^n$ の微分を考えてみる

$$
\begin{align}
\frac{d}{dx}x^n &= \lim_{h\to0}\frac{(x+h)^n - x^n}{h} \\
&= \lim_{h\to0}\frac{x^n + nhx^{n-1} + {}_nC_2h^2x^{n-2} + \cdots + {}_nC_nh^n - x^n}{h} \quad (二項定理より) \\
&= \lim_{h\to0}\frac{nhx^{n-1} + {}_nC_2h^2x^{n-2} + \cdots + {}_nC_nh^n}{h} \\
&= \lim_{h\to0}(nx^{n-1} + {}_nC_2hx^{n-2} + \cdots + {}_nC_nh^{n-1}) \\
&= nx^{n-1} \quad (h が掛かっている部分は全て消える)
\end{align}
$$

つまり $\frac{d}{dx}x^n = nx^{n-1}$ となる

これはよく使うため、公式として覚えておくと良い

この公式を用いて具体的な関数として $x^4-2x^2+3$ の導関数を考えてみる

$$
\begin{align}
\frac{d}{dx}(x^4-2x^2+3) &= \frac{d}{dx}(x^4) + \frac{d}{dx}(-2x^2) + \frac{d}{dx}(3) \\
&= \frac{d}{dx}(x^4) - 2\frac{d}{dx}(x^2) + \frac{d}{dx}(3) \\
&= 4x^3 - 2\times 2x + 0 \\
&= 4x^3 - 4x
\end{align}
$$

#### 演習
次の関数の導関数を求めよ

1. $f(x) = x^3$
2. $f(x) = x^5 + 3x^2 - 2x + 2$

#### 解答
1. $$
\frac{d}{dx}(x^3) = 3x^2
$$
2. $$
\frac{d}{dx}(x^5 + 3x^2 - 2x + 2) = 5x^4 + 6x - 2
$$

```julia
using SymPy

@vars x
f(x) = x^3

# f(x) の x に対する導関数を求める
diff(f(x), x)
```

$\begin{equation*}3 x^{2}\end{equation*}$

```julia
f(x) = x^5 + 3x^2 - 2x + 2
diff(f(x), x)
```

$\begin{equation*}5 x^{4} + 6 x - 2\end{equation*}$

### 積・商の微分と高階道関数
関数の積の微分については、次の式が成り立つ

$$
\frac{d}{dx}(f(x)g(x)) = f'(x)g(x) + f(x)g'(x)
$$

この式は以下により証明される

$$
\begin{align}
\frac{d}{dx}(f(x)g(x)) &= \lim_{h\to0}\frac{f(x+h)g(x+h) - f(x)g(x)}{h} \\
&= \lim_{h\to0}\frac{f(x+h)g(x+h) - f(x)g(x+h) + f(x)g(x+h) - f(x)g(x)}{h} \\
&= \lim_{h\to0}\frac{(f(x+h) - f(x))g(x+h) + f(x)(g(x+h) - g(x))}{h} \\
&= \lim_{h\to0}\frac{f(x+h) - f(x)}{h}\cdot\lim_{h\to0}g(x+h) + \lim_{h\to0}f(x)\cdot\lim_{h\to0}\frac{g(x+h) - g(x)}{h} \\
&= f'(x)g(x) + f(x)g'(x)
\end{align}
$$

また、関数の商の微分については、次の式が成り立つ

$$
\frac{d}{dx}\frac{f(x)}{g(x)} = \frac{f'(x)g(x) - f(x)g'(x)}{g(x)^2}
$$

上記を証明するには、まず $\frac{d}{dx}\left(\frac{1}{g(x)}\right)$ の計算を行う

$$
\begin{align}
\frac{d}{dx}\left(\frac{1}{g(x)}\right) &= \lim_{h\to0}\frac{\frac{1}{g(x+h)} - \frac{1}{g(x)}}{h}) \\
&= \lim_{h\to0}\frac{g(x) - g(x+h)}{hg(x)g(x+h)} \\
&= \lim_{h\to0}\frac{-(g(x+h) - g(x))}{h}\cdot\lim_{h\to0}\frac{1}{g(x)g(x+h)} \\
&= -\frac{g'(x)}{g(x)^2}
\end{align}
$$

次に、上式を $\frac{d}{dx}\frac{f(x)}{g(x)}$ に当てはめて積の微分の公式を使うと以下のようになる

$$
\begin{align}
\frac{d}{dx}\left(\frac{f(x)}{g(x)}\right) &= \frac{d}{dx}\left(f(x)\cdot\frac{1}{g(x)}\right) \\
&= f'(x)\cdot\frac{1}{g(x)} + f(x)\cdot\left(\frac{1}{g(x)}\right)' \\
&= f'(x)\cdot\frac{1}{g(x)} + f(x)\cdot\left(-\frac{g'(x)}{g(x)^2}\right) \\
&= \frac{f'(x)g(x) - f(x)g'(x)}{g(x)^2}
\end{align}
$$

関数 $y=f(x)$ の導関数 $f'(x)$ のさらに導関数をとることがある

それを **2階導関数** と呼び、$f''(x)$, $\frac{d^2}{dx^2}f(x)$, $\frac{d^2y}{dx^2}$ などの書き方をする

さらに一般的には微分を $n$ 回行うこともあり、$f(x)$ を $n$ 回微分したものを$n$階導関数と呼び、$f^(n)(x)$ または $\frac{d^n}{dx^n}f(x)$ などで表す

すべての関数が $n$ 回微分可能とは限らず、特に何度でも（無限回）微分可能な関数を **滑らか** であるという

例えば任意の多項式関数は滑らかである

実際、$n$次元を微分すると$n-1$次元になるため、何度か微分すると定数になり、定数は何度でも微分可能（永遠に 0 になり続ける）、すなわち滑らかである

#### 演習1
次の関数の導関数を求めよ

1. $f(x) = (x^2 + 1)(x^2 + 2x + 5)$
2. $f(x) = \frac{x^2 + x + 1}{x^2 + 1}$

#### 演習2
次の関数の2階導関数を求めよ

1. $f(x) = x^3$
2. $f(x) = \frac{1}{x^2 + 1}$

#### 解答1
1. $$
\begin{align}
\frac{d}{dx}((x^2 + 1)(x^2 + 2x + 5)) &= \frac{d}{dx}(x^2 + 1)\cdot(x^2 + 2x + 5) + (x^2 + 1)\cdot\frac{d}{dx}(x^2 + 2x + 5) \\
&= 2x(x^2 + 2x + 5) + (x^2 + 1)(2x + 2) \\
&= 2x^3 + 4x^2 + 10x + 2x^3 + 2x^2 + 2x + 2 \\
&= 4x^3 + 6x^2 + 12x + 2
\end{align}
$$
2. $$
\begin{align}
\frac{d}{dx}\left(\frac{x^2 + x + 1}{x^2 + 1}\right) &= \frac{(x^2 + x + 1)'\cdot(x^2 + 1) - (x^2 + x + 1)\cdot(x^2 + 1)'}{(x^2 + 1)^2} \\
&= \frac{(2x + 1)(x^2 + 1) - (x^2 + x + 1)\cdot2x}{(x^2 + 1)^2} \\
&= \frac{1 - x^2}{(x^2 + 1)^2}
\end{align}
$$

#### 解答2
1. $$
\begin{align}
\frac{d}{dx}x^3 &= 3x^2 \\
\frac{d^2}{dx^2}x^3 &= (3x^2)' = 6x
\end{align}
$$
2. $$
\begin{align}
\frac{d}{dx}\frac{1}{x^2 + 1} &= \frac{-(x^2 + 1)'}{(x^2 + 1)^2} \\
&= \frac{-2x}{(x^2 + 1)^2} \\
\frac{d^2}{dx^2}\frac{1}{x^2 + 1} &= \left(\frac{-2x}{(x^2 + 1)^2}\right)' \\
&= \frac{(-2x)'\cdot(x^2 + 1)^2 - (-2x)\cdot((x^2 + 1)^2)'}{(x^2 + 1)^4} \\
&= \frac{-2(x^2 + 1)^2 + 8x^2(x^2 + 1)}{(x^2 + 1)^4} \\
&= \frac{6x^2 - 2}{(x^2 + 1)^3}
\end{align}
$$

```julia
@vars x
f(x) = (x^2 + 1) * (x^2 + 2x + 5)

# 微分して級数展開する
diff(f(x), x) |> expand
```

$\begin{equation*}4 x^{3} + 6 x^{2} + 12 x + 2\end{equation*}$

```julia
f(x) = (x^2 + x + 1) / (x^2 + 1)

# 微分して因数分解する
diff(f(x), x) |> factor
```

$\begin{equation*}- \frac{\left(x - 1\right) \left(x + 1\right)}{\left(x^{2} + 1\right)^{2}}\end{equation*}$

```julia
f(x) = x^3

# 2階導関数: 微分して微分する
diff(f(x), x) |> diff
```

$\begin{equation*}6 x\end{equation*}$

```julia
f(x) = 1 / (x^2 + 1)
diff(f(x), x) |> diff |> factor
```

$\begin{equation*}\frac{2 \left(3 x^{2} - 1\right)}{\left(x^{2} + 1\right)^{3}}\end{equation*}$

### 合成関数の微分と逆関数の微分
$w$ についての関数 $y=f(w)$ と $x$ についての関数 $w=g(x)$ が与えられた時、関数 $y=f(g(x))$ は **合成関数** と呼ばれる

これはつまり $x$ に対して、関数 $g$ を作用させた後に $f$ を作用させるという関数である

$f(x)$ と $g(x)$ が微分可能である時、合成関数の微分について次の式が成り立つ

$$
\frac{dy}{dx} = \frac{dy}{dw}\cdot\frac{dw}{dx}
$$

次に逆関数について考えてみる

関数 $y=f(x)$ について、もしこの関数の値域にあるすべての $y$ に対して $y=f(x)$ となる $x$ が1つしかないという条件を満たすとすると、$y$ を1つ決めると $y=f(x)$ を満たす $x$ は1つ決まることになり、この対応関係も関数 $x=g(y)$ とみなすことが出来る

この関数 $x=g(y)$ を $f(x)$ の **逆関数** と呼び、$f^{-1}$ のように書くことがある

逆関数の性質としては以下が成り立つ

$$
\begin{align}
f^{-1}(f(x)) &= x \\
f(f^{-1}(x)) &= x
\end{align}
$$

#### 演習1
次の関数の導関数を、合成関数の性質を利用して求めよ

1. $f(x) = (2x + 1)^5$
2. $f(x) = (x^3 + x^2 + x + 1)^3$

#### 演習2
次の関数の逆関数を求めよ

1. $$
f(x) = e^{2x}
$$
2. $$
f(x) = \frac{1}{1 + \ln{x}}\quad\begin{pmatrix}x\gt0, \\ x\neq\frac{1}{e}\end{pmatrix}
$$
3. $$
f(x) = \frac{1}{1 + e^{-x}}
$$

#### 解答1

#### 解答2

### 指数関数と対数関数の微分
自然対数の底 $e$ の定義 $e = \lim_{n\to\infty}(1 + \frac{1}{n})^n$ において、$\frac{1}{n}=t$ とおくと、$n\to\infty$ のとき $t\to0$ なので、

$$
e = \lim_{t\to0}(1+t)^\frac{1}{t}
$$

となる

ここで $\lim_{h\to0}\frac{e^h-1}{h}$ の極限を計算したいとする

$e^h-1=u$ とおくと、$h\to0$ のとき $u\to0$ であり、$e^h=1+u$ より $h=\ln(1+u)$ となり、以下が導かれる

$$
\begin{align}
\lim_{h\to0}\frac{e^h-1}{h} &= \lim_{u\to0}\frac{u}{\ln(1+u)} \\
&= \lim_{u\to0}\frac{1}{\frac{1}{u}\ln(1+u)} \\
&= \lim_{u\to0}\frac{1}{\ln(1+u)^\frac{1}{u}} \\
&= \lim_{u\to0}\frac{1}{\ln{e}} \\
&= 1
\end{align}
$$

上記の結果を利用して指数関数 $y=e^x$ の微分を行う

$$
\begin{align}
\frac{d}{dx}e^x &= \lim_{h\to0}\frac{e^{x+h}-e^x}{h} \\
&= \lim_{h\to0}\frac{e^x(e^h-1)}{h} \\
&= e^x\cdot\lim_{h\to0}\frac{e^h-1}{h} \\
&= e^x
\end{align}
$$

上記のように指数関数の微分は、元の関数と同じになる

このことから、$e$ という数が微分において特別な意味を持つことが分かる

すなわち、$e$ は指数関数の微分の計算を簡単にすることが出来るということである

実際、$e$ の代わりに任意定数 $a$ ($a\gt0$) を底とする指数関数 $y=a^x$ の微分を行ってみると、

$$
\frac{d}{dx}a^x = e^{x\ln{a}}\cdot\ln{a} = a^x\ln{a}
$$

となり、あまりきれいな形にならない

次に、対数関数 $y=\ln{x}$ の微分を考えてみると、以下のようになる

$$
\begin{align}
\frac{d}{dx}\ln{x} &= \lim_{h\to0}\frac{\ln(x+h)-\ln{x}}{h} \\
&= \lim_{h\to0}\left(\frac{1}{h}\ln\frac{x+h}{x}\right) \\
&= \frac{1}{x}\lim_{h\to0}\left(\frac{x}{h}\ln(1+\frac{h}{x})\right) \\
&= \frac{1}{x}\lim_{h\to0}\ln\left(1+\frac{h}{x}\right)^\frac{x}{h}
\end{align}
$$

ここで $\frac{h}{x}=t$ とおくと、$h\to0$ のとき $t\to0$ であるため、

$$
\lim_{h\to0}\ln\left(1+\frac{h}{x}\right)^\frac{x}{h} = \lim_{t\to0}\ln(1+t)^\frac{1}{t}=\ln{e}=1
$$

となり、つまり、

$$
\frac{d}{dx}\ln{x} = \frac{1}{x}
$$

である

これも、対数の底を $e$ にしたことにより結果が簡潔になることが分かる

実際、一般の $a\gt0$ を底とする対数関数 $y=\log_ax$ の導関数を求めると、$\log_ax=\frac{\log_ex}{\log_ea}$ より、

$$
\frac{d}{dx}\log_ax = \frac{1}{x\ln{a}}
$$

という、やや複雑な形の関数となる

以上より、指数関数と対数関数で、$e$ という数が特別な意味を持つことが確認できた

逆に考えれば、指数・対数関数を微分した結果がシンプルになるように決められた数が $e$ であるとも言える

なお、関数 $y=\ln{x}$ の微分の話をしたときは $x\gt0$ であることを暗黙的に仮定していた

この関数は $x\gt0$ の定義域でしか定義されないためである

そこで $\ln|x|$ の微分を考えてみる

この関数は $x\neq0$ で定義されているため、

$$
\ln|x| = \left\{
\begin{array}{cc}
\ln{x} & (x\gt0) \\
\ln(-x) & (x\lt0)
\end{array}
\right.
$$

となる

$x\gt0$ の場合はすでに計算済みのため、ここでは $x\lt0$ の範囲での微分を考える

$$
\begin{align}
\frac{d}{dx}\ln(-x) &= \frac{1}{-x}\times(-1) \quad (\because 合成関数の微分) \\
&= \frac{1}{x}
\end{align}
$$

したがって、$x\gt0$ でも $x\lt0$ でも等しく導関数は $\frac{1}{x}$ となり、

$$
\frac{d}{dx}\ln|x| = \frac{1}{x}
$$

を示すことができた

### べき乗の微分
$n$ が自然数のとき $\frac{d}{dx}x^n = nx^{n-1}$ が成り立つことはすでに示したが、この式は $n$ を自然数に限定しなくても成り立つ

$$
\frac{d}{dx}x^\alpha = \alpha{x}^{\alpha-1} \quad (\alpha:\ \alpha\geq0 を満たす任意の実数)
$$

### 関数の増減と極大・極小
導関数の応用例として、関数の増減を調べてグラフの概形を推測するというものがある

例えば次のような関数について考えてみる

$$
f(x) = x^3-3x^2-9x+5
$$

この導関数は以下のようになる

$$
f'(x) = 3x^2-6x-9 = 3(x+1)(x-3)
$$

上記は $x\lt-1$ では正になり、$x=-1$ では 0 に、$-1\lt{x}\lt3$ では負に、$x=3$ では 0 に、$x\gt3$ では正になる

これを表で表すと以下のようになる

| x | ... | -1 | ... | 3 | ... |
|:--:|:--:|:--:|:--:|:--:|:--:|
|f'(x)| + | 0 | - | 0 | + |
|f'(x)| ↗ | 極大 | ➘ | 極小 | ↗ |

導関数とは接線の傾きを表していたため、これが正であることは、関数 $f(x)$ はその区間で増加していることになる

同様に、導関数が負の範囲では $f(x)$ は減少している

これを表に表すと、上記の表の3行目のようになり、この表を **増減表** と呼ぶ

特に導関数が 0 に等しくなるときが重要で、このとき $f(x)$ が極大または極小になることがある

- **極大**:
    - 十分に小さい近傍で最大であること
    - 極大のときの関数の値を **極大値** と呼ぶ
- **極小**:
    - 十分に小さい近傍で最小であること
    - 極小のときの関数の値を **極小値** と呼ぶ

```julia
using Plots

f(x) = x^3 - 3x^2 - 9x + 5
plot(f; x_lims=(-5, 5), y_lims=(-25, 25))
```

極大値と極小値をまとめて **極値** と呼ぶこともある

導関数が接線の傾きを表すことを思い出すと、極値をとる点で導関数が 0 になるのは直感的に分かる

一方、導関数が 0 であっても、必ずしも極大または極小とは限らないため注意が必要である

例えば $f(x)=x^3$ という関数を考えると、この導関数は $f'(x)=3x^2$ であるため、$x=0$ で $f'(x)=0$ となる

しかし $f'(x)=3x^2$ は $x\lt0$ でも $x\gt0$ でも正なので、$f(x)$ は $x=0$ の左でも右でも増加していることになり、$x=0$ では極値をとらない

```julia
f(x) = x^3
plot(f; x_lims=(-5, 5), y_lims=(-25, 25))
```

このように、$f'(x)=0$ を満たす $x$ で極大になるか、極小になるか、あるいは極値をとらないか調べるには、増減表を書くか2階導関数を調べる必要がある

#### 演習1
次の関数の極大と極小を求めよ

$$
f(x) = x^3 - 3x
$$

#### 演習2
次の関数の最小値を求めよ

$$
f(x) = 3x^4 - 4x^3 - 36x^2
$$

#### 解答1
$$
f'(x) = 3x^2 - 3 = 3(x+1)(x-1)
$$

| x | ... | -1 | ... | 1 |
|:--:|:--:|:--:|:--:|:--:|
|f'(x)| + | 0 | - | 0 | + |
|f'(x)| ↗ | 極大 | ➘ | 極小 | ↗ |

以上より、極大値 $f(-1)=2$, 極小値 $f(1)=-2$

#### 解答2
$$
f'(x) = 12x^3 - 12x^2 - 72x = 12x(x-3)(x+2)
$$

| x | ... | -2 | ... | 0 | ... | 3 |
|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
|f'(x)| - | 0 | + | 0 | - | 0 | + |
|f'(x)| ➘ | 極小 | ↗ | 極大 | ➘ | 極小 | ↗ |

増減表より、極小値は2つあり、最小値はそのうちのいずれかとなる

$$
\begin{align}
f(-2) &= -64 \\
f(3) &= -189
\end{align}
$$

より、最小値は $f(3) = -189$

```julia

```


| |Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section18.5

```python
#
# Pierre Proulx
#
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp
from IPython.display import *
sp.init_printing(use_latex=True)
%matplotlib inline
```

```python
# Paramètres, variables et fonctions

eta=sp.symbols('eta')       # eta est égal à x/(sp.sqrt(4*DAB*z/Vmax))
phi=sp.Function('phi')(eta) # phi est Ca/Ca0
Ca,Ca0=sp.symbols('C_A C_A0')
```

```python
#
# La solution est similaire à celle du chapitre 4,
# on utilisera d'abord cette solution ensuite on insèrera les
# variables voulues
#
eq1=sp.Eq(phi.diff(eta,eta)+2*eta*phi.diff(eta),0)
display(eq1)
phi=sp.dsolve(eq1)
display(phi)
```

```python
cl1=sp.Eq(phi.rhs.subs(eta,0)-1,0)
cl2=sp.Eq(phi.rhs.subs(eta,sp.oo),0)
constantes=sp.solve((cl1,cl2),sp.symbols('C1 C2'),dict=True)
display(constantes)
phi=phi.subs(constantes[0])
display(phi)
etaCh18=x/(sp.sqrt(4*DAB*z/Vmax))
phi=phi.subs(eta,etaCh18)
display(phi.rhs)
```

```python
#
# On pourrait maintenant voir l'effet des paramètres sur le profil
# afin de comparer avec la figure 18.5-1
#
# Valeurs estimées des paramètres en utilisant la diffusivité de l'hélium dans l'eau, c'est
# un tout petit atome qui diffusera donc très rapidement
#
dico={'D_AB':7e-9,'V_m':.1}
Cplot=phi.rhs.subs(dico)
delta=0.003                          # film de 3 millimètres, essayez de varier la position z pour voir
Vzplot=1-x**2/delta**2               # la combinaison profil de concentration-profil de vitesse
#                                      comme sur la figure 18.5-1.
#                                      La simplification v=v_max est-elle justifiée???
#
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize']=10,8
p=sp.plot((Cplot.subs(z,1),(x,0,delta)),(Vzplot,(x,0,delta)),show=False,legend=True)
p[0].line_color='blue'
p[0].label='concentration'
p[1].line_color='red'
p[1].label='vitesse'
p.show()
```

```python

```