```python
%%capture
# Load the quadrotor simulator from a previous notebook
%run quadsim.ipynb
```

```python
from IPython.core.display import HTML
HTML('<style>.impl-inner{background-color:#e4ffe6} .impl-outer{background-color:#e4ebff} .text_cell_render p { text-align: justify; text-justify: inter-word;}</style>')
```

# Sliding Mode Control

This discussion on sliding mode control for a multirotor closely follows the work of L'Afflitto et al. [1], in the notation of Khalil [2]. A diagram of the control scheme (taken from [1]) is shown below. In sliding mode control, trajectories of the system are steered to a subspace of the state space (i.e., the sliding manifold) in finite time. Once on this sliding surface, the system can asymptotically converge to the origin without regard to matched uncertainties. In [1], the control architecture shown below is used with sliding mode control to design an autopilot that steers a quadrotor to a given location with a given yaw angle.

A more introductory implementation of SMC is found in the [notebook](smc_cranfield.ipynb) based on Shaik's work.

## Theory - System Modeling

Consider a nonlinear system of the form [2, eq 14.1]
\begin{equation}\label{eq:general_nonlinear_sys}
\dot{x} = f(x) + B(x)[G(x)E(x)u + \delta(t,x,u)],\quad x \in \mathbb{R}^n, \ u \in \mathbb{R}^p
\end{equation}
where $f$, $B$, and $E$ are known, while $G$ and $\delta$ could be uncertain. We assume that $E(x)$ is a nonsingular matrix and that $G(x)$ is a diagonal matrix whose elements are positive and bounded away from zero.

Suppose that $f(0) = 0$ so that, in the absence of $\delta$, the origin is an open-loop equilibrium point. **Our goal is to design a state feedback control law to stabilize the origin for all uncertainties in $G$ and $\delta$.**

Given a diffeomorphism $T$, \eqref{eq:general_nonlinear_sys} can be written in *regular form* [2, eq 14.4] as:
\begin{align}
\dot\eta &= f_a(\eta, \xi) \label{eq:normal-eta} \\
\dot\xi  &= f_b(\eta, \xi) + G(x)E(x)u + \delta(t, x, u). \label{eq:normal-xi}
\end{align}

Note that a large class of mechanical systems may be written in regular form, with the kinematics captured by \eqref{eq:normal-eta} and the dynamics captured by \eqref{eq:normal-xi}. This is great because kinematics are typically assumed to be perfectly known, while the dynamics of a system are subject to modeling errors and uncertainties in the system's inertial properties [1].

## Practice - System Modeling
In keeping with the architecture in the above figure from [1], we will consider the inner loop and the outer loop as separate nonlinear systems, writing each in the form of \eqref{eq:general_nonlinear_sys} and creating two sliding mode controllers that work together to stabilize the system around trajectories.

For the purposes of control, we consider the inner loop kinematics of a quadrotor to be
\begin{align}
\dot{r_z} &= v_z \\
\dot{\phi} &\approx p \\
\dot{\theta} &\approx q \\
\dot{\psi} &\approx r \\
\end{align}
where we have assumed small angles s.t. $\Gamma \approx I_{3 \times 3}$. The inner loop dynamics are
\begin{align}
\ddot{r_z} &= g - \frac{F}{m}\cos\phi\cos\theta \\
\dot\omega &= -I^{-1} (\omega \times I\omega + M).
\end{align}

In our specific case, we can express the quadrotor system in the form of \eqref{eq:general_nonlinear_sys} with the following definitions:
\begin{align}
x &\triangleq
\begin{bmatrix} x_1 & x_2 & x_3 & x_4 & x_5 & x_6 & x_7 & x_8 \end{bmatrix}^\top =
\begin{bmatrix} r_z & \phi & \theta & \psi & \dot{r}_z & p & q & r \end{bmatrix}^\top \\
u &\triangleq
\begin{bmatrix} u_1 & u_2 & u_3 & u_4 \end{bmatrix}^\top =
\begin{bmatrix} F & \tau_\phi & \tau_\theta & \tau_\psi \end{bmatrix}^\top.
\end{align}

Which evolves as
\begin{align}\label{eq:inner-loop-dynamics-ss}
\dot{x} =
\begin{bmatrix} \dot{x}_1 \\ \dot{x}_2 \\ \dot{x}_3 \\ \dot{x}_4 \\ \dot{x}_5 \\ \dot{x}_6 \\ \dot{x}_7 \\ \dot{x}_8 \end{bmatrix} =
\begin{bmatrix} \dot{r}_z \\ \dot\phi \\ \dot\theta \\ \dot\psi \\ \ddot{r}_z \\ \dot{p} \\ \dot{q} \\ \dot{r} \end{bmatrix} =
\begin{bmatrix}
    x_5 \\ x_6 \\ x_7 \\ x_8 \\
    g - \frac{u_1}{m}\cos x_2 \cos x_3 \\
    \frac{J_y - J_z}{J_x} x_7 x_8 + \frac{1}{J_x} u_2 \\
    \frac{J_z - J_x}{J_y} x_6 x_8 + \frac{1}{J_y} u_3 \\
    \frac{J_x - J_y}{J_z} x_6 x_7 + \frac{1}{J_z} u_4
\end{bmatrix}
\end{align}

In order to transform these inner-loop dynamics into the regular form, we must identify the certain and uncertain components. We can break up \eqref{eq:inner-loop-dynamics-ss} into
\begin{align}
\dot{x} =
\underbrace{\begin{bmatrix}
    x_5 \\ x_6 \\ x_7 \\ x_8 \\
    g \\
    \frac{J_y - J_z}{J_x} x_7 x_8 \\
    \frac{J_z - J_x}{J_y} x_6 x_8 \\
    \frac{J_x - J_y}{J_z} x_6 x_7
\end{bmatrix}}_{f(x)} +
\underbrace{\begin{bmatrix}
    0_{4\times 4} \\ \hdashline
    I_{4\times 4}
\end{bmatrix}}_{B(x)}
\left(
\underbrace{\begin{bmatrix}
\frac{\cos x_2\cos x_3}{m} &0&0&0 \\
0& \frac{1}{J_x} &0&0 \\
0&0& \frac{1}{J_y} &0 \\
0&0&0& \frac{1}{J_z}
\end{bmatrix}}_{G(x)}
\underbrace{\begin{bmatrix}
-1&0&0&0 \\
0&1&0&0 \\
0&0&1&0 \\
0&0&0&1
\end{bmatrix}}_{E(x)}
\underbrace{\begin{bmatrix}
u_1 \\ u_2 \\ u_3 \\ u_4
\end{bmatrix}}_{u} +
\underbrace{0_{4\times 1}}_{\delta(t, x, u)}
\right).
\end{align}

Note how $G(x)$ contains the mass and the inertia terms, which are likely to be uncertain.

We then use the identity diffeomorphism $T:D\to\mathbb{R}^{n=8}, x\mapsto x$ to transform this system into the regular form:
\begin{align} \label{eq:inner-regular-transform}
\eta &=
\begin{bmatrix} x_1 & x_2 & x_3 & x_4 \end{bmatrix}^\top =
\begin{bmatrix} r_z & \phi & \theta & \psi \end{bmatrix}^\top \in \mathbb{R}^{n-p=4} \\
\xi &=
\begin{bmatrix} x_5 & x_6 & x_7 & x_8 \end{bmatrix}^\top =
\begin{bmatrix} \dot{r_z} & p & q & r \end{bmatrix}^\top \in \mathbb{R}^{p=4},
\end{align}

which evolves as

\begin{align}
\dot\eta &=
\begin{bmatrix} \dot{x_1} \\ \dot{x_2} \\ \dot{x_3} \\ \dot{x_4} \end{bmatrix} =
\begin{bmatrix} x_5 \\ x_6 \\ x_7 \\ x_8 \end{bmatrix} = f_a(\eta, \xi) = \xi \\
\dot\xi &=
\begin{bmatrix} \dot{x_5} \\ \dot{x_6} \\ \dot{x_7} \\ \dot{x_8} \end{bmatrix} =
\underbrace{\begin{bmatrix}
    g \\
    \frac{J_y - J_z}{J_x} x_7 x_8 \\
    \frac{J_z - J_x}{J_y} x_6 x_8 \\
    \frac{J_x - J_y}{J_z} x_6 x_7
\end{bmatrix}}_{f_b(\eta,\xi)} +
\underbrace{\begin{bmatrix}
\frac{\cos x_2\cos x_3}{m} & 0_{1\times 3} \\
0_{3\times 1} & I^{-1}
\end{bmatrix}}_{G(x)}
\underbrace{\begin{bmatrix}
-1 & 0_{1\times 3} \\
0_{3\times 1} & I_{3\times 3}
\end{bmatrix}}_{E(x)}
\underbrace{\begin{bmatrix}
u_1 \\ u_2 \\ u_3 \\ u_4
\end{bmatrix}}_{u} +
\underbrace{0_{4\times 1}}_{\delta(t, x, u)}
\end{align}

The outer loop kinematics are
$$
\begin{align}
\dot{r}_x &= v_x \\
\dot{r}_y &= v_y
\end{align}
$$
with dynamics
$$
\begin{align}
\ddot{r}_x &= -\frac{u_1}{m}\left( \cos\phi\sin\theta\cos\psi + \sin\phi\sin\psi \right) \\
\ddot{r}_y &= -\frac{u_1}{m}\left( \cos\phi\sin\theta\sin\psi - \sin\phi\cos\psi \right).
\end{align}
$$

Expressing the outer loop in state-space form yields
$$
\begin{align}
x &\triangleq
\begin{bmatrix} x_1 & x_2 & x_3 & x_4 \end{bmatrix}^\top =
\begin{bmatrix} r_x & r_y & \dot{r}_x & \dot{r}_y \end{bmatrix}^\top \\
u &\triangleq
\begin{bmatrix} d_1 & d_2 \end{bmatrix}^\top =
\begin{bmatrix} \theta^d & \phi^d \end{bmatrix}^\top.
\end{align}
$$

Using the small-angle approximation ($\cos\theta\approx 1$; $\sin\theta\approx\theta$), we can write the evolution of the state vector as
$$
\begin{align}\label{eq:outer-loop-dynamics-ss}
\dot{x} =
\begin{bmatrix} \dot{x}_1 \\ \dot{x}_2 \\ \dot{x}_3 \\ \dot{x}_4 \end{bmatrix} =
\begin{bmatrix} \dot{r}_x \\ \dot{r}_y \\ \ddot{r}_x \\ \ddot{r}_y \end{bmatrix} =
\begin{bmatrix}
    x_3 \\ x_4 \\
    -\frac{u_1}{m}\left( \cos\phi\sin\theta\cos\psi + \sin\phi\sin\psi \right) \\
    -\frac{u_1}{m}\left( \cos\phi\sin\theta\sin\psi - \sin\phi\cos\psi \right)
\end{bmatrix} \approx
\begin{bmatrix}
    x_3 \\ x_4 \\
    -\frac{u_1}{m}\left( \theta\cos\psi + \phi\sin\psi \right) \\
    -\frac{u_1}{m}\left( \theta\sin\psi - \phi\cos\psi \right)
\end{bmatrix}.
\end{align}
$$

Similar as with the inner loop, we break \eqref{eq:outer-loop-dynamics-ss} into certain and uncertain components so that it is of the form of \eqref{eq:general_nonlinear_sys}.
$$
\begin{align}
\dot{x} =
\underbrace{\begin{bmatrix}
    x_3 \\ x_4 \\ 0 \\ 0
\end{bmatrix}}_{f(x)} +
\underbrace{\begin{bmatrix}
    0_{2\times 2} \\ \hdashline
    I_{2\times 2}
\end{bmatrix}}_{B(x)}
\left(
\underbrace{\left( \frac{1}{m} I_{2\times 2} \right)}_{G(x)}
\underbrace{
\begin{bmatrix}-u_1&0\\0&u_1\end{bmatrix}
\begin{bmatrix}
\cos\psi&\sin\psi \\
-\sin\psi&\cos\psi
\end{bmatrix}
}_{E(x)}
\underbrace{\begin{bmatrix}
\theta^d \\ \phi^d
\end{bmatrix}}_{u} +
\underbrace{0_{4\times 1}}_{\delta(t, x, u)}
\right).
\end{align}
$$

Again using the identity diffeomorphism $T:D\to\mathbb{R}^{n=4}, x\mapsto x$, we transform the system into regular form:
$$
\begin{align}
\eta &=
\begin{bmatrix} x_1 & x_2 \end{bmatrix}^\top =
\begin{bmatrix} r_x & r_y \end{bmatrix}^\top \in \mathbb{R}^{n-p=2} \\
\xi &=
\begin{bmatrix} x_3 & x_4 \end{bmatrix}^\top =
\begin{bmatrix} \dot{r}_x & \dot{r}_y \end{bmatrix}^\top \in \mathbb{R}^{p=2},
\end{align}
$$
which evolves as
$$
\begin{align}
\dot\eta &=
\begin{bmatrix} \dot{x}_1 \\ \dot{x}_2 \end{bmatrix} =
\begin{bmatrix} x_3 \\ x_4 \end{bmatrix} = f_a(\eta, \xi) = \xi \\
\dot\xi &=
\begin{bmatrix} \dot{x}_3 \\ \dot{x}_4 \end{bmatrix} =
\underbrace{\begin{bmatrix} 0 \\ 0 \end{bmatrix}}_{f_b(\eta,\xi)} +
\underbrace{\left( \frac{1}{m} I_{2\times 2} \right)}_{G(x)}
\underbrace{
\begin{bmatrix}-u_1&0\\0&u_1\end{bmatrix}
\begin{bmatrix}
\cos\psi&\sin\psi \\
-\sin\psi&\cos\psi
\end{bmatrix}
}_{E(x)}
\underbrace{\begin{bmatrix}
\theta^d \\ \phi^d
\end{bmatrix}}_{u} +
\underbrace{0_{4\times 1}}_{\delta(t, x, u)}
\end{align}
$$

## Theory - Stabilization

Given a system modelled in regular form, a stabilizing sliding mode controller can be designed.

> We start by selecting the sliding manifold to be $s = \xi - \phi(\eta) = 0$ so that when motion is restricted to the manifold, the reduced-order model $\dot\eta = f_a(\eta, \phi(\eta))$ has an asymptotically stable origin. The design of $\phi(\eta)$ amounts to solving a stabilization problem for the system $\dot\eta = f_a(\eta,\xi)$ with $\xi$ viewed as the control input. This stabilization problem may be solved using any other technique, such as backstepping, feedback linearzation, etc. Upon designing a stabilizing controller $\phi(\eta)$, we must design an input $u$ that steers trajectories of $\dot\xi$ to the manifold $s=0$ (i.e., steers $\xi$ to $\phi(\eta)$) in finite time and maintains it there for all future time.

Khalil, p. 564 [2]

We can achieve this by inspecting the time-evolution of the manifold $s$, which is
$$
\begin{equation}\label{eq:general-sdot}
\dot{s} = f_b(\eta,\xi) - \frac{\partial\phi}{\partial\eta} f_a(\eta,\xi) + G(x)E(x)u + \delta(t,x,u)
\end{equation}
$$

The control is given by
$$
\begin{equation}\label{eq:general-smc-control}
u = E^{-1}(x)\left\{ -L(x)\left[ f_b(\eta,\xi) - \frac{\partial\phi}{\partial\eta} f_a(\eta,\xi) \right] + v \right\}
\end{equation}
$$
where
\begin{equation}
v = -\beta(x) \text{sgn}(s).
\end{equation}

If $\hat{G}(x)$ is an approximation of the uncertain matrix $G(x)$ and we have access to it, then we let $L(x) = \hat{G}^{-1}$. Otherwise, let $L(x)=0$ and there will be no continuous component to the control---it will rely soley on the switching characteristic of $v$.

## Practice - Inner Loop Stabilization
For the sake of notation and understanding, we will first design and implement a sliding mode controller to stabilize the inner loop to the origin. With this implementation, we will be able to see how deviations from $\begin{bmatrix} r_x & \phi & \theta & \psi \end{bmatrix}^\top = \mathbf{0}$ are regulated back to $\mathbf{0}$.

The inner loop control strategy for stabilization about the origin is defined as follows:
$$
\begin{equation}
s = \xi - \phi(\eta) = \xi + K_1\eta,
\end{equation}
$$
where $K_1$ is a positive-definite diagonal matrix. Therefore, $\frac{\partial\phi}{\partial\eta} = -K_1$. Rearranging the control in equation \eqref{eq:general-smc-control}, we have the expression [1, eq 23]
$$
\begin{equation}\label{eq:smc-control}
u =  u_{eq} + E^{-1}(x)v,
\end{equation}
$$
where
$$
u_{eq} = -E^{-1}L(x)\left[ f_b(\eta,\xi) - \frac{\partial\phi}{\partial\eta}f_a(\eta,\xi) \right]
$$
and
$$
v = -\beta(x)\text{sgn}(s).
$$

Given that we have estimates of the inertial properties of the quadrotor, we let $L(x)=\hat{G}^{-1}$. Using the definitions from our inner loop modeling above and plugging them in to \eqref{eq:smc-control}, we have
$$
\begin{equation}\label{eq:inner-stabilizing-control}
u = -
\underbrace{\begin{bmatrix} -1 & 0_{1\times 3} \\ 0_{3\times 1} & I_{3\times 3} \end{bmatrix}}_{E^{-1}(x)}
\underbrace{\begin{bmatrix} \frac{m}{\cos x_2\cos x_3} & 0_{1\times 3} \\ 0_{3\times 1} & J \end{bmatrix}}_{G^{-1}(x)}
\left\{
\underbrace{\begin{bmatrix}
g \\
\frac{J_y - J_z}{J_x} x_7 x_8 \\
\frac{J_z - J_x}{J_y} x_6 x_8 \\
\frac{J_x - J_y}{J_z} x_6 x_7
\end{bmatrix}}_{f_b(\eta,\xi)}
+ K_1\xi
\right\}
+
\underbrace{\begin{bmatrix} -1 & 0_{1\times 3} \\ 0_{3\times 1} & I_{3\times 3} \end{bmatrix}}_{E^{-1}(x)} v.
\end{equation}
$$

Simplifying and breaking up into the 4 individual control channels gives
$$
\begin{align}
u_1 &= \frac{m}{\cos\phi\cos\theta}\left[ g +k_{r_z}\dot{r}_z\right] + \beta_{r_z}\text{sgn}(s_{r_z}) \label{eq:shaik-u1} \\
u_2 &= -J_x\left[ \frac{J_y - J_z}{J_x} qr + k_\phi \dot{\phi}\right] - \beta_\phi \text{sgn}(s_\phi) \\
u_3 &= -J_y\left[ \frac{J_z - J_x}{J_y} pr + k_\theta \dot{\theta}\right] - \beta_\theta \text{sgn}(s_\theta) \\
u_4 &= -J_z\left[ \frac{J_x - J_y}{J_z} pq + k_\psi \dot{\psi}\right] - \beta_\psi \text{sgn}(s_\psi). \label{eq:shaik-u4}
\end{align}
$$

By plugging these control equations into \eqref{eq:general-sdot}, it can be verified that the known components are canceled and the switching component is left behind, which causes $\dot{V} = -c(x)\left|s\right| < 0$.

    Note one difference from Shaik's SMC implementation is that the switching component is inside the bracket. This would seem to minimize the analytical upper bound necessary to prove that the control is stable in the sense of Lyapunov, but doesn't seem to make a difference in practice.

Another difference between this and Shaik's implementation is that there are sign differences. This comes from how Shaik defined the tracking error. Instead of the more common $e \triangleq x - x^d$, he defined the error as $e \triangleq x^d - x$. This causes the sign of the surface to be flipped, which causes $\text{sgn}(-s) = -\text{sgn}(s)$.

```python
class StabilizingSMC(Controller):
    """Sliding Mode Controller

    This implementation of SMC only performs stabilization
    of the inner loop. That means that there is no way
    for the inner loop to track a reference other than zero.

    This controller can be seen in action if the initial
    attitude and/or altitude state of the quadrotor is non-zero.

    Also note that this controller uses sgn(x) instead of sat(x).
    As a result of this, chattering in the control effort and in
    the state can be seen. This is addressed later in the notebook.

    Inertial quantities are deviated from the true simulation
    values so as to represent SMC's robustness to uncertainty.
    """
    def __init__(self):
        self.name = "SMC: Stabilized Inner Loop"

        # estimates of the physical properties of the quadrotor
        self.g = 9.8
        self.mass = 4
        self.I = np.diag(np.array([0.1, 0.1, 0.1]))

        # proportional control law \phi(\eta) that stabilizes the inner \dot\eta = f_a system
        self.K1 = np.diag(np.array([3,3,4,1]))

    def inner(self, state):

        def fa(eta, xi):
            return xi

        def fb(eta, xi):
            # extract omega from xi
            omega = xi[1:4]

            r = np.zeros((4,1))
            r[0,0] = self.g
            r[1:4] = -np.linalg.inv(self.I).dot(np.cross(omega, self.I.dot(omega), axis=0))
            return r

        # Transform the system into the form of HK (14.4) and (14.5)
        eta = np.array([state.flatten()[[2, 6, 7, 8]]]).T # r_z, ph, th, ps
        xi = np.array([state.flatten()[[5, 9, 10, 11]]]).T # \dot{r_z}, p, q, r

        ph = eta.flatten()[1]
        th = eta.flatten()[2]
        ps = eta.flatten()[3]

        # Build the inv(E(x)) matrix
        Einv = np.diag(np.array([-1,1,1,1]))

        # Build the L(x) = inv(G(x)) matrix
        L = np.eye(4)
        L[0, 0] = self.mass/(np.cos(ph)*np.cos(th))
        L[1:4, 1:4] = self.I

        # continuous control to cancel known terms: equation (18) of [1].
        ueq = -Einv.dot(L.dot(fb(eta, xi) + self.K1.dot(fa(eta, xi))))

        # Build the matrix of sliding surfaces
        SS = xi + self.K1.dot(eta)

        # Gains
        Beta = np.diag(np.array([6,1,7,2]))

        # switching components of the control: equation (20) of [1]
        # (but with a boundary on the sliding surface to mitigate chattering)
        gamma = -Beta.dot(np.sign(SS))

        # equation (23) of [1]
        u = ueq + Einv.dot(gamma)

        return u

    def update(self, commanded, state, pkt, Ts):

        #
        # Inner Loop
        #

        u = self.inner(state)

        # actuator commands
        return u, commanded
```

```python
quad = Quadrotor(r=np.array([[0],[0],[5]]),
                 v=np.array([[0],[0],[0]]),
               Phi=np.array([[0.1],[-0.8],[3.15]]))
sim = Simulator(quad=quad, ctrl=StabilizingSMC())
sim.run(15, Ts=0.01)
sim.plot()
```

## Aside - Chattering
The discontinuous switching component $\text{sgn}(s)$ is used to maintain trajectories on the surface $s=0$. However, because there is uncertainty in every system, $s$ is typically always varying about $0$, causing the control to rapidly switch signs so that $s=0$ can be maintained. This creates an undesired effect known as chattering, which can be seen in the control effort from the inner loop stabilization controller above. To alleviate this, a margin is created around the surface $s=0$ so that the sensitivity is reduced and as long as trajectories are in the *neighborhood* of $s=0$ the control is successful. This can be implemented by replacing the signum function $\text{sgn}(s)$ with a saturation function, as shown below. The transition region can be controlled by scaling the surface by a variable $\epsilon$, such as $\text{sat}(\frac{s}{\epsilon})$, where $0.1 \le \epsilon \le 1$ is a typical range.

A vectorized saturation function is implemented using `numpy` below:

```python
def sat(v):
    v = np.copy(v)
    v[np.abs(v) > 1] = np.sign(v[np.abs(v) > 1])
    return v
```

## Theory - Tracking

In general, tracking a trajectory amounts to stabilizing the error between the current and the desired to zero. To this end, we rewrite the state-space representation as an error-state representation. Define the error as $\tilde{x_i} \triangleq x_i - x_i^d$, where $x_i^d$ is the desired reference value of $x_i$. The sliding surface must also be rewritten in terms of error. The vector of all reference signals is defined as $y$.

In this error representation, it is assumed that $y \in C^2$ and that $y$, $\dot y$, and $\ddot y$ are all available online. Given an arbitrary twice-differentiable signal, its derivative may numerically be computed using a [*dirty derivative*](utils.ipynb#Derivative-Filter).

## Practice - Tracking

We will redefine the state-space systems of the inner and outer loops to use the error-state representation.

We start with the regular form of the inner loop, as written in equation \eqref{eq:inner-regular-transform}. Expressing as an error state, we have
$$
\begin{align}
\tilde\eta &= \eta - y =
\begin{bmatrix} \tilde{x}_1 & \tilde{x}_2 & \tilde{x}_3 & \tilde{x}_4 \end{bmatrix}^\top =
\begin{bmatrix} r_z-r_z^d & \phi-\phi^d & \theta-\theta^d & \psi-\psi^d \end{bmatrix}^\top \\
\tilde\xi &= \xi - \dot y =
\begin{bmatrix} \tilde{x}_5 & \tilde{x}_6 & \tilde{x}_7 & \tilde{x}_8 \end{bmatrix}^\top =
\begin{bmatrix} \dot{r}_z-\dot{r}_z^d & p-\dot\phi^d & q-\dot\theta^d & r-\dot\psi^d \end{bmatrix}^\top \\
y &= \begin{bmatrix} r_z^d & \phi^d & \theta^d & \psi^d \end{bmatrix}^\top
\end{align}
$$

which evolves as

$$
\begin{align}
\dot{\tilde\eta} &=
\begin{bmatrix} \dot{\tilde x}_1 \\ \dot{\tilde x}_2 \\ \dot{\tilde x}_3 \\ \dot{\tilde x}_4 \end{bmatrix} =
\begin{bmatrix} \tilde x_5 \\ \tilde x_6 \\ \tilde x_7 \\ \tilde x_8 \end{bmatrix} = f_a(\tilde\eta, \tilde\xi) = \tilde\xi \\
\dot{\tilde\xi} &=
\begin{bmatrix} \dot{\tilde x}_5 \\ \dot{\tilde x}_6 \\ \dot{\tilde x}_7 \\ \dot{\tilde x}_8 \end{bmatrix} =
\underbrace{\begin{bmatrix}
    g \\
    \frac{J_y - J_z}{J_x} qr \\
    \frac{J_z - J_x}{J_y} pr \\
    \frac{J_x - J_y}{J_z} pq
\end{bmatrix}}_{f_b(\eta,\xi)} -
\underbrace{\begin{bmatrix}
    \ddot{r}_z^d \\
    \ddot{\phi}^d \\
    \ddot{\theta}^d \\
    \ddot{\psi}^d
\end{bmatrix}}_{\ddot y} +
\underbrace{\begin{bmatrix}
\frac{\cos \phi\cos \theta}{m} & 0_{1\times 3} \\
0_{3\times 1} & J^{-1}
\end{bmatrix}}_{G(x)}
\underbrace{\begin{bmatrix}
-1 & 0_{1\times 3} \\
0_{3\times 1} & I_{3\times 3}
\end{bmatrix}}_{E(x)}
\underbrace{\begin{bmatrix}
u_1 \\ u_2 \\ u_3 \\ u_4
\end{bmatrix}}_{u}.
\end{align}
$$

Therefore, the new tracking control is very similar to the stabilizing control \eqref{eq:inner-stabilizing-control}, but with the subtraction of the reference signals $y$ and their first and second derivatives:
$$
\begin{equation}
u = -
\underbrace{\begin{bmatrix} -1 & 0_{1\times 3} \\ 0_{3\times 1} & I_{3\times 3} \end{bmatrix}}_{E^{-1}(x)}
\underbrace{\begin{bmatrix} \frac{m}{\cos \phi\cos \theta} & 0_{1\times 3} \\ 0_{3\times 1} & J \end{bmatrix}}_{G^{-1}(x)}
\left\{
\underbrace{\begin{bmatrix}
g \\
\frac{J_y - J_z}{J_x} qr \\
\frac{J_z - J_x}{J_y} pr \\
\frac{J_x - J_y}{J_z} pq
\end{bmatrix}}_{f_b(\eta,\xi)} -
\underbrace{\begin{bmatrix}
    \ddot{r}_z^d \\
    \ddot{\phi}^d \\
    \ddot{\theta}^d \\
    \ddot{\psi}^d
\end{bmatrix}}_{\ddot y}
+ K_1(\xi-\dot y)
\right\}
+
\underbrace{\begin{bmatrix} -1 & 0_{1\times 3} \\ 0_{3\times 1} & I_{3\times 3} \end{bmatrix}}_{E^{-1}(x)} v,
\end{equation}
$$
where
$$
\begin{equation}
v = -\beta(x)\text{sgn}(\tilde s)
\end{equation}
$$
and the sliding surface is redefined to be
$$
\begin{equation}
\tilde s = \tilde\xi + K_1\tilde\eta = (\xi-\dot y) + K_1(\eta-y).
\end{equation}
$$

Following the same pattern as was done for the inner loop, the outer loop is rewritten in an error-state representation, with the reference signal defined as
$$
\begin{equation}
y = \begin{bmatrix} r_x^d & r_y^d \end{bmatrix}^\top.
\end{equation}
$$

The regular form of the error state,
$$
\begin{align}
\dot{\tilde\eta} &=
\begin{bmatrix} \dot{\tilde x}_1 \\ \dot{\tilde x}_2 \end{bmatrix} =
\begin{bmatrix} \tilde x_3 \\ \tilde x_4 \end{bmatrix} = f_a(\tilde\eta, \tilde\xi) = \tilde\xi \\
\dot{\tilde\xi} &=
\begin{bmatrix} \dot{\tilde x}_3 \\ \dot{\tilde x}_4 \end{bmatrix} =
\underbrace{\begin{bmatrix} 0 \\ 0 \end{bmatrix}}_{f_b(\eta,\xi)} -
\underbrace{\begin{bmatrix} \ddot{r}_x^d \\ \ddot{r}_y^d \end{bmatrix}}_{\ddot y} +
\underbrace{\left( \frac{1}{m} I_{2\times 2} \right)}_{G(x)}
\underbrace{
\begin{bmatrix}-u_1&0\\0&u_1\end{bmatrix}
\begin{bmatrix} \cos\psi&\sin\psi \\ -\sin\psi&\cos\psi \end{bmatrix}
}_{E(x)}
\underbrace{\begin{bmatrix}
\theta^d \\ \phi^d
\end{bmatrix}}_{u},
\end{align}
$$
leads to the tracking control
$$
\begin{equation}
u = -
\underbrace{
\begin{bmatrix} \cos\psi&-\sin\psi \\ \sin\psi&\cos\psi \end{bmatrix}
\begin{bmatrix}-\frac{1}{u_1}&0\\0&\frac{1}{u_1}\end{bmatrix}
}_{E^{-1}(x)}
\underbrace{\begin{bmatrix} m&0\\0&m \end{bmatrix}}_{G^{-1}(x)}
\left\{
\underbrace{\begin{bmatrix} 0\\0 \end{bmatrix}}_{f_b(\eta,\xi)} -
\underbrace{\begin{bmatrix} \ddot{r}_x^d \\ \ddot{r}_y^d \end{bmatrix}}_{\ddot y}
+ K_2(\xi-\dot y)
\right\}
+ E^{-1}(x) v,
\end{equation}
$$
where
$$
\begin{equation}
v = -\beta(x)\text{sgn}(\tilde s)
\end{equation}
$$
and the sliding surface is redefined to be
$$
\begin{equation}
\tilde s = \tilde\xi + K_2\tilde\eta = (\xi-\dot y) + K_2(\eta-y).
\end{equation}
$$

```python
class SMC(Controller):
    """Sliding Mode Controller

    This implementation of SMC includes the ability
    to track reference signals, as opposed to the
    StabilizingSMC class above. In addition, this
    class utilizes a saturation function instead
    of the naive signum function for the switching
    component of the controller.
    """
    def __init__(self):
        self.name = "SMC"

        # dirty-derivative filters
        self.dyidt = DirtyDerivative(order=1, tau=0.0125)
        self.dyidt2 = DirtyDerivative(order=2, tau=0.025)
        self.dyodt = DirtyDerivative(order=1, tau=0.025)
        self.dyodt2 = DirtyDerivative(order=2, tau=0.05)

        # Value of total thrust from last iteration
        self.u1_d1 = 0

        # estimates of the physical properties of the quadrotor
        self.g = 9.81
        self.mass = 3.81
        self.I = np.diag(np.array([0.1, 0.1, 0.1]))

        # proportional control law \phi(\eta) that stabilizes the inner \dot\eta = f_a system
        self.K1 = np.diag(np.array([2,5,5,1]))

        # proportional control law \phi(\eta) that stabilizes the outer \dot\eta = f_a system
        self.K2 = np.diag(np.array([1,1]))

    def inner(self, desired, state, Ts):

        def fa(eta, xi):
            return xi

        def fb(eta, xi):
            # extract omega from xi
            omega = xi[1:4]

            r = np.zeros((4,1))
            r[0,0] = self.g
            r[1:4] = -np.linalg.inv(self.I).dot(np.cross(omega, self.I.dot(omega), axis=0))
            return r

        # Create the reference signal vector
        y = np.atleast_2d(desired).T # make a col vector
        ydot = self.dyidt.update(y, Ts)
        yddot = self.dyidt2.update(ydot, Ts)

        # Transform the system into the form of HK (14.4) and (14.5)
        eta = np.array([state.flatten()[[2, 6, 7, 8]]]).T # r_z, ph, th, ps
        xi = np.array([state.flatten()[[5, 9, 10, 11]]]).T # \dot{r_z}, p, q, r

        # for convenience
        ph = eta.flatten()[1]
        th = eta.flatten()[2]
        ps = eta.flatten()[3]

        # Build the inv(E(x)) matrix
        Einv = np.diag(np.array([-1,1,1,1]))

        # Build the L(x) = inv(G(x)) matrix
        L = np.eye(4)
        L[0, 0] = self.mass/(np.cos(ph)*np.cos(th))
        L[1:4, 1:4] = self.I

        # continuous control to cancel known terms: equation (18) of [1].
        ueq = -Einv.dot(L.dot( (fb(eta, xi) - yddot) + self.K1.dot( fa(eta-y, xi-ydot) )))

        # Build the matrix of sliding surfaces
        SS = (xi-ydot) + self.K1.dot(eta-y)

        # Gains
        Beta = np.diag(np.array([11,5,5,2]))

        # switching components of the control: equation (20) of [1]
        # (but with a boundary on the sliding surface to mitigate chattering)
        gamma = -Beta.dot(sat( SS/1 ))

        # equation (23) of [1]
        u = ueq + Einv.dot(gamma)

        return u

    def outer(self, desired, u1, state, Ts):

        # Make sure the thrust is non-zero
        if u1 == 0:
            return np.array([0,0])

        def fa(eta, xi):
            return xi

        def fb(eta, xi):
            return np.zeros((2,1))

        # Create the reference signal vector
        y = np.atleast_2d(desired).T # make a col vector
        ydot = self.dyodt.update(y, Ts)
        yddot = self.dyodt2.update(ydot, Ts)

        # Transform the system into the form of HK (14.4) and (14.5)
        eta = np.array([state.flatten()[[0, 1]]]).T # r_x, r_y
        xi = np.array([state.flatten()[[3, 4]]]).T # \dot{r}_x, \dot{r}_y

        ps = state.flatten()[8]

        # Build the inv(E(x)) matrix
        R = np.array([
            [ np.cos(ps), np.sin(ps)],
            [-np.sin(ps), np.cos(ps)],
        ])
        Einv = R.T.dot(np.diag(np.array([-1.0/u1,1.0/u1])))

        # Build the L(x) = inv(G(x)) matrix
        L = self.mass*np.eye(2)

        # continuous control to cancel known terms: equation (18) of [1].
        ueq = -Einv.dot(L.dot( (fb(eta, xi) - yddot) + self.K2.dot( fa(eta-y, xi-ydot) )))

        # Build the matrix of sliding surfaces
        SS = (xi-ydot) + self.K2.dot(eta-y)

        # Gains
        Beta = np.diag(np.array([15,15]))

        # switching components of the control: equation (20) of [1]
        # (but with a boundary on the sliding surface to mitigate chattering)
        gamma = -Beta.dot(sat(SS/1))

        # equation (23) of [1]
        u = ueq + Einv.dot(gamma)

        return u

    def update(self, commanded, state, pkt, Ts):

        #
        # Outer Loop
        #

        ref = self.outer(commanded.flatten()[[0, 1]], self.u1_d1, state, Ts)

        #
        # Inner Loop
        #

        # build reference signal for inner loop
        rz_ref = commanded.flatten()[2]
        ph_ref = ref.flatten()[1]
        th_ref = ref.flatten()[0]
        ps_ref = commanded.flatten()[8]
        desired = np.array([rz_ref, ph_ref, th_ref, ps_ref])

        u = self.inner(desired, state, Ts)

        # Save total thrust for the outer loop
        self.u1_d1 = u.flatten()[0]

        # update the commanded states
        commanded[6] = ph_ref
        commanded[7] = th_ref

        # actuator commands
        return u, commanded
```

```python
# Instantiate a quadrotor model with the given initial conditions
quad = Quadrotor(r=np.array([[0],[0],[-10]]),
                 v=np.array([[0],[0],[0]]),
               Phi=np.array([[-0.1],[0],[0]]))

# Instantiate a tracking sliding mode controller
ctrl = SMC()

# Setup a setpoint commander
cmdr = Commander(default=True)
cmdr.attitude(np.array([0,0,np.pi/2]))

def set_position(i, Ts):
    f = 0.25
    x = np.cos(2*np.pi*f*i*Ts)
    return np.array([x, 2, -1])

cmdr.position(set_position)

# Run the simulation
sim = Simulator(quad, ctrl, cmdr=cmdr)
sim.run(20, Ts=0.01)
sim.plot()
```

## Conclusion

Sliding mode control is a fairly straightforward controller to implement with low computational demands (i.e., no optimization---it is all just linear algebra). Although there are various ways to approach the design of a sliding mode controller, the principle is the same:

1. Identify a sliding surface such as $s = \xi - \phi(\eta) = 0$ that you can drive system trajectories to. Once the system is on this manifold, it *slides* to the origin using the control law $\phi(\eta)$, which is typically designed to be linear (i.e., $\phi(\eta) = -K\eta$) but could be anything.

1. Choose $u$ to drive all system trajectories to the manifold $s=0$. While Khalil gives a general control form in equation \eqref{eq:general-smc-control}, the fundamental principle is Lyapunov stabiliy analysis. The control $u$ must be designed such that $\dot{V} = s\dot{s} < 0$. The control $u$ is often chosen with a continuous part that cancels any known terms from the model, and a discontinuous switching component $\beta(x)\text{sgn}(s)$ that guarantees a that the Lyapunov function is negative-definite in the direction of the system's trajectories.

1. Because $\text{sgn}(s)$ causes chattering, use $\text{sat}(\frac{s}{\epsilon})$ instead, where $\epsilon$ is typically small and can be tuned to change the transition region around the manifold $s=0$.

Other thoughts:
- Ensure that your derivatives are being computed corectly! For an $n$th-order derivative, make sure to wait until you have seen $n$ measurements.

## References

[1] A. L’Afflitto, R. Anderson, and K. Mohammadi, “An Introduction to Nonlinear Robust Control for Unmanned Quadrotor Aircraft,” IEEE Control Syst. Mag., vol. 38, no. June, pp. 102–121, 2018.

[2] H. K. Khalil, Nonlinear Systems. Princeton, NJ: Prentice Hall, 2002.

[3] M. K. Shaik and J. F. Whidborne, “Robust Sliding Mode Control of a Quadrotor,” 11th Int. Conf. Control, pp. 1–5, 2016.


# Gefitinib PK Analysis

Run population anaysis of Gefitinib PK data. First we reproduce the steps performed in the publication by Eigenmann et. al.. The next step is to challenge the modelling assumptions.

## 1. Reproduce PK Analysis

### 1.1 Import data

```python
import os
import pandas as pd

# Import data
path = os.getcwd()
data_raw = pd.read_csv(path + '/data/PK_LXF_gefi.csv', sep=';')

# Filter relevant information
data = data_raw[['#ID', 'TIME', 'Y', 'DOSE GROUP', 'DOSE', 'BW']]

# Convert TIME and Y to numeric values (currently strings)
data['TIME'] = pd.to_numeric(data['TIME'], errors='coerce')
data['Y'] = pd.to_numeric(data['Y'], errors='coerce')

# Sort TIME values (for plotting convenience)
data.sort_values(by='TIME', inplace=True)

# Filter NaNs
data = data[data['Y'].notnull()]

# Show data
data
```

      #ID
      TIME
      Y
      DOSE GROUP
      DOSE
      BW

      31
      25
      9.991667
      87.600
      100.00
      .
      24.1

      130
      149
      9.992361
      3.960
      25.00
      .
      23.4

      175
      99
      9.996528
      0.488
      6.25
      .
      23.0

      49
      124
      10.010417
      2000.000
      100.00
      .
      25.2

      202
      156
      10.010417
      78.000
      6.25
      .
      30.4

      76
      41
      10.010417
      468.000
      25.00
      .
      24.1

      13
      22
      10.020833
      3190.000
      100.00
      .
      24.4

      139
      164
      10.020833
      327.000
      25.00
      .
      24.6

      166
      75
      10.022222
      53.500
      6.25
      .
      24.6

      193
      138
      10.040278
      97.500
      6.25
      .
      27.5

      94
      68
      10.040278
      461.000
      25.00
      .
      30.4

      67
      157
      10.044444
      2900.000
      100.00
      .
      24.0

      121
      133
      10.102778
      514.000
      25.00
      .
      22.8

      211
      165
      10.103472
      142.000
      6.25
      .
      24.6

      4
      15
      10.104861
      2180.000
      100.00
      .
      25.9

      148
      4
      10.205556
      83.900
      6.25
      .
      29.3

      85
      62
      10.206250
      272.000
      25.00
      .
      24.9

      22
      24
      10.206944
      1870.000
      100.00
      .
      26.9

      184
      125
      10.409722
      8.030
      6.25
      .
      31.5

      58
      132
      10.413889
      1090.000
      100.00
      .
      21.9

      103
      117
      10.413889
      193.000
      25.00
      .
      27.9

      112
      130
      10.997222
      9.980
      25.00
      .
      19.5

      157
      42
      10.997222
      0.488
      6.25
      .
      24.7

      41
      78
      11.000000
      31.400
      100.00
      .
      20.2

      25
      24
      15.994444
      412.000
      100.00
      .
      25.2

      88
      62
      15.994444
      2.100
      25.00
      .
      25.1

      160
      42
      15.995139
      2.830
      6.25
      .
      26.2

      206
      156
      16.009722
      45.400
      6.25
      .
      29.3

      98
      68
      16.010417
      273.000
      25.00
      .
      30.3

      71
      157
      16.011111
      2450.000
      100.00
      .
      23.0

      197
      138
      16.021528
      27.900
      6.25
      .
      26.9

      107
      117
      16.022222
      437.000
      25.00
      .
      28.3

      53
      124
      16.022222
      2610.000
      100.00
      .
      25.0

      125
      133
      16.040972
      420.000
      25.00
      .
      23.0

      215
      165
      16.040972
      37.400
      6.25
      .
      24.4

      8
      15
      16.041667
      2280.000
      100.00
      .
      21.1

      170
      75
      16.100694
      81.800
      6.25
      .
      24.2

      116
      130
      16.102083
      440.000
      25.00
      .
      20.4

      62
      132
      16.102778
      2390.000
      100.00
      .
      24.0

      44
      78
      16.210417
      1510.000
      100.00
      .
      19.2

      143
      164
      16.212500
      209.000
      25.00
      .
      24.5

      188
      125
      16.214583
      35.000
      6.25
      .
      30.4

      152
      4
      16.415278
      24.000
      6.25
      .
      27.6

      80
      41
      16.415278
      239.000
      25.00
      .
      23.3

      17
      22
      16.418056
      2140.000
      100.00
      .
      21.6

      35
      25
      16.993056
      100.000
      100.00
      .
      23.4

      179
      99
      16.994444
      0.488
      6.25
      .
      23.7

      134
      149
      16.994444
      8.560
      25.00
      .
      23.6

### 1.2 Sort data into dosing groups

Each dose group has been dosed daily from day 2 to 16. PK samples have been taken on day 10 and 16.

```python
# Get group identifiers
groups = data['DOSE GROUP'].unique()

# Sort into groups
data_one = data[data['DOSE GROUP'] == groups[0]]
data_two = data[data['DOSE GROUP'] == groups[1]]
data_three = data[data['DOSE GROUP'] == groups[2]]

# Show different dose groups
groups
```

    array([100.  ,  25.  ,   6.25])

### 1.3 Visualise dose group one

```python
import matplotlib.pyplot as plt

# Display dose group
print(groups[0])

# Get unique animal IDs
ids = data_one['#ID'].unique()

# Plot measurements
fig = plt.figure(figsize=(12, 6))

for i in ids:
    # Mask for individual
    mask = data_one['#ID'] == i
    time = data_one[mask]['TIME']
    volume = data_one[mask]['Y']

    # Filter out Nan values
    mask = volume.notnull()
    time = time[mask]
    volume = volume[mask]

    # Create semi log plot
    plt.scatter(time, volume, edgecolor='black')

# Set y axis to logscale
plt.yscale('log')

# Label axes
plt.xlabel('Time in [day]')
plt.ylabel('Plasma concentration in [mg/L]')

plt.show()
```

### 1.4 Visualise dose group two

```python
import matplotlib.pyplot as plt

# Display dose group
print(groups[1])

# Get unique animal IDs
ids = data_two['#ID'].unique()

# Plot measurements
fig = plt.figure(figsize=(12, 6))

for i in ids:
    # Mask for individual
    mask = data_two['#ID'] == i
    time = data_two[mask]['TIME']
    volume = data_two[mask]['Y']

    # Filter out Nan values
    mask = volume.notnull()
    time = time[mask]
    volume = volume[mask]

    # Create semi log plot
    plt.scatter(time, volume, edgecolor='black')

# Set y axis to logscale
plt.yscale('log')

# Label axes
plt.xlabel('Time in [day]')
plt.ylabel('Plasma concentration in [mg/L]')

plt.show()
```

### 1.5 Visualise dose group three

```python
import matplotlib.pyplot as plt

# Display dose group
print(groups[2])

# Get unique animal IDs
ids = data_three['#ID'].unique()

# Plot measurements
fig = plt.figure(figsize=(12, 6))

for i in ids:
    # Mask for individual
    mask = data_three['#ID'] == i
    time = data_three[mask]['TIME']
    volume = data_three[mask]['Y']

    # Filter out Nan values
    mask = volume.notnull()
    time = time[mask]
    volume = volume[mask]

    # Create semi log plot
    plt.scatter(time, volume, edgecolor='black')

# Set y axis to logscale
plt.yscale('log')

# Label axes
plt.xlabel('Time in [day]')
plt.ylabel('Plasma concentration in [mg/L]')

plt.show()
```

### 1.2 Build Structural Model (pints.ForwardModel)

#### 1.2.1 Build Myokit Model

```python
import myokit

from pkpd import model as m

# Build 1 compartmental PK model with default parameters
model = m.create_one_comp_pk_model()

# Validate model
model.validate()

# Check units
model.check_units(mode=myokit.UNIT_TOLERANT)

# Print model
print(model.code())
```

    [[model]]
    # Initial values
    central.amount = 0

    [central]
    dot(amount) = -k_e * amount
        in [mg]
    conc = amount / volume
        in [mg/L]
    k_e = 0
        in [1/day]
    time = 0 bind time
        in [day]
    volume = 1
        in [L]

#### 1.2.2 Set oral administration

```python
import myokit

class DosingRegimen(object):
    def __init__(self):
        self._amount = None
        self._duration = None
        self._periodicity = 0
        self._multiplier = 0
        self._dosing_regimen = None
        self._indirect_admin = False
        self._dosing_compartment = 'central'

    def __call__(self, model, amount, duration=None, periodicity=0, multiplier=0):
        """
        Returns the myokit.Model with appropriate structural adjustments for dosing,
        and a myokit.Protocol with the specified schedule.
        """
        if

    def set_amdin(indirect=False):
        self.indirect_admin = indirect

def create_dosing_regimen(model, compartment, administration, amount, duration=None, periodicity=0, multiplier=0):
    """
    Returns a myokit.Protocol object with the specified dosing regimen, and alters the structural model,
    by addition of bolus injection or dosing compartment.

    model -- myokit.Model
    compartment -- compartment with bolus injection or connection to dosing compartment
    administration -- type of administration: Injection into compartment or dosing compartment
    amount -- Applied dose
    duration -- How long did it take to apply the dose, if None duration is set to the smallest numerically stable duration,
                i.e. such that duration > 1e-6 and amount / duration < some upper value.
    periodicity -- In which intervals is dose applied. If 0 dose is applied once.
    multplier -- How often is dose applied? If 0, it's applied once for non-period and indefinitely for periodic schedules.
    """
    if not isinstance(model, myokit.Model):
        raise ValueError
    if not model.has_component(compartment):
        raise ValueError
    if administration != 'dosing compartment':
        raise NotImplementedError

    # Set dosing compartment
    dose_comp = model.get(compartment)
    if administration == 'dosing compartment':
        dose_comp = model.add_component('dose')

    # Add dose rate and regimen to dose compartment
    dose_rate = dose_comp.add_variable('dose_rate')
    regimen = dose_comp.add_variable('regimen')

    # define dosing regimen
    duration = 0.001  # [1 / d] how long does it take for dose to be in dosing compartment?
    dose_rate.set_rhs(25 * 0.2 / duration)  # 100 mg / kg / d let's estimate weight of mouse 0.2 kg
    dosing_regimen = myokit.Protocol()
    dosing_regimen.schedule(level=1, start=0, duration=duration)
```

#### 1.2.3 Build pints model

```python
# Build pints model
pass
```

### 1.3 Build population model

This is the Hierarchical Log Prior of the model

\begin{align}
\text{p}(\psi | \theta )\text{p}(\theta )
\end{align}

Should take a number of means and variances to construct $\text{p}(\theta )$ (Gaussian for means of $\text{p}(\psi | \theta )$ and half cauchy for variances of $\text{p}(\psi | \theta )$.

Think about whether this can be generalised for any structure.

Potentially best way: build a Hierarchical Log Posterior by combining a Likelihood and prior similar to PosteriorLogLikelihood, but appending the input params of the prior instead to the model params (this is the p(y|psi)p(psi|theta) bit). You have to provide posteriors for the population level paramers.

In that way a posterior can be constructed by passing any problem, any population distribution and any priors of the population distirbution parameters!

```python
# Build hierarchical model
pass
```

### 1.4 Build error model

This can be done by just using Loglikelihood from pints

```python
pass
```

### Run inference

```python
pass
```

### Visualise prediction

```python

```


```python
# 그래프, 수학 기능 추가
# Add graph and math features
import pylab as py
import numpy as np
import numpy.linalg as nl
# 기호 연산 기능 추가
# Add symbolic operation capability
import sympy as sy

```

```python
sy.init_printing()

```

# 상미분방정식을 위한 전진 오일러법
Forward Euler Method for Ordinary Differntial Equation

[](https://www.youtube.com/watch?v=q87L9R9v274)

## 여러 $(t, x)$ 지점에서의 기울기
Slopes at $(t, x)$ points

다시 한번 주어진 미분 방정식을 생각해 보자.
Let's think about the first order differential equation again.

$$
\left\{
    \begin{align}
        a_0 \frac{d}{dt}x(t)+a_1 x(t)&=0 \\
        x(0)&=x_0 \\
    \end{align}
\right.
$$

미분항을 남기고 나머지를 등호의 오른쪽으로 옮겨 보자.
Let's move terms except the differential to the right side of the equal sign.

$$
a_0 \frac{d}{dt}x(t)=-a_1 x(t)
$$

양변을 $a_0$로 나누어 보자.
Let's divide both sides with $a_0$.

$$
\frac{d}{dt}x(t)=-\frac{a_1}{a_0} x(t)
$$

이 식의 의미를 한번 생각해 보자.
Let's think about the meaning of this equation.

위 미분방정식을 만족시키는 어떤 함수 $x(t)$의 $t=t_i$, $x=x_j$ 점에서의 $t$에 대한 기울기는 다음과 같을 것이라는 의미이다.

This equation indicates that a function $x(t)$ satisfying the differential equation above would have a slope as follows at a point of $t=t_i$ and $x=x_j$.

$$
\left.\frac{d}{dt}x\right|_{\left(t, x\right)=\left(t_i,x_j\right)}=-\frac{a_1}{a_0} x_j
$$

이런식으로 $t$의 변화에 따른 $x$의 기울기 $\frac{d}{dt}x(t)$ 를 모든 $(t, x)$ 점에서 구할 수 있다.

In this way, we can find all the $\frac{d}{dx}x(t)$, slopes of $x$ with respect to the change of $t$ at all $(t, x)$ points.

## 방향장 기울기 시각화
Visualizing the slopes in the [Direction Field](https://en.wikipedia.org/wiki/Direction_field)

다음 예를 생각해 보자.

Let's think about an example as follows.

```python
a_0, a_1 = 2.0, 1.0

```

$$
\left\{
    \begin{align}
        2 \frac{d}{dt}x(t)+ x(t)&=0 \\
        x(0)&=x_0 \\
    \end{align}
\right.
$$

기울기를 계산하는 파이썬 함수를 생각해 보자.
Let's think about a python function calculating the slope.

```python
def dx_dt(x, t):
    return - a_1 * x / a_0

```

예를 들어 $0 \le t \le 10$, $-6 \le x \le 6$ 인 영역에서 기울기를 그려 보자.

Let's plot slopes within the region of $0 \le t \le 10$ and $-6 \le x \le 6$.

```python
t_array = py.linspace(0, 10)
x_array = py.linspace(-6, 6)

```

```python
def ode_slope_1state_interval(func, delta_t, delta_x, te, ti, x_max, x_min):
    time_list = np.arange(ti, te, delta_t)
    x_list = np.arange(x_min, x_max + 0.5 * delta_x, delta_x)
    ode_slope_1state(func, x_list, time_list)
    return time_list

def ode_slope_1state(func, x_list, time_list):
    """
    Plot field of arrows indicating derivatives of the state
    :param func:
    :param x_list:
    :param time_list:
    :return:
    """
    time_mesh, x_mesh = np.meshgrid(time_list, x_list)
    u_mesh = np.ones_like(x_mesh)
    v_mesh = func(x_mesh, time_mesh)
    # magnitude as color
    color_mesh = np.sqrt(u_mesh * u_mesh + v_mesh * v_mesh)

    # https://stackoverflow.com/questions/29589119/plot-width-settings-in-ipython-notebook
    py.figure(figsize=(12, 12))
    py.quiver(time_mesh, x_mesh, u_mesh, v_mesh, color_mesh, angles='xy')
    py.xlabel('t')
    py.ylabel('x')
    py.xlim((time_list[0] - (time_list[1] - time_list[0]) * 0.125, time_list[-1]))
    py.ylim((min(x_list) - (x_list[1] - x_list[0]) * 0.125,
                max(x_list) + (x_list[-1] - x_list[-2]) * 0.125))
    py.grid(True)

ode_slope_1state(dx_dt, x_array, t_array)
py.savefig('slopes_t_x.png', dpi=300)

```

 위와 같이 모든 점에서의 기울기의 그림을 **방향장** 또는 **기울기장** 이라고 한다.

 As above, a plot of the slopes of all points is called a [**Direction Field**](https://en.wikipedia.org/wiki/Direction_field) or **Slope Field**

엄밀해를 겹쳐 그려 보자
Let's overlap the curve of the exact solution.

$$x(t)=x_0 e^{-\frac{a_1}{a_0} t}$$

```python
x_0 = 4.5

def exact(t):
    return x_0 * py.exp((-a_1 / a_0) * t)

x_exact_array = exact(t_array)

ode_slope_1state(dx_dt, x_array, t_array)
py.plot(t_array, x_exact_array, label='exact')

py.legend(loc=0, fontsize='xx-large')

py.savefig('slopes_t_x.svg')

```

$t=0$에서의 $x$의 초기값에 따라 엄밀해가 달라질 수 있음을 기억하자.
Let's remember that the exact solution may vary depending on the initial value of $x$ at $t=0$.

이것을 이용해서 미분방정식의 해를 구해볼 수 있을까?
Using this, can we find a solution of a differential equation?

## 전진 오일러법
Forward Euler Method

$\left(t,x\right) = \left(0, x_0\right)$ 에서의 $x(t)$의 기울기를 생각해 보자.

Let's think about the slope of $x(t)$ at $\left(t,x\right) = \left(0, x_0\right)$.

$$
\left.\frac{d}{dt}x\right|_{\left(t, x\right)=\left(0,x_0\right)}=-\frac{a_1}{a_0} x_0=s_0
$$

그렇다면, $(0, x_0)$점을 지나고 기울기가 $s_0=-\frac{a_1}{a_0} x_0$ 인 직선을 생각할 수 있다.

Then, we can think about a line passing through $(0, x_0)$ with the slope of $s_0=-\frac{a_1}{a_0} x_0$.

$$
x=-\frac{a_1}{a_0} x_0 \left(t -  0 \right) + x_0=s_0\left(t -  0 \right) + x_0
$$

이 직선을 따라 $t_1=t_0 + \Delta t = 0 + 0.5$ 까지 *전진*해 보자.

Following this line, let's move *forward* to $t_1=t_0 + \Delta t = 0 + 0.5$.

```python
# An initial point
t_0 = 0
x_0 = 4.5
# time step
delta_t = 0.5

# Indicate the initial point
py.plot(t_0, x_0, 'o')
py.text(t_0, x_0, '$(t_0, x_0)$')

# Slope at the initial point
s_0 = dx_dt(x_0, t_0)

# Straight line to next time step
t_0_array = py.linspace(0, delta_t)
x_0_array = s_0 * (t_0_array - t_0) + x_0

# (t_1, x_1) point
t_1, x_1 = t_0_array[-1], x_0_array[-1]

py.plot(t_0_array, x_0_array, label='$x=s_0(t-0)+x_0$')
py.plot(t_1, x_1, 'o')
py.text(t_1, x_1, '$(t_1, x_1)$')

py.axis('equal')

py.xlim(left=-4, right=4)
py.ylim(bottom=0, top=6)

py.xlabel('t(sec)')
py.ylabel('x(m)')

py.legend(loc=0)
py.grid(True)

```

$(t_1, x_1)$ 에서의 기울기 $s_1$ 과 그러한 기울기를 가지고 $\left(t_1, x_1\right)$ 을 지나는 직선은 다음과 같다.

The slope $s_1$ at $\left(t_1, x_1\right)$ and another line with such slope and passing $\left(t_1, x_1\right)$ would be as follows.

$$
\begin{align}
\left.\frac{d}{dt}x\right|_{\left(t, x\right) = \left(t_1,x_1\right)} &=-\frac{a_1}{a_0} x_1=s_1 \\
x & = s_1\left(t - t_1 \right) + x_1
\end{align}
$$

```python
# Indicate the initial point
py.plot(t_0, x_0, 'o')
py.text(t_0, x_0, '$(t_0, x_0)$')

# Indicate the line from (t_0, x_0) with slope s_0
py.plot(t_0_array, x_0_array, label='$x=s_0(t-0)+x_0$')

# Indicate (t_1, x_1) point
py.plot(t_1, x_1, 'o')
py.text(t_1, x_1, '$(t_1, x_1)$')

# Slope at (t_1, x_1) point

s_1 = dx_dt(x_1, t_1)

# Straight line to next time step
t_1_array = py.linspace(t_1, t_1 + delta_t)
x_1_array = s_1 * (t_1_array - t_1) + x_1

# (t_2, x_2) point
t_2, x_2 = t_1_array[-1], x_1_array[-1]

# Indicate the line from (t_1, x_1) with slope s_1
py.plot(t_1_array, x_1_array, label='$x=s_1(t-1)+x_1$')

# Indicate (t_1, x_1) point
py.plot(t_2, x_2, 'o')
py.text(t_2, x_2, '$(t_2, x_2)$')

py.axis('equal')

py.xlim(left=-4, right=4)
py.ylim(bottom=0, top=6)

py.xlabel('t(sec)')
py.ylabel('x(m)')

py.legend(loc=0)
py.grid(True)

```

엄밀해와 비교해 보자.
Let's compare with the exact solution.

```python
# Indicate the initial point
py.plot(t_0, x_0, 'o')
py.text(t_0, x_0, '$(t_0, x_0)$')

# Indicate the line from (t_0, x_0) with slope s_0
py.plot(t_0_array, x_0_array, label='$x=s_0(t-0)+x_0$')

# Indicate (t_1, x_1) point
py.plot(t_1, x_1, 'o')
py.text(t_1, x_1, '$(t_1, x_1)$')

# Slope at (t_1, x_1) point

s_1 = dx_dt(x_1, t_1)

# Straight line to next time step
t_1_array = py.linspace(t_1, t_1 + delta_t)
x_1_array = s_1 * (t_1_array - t_1) + x_1

# (t_2, x_2) point
t_2, x_2 = t_1_array[-1], x_1_array[-1]

# Indicate the line from (t_1, x_1) with slope s_1
py.plot(t_1_array, x_1_array, label='$x=s_1(t-1)+x_1$')

# Indicate (t_1, x_1) point
py.plot(t_2, x_2, 'o')
py.text(t_2, x_2, '$(t_2, x_2)$')

# Indicate the exact solution
t_array = py.linspace(t_0, t_2)
exact_x_array = exact(t_array)
py.plot(t_array, exact_x_array, label='exact')

py.axis('equal')

py.xlim(left=-4, right=4)
py.ylim(bottom=0, top=6)

py.xlabel('t(sec)')
py.ylabel('x(m)')

py.legend(loc=0)
py.grid(True)

```

$t$값이 커 짐에 따라, 엄밀해와 $x_1$의 오차 보다 엄밀해와 $x_2$ 사이의 오차가 커지는 것을 볼 수 있다.

As $t$ increases, the error between $x_2$ and the exact solution is larger than the error between $x_1$ and the exact solution.

## 함수로 변환
Convert to a function

사용상 편리를 위해 함수를 만들어 보자.
To make it easier to use, let's make it a function.

```python
def forward_euler(f, t_array, x_0):
    time_list = [t_array[0]]
    result_list = [x_0]

    x_i = x_0

    for k, t_i in enumerate(t_array[:-1]):
        # time step
        delta_t = t_array[k+1] - t_array[k]

        # slope
        s_i = f(x_i, t_i)

        # x[i + 1]
        x_i_plus_1 = x_i + s_i * delta_t

        time_list.append(t_array[k+1])
        result_list.append(x_i_plus_1)

        x_i = x_i_plus_1

    return time_list, result_list

```

다시 그려 보자.
Let's plot again.

```python
# Time step interval
delta_t = 0.5

# Time array
t_sec_array = np.arange(0, 1 + delta_t*0.5, delta_t)

# Initial state
x_0 = 4.5

# *** new function ***
t, x = forward_euler(dx_dt, t_sec_array, x_0)
# *** new function ***

py.plot(t, x, 'o-')

for k in range(len(t)):
    py.text(t[k], x[k], f'$(t_{k}, x_{k})$')

# Indicate the exact solution
t_array = py.linspace(t_0, t_2)
exact_x_array = exact(t_array)
py.plot(t_array, exact_x_array, label='exact')

py.axis('equal')

py.xlim(left=-4, right=4)
py.ylim(bottom=0, top=6)

py.xlabel('t(sec)')
py.ylabel('x(m)')

py.legend(loc=0)
py.grid(True)

```

## $\Delta t$ 간격의 영향
Influence of $\Delta t$ interval

오차를 줄일 수 있는 좋은 방법이 없을까?
Is there a good way to reduce the error?

$\Delta t=0.5$ 를 $\Delta t=0.1$로 한번 줄여 보자.
Let's make $\Delta t=0.5$ to $\Delta t=0.1$.

```python
# Time step interval
delta_t = 0.1

# Time array
t_sec_array = np.arange(0, 1 + delta_t*0.5, delta_t)

# Initial state
x_0 = 4.5

# *** new function ***
t_01, x_01 = forward_euler(dx_dt, t_sec_array, x_0)

```

```python
py.plot(t, x, 'o-', label='$\Delta t=0.5$')
py.plot(t_01, x_01, '.-', label='$\Delta t=0.1$')

# Indicate the exact solution
t_array = py.linspace(t_0, t_2)
exact_x_array = exact(t_array)
py.plot(t_array, exact_x_array, label='exact')

py.axis('equal')

py.xlim(left=-4, right=4)
py.ylim(bottom=0, top=6)

py.xlabel('t(sec)')
py.ylabel('x(m)')

py.legend(loc=0)
py.grid(True)

```

$\Delta t$ 간격을 줄이면 오차에 어떤 영향을 미쳤는가?
How did reducing $\Delta t$ interval influence the error?

## Final Bell
마지막 종

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```


```python
from bokeh.io import output_notebook,show
output_notebook()
```

    Loading BokehJS ...

```python
import sympy as sm
import numpy as np
from scipy import optimize
from scipy import interpolate

## Micro optimization
def total_utility(c, weight, theta):
    '''
    Sums utility for c for multiple years
    c is an array
    '''
    uts = (c**(1-theta)-1)/(1-theta)
    # sum of utitity
    t_u = np.dot(uts,weight)

    return t_u

def prod(k,l,alpha,a):
    return (k**alpha)*((a*l)**(1-alpha))

def tot_ut_multiple_sks_quick(sks, k0, l, b, weight, theta, alpha, delta):
    '''
    Finds total utitilty for a set of years with a savingsrate for each year
    '''
    t = len(sks)
    k_short = np.empty(t)
    k_short[0] = k0

    for i in range(1,t):
        k_short[i]=sks[i-1]*prod(k_short[i-1],l[i-1],alpha,b)+(1-delta)*k_short[i-1]

    y_short = prod(k_short,l,alpha,b)

    return total_utility(y_short*(1-sks)/l, weight, theta)
```

```python
def optimal_sks(t, b, n, weight, delta, alpha, theta, k0, l0):
    '''This funcition also solves the maximization problem but instead of taking
    a vector of populations growth, it only take l0 and calculates the rest
    this is for use in the simulation of the macro model, since the population is changing in every period.
    '''
    l = np.array([l0*(1+n)**i for i in range(t)])
    obj = lambda sks: -tot_ut_multiple_sks_quick(sks, k0, l, b, weight, theta, alpha, delta)
    sks0 = np.linspace(alpha,0,t)

    bounds = np.full((t,2),[1e-10,1])
    res = optimize.minimize(obj, sks0, method='SLSQP',
        bounds=bounds,)

    if res.success:
        return res.x[0]
    else:
        print('Optimization was sadly not succesfull')

# Macro, the solow walk
def capitalakku(b,k,l,sk,alpha,delta):
    return prod(k,l,alpha,b)*sk+(1-delta)*k

def mod_solowwalk(k0, b, l0, n, alpha, delta, weight, theta, t, t_big):
    '''Simulates the modifed solow model. In each period, the representative household
    calculates the preffered savingsrate in that period and the next period is simulated
    using that savings rate.
    '''

    k_path = np.array([k0])
    l_path = np.array([l0*(1+n)**i for i in list(range(t_big))])
    y_path = np.array([prod(k_path[0],l_path[0],alpha,b)])

    sk_path = np.array([optimal_sks2(
        t, b, n, weight, delta, alpha, theta, k_path[0],l_path[0])])

    for i in range(1,t_big):
        k_plus = capitalakku(b,k_path[i-1],l_path[i-1],sk_path[i-1],alpha,delta)
        y_plus = prod(k_plus,l_path[i-1],alpha,b)
        sk_plus = np.array([optimal_sks2(t, b, n, weight, delta, alpha, theta, k_plus,l_path[i])])

        k_path = np.append(k_path, k_plus)
        y_path = np.append(y_path, y_plus)
        sk_path = np.append(sk_path,sk_plus)

    k_pr_path = k_path/l_path
    y_pr_path = y_path/l_path
    return k_pr_path, y_pr_path, sk_path
```

```python
# Plotting function:
from bokeh.io import output_notebook, push_notebook,show
from bokeh.plotting import figure, show, output_file
from bokeh.models import ColumnDataSource, HoverTool, NumeralTickFormatter

def plotting(x,y_names, x_array, y_arrays,y_name ='Savings rate',title = 'Figure'
            , colors= ['red','blue','green','purple','yellow'],
             legendlocation="top_center",tools="pan,wheel_zoom,box_zoom,reset,save"):

    '''
    Bokeh plotting
    '''

    # Bokeh needs a name for the data that neither has spaces nor numbers
    # because we want the option to do this we define abitrairy calls via the alphabeth.
    calls = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
     'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
    y_calls = []

    for i in range(len(y_names)):
        y_calls.append(calls[i])

    tooltips=[(f'{x}','@x{0,0.00}')]

    for yn,yc in zip(y_names,y_calls):
        text = '@'+f'{yc}'+'{0.00}'
        tooltips.append((f'{y_name} for {yn}',text))

    hover = HoverTool(tooltips=tooltips)
    data = {'x': x_array}
    for yc, y_array in zip(y_calls,y_arrays):
        data[f'{yc}']=y_array

    source = ColumnDataSource(data)

    p = figure(title=f'{title}',tools=[hover,tools],
               x_axis_label=f'{x}', y_axis_label=f'{y_name}')
    for i,(yc,yn) in enumerate(zip(y_calls,y_names)):
        p.line(x='x', y=f'{yc}', source=source,
           legend= f'{yn}', color = colors[i])

    p.legend.location = legendlocation

    show(p,notebook_handle=True)
```

```python
# Parameters
#Macro:
alpha = 1/3
delta = 0.05
n = 0.008
sk = 0.2

# production:
b = 1
k0 = 5
l0 = 1

# Our timeline:
timeline = 200
#Micro:
theta = 0.5
beta = 0.99

t = 40

# time preference utility weights and population growth
# are precomputed to save time:
weight = np.array([beta**i for i in range(t)])
```

```python
k_n = 150
theta = 0.5
sks = np.zeros(k_n)
k0s = np.linspace(1e-4,30,k_n)

for i in range(k_n):
        sks[i] = optimal_sks(
            t, b, n, weight, delta, alpha, theta, k0s[i],l0)

```

```python
''' We use Bokeh to plot and have defined a function in model_funcs.py
that calls the plot to bokeh '''

# \u03B8 is code for theta
thetas_names = ['\u03B8 = 0.5']

plotting("Capital pr. capita",thetas_names, k0s, [sks])
```

```python
def solve_micro(t, b, n, weight, delta, alpha, theta,l0,k_min=1e-4, k_high=30, precision=150):
    sks = np.zeros(precision)
    k0s = np.linspace(k_min,k_high,k_n)
    for i in range(k_n):
            sks[i] = optimal_sks(
                t, b, n, weight, delta, alpha, theta, k0s[i],l0)
    sk_interp = interpolate.RegularGridInterpolator([k0s], sks,
        bounds_error=False,fill_value=None)
    return sks, k0s, sk_interp
```

```python
sks2, k0s2, sk_interp2 = solve_micro(t, b, n, weight, delta, alpha, theta,l0)
```

```python
sk_interp = interpolate.RegularGridInterpolator([k0s], sks,
        bounds_error=False,fill_value=None)
```

```python
rand = np.random.uniform(1,30)
assert sk_interp2([rand]) == sk_interp([rand])
```

```python
def new_mod_solowwalk(k0, l0, b,n, alpha, delta, sk_interp, timeline):
    '''
    Simulates the modifed solow model. In each period, the representative household
    calculates the preffered savingsrate in that period and the next period is simulated
    using that savings rate.
    '''
    k_path = np.array([k0])
    l_path = np.array([l0*(1+n)**i for i in list(range(timeline))])
    y_path = np.array([prod(k_path[0],l_path[0],alpha,b)])

    sk_path = np.array(sk_interp([k_path[0]/l_path[0]]))

    for i in range(1,timeline):
        k_plus = capitalakku(b,k_path[i-1],l_path[i-1],sk_path[i-1],alpha,delta)
        y_plus = prod(k_plus,l_path[i-1],alpha,b)
        sk_plus = np.array(sk_interp([k_plus/l_path[i]]))

        k_path = np.append(k_path, k_plus)
        y_path = np.append(y_path, y_plus)
        sk_path = np.append(sk_path,sk_plus)

    k_pr_path = k_path/l_path
    y_pr_path = y_path/l_path
    return k_pr_path, y_pr_path, sk_path
```

```python
k_pr_path, y_pr_path, sk_path = new_mod_solowwalk(k0, l0, b, n, alpha, delta, sk_interp, timeline)
```

```python
def find_ssk_sk(k,b,delta,n,alpha):
    return (k**(1-alpha)*(delta+n))/b

def find_ss(sk_interp, b, n, beta, delta, alpha, bracket):
    '''
    Finding the steady state of the model by figuring out which mount of capital,
    that makes the optimal savings rate chosen by the consumer equal
    to the savings rate the implies that this amount of capital is the steady state.
    '''

    obj = lambda k_star: find_ssk_sk(k_star,b,delta,n,alpha)-sk_interp([k_star])[0]
    res = optimize.root_scalar(obj,method='brentq',bracket=bracket)
    if res.converged:
        k_star = res.root
        sk_star = find_ssk_sk(k_star,b,delta,n,alpha)
        return k_star,sk_star
    else:
        print('Convergence failed')
```

```python
k_star, sk_star = find_ss(sk_interp, b, n, beta, delta, alpha, [0,30])
k_stars = [k_star for i in range(timeline)]
sk_stars = [sk_star for i in range(timeline)]
```

```python
plotting("T",['Mod_solow','Steady state'], range(timeline), [k_pr_path,k_stars],y_name ='Capital pr. capita')
```

```python
plotting("T",['Mod_solow','Steady state'], range(timeline), [sk_path,sk_stars], ['ms','ss'])
```


```python
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

```python
import seaborn as sns
sns.set_style('ticks', {'axes.grid': True})
sns.set_context('poster', font_scale=2.0)
%config InlineBackend.figure_format = 'retina'

plt.rcParams['figure.figsize'] = (12.8, 9.6)
plt.rcParams['figure.facecolor'] = 'white'
```

# lattice spacing and parameter

```python
hbarc = 0.197e3 # MeV fm
lat_unit = 0.1 # 0.1 fm
ainv = hbarc/lat_unit
```

# generate mock-up data

We consider the mock-up data given by
\begin{equation}
R(t) = b_0 e^{-\Delta E_\mathrm{BB} t} + b_1 e^{-(\delta E_\mathrm{el} + \Delta E_\mathrm{BB})t}
+ c_0 e^{-(\delta E_\mathrm{inel} + \Delta E_\mathrm{BB})t}
\end{equation}
where the ground state energy shift $\Delta E_\mathrm{BB}$, the gap of the first excited elastic state $\delta E_\mathrm{el}$,
and the gap of the lowest inelastic state $\delta E_\mathrm{inel}$.

In this example, we consider $b_1/b_0 = \pm 0.1, 0$ and $c_0/b_0 = 0.01$ with $\delta E_\mathrm{el} = 50$ MeV
and $\delta E_\mathrm{inel} = 500$ MeV.

```python
def generate_mockup(ts, Eel=50, b1b0=0.1, Einel=500, c0b0=0.01, sigma0=1.0e-4, sn_fac=0.3, Nsample=1000, seed=0):
    """
    Return effective energy shifts with fluctuations.
    (We neglect the correlation of the data for simplicity.)

    Parametrs:
    ts : np.ndarray
        Euclidean time
    Eel : float
        elastic state gap in MeV
    b1b0 : float
        contamination of the first excited elastic state (default = 0.1)
    c1b0 : float
        contamination of the lowest inelastic state (default = 0.01)
    sigma0 : float
        magnitude of the fluctuation
    sn_fac : float
        exponent of the growth of the noise
    Nsample : integer
        number of sampling to estimate the errors
    seed : integer
        seed for the random numbers

    Returns:
    Eeffs : np.ndarray
        the effective energy shifts
    Eeff_errs : np.ndarray
        the error of the effective energy shifts
    """
    np.random.seed(seed)
    R = lambda t: 1.0 + b1b0 * np.exp(-Eel*t/ainv) + c0b0 * np.exp(-Einel*t/ainv)
    sigma = lambda t: sigma0 * np.exp(sn_fac * t)
    Eeffs, Eeff_errs = [], []

    Ri = R(ts[0]) * (1.0 + sigma(ts[0]) * np.random.randn(Nsample))
    for t in ts:
        Rni = R(t+1) * (1.0 + sigma(t+1) * np.random.randn(Nsample))
        yi = np.log(Ri/Rni)
        Eeffs.append(yi.mean())
        Eeff_errs.append(yi.std() / np.sqrt(Nsample - 1))

        Ri = Rni # update correlator values
    return np.array(Eeffs), np.array(Eeff_errs)
```

```python
def test_plot(Eel=50, b1b0=0.1, Einel=500, c0b0=0.01, yr=(-10, 10), seed=1):
    """
    test the behavior of the plateau for given parameters and random seed
    """
    fig, ax = plt.subplots()
    ts = np.arange(1, 24)

    Eeff, Eeff_err = generate_mockup(ts, b1b0=b1b0, c0b0=c0b0, seed=seed)
    ax.errorbar(ts * lat_unit, Eeff*ainv, Eeff_err*ainv,
               fmt='o', color='red', capsize=10, capthick=2,
               label=r'$c_0/b_0 = {:.2f}$, $b_1/b_0 = {:.2f}$'.format(c0b0, b1b0))

    ax.set_ylim(yr)
    ax.set_xlabel(r'$t$ [fm]', fontsize=48)
    ax.set_ylabel(r'$\Delta E_\mathrm{BB}^\mathrm{eff}(t) - \Delta E_\mathrm{BB}$ [MeV]',
                 fontsize=48)
    ax.axhline(0, c='black')
    leg = ax.legend(frameon=True)
    leg.get_frame().set_edgecolor('black')
    leg.get_frame().set_linewidth(2.0)

```

```python
test_plot(b1b0= -0.1, seed=2)
```

```python
test_plot(b1b0= 0.1, seed=4)
```

```python
test_plot(b1b0= 0.0, seed=10)
```

```python
fig, ax = plt.subplots()
ts = np.arange(1, 24)
c0b0 = 0.01
for b1b0, seed, fmt, col in zip([0.1, 0, -0.1], [2, 10, 4], ['s', 'D', 'o'], ['blue', 'black', 'red']):
    Eeff, Eeff_err = generate_mockup(ts, b1b0=b1b0,seed=seed)
    ax.errorbar(ts * lat_unit, Eeff*ainv, Eeff_err*ainv,
               fmt=fmt, color=col, capsize=10, capthick=2,
               label=r'$c_0/b_0 = {:.2f}$, $b_1/b_0 = {:.1f}$'.format(c0b0, b1b0))

leg = ax.legend(frameon=True, fontsize=28)
leg.get_frame().set_edgecolor('black')
leg.get_frame().set_linewidth(2.0)
ax.set_ylim(-11, 11)
ax.axhline(0, color='black')
ax.set_xlabel(r'$t$ [fm]', size=48)
ax.set_ylabel(r'$\Delta E_\mathrm{BB}^\mathrm{eff}(t) - \Delta E_\mathrm{BB}$ [MeV]', size=48);
```

# effective energy shift without noise

```python
R = lambda b1b0, Eel, c0b0, Einel, t: 1.0 + b1b0*np.exp(-Eel*t) + c0b0*np.exp(-Einel*t)
dE_exact = lambda b1b0, Eel, c0b0, Einel, t: np.log(R(b1b0, Eel, c0b0, Einel, t)/R(b1b0, Eel, c0b0, Einel, t+1))

fig, ax = plt.subplots()

tls = np.linspace(1, 100, 200)
for b1b0, c, ls in zip([0.1, 0, -0.1], ['blue', 'black', 'red'], ['--', '-', '-.']):
    ex = dE_exact(b1b0, 50/ainv, 0.01, 500/ainv, tls)
    ax.plot(tls * lat_unit, ex* ainv, c=c, lw=5, linestyle=ls, label=r'$b_1/b_0 = {}$'.format(b1b0), zorder=3)

leg = ax.legend(loc='lower left', ncol=2, frameon=True, fontsize=28)
leg.get_frame().set_edgecolor('black')
leg.get_frame().set_linewidth(2.0)

ax.set_xlabel(r'$t$ [fm]', size=48)
ax.set_ylabel(r'$\Delta E_{\mathrm{BB}}^{\mathrm{eff}}(t) - \Delta E_{\mathrm{BB}}$ [MeV]', size=48)
ax.set_ylim(-8, 8)
ax.axhline(0, color='black');
```

```python

```


# Práctica 3 - Dinámica de manipuladores

## List comprehensions

En esta práctica nos enfocaremos en temas avanzados de programación que se aplican directamente al lenguaje de programación Python y a algunos otros lenguajes de programación. En primer lugar veamos la instrucción ```range```:

```python
range(5)
```

Mmmm..., realmente no hizo nada... bueno, el objetivo de esta función de la libreria standard de Python es crear un arreglo de enteros desde $0$ hasta el numero que le demos como argumento:

```python
for i in range(5):
    print(i)
```

En este caso es obvio como se estan generando los numeros, desde $0$ hasta ¿$4$?. Mas bien hasta que el numero de elementos sea el numero dado, o bien $n-1$; de cualquier manera, esta forma de utilizar el ciclo ```for``` es la que nos da el vinculo con el ```for``` tradicional de otros lenguajes de programación, pero en Python tenemos una manera diferente de usar este mismo ```for```, *List Comprehensions*:

```python
[i for i in range(5)]
```

Si recuerdas la práctica 1, para crear un arreglo con numeros adentro, podiamos utilizar el siguiente código:

```python
A = []
for i in range(5):
    A.append(i)

A
```

Sin embargo, comparandolo con la simplicidad del código anterior, este enfoque palidece, por lo que en Python es normal ver este tipo de construcciones, podemos diseccionarlo en dos partes:

```Python
[i for i in range(5)]
```

en la primera, ```i``` ocupa el lugar del elemento que nos dicta que es lo que vamos a guardar en el arreglo final; en este caso, es el numero a secas, por lo que no hacemos ninguna operación, pero si queremos guardar en un arreglo, por ejemplo, los cuadrados de los primeros $10$ enteros positivos, tendremos:

```python
[i**2 for i in range(10)]
```

La segunda parte de este código,

```Python
[i for i in range(5)]
```

es el ```for```, esta es la razon por la que usamos ```i``` anteriormente, da lo mismo escribir:

```python
[x**2 for x in range(10)]
```

y por supuesto **no** funcionará si:

```python
[x**2 for i in range(10)]
```

Pero regresemos por un momento al primer pedazo de código que escribimos:

```python
range(5)
```

Si ese... lo que esta pasando aqui es que Python realmente no quiere evaluar esta expresión hasta que sea completamente necesario, a este concepto se le llama *Lazy Evaluation*, o evaluación floja; a pesar de la mala reputación que tiene Python por ser lento, y por ocupar demasiada memoria de ejecución, tiene caracteristicas bastante interesantes, como el hecho de no evaluar expresiones que no van a ser necesarias.

Hagamos un ejemplo para que quede mas claro:

```python
if False:
    A = []
    for i in range(5):
        A.append(i)
else:
    pass
```

En este código las lineas dentro del ```if``` nunca serán ejecutadas, por lo que si tuvieramos un lenguaje de programación compilado (como C), en primer lugar tendría que reservar espacio en memoria para el arreglo que va a guardar los datos de ```A```, sin embargo nunca se utilizaria ese espacio, en Python nunca se ejecutarán esas lineas y lo demostraremos:

```python
%%timeit
A = [i for i in range(5)]
```

Una de las funciones especiales de Jupyter (no de Python) es que si escribo ```%%timeit``` al inicio de cualquier celda, la ejecutará muchas veces y me reportará el promedio de tiempo que se tardó en ejecutar esta celda, en este caso nuestro programa se tardó $739ns$ en promedio.

Si por el otro lado, ejecuto las lineas en donde saltará especificamente la creación de este arreglo tendremos:

```python
%%timeit
if False:
    A =[]
    for i in range(5):
        A.append(i)
else:
    pass
```

En esta ocasión se tarda $12.4ns$ ya que en realidad no esta haciendo nada...

Por cierto, una cosa más... las list comprehensions funcionan no solo con ```range```, sino con cualquier arreglo, como en el siguiente caso:

```python
[x**2 + 1 for x in [2,4,6,8,10]]
```

o incluso este:

```python
[caracter + " abc" for caracter in "pizza"]
```

---
## Ejercicio

Utilizando list comprehensions:

* Genera un arreglo con los simbolos de sympy que representan la longitud de 5 eslabones, recuerda que para generar el simbolo de $l_1$ tenemos que escribir:

```Python
var("l1")
```

```python
from sympy.physics.mechanics import mechanics_printing
mechanics_printing()
```

```python
from sympy import var
# ESCRIBE TU CODIGO AQUI
raise NotImplementedError
longs
```

```python
from nose.tools import assert_equal
from sympy import var
assert_equal(longs, [var("l1"), var("l2"), var("l3"), var("l4"), var("l5")])
```

* Genera un arreglo con las matrices simbolicas de sympy que contengan las matrices de transformación homogéneas de los 5 grados de libertad descritos en el siguiente arreglo con parametros DH:

```python
parametros_DH = [[ "0", "l1", "0", "q1"],
                 ["l2",  "0", "0", "q2"],
                 [ "0", "l3", "0", "q3"],
                 ["l4",  "0", "0", "q4"],
                 [ "0", "l5", "0", "q5"]]
```

```python
from sympy import var
# ESCRIBE TU CODIGO AQUI
raise NotImplementedError
matrices
```

```python
from nose.tools import assert_equal
from sympy import Matrix, var, sin, cos
l1, q1, cero = var("l1"), var("q1"), var("0")
A1 = Matrix([[cos(q1), -sin(q1)*cos(cero), sin(cero)*sin(q1), cero*cos(q1)],
             [sin(q1), cos(cero)*cos(q1), -sin(cero)*cos(q1), cero*sin(q1)],
             [0, sin(cero), cos(cero), l1],
             [0, 0, 0, 1]])
assert_equal(matrices[0], A1)
l2, q2, cero = var("l2"), var("q2"), var("0")
A2 = Matrix([[cos(q2), -sin(q2)*cos(cero), sin(cero)*sin(q2), l2*cos(q2)],
             [sin(q2), cos(cero)*cos(q2), -sin(cero)*cos(q2), l2*sin(q2)],
             [0, sin(cero), cos(cero), cero],
             [0, 0, 0, 1]])
assert_equal(matrices[1], A2)
l3, q3, cero = var("l3"), var("q3"), var("0")
A3 = Matrix([[cos(q3), -sin(q3)*cos(cero), sin(cero)*sin(q3), cero*cos(q3)],
             [sin(q3), cos(cero)*cos(q3), -sin(cero)*cos(q3), cero*sin(q3)],
             [0, sin(cero), cos(cero), l3],
             [0, 0, 0, 1]])
assert_equal(matrices[2], A3)
l4, q4, cero = var("l4"), var("q4"), var("0")
A4 = Matrix([[cos(q4), -sin(q4)*cos(cero), sin(cero)*sin(q4), l4*cos(q4)],
             [sin(q4), cos(cero)*cos(q4), -sin(cero)*cos(q4), l4*sin(q4)],
             [0, sin(cero), cos(cero), cero],
             [0, 0, 0, 1]])
assert_equal(matrices[3], A4)
l5, q5, cero = var("l5"), var("q5"), var("0")
A5 = Matrix([[cos(q5), -sin(q5)*cos(cero), sin(cero)*sin(q5), cero*cos(q5)],
             [sin(q5), cos(cero)*cos(q5), -sin(cero)*cos(q5), cero*sin(q5)],
             [0, sin(cero), cos(cero), l5],
             [0, 0, 0, 1]])
assert_equal(matrices[4], A5)
```


#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1yV41167Di', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1yV41167Di

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")

```

```python
#@title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
  """Plot a step histogram given counts over bins."""
  if ax is None:
    _, ax = plt.subplots()

  # duplicate the first element of `counts` to match bin edges
  counts = np.insert(counts, 0, counts[0])

  ax.fill_between(bins, counts, step="pre", alpha=0.4, **kwargs)  # area shading
  ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

  for x in vlines:
    ax.axvline(x, color='r', linestyle='dotted')  # vertical line

  if ax_args is None:
    ax_args = {}

  # heuristically set max y to leave a bit of room
  ymin, ymax = ax_args.get('ylim', [None, None])
  if ymax is None:
    ymax = np.max(counts)
    if ax_args.get('yscale', 'linear') == 'log':
      ymax *= 1.5
    else:
      ymax *= 1.1
      if ymin is None:
        ymin = 0

  if ymax == ymin:
    ymax = None

  ax_args['ylim'] = [ymin, ymax]

  ax.set(**ax_args)
  ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
  fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

  # membrane voltage trace
  ax1.plot(v[0:100])
  ax1.set(xlabel='Time', ylabel='Voltage')
  # plot spike events
  for x in spike_times:
    if x >= 100:
      break
    ax1.axvline(x, color='red')

  # ISI distribution
  isi = np.diff(spike_times)
  n_bins = np.arange(isi.min(), isi.max() + 2) - .5
  counts, bins = np.histogram(isi, n_bins)
  vlines = []
  if len(isi) > 0:
    vlines = [np.mean(isi)]
  xmax = max(20, int(bins[-1])+5)
  histogram(counts, bins, vlines=vlines, ax=ax2, ax_args={
    'xlabel': 'Inter-spike interval',
    'ylabel': 'Number of intervals',
    'xlim': [0, xmax]
  })
  plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes

  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ################################################################################
  # Students: compute dv, then comment out or remove the next line
  raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = ...

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines after completing the lif_neuron function
# v, spike_times = lif_neuron()
# plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  """ Simulate a linear integrate-and-fire neuron.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    rate (int): The mean rate of incoming spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    dv = alpha * exc[i]

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

v, spike_times = lif_neuron()
with plt.xkcd():
  plot_neuron_stats(v, spike_times)
```

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
#@title

#@markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  exc = stats.poisson(rate).rvs(n_steps)
  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = alpha * exc[i]
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0
  return v, spike_times

@widgets.interact(
  n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
  rate=widgets.IntSlider(10, min=5, max=20)
)
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
  v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 2: Linear-IF models
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1iZ4y1u7en', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1iZ4y1u7en

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """

  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []

  ###############################################################################
  # Students: compute dv, then comment out or remove the next line
  raise NotImplementedError("Excercise: compute the change in membrane potential")
  ################################################################################

  for i in range(1, n_steps):

    dv = ...

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

# Uncomment these lines do make the plot once you've completed the function
#v, spike_times = lif_neuron_inh()
#plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):

    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])

    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

v, spike_times = lif_neuron_inh()

with plt.xkcd():
  plot_neuron_stats(v, spike_times)
```

## Interactive Demo: LIF + inhibition neuron

```python
#@title
#@markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
  and inhibitory inputs.

  Args:
    n_steps (int): The number of time steps to simulate the neuron's activity.
    alpha (float): The input scaling factor
    beta (float): The membrane potential leakage factor
    exc_rate (int): The mean rate of the incoming excitatory spikes
    inh_rate (int): The mean rate of the incoming inhibitory spikes
  """
  # precompute Poisson samples for speed
  exc = stats.poisson(exc_rate).rvs(n_steps)
  inh = stats.poisson(inh_rate).rvs(n_steps)

  v = np.zeros(n_steps)
  spike_times = []
  for i in range(1, n_steps):
    dv = -beta * v[i-1] + alpha * (exc[i] - inh[i])
    v[i] = v[i-1] + dv
    if v[i] > 1:
      spike_times.append(i)
      v[i] = 0

  return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.5, beta=0.1, exc_rate=10, inh_rate=10):
  v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate, inh_rate)
  plot_neuron_stats(v, spike_times)
```

```python
#@title Video 3: LIF + inhibition
from IPython.display import IFrame
class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
        self.id=id
        src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
        super(BiliVideo, self).__init__(src, width, height, **kwargs)

video = BiliVideo(id='BV1nV41167mS', width=854, height=480, fs=1)
print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
video
```

    Video available at https://www.bilibili.com/video/BV1nV41167mS

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).