# Lead-Acid Models

We compare a standard porous-electrode model for lead-acid batteries with two asymptotic reductions. For a more in-depth introduction to PyBaMM models, see the [SPM notebook](./SPM.ipynb). Further details on the models can be found in [[4]](#References).

```python
%pip install pybamm -q    # install PyBaMM if it is not installed
import pybamm
import numpy as np
import os
import matplotlib.pyplot as plt
os.chdir(pybamm.__path__[0]+'/..')
```

    Note: you may need to restart the kernel to use updated packages.

### "Full" model

#### Electrolyte Concentration

$$
    \frac{\partial }{\partial t}\left(\epsilon c\right) = -\frac{\partial N}{\partial x} + sj, \\
    N = -\frac{\epsilon^b  D(c)}{\mathcal{C}_\text{e}}  \frac{\partial c}{\partial x}\\
    N\big|_{x=0}= N\big|_{x=1}=0, \\
    c\big|_{t=0} = 1
$$

#### Porosity

$$
    \frac{\partial \epsilon}{\partial t} = -\beta^\text{surf}j, \\
    \epsilon\big|_{t=0} = \epsilon^0
$$

#### Electrolyte Current

$$
    \frac{\partial i_{\text{e}}}{\partial x} = j, \\
    \mathcal{C}_\text{e} i_{\text{e}} = \epsilon_k^b \kappa(c)
        \left( \chi \frac{\partial}{\partial x}\log(c)
        - \frac{\partial\phi_{\text{e}}}{\partial x}\right)\\
    i_{\text{e}}\big|_{x=0}= i_{\text{e}}\big|_{x=1}=0,
$$

#### Electrode Current

$$
    \frac{\partial i_{\text{s}}}{\partial x} = -j,\\
    i_{\text{s}} = -\sigma\frac{\partial\phi_{\text{s}}}{\partial x},\\
    \phi_{\text{s}}\big|_{x=0}
        = i_{\text{s}}\big|_{x=l_\text{n}}
        = i_{\text{s}}\big|_{x=1-l_\text{p}} = 0, \\
    i_{\text{s}}\big|_{x=1}=\mathcal{I},
$$

#### interfacial current density

$$
j = \begin{cases}
    2j_0(c) \sinh\left(\eta\right), \quad &0 < x < l_\text{n} \\
    0, \quad &l_\text{n} < x < 1-l_\text{p} \\
    2j_0(c) \sinh\left(\eta\right), \quad &1-l_\text{p} < x < 1
\end{cases}
\\
\eta = \phi_{\text{s}} - \phi_\text{e} - U(c),
$$

This model is implemented in PyBaMM as the `Full` model

```python
full = pybamm.lead_acid.Full()
```

### "Leading-order" model

$$
    \frac{\mathrm{d} }{\mathrm{d} t}\left(\epsilon c\right) = (s_\text{n} - s_\text{p})\mathrm{I}, \\
    \frac{\mathrm{d} \epsilon}{\mathrm{d} t} = -\beta^\text{surf}j, \\
    j = \begin{cases}
        \mathrm{I}/l_\text{n}, \quad &0 < x < l_\text{n} \\
        0, \quad &l_\text{n} < x < 1-l_\text{p} \\
        -\mathrm{I}/l_\text{p}, \quad &1-l_\text{p} < x < 1
    \end{cases} \\
    \phi_\text{e} = -U_\text{n}(c) + \sinh^{-1}\left(\frac{\mathrm{I}}{2l_\text{n}j_{0\text{n}}(c)}\right) \\
    V = -\phi_\text{e} + U_\text{p}(c) - \sinh^{-1}\left(\frac{\mathrm{I}}{2l_\text{p}j_{0\text{p}}(c)}\right) \\
$$

This model is implemented in PyBaMM as `LOQS` (leading-order quasi-static)

```python
loqs = pybamm.lead_acid.LOQS()
```

### "Composite" model

#### Electrolyte Concentration

$$
    \frac{\partial }{\partial t}\left(\epsilon c\right) = -\frac{\partial N}{\partial x} + sj, \\
    N = -\frac{\epsilon^b  D(c)}{\mathcal{C}_\text{e}}  \frac{\partial c}{\partial x}\\
    N\big|_{x=0}= N\big|_{x=1}=0, \\
    c\big|_{t=0} = 1
$$

#### Porosity

$$
    \frac{\partial \epsilon}{\partial t} = -\beta^\text{surf}j, \\
    \epsilon\big|_{t=0} = \epsilon^0
$$

#### Interfacial current density

$$
j = \begin{cases}
        \mathrm{I}/l_\text{n}, \quad &0 < x < l_\text{n} \\
        0, \quad &l_\text{n} < x < 1-l_\text{p} \\
        -\mathrm{I}/l_\text{p}, \quad &1-l_\text{p} < x < 1
    \end{cases} \\
$$

#### Voltage

$$
V = U_{\text{eq}} + \eta_r + \eta_c + \Delta\Phi_{\text{Elec}} + \Delta\Phi_{\text{Solid}}
$$
where
\begin{align}
    & U_{\text{eq}} = U_p(\bar{c}_p) - U_n(\bar{c}_n), \\
     &\eta_{r} = -\sinh^{-1}\left(\frac{\mathcal{I}}{2\bar{j}_{0n} l_n}\right)
     -\sinh^{-1}\left(\frac{\mathcal{I}}{2\bar{j}_{0p} l_p}\right), \\
     &\eta_c =  \mathcal{C}_e\chi\left(\log(\bar{c}_{p}) - \log(\bar{c}_{n})\right), \\
    &\Delta \Phi_{\text{Elec}}= -\frac{\mathcal{C}_e \mathcal{I}}{\kappa(\bar{c})}\left(\frac{l_n}{3\epsilon_n^b} + \frac{l_s}{\epsilon_{s}^b} + \frac{l_p}{3\epsilon_p^b} \right),
     \label{eqn:SPMe:electrolyte_ohmic_losses} \\
     &\Delta \Phi_{\text{Solid}} =  -\frac{\mathcal{I}}{3}\left(\frac{l_p}{\sigma_p} + \frac{l_n}{\sigma_n} \right),
\end{align}
with
\begin{equation}
    \bar{\cdot}_{\text{n}} =  \frac{1}{l_n}\int_0^{l_n} \cdot_{\text{n}} \, \text{d}x, \quad
    \bar{\cdot}_{\text{p}} =  \frac{1}{l_p}\int_{1-l_p}^{1} \cdot_{\text{p}} \, \text{d}x,
\end{equation}

This model is implemented in PyBaMM as `Composite`

```python
composite = pybamm.lead_acid.Composite()
```

## Solving the models

We load process parameters for each model, using the same set of (default) parameters for all. In anticipation of changing the current later, we make current an input parameter

```python
# load models
models = [loqs, composite, full]

# process parameters
param = models[0].default_parameter_values
param["Current function [A]"] = "[input]"
for model in models:
    param.process_model(model)
```

Then, we discretise the models, using the default settings

```python
for model in models:
    # load and process default geometry
    geometry = model.default_geometry
    param.process_geometry(geometry)

    # discretise using default settings
    mesh = pybamm.Mesh(geometry, model.default_submesh_types, model.default_var_pts)
    disc = pybamm.Discretisation(mesh, model.default_spatial_methods)
    disc.process_model(model)
```

Finally, we solve each model using CasADi's solver and a current of 1A

```python
timer = pybamm.Timer()
solutions = {}
t_eval = np.linspace(0, 3600 * 17, 100) # time in seconds
solver = pybamm.CasadiSolver()
for model in models:
    timer.reset()
    solution = solver.solve(model, t_eval, inputs={"Current function [A]": 1})
    print("Solved the {} in {}".format(model.name, timer.time()))
    solutions[model] = solution
```

    Solved the LOQS model in 125.991 ms
    Solved the Composite model in 1.442 s
    Solved the Full model in 1.688 s

## Results

To plot the results, the variables are extracted from the solutions dictionary. For example, we can compare the voltages:

```python
for model in models:
    time = solutions[model]["Time [h]"].entries
    voltage = solutions[model]["Terminal voltage [V]"].entries
    plt.plot(time, voltage, lw=2, label=model.name)
plt.xlabel("Time [h]", fontsize=15)
plt.ylabel("Terminal voltage [V]", fontsize=15)
plt.legend(fontsize=15)
plt.show()
```

Alternatively, using `QuickPlot`, we can compare the values of some variables

```python
solution_values = [solutions[model] for model in models]
quick_plot = pybamm.QuickPlot(solution_values)
quick_plot.dynamic_plot();
```

    interactive(children=(FloatSlider(value=0.0, description='t', max=17.0, step=0.17), Output()), _dom_classes=('…

If we update the current, setting it to be 20 A, we observe a greater discrepancy between the full model and the reduced-order models.

```python
t_eval = np.linspace(0, 3600, 100)
for model in models:
    solutions[model] = solver.solve(model, t_eval, inputs={"Current function [A]": 20})

# Plot
solution_values = [solutions[model] for model in models]
quick_plot = pybamm.QuickPlot(solution_values)
quick_plot.dynamic_plot();
```

    interactive(children=(FloatSlider(value=0.0, description='t', max=3563.636363636364, step=35.63636363636364), …

## References

The relevant papers for this notebook are:

```python
pybamm.print_citations()
```

    [1] Joel A. E. Andersson, Joris Gillis, Greg Horn, James B. Rawlings, and Moritz Diehl. CasADi – A software framework for nonlinear optimization and optimal control. Mathematical Programming Computation, 11(1):1–36, 2019. doi:10.1007/s12532-018-0139-4.
    [2] Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, and others. Array programming with NumPy. Nature, 585(7825):357–362, 2020. doi:10.1038/s41586-020-2649-2.
    [3] Valentin Sulzer, S. Jon Chapman, Colin P. Please, David A. Howey, and Charles W. Monroe. Faster Lead-Acid Battery Simulations from Porous-Electrode Theory: Part I. Physical Model. Journal of The Electrochemical Society, 166(12):A2363–A2371, 2019. doi:10.1149/2.0301910jes.
    [4] Valentin Sulzer, S. Jon Chapman, Colin P. Please, David A. Howey, and Charles W. Monroe. Faster Lead-Acid Battery Simulations from Porous-Electrode Theory: Part II. Asymptotic Analysis. Journal of The Electrochemical Society, 166(12):A2372–A2382, 2019. doi:10.1149/2.0441908jes.
    [5] Valentin Sulzer, Scott G. Marquis, Robert Timms, Martin Robinson, and S. Jon Chapman. Python Battery Mathematical Modelling (PyBaMM). ECSarXiv. February, 2020. doi:10.1149/osf.io/67ckj.


# Introduction (part 2): `Operator` `apply` and `arguments`

## Goals

- Describe two fundamental user APIs:
    * `Operator.apply`
    * `Operator.arguments`
- Describe the `Data` objects associated with functions
- Investigate how these `Data` objects are updated during time loops

We will use a trivial `Operator` that, at each time step, increments by 1 all points in the physical domain.

```python
from devito import Grid, TimeFunction, Eq, Operator

grid = Grid(shape=(4, 4))
u = TimeFunction(name='u', grid=grid, save=3)
op = Operator(Eq(u.forward, u + 1))
```

To run `op`, we have to "`apply`" it.

```python
#NBVAL_IGNORE_OUTPUT
summary = op.apply()
```

    Operator `Kernel` run in 0.01 s

Under-the-hood, some code has been generated (`print(op)` to display the generated code), JIT-compiled, and executed. Since no additional arguments have been passed, `op` has used `u` as input. We can verify that the content of `u.data` is as expected

```python
u.dimensions, u.shape
```

    ((time, x, y), (3, 4, 4))

```python
u.data
```

    Data([[[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]],

          [[1., 1., 1., 1.],
           [1., 1., 1., 1.],
           [1., 1., 1., 1.],
           [1., 1., 1., 1.]],

          [[2., 2., 2., 2.],
           [2., 2., 2., 2.],
           [2., 2., 2., 2.],
           [2., 2., 2., 2.]]], dtype=float32)

In particular, we observe that:

* `u` has size 3 along the time dimension, since it was built with `save=3`. Therefore `op` could only execute 2 timesteps, namely time=0 and time=1; given `Eq(u.forward, u + 1)`, executing time=2 would cause out-of-bounds access errors. Devito figures this out automatically and sets appropriate minimum and maximum iteration points.
* All 16 points in each timeslice of the 4x4 `Grid` have been computed.

To access all default arguments used by `op` *without* running the `Operator`, one can run

```python
#NBVAL_IGNORE_OUTPUT
op.arguments()
```

    {'u': <cparam 'P' (0x7f48f89b7bb0)>,
     'time_m': 0,
     'time_size': 3,
     'time_M': 1,
     'x_m': 0,
     'x_size': 4,
     'x_M': 3,
     'y_m': 0,
     'y_size': 4,
     'y_M': 3,
     'timers': <cparam 'P' (0x7f48f8d24790)>}

`'u'` stores a pointer to the allocated data; `'timers'` stores a pointer to a data structure used for C-level performance profiling.

One may want to replace some of these default arguments. For example, we could increase the minimum iteration point along the spatial Dimensions `x` and `y`, and execute only the very first timestep:

```python
#NBVAL_IGNORE_OUTPUT
u.data[:] = 0.  # Explicit reset to initial value
summary = op.apply(x_m=2, y_m=2, time_M=0)
```

    Operator `Kernel` run in 0.01 s

We look again at the computed data to convince ourselves that everything went as intended to go

```python
u.data
```

    Data([[[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]],

          [[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 1., 1.],
           [0., 0., 1., 1.]],

          [[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]]], dtype=float32)

Given a generic `Dimension` `d`, the naming convention is such that:

* `d_m` is the minimum iteration point
* `d_M` is the maximum iteration point

Hence, `op.apply(..., d_m=4, d_M=7, ...)` will run `op` in the compact interval `[4, 7]` along `d`. For historical reasons, `d=...` aliases to `d_M=...`; in many Devito examples it happens to see `op.apply(..., time=10, ...)` -- this is just equivalent to `op.apply(..., time_M=10, ...)`.

If we try to specify an invalid iteration extreme, Devito will raise an exception.

```python
from devito.exceptions import InvalidArgument
try:
    op.apply(time_M=2)
except InvalidArgument as e:
    print(e)
```

    OOB detected due to time_M=2

The same `Operator` can be applied to a different `TimeFunction`. For example:

```python
#NBVAL_IGNORE_OUTPUT
u2 = TimeFunction(name='u', grid=grid, save=5)
summary = op.apply(u=u2)
```

    Operator `Kernel` run in 0.01 s

```python
u2.data
```

    Data([[[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]],

          [[1., 1., 1., 1.],
           [1., 1., 1., 1.],
           [1., 1., 1., 1.],
           [1., 1., 1., 1.]],

          [[2., 2., 2., 2.],
           [2., 2., 2., 2.],
           [2., 2., 2., 2.],
           [2., 2., 2., 2.]],

          [[3., 3., 3., 3.],
           [3., 3., 3., 3.],
           [3., 3., 3., 3.],
           [3., 3., 3., 3.]],

          [[4., 4., 4., 4.],
           [4., 4., 4., 4.],
           [4., 4., 4., 4.],
           [4., 4., 4., 4.]]], dtype=float32)

Note that this was the third call to `op.apply`, but code generation and JIT-compilation only occurred upon the very first call.

There is one relevant case in which the maximum iteration point along the time dimension must be specified -- whenever `save` is unset, as in such a case the `Operator` wouldn't know for how many iterations to run.

```python
v = TimeFunction(name='v', grid=grid)
op2 = Operator(Eq(v.forward, v + 1))
try:
    op2.apply()
except ValueError as e:
    print(e)
```

    No value found for parameter time_M

```python
#NBVAL_IGNORE_OUTPUT
summary = op2.apply(time_M=4)
```

    Operator `Kernel` run in 0.01 s

```python
v.data
```

    Data([[[4., 4., 4., 4.],
           [4., 4., 4., 4.],
           [4., 4., 4., 4.],
           [4., 4., 4., 4.]],

          [[5., 5., 5., 5.],
           [5., 5., 5., 5.],
           [5., 5., 5., 5.],
           [5., 5., 5., 5.]]], dtype=float32)

The `summary` variable can be inspected to retrieve performance metrics.

```python
#NBVAL_IGNORE_OUTPUT
summary
```

    PerformanceSummary([(PerfKey(name='section0', rank=None),
                         PerfEntry(time=1e-06, gflopss=0.0, gpointss=0.0, oi=0.0, ops=0, itershapes=[]))])

We observe that basically all entries except for the execution time are fixed at 0. This is because by default Devito avoids to compute performance metrics, to minimize the processing time before returning control to the user (in complex `Operators`, the processing time to retrieve, for instance, the operation count or the memory footprint could be significant). To compute all performance metrics, a user could either export the environment variable `DEVITO_PROFILING` to `'advanced'` or change the profiling level programmatically *before* the `Operator` is constructed

```python
#NBVAL_IGNORE_OUTPUT
from devito import configuration
configuration['profiling'] = 'advanced'

op = Operator(Eq(u.forward, u*u + 1.))
op.apply()
```

    Operator `Kernel` run in 0.01 s

    PerformanceSummary([(PerfKey(name='section0', rank=None),
                         PerfEntry(time=1e-06, gflopss=0.064, gpointss=0.032, oi=0.16666666666666666, ops=2, itershapes=((2, 4, 4),)))])

A `PerformanceSummary` will contain as many entries as "sections" in the generated code. Currently, there is no way to automatically tie a compiler-generated section to the user-provided `Eq`s (in general, there can be more than one `Eq` in a section). The only option is to look at the generated code and search for bodies of code wrapped within C comments such as
```
/* Begin section0 */
<code>
/* End section0 \*/"
```
For example

```python
# Uncomment me and search for /* Begin section0 */ ... /* End section0 */
# print(op)
```

In the `PerformanceSummary`, associated to `section0` is a `PerfEntry`, whose entries represent:

* time: The time, in seconds, to execute the section.
* gflopss: Performance of the section in Gigaflops per second.
* gpointss: Performance of the section in Gigapoints per second.
* oi: Operational intensity.
* ops: Floating point operations at each (innermost loop) iteration.
* itershape: The shape of the iteration space.

# Excersie: Comparison against exact solution

Consider the following boundary value problem for the 1D wave equation:

\begin{align}
  \frac{1}{c^2}\frac{\partial^2 u}{\partial t^2}(x,t)-\frac{\partial^2 u}{\partial x^2}(x,t)&=0, \\
  u(0,t)=u(10,t)&= 0, \\
  \frac{\partial u}{\partial t}(x,0)&=0, \\
  u(x,0)&=
    \begin{cases}
      0, & 0 \leq x < 4,\\
      1, & 4 \leq x \leq 6,\\
      0, & 6 < x \leq 10.
    \end{cases}
\end{align}

The initial state is illustrated in the figure below:

It can be shown that the analytical solution to this problem is:
\begin{equation}
 u(x,t)=\sum_{n=1}^{\infty}A_{n}\sin\left(\frac{n\pi x}{10}\right)\cos\left(\frac{n\pi ct}{10}\right),
\end{equation}
where
\begin{equation}
  A_n=\frac{2}{n\pi}\left[\cos\left(\frac{4n\pi}{10}\right)-\cos\left(\frac{6n\pi}{10}\right)\right].
\end{equation}

Below is a snippet of code to plot the exact result at any point in time. You can use it to explore the evolution of the wave:

```python
from numpy import sin, cos, pi, linspace, shape, zeros, mod

import matplotlib.pyplot as plt

# Global constants
L = 10. # Define length of domain as a global variable
N = 2000 # Number of terms in the Fourier sine series
l = 1001 # Define number of points in domain
c = 1.0 # Speed of sound

x = linspace(0, L, l) # x axis for calling u(x, t) at given t

# Calculate exact solution

def u_e(x, t):
    s = zeros(x.size)
    for i in range(1,N+1):
        s+=A_n(i)*sin(i*pi*x/L)*cos(i*pi*c*t/L)
    return s

def A_n(n):
    const = 2/n/pi*(cos(4*n*pi/L)-cos(6*n*pi/L))
    return const
```

```python
%matplotlib inline
# Plot solution at a point T in time
T = 0
plt.plot(x, u_e(x, T))
plt.xlabel('x')
plt.ylabel('u(x,0)')
plt.show()
```

How would you write a Devito code to compute the evolution in time of the wave?

```python
# Some hints:

# Required imports
from devito import Grid, TimeFunction, Eq, solve, Operator

# Define the grid/computational domain
grid = Grid(shape=(l), extent=(L))
t_s = grid.stepping_dim

# Wavefield
u = TimeFunction(name='u', grid=grid, time_order=2, space_order=2)

dx = abs(x[1]-x[0])
# Critical time-step
dt = 0.004 # (dt < 0.42*dx/c)

# Set the initial conditions
u0 = u_e(x, 0)
u1 = u_e(x, dt)
u.data[0,:] = u0
u.data[1,:] = u1

# Boundary conditions can be defined by additional equations:
bc = [Eq(u[t_s+1, 0], 0)]
bc += [Eq(u[t_s+1, -1], 0)]

# Note -- Boundary conditions can be added to an operator as follows:
# op = Operator([eq]+bc)
```

See `nb2_exercise.ipynb` in additional the `additional_material` folder for a working solution.


# Complex biophysical models in NeuroDyn

## Overview

In the **NeuroDyn Python model** notebook we have seen how we can define Hodgkin-Huxley and NeuroDyn models, how to simulate them, as well as how to fit the parameters of the NeuroDyn model in order to replicate the biophysical data of the squid giant axon.

Although the forms of the NeuroDyn currents are fixed to the forms of the three currents of the original Hodgkin-Huxley paper, the chip offers full flexibility in defining the dynamics of the gating variables, as well as changing the conductance parameters such as maximal conductances and reversal potentials. This flexibility allows us to use the NeuroDyn equations to replicate different types of neurons. This notebook provides an introduction to understanding how more complex biological neurons can be replicated within the NeuroDyn architecture.

We start with a biological motivation, by considering the famous model of a bursting neuron Aplysia R-15 and how its dynamical structure leads to the generation of bursts of spikes. Next, we try to mimick this basic dynamical structure using purely an interconnection of Hodgkin-Huxley models. By using the fitting techniques shown in the previous notebook, this technique can be explored to design interconnections of NeuroDyn neurons that would showcase more complex dynamical behaviors such as neuronal bursting and post-inhibitory rebound.

## Neuronal bursting

Bursting is one of the fundamental signalling modes of neurons and is ubiquitous in many different types of neurons and neural networks. Importantly, generating a burst of spikes after a hyperpolarizing current pulse is a crucial mechanism for the generation of intrinsic rhythms in half-center oscillators, which themselves are the building blocks of more complex central pattern generating networks.

 Fig. 1: Bursting in subthalamic nucleus neuron (Beurrier, 1999)

Intrinsic generation of bursts of spikes stems from the interplay of fast processes that generate individual spikes and the slower processes that modulate the spiking behavior and define the burst duration.

We have seen in the previous notebook how NeuroDyn can be set up to replicate the spiking behavior of the Hodgkin-Huxley model. In contrast, bursting models of neurons are generally more complex, consisting of several additional ionic currents that modulate the slower burst-generating process. These currents are mostly calcium-dependent and act to initiate and terminate the fast spike-generating processes dictated by the sodium and potassium currents.

Since we cannot add additional ionic currents to a single NeuroDyn neuron, we can achieve a similar affect by *interconnecting* several NeuroDyn neurons through resistive connections or short circuits.

To motivate this, let's first look at an example model of bursting and understand how the slower currents compare to the faster currents of the Hodgkin-Huxley model.

### Example model (Aplysia R-15)

As an example, let's take a look at the Aplysia R-15 bursting neuronal model, a well-studied bursting neuron. The membrane equation has five ionic currents: the leak, sodium and potassium currents that are common to the Hodgkin-Huxley model, and in addition, an inward calcium current and an outward calcium-activated potassium current. The calcium current is slower than the potassium current and its activation has an *excitatory* effect, while the calcium-activated potassium current is significantly slower than this and has an *inhibitory* effect.

The calcium and the calcium-activated potassium can be thought of as playing the roles of the standard sodium and potassium currents respectively, but in a *slower* timescale. This way, we can think of these currents being responsible for *slow excitability*, i.e. the generation of slow spikes, in contrast to the Hodgkin-Huxley currents, which generate fast spikes and therefore give rise to the *fast excitability*.

This **fast + slow** structure is what we aim to recreate through the interconnection of basic Hodgkin-Huxley or NeuroDyn blocks.

Let's take a look at the model to get a clearer picture of this. The membrane equation has the following form:

\begin{equation}
C\frac{dV}{dt} = - I_l - I_{Na} - I_K - I_{Ca} - I_{K-Ca} + I_{app},
\end{equation}

The Hodgkin-Huxley currents have the same forms as previously discussed, while the two additional currents $I_{Ca}$ and $I_{K-Ca}$ are modelled as:

\begin{align}
\label{eq:calcium}
I_{Ca} &= \bar{g}_{Ca} \, x (V - E_{Ca})\\
\label{eq:calcium_activated_potassium}
I_{K-Ca} &= \bar{g}_{K-Ca} \, \frac{c}{0.5 + c} (V - E_K)
\end{align}

The calcium activation variable $x$ and the calcium concentration $c$ have the first-order dynamics:

\begin{align}
\label{eq:xdot}
\tau_x \dot{x} &= x_{\infty}(V) - x \\
\label{eq:cdot}
\tau_{c}\dot{c} &= (K_c x (E_{Ca} - V) - c)
\end{align}

First, let's check the time plot of the membrane equation to see how the neuron behaves with the default parameters:

```python
# Aplysia R-15 neuronal model
import matplotlib.pyplot as plt
import numpy as np
np.seterr(all="ignore")
from scipy.integrate import solve_ivp

# Alpha activation function for variable x
def alphafun(V, x):
    if (x == 'm'):
        V = np.asarray(V)
        y = np.zeros(V.shape)
        y[V != 50] = 0.1 * (50 - V) / (np.exp((50 - V)/10) - 1)
        y[V == 50] = 1

    if (x == 'h'):
        y = 0.07 * np.exp((25-V)/20)

    if (x == 'n'):
        V = np.asarray(V)
        y = np.zeros(V.shape)
        y[V != 55] = 0.01 * (55 - V) / (np.exp((55 - V)/10) - 1)
        y[V == 55] = 0.1

    return y

# Beta activation function for variable x
def betafun(V, x):
    if (x == 'm'):
        y = 4 * np.exp((25-V)/18)

    if (x == 'h'):
        y = 1 / (np.exp((55-V)/10) + 1)

    if (x == 'n'):
        y = 0.125 * np.exp((45-V)/80)

    return y

# x_inf(V) function for variable 'x'
def x_inf(V, x):
    # Define constants for gating variable 'x'
    A = 0.15
    B = -50

    # Shifted voltage constants
    C1 = 127 / 105
    C2 = 8265 / 105

    # Constant for Ca dynamics
    Kc = 0.0085

    if (x == 'x'):
        y = 1 / (np.exp(A * (B - V)) + 1)
    elif(x == 'c'):
        c = (Kc / (np.exp(A * (B - V)) + 1)) * (Eca - V)
        y = c / (0.5 + c)
    else:
        V = C1*V + C2
        y = alphafun(V,x) / (alphafun(V,x) + betafun(V,x))

    return y

# tau(V) function for variable 'x'
def tau(V, x):
    # Shifted voltage constants
    C1 = 127 / 105
    C2 = 8265 / 105

    V = np.asarray(V)

    if (x == 'x'):
        y = np.ones(V.shape) * 235
    elif (x == 'c'):
        y = np.ones(V.shape) * 1 / 0.003
    else:
        # If x == 'n' or x == 'h'
        V = C1*V + C2
        y = 12.5 / (alphafun(V,x) + betafun(V,x)) # includes *1/lambda = 12.5

    return y

# Calcium dynamics constants
rho = 0.0003
Kc = 0.0085

# Maximum conductances in mmho/ cm^2
gna = 4
gk = 0.3
gca = 0.004
gkca = 0.03
gl = 0.003

# Nernst potentials in mV
# Note: resting potential != 0mV, not normalized as in Hodgkin-Huxley
Ena = 30
Ek = -75
Eca = 140
El = -40

# Constant current stimulus (in uA / cm^2)
I0 = 0

def Iapp(t):
    return I0

# Length of simulation (in ms)
T = 80*1000
t = (0, T)

# Initial state x = [V0, h0, n0, x0, c0]
y0 = np.array([0, 0, 0, 0, 0])

# Dynamics of the model
def ode(t, y):
    V, h, n, x, c = y
    I = Iapp(t)

    dV = -gna * x_inf(V,'m')**3 * h * (V - Ena) - gca * x * (V - Eca) - (gk*n**4 + gkca * c / (0.5 + c)) * (V - Ek) -gl * (V - El) + I
    dh = (x_inf(V,'h') - h) / tau(V,'h')
    dn = (x_inf(V,'n') - n) / tau(V,'n')
    dx = (x_inf(V,'x') - x) / tau(V,'x')
    dc = rho * (Kc*x*(Eca - V) - c)

    dy = np.asarray([dV, dh, dn, dx, dc])

    return dy

# Simulate the model
sol = solve_ivp(ode, t, y0)

plt.figure()
plt.plot(sol.t, sol.y[0])
plt.show()
```

As we can see, the neuron generates a bursting waveform without an external stimulus. In order to get a better idea of the underlying dynamics, let's take a look at the gating variable forms. We want to get an understanding of the qualitative differences between the two slow currents $I_{Ca}$ and $I_{K-Ca}$ compared to the faster sodium and potassium currents.

Let's first look at the steady-state functions of the gating variables (note that the $I_{K-Ca}$ has a different form to other currents, but we can analyze it in a similar way).

```python
V = np.arange(Ek, Ena, 0.1)

plt.figure()
plt.plot(V, x_inf(V, 'm'), label='m')
plt.plot(V, x_inf(V, 'h'), label='h')
plt.plot(V, x_inf(V, 'n'), label='n')
plt.plot(V, x_inf(V, 'x'), label='x')
plt.plot(V, x_inf(V, 'c'), label='c / (0.5 + c)')

plt.legend(loc = 'lower right')
plt.show()
```

Notice that the activation of the calcium current $x$, and the activation of the calcium-activated potassium current $\frac{c}{0.5+c}$ activate at a lower voltage compared to the activation of sodium $m$ and activation of potassium $n$. Let's take a look at this seperately by comparing the activation of sodium with calcium, and potassium with calcium-activated potassium:

```python
plt.figure()
plt.plot(V, x_inf(V, 'm'), label='m')
plt.plot(V, x_inf(V, 'x'), label='x')
plt.legend()

plt.figure()
plt.plot(V, x_inf(V, 'n'), label='n')
plt.plot(V, x_inf(V, 'c'), label='c / (0.5 + c)')
plt.legend()

plt.show()
```

We can see that both slow gating variables activate at *lower* voltages than the faster gating variables of sodium and potassium. This ensures that the threshold of the slow spiking is lower than the threshold of the fast spiking.

Now, let's take a look at the time-constants of the gating variables. We can plot the time-constant functions on a log scale for a clearer overview:

```python
V = np.arange(Ek, Ena, 0.1)

plt.figure()
plt.semilogy(V, tau(V, 'm'), label='m')
plt.semilogy(V, tau(V, 'h'), label='h')
plt.semilogy(V, tau(V, 'n'), label='n')
plt.semilogy(V, tau(V, 'x'), label='x')
plt.semilogy(V, tau(V, 'c'), label='c')

plt.legend()
plt.show()
```

We can clearly see the separation of the timescales: the activation variables of the calcium and calcium-activated potassium are significantly slower than the gating variables of the Hodgkin-Huxley currents. Let's take a look again one by one, comparing sodium with calcium, and potassium with calcium-activated potassium:

```python
plt.figure()
plt.semilogy(V, tau(V, 'm'), label='m')
plt.semilogy(V, tau(V, 'x'), label='x')
plt.legend()

plt.figure()
plt.semilogy(V, tau(V, 'n'), label='n')
plt.semilogy(V, tau(V, 'c'), label='c')
plt.legend()

plt.show()
```

Summarizing, we can see that bursting can be achieved by adding additional currents to the Hodgkin-Huxley equation that have a similar effect as sodium and potassium, but with slower activations. These currents activate in a **lower voltage range**, ensuring that the slow threshold is below the threshold of fast spiking, and they activate on a **slower timescale**. Let's see if we can recreate this structure using only interconnections of Hodgkin-Huxley neurons, changing the parameters of the additional currents so that they retain these two important properties.

## Bursting through interconnection of Hodgkin-Huxley neurons

Let's see how we can construct a bursting neurons throgh the interconnection of Hodgkin-Huxley neurons. Similar to our biological example, we would like the first neuron to act as a standard Hodgkin-Huxley model, generating *fast excitability*. In turn, the second model needs to provide the *slow excitability*, so that the interconnection of the two will lead to the creation of bursts of spikes.

### Loading the model

First, let's load the model and the required modules:

```python
import matplotlib.pyplot as plt
from cb_models import HHModel, NeuroDynModel, ShortCircuit

# **Ignore overflow warnings**
import numpy as np
d = np.seterr(all="ignore")
```

### Short-circuiting neurons

Now let's start by defining two Hodgkin-Huxley neurons and short circuiting them. We can use the class `ShortCircuit` for this, which takes a list of neurons, and short-circuits them so that the membrane voltages of all the neurons are equal. Alternatively, we can create a network of neurons and connect them with a low-resistance connections, mirroring the structure of multiple compartment models. The network class is described in the notebook **Networks in NeuroDyn**.

Let's try this out on the default Hodgkin-Huxley models first. If we interconnect two identical neurons like this, the capacitance and all maximal conductance parameters will effectively double due to the parallel interconnection of all the elements. This means that by also doubling the injected current compared to a single Hodgkin-Huxley neuron, we would expect to see the same behavior. Let's try this out:

```python
neuron1 = HHModel()
neuron2 = HHModel()

neuron_connection = ShortCircuit([neuron1, neuron2])

T = 200
t = (0, T)

I0 = 8
iapp1 = lambda t: I0
iapp2 = lambda t: 2*I0

x01 = [0,0,0,0]
x02 = x01 + [0,0,0]

sol1 = neuron1.simulate(t, x01, iapp1)
sol2 = neuron_connection.simulate(t, x02, iapp2)

plt.figure()
plt.plot(sol1.t, sol1.y[0])
plt.plot(sol2.t, sol2.y[0])
plt.show()
```

We see that the interconnection seems to work - we get the same behavior with the two-neuron interconnection as with the original neuron!

Note that we need to pass an initial state vector of the appropriate size: in this case we had two neurons with six gating variable dynamics in total, and a single membrane voltage.

### Setting the parameters of the second neuron

Let's see now if by appropriately changing the parameters of the second neuron, we can get the interconnection to burst. To do this, we can follow the underlying principles discussed for the Aplysia R-15 model.

#### Turning off sodium inactivation

To start, we can fix the sodium inactivation to $1$ so that it resembles more the calcium current with a single activation variable. This can be done by setting $\beta_h(V) = 0$ since then the $h$ equation becomes

\begin{equation}
\dot{h} = \alpha_h(V) (1 - h)
\end{equation}

so that the steady-state value is $h = 1$ for all $V$. We can do this by setting the parameter `bA` of the gating variable to `0`.

```python
# Set h = 1
V = np.arange(neuron1.Ek, neuron1.Ena, 0.1)

neuron2.h.bA = 0

plt.figure()
plt.title("$h_{\infty}(V)$")
plt.plot(V, neuron2.h.inf(V))
plt.show()
```

#### Shifting the steady-state functions of $m$ and $n$

Now, let's shift the steady-state functions of sodium and potassium activation to a lower voltage range, so that the threshold voltage is also shifted as a result. Remember that

\begin{equation}
m_{\infty}(V) = \frac{\alpha_m(V)}{\alpha_m(V) + \beta_m(V)}
\end{equation}

so that adding the same voltage shift to $\alpha_m(V + \delta V)$ and $\beta_m(V + \delta V)$ functions will shift $m_{\infty}(V + \delta V)$ by the same amount.

We can do that by changing the parameter $V_h$ of the $m$ and $n$ variables, which represents the half-activation voltage. We need to change this parameter for both $\alpha(V)$ and $\beta(V)$ functions, so the corresponding variable names are `aVh` and `bVh`. We can then apply this to both $m$ and $n$ gating variables.

```python
# Shift m and n for neuron 2
vshift = 20

# Shift m
neuron2.m.aVh = neuron1.m.aVh - vshift
neuron2.m.bVh = neuron1.m.bVh - vshift

# Shift n
neuron2.n.aVh = neuron1.n.aVh - vshift
neuron2.n.bVh = neuron1.n.bVh - vshift

plt.figure()
plt.plot(V, neuron1.m.inf(V), label = 'm1')
plt.plot(V, neuron2.m.inf(V), label = 'm2')
plt.legend()
plt.show()

plt.figure()
plt.plot(V, neuron1.n.inf(V), label = 'n1')
plt.plot(V, neuron2.n.inf(V), label = 'n2')
plt.legend()
plt.show()
```

#### Scaling the time constants of $m$ and $n$

The changes we have done so far had no effect on the timescales of the gating variables $m$ and $n$ and purely affected the steady-state functions. Let's now scale the time constants of the activation variables so that they activate more slowly compared to the original Hodgkin-Huxley currents. Remember that the time constant functions are defined as

\begin{equation}
\tau(V) = \frac{1}{\alpha(V) + \beta(V)}
\end{equation}

so that for example multiplying both $\alpha(V)$ and $\beta(V)$ by $0.1$ will make the time constant $10$ times larger.

We can again do this by scaling the gain constants of the $\alpha(V)$ and $\beta(V)$ functions, represented by the parameters `aA` and `bA` within the gating variables.

```python
# Scaling the time constants for m and n in neuron 2
t_scale1 = 15
t_scale2 = 30

# Scale the timescale for neuron 2 m variable
neuron2.m.aA = neuron1.m.aA / t_scale1
neuron2.m.bA = neuron1.m.bA / t_scale1

# Scale the timescale for neuron 2 n variable
neuron2.n.aA = neuron1.n.aA / t_scale2
neuron2.n.bA = neuron1.n.bA / t_scale2

plt.figure()
plt.semilogy(V, neuron1.m.tau(V), label = 'm1')
plt.semilogy(V, neuron2.m.tau(V), label = 'm2')
plt.legend()
plt.show()

plt.figure()
plt.semilogy(V, neuron1.n.tau(V), label = 'n1')
plt.semilogy(V, neuron2.n.tau(V), label = 'n2')
plt.legend()
plt.show()
```

#### Setting the maximal conductances and reversal potentials

Finally, we need to change the maximal conductance parameters, as well as the reversal potentials.

Firstly, we can set the maximal conductance of the leak to $g_l = 0$ since the leak current can be solely provided by the first neuron. Secondly, we can set the reversal potential for the sodium current to be higher, so that it reflects the calcium reversal potential for which $E_{Ca} > E_{Na}$. For the potassium current, we keep the same reversal potential as we are trying to emulate a slow *potassium* current.

The maximal conductance parameters of the sodium and potassium we can use as the control variables in order to tune the bursting oscillation.

```python
# Turn off leak current for neuron 2
neuron2.gl = 0

# Set reversal potential for neuron 2 sodium to calcium reversal
neuron2.Ena = 200

# Change maximal conductances of sodium and potassium for neuron 2
neuron2.gna = 1
neuron2.gk = 3
```

#### Simulating the interconnection

Let's now look at how the system behaves over time. We can go back to the previous sections and change the parameters to see how it changes the oscillating behavior.

```python
# Simulate HH short-circuit

T = 2000
t = (0, T)

I0 = -5
iapp = lambda t: I0

x0 = [0,0,0,0,0,0,0]

sol = neuron_connection.simulate(t, x0, iapp)

plt.figure()
plt.plot(sol.t, sol.y[0])
plt.show()
```

We should now observe that the interconnected system exhibits intrinsic bursting! You can try changing the parameters of the two slow currents in order to see how they affect the bursting behavior, as well as try obtaining different types of bursting waveforms.

### Post-inhibitory rebound

An important consequence of the internal burst-generating dynamics is what is called a **post-inhibitory rebound** (PIR). This is a phenomenon where an initially silent neuron can fire a transient burst of spikes after a prolonged period of hyperpolarization. This mechanism is fundamental for the generation of sustained oscillations in half-center oscillators, where mutually inhibiting neurons hyperpolarize each other as a result of firing a burst of spikes.

Let's see how our model behaves in response to a long hyperpolarizing pulse:

```python
# Post-inhibitory rebound

def pulse(t, t_start, t_length, mag):
    I = (t>=t_start)*mag - (t>=(t_start+t_length))*mag
    return I

T = 3000
t = (0, T)

I0 = -6
pulse_mag = 2

def Iapp(t):
    t_start = T / 2
    t_length = T / 5
    return I0 - pulse(t, t_start, t_length, pulse_mag)

x0 = [0,0,0,0,0,0,0]

sol = neuron_connection.simulate(t, x0, Iapp)

plt.figure()
plt.plot(sol.t, sol.y[0])
plt.show()
```

We can try different values of the pulse magnitude - you should notice that for a small pulse the system will generate a small rebound bump. In this case, the rebound is not large enough to activate the fast spiking dynamics and the system goes back to rest after the transient. However, if the pulse is large enough the rebound will move the system through the spiking regime before returning to rest, thus creating a transient burst. We can use the post-inhibitory rebound as the basic mechanism for generating bursting oscillations in inhibitory networks.

## Next step: bursting through interconnection of NeuroDyn circuits

The procedure discussed so far can be used as a first step to designing more complex neuronal behaviors in NeuroDyn hardware. By using the fitting procedures described in the **NeuroDyn Python model** notebook, we can first design an interconnection of Hodgkin-Huxley models that would give us the desired behavior of the system, and then translate the Hodgkin-Huxley parameters to the parameters that can be set within the hardware.

The additional challenge here is ensuring that the parameters can be fitted within the physical constraints. Due to this, we might find that certain regimes may not be directly translated to the NeuroDyn equations. Importantly, remember that the current $I_{master}$ in NeuroDyn determines the effective scale of the maximal conductances and $\alpha$ and $\beta$ coefficients. By decreasing the $I_{master}$ of a neuron, its time-scale is also effictively decreased, and we can use this to design the slower neuron.

```python

```

Complex biophysical models in NeuroDyn.ipynb


[Table of Contents](./table_of_contents.ipynb)

# Designing Nonlinear Kalman Filters

```python
#format the book
%matplotlib inline
from __future__ import division, print_function
from book_format import load_style
load_style()
```

<style>
@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');
@import url('http://fonts.googleapis.com/css?family=Lora');

.CodeMirror pre {
    font-family: 'Source Code Pro', Consolas, monocco, monospace;
}
    div.cell{
        //width: 950px;
        margin-left: 0% !important;
        margin-right: auto;
    }
    div.text_cell_render{
        font-family: 'Lora';
        line-height: 125%;
        font-size: 100%;
        text-align: justify;
        text-justify:inter-word;
    }
    div.text_cell code {
        background: transparent;
        color: #000000;
        font-weight: 400;
        font-size: 11pt;
        font-family:  'Source Code Pro', Consolas, monocco, monospace;
   }
    h1 {
        font-family: 'Open sans',verdana,arial,sans-serif;
	}

    div.input_area {
        background: #F6F6F9;
        border: 1px solid #586e75;
    }

    .text_cell_render h1 {
        font-weight: 200;
        font-size: 30pt;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h2 {
        font-family: 'Open sans',verdana,arial,sans-serif;
        text-align: left;
    }
    .text_cell_render h2 {
        font-weight: 200;
        font-size: 16pt;
        font-style: italic;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1.5em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h3 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h3 {
        font-weight: 200;
        font-size: 14pt;
        line-height: 100%;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 2em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h4 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h4 {
        font-weight: 100;
        font-size: 14pt;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
        white-space: nowrap;
    }
    h5 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }

    .text_cell_render h5 {
        font-weight: 200;
        font-style: normal;
        color: #1d3b84;
        font-size: 16pt;
        margin-bottom: 0em;
        margin-top: 0.5em;
        display: block;
        white-space: nowrap;
    }
    div.output_subarea.output_text.output_pyout {
        overflow-x: auto;
        overflow-y: visible;
        max-height: 5000000px;
    }
    div.output_subarea.output_stream.output_stdout.output_text {
        overflow-x: auto;
        overflow-y: visible;
        max-height: 5000000px;
    }
    div.output_wrapper{
        margin-top:0.2em;
        margin-bottom:0.2em;
}

    code{
        font-size: 6pt;

    }
    .rendered_html code{
    background-color: transparent;
    }
    ul{
        margin: 2em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li li{
        padding-left: 0.2em;
        margin-bottom: 0.2em;
        margin-top: 0.2em;
    }
    ol{
        margin: 2em;
    }
    ol li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.2em;
    }
    a:link{
       color:#447adb;
    }
    a:visited{
       color: #1d3b84;
    }
    a:hover{
       color: #1d3b84;
    }
    a:focus{
       color:#447adb;
    }
    a:active{
       font-weight: bold;
       color:#447adb;
    }
    .rendered_html :link {
       text-decoration: underline;
    }
    .rendered_html :hover {
       text-decoration: none;
    }
    .rendered_html :visited {
      text-decoration: none;
    }
    .rendered_html :focus {
      text-decoration: none;
    }
    .rendered_html :active {
      text-decoration: none;
    }
    .warning{
        color: rgb( 240, 20, 20 )
    }
    hr {
      color: #f3f3f3;
      background-color: #f3f3f3;
      height: 1px;
    }
    blockquote{
      display:block;
      background: #fcfcfc;
      border-left: 5px solid #c76c0c;
      font-family: 'Open sans',verdana,arial,sans-serif;
      width:680px;
      padding: 10px 10px 10px 10px;
      text-align:justify;
      text-justify:inter-word;
      }
      blockquote p {
        margin-bottom: 0;
        line-height: 125%;
        font-size: 100%;
      }
</style>

## Introduction

** Author's note: I was initially planning to have a design nonlinear chapter that compares various approaches. This may or may not happen, but for now this chapter has no useful content and I suggest not reading it. **

We see that the Kalman filter reasonably tracks the ball. However, as already explained, this is a silly example; we can predict trajectories in a vacuum with arbitrary precision; using a Kalman filter in this example is a needless complication.

### Kalman Filter with Air Drag

I will dispense with the step 1, step 2, type approach and proceed in a more natural style that you would use in a non-toy engineering problem. We have already developed a Kalman filter that does excellently at tracking a ball in a vacuum, but that does not incorporate the effects of air drag into the model. We know that the process model is implemented with $\textbf{F}$, so we will turn our attention to that immediately.

Notionally, the computation that $\textbf{F}$ computes is

$$x' = Fx$$

With no air drag, we had

$$
\mathbf{F} = \begin{bmatrix}
1 & \Delta t & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 \\
0 & 0 & 1 & \Delta t & \frac{1}{2}{\Delta t}^2 \\
0 & 0 & 0 & 1 & \Delta t \\
0 & 0 & 0 & 0 & 1
\end{bmatrix}
$$

which corresponds to the equations

$$
\begin{aligned}
x &= x + v_x \Delta t \\
v_x &= v_x \\
\\
y &= y + v_y \Delta t + \frac{a_y}{2} {\Delta t}^2 \\
v_y &= v_y + a_y \Delta t \\
a_y &= a_y
\end{aligned}
$$

From the section above we know that our new Euler equations must be

$$
\begin{aligned}
x &= x + v_x \Delta t \\
v_x &= v_x \\
\\
y &= y + v_y \Delta t + \frac{a_y}{2} {\Delta t}^2 \\
v_y &= v_y + a_y \Delta t \\
a_y &= a_y
\end{aligned}
$$

## Realistic 2D Position Sensors

The position sensor in the last example are not very realistic. In general there is no 'raw' sensor that provides (x,y) coordinates. We have GPS, but GPS already uses a Kalman filter to create a filtered output; we should not be able to improve the signal by passing it through another Kalman filter unless we incorporate additional sensors to provide additional information. We will tackle that problem later.

Consider the following set up. In an open field we put two transmitters at a known location, each transmitting a signal that we can detect. We process the signal and determine  how far we are from that signal, with some noise. First, let's look at a visual depiction of that.

```python
import matplotlib.pyplot as plt

circle1=plt.Circle((-4, 0), 5, color='#004080',
                   fill=False, linewidth=20, alpha=.7)
circle2=plt.Circle((4, 0), 5, color='#E24A33',
                   fill=False, linewidth=5, alpha=.7)

fig = plt.gcf()
ax = fig.gca()

plt.axis('equal')
plt.xlim((-10, 10))
plt.ylim((-10, 10))

plt.plot ([-4, 0], [0, 3], c='#004080')
plt.plot ([4, 0], [0, 3], c='#E24A33')
plt.text(-4, -.5, "A", fontsize=16, horizontalalignment='center')
plt.text(4, -.5, "B", fontsize=16, horizontalalignment='center')

ax.add_artist(circle1)
ax.add_artist(circle2)
plt.show()
```

Here I have attempted to show transmitter A, drawn in red, at (-4,0) and a second one B, drawn in blue, at (4,0). The red and blue circles show the range from the transmitters to the robot, with the width illustrating the effect of the $1\sigma$ angular error for each transmitter. Here I have given the blue transmitter more error than the red one. The most probable position for the robot is where the two circles intersect, which I have depicted with the red and blue lines. You will object that we have two intersections, not one, but we will see how we deal with that when we design the measurement function.

This is a very common sensor set up. Aircraft still use this system to navigate, where it is called DME (Distance Measuring Equipment). Today GPS is a much more common navigation system, but I have worked on an aircraft where we integrated sensors like this into our filter along with the GPS, INS, altimeters, etc. We will tackle what is called *multi-sensor fusion* later; for now we will just address this simple configuration.

The first step is to design our state variables. We will assume that the robot is traveling in a straight direction with constant velocity. This is unlikely to be true for a long period of time, but is acceptable for short periods of time. This does not differ from the previous problem - we will want to track the values for the robot's position and velocity. Hence,

$$\mathbf{x} =
\begin{bmatrix}x\\v_x\\y\\v_y\end{bmatrix}$$

The next step is to design the state transition function. This also will be the same as the previous problem, so without further ado,

$$
\mathbf{x}' = \begin{bmatrix}1& \Delta t& 0& 0\\0& 1& 0& 0\\0& 0& 1& \Delta t\\ 0& 0& 0& 1\end{bmatrix}\mathbf{x}$$

The next step is to design the control inputs. We have none, so we set ${\mathbf{B}}=0$.

The next step is to design the measurement function $\mathbf{z} = \mathbf{Hx}$. We can model the measurement using the Pythagorean theorem.

$$
z_a = \sqrt{(x-x_A)^2 + (y-y_A)^2} + v_a\\[1em]
z_b = \sqrt{(x-x_B])^2 + (y-y_B)^2} + v_b
$$

where $v_a$ and $v_b$ are white noise.

We see an immediate problem. The Kalman filter is designed for linear equations, and this is obviously nonlinear. In the next chapters we will look at several ways to handle nonlinear problems in a robust way, but for now we will do something simpler. If we know the approximate position of the robot than we can linearize these equations around that point. I could develop the generalized mathematics for this technique now, but instead let me just present the worked example to give context to that development.

Instead of computing $\mathbf{H}$ we will compute the partial derivative of $\mathbf{H}$ with respect to the robot's position $\mathbf{x}$. You are probably familiar with the concept of partial derivative, but if not, it just means how $\mathbf{H}$ changes with respect to the robot's position. It is computed as the partial derivative of $\mathbf{H}$ as follows:

$$\frac{\partial \mathbf{h}}{\partial \mathbf{x}} =
\begin{bmatrix}
\frac{\partial h_1}{\partial x_1} & \frac{\partial h_1}{\partial x_2} &\dots \\
\frac{\partial h_2}{\partial x_1} & \frac{\partial h_2}{\partial x_2} &\dots \\
\vdots & \vdots
\end{bmatrix}
$$

Let's work the first partial derivative. We want to find

$$\frac{\partial }{\partial x} \sqrt{(x-x_A)^2 + (y-y_A)^2}
$$

Which we compute as

$$
\begin{aligned}
\frac{\partial h_1}{\partial x} &= ((x-x_A)^2 + (y-y_A)^2))^\frac{1}{2} \\
&= \frac{1}{2}\times 2(x-x_a)\times ((x-x_A)^2 + (y-y_A)^2))^{-\frac{1}{2}} \\
&= \frac{x_r - x_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}}
\end{aligned}
$$

We continue this computation for the partial derivatives of the two distance equations with respect to $x$, $y$, $dx$ and $dy$, yielding

$$\frac{\partial\mathbf{h}}{\partial\mathbf{x}}=
\begin{bmatrix}
\frac{x_r - x_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}} & 0 &
\frac{y_r - y_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}} & 0 \\
\frac{x_r - x_B}{\sqrt{(x_r-x_B)^2 + (y_r-y_B)^2}} & 0 &
\frac{y_r - y_B}{\sqrt{(x_r-x_B)^2 + (y_r-y_B)^2}} & 0 \\
\end{bmatrix}
$$

That is pretty painful, and these are very simple equations. Computing the Jacobian can be extremely difficult or even impossible for more complicated systems. However, there is an easy way to get Python to do the work for you by using the SymPy module [1]. SymPy is a Python library for symbolic mathematics. The full scope of its abilities are beyond this book, but it can perform algebra, integrate and differentiate equations, find solutions to differential equations, and much more. We will use it to compute our Jacobian!

First, a simple example. We will import SymPy, initialize its pretty print functionality (which will print equations using LaTeX). We will then declare a symbol for NumPy to use.

```python
import sympy
from sympy import init_printing
init_printing(use_latex='png')

phi, x = sympy.symbols('\phi, x')
phi
```

Notice how we use a latex expression for the symbol `phi`. This is not necessary, but if you do it will render as LaTeX when output. Now let's do some math. What is the derivative of $\sqrt{\phi}$?

```python
sympy.diff('sqrt(phi)')
```

We can factor equations.

```python
sympy.factor('phi**3 -phi**2 + phi - 1')
```

SymPy has a remarkable list of features, and as much as I enjoy exercising its features we cannot cover them all here. Instead, let's compute our Jacobian.

```python
from sympy import symbols, Matrix
phi = symbols('\phi')
phi

x, y, xa, xb, ya, yb, dx, dy = symbols('x y x_a x_b y_a y_b dx dy')

H = Matrix([[sympy.sqrt((x-xa)**2 + (y-ya)**2)],
            [sympy.sqrt((x-xb)**2 + (y-yb)**2)]])

state = Matrix([x, dx, y, dy])
H.jacobian(state)
```

In a nutshell, the entry (0,0) contains the difference between the x coordinate of the robot and transmitter A's x coordinate divided by the distance between the robot and A. (2,0) contains the same, except for the y coordinates of the robot and transmitters. The bottom row contains the same computations, except for transmitter B. The 0 entries account for the velocity components of the state variables; naturally the range does not provide us with velocity.

The values in this matrix change as the robot's position changes, so this is no longer a constant; we will have to recompute it for every time step of the filter.

If you look at this you may realize that this is just a computation of x/dist and y/dist, so we can switch this to a trigonometic form with no loss of generality:

$$\frac{\partial\mathbf{h}}{\partial\mathbf{x}}=
\begin{bmatrix}
-\cos{\theta_A} & 0 & -\sin{\theta_A} & 0 \\
-\cos{\theta_B} & 0 & -\sin{\theta_B} & 0
\end{bmatrix}
$$

However, this raises a huge problem. We are no longer computing $\mathbf{H}$, but $\Delta\mathbf{H}$, the change of $\mathbf{H}$. If we passed this into our Kalman filter without altering the rest of the design the output would be nonsense. Recall, for example, that we multiply $\mathbf{Hx}$ to generate the measurements that would result from the given estimate of $\mathbf{x}$ But now that $\mathbf{H}$ is linearized around our position it contains the *change* in the measurement function.

We are forced, therefore, to use the *change* in $\mathbf{x}$ for our state variables. So we have to go back and redesign our state variables.

>Please note this is a completely normal occurrence in designing Kalman filters. The textbooks present examples like this as *fait accompli*, as if it is trivially obvious that the state variables needed to be velocities, not positions. Perhaps once you do enough of these problems it would be trivially obvious, but at that point why are you reading a textbook? I find myself reading through a presentation multiple times, trying to figure out why they made a choice, finally to realize that it is because of the consequences of something on the next page. My presentation is longer, but it reflects what actually happens when you design a filter. You make what seem reasonable design choices, and as you move forward you discover properties that require you to recast your earlier steps. As a result, I am going to somewhat abandon my **step 1**, **step 2**, etc.,  approach, since so many real problems are not quite that straightforward.

If our state variables contain the velocities of the robot and not the position then how do we track where the robot is? We can't. Kalman filters that are linearized in this fashion use what is called a *nominal trajectory* - i.e. you assume a position and track direction, and then apply the changes in velocity and acceleration to compute the changes in that trajectory. How could it be otherwise? Recall the graphic showing the intersection of the two range circles - there are two areas of intersection. Think of what this would look like if the two transmitters were very close to each other - the intersections would be two very long crescent shapes. This Kalman filter, as designed, has no way of knowing your true position from only distance measurements to the transmitters. Perhaps your mind is already leaping to ways of working around this problem. If so, stay engaged, as later sections and chapters will provide you with these techniques. Presenting the full solution all at once leads to more confusion than insight, in my opinion.

So let's redesign our *state transition function*. We are assuming constant velocity and no acceleration, giving state equations of
$$
\dot{x}' = \dot{x} \\
\ddot{x}' = 0 \\
\dot{y}' = \dot{y} \\
\dot{y}' = 0$$

This gives us the the *state transition function* of

$$
\mathbf{F} = \begin{bmatrix}0 &1 & 0& 0\\0& 0& 0& 0\\0& 0& 0& 1\\ 0& 0& 0& 0\end{bmatrix}$$

A final complication comes from the measurements that we pass in. $\mathbf{Hx}$ is now computing the *change* in the measurement from our nominal position, so the measurement that we pass in needs to be not the range to A and B, but the *change* in range from our measured range to our nominal position.

There is a lot here to take in, so let's work through the code bit by bit. First we will define a function to compute $\frac{\partial\mathbf{h}}{\partial\mathbf{x}}$ for each time step.

```python
from math import sin, cos, atan2

def H_of(pos, pos_A, pos_B):
    """ Given the position of our object at 'pos' in 2D, and two
    transmitters A and B at positions 'pos_A' and 'pos_B', return
    the partial derivative of H
    """

    theta_a = atan2(pos_a[1] - pos[1], pos_a[0] - pos[0])
    theta_b = atan2(pos_b[1] - pos[1], pos_b[0] - pos[0])

    return np.array([[0, -cos(theta_a), 0, -sin(theta_a)],
                     [0, -cos(theta_b), 0, -sin(theta_b)]])
```

Now we need to create our simulated sensor.

```python
from numpy.random import randn

class DMESensor(object):
    def __init__(self, pos_a, pos_b, noise_factor=1.0):
        self.A = pos_a
        self.B = pos_b
        self.noise_factor = noise_factor

    def range_of(self, pos):
        """ returns tuple containing noisy range data to A and B
        given a position 'pos'
        """

        ra = math.sqrt((self.A[0] - pos[0])**2 + (self.A[1] - pos[1])**2)
        rb = math.sqrt((self.B[0] - pos[0])**2 + (self.B[1] - pos[1])**2)

        return (ra + randn()*self.noise_factor,
                rb + randn()*self.noise_factor)
```

Finally, we are ready for the Kalman filter code. I will position the transmitters at x=-100 and 100, both with y=-20. This gives me enough space to get good triangulation from both as the robot moves. I will start the robot at (0,0) and move by (1,1) each time step.

```python
import kf_book.book_plots as bp
from filterpy.kalman import KalmanFilter
import math
import numpy as np

pos_a = (100, -20)
pos_b = (-100, -20)

f1 = KalmanFilter(dim_x=4, dim_z=2)

f1.F = np.array ([[0, 1, 0, 0],
                  [0, 0, 0, 0],
                  [0, 0, 0, 1],
                  [0, 0, 0, 0]], dtype=float)

f1.R *= 1.
f1.Q *= .1

f1.x = np.array([[1, 0, 1, 0]], dtype=float).T
f1.P = np.eye(4) * 5.

# initialize storage and other variables for the run
count = 30
xs, ys = [], []
pxs, pys = [], []

# create the simulated sensor
d = DMESensor(pos_a, pos_b, noise_factor=3.)

# pos will contain our nominal position since the filter does not
# maintain position.
pos = [0, 0]

for i in range(count):
    # move (1,1) each step, so just use i
    pos = [i, i]

    # compute the difference in range between the nominal track
    # and measured ranges
    ra,rb = d.range_of(pos)
    rx,ry = d.range_of((pos[0] + f1.x[0, 0], pos[1] + f1.x[2, 0]))
    z = np.array([[ra - rx], [rb - ry]])

    # compute linearized H for this time step
    f1.H = H_of (pos, pos_a, pos_b)

    # store stuff so we can plot it later
    xs.append(f1.x[0, 0]+i)
    ys.append(f1.x[2, 0]+i)
    pxs.append(pos[0])
    pys.append(pos[1])

    # perform the Kalman filter steps
    f1.predict()
    f1.update(z)

bp.plot_filter(xs, ys)
bp.plot_track(pxs, pys)
plt.legend(loc=2)
plt.show()
```

## Linearizing the Kalman Filter

Now that we have seen an example of linearizing the Kalman filter we are in a position to better understand the math.

We start by assuming some function $\mathbf f$

## Example: A falling Ball

**author's note: ignore this section for now. **

In the **Designing Kalman Filters** chapter I first considered tracking a ball in a vacuum, and then in the atmosphere. The Kalman filter performed very well for vacuum, but diverged from the ball's path in the atmosphere. Let us look at the output; to avoid littering this chapter with code from that chapter I have placed it all in the file `ekf_internal.py'.

```python
import kf_book.ekf_internal as ekf
ekf.plot_ball()
```

We can artificially force the Kalman filter to track the ball by making $Q$ large. That would cause the filter to mistrust its prediction, and scale the kalman gain $K$ to strongly favor the measurments. However, this is not a valid approach. If the Kalman filter is correctly predicting the process we should not 'lie' to the filter by telling it there are process errors that do not exist. We may get away with that for some problems, in some conditions, but in general the Kalman filter's performance will be substandard.

Recall from the **Designing Kalman Filters** chapter that the acceleration is

$$a_x = (0.0039 + \frac{0.0058}{1+\exp{[(v-35)/5]}})*v*v_x \\
a_y = (0.0039 + \frac{0.0058}{1+\exp{[(v-35)/5]}})*v*v_y- g
$$

These equations will be *very* unpleasant to work with while we develop this subject, so for now I will retreat to a simpler one dimensional problem using this simplified equation for acceleration that does not take the nonlinearity of the drag coefficient into account:

$$\begin{aligned}
\ddot{y} &= \frac{0.0034ge^{-y/20000}\dot{y}^2}{2\beta} - g \\
\ddot{x} &= \frac{0.0034ge^{-x/20000}\dot{x}^2}{2\beta}
\end{aligned}$$

Here $\beta$ is the ballistic coefficient, where a high number indicates a low drag.

This is still nonlinear, so we need to linearize this equation at the current state point. If our state is position and velocity, we need an equation for some arbitrarily small change in $\mathbf{x}$, like so:

$$ \begin{bmatrix}\Delta \dot{x} \\ \Delta \ddot{x} \\ \Delta \dot{y} \\ \Delta \ddot{y}\end{bmatrix} =
\large\begin{bmatrix}
\frac{\partial \dot{x}}{\partial x} &
\frac{\partial \dot{x}}{\partial \dot{x}} &
\frac{\partial \dot{x}}{\partial y} &
\frac{\partial \dot{x}}{\partial \dot{y}} \\
\frac{\partial \ddot{x}}{\partial x} &
\frac{\partial \ddot{x}}{\partial \dot{x}}&
\frac{\partial \ddot{x}}{\partial y}&
\frac{\partial \dot{x}}{\partial \dot{y}}\\
\frac{\partial \dot{y}}{\partial x} &
\frac{\partial \dot{y}}{\partial \dot{x}} &
\frac{\partial \dot{y}}{\partial y} &
\frac{\partial \dot{y}}{\partial \dot{y}} \\
\frac{\partial \ddot{y}}{\partial x} &
\frac{\partial \ddot{y}}{\partial \dot{x}}&
\frac{\partial \ddot{y}}{\partial y}&
\frac{\partial \dot{y}}{\partial \dot{y}}
\end{bmatrix}\normalsize
\begin{bmatrix}\Delta x \\ \Delta \dot{x} \\ \Delta \dot{y} \\ \Delta \ddot{y}\end{bmatrix}$$

The equations do not contain both an x and a y, so any partial derivative with both in it must be equal to zero. We also know that $\large\frac{\partial \dot{x}}{\partial x}\normalsize = 0$ and that  $\large\frac{\partial \dot{x}}{\partial \dot{x}}\normalsize = 1$, so our matrix ends up being

$$\mathbf{F} = \begin{bmatrix}0&1&0&0 \\
\frac{0.0034e^{-x/22000}\dot{x}^2g}{44000\beta}&0&0&0
\end{bmatrix}$$

$$\begin{aligned}\ddot{x} &= -\frac{1}{2}C_d\rho A \dot{x}\\
\ddot{y} &= -\frac{1}{2}C_d\rho A \dot{y}-g\end{aligned}$$

```python
from sympy.abc import *
from sympy import *

init_printing(pretty_print=True, use_latex='mathjax')

x1 = (0.0034*g*exp(-x/22000)*((x)**2))/(2*b) - g

x2 = (a*g*exp(-x/c)*(Derivative(x)**2))/(2*b) - g

#pprint(x1)
#pprint(Derivative(x)*Derivative(x,n=2))
#pprint(diff(x2, x))
```

** orphan text
This approach has many issues. First, of course, is the fact that the linearization does not produce an exact answer. More importantly, we are not linearizing the actual path, but our filter's estimation of the path. We linearize the estimation because it is statistically likely to be correct; but of course it is not required to be. So if the filter's output is bad that will cause us to linearize an incorrect estimate, which will almost certainly lead to an even worse estimate. In these cases the filter will quickly diverge. This is where the 'black art' of Kalman filter comes in. We are trying to linearize an estimate, and there is no guarantee that the filter will be stable. A vast amount of the literature on Kalman filters is devoted to this problem. Another issue is that we need to linearize the system using analytic methods. It may be difficult or impossible to find an analytic solution to some problems. In other cases we may be able to find the linearization, but the computation is very expensive. **

## References

[1] http://sympy.org


# Class Reconstruction
## Introduction
Classification is the problem of taking an unknown datum $\vec{x} \in \mathbb{R}^N$ and assigning it to a class label $c \in \mathcal{C}$, where $\mathcal{C}$ is the set of discrete class labels.  In this instance, $\vec{x}$ is assumed to exist in a high-dimensional feature space, though this is not a requirement.  If $\mathcal{C}$ is known in advance then this problem falls under the domain of supervised learning.  Now, there are [many ways](http://scikit-learn.org/stable/supervised_learning.html#supervised-learning) to perform supervised learning but this notebook will focus on training a relatively simple neural network using PyTorch.

One way to view classification is as the non-linear projection of some high-dimensional space to a lower-dimensional one.  The projection is done in a way such that regions in the high-dimensional space are associated with a specific coordinate in the lower-dimensional space.  For example, if $\mathcal{C} = \{c_0, c_1, \dots, c_{L-1} \}$, where $L$ is the number of labels, then a vector in the "class space" can be $\vec{c}_0 = \begin{bmatrix} 1 & 0 & \dots & 0 \end{bmatrix}^T$, $\vec{c}_0 = \begin{bmatrix} 0 & 1 & \dots & 0 \end{bmatrix}^T$ and so on.  This is known as [one-hot](https://en.wikipedia.org/wiki/One-hot) encoding, and maps an $N$-dimensional space to an $L$ dimensional one, where $L < N$.  The example provided below will use this type of encoding for classifying the [MNIST dataset](http://yann.lecun.com/exdb/mnist/).

Something that is lost with classification is the ability to go from a class label back to an *instance* of that class.  This is because this type of non-linear projection operation is not invertible.  There is no one-to-one mapping between a class instance and the class label; it is a many-to-one relationship.  However, there are networks that would support doing both.  Namely, a slightly modified autoencoder.

## Script Setup
The constants below control how the training is done.  The parameters are batch size, number of epochs and an 'alpha' parameter used by the objective, or cost, function.

```python
# Notebook Constants
BATCH_SIZE = 50
EPOCHS = 10
ALPHA = 0.5
```

The datasets are loaded using a utility `load_dataset` function.  This can load either the original MNIST dataset or the Fashion-MNIST dataset.

```python
from utilities import load_dataset

# The Fashion-MNIST dataset can be loaded by changing 'mnist' to 'fashion'.
train, test = load_dataset('mnist', BATCH_SIZE)

print('Training Examples: %d' % (len(train)*BATCH_SIZE))
print('Testing Examples:  %d' % (len(test)*BATCH_SIZE))
```

    Training Examples: 60000
    Testing Examples:  10000

```python
# Check to see if CUDA is available.
import torch
if torch.cuda.is_available():
    device = torch.device('cuda')
    print('Training with CUDA.')
else:
    device = torch.device('cpu')
    print('Training with CPU.')
```

    Training with CPU.

## Network Design
The network is comprised of a classification frontend and a reconstruction backend.  It uses traditional linear units with sigmoidal activation functions.  The structure is
```
Input
  |
  V
 784 -> 256 -> 16 -> 10 -> 10 -> 16 -> 256 -> 784
                        |                      |
                        V                      V
                      Label              Reconstruction

  |<--- Classifier --->| |<---- Generator ---->|
```

When provided with an input image (28x28 pixels), the network will produce a one-hot encoding of the particular class as well as a new, 28x28 pixel image of what the network believes the class looks like.  This being a feedforward network, it is possible to provide a label directly and obtain a representation of what the generator believes the label to look like.

```python
from network import Reconstructor
model = Reconstructor().to(device)
```

## Training
The training loop is straightforward: loop over the dataset `EPOCH` times, randomly shuffling the dataset in `BATCH_SIZE` batches.  As discussed in the introduction, the purpose of this network is to both classify MNIST digits as well as to learn the digits from their classes.  As such, the cost function is defined as
\begin{equation}
    E\left( \vec{l}_p, \vec{x}_p \right) = (1 - \alpha)E_c(\vec{l}_p) + \alpha E_r(\vec{x}_p),
\end{equation}
where $E_c(\cdot)$ is the classification error and $E_r(\cdot)$ is the reconstruction error.  This cost function is designed so that it is the convex sum between these two errors.

Please note that, as the function has been written, the aggregate cost assumes that the magnitudes of $E_c(\cdot)$ and $E_r(\cdot)$ are similar.  I.e., their respective maximum values will be close to one another.  If they are not then it is possible for one of the costs to dominate the aggregate cost.

### Effect of $\alpha$
The $\alpha$ parameter switches the network between one of three modes.  When $\alpha = 0$, the network is trained to only be a classifier.  The output of the generator will be random noise because there is no cost associated with reconstructing the input images.  Conversely, when $\alpha = 1$, the network acts as an autoencoder.  The representation that it learns at the label output will depend on how the network is trained.  It will be able to recreate the input but it will not be able to classify anything.  Finally, if $\alpha > 0$ and $\alpha < 1$ then the network will be trained to do both.

```python
from tqdm import tqdm_notebook

optimizer = torch.optim.Adam(model.parameters())
loss_classification = torch.nn.CrossEntropyLoss()
loss_reconstruction = torch.nn.MSELoss()

def trange(count, desc):
    rng = range(count)
    return tqdm_notebook(rng, desc=desc, total=count)

losses_all = []
losses_cls = []
losses_rep = []

for epoch in trange(EPOCHS, 'Epoch'):
    for i, (img, lbl) in enumerate(train):
        img = img.reshape(-1, 28*28).to(device)
        lbl = lbl.to(device)

        # Compute the model's loss function.
        enc, rep = model(img)
        loss_cls = loss_classification(enc, lbl)
        loss_rep = loss_reconstruction(rep, img)
        loss = (1.0 - ALPHA)*loss_cls + ALPHA*loss_rep

        # Run the optimizer and update the model weights.
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        losses_all.append(loss.item())
        losses_cls.append(loss_cls.item())
        losses_rep.append(loss_rep.item())

# Plot the costs over time.
%matplotlib inline
import matplotlib.pyplot as plt

plt.figure(figsize=(18, 6))

plt.subplot(1, 3, 1)
plt.plot(losses_all)
plt.xlabel('Iteration')
plt.ylabel('Cost (All)')

plt.subplot(1, 3, 2)
plt.plot(losses_cls)
plt.xlabel('Iteration')
plt.ylabel('Cost (Classification)')

plt.subplot(1, 3, 3)
plt.plot(losses_rep)
plt.xlabel('Iteration')
plt.ylabel('Cost (Reconstruction)')

plt.show()
```

## Network Outputs
While there is only one cost being optimized, the network has two outputs.  The first, the output of the classifier, can be assessed by looking at the test set.  This provides a measureable accuracy metric for the classifier.  The other output, the output of generator, is different.

### Classifier Accuracy
The accuracy of the classifier is defined as
\begin{equation}
    \mathrm{Accuracy} = \frac{N_\mathrm{correct}}{N_\mathrm{total}} \times 100\%.
\end{equation}

```python
# Compute the accuracy of the classifier.
with torch.no_grad():
    num_correct = 0
    num_total = 0
    for i, (img, lbl) in enumerate(test):
        img = img.reshape(-1, 28*28).to(device)
        lbl = lbl.to(device)

        out = model.classifier(img)
        _, prediction = torch.max(out.data, 1)
        num_total += lbl.shape[0]
        num_correct += (prediction == lbl).sum().item()

print('Accuracy: %g %% (%d/%d)' % (100.0*num_correct/num_total, num_correct, num_total))
```

    Accuracy: 97.6 % (9760/10000)

### Generator Accuracy
The output of the generator is defined as the solution that minimizes the $L_2$-norm between a particular input label $\vec{c}_i$ and its associated input image $\vec{x}$.  As such, it is a bit more difficult to assign an accuracy metric that is meaningful in the same way that accuracy is.  In this case, it is possible to show what the network produces when given a particular label as a qualitative assessment.

```python
# Output what the network believes a label looks like.
import numpy as np
with torch.no_grad():
    fig, axs = plt.subplots(nrows=2, ncols=5)
    axes = axs.flatten()
    for i in range(10):
        enc = np.zeros((1, 10))
        enc[0, i] = 1

        enc = torch.tensor(enc, dtype=torch.float).to(device)
        out = model.generator(enc).reshape(28, 28)

        axes[i].imshow(out)
        axes[i].set_title('ID: %d' % i)

    fig.subplots_adjust(hspace=0.5, wspace=0.5)
```

## Conclusion
This notebook has shown how a simple feedfoward neural network can be used both for classification and sample generation.  This was accomplished using an autoencoder-style network and a specially-crafted cost function.  The cost function had a tuneable parameter, $\alpha$ that would allow the network to be be trained just for classification, reconstruction or a combination of both.  The purpose was not to create a state-of-the-art deep network but rather to demonstrate how a single network could be trained to perform multiple tasks.


[View in Colaboratory](https://colab.research.google.com/github/yepuv1/primality/blob/master/primality_nn.ipynb)

# Primality test using ANN.

The aim is to train a ANN Model on a small sample of primes from a finite prime numbers set.  In this notebook, I am generating the primes within 1M, which has 78K primes. The model is trained on a sample of 5k from this 78k prime set. What is interesting is that the model accuracy is 99.6% when testing for primality of the 78k primes. The accuracy drops rapidly for samples outside this 78k range of primes.

```
import math
import numpy as np
import h5py
import matplotlib.pyplot as plt
import tensorflow as tf
import sympy
from tensorflow.python.framework import ops
from sympy import primerange, sieve, composite, prime
import pickle
import timeit
%matplotlib inline
np.random.seed(1)
```

```
print(tf.__version__)
device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  raise SystemError('GPU device not found')
print('Found GPU at: {}'.format(device_name))
```

    1.11.0
    Found GPU at: /device:GPU:0

```
def convert_dec2bin(num_list,feature_size):
  p_x = None
  for p in num_list:
    pb = bin(p)[2:].zfill(32)
    li = [ int(i) for i in pb]
    a1 = np.array(li).reshape((feature_size,1))
    if p_x is None:
      p_x = a1
    else:
      p_x = np.concatenate((p_x, a1), axis=1)
  return p_x
```

```
def generate_sequence_primes(between):
  start, end = between
  p_num = [i for i in primerange(start, end)]
  return p_num, len(p_num)
```

```
def generate_random_primes(size, primes):
  le = len(primes)
  p_num = [primes[i] for i in np.random.randint(le , size=size)]
  return p_num, size
```

```
def generate_composite(size):
  c_num = np.random.randint(10000000, size=size)
  return c_num, size

```

```
def generate_x_y(primes, composites=None, feature_size=32):
  c_examples_size = 0
  p_examples_size = 0

  only_primes = False
  if composites is None:
    only_primes = True

  p_examples_size = len(primes)
  p_x = convert_dec2bin(primes, feature_size)
  p_y = np.ones(shape=(1,p_examples_size), dtype=np.int32)

  X = None
  Y = None

  if not only_primes:
    c_examples_size = len(composites)
    c_x = convert_dec2bin(composites, feature_size)
    c_y = np.zeros(shape=(1,c_examples_size), dtype=np.int32)
    X = np.concatenate((c_x, p_x), axis=1)
    size = p_examples_size + c_examples_size
    Y = np.append(c_y, p_y).reshape((1, size))

  else:
    X = p_x
    Y = p_y
    size = p_examples_size

  return X, Y, size

```

```
def one_hot_matrix(labels, C):
    """
    Creates a matrix where the i-th row corresponds to the ith class number and the jth column
                     corresponds to the jth training example. So if example j had a label i. Then entry (i,j)
                     will be 1.

    Arguments:
    labels -- vector containing the labels
    C -- number of classes, the depth of the one hot dimension

    Returns:
    one_hot -- one hot matrix
    """

    C = tf.constant(dtype=tf.int32, name="C", value=C)
    one_hot_matrix = tf.one_hot(labels,depth=C, axis=0)
    with tf.Session() as sess:
      one_hot = sess.run(one_hot_matrix)

    return one_hot

```

```
def create_placeholders(n_x, n_y):
    """
    Creates the placeholders for the tensorflow session.

    Arguments:
    n_x -- scalar, size of input vector
    n_y -- scalar, number of classes (from 0 to 1, so -> 2)

    Returns:
    X -- placeholder for the data input, of shape [n_x, None] and dtype "float"
    Y -- placeholder for the input labels, of shape [n_y, None] and dtype "float"

    Tips:
    - You will use None because it let's us be flexible on the number of examples you will for the placeholders.
      In fact, the number of examples during test/train is different.
    """

    X = tf.placeholder(tf.float32, shape=(n_x, None))
    Y = tf.placeholder(tf.float32, shape=(n_y, None))

    return X, Y
```

```
def initialize_parameters(dic_of_layer_sizes):
    """
    Initializes parameters to build a neural network with tensorflow. The shapes are:
                        W1 : [25, 2]
                        b1 : [25, 1]
                        W2 : [12, 25]
                        b2 : [12, 1]
                        W3 : [6, 12]
                        b3 : [6, 1]

    Returns:
    parameters -- a dictionary of tensors containing W1, b1, W2, b2, W3, b3
    """

    tf.set_random_seed(1)

    parameters ={}
    for key, value in dic_of_layer_sizes.items():
      print(key)
      print(value)
      parameters[key]=tf.get_variable(key, value, initializer = tf.contrib.layers.xavier_initializer(seed = 1))

    return parameters
```

```
def forward_propagation(X, parameters):
    """
    Implements the forward propagation for the model: LINEAR -> RELU -> LINEAR ->  SOFTMAX

    Arguments:
    X -- input dataset placeholder, of shape (input size, number of examples)
    parameters -- python dictionary containing your parameters "W1", "b1", "W2", "b2", "W3", "b3"
                  the shapes are given in initialize_parameters

    Returns:
    Z3 -- the output of the last LINEAR unit
    """

    # Retrieve the parameters from the dictionary "parameters"
    W1 = parameters['W1']
    b1 = parameters['b1']
    W2 = parameters['W2']
    b2 = parameters['b2']

    Z1 = tf.add(tf.matmul(W1, X), b1)  # LINEAR
    A1 = tf.nn.relu(Z1)                # RELU
    Z2 = tf.add(tf.matmul(W2, A1), b2) # LINEAR

    return Z2
```

```
def compute_cost(Z3, Y):
    """
    Computes the cost

    Arguments:
    Z3 -- output of forward propagation (output of the last LINEAR unit), of shape (6, number of examples)
    Y -- "true" labels vector placeholder, same shape as Z3

    Returns:
    cost - Tensor of the cost function
    """

    # to fit the tensorflow requirement for tf.nn.softmax_cross_entropy_with_logits(...,...)
    logits = tf.transpose(Z3)
    labels = tf.transpose(Y)

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = logits, labels = labels))

    return cost
```

```
def random_mini_batches(X, Y, mini_batch_size = 64, seed = 0):
    """
    Creates a list of random minibatches from (X, Y)

    Arguments:
    X -- input data, of shape (input size, number of examples)
    Y -- true "label" vector (containing 0 if cat, 1 if non-cat), of shape (1, number of examples)
    mini_batch_size - size of the mini-batches, integer
    seed -- this is only for the purpose of grading, so that you're "random minibatches are the same as ours.

    Returns:
    mini_batches -- list of synchronous (mini_batch_X, mini_batch_Y)
    """

    m = X.shape[1]                  # number of training examples
    mini_batches = []
    np.random.seed(seed)

    # Step 1: Shuffle (X, Y)
    permutation = list(np.random.permutation(m))
    shuffled_X = X[:, permutation]
    shuffled_Y = Y[:, permutation].reshape((Y.shape[0],m))

    # Step 2: Partition (shuffled_X, shuffled_Y). Minus the end case.
    num_complete_minibatches = math.floor(m/mini_batch_size) # number of mini batches of size mini_batch_size in your partitionning
    for k in range(0, num_complete_minibatches):
        mini_batch_X = shuffled_X[:, k * mini_batch_size : k * mini_batch_size + mini_batch_size]
        mini_batch_Y = shuffled_Y[:, k * mini_batch_size : k * mini_batch_size + mini_batch_size]
        mini_batch = (mini_batch_X, mini_batch_Y)
        mini_batches.append(mini_batch)

    # Handling the end case (last mini-batch < mini_batch_size)
    if m % mini_batch_size != 0:
        mini_batch_X = shuffled_X[:, num_complete_minibatches * mini_batch_size : m]
        mini_batch_Y = shuffled_Y[:, num_complete_minibatches * mini_batch_size : m]
        mini_batch = (mini_batch_X, mini_batch_Y)
        mini_batches.append(mini_batch)

    return mini_batches
```

```
def model(X_train, Y_train, X_test, Y_test,dic_of_layer_sizes,learning_rate = 0.0001,
          num_epochs = 1024, minibatch_size = 32, print_cost = True):
    """
    Implements a three-layer tensorflow neural network: LINEAR->RELU->LINEAR->RELU->LINEAR->SOFTMAX.

    Arguments:
    X_train -- training set, of shape (input size = 12288, number of training examples = 1080)
    Y_train -- test set, of shape (output size = 6, number of training examples = 1080)
    X_test -- training set, of shape (input size = 12288, number of training examples = 120)
    Y_test -- test set, of shape (output size = 6, number of test examples = 120)
    dic_of_layer_sizes - dictionary of W<n> b<n> sizes as [x,y]
    learning_rate -- learning rate of the optimization
    num_epochs -- number of epochs of the optimization loop
    minibatch_size -- size of a minibatch
    print_cost -- True to print the cost every 100 epochs

    Returns:
    parameters -- parameters learnt by the model. They can then be used to predict.
    """
    config = tf.ConfigProto()
    config.gpu_options.allow_growth = True

    with tf.device('/gpu:0'):

      ops.reset_default_graph()                         # to be able to rerun the model without overwriting tf variables
      tf.set_random_seed(1)                             # to keep consistent results
      seed = 3                                          # to keep consistent results
      (n_x, m) = X_train.shape                          # (n_x: input size, m : number of examples in the train set)
      n_y = Y_train.shape[0]                            # n_y : output size
      costs = []                                        # To keep track of the cost

      # Create Placeholders of shape (n_x, n_y)
      X, Y = create_placeholders(n_x, n_y)

      # Initialize parameters
      parameters = initialize_parameters(dic_of_layer_sizes)

      # Forward propagation: Build the forward propagation in the tensorflow graph
      Z3 = forward_propagation(X=X, parameters=parameters)

      # Cost function: Add cost function to tensorflow graph
      cost = compute_cost(Y=Y,Z3=Z3)

      # Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer.
      optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost)

      # Initialize all the variables
      init = tf.global_variables_initializer()

    # Start the session to compute the tensorflow graph
    with tf.Session(config=config) as sess:

        # Run the initialization
        sess.run(init)

        # Do the training loop
        for epoch in range(num_epochs):

            epoch_cost = 0.                       # Defines a cost related to an epoch
            num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set
            seed = seed + 1
            minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)

            for minibatch in minibatches:

                # Select a minibatch
                (minibatch_X, minibatch_Y) = minibatch

                # IMPORTANT: The line that runs the graph on a minibatch.
                # Run the session to execute the "optimizer" and the "cost", the feedict should contain a minibatch for (X,Y).

                _ , minibatch_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})

                epoch_cost += minibatch_cost / num_minibatches

            # Print the cost every epoch
            if print_cost == True and epoch % 100 == 0:
                print ("Cost after epoch %i: %f" % (epoch, epoch_cost))
            if print_cost == True and epoch % 5 == 0:
                costs.append(epoch_cost)

        # plot the cost
        plt.plot(np.squeeze(costs))
        plt.ylabel('cost')
        plt.xlabel('iterations (per tens)')
        plt.title("Learning rate =" + str(learning_rate))
        plt.show()

        # lets save the parameters in a variable
        parameters = sess.run(parameters)
        print ("Parameters have been trained!")

        # Calculate the correct predictions
        correct_prediction = tf.equal(tf.argmax(Z3), tf.argmax(Y))

        # Calculate accuracy on the test set
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

        print ("Train Accuracy:", accuracy.eval({X: X_train, Y: Y_train}))
        print ("Test Accuracy:", accuracy.eval({X: X_test, Y: Y_test}))

        return parameters
```

## Generate Primes (2, 1000000)

```
PRIMES_FILENAME = "1_million_primes.p"
def save_primes(file):
  p_num = [i for i in primerange(2, 1000000)]
  pickle.dump( p_num, open(file, "wb" ))
  return p_num

def load_primes(file):
 return pickle.load(open(file, "rb" ))

PRIMES = save_primes(PRIMES_FILENAME)
```

```
print(len(PRIMES))
```

    78498

## Model Training

1.   Sample 5000 primes from PRIMES generated above
2.   Generate 5000 composite numbers
3.   Sample 100 primes from PRIMES  for dev and make sure they are not in Training set.

```
def main():
  examples_size = 5000
  c, size = generate_composite(examples_size)
  p, size = generate_random_primes(examples_size,PRIMES)
  s_p_train = set(p)
  X_train, Y_train, training_size = generate_x_y(p,c)

  examples_size = 100
  c, size = generate_composite(examples_size)
  p, size = generate_random_primes(examples_size,PRIMES)

  # remove the primes common for both test and train
  s_p_test = set(p)
  s_p_test = s_p_test - s_p_train
  X_test, Y_test, test_size = generate_x_y(list(s_p_test),c)

  Y_train = one_hot_matrix(Y_train.reshape(training_size), 2)
  Y_test = one_hot_matrix(Y_test.reshape(test_size), 2)

  print ("number of training examples = " + str(X_train.shape[1]))
  print ("number of test examples = " + str(X_test.shape[1]))
  print ("X_train shape: " + str(X_train.shape))
  print ("Y_train shape: " + str(Y_train.shape))
  print ("X_test shape: " + str(X_test.shape))
  print ("Y_test shape: " + str(Y_test.shape))

  dic_of_layer_sizes ={
      "W1":[64,32],
      "b1":[64,1],
      "W2":[2,64],
      "b2":[2,1]
  }

  parameters = model(
      X_train,
      Y_train,
      X_test,
      Y_test,
      dic_of_layer_sizes,
      num_epochs=1500,
      minibatch_size=64
  )
  return parameters

parameters = main()
```

##  Inference - Random sample test.

Test for random sample of primes  of sizes 10k to 50k from the population PRIMES

```
%%time
start = 10000
end = 50000
step_size = 5000

accuracies = []
for p_size in range(start, end, step_size):
  p, size = generate_random_primes(p_size, PRIMES)
  X, Y, size = generate_x_y(p,None)
  print(X.shape)
  Y = one_hot_matrix(Y.reshape(size), 2)
  init = tf.global_variables_initializer()
  with tf.Session() as session:
    session.run(init)
    z3 = forward_propagation(np.float32(X), parameters)
    p = tf.argmax(z3)
    res = session.run(p)

    # Calculate the correct predictions
    correct_prediction = tf.equal(res, tf.argmax(Y))
    # Calculate accuracy on the test set
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
    accuracies.append(accuracy.eval())
    print ("Accuracy:", accuracy.eval())

# plot the cost
plt.plot(np.squeeze(accuracies))
plt.ylabel('accuracies')
plt.xlabel('iterations')
plt.title("Step size =" + str(step_size))
plt.show()

```

## Inference - Sequence test.

Generate primes from 2 to 100k and test for primality.

```
%%time
start = 2
end = 1000000
step_size = 1000

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
accuracies = []
with tf.device('/gpu:0'):
  p, size = generate_sequence_primes((start, end))
  X, Y, size = generate_x_y(p,None)
  print(X.shape)
  Y = one_hot_matrix(Y.reshape(size), 2)
  init = tf.global_variables_initializer()
with tf.Session(config=config) as session:
  session.run(init)
  z3 = forward_propagation(np.float32(X), parameters)
  p = tf.argmax(z3)
  res = session.run(p)

  # Calculate the correct predictions
  correct_prediction = tf.equal(res, tf.argmax(Y))
  # Calculate accuracy on the test set
  accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
  accuracies.append(accuracy.eval())
  print ("Accuracy:", accuracy.eval())

```

    (32, 78498)
    Accuracy: 0.99672604
    CPU times: user 3min 37s, sys: 7.27 s, total: 3min 45s
    Wall time: 3min 45s

## Inference - Accuracy

For primes out side the sample space, the accuracy drops.

```
%%time
start = 1000000
end = 1100000
step_size = 1000

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
accuracies = []
with tf.device('/gpu:0'):
    p, size = generate_sequence_primes((start, end))
    X, Y, size = generate_x_y(p,None)
    print(X.shape)
    Y = one_hot_matrix(Y.reshape(size), 2)
    init = tf.global_variables_initializer()
    with tf.Session(config=config) as session:
      session.run(init)
      z3 = forward_propagation(np.float32(X), parameters)
      p = tf.argmax(z3)
      res = session.run(p)

      # Calculate the correct predictions
      correct_prediction = tf.equal(res, tf.argmax(Y))
      # Calculate accuracy on the test set
      accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
      accuracies.append(accuracy.eval())
      print ("Accuracy:", accuracy.eval())

```

    (32, 7216)
    Accuracy: 0.48406318
    CPU times: user 1.73 s, sys: 98.9 ms, total: 1.82 s
    Wall time: 1.83 s