###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, G.F. Forsyth, C.D. Cooper. Partly based on content  by David Ketcheson, also under CC-BY.

# Phugoid model: bonus!

_The phugoid model of glider flight_ has been such a fun problem to showcase the power of numerical solution of differential equations, we thought you'd enjoy a bonus notebook. The previous lessons were:

* [Phugoid motion](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_01_Phugoid_Theory.ipynb) —Lays the groundwork for our fun problem, with some context, a little history and a description of the physics of phugoids: curves representing the trajectory of a glider exchanging potential and kinetic energy, with no drag.
* [Phugoid oscillation](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_02_Phugoid_Oscillation.ipynb) —Develops the simple harmonic motion of an aircraft experiencing a small perturbation from the horizontal trajectory: our opportunity to introduce Euler's method, and study its convergence via an exact solution.
* [Full phugoid motion](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb) —The full model takes into account the force of drag and results in a system of two nonlinear equations. We obtain the trajectories using Euler's method in vectorized form, introduce grid-convergence analysis and finish with the paper-airplane challenge!

That is a fantastic foundation for numerical methods. It's a good time to complement it with some theory: the first screencast of the course uses Taylor series to show that _Euler's method is a first-order method_, and we also show you graphical interpretations. Many problems require a more accurate method, though: second order or higher. Among the most popular higher-order methods that we can mention are the _Runge-Kutta methods_, developed around 1900: more than 100 years after Euler published his book containing the method now named after him!

### Euler's method is a first-order method

In this screencast, we use a Taylor series expansion to analyze Euler's method and show that it incurs a truncation error of first order. We also use a graphical interpretation to motivate the _modified_ Euler method, which achieves second order.

```python
from IPython.display import YouTubeVideo
YouTubeVideo('6i6qhqDCViA')
```

## Second-order methods

The notebook on phugoid oscillation (lesson 2) included a study of the accuracy obtained with Euler's method, using the exact solution for the simple harmonic motion. We made a _convergence plot_ and saw that as $\Delta t$ gets smaller, the error also gets smaller.

We could have drawn a line with a slope equal to 1 on that log-log plot, and you would have seen that it was parallel to the convergence line. A slope equal to 1 on a log-log convergence plot is an indication that we have a first-order method: the error scales as ${\mathcal O}(\Delta t)$.

In lesson 3, using the full phugoid model (which is nonlinear and does not have an exact solution), we did a _grid-convergence study_ with three different grids, and obtained the _observed_ order of convergence—it was very close to 1, indicating a slope of 1 on a log-log plot.

Another way to look at an ${\mathcal O}(\Delta t)$ method is to say that the error scales _linearly_ with the step size, or that they are proportional:

$$
e \propto \Delta t
$$

where $e$ stands for the error. To get more accuracy, we could use a _second-order_ method, in which the error is ${\mathcal O}(\Delta t^2)$. In general, we say that a method is of order $p$ when the error is proportional to $(\Delta t)^p$.

In the screencast titled "Euler's method is a first-order method," we used a graphical interpretation to get an idea for improving it: by estimating an intermediate point, like the **midpoint**, we can get a better approximation of the area under the curve of $u^\prime$. The scheme has two steps and is written as:

$$
\begin{align}
u_{n+1/2}   & = u_n + \frac{\Delta t}{2} f(u_n) \\
u_{n+1} & = u_n + \Delta t \,\, f(u_{n+1/2})
\end{align}
$$

This method is known as the *explicit midpoint method* or the *modified Euler method*, and it is a second-order method. Notice that we had to apply the right-hand side, $~f(u)$, twice. This idea can be extended: we could imagine estimating additional points between $u_{n}$ and $u_{n+1}$ and evaluating $~f(u)$ at the intermediate points to get higher accuracy—that's the idea behind Runge-Kutta methods.

### Runge-Kutta methods

In the modified Euler method, we improve the accuracy over Euler's method by evaluating the right-hand side of the differential equation at an intermediate point: the midpoint. The same idea can be applied again, and the function $f(u)$ can be evaluated at more intermediate points, improving the accuracy even more. This is the basis of the famous *Runge-Kutta (RK) methods*, going back to Carl Runge and Martin Kutta. The modified Euler method corresponds to _second-order_ Runge-Kutta.

Here's a bit of historical coincidence that will blow your mind: Carl Runge's daughter Iris—an accomplished applied mathematician in her own right—worked assiduously over the summer of 1909 to translate Lanchester's _"Aerodonetics."_ She also reproduced his graphical method to draw the phugoid curves (Tobies, 2012).

### Phugoid model with 2nd-order RK

Let's compute the motion of a glider under the full phugoid model using the second-order Runge-Kutta method. We'll build on the _paper airplane challenge_ of lesson 3 now, and look for the horizontal distance that the plane travels until the moment it touches the ground.

As usual, let's start by importing the libraries and modules that we need, and setting up the model parameters. We also set some default plotting formats by modifying entries of the `rcParams` dictionary.

```python
import math
import numpy
from matplotlib import pyplot
%matplotlib inline
```

```python
# Set the font family and size to use for Matplotlib figures.
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
```

In the paper-airplane challenge of lesson 3, we suggested an $L/D=5.0$ as a realistic value for paper airplanes, according to experiments, and a trim velocity of 4.9 m/s. Let's start with those values, but you could experiment changing these a bit. _What do you think will happen if you make $L/D$ higher?_

```python
# Set parameters.
g = 9.81  # gravitational acceleration (m.s^{-2})
vt = 4.9  # trim velocity (m.s)
CD = 1.0 / 5.0  # drag coefficient
CL = 1.0  # lift coefficient

# Set initial conditions.
v0 = 6.5  # start at the trim velocity
theta0 = -0.1  # trajectory angle
x0 = 0.0  # horizontal position
y0 = 2.0  # vertical position (altitude)
```

Among the initial parameters that we suggest for your first experiment, we are starting with a velocity a little higher than the trim velocity, launch the paper airplane with a negative initial angle, and take the initial height to be 2 meters—all sound like reasonable choices.

Now, we can define a few functions to carry out the computation:
* The right-hand side of the phugoid model from [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb),
* One step of the Euler's method that we learned in [Lesson 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_02_Phugoid_Oscillation.ipynb), and
* Differences with respect to a fine grid, as in [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb).

```python
def rhs_phugoid(u, CL, CD, g, vt):
    """
    Returns the right-hand side of the phugoid system of equations.

    Parameters
    ----------
    u : list or numpy.ndarray
        Solution at the previous time step
        as a list or 1D array of four floats.
    CL : float
        Lift coefficient.
    CD : float
        Drag coefficient.
    g : float
        Gravitational acceleration.
    vt : float
        Trim velocity.

    Returns
    -------
    rhs : numpy.ndarray
        The right-hand side of the system
        as a 1D array of four floats.
    """
    v, theta, x, y = u
    rhs = numpy.array([-g * math.sin(theta) - CD / CL * g / vt**2 * v**2,
                       -g * math.cos(theta) / v + g / vt**2 * v,
                       v * math.cos(theta),
                       v * math.sin(theta)])
    return rhs

def euler_step(u, f, dt, *args):
    """
    Returns the solution at the next time step using Euler's method.

    Parameters
    ----------
    u : numpy.ndarray
        Solution at the previous time step
        as a 1D array of floats.
    f : function
        Function to compute the right-hand side of the system.
    dt : float
        Time-step size.
    args : tuple, optional
        Positional arguments to pass to the function f.

    Returns
    -------
    u_new : numpy.ndarray
        The solution at the next time step
        as a 1D array of floats.
    """
    u_new = u + dt * f(u, *args)
    return u_new

def l1_diff(u_coarse, u_fine, dt):
    """
    Returns the difference in the L1-norm between the solution on
    a coarse grid and the solution on a fine grid.

    Parameters
    ----------
    u_coarse : numpy.ndarray
        Solution on the coarse grid as a 1D array of floats.
    u_fine : numpy.ndarray
        Solution on the fine grid as a 1D array of floats.
    dt : float
        Time-step size.

    Returns
    -------
    diff : float
        The difference between the two solution in the L1-norm
        scaled by the time-step size.
    """
    N_coarse = u_coarse.shape[0]
    N_fine = u_fine.shape[0]
    ratio = math.ceil(N_fine / N_coarse)
    diff = dt * numpy.sum(numpy.abs(u_coarse - u_fine[::ratio]))
    return diff
```

Next, we also need to define the function `rk2_step()` that computes the next time step using the *modified Euler* method of equations $(1)$ and $(2)$, above, otherwise known as 2nd-order Runge-Kutta or RK2. This function will be called over and over again within the time loop.

```python
def rk2_step(u, f, dt, *args):
    """
    Returns the solution at the next time step using 2nd-order
    Runge-Kutta method.

    Parameters
    ----------
    u : numpy.ndarray
        Solution at the previous time step
        as a 1D array of floats.
    f : function
        Function to compute the right-hand side of the system.
    dt : float
        Time-step size.
    args : tuple, optional
        Positional arguments to pass to the function f.

    Returns
    -------
    u_new : numpy.ndarray
        The solution at the next time step
        as a 1D array of floats.
    """
    u_star = u + 0.5 * dt * f(u, *args)
    u_new = u + dt * f(u_star, *args)
    return u_new
```

Like in [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb), we first need to set up the time discretization, then initialize arrays to save the solution and we are set to go! The only difference this time is that we are using _both_ Euler's method and 2nd-order Runge-Kutta to get a solution, to compare the two.

```python
T = 15.0  # length of the time interval
dt = 0.01  # time-step size
N = int(T / dt) + 1  # number of time steps

# Create arrays to store the solution at each time step.
u_euler = numpy.empty((N, 4))
u_rk2 = numpy.empty((N, 4))

# Set the initial conditions.
u_euler[0] = numpy.array([v0, theta0, x0, y0])
u_rk2[0] = numpy.array([v0, theta0, x0, y0])

# Time integration with both method.
for n in range(N - 1):
    u_euler[n + 1] = euler_step(u_euler[n], rhs_phugoid, dt,
                                CL, CD, g, vt)
    u_rk2[n + 1] = rk2_step(u_rk2[n], rhs_phugoid, dt,
                            CL, CD, g, vt)
```

Now we can get the position of the glider in time, according to both Euler's method and the 2nd-order Runge-Kutta method, by extracting the appropriate portions of the solution arrays:

```python
# Get the glider's position over the time.
x_euler = u_euler[:, 2]
y_euler = u_euler[:, 3]
x_rk2 = u_rk2[:, 2]
y_rk2 = u_rk2[:, 3]
```

##### How far will it fly before touching the ground?

As the $y$-axis measures the vertical coordinate with respect to the ground, negative values of $y$ don't have any physical meaning: the glider would have hit the ground by then! To find out if there are any negative $y$ values we can use the handy function [`numpy.where`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html). This function returns the **indices** of the elements in an array that match a given condition. For example, `numpy.where(y_euler<0)[0]` gives an array of the indices `i` where `y_euler[i]<0` (the `[0]` is necessary as `numpy.where` returns an array, which in this case contains a single line). If no elements of the array match the condition, the array of indices comes out empty.

From the physical problem, we know that once there is one negative value, the glider has hit the ground and all the remaining time-steps are unphysical. Therefore, we are interested in finding the _first_ index where the condition applies, given by `numpy.where(y_euler<0)[0][0]`—do read the  documentation of the function if you need to!

```python
# Get the index of the first negative element of y_euler.
idx_negative_euler = numpy.where(y_euler < 0.0)[0]
if len(idx_negative_euler) == 0:
    idx_ground_euler = N - 1
    print('[Euler] Glider has not touched ground yet!')
else:
    idx_ground_euler = idx_negative_euler[0]
# Get the index of the first negative element of y_rk2.
idx_negative_rk2 = numpy.where(y_rk2 < 0.0)[0]
if len(idx_negative_rk2) == 0:
    idx_ground_rk2 = N - 1
    print('[RK2] Glider has not touched ground yet!')
else:
    idx_ground_rk2 = idx_negative_rk2[0]
```

##### Do Euler and RK2 produce the same solution?

An easy way to compare the numerical results obtained with the Euler and 2nd-order Runge-Kutta methods is using [`numpy.allclose`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.allclose.html). This function compares each element of two arrays and returns `True` if each comparison is within some relative tolerance. Here, we use the default tolerance: $10^{-5}$.

```python
# Check if to two scheme leads to the same numerical solution.
print('Are the x-values close? {}'.format(numpy.allclose(x_euler, x_rk2)))
print('Are the y-values close? {}'.format(numpy.allclose(y_euler, y_rk2)))
```

    Are the x-values close? False
    Are the y-values close? False

Hmmm, they do differ. Maybe $10^{-5}$ is too tight a tolerance, considering we're using a somewhat coarse grid with first- and second-order methods. Perhaps we can assess this visually, by plotting the glider's path? Study the code below, where we are plotting the path twice, taking a closer look in the second plot by "zooming in" to the beginning of the flight.

```python
print('Distance traveled: {:.3f}'.format(x_rk2[idx_ground_rk2 - 1]))

# Plot the glider's path for both schemes.
pyplot.figure(figsize=(9.0, 6.0))
pyplot.subplot(121)
pyplot.grid()
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.plot(x_euler[:idx_ground_euler], y_euler[:idx_ground_euler],
            label='Euler')
pyplot.plot(x_rk2[:idx_ground_rk2], y_rk2[:idx_ground_rk2],
            label='RK2')
pyplot.legend();
# Let's take a closer look!
pyplot.subplot(122)
pyplot.grid()
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.plot(x_euler, y_euler, label='Euler')
pyplot.plot(x_rk2, y_rk2, label='RK2')
pyplot.xlim(0.0, 5.0)
pyplot.ylim(1.8, 2.5);
```

From far away, the Euler and RK2 methods seem to be producing similar answers. However, if we take a closer look, small differences become evident. Keep in mind that we are solving the same equation and both methods will converge to the same solution as we refine the grid. However, they converge to that solution at different rates: RK2 gets more accurate faster, as you make $\Delta t$ smaller.

### Grid-convergence

Just like in [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb), we want to do a grid-convergence study with RK2, to see if we indeed observe the expected rate of convergence. It is always an important step in a numerical solution to investigate whether the method is behaving the way we expect it to: this needs to be confirmed experimentally for every new problem we solve and for every new method we apply!

In the code below, a `for`-loop computes the solution on different time grids, with the coarsest and finest grid differing by 100x. We can use the difference between solutions to investigate convergence, as before.

```python
# Set the list of time-step sizes to investigate.
dt_values = [0.1, 0.05, 0.01, 0.005, 0.001]

# Create an empty list to store the solution for each time-step size.
u_values = []

for dt in dt_values:
    N = int(T / dt) + 1  # number of time steps
    # Set initial conditions.
    u = numpy.empty((N, 4))
    u[0] = numpy.array([v0, theta0, x0, y0])
    # Time integration using RK2 method.
    for n in range(N - 1):
        u[n + 1] = rk2_step(u[n], rhs_phugoid, dt, CL, CD, g, vt)
    u_values.append(u)
```

Once those runs are done, we compute the difference between each numerical solution and the fine-grid solution.

```python
# Compute the differences in the x-position for all grids.
diff_values = []
for u, dt in zip(u_values, dt_values):
    diff = l1_diff(u[:, 2], u_values[-1][:, 2], dt)
    diff_values.append(diff)
```

And now we plot!

```python
# Plot difference versus the time-step size.
pyplot.figure(figsize=(6.0, 6.0))
pyplot.title('L1-norm of the difference vs. time-step size')
pyplot.xlabel('$\Delta t$')
pyplot.ylabel('Difference')
pyplot.grid()
pyplot.loglog(dt_values[:-1], diff_values[:-1],
              color='C0', linestyle='--', marker='o')
pyplot.axis('equal');
```

This is looking good! The difference relative to our fine-grid solution is decreasing with the mesh size at a faster rate than in [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb), but *how much faster?* When we computed the observed order of convergence with Euler's method, we got a value close to 1—it's a first-order method. Can you guess what we'll get now with RK2?

To compute the observed order of convergence, we use three grid resolutions that are refined at a constant rate, in this case $r=2$.

```python
r = 2  # time-step size refinement ratio
h = 0.001  # finest time-step size

dt_values = [h, r * h, r**2 * h]
u_values = []

for dt in dt_values:
    N = int(T / dt) + 1  # number of time steps
    # Set initial conditions.
    u = numpy.empty((N, 4))
    u[0] = numpy.array([v0, theta0, x0, y0])
    # Time integration using RK2.
    for n in range(N - 1):
        u[n + 1] = rk2_step(u[n], rhs_phugoid, dt, CL, CD, g, vt)
    # Store the solution for the present time grid.
    u_values.append(u)

# Compute the observed order of convergence.
p = (math.log(l1_diff(u_values[2], u_values[1], dt_values[2]) /
              l1_diff(u_values[1], u_values[0], dt_values[1])) /
     math.log(r))

print('Observed order of convergence: p = {:.3f}'.format(p))
```

    Observed order of convergence: p = 1.996

Probably you're not too surprised to see that the observed order of convergence is close to $2$. Because we used a second-order method! This means that the numerical solution is converging with the grid resolution twice as fast compared with Euler's method in [Lesson 3](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_03_PhugoidFullModel.ipynb), or in other words, the error scales as ${\mathcal O}(\Delta t^2)$. That is a lot faster! However, we are paying a price here: second-order Runge-Kutta requires more computations per iteration.

##### Challenge task

How much longer does it take to get the solution with RK2, compared to Euler's method? Run the same solution (same time grid, same parameters), but find a way to *time* the calculation with Python, and compare the runtimes.

## Multi-step methods

The screencast *"Euler's method is a first-order method"* motivated graphically an idea to get increased accuracy: using intermediate points between $u_{n}$ and $u_{n+1}$ and evaluating the right-hand side of the differential equation at those intermediate points. The idea is to somehow get a better approximation using more data from the function $f(u)$.

Another way to bring more information about $f(u)$ into the numerical solution is to look at time data $t\lt t_{n}$. For example, we can involve in the calculation of the solution $u_{n+1}$ the known solution at $u_{n-1}$, in addition to $u_{n}$. Schemes that use this idea are called _multi-step methods_.

A classical multi-step method achieves second order by applying a _centered difference_ approximation of the derivative $u'$:

$$
u'(t) \approx \frac{u_{n+1} - u_{n-1}}{2\Delta t}
$$

Isolate the future value of the solution $u_{n+1}$ and apply the differential equation $u'=f(u)$, to get the following formula for this method:

$$
u_{n+1} = u_{n-1} + 2\Delta t \, f(u_n)
$$

This scheme is known as the **leapfrog method**. Notice that it is using the right-hand side of the differential equation, $f(u)$, evaluated at the _midpoint_ between $u_{n-1}$ and $u_{n+1}$, where the time interval between these two solutions is $2\Delta t$. Why is it called "leapfrog"? If you imagine for a moment all of the _even_ indices $n$ of the numerical solution, you notice that these solution values are computed using the slope estimated from _odd_ values $n$, and vice-versa.

Let's define a function that computes the numerical solution using the leapfrog method:

```python
def leapfrog_step(u_prev, u, f, dt, *args):
    """
    Returns the solution at the next time step using
    the leapfrog method.

    Parameters
    ----------
    u_prev : numpy.ndarray
        Solution at the time step n-1
        as a 1D array of floats.
    u : numpy.ndarray
        Solution at the previous time step
        as a 1D array of floats.
    f : function
        Function to compute the right-hand side of the system.
    dt : float
        Time-step size.
    args : tuple, optional
        Positional arguments to pass to the function f.

    Returns
    -------
    u_new : numpy.ndarray
        The solution at the next time step
        as a 1D array of floats.
    """
    u_new = u_prev + 2.0 * dt * f(u, *args)
    return u_new
```

But wait ... what will we do at the _initial_ time step, when we don't have information for $u_{n-1}$? This is an issue with all multi-step methods: we say that they are _not self-starting_. In the first time step, we need to use another method to get the first "kick"—either Euler's method or 2nd-order Runge Kutta could do: let's use RK2, since it's also second order.

For this calculation, we are going to re-enter the model parameters in the code cell below, so that later on we can experiment here using the leapfrog method and different starting values. At the end of this notebook, we'll give you some other model parameters to try that will create a very interesting situation!

```python
# Set parameters.
g = 9.81  # gravitational acceleration (m.s^{-2})
vt = 4.9  # trim velocity (m.s)
CD = 1.0 / 5.0  # drag coefficient
CL = 1.0  # lift coefficient

# Set initial conditions.
v0 = 6.5  # start at the trim velocity
theta0 = -0.1  # trajectory angle
x0 = 0.0  # horizontal position
y0 = 2.0  # vertical position (altitude)

T = 15.0  # length of the time interval
dt = 0.01  # time-step size
N = int(T / dt) + 1  # number of time steps

# Create arrays to store the solution at each time step.
u_leapfrog = numpy.empty((N, 4))
# Set the initial conditions.
u_leapfrog[0] = numpy.array([v0, theta0, x0, y0])
# Use the RK2 method for the first time step.
u_leapfrog[1] = rk2_step(u_leapfrog[0], rhs_phugoid, dt, CL, CD, g, vt)
```

Now we have all the required information to loop in time using the leapfrog method. The code cell below calls the leapfrog function for each time step.

```python
# Time integration using the leapfrog method.
for n in range(1, N - 1):
    u_leapfrog[n + 1] = leapfrog_step(u_leapfrog[n - 1], u_leapfrog[n],
                                      rhs_phugoid, dt, CL, CD, g, vt)
```

Like before, we extract from the solution array the information about the glider's position in time and find where it reaches the ground.

```python
# Get the glider's position over the time.
x_leapfrog = u_leapfrog[:, 2]
y_leapfrog = u_leapfrog[:, 3]

# Get the index of the first negative element of y_leapfrog.
idx_negative_leapfrog = numpy.where(y_leapfrog < 0.0)[0]
if len(idx_negative_leapfrog) == 0:
    idx_ground_leapfrog = N - 1
    print('[leapfrog] Glider has not touched ground yet!')
else:
    idx_ground_leapfrog = idx_negative_leapfrog[0]
```

Plotting the glider's trajectory with both the leapfrog and RK2 methods, we find that the solutions are very close to each other now: we don't see the differences that were apparent when we compared Euler's method and RK2.

```python
print('Distance traveled: {:.3f}'.format(x_leapfrog[idx_ground_leapfrog - 1]))

# Plot the glider's path for the leapfrog scheme.
pyplot.figure(figsize=(9.0, 6.0))
pyplot.subplot(121)
pyplot.grid()
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.plot(x_leapfrog[:idx_ground_leapfrog],
            y_leapfrog[:idx_ground_leapfrog])
# Let's take a closer look!
pyplot.subplot(122)
pyplot.grid()
pyplot.xlabel('x')
pyplot.ylabel('y')
pyplot.plot(x_leapfrog, y_leapfrog)
pyplot.xlim(0.0, 5.0)
pyplot.ylim(1.8, 2.5);
```

What about the observed order of convergence? We'll repeat the process we have used before, with a grid-refinement ratio $r=2$ ... here we go:

```python
r = 2  # time-step size refinement ratio
h = 0.001  # finest time-step size

dt_values = [h, r * h, r**2 * h]
u_values = []

for dt in dt_values:
    N = int(T / dt) + 1  # number of time steps
    # Set initial conditions.
    u = numpy.empty((N, 4))
    u[0] = numpy.array([v0, theta0, x0, y0])
    # Use RK2 for the first time step.
    u[1] = rk2_step(u[0], rhs_phugoid, dt, CL, CD, g, vt)
    # Time integration using the leapfrog scheme.
    for n in range(1, N - 1):
        u[n + 1] = leapfrog_step(u[n - 1], u[n], rhs_phugoid, dt,
                                 CL, CD, g, vt)
    # Store the solution for the present time grid.
    u_values.append(u)

# Compute the observed order of convergence.
p = (math.log(l1_diff(u_values[2][:, 2], u_values[1][:, 2],
                      dt_values[2]) /
              l1_diff(u_values[1][:, 2], u_values[0][:, 2],
                      dt_values[1])) /
     math.log(r))

print('Observed order of convergence: p = {:.3f}'.format(p))
```

    Observed order of convergence: p = 2.187

We now have numerical evidence that our calculation with the leapfrog method indeed exhibits second-order convergence, i.e., the method is ${\mathcal O}(\Delta t^2)$. _The leapfrog method is a second-order method_. Good job!

### But chew on this ...

Go back to the cell that re-enters the model parameters, just above the leapfrog-method time loop, and change the following: the initial height `y0` to 25, and the final time `T` to 36. Now re-run the leapfrog calculation and the two code cells below that, which extract the glider's position and plot it.

_What is going on?_

## Reference

Tobies, R. "Iris Runge: A life at the crossroads of mathematics, science and industry," Springer Basel, 1st ed. (2012). [Read on Google books, page 73](http://books.google.com/books?id=EDm0eQqFUQ4C&lpg=PA73&dq=%22I%20have%20been%20making%20good%20progress%20with%20Lanchester.%20The%20second%20chapter%20is%20already%20on%20your%20desk%22&pg=PA73#v=onepage&q=%22I%20have%20been%20making%20good%20progress%20with%20Lanchester.%20The%20second%20chapter%20is%20already%20on%20your%20desk%22&f=false).

---

###### The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "Source Code Pro";
    font-size: 90%;
}

.alert-box {
    padding:10px 10px 10px 36px;
    margin:5px;
}

.success {
    color:#666600;
    background:rgb(240,242,229);
}
</style>


```python
import numpy as np
import MDAnalysis
import sympy as sp
```

**Micro to macroscopic heat transfer**

Fourier's law relates the macroscopic heat current to the temperature gradient:

$J=-\lambda\nabla T$

where we find $\lambda$ using the Green-Kubo formula:

$\lambda= \frac{1}{3VK_{B}T^2} \int_{0}^{\infty} \langle j(0) \cdot j(t) \rangle \, \mathrm{d}t$

For the integration component, we're going to preprocess our trajectory into the time-stamped microscopic heat currents, $j$, and use a numerical method to perform the integration.

```python
def gk_heat_current(kb, volume, temperature, integrated_heat_current):
    gk_hc = 1 / (3 * volume * kb * temperature**2) * integrated_heat_current
    return gk_hc
```

I'm using the lowercase $j$ to signify the microscopic heat current:

$j(t) = \frac{d}{dt} \sum_{i=1}^N r_i e_i $

$r_i$ is the atomic position and $e_i$ is the microscopic site energy. The microscopic site energy is a combination of kinetic and potential energies:

$e_i = \frac{1}{2} [ m_i |v_i|^2 + \sum_{j}{\phi(r_{ij})}]$

```python
def site_energy(mass, velocities, lj_potentials):
    se = 0.5 * (mass * abs(velocities)**2 + np.sum(lj_potentials, axis=0))
    return se
```

where $v_i$ is the velocity of particle $i$ and $\phi$ is the interatomic, Lennard Jones potential:

$\phi(r_{ij}) = 4 \epsilon [( \frac{\sigma}{r_{ij}} )^{12} - ( \frac{\sigma}{r_{ij}} )^6]$

```python
def lj_potential(distance, epsilon, sigma):
    lj = 4 * epsilon * ((sigma/distance)**12 - (sigma/distance)**6)
    np.nan_to_num(lj, copy=False)
    return lj
```

where $r_{ij}$ are the interatomic distances. This simplifies to the final experession:

$j(t) = \sum_{i}{v_{i} e_{i}} + \frac{1}{2} \sum_{i}\sum_{j>i}[{(v_{j}+v_{i}) \cdot F_{ij}]r_{ij}}$

where $F_{ij}$ is the force exerted on atom $i$ from its LJ interaction with atom $j$: $F_{ij} = -\frac{\partial \phi}{\partial r_{ij}}$

```python
def microscopic_heat_current(site_energies, velocities, distances, forces):
    micro_hc = np.sum(velocities * site_energies) + 0.5 * np.sum(np.sum(velocities *
                                                             np.nan_to_num(forces)) * distances)
    return micro_hc
```

**Calculating in MDAnalysis**

There are $\sum_{i=1}^{N-1}{i}$ pairwise distances and $N$ velocities we need to calculate to compute the microscopic energy of every atom in our system

```python
# before we start lets define some variables

kb = 0.0083144621 #KJ/mol K
sigma = .34 #nm
epsilon = 0.99774 #KJ/mol
mass = 39.948 #AU
timestep = 0.002 # in ps

# should the timestep be the md timestep
# or should the xtc write frequency be
# taken into account?
```

```python
u = MDAnalysis.Universe("argon.pdb", "traj.trr")
u.atoms[u.atoms.types == 'A'].masses = mass
pdbtrj = "argon.pdb"
heat_current = []

with MDAnalysis.Writer(pdbtrj, multiframe=True, bonds=None, n_atoms=u.atoms.n_atoms) as PDB:
    i = 0
    for ts in u.trajectory[:10]:
        pairwise_distances = np.zeros((u.atoms.n_atoms,u.atoms.n_atoms))
        pairwise_velocities = np.zeros((u.atoms.n_atoms,u.atoms.n_atoms))
        for atom_a in range(u.atoms.n_atoms):
            for atom_b in range(atom_a + 1 , u.atoms.n_atoms):

                # calculate distances
                distance = np.sqrt(np.sum((u.atoms[atom_a].position -
                                           u.atoms[atom_b].position)**2))/10 #A to nm
                pairwise_distances[atom_a,atom_b] = distance

                # calculate d(r_ij) / dt
                relative_velocity = np.sqrt(np.sum((u.atoms[atom_a].velocity -
                                           u.atoms[atom_b].velocity)**2))
                pairwise_velocities[atom_a,atom_b] = relative_velocity

        #compute velocities
        velocities = np.sqrt(np.sum(u.atoms.velocities**2, axis=1))

        #compute forces
        forces = np.sqrt(np.sum(u.atoms.forces**2, axis=1))

        # 1) compute LJ
        lj_potentials = lj_potential(pairwise_distances, epsilon, sigma)

        # 2) compute site energy
        site_energies = site_energy(mass, velocities, lj_potentials)

        if i > 0:
            pairwise_forces = -(lj_potentials -
                                previous_lj_potentials) / (pairwise_distances -
                                                           previous_pairwise_distances)
            # 3) compute micro heat current
            micro_hc = microscopic_heat_current(site_energies, pairwise_velocities,
                                                pairwise_distances, pairwise_forces)
            heat_current.append(micro_hc)

        previous_lj_potentials = lj_potentials
        previous_pairwise_distances = pairwise_distances

        PDB.write(u.atoms)
        i += 1
        if i < 10:
            print("Frame {}: "
                  "min/max velocities: {:.1f}...{:.1f} nm/ps".format(ts.frame,
                                                                 velocities.min(),
                                                                 velocities.max()))
            print("Frame {}: "
                  "min/max lj_potentials: {:.1f}...{:.1f} KJ/mol".format(ts.frame,
                                                                 np.nanmin(lj_potentials),
                                                                 np.nanmax(lj_potentials)))
        if 10 > i > 1:
            print("Frame {}: "
                  "micro heat current: {:.1f} KJ".format(ts.frame, micro_hc))

print("Wrote PDB trajectory for Argon with distances in bfactor field")
```

    /home/wesleybeckner/.local/lib/python3.6/site-packages/MDAnalysis/topology/guessers.py:72: UserWarning: Failed to guess the mass for the following atom types: A
      warnings.warn("Failed to guess the mass for the following atom types: {}".format(atom_type))
    /home/wesleybeckner/anaconda3/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py:2: RuntimeWarning: divide by zero encountered in true_divide

    /home/wesleybeckner/anaconda3/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in subtract

    Frame 0: min/max velocities: 0.1...5.5 A
    Frame 0: min/max lj_potentials: -0.9...0.0 KJ/mol

    /home/wesleybeckner/anaconda3/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py:39: RuntimeWarning: invalid value encountered in true_divide

    Frame 1: min/max velocities: 0.1...5.0 A
    Frame 1: min/max lj_potentials: -1.0...4.2 KJ/mol
    Frame 1: micro heat current: 3716012473.9 KJ
    Frame 2: min/max velocities: 0.2...6.0 A
    Frame 2: min/max lj_potentials: -1.0...3.1 KJ/mol
    Frame 2: micro heat current: 4591803381.9 KJ
    Frame 3: min/max velocities: 0.2...6.0 A
    Frame 3: min/max lj_potentials: -1.0...10.2 KJ/mol
    Frame 3: micro heat current: 2902303375.6 KJ
    Frame 4: min/max velocities: 0.3...7.9 A
    Frame 4: min/max lj_potentials: -1.0...3.0 KJ/mol
    Frame 4: micro heat current: 2165019219.1 KJ
    Frame 5: min/max velocities: 0.3...5.5 A
    Frame 5: min/max lj_potentials: -1.0...3.8 KJ/mol
    Frame 5: micro heat current: 1714668810.8 KJ
    Frame 6: min/max velocities: 0.2...5.7 A
    Frame 6: min/max lj_potentials: -1.0...2.9 KJ/mol
    Frame 6: micro heat current: 1067363987.0 KJ
    Frame 7: min/max velocities: 0.3...5.9 A
    Frame 7: min/max lj_potentials: -1.0...3.8 KJ/mol
    Frame 7: micro heat current: 979540643.7 KJ
    Frame 8: min/max velocities: 0.2...6.1 A
    Frame 8: min/max lj_potentials: -1.0...2.7 KJ/mol
    Frame 8: micro heat current: 1384076194.7 KJ
    Wrote PDB trajectory for Argon with distances in bfactor field

**Unit Conversions**

The following are from the GROMACS manual:

| Quanitity | Symbol | Units |
|---|---|---|
| velocity | *v* | $nm \space ps^{-1} \space or \space 1000 \space m s^{-1}$ |
| length | r | $10^{-9} m$ |
| mass | m | $ 1.660 538 921 x 10^{-27} kg $ |
| time | t | $ 10^{-12} s $ |
| temperature | K | $K$ |
| force | F | $KJ \space mol^{-1} \space nm^{-1}$ |

```python
u.atoms[100].position
```

    array([ 3.641   , 14.565001, 14.565001], dtype=float32)

```python
print(pairwise_velocities[:3])
print(pairwise_forces[:3])
print(pairwise_distances[:3])
```

    [[0.         3.62564802 5.1666379  ... 3.4837029  2.53749013 3.05304861]
     [0.         0.         4.17716503 ... 1.83053863 2.49247622 2.5765872 ]
     [0.         0.         0.         ... 3.2197969  2.74619532 4.67313719]]
    [[            nan -3.28339070e-06 -6.21306281e-06 ... -1.47709258e-02
      -1.09398735e-01 -4.43879878e+00]
     [            nan             nan -2.32991165e-06 ... -2.17548511e-05
      -1.48424816e-05 -7.42463331e-06]
     [            nan             nan             nan ... -3.24581068e-05
      -2.39331840e-05 -1.31726383e-05]]
    [[0.         3.7890213  3.47293701 ... 1.13412371 0.85463076 0.46293979]
     [0.         0.         4.00438271 ... 2.90923347 3.05787697 3.39182014]
     [0.         0.         0.         ... 2.74365005 2.86722374 3.14381886]]

```python
# np.save("argon_heat_currents_2.npy", heat_current)
```

```python
hc = np.load("argon_heat_currents_2.npy")
# hc = heat_current
```

```python
import matplotlib.pyplot as plt
```

```python
plt.plot(hc)
```

We have the heat currents, now we need to compute the integral

$\lambda= \frac{1}{3VK_{B}T^2} \int_{0}^{\infty} \langle j(0) \cdot j(t) \rangle \, \mathrm{d}t$

```python
# create timestep array (ps)
# timestep is dt * nstxout
# 0.002 * 100
dt_array = np.arange(0,1000,.2)
```

```python
from scipy.integrate import simps
```

```python
integrated_hc = simps(np.dot(hc[0], hc))#, dt_array)
```

```python
box_length = 29.130 # A
volume = box_length**3 # A^3
temperature = 100 # K
```

```python
# compute the thermal conductivity
# need to convert to W/M K
gk_heat_current(kb, volume, temperature, integrated_hc)
```

    195334669246026.1

# Things to look at

https://aip.scitation.org/doi/10.1063/1.4749265

https://courses.physics.illinois.edu/phys466/sp2011/projects/2004/Team1/index.html

https://pubs.acs.org/doi/10.1021/acs.analchem.7b05350


# Neural Model - Binaural Integration

```python
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:60% !important; }</style>"))
```

<style>.container { width:60% !important; }</style>

```python
%pwd
%cd ..
%pwd
```

    /home/oesst/ownCloud/PhD/Code/Python/hrtf_neural_model

    '/home/oesst/ownCloud/PhD/Code/Python/hrtf_neural_model'

In this notebook we present a first implementation of a neural model for sound source elevation estimation based on the computational HRTF model.
We use $binaural = \frac{Left}{Left + Right} $ an integration stage since it is more biologically plausible than the regular integration $\frac{Left}{Right}$.

To achieve similar responses we apply the same dynamical adaptation of the decay rate in the $r_{ipsi}$ neurons as demonstrated before (see Notebook *Neural Model - Gaussian Filtering*), where

\begin{equation}
 \tau \dot r^{ipsi}_\omega = - \alpha \cdot r_\omega + (\beta - r_\omega) \cdot {EX}_{\omega} - \kappa \cdot r_\omega \cdot {In}_{\omega}
\end{equation}

is extended to

\begin{equation}
 \tau \dot r^{ipsi}_\omega = - \alpha \cdot r_\omega \cdot {EX}_{\omega}  + (\beta - r_\omega) \cdot {EX}_{\omega} - \kappa \cdot r_\omega \cdot {In}_{\omega}
\end{equation}

```python
# Import necessary modules
from pathlib import Path
from src.data import generateData
from src.features import helpers as hp
from src.features import helpers_vis as hpVis
import numpy as np
import matplotlib.pyplot as plt
import os
from IPython.display import clear_output
from scipy.ndimage import gaussian_filter1d,convolve1d

ROOT = Path(os.getcwd()).resolve().parents[0]

SOUND_FILES = ROOT / 'data/raw/sound_samples/'
# create a list of the sound files
SOUND_FILES = list(SOUND_FILES.glob('**/*.wav'))
```

```python
############################################################
#                  Simulation functions                    #
############################################################

tau = 0.005

# Defines the output transfer function of q_A_SC
def out_thres(q,threshold = 0.0,slope=1):
    return np.minimum(np.maximum((q-threshold)*slope,0),1)

# Defines the sigmoidal transfer function
def out_sigmoid(x,slope =5,operation_point=0.0):
#     return 1 / (1 + np.exp(-(x-working_point)*slope) )
    tmp = (x-operation_point)*slope
    return (2 / (1 + np.exp(-4*tmp**2)))-1

#define a gauss function
def gauss(x,mean,sigma):
    if sigma == 0.0:
        return np.zeros(x.shape)
    else:
        tmp = np.exp(-(x-mean)**2 /( 2 * sigma**2 ))
        return tmp/np.max(tmp)

# define the ODE for inhibitory input neurons
def ode_p_in(p ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1.0
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha=1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_p  = -alpha *p  + (beta -p )*excitatory_in

    return d_p /tau

# define the ODE for gaussian filter neurons
def ode_r_in(r,excitatory_in,inhibitory_in):
    # tau defines how fast the membrane potential builds up
#     tau = 1.0
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 200
    # gamma defines the subtractive influence of the inhibitory input
    gamma = 0.0
    # kappa defines the divisive influence of the inhibitory input
    kappa = 200

    # calculate the change of r_Alearn
    d_r  = -alpha  * r * excitatory_in  + (beta -r ) * excitatory_in  - (gamma  + kappa  * r ) * inhibitory_in

    return d_r / tau

# define the ODE for neuron p_sum
def ode_p_sum(p ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_p  = -alpha *p  + (beta -p )*excitatory_in

    return d_p /tau

# define the ODE for integration neurons
def ode_r(r,excitatory_in,inhibitory_in=0):
    # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 2
    # gamma defines the subtractive influence of the inhibitory input
    gamma = 0
    # kappa defines the divisive influence of the inhibitory input
    kappa = 1

    # calculate the change of r_Alearn
    d_r  = -alpha  * r * excitatory_in   + (beta -r ) * excitatory_in  - (gamma  + kappa  * r ) * inhibitory_in

    return d_r /tau

# define the ODE for read out neurons
def ode_q_sum(q ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_q  = -alpha *q  + (beta -q )*excitatory_in

    return d_q /tau

```

### Parameters

```python
########################################################################
######################## Set parameters ################################
########################################################################
azimuth = 12
snr = 0.0
freq_bands = 128
participant_number = 1

normalize = False
time_window = 0.1  # time window in sec

# filtering parameters
normalization_type = 'sum_1'
sigma_smoothing = 0
sigma_gauss_norm = 1

# use the mean subtracted map as the learned map
mean_subtracted_map = True

ear = 'ipsi'

elevations = np.arange(0, 25, 1)

############################################################
#                  Simulation parameter                    #
############################################################

# Time step
dt = 0.0001 # ->

# Solve differential equation from time 0 to time T
T = 0.3#
# Descretize time into equal steps
ts = np.linspace (0 , T , int ( T / dt )+1)

```

```python

```

### Simulation

```python
#generate inputs
psd_all_c, psd_all_i = generateData.create_data(freq_bands, participant_number, snr, normalize, azimuth, time_window)

# Take only given elevations
input_c = psd_all_c[:, elevations, :]
input_i = psd_all_i[:, elevations, :]

# normalize inputs over frequencies
input_c = input_c / input_c.sum(2)[:,:,np.newaxis]
input_i = input_i / input_i.sum(2)[:,:,np.newaxis]

# Define neurons
# sounds_types = np.array([0,4,6,9,10])
sounds_types = np.arange(0,20)
# sounds_types = np.array([0])

n_sounds = len(sounds_types)
# elevations_angles = np.array([0])
elevations_angles = np.arange(0,25,1)
n_elevations = len(elevations_angles)

sigma = 3

p_in_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
p_in_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

r_in_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
r_in_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

p_sum_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
p_sum_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

r_ipsi = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

q = np.zeros((n_sounds,n_elevations,len(ts),len(elevations)))

x_kernel = np.arange(freq_bands)

gauss_kernel = gauss(np.arange(-4*sigma,4*sigma),0,sigma)

for i_sound,sound in enumerate(sounds_types):
    for i_ele,ele in enumerate(elevations_angles):

        in_i = input_i[sound,ele]
        in_c = input_c[sound,ele]

        # since the input does not change over time. We can do this calculation ouside the loop
        excitatory_in_p_i = convolve1d(out_thres(in_i),weights=gauss_kernel,axis=0,mode='reflect')
        excitatory_in_p_c = convolve1d(out_thres(in_c),weights=gauss_kernel,axis=0,mode='reflect')

        for t in range(0,len(ts)-1):

            ## p_In_ipsi neuron
            # feed inputs ipsi inhibition
            p_in_i[i_sound,i_ele, t+1,:] = p_in_i[i_sound,i_ele, t,:] + dt* ode_p_in(p_in_i[i_sound,i_ele,t,:],excitatory_in_p_i)

            ## r_In_ipsi neuron
            excitatory_in = out_thres(in_i)
            inhibitory_in = out_thres(p_in_i[i_sound,i_ele, t,:])
            r_in_i[i_sound,i_ele, t+1,:] = r_in_i[i_sound,i_ele, t,:] + dt* ode_r_in(r_in_i[i_sound,i_ele,t,:],excitatory_in,inhibitory_in)

            ## p_In_contra neuron
            # feed inputs ipsi inhibition
            p_in_c[i_sound,i_ele, t+1,:] = p_in_c[i_sound,i_ele, t,:] + dt* ode_p_in(p_in_c[i_sound,i_ele,t,:],excitatory_in_p_c)

            ## r_In_contra neuron
            excitatory_in = out_thres(in_c)
            inhibitory_in = out_thres(p_in_c[i_sound,i_ele, t,:])
            r_in_c[i_sound,i_ele, t+1,:] = r_in_c[i_sound,i_ele, t,:] + dt* ode_r_in(r_in_c[i_sound,i_ele,t,:],excitatory_in,inhibitory_in)

            ## p_sum neurons
            excitatory_in = out_thres(r_in_i[i_sound,i_ele, t,:])
            p_sum_i[i_sound,i_ele, t+1,:] = p_sum_i[i_sound,i_ele, t,:] + dt* ode_p_sum( p_sum_i[i_sound,i_ele, t,:],excitatory_in)

            excitatory_in = out_thres(r_in_c[i_sound,i_ele, t,:])
            p_sum_c[i_sound,i_ele, t+1,:] = p_sum_c[i_sound,i_ele, t,:] + dt* ode_p_sum(p_sum_c[i_sound,i_ele, t,:],excitatory_in)

            ## r_ipsi neuron
            excitatory_in = out_thres(r_in_i[i_sound,i_ele, t,:])
            inhibitory_in = out_thres(p_sum_c[i_sound,i_ele, t,:]) # + out_thres(p_sum_i[i_sound,i_ele, t,:])
            r_ipsi[i_sound,i_ele, t+1,:] = r_ipsi[i_sound,i_ele, t,:] + dt* ode_r(r_ipsi[i_sound,i_ele,t,:],excitatory_in,inhibitory_in)

            if t%500 == 0:
                clear_output(wait=True)
                print('Sound No: '+str(i_sound+1)+' of '+str(n_sounds)+'.\n  -> Elevation : '+str(i_ele+1)+' of '+str(n_elevations)+'.\n  -> Time : '+str(t))
```

    Sound No: 20 of 20.
      -> Elevation : 25 of 25.
      -> Time : 2500

## Single Sound Comparison of Different Model Stages

```python
fig = plt.figure(figsize=(10,10))

axs = fig.subplots(nrows=5,ncols=2,squeeze=False)

sound = 0
filter_sigma = 1

in_i = np.squeeze(input_i[sounds_types[sound],:,:])
in_c = np.squeeze(input_c[sounds_types[sound],:,:])

ax = axs[0,0]
ax.set_title('Ipsi')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[0,1]
ax.set_title('Gaussian Model')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[1,0]
tmp1 = np.squeeze(p_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('P Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,1]
tmp1 = in_i
tmp1 = gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[2,0]
tmp1 = np.squeeze(r_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('R_in Neuron')
fig.colorbar(a,ax=ax)

ax = axs[2,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
ax.set_ylabel('S/Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[3,0]
tmp1 = np.squeeze(p_sum_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('P_sum_i Neuron')
fig.colorbar(a,ax=ax)

ax = axs[3,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh((tmp1+tmp2))
ax.set_ylabel('SUM ')
fig.colorbar(a,ax=ax)

ax = axs[4,0]
tmp1 = (np.squeeze(r_ipsi[sound,:,-1:,:]))
a = ax.pcolormesh(tmp1)
ax.set_ylabel('R_ipsi Neuron')
fig.colorbar(a,ax=ax)

ax = axs[4,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1/(tmp1+tmp2))
ax.set_ylabel('R_in Neuron')
fig.colorbar(a,ax=ax)

plt.show()

```

## Comparison of all Sounds

```python
fig = plt.figure(figsize=(10,10))

sounds_types = np.arange(0,20)

axs = fig.subplots(nrows=len(sounds_types),ncols=2,squeeze=False)

for i_sounds,sound in enumerate(sounds_types):

    in_i = np.squeeze(input_i[sound,:,:])
    in_c = np.squeeze(input_c[sound,:,:])

    ax = axs[i_sounds,0]
    tmp1 = np.squeeze(r_ipsi[i_sounds,:,-1:,:])
    a = ax.pcolormesh(tmp1)
    if i_sounds == 0:
        ax.set_title('R_ipsi Neuron')
    fig.colorbar(a,ax=ax)

    ax = axs[i_sounds,1]
    if i_sounds == 0:
        ax.set_title('Binaural Integration')
    tmp1 = in_i
    tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=1, mode='nearest', axis=1)

    tmp2 = in_c
    tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=1, mode='nearest', axis=1)
    a = ax.pcolormesh(tmp1/(tmp1+tmp2))
#     fig.colorbar(a,ax=ax)

plt.show()
```

## Averaged Map

```python
all_maps = np.zeros((len(sounds_types),len(elevations),freq_bands))
all_maps_neural = np.zeros((len(sounds_types),len(elevations),freq_bands))

for i_sounds,sound in enumerate(sounds_types):

    in_i = np.squeeze(input_i[sound,:,:])
    in_c = np.squeeze(input_c[sound,:,:])

    # Neural Map
    tmp1 = np.squeeze(r_ipsi[i_sounds,:,-1:,:])
    all_maps_neural[i_sounds,:,:] = tmp1

    # Comp Map
    tmp1 = in_i
    tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=1, mode='nearest', axis=1)

    tmp2 = in_c
    tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=1, mode='nearest', axis=1)
    tmp3 = tmp1/(tmp1+tmp2)
    all_maps[i_sounds,:,:] = tmp3

fig = plt.figure(figsize=(10,5))

axs = fig.subplots(nrows=1,ncols=2,squeeze=False)

ax = axs[0,0]
ax.set_title('Neural Averaged Map')
tmp = np.mean(all_maps_neural,axis=0)
c = ax.pcolormesh(tmp)
# plt.colorbar(c)

ax = axs[0,1]
ax.set_title('Comp. Averaged Map')
tmp = np.mean(all_maps,axis=0)
c = ax.pcolormesh(tmp)
# plt.colorbar(c)

plt.show()
```

```python

```

```python
fig = plt.figure(figsize=(20,5))

axs = fig.subplots(nrows=2,ncols=5,squeeze=False)

sound = 0
filter_sigma = 1

in_i = np.squeeze(input_i[sounds_types[sound],:,:])
in_c = np.squeeze(input_c[sounds_types[sound],:,:])

ax = axs[0,0]
ax.set_ylabel('Ipsi')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
ax.set_title('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[1,0]
ax.set_ylabel('Contra')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
# ax.set_title('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[0,1]
tmp1 = np.squeeze(p_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_title('P Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,1]
tmp1 = in_i
tmp1 = gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
# ax.set_title('Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[0,2]
tmp1 = np.squeeze(r_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_title('R_in Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,2]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
# ax.set_title('S/Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[0,3]
tmp1 = np.squeeze(p_sum_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_title('P_sum_i Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,3]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh((tmp1+tmp2))
# ax.set_title('SUM ')
fig.colorbar(a,ax=ax)

ax = axs[0,4]
tmp1 = (np.squeeze(r_ipsi[sound,:,-1:,:]))
a = ax.pcolormesh(tmp1)
ax.set_title('R_ipsi Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,4]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1/(tmp1+tmp2))
# ax.set_title('R_in Neuron')
fig.colorbar(a,ax=ax)

plt.tight_layout()
plt.show()

```

```python

```

notebooks/Neural Model - Binaural Integration.ipynb


```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import control
from ipywidgets import interact

ts = np.linspace(0, 10, 1000)

def pzmap_k(kc):
    plt.rcParams['figure.figsize'] = 10,4

    print(kc)
    ax_pzmap = plt.subplot2grid((1, 2), (0, 0))
    ax_resp = plt.subplot2grid((1, 2), (0, 1))

    gLC = control.feedback(kc*Gp,1)
    _, yLC = control.step_response(gLC, ts)

    pLC = gLC.pole()
    cLC = gLC.zero()

    ax_pzmap.plot(pLA.real, pLA.imag, 'rx',
                  cLA.real, cLA.imag, 'ro',
                  pLC.real, pLC.imag, 'bx',
                  cLC.real, cLC.imag, 'bo')
    ax_pzmap.legend(('p LA', 'c LA', 'p LC','c LC'))
    ax_pzmap.axis(visualizacion_s)
    ax_pzmap.grid()
    ax_resp.plot(ts,yLA,'r',ts,yLC,'b')
    ax_resp.axis(visualizacion_t)
    ax_resp.grid()

def rlocus_int(kc):
    print(kc)
    plt.rcParams['figure.figsize'] = 10,4
    ax_pzmap = plt.subplot2grid((1, 2), (0, 0))
    ax_resp = plt.subplot2grid((1, 2), (0, 1))
    _, y = control.step_response(Gp, ts)

    gLC = control.feedback(kc*Gp,1)
    _, yLC = control.step_response(gLC, ts)

    rlist,klist = control.root_locus(Gp,plot = False);

    pLC = gLC.pole()
    cLC = gLC.zero()

    ax_pzmap.plot(pLA.real, pLA.imag, 'rx',
                  cLA.real, cLA.imag, 'ro',
                  pLC.real, pLC.imag, 'bx',
                  cLC.real, cLC.imag, 'bo',
                  rlist.real, rlist.imag, '-k')
    ax_pzmap.legend(('p LA', 'c LA', 'p LC','c LC','LGDR'))
    ax_pzmap.axis(visualizacion_s)
    ax_pzmap.grid()
    ax_resp.plot(ts,yLA,'r',ts,yLC,'b')
    ax_resp.axis(visualizacion_t)
    ax_resp.grid()

```

# Lugar Geométrico de las Raíces

Dado el bucle de control y cosideraciones en los sistemas que lo componen, la función de transferencia de un proceso controlado es:

\begin{equation}
\frac{C(s)}{R(s)} = \frac{G_c(s)G_p(s)}{1 + G_c(s)G_p(s)H(s)}
\end{equation}

Para efectos prácticos considere que $H(s)=1$. Si además se tiene en cuenta que los sistemas son LTI, puede considerarse que sus modelos corresponden a una fracción de polinomios.

$$G_c(s)= \frac{N_c(s)}{D_c(s)}$$
$$G_p(s)= \frac{N_p(s)}{D_p(s)}$$

Y la función de transferencia de lazo cerrado queda:

$$\frac{C(s)}{R(s)} = \frac{\frac{N_c(s)N_p(s)}{D_c(s)D_p(s)}}{\frac{D_c(s)D_p(s) + N_c(s)N_p(s)}{D_c(s)D_p(s)}} = \frac{N_c(s)N_p(s)}{D_c(s)D_p(s) + N_c(s)N_p(s)}$$

Cuando se considera que $G_c(s)=k_c$, la función de transferencia de lazo cerrado se reduce a:

\begin{equation}
\frac{C(s)}{R(s)} = \frac{k_c G_p(s)}{1 + k_c G_p(s)}
\end{equation}

o

$$\frac{C(s)}{R(s)} = \frac{k_cN_p(s)}{D_p(s) + k_cN_p(s)}$$

Observe que:
- Los polos del sistema en lazo cerrado dependen de qué valor tome $k_c$.
- Los ceros del sistema en lazo cerrado son los mismos que en lazo abierto al menos si $k_c   \neq 0$.

**¿Cómo varía la ubicación de los polos en función de $k_c$?**

**Ejemplo**

Suponga un proceso modelado por:

$$G_p(s) = \frac{1}{s - 1}$$

y una estrategia de contro definida por:

$$G_c(s) = k_c$$

```python
# Se define la función de transferencia del proceso
Gp = control.tf([1], [1,-1])
Gp
```

$$\frac{1}{s - 1}$$

Varíe $k_c$ y observe cómo varía la posición de los polos del sistema.

```python
visualizacion_s = [-10, 10, -5, 5]
visualizacion_t = [0, 10, -2, 2]

pLA = Gp.pole()
cLA = Gp.zero()
_, yLA = control.step_response(Gp, ts)

interact(pzmap_k,
         kc=(-10., 10.))
```

    interactive(children=(FloatSlider(value=0.0, description='kc', max=10.0, min=-10.0), Output()), _dom_classes=(…

    <function __main__.pzmap_k(kc)>

¿Qué pasa con un sistema de segundo orden?

```python
# Se define la función de transferencia del proceso
# Gp = control.tf(2, [1,4,1])
Gp = control.tf(2, [1,3,2,0])

Gp

```

$$\frac{2}{s^3 + 3 s^2 + 2 s}$$

```python
visualizacion_s = [-10, 10, -5, 5]
visualizacion_t = [0, 10, -2, 2]

pLA = Gp.pole()
cLA = Gp.zero()
_, yLA = control.step_response(Gp, ts)

interact(pzmap_k,
         kc=(0., 10.))
```

    interactive(children=(FloatSlider(value=5.0, description='kc', max=10.0), Output()), _dom_classes=('widget-int…

    <function __main__.pzmap_k(kc)>

Observe que en ambos casos, al variar el parámetro $k_c>0$,los polos del sistema en lazo cerrado describen trayectorias contiguas en el plano complejo $s$. A estas trayectorias se les conoce como [**Lugar Geométrico de las Raíces**](https://en.wikipedia.org/wiki/Root_locus).

En este caso se ha variado el valor $k_c$, sin embargo es posible generar lugares variando un parámetro diferente dentro de $G_c(s)$ o $H(s)$.

Hay técnicas numéricas para dibujar una aproximación del Lugar Geométrico de las Raíces, sin embargo, se usarán técnicas computacionales.

**Ejemplo**

Considere un sistema de orden 3.

$$G_p(s) = \frac{2}{s \left ( s^2 + 3s + 2\right ) }$$

Construya el Lugar de las raíces.

```python
# Se define la función de transferencia del proceso
Gp = control.tf([1,4,3], [1,3,2])
Gp
```

$$\frac{s^2 + 4 s + 3}{s^2 + 3 s + 2}$$

```python
visualizacion_s = [-5, 5, -5, 5]
visualizacion_t = [0, 10, -2, 4]

pLA = Gp.pole()
cLA = Gp.zero()
_, yLA = control.step_response(Gp, ts)

rlocus_int(0.3)
```

```python
visualizacion_s = [-5, 5, -5, 5]
visualizacion_t = [0, 10, -2, 4]

pLA = Gp.pole()
cLA = Gp.zero()
_, yLA = control.step_response(Gp, ts)

interact(rlocus_int,
         kc=(0., 5.))
```

    interactive(children=(FloatSlider(value=2.5, description='kc', max=5.0), Output()), _dom_classes=('widget-inte…

    <function __main__.rlocus_int(kc)>

Observe:
- Este sistema es inestable en lazo abierto.
- Al cerrar el lazo, el sistema se estabiliza pero si la ganancia sobrepasa un valor, se vuelve inestable nuevamente.

Retomando la expresión

$$\frac{C(s)}{R(s)} = \frac{k_cN_p(s)}{D_p(s) + k_cN_p(s)}$$

Se llama ecuación característica a $D_p(s) + k_cN_p(s) = 0$. Esta permite definir la ubicación de los polos de lazo cerrado.

**Preguntas**

- ¿Hacia dónde se mueven los polos de lazo cerrado cuando $k_c \rightarrow \infty$?
- ¿Puede encontrar los rangos de $k_c$ que hacen estable al sistema?

Repita los análisis realizados con:

$$G_p(s) = \frac{s+3}{\left (s+2 \right )\left (s+1 \right )} $$

```python

```


# Music Machine Learning - Sampling

### Author: Philippe Esling (esling@ircam.fr)

In this course we will cover
1. Defining [sampling](#sampling) procedure with Monte-Carlo.
2. Motivating the use of [Markov chains](#markov) in sampling.
2. Implementing the [Gibbs sampling](#gibbs) algorithm.

We begin with the standard imports

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
import numpy as np
from helper_plot import hdr_plot_style
hdr_plot_style()
```

## Sampling

The main idea of sampling, is that we have a very complicated distribution, for which we do not know either the closed-form or its parameters. Although it is hard to find an adequate estimation of this distribution, we can try to solve simpler problems, by estimating _expected values_ of this distribution. To do so, we will define an _estimator_, for which we know that it will provide an accurate evaluation if we have a sufficient number of samples.

#### Example with $\pi$

A simple example to understand the underlying logic is that of trying to approximate $\pi$. We know that we have the following relations, which means that we can estimate $\pi$ as a form of expectation, by drawing _random samples_

$$
\begin{align}
\frac{\pi}{4} &= \mathbb{E}\left[ x^{2} + y^{2} \leq 1 \right] \\
&\approx \frac{1}{M} \sum_{s=1}^{M} \left[ x_{s}^{2} + y_{s}^{2} \leq 1 \right]
\end{align}
$$

with $x_{s},y_{s}\sim \mathcal{U}(0,1)$

```python
import random as r
import math as m
# Number of darts that land inside.
inside = 0
inside_pts = []
outside_pts = []
# Total number of darts to throw.
total = 10000000
# Iterate for the number of darts.
for i in range(0, total):
    # Generate random x, y in [0, 1].
    x2 = r.random() ** 2
    y2 = r.random() ** 2
    # Increment if inside unit circle.
    if (x2 + y2) <= 1.0:
        inside += 1
        inside_pts.append([m.sqrt(x2), m.sqrt(y2)])
    else:
        outside_pts.append([m.sqrt(x2), m.sqrt(y2)])
# Compute estimator
pi = (float(inside) / total) * 4
print(pi)
# Plot the points
inside_pts = np.array(inside_pts)
outside_pts = np.array(outside_pts)
#plt.figure(figsize=(10,8))
#plt.scatter(inside_pts[:, 0], inside_pts[:, 1], s=10)
#plt.scatter(outside_pts[:, 0], outside_pts[:, 1], s=10, c='r');
```

    3.1415432

Advantages of Monte-Carlo
- Very simple to program
- Universally applicable to lots of problems
- Scalable to parallelization

Problems
- Usually very slow
- Better solutions might exist

## Formal introduction

The goal of Monte-Carlo methods is to _estimate expected values by sampling_
So we can define
$$
\mathbb{E}_{p(x)} f(x) \approx \frac{1}{M} \sum_{s=1}^{M} f(x_{s})
$$
with $x_{s} \sim p(x)$. This has nice co-properties
- This is an _unbiased_ estimator
- Guarantees on the convergence

### Why estimate expected values ?

Remember the goal of full Bayesian inference
$$
\begin{align}
p(y \vert x, Y_{train}, X_{train}) &= \int p(y \vert x, w) p(w \vert y_{train}, x_{train}) dw \\
&= \mathbb{E}_{p(w\vert Y_{train},X_{train})} p(y \vert x, w)
\end{align}
$$
Here we can use sample to simplify the problem by _sampling_

The same situation applies for the M-step of the EM-algorithm
$$
\underset{\theta}{\max} \mathbb{E}_{q} \log p(\mathbf{x},\mathbf{t} \vert \theta)
$$

## Sampling from 1d distributions

We start from the simplest case, where we have a discrete 1-d distribution. Imagine we want to sample from a Gaussian, so we can use the central limit theorem
$$
z = \sum_{i=1}^{12} x_{i} - 6
$$
If we take $x_{i} \sim \mathcal{U}[0,1]$, then we have $p(z)\approx\mathcal{N}(0,1)$.
Of course this is a rather crude approximation, we can rather use a more refined process

Now imagine we have a more complex distribution like this

```python
import math
import scipy.stats as stats
x = np.linspace(-7, 8, 100)
plt.figure(figsize=(10,8))
plt.plot(x, stats.norm.pdf(x, -1, 1.1) + stats.norm.pdf(x, 2, 0.9), linewidth = 5)
```

We can upper-bound our distribution by an approximation, for instance we have that
$$
\begin{align}
q(x) &= \mathcal{N}(1,3) \\
p(x) &\leq 2q(x)
\end{align}
$$
We use the constant $2$ as it is impossible to upper bound a distribution without multiplying it with a constant.

```python
import math
import scipy.stats as stats
x = np.linspace(-7, 8, 100)
plt.plot(x, 0.5 * stats.norm.pdf(x, -1, 1.1) + 0.5 * stats.norm.pdf(x, 2, 0.9), linewidth=3)
plt.plot(x, stats.norm.pdf(x, 1, 3) * 2, c='r', linewidth=3)
plt.savefig('sampling_approx.pdf')
```

```python
x = np.linspace(-7, 8, 100)
mean = 1;
plt.plot(x, stats.norm.pdf(x, 0, 3) + mean, c='r', linewidth=3)
```

- Now note that we can sample from this approximation $q$ such that $\tilde{x}\sim q(x)$
- So how can we sample from our distribution $p$
- We now that we have a higher probability in $q$
- Therefore we need to *reject* some of the points
- The rule to reject should be probabilistic
    - Should be proportional to the height of the curves
    - So if we sample $\tilde{x}$ and take its coordinates $y \sim \mathcal{U}[0, 2q(\tilde{x})]$.
    - We can accept $\tilde{x}$ with probability $\frac{p(x)}{2q(x)}$ if $y\leq p(x)$
How much points do we accept ? Knowing that we have $p(x)\leq Mq(x)$, we accept $\frac{1}{M}$ points on average

Note that we can even use this _rejection sampling_ method if we know the distribution up to a normalization factor as
$$
\begin{align}
\frac{\hat{p}(x)}{Z} &\leq Mq(x) \\
\hat{p}(x) &\leq ZMq(x) \\
\end{align}
$$
For instance, we can define our distributions $p$ and $q$ as follows

```python
import scipy.stats as st
def p(x):
    return st.norm.pdf(x, loc=30, scale=10) + st.norm.pdf(x, loc=80, scale=20)
def q(x):
    return st.norm.pdf(x, loc=50, scale=30)
```

Here, you can implement the algorithm for rejection sampling and test it out

```python
x = np.arange(-50, 151)
k = max(p(x) / q(x))

def rejection_sampling(iter=1000):
    samples = []

    ######################
    # YOUR CODE GOES HERE
    ######################

    ######################
    # Solution
    for i in range(iter):

        z = np.random.normal(50, 30)
        u = np.random.uniform(0, k*q(z))

        if u <= p(z):
            samples.append(z)

    ######################
    return np.array(samples)

plt.plot(x, p(x))
plt.plot(x, k*q(x))
plt.show()
s = rejection_sampling(iter=10000)
sns.distplot(s);
s.shape
```

Summary of rejection sampling
+ Pro : Works for most distributions
- Con : If $q$ and $p$ too different ($M$ large), rejects most points
- Con : $M$ is large for d-dimensional distributions

## Markov chains

**Here explain the principle of Markov chains**

Using Markov chain for sampling
- We want to sample from p(x)
- Build a Markov chain that converges to $p(x)$
- Start from any $x^{0}$
- For $k=0,1,\cdots$
$$
x^{k+1}\sim T(x^{k}\rightarrow x^{k+1})
$$
- Eventually $x^{k}$ will look like samples from p(x)

### Do chains always converge ?

- Imagine a binary distribution
- Will never converge
- We need chains that do converge

**Definition** : A distribution $\pi$ is called _stationary_ if
$$
\pi(x')=\sum_{x} T(x\rightarrow x') \pi(x)
$$

**Theorem** : If $T(x\rightarrow x') > 0$ for all $x, x'$, then there exists a unique $\pi$ such that
$$
\pi(x')=\sum_{x} T(x\rightarrow x') \pi(x)
$$
And the Markov chain converges to $\pi$ from any starting point.

## Gibbs sampling

One of the easiest method to construct Markov chains to sample from a distribution. Say we have a 3-d distribution that we know up to norrmalization.
$$
p(x_{1}, x_{2}, x_{3}) = \frac{\hat{p}(x_{1}, x_{2}, x_{3})}{Z}
$$
The method of Gibbs sampling starts with $(x^{0}_{1}, x^{0}_{2}, x^{0}_{3})$ that we can set to $(0,0,0)$ for instance.

Then we are going to sample each dimension at a time by sampling from the conditional
$$
\begin{align}
x^{1}_{1} \sim &p(x_{1} \vert x_{2} = x^{0}_{2}, x_{3} = x^{0}_{3}) \\
&= \frac{\hat{p}(x_{1}, x^{0}_{2}, x^{0}_{3})}{Z}
\end{align}
$$
Then we reapply the same idea for the next dimensions.
$$
\begin{align}
x^{1}_{2} &\sim p(x_{2} \vert x_{1} = x^{1}_{1}, x_{3} = x^{0}_{3}) \\
x^{1}_{3} &\sim p(x_{3} \vert x_{1} = x^{1}_{1}, x_{2} = x^{1}_{2}) \\
\end{align}
$$
Note that in this case, we cannot parallelize these sampling steps.

We can prove that Gibbs sampling indeed converges to the desired distribution $p$. So we want to prove
$$
p(x',y',z') = \sum_{x,y,z} q(x,y,z\rightarrow x',y',z') p(x,y,z)
$$

Pros :
- Reduce multidimensional sampling to sequence of 1d sampling
- A few lines of code
Cons :
- Highly correlated samples
- Slow convergence (mixing)
- Cannot be parallelized

## Metropolis-Hastings

- Sometimes Gibbs samples are too correlated
- Idea of MH is to apply rejection sampling to Markov chains
Let's start with any Markov chain $Q$
- Sample $x'$ from a _wrong_ $Q(x^{k}\rightarrow x')$ for $k=1,2,\cdots$
- Use a critic to accept proposal $x'$ with probabiility $A(x^{k}\rightarrow x')$
- Otherwise stay at $x^{k}$, such that $x^{k+1}=x^{k}$
So overall, the scheme works as
$$
\begin{align}
T(x\leftarrow x') &= Q(x\leftarrow x')A(x\leftarrow x'), \forall x\neq x'
T(x\leftarrow x) &= Q(x\leftarrow x)A(x\leftarrow x)\\
&+ \sum_{x\neq x'} Q(x\leftarrow x')(1 - A(x\leftarrow x'))
\end{align}
$$

How to choose A : $\pi(x')=\sum_{x} T(x\rightarrow x') \pi(x)$, we need to introduce the _detailed balance_ condition

## [Bonus] Gibbs Sampling implementation

The Gibbs sampler is the simplest and most prevalent MCMC algorithm. If a posterior has $k$ parameters to be estimated, we may condition each parameter on current values of the other $k-1$ parameters, and sample from the resultant distributional form (usually easier), and repeat this operation on the other parameters in turn. This procedure generates samples from the posterior distribution. Note that we have now combined Markov chains (conditional independence) and Monte Carlo techniques (estimation by simulation) to yield Markov chain Monte Carlo.

Here is a stereotypical Gibbs sampling algorithm:

1.  Choose starting values for states (parameters):
    ${\bf \theta} = [\theta_1^{(0)},\theta_2^{(0)},\ldots,\theta_k^{(0)}]$.

2.  Initialize counter $j=1$.

3.  Draw the following values from each of the $k$ conditional
    distributions:

    $$\begin{aligned}
    \theta_1^{(j)} &\sim \pi(\theta_1 | \theta_2^{(j-1)},\theta_3^{(j-1)},\ldots,\theta_{k-1}^{(j-1)},\theta_k^{(j-1)}) \\
    \theta_2^{(j)} &\sim \pi(\theta_2 | \theta_1^{(j)},\theta_3^{(j-1)},\ldots,\theta_{k-1}^{(j-1)},\theta_k^{(j-1)}) \\
    \theta_3^{(j)} &\sim \pi(\theta_3 | \theta_1^{(j)},\theta_2^{(j)},\ldots,\theta_{k-1}^{(j-1)},\theta_k^{(j-1)}) \\
    \vdots \\
    \theta_{k-1}^{(j)} &\sim \pi(\theta_{k-1} | \theta_1^{(j)},\theta_2^{(j)},\ldots,\theta_{k-2}^{(j)},\theta_k^{(j-1)}) \\
    \theta_k^{(j)} &\sim \pi(\theta_k | \theta_1^{(j)},\theta_2^{(j)},\theta_4^{(j)},\ldots,\theta_{k-2}^{(j)},\theta_{k-1}^{(j)})\end{aligned}$$

4.  Increment $j$ and repeat until convergence occurs.

As we can see from the algorithm, each distribution is conditioned on the last iteration of its chain values, constituting a Markov chain as advertised. The Gibbs sampler has all of the important properties outlined in the previous section: it is aperiodic, homogeneous and ergodic. Once the sampler converges, all subsequent samples are from the target distribution. This convergence occurs at a geometric rate.

#### Example: Inferring patterns in UK coal mining disasters

Let's try to model a more interesting example, a time series of recorded coal mining
disasters in the UK from 1851 to 1962.

Occurrences of disasters in the time series is thought to be derived from a
Poisson process with a large rate parameter in the early part of the time
series, and from one with a smaller rate in the later part. We are interested
in locating the change point in the series, which perhaps is related to changes
in mining safety regulations.

```python
disasters_array = np.array([4, 5, 4, 0, 1, 4, 3, 4, 0, 6, 3, 3, 4, 0, 2, 6,
                            3, 3, 5, 4, 5, 3, 1, 4, 4, 1, 5, 5, 3, 4, 2, 5,
                            2, 2, 3, 4, 2, 1, 3, 2, 2, 1, 1, 1, 1, 3, 0, 0,
                            1, 0, 1, 1, 0, 0, 3, 1, 0, 3, 2, 2, 0, 1, 1, 1,
                            0, 1, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 1, 1, 0, 2,
                            3, 3, 1, 1, 2, 1, 1, 1, 1, 2, 4, 2, 0, 0, 1, 4,
                            0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1])

n_count_data = len(disasters_array)
```

We are going to use Poisson random variables for this type of count data. Denoting year $i$'s accident count by $y_i$,

$$y_i \sim \text{Poisson}(\lambda).$$

For those unfamiliar, Poisson random variables look like this:

```python
plt.figure(figsize=(10, 8))
for l in [1, 5, 12, 25]:
    plt.hist(np.random.poisson(l, 1000), alpha=0.5)
```

The modeling problem is about estimating the values of the $\lambda$ parameters. Looking at the time series above, it appears that the rate declines over time.

A **changepoint model** identifies a point (here, a year) after which the parameter $\lambda$ drops to a lower value. Let us call this point in time $\tau$. So we are estimating two $\lambda$ parameters:
$\lambda = \lambda_1$ if $t \lt \tau$ and $\lambda = \lambda_2$ if $t \geq \tau$.

We need to assign prior probabilities to both $\{\lambda_1, \lambda_2\}$. The gamma distribution not only provides a continuous density function for positive numbers, but it is also *conjugate* with the Poisson sampling distribution.

```python
lambda1_lambda2 = [(0.1, 100), (1, 100), (1, 10), (10, 10)]
```

```python
plt.figure(figsize=(10, 8))
for p in lambda1_lambda2:
    plt.hist(np.random.gamma(*p, size=1000), alpha=0.5)
```

We will specify suitably vague hyperparameters $\alpha$ and $\beta$ for both priors:

\begin{align}
\lambda_1 &\sim \text{Gamma}(1, 10), \\
\lambda_2 &\sim \text{Gamma}(1, 10).
\end{align}

Since we do not have any intuition about the location of the changepoint (unless we visualize the data), we will assign a discrete uniform prior over the entire observation period [1851, 1962]:

\begin{align}
&\tau \sim \text{DiscreteUniform(1851, 1962)}\\
&\Rightarrow P(\tau = k) = \frac{1}{111}.
\end{align}

### Implementing Gibbs sampling

We are interested in estimating the joint posterior of $\lambda_1, \lambda_2$ and $\tau$ given the array of annnual disaster counts $\mathbf{y}$. This gives:

$$
 P( \lambda_1, \lambda_2, \tau | \mathbf{y} ) \propto P(\mathbf{y} | \lambda_1, \lambda_2, \tau ) P(\lambda_1, \lambda_2, \tau)
$$

To employ Gibbs sampling, we need to factor the joint posterior into the product of conditional expressions:

$$
 P(\lambda_1, \lambda_2, \tau | \mathbf{y}) \propto P(y_{t \lt \tau} | \lambda_1, \tau) P(y_{t \geq \tau} | \lambda_2, \tau) P(\lambda_1) P(\lambda_2) P(\tau)
$$

which we have specified as:

$$\begin{aligned}
P( \lambda_1, \lambda_2, \tau | \mathbf{y} ) &\propto \left[\prod_{t=1851}^{\tau} \text{Poi}(y_t|\lambda_1) \prod_{t=\tau+1}^{1962} \text{Poi}(y_t|\lambda_2) \right] \text{Gamma}(\lambda_1|\alpha,\beta) \text{Gamma}(\lambda_2|\alpha, \beta) \frac{1}{111} \\
&\propto \left[\prod_{t=1851}^{\tau} e^{-\lambda_1}\lambda_1^{y_t} \prod_{t=\tau+1}^{1962} e^{-\lambda_2} \lambda_2^{y_t} \right] \lambda_1^{\alpha-1} e^{-\beta\lambda_1} \lambda_2^{\alpha-1} e^{-\beta\lambda_2} \\
&\propto \lambda_1^{\sum_{t=1851}^{\tau} y_t +\alpha-1} e^{-(\beta+\tau)\lambda_1} \lambda_2^{\sum_{t=\tau+1}^{1962} y_i + \alpha-1} e^{-\beta\lambda_2}
\end{aligned}$$

So, the full conditionals are known, and critically for Gibbs, can easily be sampled from.

$$\lambda_1 \sim \text{Gamma}(\sum_{t=1851}^{\tau} y_t +\alpha, \tau+\beta)$$
$$\lambda_2 \sim \text{Gamma}(\sum_{t=\tau+1}^{1962} y_i + \alpha, 1962-\tau+\beta)$$
$$\tau \sim \text{Categorical}\left( \frac{\lambda_1^{\sum_{t=1851}^{\tau} y_t +\alpha-1} e^{-(\beta+\tau)\lambda_1} \lambda_2^{\sum_{t=\tau+1}^{1962} y_i + \alpha-1} e^{-\beta\lambda_2}}{\sum_{k=1851}^{1962} \lambda_1^{\sum_{t=1851}^{\tau} y_t +\alpha-1} e^{-(\beta+\tau)\lambda_1} \lambda_2^{\sum_{t=\tau+1}^{1962} y_i + \alpha-1} e^{-\beta\lambda_2}} \right)$$

Implementing this in Python requires random number generators for both the gamma and discrete uniform distributions. We can leverage NumPy for this:

```python
# Function to draw random gamma variate
rgamma = np.random.gamma

def rcategorical(probs, n=None):
    # Function to draw random categorical variate
    return np.array(probs).cumsum().searchsorted(np.random.sample(n))
```

Next, in order to generate probabilities for the conditional posterior of $\tau$, we need the kernel of the gamma density:

\\[\lambda^{\alpha-1} e^{-\beta \lambda}\\]

```python
dgamma = lambda lam, a, b: lam**(a - 1) * np.exp(-b * lam)
```

Diffuse hyperpriors for the gamma priors on $\{\lambda_1, \lambda_2\}$:

```python
alpha, beta = 1., 10
```

For computational efficiency, it is best to pre-allocate memory to store the sampled values. We need 3 arrays, each with length equal to the number of iterations we plan to run:

```python
# Specify number of iterations
n_iterations = 1000
# Initialize trace of samples
lambda1, lambda2, tau = np.empty((3, n_iterations + 1))
```

The penultimate step initializes the model paramters to arbitrary values:

```python
lambda1[0] = 6
lambda2[0] = 2
tau[0] = 50
```

Now we can run the Gibbs sampler.

```python
# Sample from conditionals
for i in range(n_iterations):

    ######################
    # YOUR CODE GOES HERE
    ######################

    ######################
    # Solution
    # Sample early mean
    lambda1[i + 1] = rgamma(disasters_array[:int(tau[i])].sum() + alpha, 1./(tau[i] + beta))

    # Sample late mean
    lambda2[i + 1] = rgamma(disasters_array[int(tau[i]):].sum() + alpha,
                            1./(n_count_data - tau[i] + beta))

    # Sample changepoint: first calculate probabilities (conditional)
    p = np.array([dgamma(lambda1[i + 1], disasters_array[:t].sum() + alpha, t + beta) *
                  dgamma(lambda2[i + 1], disasters_array[t:].sum() + alpha, n_count_data - t + beta)
                  for t in range(n_count_data)])

    # ... then draw sample
    tau[i + 1] = rcategorical(p/p.sum())

    ######################
```

Plotting the trace and histogram of the samples reveals the marginal posteriors of each parameter in the model.

```python
plt.figure()
plt.plot(lambda1)
plt.figure()
plt.hist(lambda1)
plt.figure()
plt.plot(lambda2)
plt.figure()
plt.hist(lambda2)
plt.figure()
plt.plot(tau)
plt.figure()
plt.hist(tau)
```


# Introduction to Brian part 2: Synapses

If you haven't yet read part 1: Neurons, go read that now.

As before we start by importing the Brian package and setting up matplotlib for IPython:

```python
from brian2 import *
%matplotlib inline
```

## The simplest Synapse

Once you have some neurons, the next step is to connect them up via synapses. We'll start out with doing the simplest possible type of synapse that causes an instantaneous change in a variable after a spike.

```python
start_scope()

eqs = '''
dv/dt = (I-v)/tau : 1
I : 1
tau : second
'''
G = NeuronGroup(2, eqs, threshold='v>1', reset='v = 0', method='exact')
G.I = [2, 0]
G.tau = [10, 100]*ms

# Comment these two lines out to see what happens without Synapses
S = Synapses(G, G, on_pre='v_post += 0.2')
S.connect(i=0, j=1)

M = StateMonitor(G, 'v', record=True)

run(100*ms)

plot(M.t/ms, M.v[0], label='Neuron 0')
plot(M.t/ms, M.v[1], label='Neuron 1')
xlabel('Time (ms)')
ylabel('v')
legend();
```

There are a few things going on here. First of all, let's recap what is going on with the ``NeuronGroup``. We've created two neurons, each of which has the same differential equation but different values for parameters I and tau. Neuron 0 has ``I=2`` and ``tau=10*ms`` which means that is driven to repeatedly spike at a fairly high rate. Neuron 1 has ``I=0`` and ``tau=100*ms`` which means that on its own - without the synapses - it won't spike at all (the driving current I is 0). You can prove this to yourself by commenting out the two lines that define the synapse.

Next we define the synapses: ``Synapses(source, target, ...)`` means that we are defining a synaptic model that goes from ``source`` to ``target``. In this case, the source and target are both the same, the group ``G``. The syntax ``on_pre='v_post += 0.2'`` means that when a spike occurs in the presynaptic neuron (hence ``on_pre``) it causes an instantaneous change to happen ``v_post += 0.2``. The ``_post`` means that the value of ``v`` referred to is the post-synaptic value, and it is increased by 0.2. So in total, what this model says is that whenever two neurons in G are connected by a synapse, when the source neuron fires a spike the target neuron will have its value of ``v`` increased by 0.2.

However, at this point we have only defined the synapse model, we haven't actually created any synapses. The next line ``S.connect(i=0, j=1)`` creates a synapse from neuron 0 to neuron 1.

## Adding a weight

In the previous section, we hard coded the weight of the synapse to be the value 0.2, but often we would to allow this to be different for different synapses. We do that by introducing synapse equations.

```python
start_scope()

eqs = '''
dv/dt = (I-v)/tau : 1
I : 1
tau : second
'''
G = NeuronGroup(3, eqs, threshold='v>1', reset='v = 0', method='exact')
G.I = [2, 0, 0]
G.tau = [10, 100, 100]*ms

# Comment these two lines out to see what happens without Synapses
S = Synapses(G, G, 'w : 1', on_pre='v_post += w')
S.connect(i=0, j=[1, 2])
S.w = 'j*0.2'

M = StateMonitor(G, 'v', record=True)

run(50*ms)

plot(M.t/ms, M.v[0], label='Neuron 0')
plot(M.t/ms, M.v[1], label='Neuron 1')
plot(M.t/ms, M.v[2], label='Neuron 2')
xlabel('Time (ms)')
ylabel('v')
legend();
```

This example behaves very similarly to the previous example, but now there's a synaptic weight variable ``w``. The string ``'w : 1'`` is an equation string, precisely the same as for neurons, that defines a single dimensionless parameter ``w``. We changed the behaviour on a spike to ``on_pre='v_post += w'`` now, so that each synapse can behave differently depending on the value of ``w``. To illustrate this, we've made a third neuron which behaves precisely the same as the second neuron, and connected neuron 0 to both neurons 1 and 2. We've also set the weights via ``S.w = 'j*0.2'``. When ``i`` and ``j`` occur in the context of synapses, ``i`` refers to the source neuron index, and ``j`` to the target neuron index. So this will give a synaptic connection from 0 to 1 with weight ``0.2=0.2*1`` and from 0 to 2 with weight ``0.4=0.2*2``.

## Introducing a delay

So far, the synapses have been instantaneous, but we can also make them act with a certain delay.

```python
start_scope()

eqs = '''
dv/dt = (I-v)/tau : 1
I : 1
tau : second
'''
G = NeuronGroup(3, eqs, threshold='v>1', reset='v = 0', method='exact')
G.I = [2, 0, 0]
G.tau = [10, 100, 100]*ms

S = Synapses(G, G, 'w : 1', on_pre='v_post += w')
S.connect(i=0, j=[1, 2])
S.w = 'j*0.2'
S.delay = 'j*2*ms'

M = StateMonitor(G, 'v', record=True)

run(50*ms)

plot(M.t/ms, M.v[0], label='Neuron 0')
plot(M.t/ms, M.v[1], label='Neuron 1')
plot(M.t/ms, M.v[2], label='Neuron 2')
xlabel('Time (ms)')
ylabel('v')
legend();
```

As you can see, that's as simple as adding a line ``S.delay = 'j*2*ms'`` so that the synapse from 0 to 1 has a delay of 2 ms, and from 0 to 2 has a delay of 4 ms.

## More complex connectivity

So far, we specified the synaptic connectivity explicitly, but for larger networks this isn't usually possible. For that, we usually want to specify some condition.

```python
start_scope()

N = 10
G = NeuronGroup(N, 'v:1')
S = Synapses(G, G)
S.connect(condition='i!=j', p=0.2)
```

Here we've created a dummy neuron group of N neurons and a dummy synapses model that doens't actually do anything just to demonstrate the connectivity. The line ``S.connect(condition='i!=j', p=0.2)`` will connect all pairs of neurons ``i`` and ``j`` with probability 0.2 as long as the condition ``i!=j`` holds. So, how can we see that connectivity? Here's a little function that will let us visualise it.

```python
def visualise_connectivity(S):
    Ns = len(S.source)
    Nt = len(S.target)
    figure(figsize=(10, 4))
    subplot(121)
    plot(zeros(Ns), arange(Ns), 'ok', ms=10)
    plot(ones(Nt), arange(Nt), 'ok', ms=10)
    for i, j in zip(S.i, S.j):
        plot([0, 1], [i, j], '-k')
    xticks([0, 1], ['Source', 'Target'])
    ylabel('Neuron index')
    xlim(-0.1, 1.1)
    ylim(-1, max(Ns, Nt))
    subplot(122)
    plot(S.i, S.j, 'ok')
    xlim(-1, Ns)
    ylim(-1, Nt)
    xlabel('Source neuron index')
    ylabel('Target neuron index')

visualise_connectivity(S)
```

There are two plots here. On the left hand side, you see a vertical line of circles indicating source neurons on the left, and a vertical line indicating target neurons on the right, and a line between two neurons that have a synapse. On the right hand side is another way of visualising the same thing. Here each black dot is a synapse, with x value the source neuron index, and y value the target neuron index.

Let's see how these figures change as we change the probability of a connection:

```python
start_scope()

N = 10
G = NeuronGroup(N, 'v:1')

for p in [0.1, 0.5, 1.0]:
    S = Synapses(G, G)
    S.connect(condition='i!=j', p=p)
    visualise_connectivity(S)
    suptitle('p = '+str(p))
```

And let's see what another connectivity condition looks like. This one will only connect neighbouring neurons.

```python
start_scope()

N = 10
G = NeuronGroup(N, 'v:1')

S = Synapses(G, G)
S.connect(condition='abs(i-j)<4 and i!=j')
visualise_connectivity(S)
```

Try using that cell to see how other connectivity conditions look like.

You can also use the generator syntax to create connections like this more efficiently. In small examples like this, it doesn't matter, but for large numbers of neurons it can be much more efficient to specify directly which neurons should be connected than to specify just a condition. Note that the following example uses `skip_if_invalid` to avoid errors at the boundaries (e.g. do not try to connect the neuron with index 1 to a neuron with index -2).

```python
start_scope()

N = 10
G = NeuronGroup(N, 'v:1')

S = Synapses(G, G)
S.connect(j='k for k in range(i-3, i+4) if i!=k', skip_if_invalid=True)
visualise_connectivity(S)
```

If each source neuron is connected to precisely one target neuron (which would be normally used with two separate groups of the same size, not with identical source and target groups as in this example), there is a special syntax that is extremely efficient. For example, 1-to-1 connectivity looks like this:

```python
start_scope()

N = 10
G = NeuronGroup(N, 'v:1')

S = Synapses(G, G)
S.connect(j='i')
visualise_connectivity(S)
```

You can also do things like specifying the value of weights with a string. Let's see an example where we assign each neuron a spatial location and have a distance-dependent connectivity function. We visualise the weight of a synapse by the size of the marker.

```python
start_scope()

N = 30
neuron_spacing = 50*umetre
width = N/4.0*neuron_spacing

# Neuron has one variable x, its position
G = NeuronGroup(N, 'x : metre')
G.x = 'i*neuron_spacing'

# All synapses are connected (excluding self-connections)
S = Synapses(G, G, 'w : 1')
S.connect(condition='i!=j')
# Weight varies with distance
S.w = 'exp(-(x_pre-x_post)**2/(2*width**2))'

scatter(S.x_pre/um, S.x_post/um, S.w*20)
xlabel('Source neuron position (um)')
ylabel('Target neuron position (um)');
```

Now try changing that function and seeing how the plot changes.

## More complex synapse models: STDP

Brian's synapse framework is very general and can do things like short-term plasticity (STP) or spike-timing dependent plasticity (STDP). Let's see how that works for STDP.

STDP is normally defined by an equation something like this:

$$\Delta w = \sum_{t_{pre}} \sum_{t_{post}} W(t_{post}-t_{pre})$$

That is, the change in synaptic weight w is the sum over all presynaptic spike times $t_{pre}$ and postsynaptic spike times $t_{post}$ of some function $W$ of the difference in these spike times. A commonly used function $W$ is:

$$W(\Delta t) = \begin{cases}
A_{pre} e^{-\Delta t/\tau_{pre}} & \Delta t>0 \\
A_{post} e^{\Delta t/\tau_{post}} & \Delta t<0
\end{cases}$$

This function looks like this:

```python
tau_pre = tau_post = 20*ms
A_pre = 0.01
A_post = -A_pre*1.05
delta_t = linspace(-50, 50, 100)*ms
W = where(delta_t>0, A_pre*exp(-delta_t/tau_pre), A_post*exp(delta_t/tau_post))
plot(delta_t/ms, W)
xlabel(r'$\Delta t$ (ms)')
ylabel('W')
axhline(0, ls='-', c='k');
```

Simulating it directly using this equation though would be very inefficient, because we would have to sum over all pairs of spikes. That would also be physiologically unrealistic because the neuron cannot remember all its previous spike times. It turns out there is a more efficient and physiologically more plausible way to get the same effect.

We define two new variables $a_{pre}$ and $a_{post}$ which are "traces" of pre- and post-synaptic activity, governed by the differential equations:
$$
\begin{align}
\tau_{pre}\frac{\mathrm{d}}{\mathrm{d}t} a_{pre} &= -a_{pre}\\
\tau_{post}\frac{\mathrm{d}}{\mathrm{d}t} a_{post} &= -a_{post}
\end{align}
$$

When a presynaptic spike occurs, the presynaptic trace is updated and the weight is modified according to the rule:

$$
\begin{align}
a_{pre} &\rightarrow a_{pre}+A_{pre}\\
w &\rightarrow w+a_{post}
\end{align}
$$

When a postsynaptic spike occurs:

$$
\begin{align}
a_{post} &\rightarrow a_{post}+A_{post}\\
w &\rightarrow w+a_{pre}
\end{align}
$$

To see that this formulation is equivalent, you just have to check that the equations sum linearly, and consider two cases: what happens if the presynaptic spike occurs before the postsynaptic spike, and vice versa. Try drawing a picture of it.

Now that we have a formulation that relies only on differential equations and spike events, we can turn that into Brian code.

```python
start_scope()

taupre = taupost = 20*ms
wmax = 0.01
Apre = 0.01
Apost = -Apre*taupre/taupost*1.05

G = NeuronGroup(1, 'v:1', threshold='v>1')

S = Synapses(G, G,
             '''
             w : 1
             dapre/dt = -apre/taupre : 1 (event-driven)
             dapost/dt = -apost/taupost : 1 (event-driven)
             ''',
             on_pre='''
             v_post += w
             apre += Apre
             w = clip(w+apost, 0, wmax)
             ''',
             on_post='''
             apost += Apost
             w = clip(w+apre, 0, wmax)
             ''')
```

There are a few things to see there. Firstly, when defining the synapses we've given a more complicated multi-line string defining three synaptic variables (``w``, ``apre`` and ``apost``). We've also got a new bit of syntax there, ``(event-driven)`` after the definitions of ``apre`` and ``apost``. What this means is that although these two variables evolve continuously over time, Brian should only update them at the time of an event (a spike). This is because we don't need the values of ``apre`` and ``apost`` except at spike times, and it is more efficient to only update them when needed.

Next we have a ``on_pre=...`` argument. The first line is ``v_post += w``: this is the line that actually applies the synaptic weight to the target neuron. The second line is ``apre += Apre`` which encodes the rule above. In the third line, we're also encoding the rule above but we've added one extra feature: we've clamped the synaptic weights between a minimum of 0 and a maximum of ``wmax`` so that the weights can't get too large or negative. The function ``clip(x, low, high)`` does this.

Finally, we have a ``on_post=...`` argument. This gives the statements to calculate when a post-synaptic neuron fires. Note that we do not modify ``v`` in this case, only the synaptic variables.

Now let's see how all the variables behave when a presynaptic spike arrives some time before a postsynaptic spike.

```python
start_scope()

taupre = taupost = 20*ms
wmax = 0.01
Apre = 0.01
Apost = -Apre*taupre/taupost*1.05

G = NeuronGroup(2, 'v:1', threshold='t>(1+i)*10*ms', refractory=100*ms)

S = Synapses(G, G,
             '''
             w : 1
             dapre/dt = -apre/taupre : 1 (clock-driven)
             dapost/dt = -apost/taupost : 1 (clock-driven)
             ''',
             on_pre='''
             v_post += w
             apre += Apre
             w = clip(w+apost, 0, wmax)
             ''',
             on_post='''
             apost += Apost
             w = clip(w+apre, 0, wmax)
             ''', method='linear')
S.connect(i=0, j=1)
M = StateMonitor(S, ['w', 'apre', 'apost'], record=True)

run(30*ms)

figure(figsize=(4, 8))
subplot(211)
plot(M.t/ms, M.apre[0], label='apre')
plot(M.t/ms, M.apost[0], label='apost')
legend()
subplot(212)
plot(M.t/ms, M.w[0], label='w')
legend(loc='best')
xlabel('Time (ms)');
```

A couple of things to note here. First of all, we've used a trick to make neuron 0 fire a spike at time 10 ms, and neuron 1 at time 20 ms. Can you see how that works?

Secondly, we've replaced the ``(event-driven)`` by ``(clock-driven)`` so you can see how ``apre`` and ``apost`` evolve over time. Try reverting this change and see what happens.

Try changing the times of the spikes to see what happens.

Finally, let's verify that this formulation is equivalent to the original one.

```python
start_scope()

taupre = taupost = 20*ms
Apre = 0.01
Apost = -Apre*taupre/taupost*1.05
tmax = 50*ms
N = 100

# Presynaptic neurons G spike at times from 0 to tmax
# Postsynaptic neurons G spike at times from tmax to 0
# So difference in spike times will vary from -tmax to +tmax
G = NeuronGroup(N, 'tspike:second', threshold='t>tspike', refractory=100*ms)
H = NeuronGroup(N, 'tspike:second', threshold='t>tspike', refractory=100*ms)
G.tspike = 'i*tmax/(N-1)'
H.tspike = '(N-1-i)*tmax/(N-1)'

S = Synapses(G, H,
             '''
             w : 1
             dapre/dt = -apre/taupre : 1 (event-driven)
             dapost/dt = -apost/taupost : 1 (event-driven)
             ''',
             on_pre='''
             apre += Apre
             w = w+apost
             ''',
             on_post='''
             apost += Apost
             w = w+apre
             ''')
S.connect(j='i')

run(tmax+1*ms)

plot((H.tspike-G.tspike)/ms, S.w)
xlabel(r'$\Delta t$ (ms)')
ylabel(r'$\Delta w$')
axhline(0, ls='-', c='k');
```

Can you see how this works?

## End of tutorial