# Tutorial 1: The Leaky Integrate-and-Fire (LIF) Neuron Model
**Week 2, Day 3: Biological Neuron Models**

**By Neuromatch Academy**

__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar

__Content reviewers:__  Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Matthew Krause, Spiros Chavlis, Michael Waskom

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives
This is Tutorial 1 of a series on implementing realistic neuron models. In this tutorial, we will build up a leaky integrate-and-fire (LIF) neuron model and study its dynamics in response to various types of inputs. In particular, we are going to write a few lines of code to:

- simulate the LIF neuron model

- drive the LIF neuron with external inputs, such as direct currents, Gaussian white noise, and Poisson spike trains, etc.

- study how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity)

Here, we will especially emphasize **identifying conditions (input statistics) under which a neuron can spike at low firing rates and in an irregular manner**. The reason for focusing on this is that in most cases, neocortical neurons spike in an irregular manner.

---
# Setup

```python
# Imports
import numpy as np
import matplotlib.pyplot as plt
```

##  Figure Settings

```python
# @title Figure Settings
import ipywidgets as widgets  # interactive display
%config InlineBackend.figure_format = 'retina'
# use NMA plot style
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
my_layout = widgets.Layout()
```

##  Helper functions

```python
# @title Helper functions

def plot_volt_trace(pars, v, sp):
  """
  Plot trajetory of membrane potential for a single neuron

  Expects:
  pars   : parameter dictionary
  v      : volt trajetory
  sp     : spike train

  Returns:
  figure of the membrane potential trajetory for a single neuron
  """

  V_th = pars['V_th']
  dt, range_t = pars['dt'], pars['range_t']
  if sp.size:
    sp_num = (sp / dt).astype(int) - 1
    v[sp_num] += 20  # draw nicer spikes

  plt.plot(pars['range_t'], v, 'b')
  plt.axhline(V_th, 0, 1, color='k', ls='--')
  plt.xlabel('Time (ms)')
  plt.ylabel('V (mV)')
  plt.legend(['Membrane\npotential', r'Threshold V$_{\mathrm{th}}$'],
             loc=[1.05, 0.75])
  plt.ylim([-80, -40])

def plot_GWN(pars, I_GWN):
  """
  Args:
    pars  : parameter dictionary
    I_GWN : Gaussian white noise input

  Returns:
    figure of the gaussian white noise input
  """

  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'][::3], I_GWN[::3], 'b')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{GWN}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()

def my_hists(isi1, isi2, cv1, cv2, sigma1, sigma2):
  """
  Args:
    isi1 : vector with inter-spike intervals
    isi2 : vector with inter-spike intervals
    cv1  : coefficient of variation for isi1
    cv2  : coefficient of variation for isi2

  Returns:
    figure with two histograms, isi1, isi2

  """
  plt.figure(figsize=(11, 4))
  my_bins = np.linspace(10, 30, 20)
  plt.subplot(121)
  plt.hist(isi1, bins=my_bins, color='b', alpha=0.5)
  plt.xlabel('ISI (ms)')
  plt.ylabel('count')
  plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma1, cv1))

  plt.subplot(122)
  plt.hist(isi2, bins=my_bins, color='b', alpha=0.5)
  plt.xlabel('ISI (ms)')
  plt.ylabel('count')
  plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma2, cv2))
  plt.tight_layout()
  plt.show()

# the function plot the raster of the Poisson spike train
def my_raster_Poisson(range_t, spike_train, n):
  """
  Generates poisson trains

  Args:
    range_t     : time sequence
    spike_train : binary spike trains, with shape (N, Lt)
    n           : number of Poisson trains plot

  Returns:
    Raster plot of the spike train
  """

  # find the number of all the spike trains
  N = spike_train.shape[0]

  # n should smaller than N:
  if n > N:
    print('The number n exceeds the size of spike trains')
    print('The number n is set to be the size of spike trains')
    n = N

  # plot rater
  i = 0
  while i < n:
    if spike_train[i, :].sum() > 0.:
      t_sp = range_t[spike_train[i, :] > 0.5]  # spike times
      plt.plot(t_sp, i * np.ones(len(t_sp)), 'k|', ms=10, markeredgewidth=2)
    i += 1
  plt.xlim([range_t[0], range_t[-1]])
  plt.ylim([-0.5, n + 0.5])
  plt.xlabel('Time (ms)', fontsize=12)
  plt.ylabel('Neuron ID', fontsize=12)
```

---
# Section 1: The Leaky Integrate-and-Fire (LIF) model

##  Video 1: Reduced Neuron Models

```python
# @title Video 1: Reduced Neuron Models
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="av456396195", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="rSExvwCVRYg", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

## Implementation of an LIF neuron model

Now, it's your turn to implement one of the simplest mathematical model of a neuron: the leaky integrate-and-fire (LIF) model. The basic idea of LIF neuron was proposed in 1907 by Louis Édouard Lapicque, long before we understood the electrophysiology of a neuron (see a translation of [Lapicque's paper](https://pubmed.ncbi.nlm.nih.gov/17968583/) ). More details of the model can be found in the book [**Theoretical neuroscience**](http://www.gatsby.ucl.ac.uk/~dayan/book/) by Peter Dayan and Laurence F. Abbott.

The subthreshold membrane potential dynamics of a LIF neuron is described by

\begin{eqnarray}
C_m\frac{dV}{dt} = -g_L(V-E_L) + I,\quad (1)
\end{eqnarray}

where $C_m$ is the membrane capacitance, $V$ is the membrane potential, $g_L$ is the leak conductance ($g_L = 1/R$, the inverse of the leak resistance $R$ mentioned in previous tutorials), $E_L$ is the resting potential, and $I$ is the external input current.

Dividing both sides of the above equation by $g_L$ gives

\begin{align}
\tau_m\frac{dV}{dt} = -(V-E_L) + \frac{I}{g_L}\,,\quad (2)
\end{align}

where the $\tau_m$ is membrane time constant and is defined as $\tau_m=C_m/g_L$.

You might wonder why dividing capacitance by conductance gives units of time! Find out yourself why this is true.

Below, we will use Eqn.(2) to simulate LIF neuron dynamics.

If $I$ is sufficiently strong such that $V$ reaches a certain threshold value $V_{\rm th}$, $V$ is reset to a reset potential $V_{\rm reset}< V_{\rm th}$, and voltage is clamped to $V_{\rm reset}$ for $\tau_{\rm ref}$ ms, mimicking the refractoriness of the neuron during an action potential:

\begin{eqnarray}
\mathrm{if}\quad V(t_{\text{sp}})\geq V_{\rm th}&:& V(t)=V_{\rm reset} \text{  for } t\in(t_{\text{sp}}, t_{\text{sp}} + \tau_{\text{ref}}]
\end{eqnarray}
where $t_{\rm sp}$ is the spike time when $V(t)$ just exceeded $V_{\rm th}$.

(__Note__: in the lecture slides, $\theta$ corresponds to the threshold voltage $V_{th}$, and $\Delta$ corresponds to the refractory time $\tau_{\rm ref}$.)

Thus, the LIF model captures the facts that a neuron:
- performs **spatial and temporal integration of synaptic inputs**
- generates a spike when the voltage reaches a certain threshold
- goes refractory during the action potential
- has a leaky membrane

The LIF model assumes that **the spatial and temporal integration of inputs is linear**. Also, membrane potential dynamics close to the spike threshold are **much slower** in LIF neurons than in real neurons.

## Coding Exercise 1: Python code to simulate the LIF neuron

We now write Python code to calculate our equation for the LIF neuron and simulate the LIF neuron dynamics. We will use the Euler method, which you saw in the linear systems case yesterday  to numerically integrate this equation:

\begin{align*}
\tau_m\frac{dV}{dt} = -(V-E_L) + \frac{I}{g_L}\,
\end{align*}

where $V$ is the membrane potential, $g_L$ is the leak conductance, $E_L$ is the resting potential, $I$ is the external input current, and $\tau_m$ is membrane time constant.

The cell below initializes a dictionary that stores parameters of the LIF neuron model and the simulation scheme. You can use `pars=default_pars(T=simulation_time, dt=time_step)` to get the parameters. Note that, `simulation_time` and `time_step` have the unit `ms`. In addition, you can add the value to a new parameter by `pars['New_param'] = value`.

 Execute this code to initialize the default parameters

```python
# @markdown Execute this code to initialize the default parameters

def default_pars(**kwargs):
  pars = {}

  # typical neuron parameters#
  pars['V_th'] = -55.     # spike threshold [mV]
  pars['V_reset'] = -75.  # reset potential [mV]
  pars['tau_m'] = 10.     # membrane time constant [ms]
  pars['g_L'] = 10.       # leak conductance [nS]
  pars['V_init'] = -75.   # initial potential [mV]
  pars['E_L'] = -75.      # leak reversal potential [mV]
  pars['tref'] = 2.       # refractory time (ms)

  # simulation parameters #
  pars['T'] = 400.  # Total duration of simulation [ms]
  pars['dt'] = .1   # Simulation time step [ms]

  # external parameters if any #
  for k in kwargs:
    pars[k] = kwargs[k]

  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized time points [ms]

  return pars

pars = default_pars()
print(pars)
```

    {'V_th': -55.0, 'V_reset': -75.0, 'tau_m': 10.0, 'g_L': 10.0, 'V_init': -75.0, 'E_L': -75.0, 'tref': 2.0, 'T': 400.0, 'dt': 0.1, 'range_t': array([0.000e+00, 1.000e-01, 2.000e-01, ..., 3.997e+02, 3.998e+02,
           3.999e+02])}

Complete the function below to simulate the LIF neuron when receiving external current inputs. You can use `v, sp = run_LIF(pars, Iinj)` to get the membrane potential (`v`) and spike train (`sp`) given the dictionary `pars` and input current `Iinj`.

```python
def run_LIF(pars, Iinj, stop=False):
  """
  Simulate the LIF dynamics with external input current

  Args:
    pars       : parameter dictionary
    Iinj       : input current [pA]. The injected current here can be a value
                 or an array
    stop       : boolean. If True, use a current pulse

  Returns:
    rec_v      : membrane potential
    rec_sp     : spike times
  """

  # Set parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, E_L = pars['V_init'], pars['E_L']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tref = pars['tref']

  # Initialize voltage
  v = np.zeros(Lt)
  v[0] = V_init

  # Set current time course
  Iinj = Iinj * np.ones(Lt)

  # If current pulse, set beginning and end to 0
  if stop:
    Iinj[:int(len(Iinj) / 2) - 1000] = 0
    Iinj[int(len(Iinj) / 2) + 1000:] = 0

  # Loop over time
  rec_spikes = []  # record spike times
  tr = 0.  # the count for refractory duration

  for it in range(Lt - 1):

    if tr > 0:  # check if in refractory period
      v[it] = V_reset  # set voltage to reset
      tr = tr - 1 # reduce running counter of refractory period

    elif v[it] >= V_th:  # if voltage over threshold
      rec_spikes.append(it)  # record spike event
      v[it] = V_reset  # reset voltage
      tr = tref / dt  # set refractory time

    ########################################################################
    ## TODO for students: compute the membrane potential v, spike train sp #
    # Fill out function and remove
    # raise NotImplementedError('Student Exercise: calculate the dv/dt and the update step!')
    ########################################################################

    # Calculate the increment of the membrane potential
    dv = (-(v[it] - E_L) + Iinj[it] / g_L) * (dt / tau_m)

    # Update the membrane potential
    v[it + 1] = v[it] + dv

  # Get spike times in ms
  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes

# Get parameters
pars = default_pars(T=500)

# Simulate LIF model
v, sp = run_LIF(pars, Iinj=100, stop=True)

# Visualize
plot_volt_trace(pars, v, sp)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_60a1e954.py)

*Example output:*

---
# Section 2: Response of an LIF model to different types of input currents

*Estimated timing to here from start of tutorial: 20 min*

In the following section, we will learn how to inject direct current and white noise to study the response of an LIF neuron.

##  Video 2: Response of the LIF neuron to different inputs

```python
# @title Video 2: Response of the LIF neuron to different inputs
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="av541417171", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="preNGdab7Kk", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

## Section 2.1: Direct current (DC)

*Estimated timing to here from start of tutorial: 30 min*

### Interactive Demo 2.1: Parameter exploration of DC input amplitude
Here's an interactive demo that shows how the LIF neuron behavior changes for DC input (constant current) with **different amplitudes**. We plot the membrane potential of an LIF neuron. You may notice that the neuron generates a spike. But this is just a cosmetic spike only for illustration purposes. In an LIF neuron, we only need to keep track of times when the neuron hit the threshold so the postsynaptic neurons can be informed of the spike.

How much DC is needed to reach the threshold (rheobase current)? How does the membrane time constant affect the frequency of the neuron?

 Make sure you execute this cell to enable the widget!

```python
# @title
# @markdown Make sure you execute this cell to enable the widget!
my_layout.width = '450px'
@widgets.interact(
    I_dc=widgets.FloatSlider(50., min=0., max=300., step=10.,
                               layout=my_layout),
    tau_m=widgets.FloatSlider(10., min=2., max=20., step=2.,
                                layout=my_layout)
)

def diff_DC(I_dc=200., tau_m=10.):
  pars = default_pars(T=100.)
  pars['tau_m'] = tau_m
  v, sp = run_LIF(pars, Iinj=I_dc)
  plot_volt_trace(pars, v, sp)
  plt.show()
```

    interactive(children=(FloatSlider(value=50.0, description='I_dc', layout=Layout(width='450px'), max=300.0, ste…

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_1058324c.py)

1. As we increase the current, we observe that at 210 pA we cross the threshold.

2. As we increase the membrane time constant (slower membrane), the firing rate
is decreased because the membrane needs more time to reach the threshold after
the reset.

## Section 2.2: Gaussian white noise (GWN) current

*Estimated timing to here from start of tutorial: 38 min*

Given the noisy nature of neuronal activity _in vivo_, neurons usually receive complex, time-varying inputs.

To mimic this, we will now investigate the neuronal response when the LIF neuron receives Gaussian white noise $\xi(t)$ with mean 0 ($\mu = 0$) and some standard deviation $\sigma$.

Note that the GWN has zero mean, that is, it describes only the fluctuations of the input received by a neuron. We can thus modify our definition of GWN to have a nonzero mean value $\mu$ that equals the DC input, since this is the average input into the cell. The cell below defines the modified gaussian white noise currents with nonzero mean $\mu$.

### Interactive Demo 2.2: LIF neuron Explorer for noisy input

The mean of the Gaussian white noise (GWN) is the amplitude of DC. Indeed, when $\sigma = 0$, GWN is just a DC.

So the question arises how does $\sigma$ of the GWN affect the spiking behavior of the neuron. For instance we may want to know
1.  how does the minimum input (i.e. $\mu$) needed to make a neuron spike change with increase in $\sigma$
2.  how does the spike regularity change with increase in $\sigma$

To get an intuition about these questions you can use the following interactive demo that shows how the LIF neuron behavior changes for noisy input with different amplitudes (the mean $\mu$) and fluctuation sizes ($\sigma$). We use a helper function to generate this noisy input current: `my_GWN(pars, mu, sig, myseed=False)`.  Note that fixing the value of the random seed (e.g., `myseed=2020`) will allow you to obtain the same result every time you run this. We then use our `run_LIF` function to simulate the LIF model.

 Execute to enable helper function `my_GWN`

```python
# @markdown Execute to enable helper function `my_GWN`

def my_GWN(pars, mu, sig, myseed=False):
  """
  Function that generates Gaussian white noise input

  Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    myseed     : random seed. int or boolean
                 the same seed will give the same
                 random number sequence

  Returns:
    I          : Gaussian white noise input
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # Set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # Generate GWN
  # we divide here by 1000 to convert units to sec.
  I_gwn = mu + sig * np.random.randn(Lt) / np.sqrt(dt / 1000.)

  return I_gwn

help(my_GWN)
```

    Help on function my_GWN in module __main__:

    my_GWN(pars, mu, sig, myseed=False)
        Function that generates Gaussian white noise input

        Args:
          pars       : parameter dictionary
          mu         : noise baseline (mean)
          sig        : noise amplitute (standard deviation)
          myseed     : random seed. int or boolean
                       the same seed will give the same
                       random number sequence

        Returns:
          I          : Gaussian white noise input

 Make sure you execute this cell to enable the widget!

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    mu_gwn=widgets.FloatSlider(200., min=100., max=300., step=5.,
                               layout=my_layout),
    sig_gwn=widgets.FloatSlider(2.5, min=0., max=5., step=.5,
                                layout=my_layout)
)

def diff_GWN_to_LIF(mu_gwn, sig_gwn):
  pars = default_pars(T=100.)
  I_GWN = my_GWN(pars, mu=mu_gwn, sig=sig_gwn)
  v, sp = run_LIF(pars, Iinj=I_GWN)
  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'][::3], I_GWN[::3], 'b')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{GWN}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()
  plt.show()
```

    interactive(children=(FloatSlider(value=200.0, description='mu_gwn', layout=Layout(width='450px'), max=300.0, …

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_2de5d8a9.py)

1) If we have bigger current fluctuations (increased sigma), the minimum input needed
to make a neuron spike is smaller as the fluctuations can help push the voltage above
threshold.

2) The standard deviation (or size of current fluctuations) dictates **the level of
irregularity of the spikes**; the higher the sigma the more irregular the observed
spikes.

### Think! 2.2: Analyzing GWN Effects on Spiking
- As we increase the input average ($\mu$) or the input fluctuation ($\sigma$), the spike count changes. How much can we increase the spike count, and what might be **the relationship between GWN mean/std or DC value and spike count**?

- We have seen above that when we inject DC, the neuron spikes in a regular manner (clock like), and this regularity is reduced when GWN is injected. The question is, how irregular can we make the neurons spiking by changing the parameters of the GWN?

We will see the answers to these questions in the next section but discuss first!

---
# Section 3: Firing rate and spike time irregularity

*Estimated timing to here from start of tutorial: 48 min*

When we plot the output firing rate as a function of GWN mean or DC value, it is called the input-output transfer function of the neuron (so simply F-I curve).

Spike regularity can be quantified as the **coefficient of variation (CV) of the inter-spike-interval (ISI)**:
\begin{align}
\text{CV}_{\text{ISI}} = \frac{std(\text{ISI})}{mean(\text{ISI})}
\end{align}

A **Poisson** train is an example of high irregularity, in which $\textbf{CV}_{\textbf{ISI}} \textbf{= 1}$. And for a clocklike (regular) process we have $\textbf{CV}_{\textbf{ISI}} \textbf{= 0}$ because of **std(ISI)=0**.

## Interactive Demo 3A: F-I Explorer for different `sig_gwn`

How does **the F-I curve of the LIF neuron change** as we increase the $\sigma$ of the GWN? We can already expect that the F-I curve will be stochastic and the results will vary from one trial to another. **But will there be any other change compared to the F-I curve measured using DC?**

Here's an interactive demo that shows how the F-I curve of a LIF neuron changes for different levels of fluctuation $\sigma$.

 Make sure you execute this cell to enable the widget!

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    sig_gwn=widgets.FloatSlider(3.0, min=0., max=6., step=0.5,
                                layout=my_layout)
)

def diff_std_affect_fI(sig_gwn):
  pars = default_pars(T=1000.)
  I_mean = np.arange(100., 400., 10.)
  spk_count = np.zeros(len(I_mean))
  spk_count_dc = np.zeros(len(I_mean))

  for idx in range(len(I_mean)):
      I_GWN = my_GWN(pars, mu=I_mean[idx], sig=sig_gwn, myseed=2020)
      v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
      v_dc, rec_sp_dc = run_LIF(pars, Iinj=I_mean[idx])
      spk_count[idx] = len(rec_spikes)
      spk_count_dc[idx] = len(rec_sp_dc)

  # Plot the F-I curve i.e. Output firing rate as a function of input mean.
  plt.figure()
  plt.plot(I_mean, spk_count, 'k',
           label=r'$\sigma_{\mathrm{GWN}}=%.2f$' % sig_gwn)
  plt.plot(I_mean, spk_count_dc, 'k--', alpha=0.5, lw=4, dashes=(2, 2),
           label='DC input')
  plt.ylabel('Spike count')
  plt.xlabel('Average injected current (pA)')
  plt.legend(loc='best')
  plt.show()
```

    interactive(children=(FloatSlider(value=3.0, description='sig_gwn', layout=Layout(width='450px'), max=6.0, ste…

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_eba2370f.py)

If we use a DC input, the F-I curve is deterministic, and we can
found its shape by solving the membrane equation of the neuron. If we have GWN,
as we increase the sigma, **the F-I curve has a more linear shape**, and the neuron
reaches its threshold using less average injected current.

## Coding Exercise 3: Compute $CV_{ISI}$ values

As shown above, the F-I curve becomes **smoother** while increasing the amplitude of the fluctuation ($\sigma$). In addition, the fluctuation can also change the irregularity of the spikes. Let's investigate the effect of $\mu=250$ with $\sigma=0.5$ vs $\sigma=3$.

Fill in the code below to compute ISI, then plot the histogram of the ISI and compute the $CV_{ISI}$. Note that, you can use `np.diff` to calculate ISI.

```python
def isi_cv_LIF(spike_times):
  """
  Calculates the inter-spike intervals (isi) and
  the coefficient of variation (cv) for a given spike_train

  Args:
    spike_times : (n, ) vector with the spike times (ndarray)

  Returns:
    isi         : (n-1,) vector with the inter-spike intervals (ms)
    cv          : coefficient of variation of isi (float)

  """
  ########################################################################
  ## TODO for students: compute the membrane potential v, spike train sp #
  # Fill out function and remove
  # raise NotImplementedError('Student Exercise: calculate the isi and the cv!')
  ########################################################################
  if len(spike_times) >= 2:
    # Compute isi
    isi = np.diff(spike_times)
    # Compute cv
    cv = isi.std() / isi.mean()
  else:
    isi = np.nan
    cv = np.nan

  return isi, cv

# Set parameters
pars = default_pars(T=1000.)
mu_gwn = 250
sig_gwn1 = 0.5
sig_gwn2 = 3.0

# Run LIF model for sigma = 0.5
I_GWN1 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn1, myseed=2020)
_, sp1 = run_LIF(pars, Iinj=I_GWN1)

# Run LIF model for sigma = 3
I_GWN2 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn2, myseed=2020)
_, sp2 = run_LIF(pars, Iinj=I_GWN2)

# Compute ISIs/CV
isi1, cv1 = isi_cv_LIF(sp1)
isi2, cv2 = isi_cv_LIF(sp2)

# Visualize
my_hists(isi1, isi2, cv1, cv2, sig_gwn1, sig_gwn2)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_27d69c89.py)

*Example output:*

## Interactive Demo 3B: Spike irregularity explorer for different `sig_gwn`

In the above illustration, we see that **the CV of inter-spike-interval (ISI) distribution depends on $\sigma$ of GWN**. What about the mean of GWN, should that also affect the CV$_{\rm ISI}$? If yes, how? Does the efficacy of $\sigma$ in increasing the CV$_{\rm ISI}$ depend on $\mu$?

In the following interactive demo, you will examine how different levels of fluctuation $\sigma$ affect the CVs for different average injected currents ($\mu$).

1. Does the standard deviation of the injected current affect the F-I curve in any qualitative manner?
2. Why does increasing the mean of GWN reduce the $CV_{ISI}$?
3.  If you plot spike count (or rate) vs. $CV_{ISI}$, should there be a relationship between the two? Try out yourself.

 Make sure you execute this cell to enable the widget!

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    sig_gwn=widgets.FloatSlider(0.0, min=0., max=10.,
                                step=0.5, layout=my_layout)
)

def diff_std_affect_fI(sig_gwn):
  pars = default_pars(T=1000.)
  I_mean = np.arange(100., 400., 20)
  spk_count = np.zeros(len(I_mean))
  cv_isi = np.empty(len(I_mean))

  for idx in range(len(I_mean)):
    I_GWN = my_GWN(pars, mu=I_mean[idx], sig=sig_gwn)
    v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
    spk_count[idx] = len(rec_spikes)
    if len(rec_spikes) > 3:
      isi = np.diff(rec_spikes)
      cv_isi[idx] = np.std(isi) / np.mean(isi)

  # Plot the F-I curve i.e. Output firing rate as a function of input mean.
  plt.figure()
  plt.plot(I_mean[spk_count > 5], cv_isi[spk_count > 5], 'bo', alpha=0.5)
  plt.xlabel('Average injected current (pA)')
  plt.ylabel(r'Spike irregularity ($\mathrm{CV}_\mathrm{ISI}$)')
  plt.ylim(-0.1, 1.5)
  plt.grid(True)
  plt.show()
```

    interactive(children=(FloatSlider(value=0.0, description='sig_gwn', layout=Layout(width='450px'), max=10.0, st…

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_c6f1c4a2.py)

1. Yes, it does. With DC input the F-I curve has a strong non-linearity but when
a neuron is driven with GWN, as we increase the $\sigma$ the non-linearity is
smoothened out. Essentially, **in this case noise is acting to suppress the
non-linearities and render a neuron as a linear system**.

2. (here is a short answer) When we increase the mean of the GWN, at some point
effective input mean is above the spike threshold and then the neuron operates
in the so called mean-driven regime -- as the input is so high all the neuron
is does is charge up to the spike threshold and reset. This essentially gives
almost regular spiking.

3. In an LIF, high firing rates are achieved for high GWN mean. Higher the mean,
higher the firing rate and lower the CV_ISI. So you will expect that as firing rate
increases, spike irregularity decreases. This is because of the spike threshold.
FOr a Poisson process there is no relationship between spike rate and spike
irregularity.

---
# Section 4: Generation of Poisson type spike trains

*In the next tutorials, we will often use Poisson type spike train to explore properties of neurons and synapses. Therefore, it is good to know how to generate Poisson type spike trains.*

Mathematically, a spike train is a Point Process. One of the simplest models of a sequence of presynaptic pulse inputs is the Poisson process. **We know that given temporal integration and refractoriness, neurons cannot behave as a Poisson Process, and Gamma Process gives a better approximation** (*find out what might be the difference in the two processes*).
Here, however, we will assume that the incoming spikes are following Poisson statistics. A question arises about how to simulate a Poisson process. The generation of the Poisson process can be realized by at least two following ways:

- By definition, for a Poisson process with rate $\lambda$, the probability of finding one event in the time window with a sufficiently small length $\Delta t$ is $P(N = 1) = \lambda \Delta t$. Therefore, in each time window, we generate a uniformly distributed random variable $r \in [0,1]$ and generate a Poisson event when $r <\lambda \Delta t$. This method allows us to generate Poisson distributed spikes in an online manner.

- The interval $t_{k+1}-t_{k}$ between two Poisson events with rate $\lambda$ follows the exponential distribution, i.e., $P(t_{k+1}-t_{k}<t) = 1 - e^{\lambda t}$. Therefore, we only need to generate a set of exponentially distributed variables $\{s_k\}$ to obtain the timing of Poisson events $t_{k+1}=t_{k}+s_{k}$. In this method, we need to generate all future spikes at once.

Below, we use the first method in a function `Poisson_generator`, which takes arguments `(pars, rate, n, myseed)`.

 Execute this cell to get a Poisson_generator function

```python
# @title
# @markdown Execute this cell to get a Poisson_generator function

def Poisson_generator(pars, rate, n, myseed=False):
  """
  Generates poisson trains

  Args:
    pars            : parameter dictionary
    rate            : noise amplitute [Hz]
    n               : number of Poisson trains
    myseed          : random seed. int or boolean

  Returns:
    pre_spike_train : spike train matrix, ith row represents whether
                      there is a spike in ith spike train over time
                      (1 if spike, 0 otherwise)
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate uniformly distributed random variables
  u_rand = np.random.rand(n, Lt)

  # generate Poisson train
  poisson_train = 1. * (u_rand < rate * (dt / 1000.))

  return poisson_train
```

```python
# we can use Poisson_generator to mimic presynaptic spike trains
pars = default_pars()
pre_spike_train = Poisson_generator(pars, rate=10, n=100, myseed=2020)
my_raster_Poisson(pars['range_t'], pre_spike_train, 100)
```

How do we make sure that the above spike trains are following Poisson statistics?
A Poisson process must have the following properties:
- The ratio of the mean and variance of spike count is 1
- **Inter-spike-intervals are exponentially distributed**
- Spike times are irregular i.e. $CV_{\rm ISI} = 1$
- Adjacent spike intervals are independent of each other.

---
# Summary

*Estimated timing of tutorial: 1 hour, 10 min*

Congratulations! You've just built a leaky integrate-and-fire (LIF) neuron model from scratch, and studied its dynamics in response to various types of inputs, having:

- simulated the LIF neuron model

- driven the LIF neuron with external inputs, such as direct current and Gaussian white noise

- studied how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity),

with a special focus on low rate and irregular firing regime to mimc real cortical neurons. The next tutorial will look at how spiking statistics may be influenced by a neuron's input statistics.

If you have extra time, look at the bonus sections below to explore a different type of noise input and learn about extensions to integrate-and-fire models.

---
# Bonus

---
## Bonus Section 1: Orenstein-Uhlenbeck Process

When a neuron receives spiking input, the synaptic current is Shot Noise -- which is a kind of colored noise and the spectrum of the noise determined by the synaptic kernel time constant. That is, a neuron is driven by **colored noise** and not GWN.

We can model colored noise using the Ohrenstein-Uhlenbeck process - filtered white noise.

We next study if the input current is temporally correlated and is modeled as an Ornstein-Uhlenbeck process $\eta(t)$, i.e., **low-pass filtered GWN with a time constant $\tau_{\eta}$**:

$$\tau_\eta \frac{d}{dt}\eta(t) = \mu-\eta(t) + \sigma_\eta\sqrt{2\tau_\eta}\xi(t).$$

**Hint:** An OU process as defined above has

$$E[\eta(t)]=\mu$$

and autocovariance

$$E[\eta(t)\eta(t+\tau)]=\sigma_\eta^2e^{-|t-\tau|/\tau_\eta},$$

which can be used to check your code.

 Execute this cell to get helper function `my_OU`

```python
# @markdown Execute this cell to get helper function `my_OU`

def my_OU(pars, mu, sig, myseed=False):
  """
  Function that produces Ornstein-Uhlenbeck input

  Args:
    pars       : parameter dictionary
    sig        : noise amplitute
    myseed     : random seed. int or boolean

  Returns:
    I_ou       : Ornstein-Uhlenbeck input current
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tau_ou = pars['tau_ou']  # [ms]

  # set random seed
  if myseed:
    np.random.seed(seed=myseed)
  else:
    np.random.seed()

  # Initialize
  noise = np.random.randn(Lt)
  I_ou = np.zeros(Lt)
  I_ou[0] = noise[0] * sig

  # generate OU
  for it in range(Lt-1):
    I_ou[it+1] = I_ou[it] + (dt / tau_ou) * (mu - I_ou[it]) + np.sqrt(2 * dt / tau_ou) * sig * noise[it + 1]

  return I_ou

help(my_OU)
```

    Help on function my_OU in module __main__:

    my_OU(pars, mu, sig, myseed=False)
        Function that produces Ornstein-Uhlenbeck input

        Args:
          pars       : parameter dictionary
          sig        : noise amplitute
          myseed     : random seed. int or boolean

        Returns:
          I_ou       : Ornstein-Uhlenbeck input current

### Bonus Interactive Demo 1: LIF Explorer with OU input
In the following, we will check how a neuron responds to a noisy current that follows the statistics of an OU process.

-  How does the OU type input change neuron responsiveness?
-  What do you think will happen to the spike pattern and rate if you increased or decreased the time constant of the OU process?

 Remember to enable the widget by running the cell!

```python
# @title

# @markdown Remember to enable the widget by running the cell!

my_layout.width = '450px'
@widgets.interact(
  tau_ou=widgets.FloatSlider(10.0, min=5., max=20.,
                             step=2.5, layout=my_layout),
  sig_ou=widgets.FloatSlider(10.0, min=5., max=40.,
                             step=2.5, layout=my_layout),
  mu_ou=widgets.FloatSlider(190.0, min=180., max=220.,
                            step=2.5, layout=my_layout)
)

def LIF_with_OU(tau_ou=10., sig_ou=40., mu_ou=200.):
  pars = default_pars(T=1000.)
  pars['tau_ou'] = tau_ou  # [ms]

  I_ou = my_OU(pars, mu_ou, sig_ou)

  v, sp = run_LIF(pars, Iinj=I_ou)

  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'], I_ou, 'b', lw=1.0)
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{\mathrm{OU}}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()
  plt.show()
```

    interactive(children=(FloatSlider(value=10.0, description='tau_ou', layout=Layout(width='450px'), max=20.0, mi…

## Think!
- How does the OU type input change neuron responsiveness?
- What do you think will happen to the spike pattern and rate if you increased or decreased the time constant of the OU process?

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_BiologicalNeuronModels/solutions/W2D3_Tutorial1_Solution_cf5b6a80.py)

In a limiting case, when the time constant of the OU process is very long and the
input current is almost flat, we expect the firing rate to decrease and neuron
will spike **more regularly**. So as the OU process time constant increases, we expect
firing rate and $CV_{ISI}$ to decrease, if all other parameters are kept constant. We
can also relate the OU process time constant to the membrane time constant as the
neuron membrane does the same operation. This way we can link to the very first
interactive demo.

---
## Bonus Section 2: Generalized Integrate-and-Fire models

LIF model is not the only abstraction of real neurons. If you want to learn about more realistic types of neuronal models, watch the Bonus Video!

###  Video 3 (Bonus): Extensions to Integrate-and-Fire models

```python
# @title Video 3 (Bonus): Extensions to Integrate-and-Fire models
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="G0b6wLhuQxE", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})


###### 20 November 2018, by Jeroen van Lidth de Jeude - [NETWORKS](http://networks.imtlucca.it/) -  [IMT School for Advanced Studies Lucca](https://www.imtlucca.it/jeroen.vanlidth)
# Detecting Core-Periphery Structures by Surprise
This notebook provides the code to detect bimodular structures as core-periphery and bipartite ones, with the **bimodular surprise** as proposed in the paper : *["Detecting Core-Periphery Structures by Surprise" - Jeroen van Lidth de Jeude, Guido Caldarelli and Tiziano Squartini (10/10/2018)](https://arxiv.org/abs/1810.04717)*

## Bimodular Network Structures
This method can both detect core-periphery structures and bipartite ones.

### Bimodular Surprise
Bimodular surprise is detects significant structures by comparing the connectivity in different areas of the graph. It uses a hypergeometric distribution to evaluate bimodular graph partitions.
\begin{equation}
S_\parallel\equiv\sum_{i\geq l_c^*}\sum_{j\geq l_{cp}^*}\frac{\binom{V_c}{i} \binom{V_{cp}}{j} \binom{V-(V_c+V_{cp})}{L-(i+j)}}{\binom{V}{L}}
\end{equation}
where $V=N(N-1)$ is the volume of the network, coinciding with the total number of nodes pairs, $L$ is the total number of links and $l^*$ is the observed number of intracluster links (i.e. *within* the individuated communities). The index $c$ labels the core part and the index $cp$ labels the core-periphery part; whenever considering bipartite networks, the core-periphery portion will be assumed to indicate the inter-layer portion.
These three types of links are visualised below for the core-periphery case.

# Code
The code consists of the calculation of the distribution and the search heuristic which is to find the network partition which minimises this value.

### Python and Packages
This code runs on Python 3.5 or any higher version.
The following Python packages are required for execution. Please make sure these are installed and updated:
on your operating system's terminal:
    - `pip install networkx`
    - or upgrading: `pip install --upgrade networkx`

```python
from scipy.special import comb
import numpy as np
import networkx as nx
from numba import jit  # numba is a package to speed up certain simple bits of code

# For diplaying figures within the notebook we require: MatPlotLib
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
```

### PACO optimizing algorithm
The following chunks contain the algorithm for the *bimodular surprise* evaluation and optimization. If you think this code seems awfully long to just evaluate and minimize a distribution function, thats not a weird observation. However, for computational speed and the precision of binomial coefficients we introduced some tricks. The simple code of the bimodular surprise can actually be found as the last function below here: `evaluate_real_surprise`.

```python
@jit
def logMultiHyperProbability(p, p_c, p_x, l, l_c, l_x):
    '''Evaluation of logarithmic hypergeometric distribution'''
    logH = logC(p_c, l_c) + logC(p_x, l_x) + logC(p - p_c - p_x, l-l_c-l_x) - logC(p, l)
    return logH

@jit
def logC2(n, k):
    if (k == n):
        x = 0
    else:
        t = n - k
        if(t < k):
            t = k
        x = sumRange(t+1, n) - sumFactorial(n - t);
    return x

@jit
def sumRange(xmin, xmax):
    csum = 0
    for i in range(xmin,xmax+1):
        csum += np.log10(i)
    return csum

@jit
def sumFactorial(n):
    csum = 0
    if n > 1:
        for i in range(2,n+1):
            csum += np.log10(i)
    return csum

@jit
def sumLogProbabilities2(nextP, P, logP):
    logP += np.log1p(nextP / P)
    return logP

@jit
def sumLogProbabilities(nextLogP, logP):
    if nextLogP == 0:
        stop = True
    else:
        stop = False
        if nextLogP > logP:
            common = nextLogP
            diffExponent = logP - common
        else:
            common = logP
            diffExponent = nextLogP - common

        logP = common + ((np.log10(1 + 10**diffExponent)) / np.log10(10))

        if (nextLogP - logP) > -4:
            stop = True

    return logP, stop

@jit
def logC(n, k):
    if k == n:
        return 0
    elif (n > 1000) & (k > 1000): # Stirling's binomial coeff approximation
        return logStirFac(n) - logStirFac(k) - logStirFac(n-k)
    else:
        t = n - k
        if t < k:
            t = k
        logC = sumRange(t + 1, n) - sumFactorial(n - t)
        return logC

@jit
def logStirFac(n):
    if n <= 1:
        return 1.0
    else:
        return -n + n*np.log10(n) + np.log10(n*(1 + 4.0*n*(1.0 + 2.0*n)))/6.0 + np.log10(np.pi)/2.0

@jit
def logBin(n, k):
    return logStirFac(n) - logStirFac(k) - logStirFac(n - k)

@jit
def surprise_bipartite_logsum(p, p_c, p_x, l, l_c, l_x):
    stop = False
    first_loop_break = False

    min_l_p = min(l, p_c+p_x)

    logP = logMultiHyperProbability(p, p_c, p_x, l, l_c, l_x)
    for l_c_loop in range(l_c, min_l_p+1):
        for l_x_loop in range(l_x, min_l_p+1 - l_c_loop):
            if (l_c_loop == l_c) & (l_x_loop == l_x):
                continue
            nextLogP = logMultiHyperProbability(p, p_c, p_x, l, l_c_loop, l_x_loop)
            [logP, stop] = sumLogProbabilities(nextLogP, logP)

            if stop:
                first_loop_break = True
                break
        if first_loop_break:
            break

    return -logP

@jit
def calculate_surprise_logsum(adjacency_matrix, core_nodes, periphery_nodes, is_directed=True):

    core_nodes = set(core_nodes)
    periphery_nodes = set(periphery_nodes)

    if (len(core_nodes) <= 1) | (len(periphery_nodes) <= 1): #not enough core/periphery nodes: bin. coef. = 0
        return 1
    assert np.sum(adjacency_matrix.diagonal()) == 0, 'Adjacency diagonal not zero: illegal self loops'

    if is_directed == True:
        p_c = len(core_nodes) * (len(core_nodes)- 1)
        p_x = len(core_nodes) * len(periphery_nodes) * 2
        #p_p = len(periphery_nodes) * (len(periphery_nodes) - 1)
        l_c = np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(core_nodes))] > 0)
        l_x = np.sum(adjacency_matrix[np.ix_(list(periphery_nodes), list(core_nodes))] > 0) + (
                        np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(periphery_nodes))] > 0) )
        l = np.sum(adjacency_matrix > 0)
        n = len(core_nodes) + len(periphery_nodes)
        p = n*(n-1)
    else: # is_directed == False : UNDIRECTED
        adjacency_matrix = np.triu(adjacency_matrix)
        p_c = int(len(core_nodes) * (len(core_nodes)- 1) /2)
        p_x = len(core_nodes) * len(periphery_nodes)
        #p_p = len(periphery_nodes) * (len(periphery_nodes) - 1)
        l_c = np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(core_nodes))] > 0)
        l_x = np.sum(adjacency_matrix[np.ix_(list(periphery_nodes), list(core_nodes))] > 0) + (
                        np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(periphery_nodes))] > 0) )
        l = np.sum(adjacency_matrix > 0)
        n = len(core_nodes) + len(periphery_nodes)
        p = int(n*(n-1) / 2)

    if (p_c + p_x) < (l_c+l_x):
        return 0
    surprise = surprise_bipartite_logsum(p, p_c, p_x, l, l_c, l_x)
    return surprise

def remove_isolated_nodes(adj):
    zero_in_degree = np.where(np.sum(adj,0) == 0)[0]
    zero_out_degree = np.where(np.sum(adj,1) == 0)[0]
    zero_degree_nodes = np.array(list(set(zero_in_degree) & set(zero_out_degree)))
    degree_nodes = np.array(list(set(np.arange(len(adj))) - set(zero_degree_nodes)))

    if len(zero_degree_nodes)>0:
        adj_new = adj[np.ix_(list(degree_nodes), list(degree_nodes))]
    else:
        adj_new = adj

    return adj_new, zero_degree_nodes

def insert_isolated_nodes(zero_degree_nodes, cluster_assignment):
    for node in zero_degree_nodes:
        cluster_assignment = np.insert(cluster_assignment, node, 1)
    return cluster_assignment

def calculate_surprise(adjacency_matrix, cluster_assignment, is_directed=True):
    core_nodes = np.where(cluster_assignment == 0)[0]
    periphery_nodes = np.where(cluster_assignment == 1)[0]

    if (len(core_nodes) <= 1) | (len(periphery_nodes) <= 1): #not enough core/periphery nodes: bin. coef. = 0
        return 1
    assert np.sum(adjacency_matrix.diagonal()) == 0, 'Adjacency diagonal not zero: illegal self loops'

    if is_directed == True:
        return calculate_surprise_logsum(adjacency_matrix, core_nodes, periphery_nodes, is_directed=True)
    else: # is_directed == False : UNDIRECTED
        return calculate_surprise_logsum(adjacency_matrix, core_nodes, periphery_nodes, is_directed=False)

    return 1

def jaccard_sorted_edges(adjacency_matrix):
    G = nx.from_numpy_matrix(adjacency_matrix)
    jacc = nx.jaccard_coefficient(G, ebunch=G.edges())
    jacc_array = []
    for u, v, p in jacc:
        jacc_array += [[u,v,p]]
    jacc_array = np.array(jacc_array)
    jacc_array = jacc_array[jacc_array[:,2].argsort()][::-1]
    sorted_edges = jacc_array[:,0:2]
    sorted_edges = sorted_edges.astype(np.int)
    return sorted_edges

@jit
def calculate_z_motifs(adjacency):
    adj = adjacency.copy()
    G = nx.from_numpy_array(adj)
    # make adjacency symetric/undirected
    adj2 = adj  + np.transpose(adj)
    adj = adj2.astype(bool).astype(int)

    z_motifs = np.zeros(len(G.edges()))
    nodes = G.nodes()
    for index,[i,j] in enumerate(G.edges()):
        for k in nodes:
            if k not in [i,j]:
                for l in nodes:
                    if (l not in [i,j,k]) & (l > k):
                        #z_motifs[index] += adj[i,k]*adj[k,l]*adj[j,l]
                        z_motifs[index] += adj[i,k]*adj[j,l]
    return z_motifs, np.array(G.edges())

def sort_edges_z_motif(adj):
    z_motifs, edges_array = calculate_z_motifs(adj)
    sort_indices = np.argsort(z_motifs)
    sorted_edges = edges_array[sort_indices[::-1]]
    return sorted_edges

def flip_random_group(a, flip_n_nodes=2):
    if (sum(a==0) <= flip_n_nodes+1):
        a[np.random.choice(np.where(a==1)[0], replace=False, size=flip_n_nodes)] = 0
        return a
    elif (sum(a==1) <= flip_n_nodes+1):
        a[np.random.choice(np.where(a==0)[0], replace=False, size=flip_n_nodes)] = 1
        return a
    elif np.random.rand() < 0.5:
        a[np.random.choice(np.where(a==0)[0], replace=False, size=flip_n_nodes)] = 1
    else:
        a[np.random.choice(np.where(a==1)[0], replace=False, size=flip_n_nodes)] = 0
    return a

def correct_partition_numbering(adjacency_matrix, cluster_assignment):
    core_nodes = np.where(cluster_assignment == 0)[0]
    periphery_nodes = np.where(cluster_assignment == 1)[0]
    l_core = np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(core_nodes))] > 0)
    l_periphery = np.sum(adjacency_matrix[np.ix_(list(periphery_nodes), list(periphery_nodes))] > 0)
    core_density = l_core / (len(core_nodes)*(len(core_nodes)-1))
    periphery_density = l_periphery / (len(periphery_nodes)*(len(periphery_nodes)-1))

    if periphery_density > core_density:
        cluster_assignment_new = 1- cluster_assignment
        return cluster_assignment_new
    return cluster_assignment

def minimize_surprise_paco(adjacency_matrix, num_sim, is_directed=True, sort_method='zmotif'):

    surprise = 0
    cluster_assignment = np.ones(len(adjacency_matrix), dtype=int)
    degrees = np.sum(adjacency_matrix,0) + np.sum(adjacency_matrix,1)
    cluster_assignment[degrees.argsort()[-3:]] = 0

    degrees = np.vstack((np.arange(len(degrees)),degrees))
    degrees = degrees[:,degrees[1,:].argsort()[::-1]]
    nodes_degree_sorted = degrees[0,:]

    if sort_method == 'zmotif':
        E_sorted = sort_edges_z_motif(adjacency_matrix)
    elif sort_method == 'jaccard':
        E_sorted = jaccard_sorted_edges(adjacency_matrix)
    else:
        assert False, '`sort_method` must be "zmotif" or "jaccard"'

    for sim in np.arange(num_sim):
        for [u,v] in E_sorted:
            cluster_assignment_temp1 = cluster_assignment.copy()
            cluster_assignment_temp2 = cluster_assignment.copy()
            if cluster_assignment[u] != cluster_assignment[v]:
                cluster_assignment_temp1[v] = cluster_assignment[u]
                cluster_assignment_temp2[u] = cluster_assignment[v]

                surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp1, is_directed)
                if (surprise_temp >= surprise):
                    cluster_assignment = cluster_assignment_temp1.copy()
                    surprise = surprise_temp

                surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp2, is_directed)
                if (surprise_temp >= surprise):
                    cluster_assignment = cluster_assignment_temp2.copy()
                    surprise = surprise_temp

            else: # cluster_assignment is the same
                cluster_assignment_temp1[v] = 1 - cluster_assignment[v] #invert assignment
                cluster_assignment_temp2[u] = 1 - cluster_assignment[u]

                surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp1, is_directed)
                if (surprise_temp >= surprise):
                    cluster_assignment = cluster_assignment_temp1.copy()
                    surprise = surprise_temp

                surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp2, is_directed)
                if (surprise_temp >= surprise):
                    cluster_assignment = cluster_assignment_temp2.copy()
                    surprise = surprise_temp
            #print(surprise)
            if surprise == 0:
                print('zero surprise reached')
                break

        # Group flipping
        for i in np.arange(100):
            cluster_assignment_temp = flip_random_group(cluster_assignment.copy(), flip_n_nodes=1)
            surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp, is_directed)
            if (surprise_temp >= surprise):
                cluster_assignment = cluster_assignment_temp.copy()
                surprise = surprise_temp

            cluster_assignment_temp = flip_random_group(cluster_assignment.copy(), flip_n_nodes=2)
            surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp, is_directed)
            if (surprise_temp >= surprise):
                cluster_assignment = cluster_assignment_temp.copy()
                surprise = surprise_temp

            cluster_assignment_temp = flip_random_group(cluster_assignment.copy(), flip_n_nodes=3)
            surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp, is_directed)
            if (surprise_temp >= surprise):
                cluster_assignment = cluster_assignment_temp.copy()
                surprise = surprise_temp

            cluster_assignment_temp = flip_random_group(cluster_assignment.copy(), flip_n_nodes=4)
            surprise_temp = calculate_surprise(adjacency_matrix, cluster_assignment_temp, is_directed)
            if (surprise_temp >= surprise):
                cluster_assignment = cluster_assignment_temp.copy()
                surprise = surprise_temp

    cluster_assignment = correct_partition_numbering(adjacency_matrix, cluster_assignment)
    return cluster_assignment, surprise

def run_surprise(G, num_sim=10, is_directed=True, sort_method='zmotif'):
    """Running the PACO optimization algoritm for cp detection.
    This fucntion takes as argument either a numpy adjacency matrix or a networkx graph.

        Args:
            G: [numpy.array or networkx.Graph] the adjacency matrix
            num_sim: [int] number of heuristic runs minimal is 1
            is_directed: [boolean] indicator if graph is directed or not
            sort_methos: ['zmotif' or 'jaccard'] the method used to sort the edges

        Returns:
            numpy.array vector is node assignment and numpy.float significance value

    """
    if (type(G) == nx.classes.graph.Graph) | (type(G) == nx.classes.digraph.DiGraph):
        adj = nx.to_numpy_array(G)
        np.fill_diagonal(adj, 0)
    else: #G is an adjacency
        adj = G

    [cluster_assignment, surprise] = minimize_surprise_paco(adj, num_sim, is_directed, sort_method)

    # evaluate the actual surprise value, not the logsum version
    #real_surprise = evaluate_real_surprise(adj, cluster_assignment, is_directed=is_directed)
    real_surprise = 10**(-surprise)

    return cluster_assignment, real_surprise

@jit
def surprise_bimodular(p, p_c, p_x, l, l_c, l_x):
    min_l_p = min(l, p_c+p_x)

    surprise = 0
    for l_c_loop in range(l_c, min_l_p+1):
        for l_x_loop in range(l_x, min_l_p+1 - l_c_loop):
            surprise += comb(p_c, l_c_loop, exact=1) * comb(p_x, l_x_loop, exact=1) * comb(p - p_c - p_x, l - l_c_loop - l_x_loop, exact=1) / comb(p, l, exact=1)
    return surprise

def evaluate_real_surprise(adjacency_matrix, partition, is_directed=True):
    core_nodes = np.where(partition == 0)[0]
    periphery_nodes = np.where(partition == 1)[0]
    core_nodes = set(core_nodes)
    periphery_nodes = set(periphery_nodes)

    if (len(core_nodes) <= 1) | (len(periphery_nodes) <= 1): #not enough core/periphery nodes: bin. coef. = 0
        return 1
    assert np.sum(adjacency_matrix.diagonal()) == 0, 'Adjacency diagonal not zero: illegal self loops'

    if is_directed == True:
        p_c = len(core_nodes) * (len(core_nodes)- 1)
        p_x = len(core_nodes) * len(periphery_nodes) * 2
        #p_p = len(periphery_nodes) * (len(periphery_nodes) - 1)
        l_c = np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(core_nodes))] > 0)
        l_x = np.sum(adjacency_matrix[np.ix_(list(periphery_nodes), list(core_nodes))] > 0) + (
                        np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(periphery_nodes))] > 0) )
        l = np.sum(adjacency_matrix > 0)
        n = len(core_nodes) + len(periphery_nodes)
        p = n*(n-1)
    else: # is_directed == False : UNDIRECTED
        adjacency_matrix = np.triu(adjacency_matrix)
        p_c = int(len(core_nodes) * (len(core_nodes)- 1) /2)
        p_x = len(core_nodes) * len(periphery_nodes)
        #p_p = len(periphery_nodes) * (len(periphery_nodes) - 1)
        l_c = np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(core_nodes))] > 0)
        l_x = np.sum(adjacency_matrix[np.ix_(list(periphery_nodes), list(core_nodes))] > 0) + (
                        np.sum(adjacency_matrix[np.ix_(list(core_nodes), list(periphery_nodes))] > 0) )
        l = np.sum(adjacency_matrix > 0)
        n = len(core_nodes) + len(periphery_nodes)
        p = int(n*(n-1) / 2)

    return surprise_bimodular(p, p_c, p_x, l, l_c, l_x)
```

# Running the Algorithm
Running the algorithm is as simple as calling the function `run_surprise` with as argument the *adjacency matrix* of the graph or a *NetworkX graph*. The adjacency matrix should be supplied in the numpy.array format. See also the examples.

The two output variables of the algorithm are: a numpy.array vector detailing the *node assignment*, and a float value of the *significance* of the partition.

## Example - Karate Club
In this example we analyze the core-periphery structure of the infamous Zachary Karate club graph: an undirected, unweighted social interactions graph. For more details on this classic network community detection dataset see: [Zachary’s Zachary karate club](https://petterhol.me/2018/01/28/zacharys-zachary-karate-club/)

```python
G_karate = nx.karate_club_graph() # Loading the graph from the NetworkX package

# Run the bimodular surprise algorithm
partitioning, significance = run_surprise(G=G_karate, is_directed=False)
```

```python
partitioning
```

    array([0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
           1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0])

The variable `partitioning` now shows the node assingment found by bimodular surprise. Nodes with value zero are in the core, nodes with value 1 in the periphery. In this case the first three nodes and the last two nodes are in the core. The order of the nodes in the partitioning is the same as the ordering of the nodes in the adjacency matrix/the ordering of the NetworkX graph.nodes().

```python
significance
```

    1.6313419062282474e-22

The variable `significance` stores the significance value (p-value) of the found partition. In general a partition with a significance value < 0.01 can be considered significant.

```python
nx.draw(G=G_karate, node_color=partitioning, with_labels=True)
_ = plt.title('Core-periphery structure of the Zachary Karate Club \n as revealed by bimodular surprise')
```

## Example - Loading an external file
In this example we load an external file. Graph files (gml, gexf, etc.) are easiest loaded using NetworkX, adjacency matrices (csv, txt, etc.) are usually imported using Numpy. Here we load a artificial core-periphery like structure from a csv file.

```python
# Loading a csv adjacency matrix from the working directory (same as where notebook is saved)
adj = np.loadtxt('./Data/artificial_core-periphery_matrix.csv', delimiter=',')
adj = adj > 0 # Force binary
adj = adj.astype(int) # Force integers

plt.imshow(adj, cmap='Blues')
plt.colorbar()
_ = plt.title('adjacency matrix of \n artificial core-periphery structure')
```

```python
# The adjacency matrix can be directly given as an input to the function
partitioning, significance = run_surprise(G=adj, is_directed=True)

print(significance, partitioning)
```

    9.81263527586384e-30 [0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1]

```python
plt.figure(figsize=(10,10))
G_external_adj = nx.from_numpy_array(adj)  # Create NetworkX graph for visualisation
nx.draw(G=G_external_adj, node_color=partitioning, with_labels=False)
_ = plt.title('Core-periphery structure of artificial network \n as revealed by bimodular surprise')
```

The bimodular surprise identifies the planted core correctly, see the visualised adjacency matrix of the data.

### Useful links
This work was done within the [NETWORKS](http://networks.imtlucca.it/) research group at [IMT School for Advanced Studies Lucca](http://www.imtlucca.it/) with [Tiziano Squartini](https://www.imtlucca.it/tiziano.squartini) and [Guido Caldarelli](http://www.guidocaldarelli.com/).

The search heuristic was adapted from [Carlo Nicolini'](https://github.com/CarloNicolini/paco) work. The surprise method is based on the [surprise](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0024195) for community detection.

For more information, please check out our [paper](https://arxiv.org/abs/1810.04717).


```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

import torch
from torch.autograd import Variable

import copy

from sklearn.metrics import r2_score
```

```python
sns.set_context("paper")
sns.set_style("ticks")

np.random.seed(101217209)
torch.manual_seed(101217209)
```

    <torch._C.Generator at 0x7f623e4b55e8>

```python
def get_data(N, min_x, max_x):
    w, b = np.random.randn(2)
    def true_model(X):
        lin_factor = w * X + b
        y = np.sin(lin_factor*10)*(lin_factor**2)
        return y
    X_true = np.arange(min_x, max_x, 0.01)
    y_true = true_model(X_true)
    span = (max_x - min_x)
    scale = 0.2
    X_obs = min_x + span*scale + np.random.rand(N)*(span - 2*scale*span)
    y_obs = true_model(X_obs) + np.random.randn(N)*0.1

    y_mean, y_std = y_obs.mean(), y_obs.std()
    y_obs = (y_obs - y_mean)/y_std
    y_true = (y_true - y_mean)/y_std
    return (X_obs, y_obs, X_true, y_true), (w, b, true_model)
```

```python
N = 40
min_x, max_x = -1, 1
(X_obs, y_obs, X_true, y_true), (w, b, true_model) = get_data(N, min_x, max_x)
```

```python
class SimpleModel(torch.nn.Module):
    def __init__(self, p=0.05, decay=0.001):
        super(SimpleModel, self).__init__()
        self.dropout_p = p
        self.decay = decay
        self.f = torch.nn.Sequential(
            torch.nn.Linear(1,20),
            torch.nn.ReLU(),
            torch.nn.Dropout(p=self.dropout_p),
            torch.nn.Linear(20, 20),
            torch.nn.Sigmoid(),
            torch.nn.Dropout(p=self.dropout_p),
            torch.nn.Linear(20,1)
        )
    def forward(self, X):
        X = Variable(torch.Tensor(X), requires_grad=False)
        return self.f(X)

def ewc_criterion(model, fisher_matrix, prior_params, alpha=1):
    ewc_loss = Variable(torch.zeros(1))
    for p, fm, prior in zip(model.parameters(), fisher_matrix, prior_params):
        param_loss = torch.pow(p - Variable(prior, requires_grad=False), 2)
        fisher_scaled_loss = alpha*Variable(fm, requires_grad=False).mul(param_loss)
        ewc_loss += fisher_scaled_loss.sum()
    return ewc_loss

def get_criterion_optimizer(model):
    criterion = torch.nn.MSELoss()
    optimizer = torch.optim.Adam(
        model.parameters(),
        #lr=0.001,
        #momentum=0.5,
        weight_decay=model.decay)
    return criterion, optimizer

def plot_predictions(model, model_simple, ax=None):
    if ax is None:
        fig, ax = plt.subplots(1,1)
    ax.plot(X_obs[:10], y_obs[:10], ls="none", marker="o", color="b", label="task 1")
    ax.plot(X_obs[10:20], y_obs[10:20], ls="none", marker="d", color="cyan", label="task 2")
    ax.plot(X_obs[20:], y_obs[20:], ls="none", marker="s", color="magenta", label="task 3")
    ax.plot(X_true, y_true, ls="-", lw=1, color="0.5", label="true")

    model.eval()
    y_pred = model(X_true[:, np.newaxis]).data.numpy()
    r_squared = r2_score(y_true, y_pred)
    ax.plot(X_true, y_pred, ls="--", lw=1, color="r", label="ewc ({:.3f})".format(r_squared))

    model_simple.eval()
    y_pred = model_simple(X_true[:, np.newaxis]).data.numpy()
    r_squared = r2_score(y_true, y_pred)
    ax.plot(X_true, y_pred, ls="--", lw=1, color="g", label="simple ({:.3f})".format(r_squared))

    ax.legend()
    ax.set_xlabel("X")
    ax.set_ylabel("Y")
    sns.despine(offset=10)
    return ax

def plot_losses(losses, losses_simple, ax=None):
    if ax is None:
        fig, ax = plt.subplots(1,1,figsize=(10,4))
    ax.plot(losses, lw=0.1, color='r', label='ewc')
    ax.plot(losses_simple, lw=0.1, color='0.5', label='simple')
    ax.legend()
    ax.set_xlabel("Epochs")
    ax.set_ylabel("Loss")
    sns.despine(offset=10)
    return ax

def get_fisher_matrix(model, criterion, X, Y, old_fisher_matrix=None):
    fisher_matrix = [torch.zeros(p.data.shape) for p in model.parameters()]
    counts = 0
    #print(X.shape, Y.shape)
    for x,y in zip(X[:, np.newaxis, np.newaxis], Y[:, np.newaxis]):
        criterion.zero_grad()
        #print(x.shape, y.shape)
        y_pred = model(x)
        #print(y_pred.data.shape)
        loss = criterion(y_pred, Variable(torch.Tensor(y), requires_grad=False))
        loss.backward()
        for i, p in enumerate(model.parameters()):
            grad_squared = torch.pow(p.grad.data, 2)
            #print(grad_squared.shape, fisher_matrix[i].shape)
            fisher_matrix[i] += grad_squared
        counts += 1
    #print(grad_squared.shape, fisher_matrix[i].shape)
    ## Calculate the means
    fisher_matrix = [k/counts for k in fisher_matrix]
    if old_fisher_matrix:
        fisher_matrix = [k + o for k,o in zip(fisher_matrix, old_fisher_matrix)]
    return fisher_matrix

def train_step(
    model, X, Y, criterion, optimizer,
    fisher_matrix=None, model_params=None,
    ecw_alpha=0, clip=10):
    y_pred = model(X[:, np.newaxis])
    loss = criterion(y_pred, Variable(torch.Tensor(Y), requires_grad=False))
    train_loss = loss.data.numpy()[0]
    if ecw_alpha > 0:
        ewc_loss = ewc_criterion(model, fisher_matrix, model_params, alpha=ecw_alpha)
        loss += ewc_loss
    loss.backward()
    torch.nn.utils.clip_grad_norm(model.parameters(),clip)
    optimizer.step()
    return train_loss
```

```python
def train_plot_compare(X_obs, y_obs, prefix="IID", clip=10):
    model = SimpleModel(p=0.01, decay=1e-3)
    model_simple = copy.deepcopy(model)

    fig, ax = plt.subplots(1,3, sharex=True, sharey=True, figsize=(18, 6))
    fig_loss, ax_loss = plt.subplots(3,1, sharex=True, sharey=False, figsize=(10, 12))

    ## Task 1

    fisher_matrix = None
    model_params = None

    for i, X, Y, alpha in zip(
        range(3),
        [X_obs[:10], X_obs[10:20], X_obs[20:]],
        [y_obs[:10], y_obs[10:20], y_obs[20:]],
        [0, 1, 1] # Alpha should increase with tasks
    ):
        title = "Task {}".format(" + ".join(str(k+1) for k in range(i+1)))
        model.train()
        criterion, optimizer = get_criterion_optimizer(model)

        model_simple.train()
        criterion_simple, optimizer_simple = get_criterion_optimizer(model_simple)

        losses = []
        losses_simple = []
        for epoch in range(10000):
            loss = train_step(
                model, X, Y,
                criterion, optimizer,
                fisher_matrix=fisher_matrix, model_params=model_params,
                ecw_alpha=alpha, clip=clip
            )
            losses.append(loss)

            loss = train_step(
                model_simple, X, Y, criterion_simple, optimizer_simple, clip=clip)
            losses_simple.append(loss)
        plot_losses(losses, losses_simple, ax=ax_loss[i])
        plot_predictions(model, model_simple, ax=ax[i])
        ax_loss[i].set_title(title)
        ax[i].set_title(title)
        print(title)

        model.eval()
        fisher_matrix = get_fisher_matrix(model, criterion, X, Y)
        model_params = [copy.deepcopy(p.data) for p in model.parameters()]

    fig.tight_layout()
    fig_loss.tight_layout()

    sns.despine(offset=10, fig=fig)
    sns.despine(offset=10, fig=fig_loss)

    fig.savefig("{}_EWC_predictions.pdf".format(prefix), bbox_inches="tight")
    fig.savefig("{}_EWC_losses.pdf".format(prefix), bbox_inches="tight")

    return model, model_simple
```

## IID samples

```python
plt.plot(X_obs[:10], y_obs[:10], ls="none", marker="o", color="b", label="task 1")
plt.plot(X_obs[10:20], y_obs[10:20], ls="none", marker="d", color="cyan", label="task 2")
plt.plot(X_obs[20:], y_obs[20:], ls="none", marker="s", color="magenta", label="task 3")
plt.plot(X_true, y_true, ls="-", lw=1, color="0.5", label="true")
plt.legend()
sns.despine(offset=10)
```

```python
model, model_simple = train_plot_compare(X_obs, y_obs, prefix="IID")
```

```python
"""
model.eval()
fisher_matrix = get_fisher_matrix(model, criterion, X_obs[:10], y_obs[:10])
model_params = [copy.deepcopy(p.data) for p in model.parameters()]

fig = plt.figure(0)
ax = fig.add_subplot(111)

el = plt.matplotlib.patches.Ellipse(
    xy=[
        model_params[0].squeeze().numpy()[0],
        model_params[1].numpy()[0]
    ],
    width=2./fisher_matrix[0].squeeze().numpy()[0],
    height=2./fisher_matrix[1].numpy()[0])
ax.add_artist(el)
#el.set_clip_box(ax.bbox)
el.set_color('0.5')
el.set_alpha(0.1)
ax.scatter(model_params[0].squeeze().numpy(), model_params[1].numpy(), s=1)
"""
```

    "\nmodel.eval()\nfisher_matrix = get_fisher_matrix(model, criterion, X_obs[:10], y_obs[:10])\nmodel_params = [copy.deepcopy(p.data) for p in model.parameters()]\n\nfig = plt.figure(0)\nax = fig.add_subplot(111)\n\n\nel = plt.matplotlib.patches.Ellipse(\n    xy=[\n        model_params[0].squeeze().numpy()[0],\n        model_params[1].numpy()[0]\n    ],\n    width=2./fisher_matrix[0].squeeze().numpy()[0],\n    height=2./fisher_matrix[1].numpy()[0])\nax.add_artist(el)\n#el.set_clip_box(ax.bbox)\nel.set_color('0.5')\nel.set_alpha(0.1)\nax.scatter(model_params[0].squeeze().numpy(), model_params[1].numpy(), s=1)\n"

## Sequential samples

```python
sorted_idx = np.argsort(X_obs)
X_obs = X_obs[sorted_idx]
y_obs = y_obs[sorted_idx]
```

```python
plt.plot(X_obs[:10], y_obs[:10], ls="none", marker="o", color="b", label="task 1")
plt.plot(X_obs[10:20], y_obs[10:20], ls="none", marker="d", color="cyan", label="task 2")
plt.plot(X_obs[20:], y_obs[20:], ls="none", marker="s", color="magenta", label="task 3")
plt.plot(X_true, y_true, ls="-", lw=1, color="0.5", label="true")
plt.legend()
sns.despine(offset=10)
```

```python
model, model_simple = train_plot_compare(X_obs, y_obs, prefix="Seq")
```

## Improvements

Before each forward pass

$$
\begin{equation}
\sigma_{\theta} = F_{\theta}^{-1/2}\\
\epsilon \sim \mathcal{N}(0, 1)\\
\theta = \theta + \sigma*\epsilon
\end{equation}
$$

```python

```


# Notebook 20: Variational autoencoder for the Ising Model with Keras

# Learning Goal

The goal of this notebook is to implement a VAE to learn a generative model for the 2D Ising model. The goal will be to understand how latent variables can capture physical quantities (such as the order parameter) and the effect of hyperparameters on VAE results.

## Overview

In this notebook, we will write a variational autoencoder (VAE) in Keras for the 2D Ising model dataset. The code in this notebook is adapted from (https://blog.keras.io/building-autoencoders-in-keras.html) and reproduces some of the results found in (https://arxiv.org/pdf/1703.02435.pdf). The goal of the notebook is to show how to implement a variational autoencoder in Keras in order to learn effective low-dimensional representations of equilibrium samples drawn from the 2D ferromagnetic Ising model with periodic boundary conditions.

## Structure of the notebook
The notebook is structured as follows.

 1. We load in the Ising dataset
 2. We construct the variational auto encoder model using Keras
 3. We train the model on a training set and then visualize the latent representation
 4. We compare the learned representation with the magnetization order parameter

### Load the dataset

```python
import pickle
from sklearn.model_selection import train_test_split
import collections

def load_data_set(root="IsingMC/", train_size = 0.5):
    """Loads the Ising dataset in the format required for training the tensorflow VAE

    Parameters
    -------

    root: str, default = "IsingMC/"
        Location of the directory containing the Ising dataset
    train_size: float, default = 0.5
        Size ratio of the training set. 1-train_size corresponds to the test set size ratio.

    """
    # The Ising dataset contains 16*10000 samples taken in T=np.arange(0.25,4.0001,0.25)
    data = pickle.load(open(root+'Ising2DFM_reSample_L40_T=All.pkl','rb'))
    data = np.unpackbits(data).astype(int).reshape(-1,1600) # decompression of data and casting to int.
    Y = np.hstack([t]*10000 for t in np.arange(0.25,4.01,0.25)) # labels

    # Here we downsample the dataset and use 1000 samples at each temperature
    tmp = np.arange(10000)
    np.random.shuffle(tmp)
    rand_idx=tmp[:10000]

    X = np.vstack(data[i*10000:(i+1)*10000][rand_idx] for i, _ in enumerate(np.arange(0.25,4.01,0.25)))
    Y = np.hstack(Y[i*10000:(i+1)*10000][rand_idx] for i, _ in enumerate(np.arange(0.25,4.01,0.25)))
    # Note that data is not currently shuffled

    return X, Y

```

## Constructing a VAE class

Here, we implement the VAE in a slightly different way than we did for the MNIST dataset. We have chosen to create a new VAE class so that the parameters can be easily changed for new data.

```python
from __future__ import print_function
import os
import numpy as np

from scipy.stats import norm

from keras.layers import Input, Dense, Lambda
from keras.models import Model
from keras import backend as K
from keras import metrics, losses
from keras.datasets import mnist

class VAE:

    def __init__(self, batch_size=100, original_dim =1600, latent_dim = 100, epochs=50, root="IsingMC/", epsilon=0.5):
        '''
         #Reference
         - Auto-Encoding Variational Bayes
           https://arxiv.org/abs/1312.6114
           This code is taken from Keras VAE tutorial available at https://blog.keras.io/building-autoencoders-in-keras.html

        Parameters
        ----------

        batch_size : int, default=100
            Size of batches for gradient descent
        original_dim : int, default =1600
            Number of features
        latent_dim: int, default = 100
            Dimensionality of the latent space
        epochs: int, default = 50
            Number of epochs for training
        '''
        self.batch_size = batch_size
        self.original_dim = original_dim
        self.latent_dim = latent_dim
        self.intermediate_dim = 256
        self.epochs = epochs
        self.epsilon_std = epsilon

    def sampling(self, args):
        ''' Sampling from the latent variables using the means and log-variances'''
        z_mean, z_log_var = args
        epsilon = K.random_normal(shape=(K.shape(z_mean)[0], self.latent_dim), mean=0.,
                                  stddev=self.epsilon_std)
        return z_mean + K.exp(z_log_var / 2) * epsilon

    def build(self):
        """ This class method constructs the VAE model
        """
        original_dim = self.original_dim
        latent_dim = self.latent_dim
        intermediate_dim = self.intermediate_dim

        # encoder
        self.x = Input(shape=(original_dim,))
        h = Dense(intermediate_dim, activation='relu')(self.x)
        self.z_mean = Dense(latent_dim)(h)
        self.z_log_var = Dense(latent_dim)(h)

        # note that "output_shape" isn't necessary with the TensorFlow backend
        z = Lambda(self.sampling, output_shape=(latent_dim,))([self.z_mean, self.z_log_var])

        # we instantiate these layers separately so as to reuse them later
        decoder_h = Dense(intermediate_dim, activation='relu')
        decoder_mean = Dense(original_dim, activation='sigmoid')
        h_decoded = decoder_h(z)
        x_decoded_mean = decoder_mean(h_decoded)

        #decoder
        decoder_input = Input(shape=(latent_dim,))
        _h_decoded = decoder_h(decoder_input)
        _x_decoded_mean = decoder_mean(_h_decoded)

        self.generator = Model(decoder_input, _x_decoded_mean)

        # end-to-end VAE model
        self.vae = Model(self.x, x_decoded_mean)

        # encoder, from inputs to latent space
        self.encoder = Model(self.x, self.z_mean)

        # decoder
        #self.decoder = Model(decoder_input, _x_decoded_mean)

        # Compute VAE loss
        self.vae.compile(optimizer='rmsprop', loss=self.vae_loss)

        # Prints a summary of the architecture used
        self.vae.summary()

    def vae_loss(self, x, x_decoded_mean):
            xent_loss = losses.binary_crossentropy(x, x_decoded_mean)
            kl_loss = - 0.5 * K.mean(1 + self.z_log_var - K.square(self.z_mean) - K.exp(self.z_log_var), axis=-1)
            return xent_loss + kl_loss

    def train(self, x_train, x_test):
        from sklearn.preprocessing import minmax_scale

        x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:]))) # flatten each sample out
        x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
        x_train = minmax_scale(x_train) # this step is required in order to use cross-entropy loss for reconstruction
        x_test = minmax_scale(x_train) # scaling features in 0,1 interval

        self.vae.fit(x_train, x_train,
                shuffle=True,
                epochs=self.epochs,
                batch_size=self.batch_size,
                validation_data=(x_test, x_test)
                )

        # build a model to project inputs on the latent space
        #encoder = Model(self.x, self.z_mean)

    def predict_latent(self, xnew):
        # build a model to project inputs on the latent space
        return self.encoder.predict(xnew)

    def generate_decoding(self, znew):
        # Generate new fantasy particles
        return self.generator.predict(znew)
```

    /Users/mbukov/anaconda3/envs/DNN/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters
    Using TensorFlow backend.

## Load the 2D Ising dataset

```python
# The y labels are the temperatures in np.arange(0.25,4.01,0.2) at which X was drawn

#Directory where data is stored
root=path_to_data=os.path.expanduser('~')+'/Dropbox/MachineLearningReview/Datasets/isingMC/'
X, Y = load_data_set(root= root)
```

## Construct a training and a validation set

```python
from sklearn.model_selection import train_test_split
xtrain, xtest, ytrain, ytest = train_test_split(X, Y, test_size=0.8)
print(xtrain.shape)
```

    (32000, 1600)

## Construct and train the variational autoencoder model

```python
model = VAE(epochs=5, latent_dim=2, epsilon=0.2) # Choose model parameters
model.build() # Construct VAE model using Keras
model.train(xtrain, xtest) # Trains VAE model based on custom loss function
```

    __________________________________________________________________________________________________
    Layer (type)                    Output Shape         Param #     Connected to
    ==================================================================================================
    input_1 (InputLayer)            (None, 1600)         0
    __________________________________________________________________________________________________
    dense_1 (Dense)                 (None, 256)          409856      input_1[0][0]
    __________________________________________________________________________________________________
    dense_2 (Dense)                 (None, 2)            514         dense_1[0][0]
    __________________________________________________________________________________________________
    dense_3 (Dense)                 (None, 2)            514         dense_1[0][0]
    __________________________________________________________________________________________________
    lambda_1 (Lambda)               (None, 2)            0           dense_2[0][0]
                                                                     dense_3[0][0]
    __________________________________________________________________________________________________
    dense_4 (Dense)                 (None, 256)          768         lambda_1[0][0]
    __________________________________________________________________________________________________
    dense_5 (Dense)                 (None, 1600)         411200      dense_4[0][0]
    ==================================================================================================
    Total params: 822,852
    Trainable params: 822,852
    Non-trainable params: 0
    __________________________________________________________________________________________________

    /Users/mbukov/anaconda3/envs/DNN/lib/python3.6/site-packages/sklearn/utils/validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64.
      warnings.warn(msg, DataConversionWarning)

    Train on 32000 samples, validate on 32000 samples
    Epoch 1/5
    32000/32000 [==============================] - 16s 508us/step - loss: 5.4376 - val_loss: 0.8315
    Epoch 2/5
    32000/32000 [==============================] - 18s 555us/step - loss: 2.9111 - val_loss: 1.8376
    Epoch 3/5
    32000/32000 [==============================] - 13s 419us/step - loss: 2.2729 - val_loss: 0.8762
    Epoch 4/5
    32000/32000 [==============================] - 13s 399us/step - loss: 1.6337 - val_loss: 0.8224
    Epoch 5/5
    32000/32000 [==============================] - 13s 408us/step - loss: 1.3037 - val_loss: 1.1051

## Encoding samples to latent space:

We predict the latent variable coordinates for the test set:

```python
zpred = model.predict_latent(xtest)
print(zpred.shape)
```

    (128000, 2)

Let's visualize this 2-dimensional space. We also color each sample according to the temperature at which it was drawn. The largest temperature is red ($T=4.0$) and lowest is blue ($T=0.25$).

```python
# To make plots pretty
golden_size = lambda width: (width, 2. * width / (1 + np.sqrt(5)))

%matplotlib inline
import matplotlib.pyplot as plt
plt.rc('font',**{'size':16})
fig, ax = plt.subplots(1,figsize=golden_size(8))
sc = ax.scatter(zpred[:,0], zpred[:,1], c=ytest/4.0, s=4, cmap="coolwarm")
ax.set_xlabel('First latent dimension of the VAE')
ax.set_ylabel('Second latent dimension of the VAE')
plt.colorbar(sc, label='$0.25\\times$Temperature')
plt.savefig('VAE_ISING_latent.png')
plt.show()
```

## Understanding the latent space embeddings

To better understand the latent space, we can plot each of the latent dimension coordinates against the corresponding magnetization of each sample.

```python
plt.rc('font',**{'size':16})
fig, ax = plt.subplots(1,2,figsize=(15,8))
ax[0].scatter(zpred[:,0], np.mean(xtest, axis=1), c=ytest/4.0, s=2, cmap="coolwarm")
ax[0].set_xlabel('First latent dimension of the VAE')
ax[0].set_ylabel('Magnetization')
sc = ax[1].scatter(zpred[:,1], np.mean(xtest, axis=1), c=ytest/4.0, s=2, cmap="coolwarm")
ax[1].set_xlabel('Second latent dimension of the VAE')
ax[1].set_ylabel('Magnetization')
plt.colorbar(sc, label='$0.25\\times$Temperature')
plt.savefig('VAE_ISING_latent_magnetization.png')
plt.show()
```

It appears that these dimensions are strongly correlated, meaning that the learned representation is effectively one-dimensional. This can be understood by the fact that in order to draw samples at high and low temperatures, we only require the information about the magnetization order parameter (we only have to draw samples from a factorized mean-field distribution):
\begin{equation}
p(s_i=\pm) = \frac{1\pm m}{2},
\end{equation}
where $p(s_i=\pm)$ is the probability that spin $i$ is up ($+$) or down ($-$), given that the magnetization sector is fixed.
Note that this is not true in the vicinity of the critical point, where mean-field theory fails as the system develops long-range correlations.

We see that the VAE correctly captures the structure of the data. The high-temperature samples cluster at intermediate values and the ordered samples with positive and negative magnetization cluster in opposite regions. This can be more effectively visualized using a 1-D histogram:

```python
# Make histogram at the
plt.hist(zpred[:,0],bins=50)
plt.show()
```

## Generating New Examples

So far in this notebook, we have shown that the latent structure of VAEs can automatically identify order parameters. This is not surprising since even the first principle component in a PCA is essentially the magnetization.

The interesting feature of VAEs is that they are also a generative model. We now ask how well the VAE can generate new examples. Our decoder returns probabilities for each pixel being 1. We then can draw random numbers to generate samples. This is done in the short function below.

One again, as in the VAE MNIST notebook, we will sample our latent space togenerate the particles in two different ways

* Sampling uniformally in the latent space
* Sampling accounting for the fact that the latent space is Gaussian so that we expect most of the data points to be centered around (0,0) and fall off exponentially in all directions. This is done by transforming the uniform grid using the inverse Cumulative Distribution Function (CDF) for the Gaussian.

```python
# Generate fantasy particles
def generate_samples(model, z_input):
    temp=model.generate_decoding(z_input).reshape(n*n,1600)
    draws=np.random.uniform(size=temp.shape)
    samples=np.array(draws<temp).astype(int)
    return samples

```

```python
# display a 2D manifold of the images
n = 5  # figure with 15x15 images
quantile_min = 0.01
quantile_max = 0.99
latent_dim=2

img_rows=40
img_cols=40

# Linear Sampling
# we will sample n points within [-15, 15] standard deviations
z1_u = np.linspace(5, -5, n)
z2_u = np.linspace(5, -5, n)
z_grid = np.dstack(np.meshgrid(z1_u, z2_u))
z_input=np.array(z_grid.reshape(n*n, latent_dim))
print(z_input.shape)
x_pred_grid = generate_samples(model,z_input) \
                     .reshape(n, n, img_rows, img_cols)

print(x_pred_grid.shape)
# Inverse CDF sampling
z1 = norm.ppf(np.linspace(quantile_min, quantile_max, n))
z2 = norm.ppf(np.linspace(quantile_max, quantile_min, n))
z_grid2 = np.dstack(np.meshgrid(z1, z2))
z_input=np.array(z_grid2.reshape(n*n, latent_dim))
x_pred_grid2 = generate_samples(model,z_input) \
                     .reshape(n, n, img_rows, img_cols)

# Plot figure
fig, ax = plt.subplots(figsize=golden_size(10))

ax.imshow(np.block(list(map(list, x_pred_grid))), cmap='gray')

ax.set_xticks(np.arange(0, n*img_rows, img_rows) + .5 * img_rows)
ax.set_xticklabels(map('{:.2f}'.format, z1), rotation=90)

ax.set_yticks(np.arange(0, n*img_cols, img_cols) + .5 * img_cols)
ax.set_yticklabels(map('{:.2f}'.format, z2))

ax.set_xlabel('$z_1$')
ax.set_ylabel('$z_2$')
ax.set_title('Uniform')
ax.grid(False)

plt.savefig('VAE_ISING_fantasy_uniform.pdf')
plt.show()

# Plot figure Inverse CDF sampling
fig, ax = plt.subplots(figsize=golden_size(10))

ax.imshow(np.block(list(map(list, x_pred_grid2))), cmap='gray_r', vmin=0, vmax=1)

ax.set_xticks(np.arange(0, n*img_rows, img_rows) + .5 * img_rows)
ax.set_xticklabels(map('{:.2f}'.format, z1), rotation=90)

ax.set_yticks(np.arange(0, n*img_cols, img_cols) + .5 * img_cols)
ax.set_yticklabels(map('{:.2f}'.format, z2))

ax.set_xlabel('$z_1$')
ax.set_ylabel('$z_2$')
ax.set_title('Inverse CDF')
ax.grid(False)
plt.savefig('VAE_ISING_fantasy_invCDF.pdf')
plt.show()

```

## Examples fail to capture critial correlations

We see that the examples look quite different from real Ising configurations -- they lack the large scale patchiness seen in the  critical region. They mostly turn out to be unstructured speckles that reflect only the average probability that a pixel is on in a region. This is not surprising since our VAE has no spatial structure and gets very little information about correlations in the binary cross-entropy which we use to measure reconstruction errors.

## Exercises

* To gain more intuition for the VAE parameters, change $\epsilon$. What happens when epsilon is $0.01$, $1$, $10$?
* Change the number of latent dimensions to 1, or 10. Visualize the 10 dimensional latent space using PCA? Are the reconstructed samples and loss functions very different?
* What do you think would happen for other spin systems where the order parameter is a vector such as nematically ordered materials?
* Replace the decoder with a (de)convolutional net that includes spatial structure? Doe this improve the performance of the VAE for generating examples?

```python

```