# Tutorial 3: Synaptic transmission - Models of static and dynamic synapses
**Week 2, Day 3: Biological Neuron Models**

**By Neuromatch Academy**

__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar

__Content reviewers:__  Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Matthew Krause, Spiros Chavlis, Michael Waskom

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

*Estimated timing of tutorial: 1 hour, 10 min*

Synapses connect neurons into neural networks or circuits. Specialized electrical synapses make direct, physical connections between neurons. In this tutorial, however, we will focus on **chemical synapses**, which are more common in the brain. These synapses do not physically join neurons. Instead, a spike in the presynaptic cell causes a chemical, or neurotransmitter, to be released into a small space between the neurons called the synaptic cleft. Once the chemical diffuses across that space, it changes the pearmeability of the postsynaptic membrane, which may result in a positive or negative change in the membrane voltage.

In this tutorial, we will model chemical synaptic transmission and study some interesting effects produced by **static synapses** and **dynamic synapses**.

First, we will start by writing code to simulate static synapses -- whose weight is always fixed.
Next, we will extend the model and model **dynamic synapses** -- whose synaptic strength is dependent on the recent spike history: synapses can either progressively increase or decrease the size of their effects on the post-synaptic neuron, based on the recent firing rate of its presynaptic partners. This feature of synapses in the brain is called **Short-Term Plasticity** and causes synapses to undergo *Facilitation* or *Depression*.

Our goals for this tutorial are to:

- simulate static synapses and study how excitation and inhibition affect the patterns in the neurons' spiking output
- define mean- or fluctuation-driven regimes
- simulate short-term dynamics of synapses (facilitation and depression)
- study how a change in pre-synaptic firing history affects the synaptic weights (i.e., PSP amplitude)

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/8djsm/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

```python
# @title Video 1: Static and dynamic synapses
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1dz4y1D7zu", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="Hbz2lj2AO_0", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

---
# Setup

```python
# Imports

import matplotlib.pyplot as plt
import numpy as np
```

```python
# @title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format='retina'
# use NMA plot style
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
my_layout = widgets.Layout()
```

```python
# @title Plotting Functions

def my_illus_LIFSYN(pars, v_fmp, v):
  """
  Illustartion of FMP and membrane voltage

  Args:
    pars  : parameters dictionary
    v_fmp : free membrane potential, mV
    v     : membrane voltage, mV

  Returns:
    plot of membrane voltage and FMP, alongside with the spiking threshold
    and the mean FMP (dashed lines)
  """

  plt.figure(figsize=(14, 5))
  plt.plot(pars['range_t'], v_fmp, 'r', lw=1.,
           label='Free mem. pot.', zorder=2)
  plt.plot(pars['range_t'], v, 'b', lw=1.,
           label='True mem. pot', zorder=1, alpha=0.7)
  plt.axhline(-55, 0, 1, color='k', lw=2., ls='--',
              label='Spike Threshold', zorder=1)
  plt.axhline(np.mean(v_fmp), 0, 1, color='r', lw=2., ls='--',
              label='Mean Free Mem. Pot.', zorder=1)
  plt.xlabel('Time (ms)')
  plt.ylabel('V (mV)')
  plt.legend(loc=[1.02, 0.68])
  plt.show()

def my_illus_STD(Poi_or_reg=False, rate=20., U0=0.5,
                 tau_d=100., tau_f=50., plot_out=True):
  """
   Only for one presynaptic train

  Args:
    Poi_or_reg : Poisson or regular input spiking trains
    rate       : Rate of input spikes, Hz
    U0         : synaptic release probability at rest
    tau_d      : synaptic depression time constant of x [ms]
    tau_f      : synaptic facilitation time constantr of u [ms]
    plot_out   : whether ot not to plot, True or False

  Returns:
    Nothing.
  """

  T_simu = 10.0 * 1000 / (1.0 * rate)  # 10 spikes in the time window
  pars = default_pars(T=T_simu)
  dt = pars['dt']

  if Poi_or_reg:
    # Poisson type spike train
    pre_spike_train = Poisson_generator(pars, rate, n=1)
    pre_spike_train = pre_spike_train.sum(axis=0)
  else:
    # Regular firing rate
    isi_num = int((1e3/rate)/dt)  # number of dt
    pre_spike_train = np.zeros(len(pars['range_t']))
    pre_spike_train[::isi_num] = 1.

  u, R, g = dynamic_syn(g_bar=1.2, tau_syn=5., U0=U0,
                        tau_d=tau_d, tau_f=tau_f,
                        pre_spike_train=pre_spike_train,
                        dt=pars['dt'])

  if plot_out:
    plt.figure(figsize=(12, 6))

    plt.subplot(221)
    plt.plot(pars['range_t'], R, 'b', label='R')
    plt.plot(pars['range_t'], u, 'r', label='u')
    plt.legend(loc='best')
    plt.xlim((0, pars['T']))
    plt.ylabel(r'$R$ or $u$ (a.u)')
    plt.subplot(223)
    spT = pre_spike_train > 0
    t_sp = pars['range_t'][spT]  #spike times
    plt.plot(t_sp, 0. * np.ones(len(t_sp)), 'k|', ms=18, markeredgewidth=2)
    plt.xlabel('Time (ms)');
    plt.xlim((0, pars['T']))
    plt.yticks([])
    plt.title('Presynaptic spikes')

    plt.subplot(122)
    plt.plot(pars['range_t'], g, 'r', label='STP synapse')
    plt.xlabel('Time (ms)')
    plt.ylabel('g (nS)')
    plt.xlim((0, pars['T']))

    plt.tight_layout()

  if not Poi_or_reg:
    return g[isi_num], g[9*isi_num]

def plot_volt_trace(pars, v, sp):
  """
  Plot trajetory of membrane potential for a single neuron

  Args:
    pars   : parameter dictionary
    v      : volt trajetory
    sp     : spike train

  Returns:
    figure of the membrane potential trajetory for a single neuron
  """

  V_th = pars['V_th']
  dt = pars['dt']
  if sp.size:
      sp_num = (sp/dt).astype(int) - 1
      v[sp_num] += 10

  plt.plot(pars['range_t'], v, 'b')
  plt.axhline(V_th, 0, 1, color='k', ls='--', lw=1.)
  plt.xlabel('Time (ms)')
  plt.ylabel('V (mV)')
```

```python
# @title Helper Functions

def my_GWN(pars, mu, sig, myseed=False):
  """
  Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    myseed     : random seed. int or boolean
                  the same seed will give the same random number sequence

  Returns:
    I          : Gaussian White Noise (GWN) input
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  # you can fix the seed of the random number generator so that the results
  # are reliable. However, when you want to generate multiple realizations
  # make sure that you change the seed for each new realization.
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate GWN
  # we divide here by 1000 to convert units to seconds.
  I_GWN = mu + sig * np.random.randn(Lt) / np.sqrt(dt / 1000.)

  return I_GWN

def Poisson_generator(pars, rate, n, myseed=False):
  """
  Generates poisson trains

  Args:
    pars       : parameter dictionary
    rate       : noise amplitute [Hz]
    n          : number of Poisson trains
    myseed     : random seed. int or boolean

  Returns:
    pre_spike_train : spike train matrix, ith row represents whether
                      there is a spike in ith spike train over time
                      (1 if spike, 0 otherwise)
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate uniformly distributed random variables
  u_rand = np.random.rand(n, Lt)

  # generate Poisson train
  poisson_train = 1. * (u_rand < rate * (dt / 1000.))

  return poisson_train

def default_pars(**kwargs):
  pars = {}

  ### typical neuron parameters###
  pars['V_th'] = -55.    # spike threshold [mV]
  pars['V_reset'] = -75. # reset potential [mV]
  pars['tau_m'] = 10.    # membrane time constant [ms]
  pars['g_L'] = 10.      # leak conductance [nS]
  pars['V_init'] = -65.  # initial potential [mV]
  pars['E_L'] = -75.     # leak reversal potential [mV]
  pars['tref'] = 2.      # refractory time (ms)

  ### simulation parameters ###
  pars['T'] = 400.  # Total duration of simulation [ms]
  pars['dt'] = .1   # Simulation time step [ms]

  ### external parameters if any ###
  for k in kwargs:
    pars[k] = kwargs[k]

  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized time points [ms]

  return pars
```

In the `Helper Function`:

- Gaussian white noise generator: `my_GWN(pars, mu, sig, myseed=False)`
- Poissonian spike train generator: `Poisson_generator(pars, rate, n, myseed=False)`
- default parameter function (as before)

---
# Section 1: Static synapses

*Estimated timing to here from start of tutorial: 20 min*

## Section 1.1: Simulate synaptic conductance dynamics

Synaptic input _in vivo_ consists of a mixture of **excitatory** neurotransmitters, which depolarizes the cell and drives it towards spike threshold, and **inhibitory** neurotransmitters that hyperpolarize it, driving it away from spike threshold. These chemicals cause specific ion channels on the postsynaptic neuron to open, resulting in a change in that neuron's conductance and, therefore, the flow of current in or out of the cell.

This process can be modelled by assuming that the presynaptic neuron's spiking activity produces transient changes in the postsynaptic neuron's conductance ($g_{\rm syn}(t)$). Typically, the conductance transient is modeled as an exponential function.

Such conductance transients can be generated using a simple ordinary differential equation (ODE):

\\

\begin{eqnarray}
\frac{dg_{\rm syn}(t)}{dt} &=& \bar{g}_{\rm syn} \sum_k \delta(t-t_k) -g_{\rm syn}(t)/\tau_{\rm syn}
\end{eqnarray}

\\

where $\bar{g}_{\rm syn}$ (often referred to as synaptic weight) is the maximum conductance elicited by each incoming spike, and $\tau_{\rm syn}$ is the synaptic time constant. Note that the summation runs over all spikes received by the neuron at time $t_k$.

Ohm's law allows us to convert conductance changes to the current as:

\\

\begin{align}
I_{\rm syn}(t) = g_{\rm syn}(t)(V(t)-E_{\rm syn})  \\
\end{align}

\\

The reversal potential $E_{\rm syn}$ determines the direction of current flow and the excitatory or inhibitory nature of the synapse.

**Thus, incoming spikes are filtered by an exponential-shaped kernel, effectively low-pass filtering the input. In other words, synaptic input is not white noise, but it is, in fact, colored noise, where the color (spectrum) of the noise is determined by the synaptic time constants of both excitatory and inhibitory synapses.**

In a neuronal network, the total synaptic input current $I_{\rm syn}$ is the sum of both excitatory and inhibitory inputs. Assuming the total excitatory and inhibitory conductances received at time $t$ are $g_E(t)$ and $g_I(t)$, and their corresponding reversal potentials are $E_E$ and $E_I$, respectively, then the total synaptic current can be described as:

\\

\begin{align}
I_{\rm syn}(V(t),t) = -g_E(t) (V-E_E) - g_I(t) (V-E_I)
\end{align}

\\

Accordingly, the membrane potential dynamics of the LIF neuron under synaptic current drive become:

\\

\begin{eqnarray}
\tau_m\frac{dV(t)}{dt} = -(V(t)-E_L) - \frac{g_E(t)}{g_L} (V(t)-E_E) - \frac{g_I(t)}{g_L} (V(t)-E_I) + \frac{I_{\rm inj}}{g_L}\quad (2)
\end{eqnarray}

\\

$I_{\rm inj}$ is an external current injected in the neuron, which is under experimental control; it can be GWN, DC, or anything else.

We will use Eq. (2) to simulate the conductance-based LIF neuron model below.

In the previous tutorials, we saw how the output of a single neuron (spike count/rate and spike time irregularity) changes when we stimulate the neuron with DC and GWN, respectively. Now, we are in a position to study how the neuron behaves when it is bombarded with both excitatory and inhibitory spikes trains -- as happens *in vivo*.

What kind of input is a neuron receiving? When we do not know, we chose the simplest option. The simplest model of input spikes is given when every input spike arrives independently of other spikes, i.e., we assume that the input is Poissonian.

## Section 1.2: Simulate LIF neuron with conductance-based synapses

We are now ready to simulate a LIF neuron with conductance-based synaptic inputs! The following code defines the LIF neuron with synaptic input modeled as conductance transients.

```python
# @markdown Execute this cell to get a function for conductance-based LIF neuron (run_LIF_cond)

def run_LIF_cond(pars, I_inj, pre_spike_train_ex, pre_spike_train_in):
  """
  Conductance-based LIF dynamics

  Args:
    pars               : parameter dictionary
    I_inj              : injected current [pA]. The injected current here
                         can be a value or an array
    pre_spike_train_ex : spike train input from presynaptic excitatory neuron
    pre_spike_train_in : spike train input from presynaptic inhibitory neuron

  Returns:
    rec_spikes         : spike times
    rec_v              : mebrane potential
    gE                 : postsynaptic excitatory conductance
    gI                 : postsynaptic inhibitory conductance

  """

  # Retrieve parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, E_L = pars['V_init'], pars['E_L']
  gE_bar, gI_bar = pars['gE_bar'], pars['gI_bar']
  VE, VI = pars['VE'], pars['VI']
  tau_syn_E, tau_syn_I = pars['tau_syn_E'], pars['tau_syn_I']
  tref = pars['tref']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # Initialize
  tr = 0.
  v = np.zeros(Lt)
  v[0] = V_init
  gE = np.zeros(Lt)
  gI = np.zeros(Lt)
  Iinj = I_inj * np.ones(Lt)  # ensure Iinj has length Lt

  if pre_spike_train_ex.max() == 0:
    pre_spike_train_ex_total = np.zeros(Lt)
  else:
    pre_spike_train_ex_total = pre_spike_train_ex.sum(axis=0) * np.ones(Lt)

  if pre_spike_train_in.max() == 0:
    pre_spike_train_in_total = np.zeros(Lt)
  else:
    pre_spike_train_in_total = pre_spike_train_in.sum(axis=0) * np.ones(Lt)

  # simulation
  rec_spikes = []  # recording spike times
  for it in range(Lt - 1):
    if tr > 0:
      v[it] = V_reset
      tr = tr - 1
    elif v[it] >= V_th:  # reset voltage and record spike event
      rec_spikes.append(it)
      v[it] = V_reset
      tr = tref / dt

    # update the synaptic conductance
    gE[it + 1] = gE[it] - (dt / tau_syn_E) * gE[it] + gE_bar * pre_spike_train_ex_total[it + 1]
    gI[it + 1] = gI[it] - (dt / tau_syn_I) * gI[it] + gI_bar * pre_spike_train_in_total[it + 1]

    # calculate the increment of the membrane potential
    dv = (dt / tau_m) * (-(v[it] - E_L) \
                         - (gE[it + 1] / g_L) * (v[it] - VE) \
                         - (gI[it + 1] / g_L) * (v[it] - VI) + Iinj[it] / g_L)

    # update membrane potential
    v[it+1] = v[it] + dv

  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes, gE, gI

print(help(run_LIF_cond))
```

    Help on function run_LIF_cond in module __main__:

    run_LIF_cond(pars, I_inj, pre_spike_train_ex, pre_spike_train_in)
        Conductance-based LIF dynamics

        Args:
          pars               : parameter dictionary
          I_inj              : injected current [pA]. The injected current here
                               can be a value or an array
          pre_spike_train_ex : spike train input from presynaptic excitatory neuron
          pre_spike_train_in : spike train input from presynaptic inhibitory neuron

        Returns:
          rec_spikes         : spike times
          rec_v              : mebrane potential
          gE                 : postsynaptic excitatory conductance
          gI                 : postsynaptic inhibitory conductance

    None

### Coding Exercise 1.2: Measure the mean free membrane potential

Let's simulate the conductance-based LIF neuron with presynaptic spike trains generated by a `Poisson_generator` with rate 10 Hz for both excitatory and inhibitory inputs. Here, we choose 80 excitatory presynaptic spike trains and 20 inhibitory ones.

Previously, we've already learned that $CV_{\rm ISI}$ can describe the irregularity of the output spike pattern. Now, we will introduce a new descriptor of the neuron membrane, i.e., the  **Free Membrane Potential (FMP)** -- the membrane potential of the neuron when its spike threshold is removed.

Although this is completely artificial, calculating this quantity allows us to get an idea of how strong the input is. We are mostly interested in knowing the mean and standard deviation (std.) of the FMP. In the exercise, you can visualize the FMP and membrane voltage with spike threshold.

```python
# Get default parameters
pars = default_pars(T=1000.)

# Add parameters
pars['gE_bar'] = 2.4    # [nS]
pars['VE'] = 0.         # [mV] excitatory reversal potential
pars['tau_syn_E'] = 2.  # [ms]
pars['gI_bar'] = 2.4    # [nS]
pars['VI'] = -80.       # [mV] inhibitory reversal potential
pars['tau_syn_I'] = 5.  # [ms]

# Generate presynaptic spike trains
pre_spike_train_ex = Poisson_generator(pars, rate=10, n=80)
pre_spike_train_in = Poisson_generator(pars, rate=10, n=20)

# Simulate conductance-based LIF model
v, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,
                                     pre_spike_train_in)

# Show spikes more clearly by setting voltage high
dt, range_t = pars['dt'], pars['range_t']
if rec_spikes.size:
  sp_num = (rec_spikes / dt).astype(int) - 1
  v[sp_num] = 10   # draw nicer spikes

####################################################################
## TODO for students: measure the free membrane potential
# In order to measure the free membrane potential, first,
# you should prevent the firing of the LIF neuron
# How to prevent a LIF neuron from firing? Increase the threshold pars['V_th'].
#raise NotImplementedError("Student exercise: change threshold and calculate FMP")
####################################################################

# Change the threshold
pars['V_th'] = 0

# Calculate FMP
v_fmp, _, _, _ = run_LIF_cond(pars, 0, pre_spike_train_ex, pre_spike_train_in)

my_illus_LIFSYN(pars, v_fmp, v)
```

```python
# to_remove solution

# Get default parameters
pars = default_pars(T=1000.)

# Add parameters
pars['gE_bar'] = 2.4    # [nS]
pars['VE'] = 0.         # [mV] excitatory reversal potential
pars['tau_syn_E'] = 2.  # [ms]
pars['gI_bar'] = 2.4    # [nS]
pars['VI'] = -80.       # [mV] inhibitory reversal potential
pars['tau_syn_I'] = 5.  # [ms]

# Generate presynaptic spike trains
pre_spike_train_ex = Poisson_generator(pars, rate=10, n=80)
pre_spike_train_in = Poisson_generator(pars, rate=10, n=20)

# Simulate conductance-based LIF model
v, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,
                                     pre_spike_train_in)

# Show spikes more clearly by setting voltage high
dt, range_t = pars['dt'], pars['range_t']
if rec_spikes.size:
  sp_num = (rec_spikes / dt).astype(int) - 1
  v[sp_num] = 10   # draw nicer spikes

# Change the threshold
pars['V_th'] = 1e3

# Calculate FMP
v_fmp, _, _, _ = run_LIF_cond(pars, 0, pre_spike_train_ex, pre_spike_train_in)

with plt.xkcd():
  my_illus_LIFSYN(pars, v_fmp, v)
```

### Interactive Demo 1.2: Conductance-based LIF Explorer with different E/I input

In the following, we can investigate how varying the ratio of excitatory to inhibitory inputs changes the firing rate and the spike time regularity (see the output text).

To change both the excitatory and inhibitory inputs, we will vary their firing rates. *However, if you wish, you can vary the strength and/or the number of these connections as well.*

Pay close attention to the mean free membrane potential (red dotted line) and its location with respect to the spike threshold (black dotted line).

Is there a relationship between the mean of the FMP and spike time irregularity ($CV_{\rm ISI}$)? What happens when the mean of the FMP is above the spike threshold vs below?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!
np.random.seed(2020)
my_layout.width = '450px'
@widgets.interact(
    inh_rate=widgets.FloatSlider(20., min=10., max=60., step=5.,
                                 layout=my_layout),
    exc_rate=widgets.FloatSlider(10., min=2., max=20., step=2.,
                                 layout=my_layout)
)

def EI_isi_regularity(exc_rate, inh_rate):

  pars = default_pars(T=1000.)
  # Add parameters
  pars['gE_bar'] = 3.     # [nS]
  pars['VE'] = 0.         # [mV] excitatory reversal potential
  pars['tau_syn_E'] = 2.  # [ms]
  pars['gI_bar'] = 3.     # [nS]
  pars['VI'] = -80.       # [mV] inhibitory reversal potential
  pars['tau_syn_I'] = 5.  # [ms]

  pre_spike_train_ex = Poisson_generator(pars, rate=exc_rate, n=80)
  pre_spike_train_in = Poisson_generator(pars, rate=inh_rate, n=20)  # 4:1

  # Lets first simulate a neuron with identical input but with no spike
  # threshold by setting the threshold to a very high value
  # so that we can look at the free membrane potential
  pars['V_th'] = 1e3
  v_fmp, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,
                                           pre_spike_train_in)

  # Now simulate a LIP with a regular spike threshold
  pars['V_th'] = -55.
  v, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,
                                       pre_spike_train_in)
  dt, range_t = pars['dt'], pars['range_t']
  if rec_spikes.size:
      sp_num = (rec_spikes / dt).astype(int) - 1
      v[sp_num] = 10  # draw nicer spikes

  spike_rate = 1e3 * len(rec_spikes) / pars['T']

  cv_isi = 0.
  if len(rec_spikes) > 3:
    isi = np.diff(rec_spikes)
    cv_isi = np.std(isi) / np.mean(isi)

  print('\n')
  plt.figure(figsize=(15, 10))
  plt.subplot(211)
  plt.text(500, -35, f'Spike rate = {spike_rate:.3f} (sp/s), Mean of Free Mem Pot = {np.mean(v_fmp):.3f}',
           fontsize=16, fontweight='bold', horizontalalignment='center',
           verticalalignment='bottom')
  plt.text(500, -38.5, f'CV ISI = {cv_isi:.3f}, STD of Free Mem Pot = {np.std(v_fmp):.3f}',
           fontsize=16, fontweight='bold', horizontalalignment='center',
           verticalalignment='bottom')

  plt.plot(pars['range_t'], v_fmp, 'r', lw=1.,
           label='Free mem. pot.', zorder=2)
  plt.plot(pars['range_t'], v, 'b', lw=1.,
           label='mem. pot with spk thr', zorder=1, alpha=0.7)
  plt.axhline(pars['V_th'], 0, 1, color='k', lw=1., ls='--',
              label='Spike Threshold', zorder=1)
  plt.axhline(np.mean(v_fmp),0, 1, color='r', lw=1., ls='--',
              label='Mean Free Mem. Pot.', zorder=1)
  plt.ylim(-76, -39)
  plt.xlabel('Time (ms)')
  plt.ylabel('V (mV)')
  plt.legend(loc=[1.02, 0.68])

  plt.subplot(223)
  plt.plot(pars['range_t'][::3], gE[::3], 'r', lw=1)
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$g_E$ (nS)')

  plt.subplot(224)
  plt.plot(pars['range_t'][::3], gI[::3], 'b', lw=1)
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$g_I$ (nS)')

  plt.tight_layout()
```

    interactive(children=(FloatSlider(value=10.0, description='exc_rate', layout=Layout(width='450px'), max=20.0, …

```python
# to_remove explanation

"""

When the mean FMP is above spike threshold, the fluctuations in the FMP are rather small,
and the neuron spikes in a fairly regular fashion. This regime, where the mean FMP
is above the spike threshold, is called **mean-driven regime**.

When the mean FMP is below the spike threshold, the fluctuations in the FMP are
large, and the neuron's spikes are driven by these fluctuations. As a consequence,
the neuron spikes in more  Poisson-like fashion. This regime, where the mean FMP is
below the spike threshold, and spikes are driven by the fluctuations, is called **fluctuation-driven regime**.

"""
```

### Think! 1.2: Excitatory and Inhibitory Balance

1. How much can you increase the spike pattern variability? Under what condition(s) might the neuron respond with Poisson-type spikes? Note that we injected Poisson-type spikes. (Think of the answer in terms of the ratio of the exc. and inh. input spike rates.)

2. Link to the balance of excitation and inhibition: one of the definitions of excitation and inhibition balance is that mean free membrane potential remains constant as excitatory and inhibitory input rates are increased. What do you think happens to the neuron firing rate as we change excitatory and inhibitory rates while keeping the neuron in balance?  See [Kuhn, Aertsen, and Rotter (2004)](https://www.jneurosci.org/content/jneuro/24/10/2345.full.pdf) for much more on this.

```python
# to_remove explanation

"""
Discussion:

1. We can push the neuron to spike almost like a Poisson neuron. Of course given
that there is a refractoriness it will never spike completely like a Poisson process.
Poisson type spike irregularity will be achieved when mean is small (far from the
spike threshold) and fluctuations are large. This will achieved when excitatory
and inhibitory rates are balanced -- i.e. ratio of exc and inh. spike rate is
constant as you vary the inout rate.

2. Firing rate will increase because fluctuations will increase as we increase
exc. and inh. rates. But if synapses are modelled as conductance as opposed to
currents, fluctuations may start decrease at high input rates because neuron time
constant will drop.

""";
```

---
# Section 2: Short-term synaptic plasticity

*Estimated timing to here from start of tutorial: 45 min*

Above, we modeled synapses with fixed weights. Now we will explore synapses whose weight change in some input conditions.

Short-term plasticity (STP) is a phenomenon in which synaptic efficacy changes over time in a way that reflects the history of presynaptic activity. Two types of STP, with opposite effects on synaptic efficacy, have been experimentally observed. They are known as Short-Term Depression (STD) and Short-Term Facilitation (STF).

The mathematical model (_for more information see [here](http://www.scholarpedia.org/article/Short-term_synaptic_plasticity)_) of STP is based on the concept of a limited pool of synaptic resources available for transmission ($R$), such as, for example, the overall amount of synaptic vesicles at the presynaptic terminals. The amount of presynaptic resource changes in a dynamic fashion depending on the recent history of spikes.

Following a presynaptic spike, (i) the fraction $u$ (release probability) of the available pool to be utilized increases due to spike-induced calcium influx to the presynaptic terminal, after which (ii) $u$ is consumed to increase the post-synaptic conductance. Between spikes, $u$ decays back to zero with time constant $\tau_f$ and $R$ recovers to 1 with time constant $\tau_d$. In summary, the dynamics of excitatory (subscript $E$) STP are given by:

\\

\begin{eqnarray}
&& \frac{du_E}{dt} &=& -\frac{u_E}{\tau_f} + U_0(1-u_E^-)\delta(t-t_{\rm sp}) \\[.5mm]
&& \frac{dR_E}{dt} &=& \frac{1-R_E}{\tau_d} - u_E^+ R_E^- \delta(t-t_{\rm sp}) \qquad (6) \\[.5mm]
&& \frac{dg_E(t)}{dt} &=& -\frac{g_E}{\tau_E} + \bar{g}_E u_E^+ R_E^- \delta(t-t_{\rm sp})
\end{eqnarray}

\\

where $U_0$ is a constant determining the increment of $u$ produced by a spike. $u_E^-$ and $R_E^-$ denote the corresponding values just before the spike arrives, whereas $u_E^+$ refers to the moment right after the spike. $\bar{g}_E$ denotes the maximum excitatory conductane, and $g_E(t)$ is calculated for all spiketimes $k$, and decays over time with a time constant $\tau_{E}$. Similarly, one can obtain the dynamics of inhibitory STP (i.e., by replacing the subscript $E$ with $I$).

The interplay between the dynamics of $u$ and $R$ determines whether the joint effect of $uR$ is dominated by *depression* or *facilitation*. In the parameter regime of $\tau_d \gg \tau_f$  and for large $U_0$, an initial spike incurs a large drop in $R$ that takes a long time to recover; therefore, the synapse is STD-dominated. In the regime of $\tau_d \ll \tau_f$ and for small $U_0$, the synaptic efficacy is increased gradually by spikes, and consequently, the synapse is STF-dominated. This phenomenological model successfully reproduces the kinetic dynamics of depressed and facilitated synapses observed in many cortical areas.

We provide a helper function below, `dynamic_syn` that integrates the above equations using the Euler method (which you saw yesterday).

```python
# @markdown Execute this cell to get helper function `dynamic_syn`

def dynamic_syn(g_bar, tau_syn, U0, tau_d, tau_f, pre_spike_train, dt):
  """
  Short-term synaptic plasticity

  Args:
    g_bar           : synaptic conductance strength
    tau_syn         : synaptic time constant [ms]
    U0              : synaptic release probability at rest
    tau_d           : synaptic depression time constant of x [ms]
    tau_f           : synaptic facilitation time constantr of u [ms]
    pre_spike_train : total spike train (number) input
                      from presynaptic neuron
    dt              : time step [ms]

  Returns:
    u               : usage of releasable neurotransmitter
    R               : fraction of synaptic neurotransmitter resources available
    g               : postsynaptic conductance

  """

  Lt = len(pre_spike_train)
  # Initialize
  u = np.zeros(Lt)
  R = np.zeros(Lt)
  R[0] = 1.
  g = np.zeros(Lt)

  # simulation
  for it in range(Lt - 1):
    # Compute du
    du = -(dt / tau_f) * u[it] + U0 * (1.0 - u[it]) * pre_spike_train[it + 1]
    u[it + 1] = u[it] + du
    # Compute dR
    dR = (dt / tau_d) * (1.0 - R[it]) - u[it + 1] * R[it] * pre_spike_train[it + 1]
    R[it + 1] = R[it] + dR
    # Compute dg
    dg = -(dt / tau_syn) * g[it] + g_bar * R[it] * u[it + 1] * pre_spike_train[it + 1]
    g[it + 1] = g[it] + dg

  return u, R, g

help(dynamic_syn)
```

    Help on function dynamic_syn in module __main__:

    dynamic_syn(g_bar, tau_syn, U0, tau_d, tau_f, pre_spike_train, dt)
        Short-term synaptic plasticity

        Args:
          g_bar           : synaptic conductance strength
          tau_syn         : synaptic time constant [ms]
          U0              : synaptic release probability at rest
          tau_d           : synaptic depression time constant of x [ms]
          tau_f           : synaptic facilitation time constantr of u [ms]
          pre_spike_train : total spike train (number) input
                            from presynaptic neuron
          dt              : time step [ms]

        Returns:
          u               : usage of releasable neurotransmitter
          R               : fraction of synaptic neurotransmitter resources available
          g               : postsynaptic conductance

## Section 2.1: Short-term syaptic depression (STD)

### Interactive Demo 2.1: STD Explorer with input rate
Below, an interactive demo that shows how Short-term synaptic depression (STD) changes for different firing rates of the presynaptic spike train and how the amplitude synaptic conductance $g$ changes with every incoming spike until it reaches its stationary state.

How does the firing rate affect the synaptic efficacy?
Does it matter if the neuron fires in a Poisson manner, rather than regularly?

**Note:** `Poi_or_Reg=1`: for *Posisson type* and `Poi_or_Reg=0`: for *regular* presynaptic spikes.

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

def my_STD_diff_rate(rate, Poi_or_Reg):
      _ = my_illus_STD(Poi_or_reg=Poi_or_Reg, rate=rate)

_ = widgets.interact(my_STD_diff_rate, rate=(10., 100.1, 5.),
                     Poi_or_Reg=(0, 1, 1))
```

    interactive(children=(FloatSlider(value=55.0, description='rate', max=100.1, min=10.0, step=5.0), IntSlider(va…

```python
# to_remove explanation

"""
Discussion:

Increasing the input rate, we decrease the synaptic efficacy, i.e., the synaptic
conductance decreases. This is the case for both Poisson or a regular spiking input.
In case of regular spiking, the synaptic conductance reaches a steady state. This
will not happen in the case of Poisson type spikes.
""";
```

Once, I asked an experimentalist about the experimental values of the PSP amplitude produced by a connection between two neocortical excitatory neurons. She asked: "At what frequency?" I was confused, but you will understand her question, now that you know that PSP amplitude depends on the spike history, and therefore on the spike rate of the presynaptic neuron.

Here, we will study how the ratio of the synaptic conductance corresponding to the first and 10th spikes change as a function of the presynaptic firing rate (experimentalists often take the ratio of first and second PSPs).

For computational efficiency, we assume that the presynaptic spikes are regular. This assumption means that we do not have to run multiple trials.

```python
# @markdown STD conductance ratio with different input rate
# Regular firing rate
input_rate = np.arange(5., 40.1, 5.)
g_1 = np.zeros(len(input_rate))  # record the the PSP at 1st spike
g_2 = np.zeros(len(input_rate))  # record the the PSP at 10th spike

for ii in range(len(input_rate)):
   g_1[ii], g_2[ii] = my_illus_STD(Poi_or_reg=False, rate=input_rate[ii],
                                   plot_out=False, U0=0.5, tau_d=100., tau_f=50)

plt.figure(figsize=(11, 4.5))

plt.subplot(121)
plt.plot(input_rate, g_1, 'm-o', label='1st Spike')
plt.plot(input_rate, g_2, 'c-o', label='10th Spike')
plt.xlabel('Rate [Hz]')
plt.ylabel('Conductance [nS]')
plt.legend()

plt.subplot(122)
plt.plot(input_rate, g_2 / g_1, 'b-o')
plt.xlabel('Rate [Hz]')
plt.ylabel(r'Conductance ratio $g_{10}/g_{1}$')
plt.tight_layout()
```

As we increase the input rate the ratio of the first to tenth spike is increased, because the tenth spike conductance becomes smaller. This is a clear evidence of synaptic depression, as using the same amount of current has a smaller effect on the neuron.

## Section 2.2: Short-term synaptic facilitation (STF)

### Interactive Demo 2.2: STF explorer with input rate
Below, we see an illustration of a short-term facilitation example. Take note of the change in the synaptic variables: `U_0`, `tau_d`, and  `tau_f`.

- for STD, `U0=0.5, tau_d=100., tau_f=50.`

- for STP, `U0=0.2, tau_d=100., tau_f=750.`

How does the synaptic conductance change as we change the input rate? What do you observe in the case of a regular input and a Poisson type one?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def my_STD_diff_rate(rate, Poi_or_Reg):
  _ = my_illus_STD(Poi_or_reg=Poi_or_Reg, rate=rate, U0=0.2, tau_d=100., tau_f=750.)

_ = widgets.interact(my_STD_diff_rate, rate=(4., 40.1, 2.),
                     Poi_or_Reg=(0, 1, 1))
```

    interactive(children=(FloatSlider(value=22.0, description='rate', max=40.1, min=4.0, step=2.0), IntSlider(valu…

### Think! 2.2: Short-term synaptic facilitation vs. depression

Here, we will study how the ratio of the synaptic conductance corresponding to the $1^{st}$ and $10^{th}$ spike changes as a function of the presynaptic rate.

Why does the ratio of the first and tenth spike conductance changes in a non-monotonic fashion for synapses with STF, even though it decreases monotonically for synapses with STD?

```python
# @markdown STF conductance ratio with different input rates
# Regular firing rate
input_rate = np.arange(2., 40.1, 2.)
g_1 = np.zeros(len(input_rate))  # record the the PSP at 1st spike
g_2 = np.zeros(len(input_rate))  # record the the PSP at 10th spike

for ii in range(len(input_rate)):
   g_1[ii], g_2[ii] = my_illus_STD(Poi_or_reg=False, rate=input_rate[ii],
                                   plot_out=False,
                                   U0=0.2, tau_d=100., tau_f=750.)

plt.figure(figsize=(11, 4.5))

plt.subplot(121)
plt.plot(input_rate, g_1, 'm-o', label='1st Spike')
plt.plot(input_rate, g_2, 'c-o', label='10th Spike')
plt.xlabel('Rate [Hz]')
plt.ylabel('Conductance [nS]')
plt.legend()

plt.subplot(122)
plt.plot(input_rate, g_2 / g_1, 'b-o',)
plt.xlabel('Rate [Hz]')
plt.ylabel(r'Conductance ratio $g_{10}/g_{1}$')
plt.tight_layout()
```

```python
# to_remove explanation

"""
Discussion:

Because we have a facilitatory synapses, as the input rate increases synaptic
resources released per spike also increase. Therefore, we expect that the synaptic
conductance will increase with input rate. However, total synaptic resources are
finite. And they recover in a finite time. Therefore, at high frequency inputs
synaptic resources are rapidly deleted at a higher rate than their recovery, so
after first few spikes, only a small number of synaptic resources are left. This
results in decrease in the steady-state synaptic conductance at high frequency inputs.
""";
```

---
# Summary

*Estimated timing of tutorial: 1 hour, 10 min*

Congratulations! You have just finished the last tutorial of this day. Here, we saw how to model conductance-based synapses and also how to incorporate short-term dynamics in synaptic weights.

We covered the:

- static synapses and how excitation and inhibition affect the neuronal output
- mean- or fluctuation-driven regimes
- short-term dynamics of synapses (both facilitation and depression)

Finally, we incorporated all the aforementioned tools to study how a change in presynaptic firing history affects the synaptic weights!

There are many interesting things that you can try on your own to develop a deeper understanding of biological synapses. A couple of those are mentioned below in the optional boxes -- if you have time.

If you have time, you can look at a conductance-based LIF model with STP in the bonus sections below. There is also a whole bonus tutorial on spike timing dependent synaptic plasticity!

---
# Bonus

---
## Bonus Section 1: Conductance-based LIF with STP

Previously, we looked only at how the presynaptic firing rate affects the presynaptic resource availability and thereby the synaptic conductance. It is straightforward to imagine that, while the synaptic conductances are changing, the output of the postsynaptic neuron will change as well.

So, let's put the STP on synapses impinging on an LIF neuron and see what happens.

```python
# @markdown Execute for helper function for conductance-based LIF neuron with STP-synapses

def run_LIF_cond_STP(pars, I_inj, pre_spike_train_ex, pre_spike_train_in):
  """
  conductance-based LIF dynamics

  Args:
    pars               : parameter dictionary
    I_inj              : injected current [pA]
                         The injected current here can be a value or an array
    pre_spike_train_ex : spike train input from presynaptic excitatory neuron (binary)
    pre_spike_train_in : spike train input from presynaptic inhibitory neuron (binary)

  Returns:
    rec_spikes : spike times
    rec_v      : mebrane potential
    gE         : postsynaptic excitatory conductance
    gI         : postsynaptic inhibitory conductance

  """

  # Retrieve parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, V_L = pars['V_init'], pars['E_L']
  gE_bar, gI_bar = pars['gE_bar'], pars['gI_bar']
  U0E, tau_dE, tau_fE = pars['U0_E'], pars['tau_d_E'], pars['tau_f_E']
  U0I, tau_dI, tau_fI = pars['U0_I'], pars['tau_d_I'], pars['tau_f_I']
  VE, VI = pars['VE'], pars['VI']
  tau_syn_E, tau_syn_I = pars['tau_syn_E'], pars['tau_syn_I']
  tref = pars['tref']

  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  nE = pre_spike_train_ex.shape[0]
  nI = pre_spike_train_in.shape[0]

  # compute conductance Excitatory synapses
  uE = np.zeros((nE, Lt))
  RE = np.zeros((nE, Lt))
  gE = np.zeros((nE, Lt))
  for ie in range(nE):
    u, R, g = dynamic_syn(gE_bar, tau_syn_E, U0E, tau_dE, tau_fE,
                          pre_spike_train_ex[ie, :], dt)

    uE[ie, :], RE[ie, :], gE[ie, :] = u, R, g

  gE_total = gE.sum(axis=0)

  # compute conductance Inhibitory synapses
  uI = np.zeros((nI, Lt))
  RI = np.zeros((nI, Lt))
  gI = np.zeros((nI, Lt))
  for ii in range(nI):
    u, R, g = dynamic_syn(gI_bar, tau_syn_I, U0I, tau_dI, tau_fI,
                          pre_spike_train_in[ii, :], dt)

    uI[ii, :], RI[ii, :], gI[ii, :] = u, R, g

  gI_total = gI.sum(axis=0)

  # Initialize
  v = np.zeros(Lt)
  v[0] = V_init
  Iinj = I_inj * np.ones(Lt)  # ensure I has length Lt

  # simulation
  rec_spikes = []  # recording spike times
  tr = 0.
  for it in range(Lt - 1):
    if tr > 0:
      v[it] = V_reset
      tr = tr - 1
    elif v[it] >= V_th:  # reset voltage and record spike event
      rec_spikes.append(it)
      v[it] = V_reset
      tr = tref / dt

    # calculate the increment of the membrane potential
    dv = (dt / tau_m) * (-(v[it] - V_L) \
                         - (gE_total[it + 1] / g_L) * (v[it] - VE) \
                         - (gI_total[it + 1] / g_L) * (v[it] - VI) + Iinj[it] / g_L)

    # update membrane potential
    v[it+1] = v[it] + dv

  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes, uE, RE, gE, RI, RI, gI

help(run_LIF_cond_STP)
```

### Bonus Interactive Demo: LIF with STP Explorer

Here we have assumed that both excitatory and inhibitory synapses show short-term depression. Change the nature of synapses and study how spike pattern variability changes.
In this demo, `tau_d = 500*tau_ratio (ms)` and  `tau_f = 300*tau_ratio (ms)`.

You should compare the output of this neuron with what you observed in the previous tutorial when synapses were assumed to be static.

_Note: it will take slightly longer time to run each case_

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def LIF_STP(tau_ratio):
  pars = default_pars(T=1000)
  pars['gE_bar'] = 1.2 * 4  # [nS]
  pars['VE'] = 0.           # [mV]
  pars['tau_syn_E'] = 5.    # [ms]
  pars['gI_bar'] = 1.6 * 4  # [nS]
  pars['VI'] = -80.         # [ms]
  pars['tau_syn_I'] = 10.   # [ms]

  # here we assume that both Exc and Inh synapses have synaptic depression
  pars['U0_E'] = 0.45
  pars['tau_d_E'] = 500. * tau_ratio  # [ms]
  pars['tau_f_E'] = 300. * tau_ratio  # [ms]

  pars['U0_I'] = 0.45
  pars['tau_d_I'] = 500. * tau_ratio  # [ms]
  pars['tau_f_I'] = 300. * tau_ratio  # [ms]

  pre_spike_train_ex = Poisson_generator(pars, rate=15, n=80)
  pre_spike_train_in = Poisson_generator(pars, rate=15, n=20)  # 4:1

  v, rec_spikes, uE, RE, gE, uI, RI, gI = run_LIF_cond_STP(pars, 0,
                                                           pre_spike_train_ex,
                                                           pre_spike_train_in)

  t_plot_range = pars['range_t'] > 200

  plt.figure(figsize=(11, 7))
  plt.subplot(211)
  plot_volt_trace(pars, v, rec_spikes)

  plt.subplot(223)
  plt.plot(pars['range_t'][t_plot_range], gE.sum(axis=0)[t_plot_range], 'r')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$g_E$ (nS)')

  plt.subplot(224)
  plt.plot(pars['range_t'][t_plot_range], gI.sum(axis=0)[t_plot_range], 'b')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$g_I$ (nS)')

  plt.tight_layout()

_ = widgets.interact(LIF_STP, tau_ratio=(0.2, 1.1, 0.2))
```

When we vary the tau_ratio we are increasing `tau_f` and `tau_d` i.e. by increasing `tau_ratio` we are increasing the synaptic depression. The effect is same on both Exc and Inh conductances.
This is visible as a clear decrease in the firing rate of the neuron from 300-400ms onwards.

Not much happens in the beginning because synaptic depression takes some time to become visible.

It is curious that while both excitatory and inhibitory conductances have depressed but output firing rate has still decreased.

There are two explanations of this:
1. excitation has depressed more than the inhibition from their starting values.
2. because synaptic conductances have depressed, membrane fluctuation size has decreased.

Which is the more likely reason?

Will the neuron show higher irregularity if the synapses have STP?

Try calculating the $CV_{\rm ISI}$ for different `tau_ratio` after simulating the LIF neuron with STP (Hint:`run_LIF_cond_STP` will help you understand the irregularity).

**Functional implications of short-term dynamics of synapses**

As you have seen above, if the firing rate is stationary, the synaptic conductance quickly reaches a fixed point. On the other hand, if the firing rate transiently changes, synaptic conductance will vary -- even if the change is as short as a single inter-spike-interval. Such small changes can be observed in a single neuron when input spikes are regular and periodic. If the input spikes are Poissonian, then one may have to perform an average over several neurons.

_Come up with other functions that short-term dynamics of synapses can be used to implement and implement them._


# Assembling DMI simulations

**Author**: Marijan Beg

**Date**: Dec 2012

**Purpose**: Illustrating how to set up a simulation including the DMI term in Finmag.

## Importing all required modules

The first step in creating the skyrmion simulation is importing all required modules.

Module **dolfin** contains some basic finite elements methods. Subsequently, it will be used for creating some basic meshes and their plotting. To import dolfin module type:

```python
import dolfin as df
```

The energy terms, required for skyrmion formation are:

* Exchange,
* DMI and
* Zeeman.

Therefore these energies must be imported into the simulation. All energy terms that finmag is capable of simulating are in **finmag.energies** module. Three crucial energy terms are imported using the following command.

```python
from finmag.energies import Exchange, DMI, Zeeman
```

    [2014-09-12 14:18:45] INFO: Finmag logging output will be appended to file: '/home/mb4e10/.finmag/global.log'
    [2014-09-12 14:18:45] DEBUG: Building modules in 'native'...
    [2014-09-12 14:18:50] DEBUG: FinMag          5430:23de5abc150f15c6eeda90e6bfc375599a63c112
    [2014-09-12 14:18:50] DEBUG: Dolfin          1.4.0                Matplotlib      1.3.1
    [2014-09-12 14:18:50] DEBUG: Numpy           1.8.1                Scipy           0.14.0
    [2014-09-12 14:18:50] DEBUG: IPython         2.1.0                Python          2.7.6
    [2014-09-12 14:18:51] DEBUG: Paraview        4.0.1-1ubuntu1       Sundials        2.5.0
    [2014-09-12 14:18:51] DEBUG: Boost-Python    <unknown>            Linux           Ubuntu 14.04.1 LTS
    [2014-09-12 14:18:51] DEBUG: Registering debug signal handler. Press Ctrl-Z at any time to stop execution and jump into the debugger.

Other energy terms, such as **Anisotropy** and **Demag** can be added in the same way.

The next class which should be imported is the **Simulation** class as a part of finmag module.

```python
from finmag import Simulation as Sim
```

Additional modules should be also added in this section if they are going to be used subsequently.

## Creating a mesh

Before the simulation can be created, a suitable mesh which coresponds to the simulated sample must be created.
In the skyrmion study most used domain is thin film sample which can be imagined as a three-dimensional box with $x$ and $y$ dimensions significantly larger than the film thickess $z$. The command for creating a 3D mesh object which coresponds to a thin film is **Box** as a part of **dolfin** module (previously imported as **df**). The required arguments for this operation are:

* The coordinates of the origin. In this case origin $(x_{0},y_{0},z_{0})=(0,0,0)$ is chosen.
* Dimensions of a three dimensional box $x$, $y$ and $z$.
* Number of mesh vertices along each coordinate ($n_\mathrm{x}$, $n_\mathrm{y}$ and $n_\mathrm{z}$).

Therefore, mesh, using the **Box** command is created with the following list of arguments: ($x_{0}$, $y_{0}$, $z_{0}$, $x$, $y$, $z$, $n_\mathrm{x}$, $n_\mathrm{y}$, $n_\mathrm{z}$)

The following command creates the mesh for thin film sample with dimensions: $x = 50$ nm, $y = 50$ nm and thickess $z = 1$ nm. Number of vertices is chosen such that their spacing is $1$ nm, thus, $n_\mathrm{x}=50$, $n_\mathrm{y}=50$ and $n_\mathrm{z}=1$.

```python
mesh = df.BoxMesh(0, 0, 0, 50e-9, 50e-9, 1e-9, 50, 50, 1)
```

The information about a created mesh can be displayed by using the **print_mesh_info** method which is a part of **finmag.util.meshes** module. After this module is imported, **mesh_info** command with a mesh object as an argument can be run.

```python
from finmag.util.meshes import print_mesh_info
print_mesh_info(mesh)
```

    ===== Mesh info: ==============================
     15000 cells (= volume elements)
     35200 facets
     10400 surface facets
     24800 interior facets
     25401 edges
      5202 vertices

    ===== Distribution of edge lengths: ===========
    0.000
    0.000 **********************************************************************
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000 *******************************************************
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000
    0.000 **************

Alternatively, mesh can be visualised by using the command **plot_mesh** which is also a part of **finmag.util.helpers** module, but unstable Matplotlib 1.3.X version must be installed. Alternatively, mesh can be visualised by dolfin's **plot** method in the following way.

```python
# IPYTHON_TEST_IGNORE_OUTPUT
from finmag.util.meshes import plot_mesh_with_paraview
plot_mesh_with_paraview(mesh, use_display=0)
```

## Magnetisation saturation parameter

The next parameter which should be defined, before the simulation object is created, is magnetisation saturation parameter $M_\mathrm{S}$. For example $M_\mathrm{S}$ with value $8\cdot10^{5}$ A/m is defined with the following command:

```python
Ms = 8e5
```

## Creation of simulation object

After mesh and $M_\mathrm{S}$ parameters are defined, **Simulation** class instance can be created. It requires two arguments:

* mesh and
* $M_\mathrm{S}$

**Simulation** class is imported from the finmag module as **Sim** and therefore its instance is created using the following command:

```python
sim1 = Sim(mesh, Ms, name='skyrmion1')
```

    [2014-09-12 14:21:13] INFO: Finmag logging output will be written to file: '/home/mb4e10/finmag/doc/ipython_notebooks_src/skyrmion1.log' (any old content will be overwritten).
    [2014-09-12 14:21:13] DEBUG: Creating DataWriter for file 'skyrmion1.ndt'
    [2014-09-12 14:21:13] INFO: Creating Sim object 'skyrmion1' (rank=0/1).
    [2014-09-12 14:21:13] INFO: <Mesh of topological dimension 3 (tetrahedra) with 5202 vertices and 15000 cells, ordered>
    [2014-09-12 14:21:13] DEBUG: Creating LLG object.

The next thing which should be done is to set the initial state of magnetisation in the sample. The initial magnetisation state is defined by using the **set_m** method as a part of **Simulation** class. The required argument is a tuple of three components $(m_\mathrm{x},m_\mathrm{y},m_\mathrm{z})$. The initial magnetisation state which points towards $+x$ direction is set in the following way:

```python
sim1.set_m((1, 0, 0))
```

The current state of the magnetisation can be visualised by saving a VTK file (in `.pvd` format) and opening it in the Paraview or Mayavi software. However, Finmag also supports interactive plotting with Paraview, which is what we use here (via the `sim1.render_scene` method).

```python
# IPYTHON_TEST_IGNORE_OUTPUT
sim1.render_scene(glyph_max_number_of_points=3000, glyph_scale_factor=3, use_display=0)
```

## Adding energy terms into the simulation

After the simulation object is created, required energy terms for the skyrmion formation must be added into the simulation.
Exchange energy term

$$
E_\mathrm{ex} = A\int_{\Omega}(\nabla \vec{M})^{2} \mathrm{d}V
$$

which is uniquely characterised by exchange energy constant $A$ can be created by using already imported **Exchange** class, where exchange energy parameter is added as an argument. If $A=1\cdot10^{-11} \mathrm{J/m}$ then the exchange object is created in the following way:

```python
A = 1e-11
exchange = Exchange(A)
```

    [2014-09-12 14:21:54] DEBUG: Creating Exchange object with method box-matrix-petsc,  in Jacobian.

Similarly, DMI energy term
\begin{equation}
E_{dmi}=D\int_{\Omega}\vec{M}\cdot(\nabla\times\vec{M})\mathrm{d}V
\end{equation}
which is characterised by Dzyaloshinskii-Moriya energy parameter can be created by using the **DMI** method which taked the value of $D$ as an argument. According to equation:
\begin{equation}
L_\mathrm{D}=\frac{4\pi A}{|D|}
\end{equation}
where $L_{D}$ is the characteristic length of spiral modulation, parameter $D$ can be estimated. For instance, in the case of MnSi, the $L_{D}$ parameter is $18$ nm and accordingly DMI energy parameter is: $$D = 6.98\cdot10^{-3} \mathrm{J/m^{2}}$$

```python
D = 6.98e-3
dmi = DMI(D)
```

    [2014-09-12 14:21:59] DEBUG: Creating DMI object with method box-matrix-petsc,  in Jacobian.

The final energy term which must be included into the simulation is Zeeman energy:
\begin{equation}
E_\mathrm{H}=-\mu_{0}\int_{\Omega}\vec{M}\cdot\vec{H}\mathrm{d}V
\end{equation}
which requires the value of $\vec{H}$ as a list of three external field components $[H_\mathrm{x},H_\mathrm{y},H_\mathrm{z}]$. If the external field is applied only in z direction with $H_\mathrm{z}=1.6\cdot10^{6} \mathrm{A/m}$, the zeeman object is created:

```python
H = [0, 0, 1600000]
zeeman = Zeeman(H)
```

After energy objects are created, they can be added into the simulator by using the **add** method:

```python
sim1.add(exchange)
sim1.add(dmi)
sim1.add(zeeman)
```

    [2014-09-12 14:22:05] DEBUG: Adding interaction Exchange to simulation.
    [2014-09-12 14:22:05] DEBUG: Adding interaction DMI to simulation.
    [2014-09-12 14:22:05] DEBUG: Adding interaction Zeeman to simulation.

We can ask the simulation object what energies it knows about:

```python
sim1.get_interaction_list()
```

    ['DMI', 'Exchange', 'Zeeman']

## Running the simulation and final state visualisation
After the whole simulation object is created, simulation can be executed. The simulation is executed by using the **run_until** command. This command takes the final time of simulation $t_{final}$ as an argument. If the final simulation time is $0.3 ns$, time integration is executed by typing (This make take a few minutes to execute.):

```python
sim1.run_until(0.3e-9)
```

    [2014-09-12 14:22:10] INFO: Create integrator sundials with kwargs={}
    [2014-09-12 14:22:10] DEBUG: Creating integrator with backend sundials.
    [2014-09-12 14:22:10] DEBUG: Updating get method for steps in TableWriter(name=skyrmion1.ndt)
    [2014-09-12 14:22:10] DEBUG: Updating get method for last_step_dt in TableWriter(name=skyrmion1.ndt)
    [2014-09-12 14:22:10] DEBUG: Updating get method for dmdt in TableWriter(name=skyrmion1.ndt)
    [2014-09-12 14:22:10] INFO: Simulation will run until t = 3e-10 s.
    [2014-09-12 14:23:15] INFO: Simulation has reached time t = 3e-10 s.

After the simulation is finished, the state of an object can be visualised as before. It can be seen that a single skyrmion is formed in the sample.

```python
# IPYTHON_TEST_IGNORE_OUTPUT
sim1.render_scene(glyph_max_number_of_points=3000, glyph_scale_factor=4, use_display=0)
```


The **Continuous Bag-of-Words model (CBOW)** is frequently used in NLP deep
learning. It is a model that tries to predict words given the context of
a few words before and a few words after the target word. This is
distinct from language modeling, since CBOW is not sequential and does
not have to be probabilistic. Typcially, CBOW is used to quickly train
word embeddings, and these embeddings are used to initialize the
embeddings of some more complicated model. Usually, this is referred to
as *pretraining embeddings*. It almost always helps performance a couple
of percent.

The CBOW model is as follows. Given a target word $w_i$ and an
$N$ context window on each side, $w_{i-1}, \dots, w_{i-N}$
and $w_{i+1}, \dots, w_{i+N}$, referring to all context words
collectively as $C$, CBOW tries to minimize

\begin{align}-\log p(w_i | C) = -\log \text{Softmax}(A(\sum_{w \in C} q_w) + b)\end{align}

where $q_w$ is the embedding for word $w$.

In this notebook, I am going to use `tensorflow 2.0.0-beta0` and take the advantage of its flexible custom model definitions to create a COBW model and will train it to get the embeddings of a toy dataset.

```python
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import *
import numpy as np
from tqdm import tqdm

print(tf.__version__)
```

    2.0.0-beta0

```python
# 2 words to the left, 2 to the right
CONTEXT_SIZE = 2
# Embedding dimension on left
EMBED_LEFT = 10
# Embedding dimension on right
EMBED_RIGHT = 10
```

```python
# Row test from Shakespeare Sonnet 2
raw_text = """When forty winters shall besiege thy brow,
And dig deep trenches in thy beauty's field,
Thy youth's proud livery so gazed on now,
Will be a totter'd weed of small worth held:
Then being asked, where all thy beauty lies,
Where all the treasure of thy lusty days;
To say, within thine own deep sunken eyes,
Were an all-eating shame, and thriftless praise.
How much more praise deserv'd thy beauty's use,
If thou couldst answer 'This fair child of mine
Shall sum my count, and make my old excuse,'
Proving his beauty by succession thine!
This were to be new made when thou art old,
And see thy blood warm when thou feel'st it cold.""".split()

# By deriving a set from `raw_text`, we deduplicate the array
vocab = set(raw_text)
vocab_size = len(vocab)

word_to_ix = {word: i for i, word in enumerate(vocab)}
data = []
for i in range(2, len(raw_text) - 2):
    context = [raw_text[i - 2], raw_text[i - 1],
               raw_text[i + 1], raw_text[i + 2]]
    target = raw_text[i]
    data.append((context, target))
print(data[:5])
```

    [(['When', 'forty', 'shall', 'besiege'], 'winters'), (['forty', 'winters', 'besiege', 'thy'], 'shall'), (['winters', 'shall', 'thy', 'brow,'], 'besiege'), (['shall', 'besiege', 'brow,', 'And'], 'thy'), (['besiege', 'thy', 'And', 'dig'], 'brow,')]

The toy dataset is ready. Let's now specify the model definition.

```python
class CBOW(Model):

    def __init__(self, vocab_size, embedding_dim_left, embedding_dim_right, context_size):
        super(CBOW, self).__init__()
        self.embeddings_left = Embedding(vocab_size, embedding_dim_left)
        self.embeddings_right = Embedding(vocab_size, embedding_dim_right)
        self.linear1 = Dense(128, activation='linear', input_dim=(context_size * embedding_dim_left))
        self.linear2 = Dense(vocab_size, activation='linear', input_dim=128)

    def call(self, inputs):
        left_embeds = tf.reshape(self.embeddings_left(inputs[:2]), [1,20])
        right_embeds = tf.reshape(self.embeddings_left(inputs[:2]), [1,20])
        left_out = tf.keras.activations.relu(self.linear1(left_embeds))
        right_out = tf.keras.activations.relu(self.linear1(right_embeds))
        out = self.linear2(left_out + right_out)
        return out
```

Let's instantiate the model.

```python
model = CBOW(vocab_size, EMBED_LEFT, EMBED_RIGHT, CONTEXT_SIZE)
```

Define our loss function and optimizer.

```python
loss_func = tf.keras.losses.SparseCategoricalCrossentropy()
optimizer = tf.keras.optimizers.SGD(learning_rate=0.001)
```

The function which will train our model. We use `GradientTape` to make use of automatic differentiation. Makes the process of Backpropagation much more flexible. Makes it also easy to chart dependencies inside the training loop.

```python
@tf.function
def model_train(features, labels):
    # Define the GradientTape context
    with tf.GradientTape() as tape:
        # Get the probabilities
        predictions = model(features)
        # Calculate the loss
        loss = loss_func(labels, predictions)
    # Get the gradients
    gradients = tape.gradient(loss, model.trainable_variables)
    # Update the weights
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    return loss
```

Finally we train the model for 10 epochs.

```python
running_loss = []
for epoch in tqdm(range(10)):
    total_loss = 0
    for context, target in data:
        context_idxs = np.array([word_to_ix[w] for w in context])
        target_idxs = np.array([word_to_ix[target]])
        loss = model_train(context_idxs, target_idxs)
        total_loss += loss
    running_loss.append(total_loss)
print(running_loss)
```

    100%|██████████| 10/10 [00:01<00:00,  8.50it/s]

    [<tf.Tensor: id=66937, shape=(), dtype=float32, numpy=1550.9991>, <tf.Tensor: id=67382, shape=(), dtype=float32, numpy=1634.5785>, <tf.Tensor: id=67827, shape=(), dtype=float32, numpy=1486.1724>, <tf.Tensor: id=68272, shape=(), dtype=float32, numpy=1476.1912>, <tf.Tensor: id=68717, shape=(), dtype=float32, numpy=1470.2799>, <tf.Tensor: id=69162, shape=(), dtype=float32, numpy=1466.611>, <tf.Tensor: id=69607, shape=(), dtype=float32, numpy=1464.6365>, <tf.Tensor: id=70052, shape=(), dtype=float32, numpy=1462.1602>, <tf.Tensor: id=70497, shape=(), dtype=float32, numpy=1458.745>, <tf.Tensor: id=70942, shape=(), dtype=float32, numpy=1455.2542>]

The loss is pretty high but it does decrease anyway. The purpose of this notebook is to show how we can combine the low level TF ops with the high-level APIs to aid utmost flexibility in model definition. The example is inspired from [this tutorial](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html).


```python
import numpy as np
from bokeh.plotting import figure, show
from bokeh.io import output_notebook
from bokeh.models import LinearColorMapper, BasicTicker, ColorBar
from bokeh.palettes import Category10
output_notebook()
```

    Loading BokehJS ...

# Introduction to Gaussian Processes - Part I
Author: [@alex_bridgland](https://twitter.com/alex_bridgland) | [@bridgo](https://github.com/Bridgo)

Blog post: http://bridg.land/posts/gaussian-processes-1
***
Gaussian processes may not be at the center of current machine learning hype but are still used at the forefront of research -- they were recently seen automatically tuning the MCTS hyperparameters for AlphaGo Zero for instance. They manage to be very easy to use while providing rich modeling capacity and uncertainty estimates.

However they can be pretty hard to grasp, especially if you're used to the type of models we see a lot of in deep learning. So hopefully this guide can fix that! It assumes a fairly minimal ML background and I aimed for a more visual & intuitive introduction without totally abandoning the theory. To get the most out of it I recommend downloading the notebook and experimenting with all the code!

## What is a Gaussian Process and why would I use one?
***
A Gaussian process (GP) is a powerful model that can be used to represent a distribution over functions. Most modern techniques in machine learning tend to avoid this by parameterising functions and then modeling these parameters (e.g. the weights in linear regression). However GPs are nonparametric models that model the function directly. This comes with one hugely important benefit: not only can we model any black-box function, __we can also model our uncertainty__. Quantifying uncertainty can be extremely valuable - for example if we are allowed to explore (request more data) we can chose to explore the areas we are least certain about to be as efficient as possible. This is the main idea behind Bayesian optimisation. For more information on the importance of uncertainty modeling see [this article](http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html) by Yarin Gal.

> If you give me several pictures of cats and dogs – and then you ask me to classify a new cat photo – I should return a prediction with rather high confidence. But if you give me a photo of an ostrich and force my hand to decide if it's a cat or a dog – I better return a prediction with very low confidence.
>
> Yarin Gal

For this introduction we will consider a simple regression setup without noise (but GPs can be extended to multiple dimensions and noisy data):
- We assume there is some hidden function $f:\mathbb{R}\rightarrow\mathbb{R}$ that we want to model.
- We have data $\mathbf{x}=[x_ 1, \ldots, x_N]^T, \mathbf{y}=[y_ 1, \ldots, y_N]^T$ where $y_i = f(x_i)$.
- We want to predict the value of $f$ at some new, unobserved points $\mathbf{x}_*$.

## Modeling Functions using Gaussians
***
The key idea behind GPs is that a function can be modeled using an infinite dimensional multivariate Gaussian distribution. In other words, every point in the input space is associated with a random variable and the joint distribution of these is modeled as a multivariate Gaussian.

Ok, so what does that mean and what does it actually look like? Well lets start with a simpler case: a unit 2D Gaussian. How can we start to view this as a distribution over functions? Here's what we have:

$$ \begin{pmatrix} y_0 \\ y_1 \end{pmatrix} \sim\mathcal{N}\left(\begin{pmatrix} 0\\ 0 \end{pmatrix}, \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\right) $$

Normally this is visualised as a 3D bell curve with the probability density represented as height. But what if, instead of visualising the whole distribution, we just sample from the distribution. Then we will have two values which we can plot on an graph. Let's do this 10 times, putting the first value at $x=0$ and the second at $x=1$ and then drawing a line between them.

```python
def plot_unit_gaussian_samples(D):
    p = figure(plot_width=800, plot_height=500, title='Samples from a unit {}D Gaussian'.format(D))

    xs = np.linspace(0, 1, D)
    for color in Category10[10]:
        ys = np.random.multivariate_normal(np.zeros(D), np.eye(D))
        p.line(xs, ys, line_width=1, color=color)
    return p
```

```python
show(plot_unit_gaussian_samples(2))
```

Looking at all these lines on a graph, it starts to look like we've just sampled 10 linear functions... What if we now use a 20-dimensional Gaussian, joining each of the sampled points in order?

```python
show(plot_unit_gaussian_samples(20))
```

These definitely look like functions of sorts but they look far too noisy to be useful for our purposes. Let's think a bit more about what we want from these samples and how we might change the distribution to get better looking samples...

The multivariate Gaussian has two parameters, its mean and covariance matrix. If we changed the mean then we would only change the overall trend (i.e. if the mean was ascending integers, e.g. `np.arange(D)` then the samples would have an overall positive linear trend) but there would still be that jagged noisy shape. For this reason we tend to leave the GP mean as zero - they actually turn out to be powerful enough to model many functions without changing this.

Instead we want some notion of _smoothness_: i.e. if two input points are close to each other then we expect the value of the function at those points to be similar. In terms of our model: random variables corresponding to nearby points should have similar values when sampled under their joint distribution (i.e. high _covariance_).

The covariance of these points is defined in the covariance matrix of the Gaussian. Suppose we have an $N$ dimensional Gaussian modeling $y_0,\ldots, y_N$, then the covariance matrix $\Sigma$ is $N\times N$ and its $(i, j)$-th element is $\Sigma_{ij}=\text{cov}(y_i, y_j)$. In other words $\Sigma$ is symmetric and stores the pairwise covariances of all the jointly modeled random variables.

## Smoothing with Kernels
***

So how should we define our covariance function? This is where the vast literature on kernels comes in handy. For our purposes we will choose a __squared exponential kernel__ which (in its simplest form) is defined by:

$$ \kappa(x, x')=\exp\left(-~\frac{(x-x')^2}{2}\right)$$

This function (which we plot in a moment) is 1 when $x=x'$ and tends to zero as its arguments drift apart.

```python
def k(xs, ys, sigma=1, l=1):
    """Squared Exponential kernel as above but designed to return the whole
    covariance matrix - i.e. the pairwise covariance of the vectors xs & ys.
    Also with two parameters which are discussed at the end."""

    # Pairwise difference matrix.
    dx = np.expand_dims(xs, 1) - np.expand_dims(ys, 0)
    return (sigma ** 2) * np.exp(-((dx / l) ** 2) / 2)

def m(x):
    """The mean function. As discussed, we can let the mean always be zero."""
    return np.zeros_like(x)
```

We can plot this kernel to show how it's maximised when $x=x'$ and then smoothly falls off as the two inputs start to differ.

```python
N = 100
x = np.linspace(-2, 2, N)
y = np.linspace(-2, 2, N)
d = k(x, y)

color_mapper = LinearColorMapper(palette="Plasma256", low=0, high=1)

p = figure(plot_width=400, plot_height=400, x_range=(-2, 2), y_range=(-2, 2),
           title='Visualisation of k(x, x\')', x_axis_label='x', y_axis_label='x\'', toolbar_location=None)
p.image(image=[d], color_mapper=color_mapper, x=-2, y=-2, dw=4, dh=4)

color_bar = ColorBar(color_mapper=color_mapper, ticker=BasicTicker(),
                     label_standoff=12, border_line_color=None, location=(0,0))

p.add_layout(color_bar, 'right')

show(p)
```

So, to get the sort of smoothness we want we will consider two random variables $y_i$ and $y_j$ plotted at $x_i$ and $x_j$ to have covariance $\text{cov}(y_i, y_j)=\kappa(x_i, x_j)$ - _the closer they are together the higher their covariance_.

Using the kernel function from above we can get this matrix with `k(xs, xs)`. Now lets try plotting another 10 samples from the 20D Gaussian, but this time with the new covariance matrix. When we do this we get:

```python
D = 20
xs = np.linspace(0, 1, D)
```

```python
ys = np.random.multivariate_normal(m(xs), k(xs, xs))
```

```python
p = figure(plot_width=800, plot_height=500)
for color in Category10[10]:
    ys = np.random.multivariate_normal(m(xs), k(xs, xs))
    p.circle(xs, ys, size=3, color=color)
    p.line(xs, ys, line_width=1, color=color)
show(p)
```

Now we have something thats starting to look like a distribution over (useful) functions! And we can see how as the number of dimensions tends to infinity we don't have to connect points any more because we will have a point for every possible choice of input.

Let's use more dimensions and see what it looks like across a bigger range of inputs:

```python
n = 100
xs = np.linspace(-5, 5, n)
K = k(xs, xs, sigma=1, l=1)
mu = m(xs)
```

```python
p = figure(plot_width=800, plot_height=500)

for color in Category10[5]:
    ys = np.random.multivariate_normal(mu, K)
    p.line(xs, ys, line_width=2, color=color)
show(p)
```

## Making Predictions using the Prior & Observations
***
Now that we have a distribution over functions, how can we use training data to model a hidden function so that we can make predictions?

First of all we need some training data! And to get that we are going to create our secret function $f$.

### The Target Function $f$
For this intro we'll use a 5th order polynomial:

$$f(x)=0.03 x^5 + 0.2 x^4 - 0.1 x^3 - 2.4 x^2 - 2.5 x + 6$$

I chose this because it has a nice wiggly graph but we could have chosen anything (try changing `coefs` below, or `f(x)` entirely, to model different functions).

```python
# coefs[i] is the coefficient of x^i
coefs = [6, -2.5, -2.4, -0.1, 0.2, 0.03]

def f(x):
    total = 0
    for exp, coef in enumerate(coefs):
        total += coef * (x ** exp)
    return total
```

```python
xs = np.linspace(-5.0, 3.5, 100)
ys = f(xs)
```

```python
p = figure(plot_width=800, plot_height=400, x_axis_label='x', y_axis_label='f(x)',
           title='The hidden function f(x)')
p.line(xs, ys, line_width=2)
show(p)
```

### Getting into the Maths

Now we get to the heart of GPs. Theres a bit more maths required but it only consists of consolidating what we have so far and using one trick to condition our joint distribution on observed data.

So far we have a way to model $p(\mathbf{y}\vert\mathbf{x})$ using a multivariate normal:

$$p(\mathbf{y}\vert\mathbf{x})=\mathcal{N}(\mathbf{y}\vert m(\mathbf{x}),\mathbf{K})$$

where $\mathbf{K}=\kappa(\mathbf{x}, \mathbf{x})$ and $m(\mathbf{x})=\mathbf{0}$.

This is a prior distribution representing the kind out outputs $\mathbf{y}$ that we expect to see over some inputs $\mathbf{x}$ _before_ we observe any data.

So we have some training data with inputs $\mathbf{x}$, and outputs $\mathbf{y}=f(\mathbf{x})$. Now lets say we have some new points $\mathbf{x}_*$ where we want to predict $\mathbf{y}_*=f(\mathbf{x}_*)$.

```python
x_obs = np.array([-4, -1.5, 0, 1.5, 2.5, 2.7])
y_obs = f(x_obs)

x_s = np.linspace(-8, 7, 80)
```

Now recalling the definition of a GP, we will model the joint distribution of all of $\mathbf{y}$ and $\mathbf{y}_*$ as:

$$ \begin{pmatrix}\mathbf{y} \\ \mathbf{y}_*\end{pmatrix} \sim\mathcal{N}\left(\begin{pmatrix}m(\mathbf{x})\\ m(\mathbf{x}_*)\end{pmatrix}, \begin{pmatrix}\mathbf{K} & \mathbf{K}_* \\ \mathbf{K}_*^T & \mathbf{K}_{**}\end{pmatrix}\right) $$

where $\mathbf{K}=\kappa(\mathbf{x}, \mathbf{x})$, $\mathbf{K}_* = \kappa(\mathbf{x}, \mathbf{x}_*)$ and $\mathbf{K}_{**}=\kappa(\mathbf{x}_*, \mathbf{x}_*)$. As before we are going to stick with a zero mean.

However this is modeling $p(\mathbf{y}, \mathbf{y}_*\vert \mathbf{x}, \mathbf{x}_*)$ and we only want a distribution over $\mathbf{y}_*$!

### Conditioning Multivariate Gaussians

Rather than deriving it from scratch we can just make use of [this standard result](https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Conditional_distributions). If we have a joint distribution over $\mathbf{y}$ and $\mathbf{y}_*$ as above, and we want to condition on the data we have for $\mathbf{y}$ then we have the following:

$$\begin{align}
p(\mathbf{y}_*\vert \mathbf{x}_*, \mathbf{x}, \mathbf{y})&=\mathcal{N}(\mathbf{y}_*\vert \mu_*, \Sigma_*)\\
\mu_*&=m(\mathbf{x}_*)+\mathbf{K}_*^T\mathbf{K}^{-1}(\mathbf{y}-m(\mathbf{x}))\\
\Sigma_*&=\mathbf{K}_{**}-\mathbf{K}_*^T\mathbf{K}^{-1}\mathbf{K}_*
\end{align}$$

Now we have a posterior distribution over $\mathbf{y}_*$ using a prior distribution and some observations!

_NB: The code below would not be used in practice since \\( \mathbf{K} \\) can often be poorly conditioned, so its inverse might be inaccurate. A better approach is covered in part II of this guide!_

**Note:** The $K_*$ terms are not necessarily square matrices.

```python
K = k(x_obs, x_obs)
K_s = k(x_obs, x_s)
K_ss = k(x_s, x_s)

K_sTKinv = np.matmul(K_s.T, np.linalg.pinv(K))

mu_s = m(x_s) + np.matmul(K_sTKinv, y_obs - m(x_obs))
Sigma_s = K_ss - np.matmul(K_sTKinv, K_s)
```

And that's it! We can now use these two parameters to draw samples from the conditional distribution. Here we plot them against the true function $f(x)$ (the dashed black line). Since we are using a GP we also have uncertainty information in the form of the variance of each random variable. We know the variance of the $i$-th will be ${\Sigma_*}_{ii}$ - in other words the variances are just the diagonal elements of $\Sigma_*$. Here we plot the samples with an uncertainty of $\pm 2$ standard deviations.

```python
p = figure(plot_width=800, plot_height=600, y_range=(-7, 8))

y_true = f(x_s)
p.line(x_s, y_true, line_width=3, color='black', alpha=0.4, line_dash='dashed', legend_label='True f(x)')

p.cross(x_obs, y_obs, size=20, legend_label='Training data')

stds = np.sqrt(Sigma_s.diagonal())
err_xs = np.concatenate((x_s, np.flip(x_s, 0))) # concatenating x_s with x_s reversed...
err_ys = np.concatenate((mu_s + 2 * stds, np.flip(mu_s - 2 * stds, 0)))
p.patch(err_xs, err_ys, alpha=0.2, line_width=0, color='grey', legend_label='Uncertainty')

count = 0
for color in Category10[3]:
    count+=1
    y_s = np.random.multivariate_normal(mu_s, Sigma_s)
    p.line(x_s, y_s, line_width=1, color=color, legend_label=f"sample {count}")

p.line(x_s, mu_s, line_width=3, color='blue', alpha=0.4, legend_label='Mean')
show(p)
```

## What next? - GP Regression and Noisy Data
***
In practice we need to do a bit more work to get good predictions. You may have noticed that the kernel implementation contained two parameters $\sigma$ and $l$. If you try changing those when sampling from the prior then you can see how $\sigma$ changes the vertical variation and $l$ changes the horizontal scale. So we would need to change these to reflect our prior belief about the hidden function $f$. For instance if we expect $f$ to have a much bigger range of outputs (for the domain we are interested in) then we would need to scale up $\sigma$ accordingly (try scaling the return value of $f$ by 100 to see what happens, then set `sigma=100`). In fact, as with anything that uses kernels, we might change our kernel entirely if we expect a different kind of function (e.g. a periodic function).

Picking the kernel is up to a human expert but choosing the parameters can be done automatically by minimising a loss term. This is the realm of Gaussian process regression.

Finally we should consider how to handle noisy data - i.e. when we can't get perfect samples of the hidden function $f$. In this case we need to factor this uncertainty into the model to get better generalisation.

These two topics will be the focus of _Introduction to Gaussian Processes - Part II_.

## Resources
***
- _Machine Learning - A Probabilistic Perspective_, Chapter 15 by Kevin P. Murphy
- [Introduction to Gaussian processes](https://www.youtube.com/watch?v=4vGiHC35j9s) on YouTube by Nando de Freitas

```python

```