# Bayesian linear regression in 1 dimension with known $\sigma^2$

Consider fitting a model of the form

\begin{equation}
y \mid x, \theta \sim \mathcal{N}\left(w_0 + w_1x, \sigma^2\right).
\end{equation}

We are given the data below.

```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
from linreg import *

X = np.array([94,96,94,95,104,106,108,113,115,121,131], dtype=np.float64)
Y = np.array([0.47,0.75,0.83,0.98,1.18,1.29,1.40,1.60,1.75,1.90,2.23])
N = len(X)

linreg = SimpleOnlineLinearRegressor()
linreg.fit(X, Y)
print(linreg.get_params())

plot_xy(X, Y)
plot_abline(linreg.get_params()['slope'], linreg.get_params()['intercept'],
            np.min(X) - 1, np.max(X) + 1)
plt.show()
```

Part (a) asks us to compute an unbiased estimate of $\sigma^2$, which is

\begin{equation}
\hat{\sigma}^2 = \frac{1}{N - 2}\sum_{i=1}^N\left(y_i - \hat{y}_i\right)^2.
\end{equation}

```python
Y_hat = linreg.predict(X)
sigma2_hat = np.dot(Y - Y_hat, Y - Y_hat)/(N-2)
print(sigma2_hat)
```

    0.0169746237612

We find that $\boxed{\hat{\sigma}^2 \approx 0.0169746237612.}$ Now, for part (b), we put a prior on $(w_0,w_1)$, where $p(w_0, w_1) = p(w_0)p(w_1)$ such that $p(w_0) \propto 1$ and $w_1 \sim \mathcal{N}(0,1)$. We can write this as

\begin{equation}
\mathbf{w} = \begin{pmatrix}w_0 \\ w_1 \end{pmatrix}
\sim \mathcal{N}\left(\mathbf{w}_0, \mathbf{V}_0\right),
~\text{where}~\mathbf{w}_0 = \begin{pmatrix} 0 \\ 0 \end{pmatrix}
~\text{and}~\mathbf{V}_0 = \begin{pmatrix}\sigma_0 & 0 \\ 0 & 1\end{pmatrix},
\end{equation}

where we let $\sigma_0 \rightarrow \infty$.

Part (c) asks us to compute the marginal posterior for the slope $p\left(w_1 \mid \mathcal{D}, \sigma^2\right)$, where we let $\sigma^2 = \hat{\sigma}^2$ computed above. First, by section section 7.6.1 in the textbook, the joint posterior has distrbution

\begin{align}
\mathbf{w} &\mid \mathcal{D}, \sigma^2 \sim \mathcal{N}\left(\mathbf{w}_N, \mathbf{V}_N\right) \\
\mathbf{V}_N &= \sigma^2\left(\sigma^2\mathbf{V}_0 + \mathbf{X}^\intercal\mathbf{X}\right)^{-1} \\
&= \frac{\sigma^2}{\sigma^2 + \sum_{i=1}^N x_i^2 - N\bar{x}^2}
\begin{pmatrix}
\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2 & -\bar{x} \\
-\bar{x} & 1
\end{pmatrix}. \\
\mathbf{w}_N &= \mathbf{V}_N\mathbf{V}_0^{-1}\mathbf{w}_0 + \frac{1}{\sigma^2}\mathbf{V}_N\mathbf{X}^\intercal \mathbf{y} \\
&= \left(\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2 - \bar{x}^2\right)^{-1}
\begin{pmatrix}
\left(\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2\right)\bar{y} - \frac{\bar{x}}{N}\sum_{i=1}^N x_iy_i \\
\frac{1}{N}\sum_{i=1}^N x_iy_i - \bar{x}\bar{y}
\end{pmatrix}\\
&= \left(\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2 - \bar{x}^2\right)^{-1}
\begin{pmatrix}
\left(\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2 - \bar{x}^2\right)\bar{y} - \bar{x}\left(\frac{1}{N}\sum_{i=1}^N x_iy_i - \bar{x}\bar{y}\right) \\
\frac{1}{N}\sum_{i=1}^N x_iy_i - \bar{x}\bar{y}
\end{pmatrix}.
\end{align}

Thus, the marginal distribution for $w_1$ is

\begin{equation}
w_1 \mid \mathcal{D}, \sigma^2 \sim \mathcal{N}\left(\left(\frac{\sigma^2}{N} + \frac{1}{N}\sum_{i=1}^N x_i^2 - \bar{x}^2\right)^{-1}\left(\frac{1}{N}\sum_{i=1}^N x_iy_i - \bar{x}\bar{y}\right),
\frac{\sigma^2}{\sigma^2 + \sum_{i=1}^N x_i^2 - N\bar{x}^2}
\right).
\end{equation}

Let us calculate these values numerically.

```python
w1_var = sigma2_hat/(sigma2_hat + np.dot(X, X) - N*np.mean(X)**2)
w1_mean = (np.dot(X,Y) - N*np.mean(X)*np.mean(Y))/(sigma2_hat + np.dot(X, X) - N*np.mean(X)**2)
print(w1_mean)
print(w1_var)
```

    0.0426509259869
    1.14229003107e-05

So we have that

\begin{align}
\mathbb{E}\left[w_1 \mid \mathcal{D}, \sigma^2\right] &\approx 0.0426509259869 \\
\operatorname{Var}\left[w_1 \mid \mathcal{D}, \sigma^2\right] &\approx 1.14229003107 \times 10^{-5}.
\end{align}

Finally part (d) asks us to compute the 95% credible interval, which is
\begin{equation}
\left[\mathbb{E}\left[w_1 \mid \mathcal{D}, \sigma^2\right] - \Phi^{-1}(0.975)\sqrt{\operatorname{Var}\left[w_1 \mid \mathcal{D}, \sigma^2\right]},
\mathbb{E}\left[w_1 \mid \mathcal{D}, \sigma^2\right] + \Phi^{-1}(0.975)\sqrt{\operatorname{Var}\left[w_1 \mid \mathcal{D}, \sigma^2\right]}\right].
\end{equation}

Let's calculate this.

```python
print(w1_mean - stats.norm.ppf(0.975)*np.sqrt(w1_var))
print(w1_mean + stats.norm.ppf(0.975)*np.sqrt(w1_var))
```

    0.0360266825466
    0.0492751694272

So, we have that

\begin{equation}
p(0.0360266825466 \leq w_1 \leq 0.0492751694272) \approx 0.95.
\end{equation}


# Neuromatch Academy: Week 3, Day 1, Tutorial 1
# Real Neurons: The Leaky Integrate-and-Fire (LIF) Neuron Model
__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar

__Content reviewers:__ Spiros Chavlis, Lorenzo Fontolan, Richard Gao, Matthew Krause

---
# Tutorial Objectives
This is Tutorial 1 of a series on implementing realistic neuron models. In this tutorial, we will build up a leaky integrate-and-fire (LIF) neuron model and study its dynamics in response to various types of inputs. In particular, we are going to write a few lines of code to:

- simulate the LIF neuron model

- drive the LIF neuron with external inputs, such as DC current, Gaussian white noise, and Poisson spike trains, etc.

- study how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity)

Here, we will especially emphasize identifying conditions (input statistics) under which a neuron can spike at low firing rates and in an irregular manner. The reason for focusing on this is that in most cases, neocortical neurons spike in an irregular manner.

---
# Setup

```python
# Imports
import matplotlib.pyplot as plt
import numpy as np
```

```python
# @title Figure Settings
import ipywidgets as widgets  # interactive display
%config InlineBackend.figure_format = 'retina'
# use NMA plot style
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
my_layout = widgets.Layout()
```

```python
# @title Helper functions

def plot_volt_trace(pars, v, sp):
  """
  Plot trajetory of membrane potential for a single neuron

  Expects:
  pars   : parameter dictionary
  v      : volt trajetory
  sp     : spike train

  Returns:
  figure of the membrane potential trajetory for a single neuron
  """

  V_th = pars['V_th']
  dt, range_t = pars['dt'], pars['range_t']
  if sp.size:
    sp_num = (sp / dt).astype(int) - 1
    v[sp_num] += 20  # draw nicer spikes

  plt.plot(pars['range_t'], v, 'b')
  plt.axhline(V_th, 0, 1, color='k', ls='--')
  plt.xlabel('Time (ms)')
  plt.ylabel('V (mV)')
  plt.legend(['Membrane\npotential', r'Threshold V$_{\mathrm{th}}$'],
             loc=[1.05, 0.75])
  plt.ylim([-80, -40])

def plot_GWN(pars, I_GWN):
  """
  Args:
    pars  : parameter dictionary
    I_GWN : Gaussian white noise input

  Returns:
    figure of the gaussian white noise input
  """

  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'][::3], I_GWN[::3], 'b')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{GWN}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()

def my_hists(isi1, isi2, cv1, cv2, sigma1, sigma2):
  """
  Args:
    isi1 : vector with inter-spike intervals
    isi2 : vector with inter-spike intervals
    cv1  : coefficient of variation for isi1
    cv2  : coefficient of variation for isi2

  Returns:
    figure with two histograms, isi1, isi2

  """
  plt.figure(figsize=(11, 4))
  my_bins = np.linspace(10, 30, 20)
  plt.subplot(121)
  plt.hist(isi1, bins=my_bins, color='b', alpha=0.5)
  plt.xlabel('ISI (ms)')
  plt.ylabel('count')
  plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma1, cv1))

  plt.subplot(122)
  plt.hist(isi2, bins=my_bins, color='b', alpha=0.5)
  plt.xlabel('ISI (ms)')
  plt.ylabel('count')
  plt.title(r'$\sigma_{GWN}=$%.1f, CV$_{\mathrm{isi}}$=%.3f' % (sigma2, cv2))
  plt.tight_layout()
  plt.show()
```

---
# Section 1: The Leaky Integrate-and-Fire (LIF) model

```python
#@title Video 1: Reduced Neuron Models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='rSExvwCVRYg', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=rSExvwCVRYg

## Implementation of an LIF neuron model

Now, it's your turn to implement one of the simplest mathematical model of a neuron: the leaky integrate-and-fire (LIF) model. The basic idea of LIF neuron was proposed in 1907 by Louis Édouard Lapicque, long before we understood the electrophysiology of a neuron (see a translation of [Lapicque's paper](https://pubmed.ncbi.nlm.nih.gov/17968583/) ). More details of the model can be found in the book [**Theoretical neuroscience**](http://www.gatsby.ucl.ac.uk/~dayan/book/) by Peter Dayan and Laurence F. Abbott.

The subthreshold membrane potential dynamics of a LIF neuron is described by

\begin{eqnarray}
C_m\frac{dV}{dt} = -g_L(V-E_L) + I,\quad (1)
\end{eqnarray}

where $C_m$ is the membrane capacitance, $V$ is the membrane potential, $g_L$ is the leak conductance ($g_L = 1/R$, the inverse of the leak resistance $R$ mentioned in previous tutorials), $E_L$ is the resting potential, and $I$ is the external input current.

Dividing both sides of the above equation by $g_L$ gives

\begin{align}
\tau_m\frac{dV}{dt} = -(V-E_L) + \frac{I}{g_L}\,,\quad (2)
\end{align}

where the $\tau_m$ is membrane time constant and is defined as $\tau_m=C_m/g_L$.

You might wonder how come dividing capacitance by conductance gives units of time! Find out yourself why it is so.

Below we will use Eqn.(1) to simulate the LIF neuron dynamics.

If $I$ is sufficiently strong such that $V$ reaches a certain threshold value $V_{\rm th}$, $V$ is reset to a reset potential $V_{\rm reset}< V_{\rm th}$ and voltage is clamped to $V_{\rm reset}$ for $\tau_{\rm ref}$ ms mimicking the refractoriness of the neuron during an action potential, _i.e._,

\begin{eqnarray}
\mathrm{if}\quad V(t_{\text{sp}})\geq V_{\rm th}&:& V(t)=V_{\rm reset} \text{  for } t\in(t_{\text{sp}}, t_{\text{sp}} + \tau_{\text{ref}}]
\end{eqnarray}
where $t_{\rm sp}$ is the spike time when $V(t)$ just exceeded $V_{\rm th}$.

(__Note__: in the lecture slides, $\theta$ corresponds to the threshold voltage $V_{th}$, and $\Delta$ corresponds to the refractory time $\tau_{\rm ref}$.)

Thus, the LIF model captures the facts that a neuron:
- performs spatial and temporal integration of synaptic inputs
- generates a spike when the voltage reaches a certain threshold
- goes refractory during the action potential
- membrane is leaky

The LIF model assumes that spatial and temporal integration of inputs is linear. Also, membrane potential dynamics close to the spike threshold is much slower in LIF neurons than in real neurons.

## Exercise 1: Python code to simulate the LIF neuron

We now **write** Python code to calculate Eqn. (2) and simulate the LIF neuron dynamics.

The cell below initializes a dictionary that stores parameters of the LIF neuron model and the simulation scheme. You can use `pars=default_pars(T=simulation_time, dt=time_step)` to get the parameters (you can try to print the dictionary `pars`). Note that, `simulation_time` and `time_step` have the unit `ms`. In addition, you can add the value to a new parameter by `pars['New_param'] = value`.

```python
# @title

# @markdown Execute this code to initialize the default parameters

def default_pars(**kwargs):
  pars = {}

  # typical neuron parameters#
  pars['V_th'] = -55.     # spike threshold [mV]
  pars['V_reset'] = -75.  # reset potential [mV]
  pars['tau_m'] = 10.     # membrane time constant [ms]
  pars['g_L'] = 10.       # leak conductance [nS]
  pars['V_init'] = -75.   # initial potential [mV]
  pars['E_L'] = -75.      # leak reversal potential [mV]
  pars['tref'] = 2.       # refractory time (ms)

  # simulation parameters #
  pars['T'] = 400.  # Total duration of simulation [ms]
  pars['dt'] = .1   # Simulation time step [ms]

  # external parameters if any #
  for k in kwargs:
    pars[k] = kwargs[k]

  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized time points [ms]

  return pars

pars = default_pars()
```

The cell below defines the function to simulate the LIF neuron when receiving external current inputs. You can use `v, sp = run_LIF(pars, Iinj)` to get the membrane potential (`v`) and spike train (`sp`) given the dictionary `pars` and input current `Iinj`.

```python
def run_LIF(pars, Iinj, stop=False):
  """
  Simulate the LIF dynamics with external input current

  Args:
    pars       : parameter dictionary
    Iinj       : input current [pA]. The injected current here can be a value
                 or an array
    stop       : boolean. If True, use a current pulse

  Returns:
    rec_v      : mebrane potential
    rec_sp     : spike times
  """

  # Set parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, E_L = pars['V_init'], pars['E_L']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tref = pars['tref']
  # Initialize voltage and current
  v = np.zeros(Lt)
  v[0] = V_init
  Iinj = Iinj * np.ones(Lt)
  if stop:
    Iinj[:int(len(Iinj) / 2) - 1000] = 0
    Iinj[int(len(Iinj) / 2) + 1000:] = 0
  tr = 0.  # the count for refractory duration
  # simulate the LIF dynamics
  rec_spikes = []  # record spike times
  for it in range(Lt - 1):
    if tr > 0:  # check for refractoriness
      v[it] = V_reset
      tr = tr - 1
    elif v[it] >= V_th:  # reset voltage and record spike event
      rec_spikes.append(it)
      v[it] = V_reset
      tr = tref / dt

    ########################################################################
    ## TODO for students: compute the membrane potential v, spike train sp #
    # Fill out function and remove
    raise NotImplementedError('Student Exercise: calculate the dv/dt and the update step!')
    ########################################################################
    # calculate the increment of the membrane potential
    dv = ...

    # update the membrane potential
    v[it+1] = ...

  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes

pars = default_pars(T=500)
# Uncomment below to test your function
# v, sp = run_LIF(pars, Iinj=100, stop=True)
# plot_volt_trace(pars, v, sp)
# plt.ylim([-80, -60])
# plt.show()
```

```python
# to_remove solution
def run_LIF(pars, Iinj, stop=False):
  """
  Simulate the LIF dynamics with external input current

  Args:
    pars       : parameter dictionary
    Iinj       : input current [pA]. The injected current here can be a value
                 or an array
    stop       : boolean. If True, use a current pulse

  Returns:
    rec_v      : mebrane potential
    rec_sp     : spike times
  """
  # Set parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, E_L = pars['V_init'], pars['E_L']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tref = pars['tref']
  # Initialize voltage and current
  v = np.zeros(Lt)
  v[0] = V_init
  Iinj = Iinj * np.ones(Lt)
  if stop:
    Iinj[:int(len(Iinj) / 2) - 1000] = 0
    Iinj[int(len(Iinj) / 2) + 1000:] = 0
  tr = 0.  # the count for refractory duration
  # simulate the LIF dynamics
  rec_spikes = []  # record spike times
  for it in range(Lt - 1):
    if tr > 0:  # check for refractoriness
      v[it] = V_reset
      tr = tr - 1
    elif v[it] >= V_th:  # reset voltage and record spike event
      rec_spikes.append(it)
      v[it] = V_reset
      tr = tref / dt
    # calculate the increment of the membrane potential
    dv = (-(v[it] - E_L) + Iinj[it] / g_L) * (dt / tau_m)

    # update the membrane potential
    v[it+1] = v[it] + dv

  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes

pars = default_pars(T=500)
# Uncomment below to test your function
v, sp = run_LIF(pars, Iinj=100, stop=True)
with plt.xkcd():
  plot_volt_trace(pars, v, sp)
  plt.ylim([-80, -60])
  plt.show()
```

---
# Section 2: Response of an LIF model to different types of input currents
In the following section, we will learn how to inject direct current and white noise to study the response of an LIF neuron.

```python
#@title Video 2: Response of the LIF neuron to different inputs
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='preNGdab7Kk', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=preNGdab7Kk

## Section 2.1: DC current

### LIF neuron driven by Constant current
Investigate the voltage response of the LIF neuron when receiving a DC current of 300 pA by `run_LIF` function.

```python
pars = default_pars(T=100)  # get the parameters
# Run the model to obtain v and sp
v, sp = run_LIF(pars, Iinj=300)

plot_volt_trace(pars, v, sp)
plt.show()
```

In the plot above, you see the membrane potential of an LIF neuron. You may notice that the neuron generates a spike. But this is just a cosmetic spike only for illustration purposes. In LIF neuron, we only need to keep track of times when the neuron hit the threshold so the postsynaptic neurons can be informed of the spike.

### Interactive Demo: Parameter exploration of DC current amplitude
Here's an interactive demo that shows how the LIF neuron behavior changes for DC input with different amplitudes.

**Remember to enable the demo by running the cell.**

```python
# @title
# @markdown Make sure you execute this cell to enable the widget!
my_layout.width = '450px'
@widgets.interact(
    I_dc=widgets.FloatSlider(50., min=0., max=300., step=10.,
                               layout=my_layout),
    tau_m=widgets.FloatSlider(10., min=2., max=20., step=2.,
                                layout=my_layout)
)

def diff_DC(I_dc=200., tau_m=10.):
  pars = default_pars(T=100.)
  pars['tau_m'] = tau_m
  v, sp = run_LIF(pars, Iinj=I_dc)
  plot_volt_trace(pars, v, sp)
  plt.show()
```

## Section 2.2: Gaussian white noise (GWN) current

Given the noisy nature of neuronal activity _in vivo_, neurons usually receive complex, time-varying inputs.

To mimic this, we will now investigate the neuronal response when the LIF neuron receives Gaussian white noise $\xi(t)$ with mean

\begin{eqnarray}
E[\xi(t)]=\mu=0,
\end{eqnarray}

and autocovariance
\begin{eqnarray}
E[\xi(t)\xi(t+\tau)]=\sigma_\xi^2 \delta(\tau)
\end{eqnarray}

Note that the GWN has the zero mean, that is, it describes the fluctuations of the input received by a neuron. Thus, at the same time, we can use the DC current as the average input into the cell, which allows us to modify the definition of GWN by having a nonzero mean value $\mu$. The cell below defines the modified gaussian white noise currents with nonzero mean $\mu$.

```python
#@title Function to generate GWN: `my_GWN(pars, mu, sig, myseed=False)`

#@markdown Run this cell before you continue!

def my_GWN(pars, mu, sig, myseed=False):
  """
  Function that generates Gaussian white noise input

  Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    myseed     : random seed. int or boolean
                 the same seed will give the same
                 random number sequence

  Returns:
    I          : Gaussian white noise input
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  # you can fix the seed of the random number generator so that the results
  # are reliable however, when you want to generate multiple realization
  # make sure that you change the seed for each new realization.
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate GWN
  # we divide here by 1000 to convert units to sec.
  I_gwn = mu + sig * np.random.randn(Lt) / np.sqrt(dt / 1000.)

  return I_gwn
```

### Exercise 2: LIF neuron driven by GWN

You can generate a noisy input with `my_GWN(pars, mu, sig, myseed=False)`. Here, $\mu=250$ and $\sigma=5$. Note that fixing the value of the random seed (e.g., `myseed=2020`) will allow you to obtain the same result every time you run this.

```python
pars = default_pars(T=100.)
sig_gwn = 5.
mu_gwn = 250.
#######################################################
## TODO for students: compute the noisy input I_GWN   #
# and then compute v, sp with inject I_GWN into LIF   #
#######################################################
# Calculate the GWN current
# I_GWN = my_GWN(pars, ..., ..., myseed=2020)
# Run the model and calculate the v and the sp
# v, sp = run_LIF(pars, ...)

# Uncomment these lines after obtaining v, sp
# plot_GWN(pars, I_GWN)
# plt.show()
```

```python
# to_remove solution
pars = default_pars(T=100.)
sig_gwn = 5.
mu_gwn = 250.
# Calculate the GWN current
I_GWN = my_GWN(pars, mu=mu_gwn, sig=sig_gwn, myseed=2020)
# Run the model and calculate the v and the sp
v, sp = run_LIF(pars, Iinj=I_GWN)

with plt.xkcd():
  plot_GWN(pars, I_GWN)
  plt.show()
```

### Interactive Demo: LIF neuron Explorer for noisy input
Here's an interactive demo that shows how the LIF neuron behavior changes for  noisy input with different amplitude and the mean $\mu$ and fluctuation $\sigma$.

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    mu_gwn=widgets.FloatSlider(200., min=100., max=300., step=5.,
                               layout=my_layout),
    sig_gwn=widgets.FloatSlider(2.5, min=0., max=5., step=.5,
                                layout=my_layout)
)

def diff_GWN_to_LIF(mu_gwn, sig_gwn):
  pars = default_pars(T=100.)
  I_GWN = my_GWN(pars, mu=mu_gwn, sig=sig_gwn)
  v, sp = run_LIF(pars, Iinj=I_GWN)
  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'][::3], I_GWN[::3], 'b')
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{GWN}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()
  plt.show()
```

## Think!
- As we increase the input average ($\mu$) or the input fluctuation ($\sigma$), the spike count changes. How much can we increase the spike count, and what might be the relationship between GWN mean/std or DC value and spike count?

- We have seen above that when we inject DC, the neuron spikes in a regular manner (clock like), and this regularity is reduced when GWN is injected. The question is, how irregular can we make the neurons spiking by changing the parameters of the GWN?

---
# Section 3: Firing rate and spike time irregularity
When we plot the output firing rate as a function of GWN mean or DC value, it is called the input-output transfer function of the neuron (so simply F-I curve).

Spike regularity can be quantified as the **coefficient of variance (CV) of the inter-spike-interval (ISI)**:
\begin{align}
\text{CV}_{\text{ISI}} = \frac{std(\text{ISI})}{mean(\text{ISI})}
\end{align}

A Poisson train is an example of high irregularity, in which $\textbf{CV}_{\textbf{ISI}} \textbf{= 1}$. And for a clock like (regular) process we have $\textbf{CV}_{\textbf{ISI}} \textbf{= 0}$ because of **std(ISI)=0**.

## Interactive Demo: F-I Explorer for different `sig_gwn`
Here's an interactive demo that shows how the F-I curve of a LIF neuron changes for different levels of fluctuation $\sigma$.

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    sig_gwn=widgets.FloatSlider(3.0, min=0., max=6., step=0.5,
                                layout=my_layout)
)

def diff_std_affect_fI(sig_gwn):
  pars = default_pars(T=1000.)
  I_mean = np.arange(100., 400., 10.)
  spk_count = np.zeros(len(I_mean))
  spk_count_dc = np.zeros(len(I_mean))

  for idx in range(len(I_mean)):
      I_GWN = my_GWN(pars, mu=I_mean[idx], sig=sig_gwn, myseed=2020)
      v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
      v_dc, rec_sp_dc = run_LIF(pars, Iinj=I_mean[idx])
      spk_count[idx] = len(rec_spikes)
      spk_count_dc[idx] = len(rec_sp_dc)

  # Plot the F-I curve i.e. Output firing rate as a function of input mean.
  plt.figure()
  plt.plot(I_mean, spk_count, 'k',
           label=r'$\sigma_{\mathrm{GWN}}=%.2f$' % sig_gwn)
  plt.plot(I_mean, spk_count_dc, 'k--', alpha=0.5, lw=4, dashes=(2, 2),
           label='DC input')
  plt.ylabel('Spike count')
  plt.xlabel('Average injected current (pA)')
  plt.legend(loc='best')
  plt.show()
```

### Exercise 3: Compute $CV_{ISI}$ values

As shown above, the F-I curve becomes smoother while increasing the amplitude of the fluctuation ($\sigma$). In addition, the fluctuation can also change the irregularity of the spikes. Let's investigate the effect of $\mu=250$, $\sigma=0.5$, and $\sigma=3$.

Fill in the code below to compute ISI, then plot the histogram of the ISI and compute the $CV_{ISI}$. Note that, you can use `np.diff` to calculate ISI.

```python
def isi_cv_LIF(spike_times):
  """
  Calculates the inter-spike intervals (isi) and
  the coefficient of variation (cv) for a given spike_train

  Args:
    spike_times : (n, ) vector with the spike times (ndarray)

  Returns:
    isi         : (n-1,) vector with the inter-spike intervals (ms)
    cv          : coefficient of variation of isi (float)

  """
  ########################################################################
  ## TODO for students: compute the membrane potential v, spike train sp #
  # Fill out function and remove
  raise NotImplementedError('Student Exercise: calculate the isi and the cv!')
  ########################################################################
  if len(spike_times) >= 2:
    # Compute isi
    isi = ...
    # Compute cv
    cv = ...
  else:
    isi = np.nan
    cv = np.nan

  return isi, cv

pars = default_pars(T=1000.)
mu_gwn = 250
sig_gwn1 = 0.5
sig_gwn2 = 3.0

I_GWN1 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn1, myseed=2020)
_, sp1 = run_LIF(pars, Iinj=I_GWN1)

I_GWN2 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn2, myseed=2020)
_, sp2 = run_LIF(pars, Iinj=I_GWN2)

# Uncomment to check your function
# isi1, cv1 = isi_cv_LIF(sp1)
# isi2, cv2 = isi_cv_LIF(sp2)

# my_hists(isi1, isi2, cv1, cv2, sig_gwn1, sig_gwn2)
```

```python
# to_remove solution
def isi_cv_LIF(spike_times):
  """
  Calculates the inter-spike intervals (isi) and
  the coefficient of variation (cv) for a given spike_train

  Args:
    spike_times : (n, ) vector with the spike times (ndarray)

  Returns:
    isi         : (n-1,) vector with the inter-spike intervals (ms)
    cv          : coefficient of variation of isi (float)

  """

  if len(spike_times) >= 2:
    # Compute isi
    isi = np.diff(spike_times)
    # Compute cv
    cv = isi.std()/isi.mean()
  else:
    isi = np.nan
    cv = np.nan

  return isi, cv

pars = default_pars(T=1000.)
mu_gwn = 250
sig_gwn1 = 0.5
sig_gwn2 = 3.0

I_GWN1 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn1, myseed=2020)
_, sp1 = run_LIF(pars, Iinj=I_GWN1)

I_GWN2 = my_GWN(pars, mu=mu_gwn, sig=sig_gwn2, myseed=2020)
_, sp2 = run_LIF(pars, Iinj=I_GWN2)

# Uncomment to check your function
isi1, cv1 = isi_cv_LIF(sp1)
isi2, cv2 = isi_cv_LIF(sp2)

with plt.xkcd():
  my_hists(isi1, isi2, cv1, cv2, sig_gwn1, sig_gwn2)
```

## Interactive Demo: Spike irregularity explorer for different `sig_gwn`
Here's an interactive demo that shows how the function of CVs on the mean current $\mu$ changes for different levels of fluctuation $\sigma$.

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

my_layout.width = '450px'
@widgets.interact(
    sig_gwn=widgets.FloatSlider(0.0, min=0., max=10.,
                                step=0.5, layout=my_layout)
)

def diff_std_affect_fI(sig_gwn):
  pars = default_pars(T=1000.)
  I_mean = np.arange(100., 400., 20)
  spk_count = np.zeros(len(I_mean))
  cv_isi = np.empty(len(I_mean))

  for idx in range(len(I_mean)):
    I_GWN = my_GWN(pars, mu=I_mean[idx], sig=sig_gwn)
    v, rec_spikes = run_LIF(pars, Iinj=I_GWN)
    spk_count[idx] = len(rec_spikes)
    if len(rec_spikes) > 3:
      isi = np.diff(rec_spikes)
      cv_isi[idx] = np.std(isi) / np.mean(isi)

  # Plot the F-I curve i.e. Output firing rate as a function of input mean.
  plt.figure()
  plt.plot(I_mean[spk_count > 5], cv_isi[spk_count > 5], 'bo', alpha=0.5)
  plt.xlabel('Average injected current (pA)')
  plt.ylabel(r'Spike irregularity ($\mathrm{CV}_\mathrm{ISI}$)')
  plt.ylim(-0.1, 1.5)
  plt.grid(True)
  plt.show()
```

### Try to answer the following:
- Does the standard deviation of the injected current affect the F-I curve in any qualitative manner?
- Why does increasing the mean of GWN reduce the $CV_{ISI}$?
- If you plot spike count (or rate) vs. $CV_{ISI}$, should there be a relationship between the two? Try out yourself.

---
# Section 4: Generation of Poisson type spike trains

*In the next tutorials, we will often use Poisson type spike train to explore properties of neurons and synapses. Therefore, it is good to know how to generate Poisson type spike trains.*

Mathematically a spike train is a Point Process. One of the simplest models of a sequence of presynaptic pulse inputs is the Poisson process. We know that given temporal integration and refractoriness, neurons cannot behave as a Poisson Process, and Gamma Process (*find out what might be the difference in the two processes*) gives a better approximation.
Here, however, we will assume that the incoming spikes are following Poisson statistics. A question arises about how to simulate a Poisson Process. The generation of the Poisson process can be realized by at least two following ways:

- By definition, for a Poisson process with rate $\lambda$, the probability of finding one event in the time window with a sufficiently small length $\Delta t$ is $P(N = 1) = \lambda \Delta t$. Therefore, in each time window, we generate a uniformly distributed random variable $r \in [0,1]$ and generate a Poisson event when $r <\lambda \Delta t$. This method allows us to generate Poisson distributed spikes in an online manner.

- The interval $t_{k+1}-t_{k}$ between two Poisson events with rate $\lambda$ follows the exponential distribution, i.e., $P(t_{k+1}-t_{k}<t) = 1 - e^{\lambda t}$. Therefore, we only need to generate a set of exponentially distributed variables $\{s_k\}$ to obtain the timing of Poisson events $t_{k+1}=t_{k}+s_{k}$. In this method, we need to generate all future spikes at once.

Below we use the first method and check the statistical property of the time interval between two Poisson events.

## Function to generate Poisson spike trains

```python
# @title `Poisson_generator(pars, rate, n, myseed`)
# @markdown Make sure you execute this cell to create Poisson spikes!

def Poisson_generator(pars, rate, n, myseed=False):
  """
  Generates poisson trains

  Args:
    pars            : parameter dictionary
    rate            : noise amplitute [Hz]
    n               : number of Poisson trains
    myseed          : random seed. int or boolean

  Returns:
    pre_spike_train : spike train matrix, ith row represents whether
                      there is a spike in ith spike train over time
                      (1 if spike, 0 otherwise)
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate uniformly distributed random variables
  u_rand = np.random.rand(n, Lt)

  # generate Poisson train
  poisson_train = 1. * (u_rand < rate * (dt / 1000.))

  return poisson_train

# the function plot the raster of the Poisson spike train
def my_raster_Poisson(range_t, spike_train, n):
  """
  Generates poisson trains

  Args:
    range_t     : time sequence
    spike_train : binary spike trains, with shape (N, Lt)
    n           : number of Poisson trains plot

  Returns:
    Raster plot of the spike train
  """

  # find the number of all the spike trains
  N = spike_train.shape[0]

  # n should smaller than N:
  if n > N:
    print('The number n exceeds the size of spike trains')
    print('The number n is set to be the size of spike trains')
    n = N

  # plot rater
  i = 0
  while i < n:
    if spike_train[i, :].sum() > 0.:
      t_sp = range_t[spike_train[i, :] > 0.5]  # spike times
      plt.plot(t_sp, i * np.ones(len(t_sp)), 'k|', ms=10, markeredgewidth=2)
    i += 1
  plt.xlim([range_t[0], range_t[-1]])
  plt.ylim([-0.5, n + 0.5])
  plt.xlabel('Time (ms)', fontsize=12)
  plt.ylabel('Neuron ID', fontsize=12)
```

```python
# we can use Poisson_generator to mimic presynaptic spike trains
pars = default_pars()
pre_spike_train = Poisson_generator(pars, rate=10, n=100, myseed=2020)
my_raster_Poisson(pars['range_t'], pre_spike_train, 100)

print(help(my_raster_Poisson))  # use `help` to learn about the function
```

How do we make sure that the above spike trains are following Poisson statistics?
A Poisson process must have the following properties:
- The ratio of the mean and variance of spike count is 1
- Inter-spike-intervals are exponentially distributed
- Spike times are irregular i.e. $CV_{\rm ISI} = 1$
- Adjacent spike intervals are independent of each other.

---
# Summary

Congratulations! You've just built a leaky integrate-and-fire (LIF) neuron model from scratch, and studied its dynamics in response to various types of inputs, having:

- simulated the LIF neuron model

- driven the LIF neuron with external inputs, such as DC current, Gaussian white noise, and Poisson spike trains, etc.

- studied how different inputs affect the  LIF neuron's output (firing rate and spike time irregularity),

with a special focus on low rate and irregular firing regime to mimc real cortical neurons. The next tutorial will look at how spiking statistics may be influenced by a neuron's input statistics.

However, if you have extra time, follow the section below to explore a different type of noise input.

---
# Bonus 1: Orenstein-Uhlenbeck Process

When a neuron received spiking input, the synaptic current is Shot Noise -- which is a kind of colored noise and the spectrum of the noise determined by the synaptic kernel time constant. That is, a neuron is driven by **colored noise** and not GWN.

We can model colored noise using the Ohrenstein-Uhlenbeck process - filtered white noise.

## Ornstein-Uhlenbeck (OU) current

We next study if the input current is temporally correlated and is modeled as an Ornstein-Uhlenbeck process $\eta(t)$, i.e., low-pass filtered GWN with a time constant $\tau_{\eta}$:

$$\tau_\eta \frac{d}{dt}\eta(t) = \mu-\eta(t) + \sigma_\eta\sqrt{2\tau_\eta}\xi(t).$$

**Hint:** An OU process as defined above has

$$E[\eta(t)]=\mu$$

and autocovariance

$$[\eta(t)\eta(t+\tau)]=\sigma_\eta^2e^{-|t-\tau|/\tau_\eta},$$

which can be used to check your code.

```python
# @title `my_OU(pars, mu, sig, myseed)`
# @markdown Ececute this cell to enable the OU process

def my_OU(pars, mu, sig, myseed=False):
  """
  Function that produces Ornstein-Uhlenbeck input

  Args:
    pars       : parameter dictionary
    sig        : noise amplitute
    myseed     : random seed. int or boolean

  Returns:
    I_ou       : Ornstein-Uhlenbeck input current
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tau_ou = pars['tau_ou']  # [ms]

  # set random seed
  if myseed:
    np.random.seed(seed=myseed)
  else:
    np.random.seed()

  # Initialize
  noise = np.random.randn(Lt)
  I_ou = np.zeros(Lt)
  I_ou[0] = noise[0] * sig

  # generate OU
  for it in range(Lt-1):
    I_ou[it+1] = I_ou[it] + (dt / tau_ou) * (mu - I_ou[it]) + np.sqrt(2 * dt / tau_ou) * sig * noise[it + 1]

  return I_ou
```

### Interactive Demo: LIF Explorer with OU input
In the following, we will check how a neuron responds to a noisy current that follows the statistics of an OU process.

```python
# @title

# @markdown Remember to enable the widget by running the cell!

my_layout.width = '450px'
@widgets.interact(
  tau_ou=widgets.FloatSlider(10.0, min=5., max=20.,
                             step=2.5, layout=my_layout),
  sig_ou=widgets.FloatSlider(10.0, min=5., max=40.,
                             step=2.5, layout=my_layout),
  mu_ou=widgets.FloatSlider(190.0, min=180., max=220.,
                            step=2.5, layout=my_layout)
)

def LIF_with_OU(tau_ou=10., sig_ou=40., mu_ou=200.):
  pars = default_pars(T=1000.)
  pars['tau_ou'] = tau_ou  # [ms]

  I_ou = my_OU(pars, mu_ou, sig_ou)

  v, sp = run_LIF(pars, Iinj=I_ou)

  plt.figure(figsize=(12, 4))
  plt.subplot(121)
  plt.plot(pars['range_t'], I_ou, 'b', lw=1.0)
  plt.xlabel('Time (ms)')
  plt.ylabel(r'$I_{\mathrm{OU}}$ (pA)')
  plt.subplot(122)
  plot_volt_trace(pars, v, sp)
  plt.tight_layout()
  plt.show()
```

## Think!
- How does the OU type input change neuron responsiveness?
- What do you think will happen to the spike pattern and rate if you increased or decreased the time constant of the OU process?

---
# Bonus 2: Generalized Integrate-and-Fire models

LIF model is not the only abstraction of real neurons. If you want to learn about more realistic types of neuronal models, watch the Bonus Video!

```python
#@title Video 3 (Bonus): Extensions to Integrate-and-Fire models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='G0b6wLhuQxE', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=G0b6wLhuQxE


## Support Vector Machines:

Support vector machines focus only on the points that are the most difficult to tell apart, whereas other classifiers pay attention to all of the points.

The intuition behind the support vector machine approach is that if a classifier is good at the most challenging comparisons (the points in B and A that are closest to each other in Figure 2), then the classifier will be even better at the easy comparisons (comparing points in B and A that are far away from each other).

**Perceptrons and other classifiers:**

Perceptrons are built by taking one point at a time and adjusting the dividing line accordingly. As soon as all of the points are separated, the perceptron algorithm stops. But it could stop anywhere. Figure 1 shows that there are a bunch of different dividing lines that separate the data. The perceptron's stopping criteria is simple: "separate the points and stop improving the line when you get 100% separation". The perceptron is not explicitly told to find the best separating line. Logistic regression and linear discriminant models are built similarly to perceptrons.

The best dividing line maximizes the distance between the B points closest to A and the A points closest to B. It's not necessary to look at all of the points to do this. In fact, incorporating feedback from points that are far away can bump the line a little too far, as seen below.

**Support Vector Machines:**

Unlike other classifiers, the support vector machine is ***explicitly*** told to find the best separating line. How? The support vector machine searches for the closest points (Figure 2), which it calls the "support vectors" (the name "support vector machine" is due to the fact that points are like vectors and that the best line "depends on" or is "supported by" the closest points).

Once it has found the closest points, the SVM draws a line connecting them (see the line labeled 'w' in Figure 2). It draws this connecting line by doing vector subtraction (point A - point B). The support vector machine then declares the best separating line to be the line that bisects -- and is perpendicular to -- the connecting line.

The support vector machine is better because when you get a new sample (new points), you will have already made a line that keeps B and A as far away from each other as possible, and so it is less likely that one will spillover across the line into the other's territory.

The paper called *[Duality and Geometry in SVM Classifiers](http://www.robots.ox.ac.uk/~cvrg/bennett00duality.pdf)* finally helped me see the light; that's where I got the images from.

## Mathematical overview of how SVM's are trained and used:

## Notations ##

In the following, scalars are denoted with italic lowercases (e.g., $y,\, b$), vectors with bold lowercases (e.g., $\mathbf{w},\, \mathbf{x}$), and matrices with italic uppercases (e.g., $W$). $\mathbf{w^T}$ is the transpose of $\mathbf{w}$, and $\|\mathbf{w}\| = \mathbf{w}^T\mathbf{w}$.

Let:

- $\mathbf{x}$ be a feature vector (i.e., the input of the SVM). $\mathbf{x} \in \mathbb{R}^n$, where $n$ is the dimension of the feature vector.
- $y$ be the class (i.e., the output of the SVM). $y \in \{ -1,1\}$, i.e. the classification task is binary.
- $\mathbf{w}$ and $b$ be the parameters of the SVM: we need to learn them using the training set.
- $(\mathbf{x}^{(i)}, y^{(i)})$ be the $i^ {\text {th}}$ sample in the dataset. Let's assume we have $N$ samples in the training set.

With $n=2$, one can represent the SVM's decision boundaries as follows:

The class $y$ is determined as follows:

$$
y^{(i)}=\left\{
                \begin{array}{ll}
                  -1 &\text{ if } \mathbf{w^T}\mathbf{x}^{(i)}+b \leq -1 \\
                  1 &\text{ if } \mathbf{w^T}\mathbf{x}^{(i)}+b \ge 1 \\
                \end{array}
              \right.
$$

which can be more concisely written as $y^{(i)} (\mathbf{w^T}\mathbf{x}^{(i)}+b) \ge 1$.

## Goal  ##

The SVM aims at satisfying two requirements:

1. The SVM should maximize the distance between the two decision boundaries. Mathematically, this means we want to maximize the distance between the hyperplane defined by $\mathbf{w^T}\mathbf{x}+b = -1$ and the hyperplane defined by $\mathbf{w^T}\mathbf{x}+b = 1$. [This distance is equal to $\frac{2}{\|\mathbf{w}\|}$](https://math.stackexchange.com/a/1306417/168764). This means we want to solve $\underset{\mathbf{w}}{\operatorname{max}} \frac{2}{\|\mathbf{w}\|}$. Equivalently we want
$\underset{\mathbf{w}}{\operatorname{min}} \frac{\|\mathbf{w}\|}{2}$.

2. The SVM should also correctly classify all $\mathbf{x}^{(i)}$, which means $y^{(i)} (\mathbf{w^T}\mathbf{x}^{(i)}+b) \ge 1, \forall i \in \{1,\dots,N\}$

Which leads us to the following quadratic optimization problem:

$$\begin{align}
\min_{\mathbf{w},b}\quad &\frac{\|\mathbf{w}\|}{2}, \\
s.t.\quad&y^{(i)} (\mathbf{w^T}\mathbf{x}^{(i)}+b) \ge 1 &\forall i \in \{1,\dots,N\}
\end{align}$$

This is the **hard-margin SVM**, as this quadratic optimization problem admits a solution iff the data is linearly separable.

One can relax the constraints by introducing so-called **slack variables** $\xi^{(i)}$. Note that each sample of the training set has its own slack variable. This gives us the following quadratic optimization problem:

$$\begin{align}
\min_{\mathbf{w},b}\quad &\frac{\|\mathbf{w}\|}{2}+ C \sum_{i=1}^{N} \xi^{(i)}, \\
s.t.\quad&y^{(i)} (\mathbf{w^T}\mathbf{x}^{(i)}+b) \ge 1 - \xi^{(i)},&\forall i \in \{1,\dots,N\} \\
\quad&\xi^{(i)}\ge0, &\forall i \in \{1,\dots,N\}
\end{align}$$

This is the **soft-margin SVM**. $C$ is a hyperparameter called **penalty of the error term**.
 ([What is the influence of C in SVMs with linear kernel?](https://stats.stackexchange.com/q/31066/12359) and [Which search range for determining SVM optimal parameters?](https://stats.stackexchange.com/q/43943/12359)).

One can add even more flexibility by introducing a function $\phi$ that maps the original feature space to a higher dimensional feature space. This allows non-linear decision boundaries. The quadratic optimization problem becomes:

$$\begin{align}
\min_{\mathbf{w},b}\quad &\frac{\|\mathbf{w}\|}{2}+ C \sum_{i=1}^{N} \xi^{(i)}, \\
s.t.\quad&y^{(i)} (\mathbf{w^T}\phi \left(\mathbf{x}^{(i)}\right)+b) \ge 1 - \xi^{(i)},&\forall i \in \{1,\dots,N\} \\
\quad&\xi^{(i)}\ge0, &\forall i \in \{1,\dots,N\}
\end{align}$$

## Optimization  ##

### Solving using the Lagrangian:

### Hence, the optimization depends on the dot product of pairs of examples
--------------------------------------------------------------------------------------------
-------------------
The quadratic optimization problem can be transformed into another optimization problem named the [**Lagrangian dual problem**](https://en.wikipedia.org/w/index.php?title=Duality_(optimization)&oldid=755311099#Convex_problems) (the previous problem is called the **primal**):

$$\begin{align}
\max_{\mathbf{\alpha}}  \quad &\min_{\mathbf{w},b}  \frac{\|\mathbf{w}\|}{2}+ C \sum_{i=1}^{N} \alpha^{(i)} \left(1-\mathbf{w^T}\phi \left(\mathbf{x}^{(i)}\right)+b)\right), \\
s.t. \quad&0 \leq  \alpha^{(i)} \leq C, &\forall i \in \{1,\dots,N\}
\end{align}$$

This optimization problem can be simplified (by setting some gradients to $0$) to:

$$\begin{align}
\max_{\mathbf{\alpha}}  \quad & \sum_{i=1}^{N} \alpha^{(i)} - \sum_{i=1}^{N}\sum_{j=1}^{N}  \left( y^{(i)}\alpha^{(i)}\phi\left(\mathbf{x}^{(i)}\right)^T \phi\left(\mathbf{x}^{(j)}\right) y^{(j)}\alpha^{(j)} \right), \\
s.t. \quad&0 \leq  \alpha^{(i)} \leq C, &\forall i \in \{1,\dots,N\}
\end{align}$$

$\mathbf{w}$ doesn't appear as $\mathbf{w}=\sum_{i =1}^{N}\alpha^{(i)}y^{(i)}\phi\left(x^{(i)}\right)$ (as stated by the [representer theorem](https://en.wikipedia.org/wiki/Representer_theorem)).

We therefore learn the $\alpha^{(i)}$ using the $(\mathbf{x}^{(i)}, y^{(i)})$ of the training set.

(FYI: [Why bother with the dual problem when fitting SVM?](https://stats.stackexchange.com/q/19181/12359) short answer: faster computation + allows to use the kernel trick, though there exist some good methods to train SVM in the primal e.g. see  {1})

## Making a prediction  ##

Once the $\alpha^{(i)}$ are learned, one can predict the class of a new sample with the feature vector $\mathbf{x}^{\text {test}}$ as follows:

\begin{align*}
y^{\text {test}}&=\text {sign}\left(\mathbf{w^T}\phi\left(\mathbf{x}^{\text {test}}\right)+b\right) \\
&= \text {sign}\left(\sum_{i =1}^{N}\alpha^{(i)}y^{(i)}\phi\left(x^{(i)}\right)^T\phi\left(\mathbf{x}^{\text {test}}\right)+b \right)
\end{align*}

The summation $\sum_{i =1}^{N}$ could seem overwhelming, since it means one has to sum over all the training samples, but the vast majority of $\alpha^{(i)}$ are $0$ (see [Why are the Lagrange multipliers sparse for SVMs?](https://stats.stackexchange.com/q/54976/12359)) so in practice it isn't an issue. (note that [one can construct special cases where all $\alpha^{{(i)}} > 0$](https://stats.stackexchange.com/q/110598/12359).) $\alpha^{{(i)}}=0$ iff $x^{{(i)}}$ is a **support vector**. The illustration above has 3 support vectors.

# Some comments on Duality and KTT conditions
### Primal problem

Picking up from @Antoni's post in between equations $(4)$ and $(5)$, recall that our original, or **primal**, optimization problem is of the form:

\begin{aligned}
	\min_{w, b} f(w,b) & = \min_{w, b}  \  \frac{1}{2} ||w||^2
	\\
	s.t. \ \  g_i(w,b) &= - y^{(i)} (w^T x^{(i)} + b) + 1 = 0
\end{aligned}

### Lagrange method
The method of Lagrange multipliers allows us to turn a constrained optimization problem into an unconstrained one of the form:

$$\mathcal{L}(w, b, \alpha) =   \frac{1}{2} ||w||^2 - \sum_i^m \alpha_i [y^{(i)} (w^T x^{(i)} + b) - 1]$$

Where  $\mathcal{L}(w, b, \alpha)$ is called the **Lagrangian** and $\alpha_i$ are called the **Lagrangian multipliers**.

Our **primal** optimization problem with the Lagrangian becomes the following: (note that the use of $min$, $max$ is not the most rigorous as we should also be using $\inf$ and $\sup$ here...)

$$ \min_{w,b} \left( \max_\alpha \mathcal{L}(w, b, \alpha)\right)$$

### Dual problem
What @Antoni and Prof. Patrick Winston have done in their derivation is assume that the optimization function and the constraints meet some technical conditions such that we can do the following:

$$ \min_{w,b} \left( \max_\alpha \mathcal{L}(w, b, \alpha)\right) =   \max_\alpha \left( \min_{w,b} \mathcal{L}(w, b, \alpha)\right)$$

This allows us to take the partial derivatives of $\mathcal{L}(w, b, \alpha)$ with respect to $w$ and $b$, equate to zero and then plug the results back into the original equation of the Lagrangian, hence generating an equivalent **dual** optimization problem of the form

\begin{aligned}
	&\max_{\alpha} \min_{w,b} \mathcal{L}(w,b,\alpha)
	\\
	& \max_{\alpha} \sum_i^m \alpha_i - \frac{1}{2} \sum_{i,j}^m y^{(i)}y^{(j)} \alpha_i \alpha_j <x^{(i)} x^{(j)}>
	\\
	& s.t. \ \alpha_i \geq 0
	\\
	& s.t. \ \sum_i^m \alpha_i y^{(i)} = 0
\end{aligned}

### Duality and KTT
Without going into excessive mathematical technicalities, these conditions are a combination of the Duality and the Karush Kuhn Tucker (KTT) conditions and allow us to solve the **dual** problem instead of the **primal** one, while ensuring that the optimal solution is the same. In our case the conditions are the following:

- The primal objective and inequality constraint functions must be convex
- The equality constraint function must be affine
- The constraints must be strictly feasible

Then there exists $w^*, \alpha^*$ which are solutions to the primal and dual problems. Moreover, the parameters $w^*, \alpha^*$ satisfy the KTT conditions below:

\begin{aligned}
	&\frac{\partial}{\partial w_i}  \mathcal{L}(w^*, \alpha^*, \beta^*) = 0 &(A)
	\\
	&\frac{\partial}{\partial \beta_i}  \mathcal{L}(w^*, \alpha^*, \beta^*) = 0 &(B)
	\\
	&\alpha_i^* g_i(w^*) = 0 &(C)
	\\
	&g_i(w^*) \leq 0  &(D)
	\\
	&\alpha_i^* \geq 0 &(E)
\end{aligned}

Moreover, if some $w^*, \alpha^*$ satisfy the KTT solutions then they are also solution to the primal and dual problem.

Equation $(C)$ above is of particular importance and is called the *dual complementarity condition*. It implies that if $\alpha_i^* > 0$ then $g_i(w^*) = 0$ which means that the constraint $g_i(w) \leq 0$ is active, i.e. it holds with equality rather than inequality. This is the explanation behind equation $(2)$ in Antoni's derivation where the inequality constraint is turned into an equality constraint.

### A intuitive but informal diagram

----------
---------
## Kernel trick  ##

One can observe that the optimization problem uses the $\phi\left(\mathbf{x}^{(i)}\right)$ only in the inner product $\phi\left(\mathbf{x}^{(i)}\right)^T \phi\left(\mathbf{x}^{(j)}\right)$. The function that maps $\left(\mathbf{x}^{(i)},\mathbf{x}^{(j)}\right)$ to the inner product $\phi\left(\mathbf{x}^{(i)}\right)^T \phi\left(\mathbf{x}^{(j)}\right)$ is [called](https://stats.stackexchange.com/a/226196/12359) a **kernel**, a.k.a. kernel function, often denoted by $k$.

One can choose $k$ so that the inner product is efficient to compute. This allows to use a potentially high feature space at a low computational cost. That is called the [**kernel trick**](https://en.wikipedia.org/w/index.php?title=Kernel_method&oldid=747269865#Mathematics:_the_kernel_trick). For a kernel function to be *valid*, i.e. usable with the kernel trick, it should satisfy [two key properties](https://stats.stackexchange.com/q/48506/12359). There exist [many kernel functions to choose from](https://stats.stackexchange.com/q/18030/12359). As a side note, the kernel trick [may be applied to other machine learning models](https://stats.stackexchange.com/q/2167/12359), in which case they are referred as [*kernelized*](https://scholar.google.com/scholar?q=kernelized+&hl=en&as_sdt=0,22).

## Going further  ##

Some interesting QAs on SVMs:

- [Best way to perform multiclass SVM](https://stats.stackexchange.com/q/21465/12359)
- [Support vector machines and regression](https://stats.stackexchange.com/q/13194/12359)
- [Understanding the different formulations for SVM](https://stats.stackexchange.com/q/157827/12359)
- [What's the difference between $\ell_1$-SVM, $\ell_2$-SVM and LS-SVM loss functions?](https://stats.stackexchange.com/q/255486/12359)
- To deal with an unbalanced dataset:
 - [Best way to handle unbalanced multiclass dataset with SVM](https://stats.stackexchange.com/q/20948/12359)
 - [A priori selection of SVM class weights](https://stats.stackexchange.com/a/24969/12359)
- [How does one interpret SVM feature weights?](https://stats.stackexchange.com/q/39243/12359)
- [Interpretating the C value in a linear SVM](https://stats.stackexchange.com/q/108639/12359)
- [Generalization bounds on SVM](https://stats.stackexchange.com/q/259788/12359)
- [General formula for the VC Dimension of a SVM](https://stats.stackexchange.com/q/255301/12359)
- [What does the "machine" in "support vector machine" and "restricted Boltzmann machine" mean?](https://stats.stackexchange.com/q/261041/12359)
- [How are SVMs = Template Matching?](https://stats.stackexchange.com/q/263587/12359)
- [Single layer NeuralNetwork with ReLU activation equal to SVM?](https://stats.stackexchange.com/q/190883/12359)
- [Comparing SVM and logistic regression](https://stats.stackexchange.com/q/95340/12359)

Other links:

-  [Least squares support vector machine](https://en.wikipedia.org/wiki/Least_squares_support_vector_machine)

----------
References:

- {1} Chapelle, Olivier. "Training a support vector machine in the primal." Neural computation 19, no. 5 (2007): 1155-1178. https://scholar.google.com/scholar?cluster=469291847682573606&hl=en&as_sdt=0,22 ; http://www.chapelle.cc/olivier/pub/neco07.pdf

## Code sample 1: SVM using cvxopt

```python
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import pickle
from cvxopt import matrix, solvers
```

```python
# Generating the data

DIM = 2
COLORS = ['red', 'blue']

# 2-D mean of ones
M1 = np.ones(DIM, )

# 2-D mean of threes
M2 = 3*np.ones(DIM, )

# 2-D covariance of 0.3
C1 = np.diag(0.3*np.ones(DIM, ))

# 2-D covariance of 0.2
C2 = np.diag(0.2*np.ones(DIM, ))

def generate_gaussian(m , c, num):
    return np.random.multivariate_normal(m, c, num)

def plot_data_with_labels(x, y):
    unique = np.unique(y)
    for li in range(len(unique)):
        x_sub = x[y == unique[li]]
        plt.scatter(x_sub[:, 0], x_sub[:, 1], c = COLORS[li])
        plt.title('Data generated from 2 gaussians')
    plt.show()

NUM = 50

if __name__ == '__main__':
    # generate 50 points from gaussian 1
    x1 = generate_gaussian(M1, C1, NUM)

    # labels
    y1 = np.ones(x1.shape[0], )

    # generate 50 points from gaussian 2
    x2 = generate_gaussian(M2, C2, NUM)
    y2 = -np.ones(x2.shape[0], )

    # join
    x = np.concatenate((x1, x2), axis = 0)
    y = np.concatenate((y1, y2), axis = 0)

    print(f" x: {x.shape} and y: {y.shape}")

    plot_data_with_labels(x, y)

    with open('gaussiandata.pickle', 'wb') as f:
        pickle.dump((x,y), f)
```

### Fitting an SVM

```python
def fit(x, y):
    NUM = x.shape[0]
    DIM = x.shape[1]
    # we'll solve the dual
    # obtain the kernel
    K = y[:, None] * x
    K = np.dot(K, K.T)
    P = matrix(K)
    q = matrix(-np.ones((NUM, 1)))
    G = matrix(-np.eye(NUM))
    h = matrix(np.zeros(NUM))
    A = matrix(y.reshape(1, -1))
    b = matrix(np.zeros(1))
    solvers.options['show_progress'] = False
    sol = solvers.qp(P, q, G, h, A, b)
    alphas = np.array(sol['x'])
    return alphas
```

Note that `sol['x']` contains the x that was part of cvxopt interface - it is our α. Now if you are familiar with SVMs, you will know that only a few of the alphas should be non-zero and they will be our support vectors. Using these alphas, we can obtain w and b from our original SVM problem. Once we do that, they will together define the decision boundary. w is equal to Σiαiyixi and b is equal to yi−wTxi for any i such that αi>0. So we do the following to obtain them.

```python
with open('gaussiandata.pickle', 'rb') as f:
    data = pickle.load(f)

x, y = data[0], data[1]

# fit svm classifier

alphas = fit(x, y)

# get weights
w = np.sum(alphas * y[:, None] * x, axis = 0)

# get bias
cond = (alphas > 1e-4).reshape(-1)
b = y[cond] - np.dot(x[cond], w)
bias = b[0]
```

Finally, we’ll plot the decision boundary for good visualizaiton. Since it will be a line in this case, we need to obtain the slope and intercept of the line from the weights and bias.

```python
def plot_data_with_labels(x, y, ax):
    unique = np.unique(y)
    for li in range(len(unique)):
        x_sub = x[y == unique[li]]
        ax.scatter(x_sub[:, 0], x_sub[:, 1], c = COLORS[li])

def plot_separator(ax, w, b):
    slope = -w[0] / w[1]
    intercept = -b / w[1]
    x = np.arange(0, 6)
    ax.plot(x, x * slope + intercept, 'k-')

# # normalize
# norm = np.linalg.norm(w)
# w, bias = w / norm, bias / norm

fig, ax = plt.subplots()
plot_separator(ax, w, bias)
plot_data_with_labels(x, y, ax)
plt.show()
```

## Code sample 2: SVM using cvxopt

**We introduce an additional variable stickily for convenience. The variable y will be equal to positive one for all positive samples and negative one for all negative samples.**

**Next, we need to address the process by which we go about maximizing the margin. To get an equation for the width of the margin, we subtract the first support vector from the one below it and the multiply the result by the unit vector of w which is always perpendicular to the decision boundary.**

**Using the constraints from above and a bit of algebra, we get the following.**

**Therefore, in order to select the optimal decision boundary, we must maximize the equation we just computed.**

**Now, in most machine learning algorithms, we’d use something like gradient descent to minimize said function, however, for support vector machines, we use the Lagrangian. In essence, using Lagrangian, we can solve for the global minimum like we’d do in high school level calculus (i.e. take the derivative of the function and make it equal to zero). The Lagrange tells us to subtract the cost function by the summation over all the constraints where each of those constraints will be multiplied by some constant alpha (normally written as lambda for the Lagrangian).**

**Then, we perform some more algebra, plugging the equations we found in the previous step back into the original equation.**

**Before we can proceed any further, we need to express the equation in terms of matrices instead of summations. The reason being, the** `qp` **function from the CVXOPT library, which we’ll use to solve the Lagrangian, accepts very specific arguments. Thus, we need to go from:**

```python
import numpy as np
import cvxopt
from sklearn.datasets.samples_generator import make_blobs
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
from sklearn.svm import LinearSVC
from matplotlib import pyplot as plt
```

    /home/abhishek/anaconda3/envs/tf2/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.datasets.samples_generator module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.datasets. Anything that cannot be imported from sklearn.datasets is now part of the private API.
      warnings.warn(message, FutureWarning)

```python
class SVM:
    def fit(self, X, y):
        n_samples, n_features = X.shape

        # P = X^T.X
        K = np.zeros((n_samples, n_samples))
        for i in range(n_samples):
            for j in range(n_samples):
                K[i,j] = np.dot(X[i], X[j])

        P = cvxopt.matrix(np.outer(y,y) * K)

        # q = -1 (1xN)
        q = cvxopt.matrix(np.ones(n_samples) * -1)

        # A = y^T
        A = cvxopt.matrix(y, (1, n_samples))

        # b=0
        b = cvxopt.matrix(0.0)

        # G = -1(NxN)
        G = cvxopt.matrix(np.diag(np.ones(n_samples) * -1))

        # h = 0 (1xN)
        h = cvxopt.matrix(np.zeros(n_samples))

        solution = cvxopt.solvers.qp(P, q, G, h, A, b)

        # Lagrange multipliers
        a = np.ravel(solution['x'])

        # Lagrange have non zero lagrange multipliers
        sv = a > 1e-5
        ind = np.arange(len(a))[sv]
        self.a = a[sv]
        self.sv = X[sv]
        self.sv_y = y[sv]

        # Intercept
        self.b = 0
        for n in range(len(self.a)):
            self.b += self.sv_y[n]
            self.b -= np.sum(self.a * self.sv_y * K[ind[n], sv])
        self.b /= len(self.a)

        # Weights
        self.w = np.zeros(n_features)
        for n in range(len(self.a)):
            self.w += self.a[n] * self.sv_y[n] * self.sv[n]

    def project(self, X):
        return np.dot(X, self.w) + self.b

    def predict(self, X):
        return np.sign(self.project(X))
```

To keep things simple, we’ll use the scikit-learn library to generate linearly separable data. We label the negative samples as -1 instead of 0. cvxopt expects the data to be in a specific format which is why we take an intermediate step.

```python
X, y = make_blobs(n_samples=250, centers=2,
                  random_state=0, cluster_std=0.60)
y[y == 0] = -1

tmp = np.ones(len(X))
y = tmp * y
```

```python
plt.scatter(X[:, 0], X[:, 1], c=y, cmap='winter')
```

```python
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
```

```python
svm = SVM()
svm.fit(X_train, y_train)
```

         pcost       dcost       gap    pres   dres
     0: -1.8226e+01 -3.4458e+01  6e+02  2e+01  2e+00
     1: -2.5252e+01 -1.8773e+01  2e+02  9e+00  7e-01
     2: -5.3459e+01 -3.2711e+01  2e+02  7e+00  6e-01
     3: -7.8360e+01 -2.6482e+01  1e+02  4e+00  3e-01
     4: -5.6818e+00 -5.1750e+00  1e+01  2e-01  1e-02
     5: -3.6906e+00 -4.1082e+00  4e-01  1e-15  8e-15
     6: -4.0061e+00 -4.0104e+00  4e-03  1e-15  6e-15
     7: -4.0094e+00 -4.0094e+00  4e-05  2e-15  7e-15
     8: -4.0094e+00 -4.0094e+00  4e-07  1e-17  5e-15
    Optimal solution found.

```python
def f(x, w, b, c=0):
    return (-w[0] * x - b + c) / w[1]

plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap='winter')

# w.x + b = 0
a0 = -4; a1 = f(a0, svm.w, svm.b)
b0 = 4; b1 = f(b0, svm.w, svm.b)
plt.plot([a0,b0], [a1,b1], 'k')

# w.x + b = 1
a0 = -4; a1 = f(a0, svm.w, svm.b, 1)
b0 = 4; b1 = f(b0, svm.w, svm.b, 1)
plt.plot([a0,b0], [a1,b1], 'k--')

# w.x + b = -1
a0 = -4; a1 = f(a0, svm.w, svm.b, -1)
b0 = 4; b1 = f(b0, svm.w, svm.b, -1)
plt.plot([a0,b0], [a1,b1], 'k--')
```

```python
y_pred = svm.predict(X_test)
confusion_matrix(y_test, y_pred)
```

    array([[29,  0],
           [ 0, 34]])

## Code sample 3: SVM using numpy at best

The **SVM (Support Vector Machine)** is a supervised machine learning algorithm typically used for binary classification problems. It’s trained by feeding a dataset with labeled examples (xᵢ, yᵢ). For instance, if your examples are email messages and your problem is spam detection, then:

- An example email message xᵢ is defined as an n-dimensional feature vector that can be plotted on n-dimensional space.
- The feature vector, as the name explains, contains features (eg. word count, link count, etc.) of your email message in numerical form
- Each feature vector is labeled with a class yᵢ
- The class yᵢ can either be a +ve or -ve (eg. spam=1, not-spam=-1)

Using this dataset the algorithm finds a hyperplane (or decision boundary) which should ideally have the following properties:

- It creates separation between examples of two classes with a maximum margin
- Its equation (w.x + b = 0) yields a value ≥ 1 for examples from+ve class and ≤-1 for examples from -ve class

How does it find this hyperplane? By finding the optimal values w* (weights/normal) and b* (intercept) which define this hyperplane. The optimal values are found by minimizing a cost function. Once the algorithm identifies these optimal values, the SVM model f(x) is then defined as shown below:

    f(x) = sign(w*.x + b*)

```python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

import statsmodels.api as sm  # for finding the p-value

from sklearn.preprocessing import MinMaxScaler  # for normalization
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, recall_score, precision_score
from sklearn.utils import shuffle
```

```python
# Reading the data

# The features in the dataset are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass.
# They describe the characteristics of the cell nuclei present in the image. Based on these features we will
# train our SVM model to detect if the mass is benign B (generally harmless) or malignant M (cancerous).

data = pd.read_csv('data.csv')
data
```

      id
      diagnosis
      radius_mean
      texture_mean
      perimeter_mean
      area_mean
      smoothness_mean
      compactness_mean
      concavity_mean
      concave points_mean
      ...
      texture_worst
      perimeter_worst
      area_worst
      smoothness_worst
      compactness_worst
      concavity_worst
      concave points_worst
      symmetry_worst
      fractal_dimension_worst
      Unnamed: 32

      0
      842302
      M
      17.99
      10.38
      122.80
      1001.0
      0.11840
      0.27760
      0.30010
      0.14710
      ...
      17.33
      184.60
      2019.0
      0.16220
      0.66560
      0.7119
      0.2654
      0.4601
      0.11890
      NaN

      1
      842517
      M
      20.57
      17.77
      132.90
      1326.0
      0.08474
      0.07864
      0.08690
      0.07017
      ...
      23.41
      158.80
      1956.0
      0.12380
      0.18660
      0.2416
      0.1860
      0.2750
      0.08902
      NaN

      2
      84300903
      M
      19.69
      21.25
      130.00
      1203.0
      0.10960
      0.15990
      0.19740
      0.12790
      ...
      25.53
      152.50
      1709.0
      0.14440
      0.42450
      0.4504
      0.2430
      0.3613
      0.08758
      NaN

      3
      84348301
      M
      11.42
      20.38
      77.58
      386.1
      0.14250
      0.28390
      0.24140
      0.10520
      ...
      26.50
      98.87
      567.7
      0.20980
      0.86630
      0.6869
      0.2575
      0.6638
      0.17300
      NaN

      4
      84358402
      M
      20.29
      14.34
      135.10
      1297.0
      0.10030
      0.13280
      0.19800
      0.10430
      ...
      16.67
      152.20
      1575.0
      0.13740
      0.20500
      0.4000
      0.1625
      0.2364
      0.07678
      NaN

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      564
      926424
      M
      21.56
      22.39
      142.00
      1479.0
      0.11100
      0.11590
      0.24390
      0.13890
      ...
      26.40
      166.10
      2027.0
      0.14100
      0.21130
      0.4107
      0.2216
      0.2060
      0.07115
      NaN

      565
      926682
      M
      20.13
      28.25
      131.20
      1261.0
      0.09780
      0.10340
      0.14400
      0.09791
      ...
      38.25
      155.00
      1731.0
      0.11660
      0.19220
      0.3215
      0.1628
      0.2572
      0.06637
      NaN

      566
      926954
      M
      16.60
      28.08
      108.30
      858.1
      0.08455
      0.10230
      0.09251
      0.05302
      ...
      34.12
      126.70
      1124.0
      0.11390
      0.30940
      0.3403
      0.1418
      0.2218
      0.07820
      NaN

      567
      927241
      M
      20.60
      29.33
      140.10
      1265.0
      0.11780
      0.27700
      0.35140
      0.15200
      ...
      39.42
      184.60
      1821.0
      0.16500
      0.86810
      0.9387
      0.2650
      0.4087
      0.12400
      NaN

      568
      92751
      B
      7.76
      24.54
      47.92
      181.0
      0.05263
      0.04362
      0.00000
      0.00000
      ...
      30.37
      59.16
      268.6
      0.08996
      0.06444
      0.0000
      0.0000
      0.2871
      0.07039
      NaN

569 rows × 33 columns

```python
def init():
    data = pd.read_csv('data.csv')

    # SVM accepts only numerical values so we transform the categories M and B into -1 and 1

    diagnosis_map = {'M':1, 'B':-1}

    data['diagnosis'] = data['diagnosis'].map(diagnosis_map)

    # Dropping last column and first column
    data.drop(data.columns[[-1, 0]], axis=1, inplace=True)

    X = data.iloc[:, 1:] # iloc gets rows (or columns) at particular positions in the index (so it only takes integers).
    Y = data.loc[:, 'diagnosis'] # loc gets rows (or columns) with particular labels from the index.

    remove_correlated_features(X)
    remove_less_significant_features(X, Y)

    # normalize the features using MinMaxScalar from sklearn.preprocessing
    X_normalized = MinMaxScaler().fit_transform(X.values)
    X = pd.DataFrame(X_normalized)

    # first insert 1 in every row for intercept b
    X.insert(loc=len(X.columns), column='intercept', value=1)

    print("Splitting dataset into train and test sets...")
    X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42)

    # train the model
    print("Started Training")

    W = sgd(X_train.to_numpy(), y_train.to_numpy())

    print("Training finished.")
    print("Weights are: {}".format(W))

    # testing the model on test set
    y_test_predicted = np.array([])
    for i in range(X_test.shape[0]):
        yp = np.sign(np.dot(W, X_test.to_numpy()[i])) #model
        y_test_predicted = np.append(y_test_predicted, yp)

    print("Accuracy on test dataset: {}".format(accuracy_score(y_test.to_numpy(), y_test_predicted)))
    print("Recall on test dataset: {}".format(recall_score(y_test.to_numpy(), y_test_predicted)))
    print("Precision on test dataset: {}".format(recall_score(y_test.to_numpy(), y_test_predicted)))
```

### Cost Function

One of the building blocks of every machine learning algorithm, it’s the function we try to minimize or maximize to achieve our objective.

**What’s our objective in SVM?** Our objective is to find a hyperplane that separates +ve and -ve examples with the largest margin while keeping the misclassification as low as possible (see Figure 3).

**How do we achieve this objective?** We will minimize the cost/objective function shown below:

You might have seen another version of a cost function that looks like this:

In this cost function, λ is essentially equal to 1/C and has the opposite effect i.e larger λ gives a wider margin and vice versa. We can use any of the above cost functions keeping in mind what each regularization parameter (C and λ) does and then tuning them accordingly. Let’s see how can we calculate the total cost as given in (1) and then we will move on to its gradient which will be used in the training phase to minimize it:

```python
def compute_cost(W, X, Y):
    # calculate hinge loss
    N = X.shape[0]
    distances = 1 - Y * (np.dot(X, W))
    distances[distances < 0] = 0  # equivalent to max(0, distance)
    hinge_loss = regularization_strength * (np.sum(distances) / N)

    # calculate cost
    cost = 1 / 2 * np.dot(W, W) + hinge_loss
    return cost
```

As you might have noticed that the intercept term b is missing. That’s because we have pushed it into the weight vector like this:

### That’s why we added an extra column with all 1s before splitting our dataset.

### Gradient of Cost Function

```python
#### ERROR: https://towardsdatascience.com/svm-implementation-from-scratch-python-2db2fc52e5c2

def calculate_cost_gradient(W, X_batch, Y_batch):
    # if only one example is passed (eg. in case of SGD)
    if type(Y_batch) == np.float64:
        Y_batch = np.array([Y_batch])
        X_batch = np.array([X_batch])  # gives multidimensional array

    distance = 1 - (Y_batch * np.dot(X_batch, W))
#     print(distance.shape, Y_batch.shape, X_batch.shape, W.shape)
    dw = np.zeros(len(W))

#     for ind, d in enumerate(distance):
#         if max(0, d) == 0:
#             di = W
#         else:
#             di = W - (regularization_strength * Y_batch[ind] * X_batch[ind])
#         dw += di
#     dw = dw/len(Y_batch)  # average

    di = W - (regularization_strength * Y_batch * X_batch)
    dw += di
    return dw
```

### Training model using SGD

Why do we minimize the cost function? Because the cost function is essentially a measure of how bad our model is doing at achieving the objective. If you look closely at J(w), to find it’s minimum, we have to:

1. Minimize ∣∣w∣∣² which maximizes margin (2/∣∣w∣∣)
2. Minimize the sum of hinge loss which minimizes misclassifications.

**How do we minimize it?**

Well, there are multiple ways but the one we will use is called Stochastic Gradient Descent or SGD.

**Gradient descent algorithm works as follows**

- Find the gradient of cost function i.e. ∇J(w’)
- Move opposite to the gradient by a certain rate i.e. w’ = w’ — ∝(∇J(w’))
- Repeat step 1–3 until convergence i.e we found w’ where J(w) is smallest

Why does it move opposite to the direction of the gradient? Because gradient is the direction of the fastest increase of the function. We need to move opposite to that direction to minimize our function J(w). Hence, the word “descent” in Gradient Descent is used.

In typical gradient descent (a.k.a vanilla gradient descent) the step 1 above is calculated using all the examples (1…N). In SGD, however, only 1 example is used at a time.

```python
def sgd(features, outputs):
    max_epochs = 5000
    weights = np.zeros(features.shape[1])
    nth = 0
    prev_cost = float("inf")
    cost_threshold = 0.01  # in percent
    # stochastic gradient descent
    for epoch in range(1, max_epochs):
        # shuffle to prevent repeating update cycles
        X, Y = shuffle(features, outputs)
        for ind, x in enumerate(X):
            ascent = calculate_cost_gradient(weights, x, Y[ind])
            weights = weights - (learning_rate * ascent)

        # convergence check on 2^nth epoch
        if epoch == 2 ** nth or epoch == max_epochs - 1:
            cost = compute_cost(weights, features, outputs)
            print("Epoch is: {} and Cost is: {}".format(epoch, cost))
            # stoppage criterion
            if abs(prev_cost - cost) < cost_threshold * prev_cost:
                return weights
            prev_cost = cost
            nth += 1
    return weights
```

```python
# After training the model using SGD we finally got the optimal weights w* which defines the best
# possible hyperplane separating two classes. Let’s test our model using this hyperplane.

# Calling init
regularization_strength = 10000
learning_rate = 0.000001
init()
```

    Splitting dataset into train and test sets...
    Started Training
    Epoch is: 1 and Cost is: 6177.387545790943
    Epoch is: 2 and Cost is: 8508.5742014718
    Epoch is: 4 and Cost is: 13366.440117739556
    Epoch is: 8 and Cost is: 23129.48617693818
    Epoch is: 16 and Cost is: 42741.3988732552
    Epoch is: 32 and Cost is: 82271.39690213982
    Epoch is: 64 and Cost is: 162350.18245808998
    Epoch is: 128 and Cost is: 326215.83550679113
    Epoch is: 256 and Cost is: 666380.748971567
    Epoch is: 512 and Cost is: 1380234.3772538519
    Epoch is: 1024 and Cost is: 2844391.8199767894
    Epoch is: 2048 and Cost is: 5490365.834151834
    Epoch is: 4096 and Cost is: 8880091.359998146
    Epoch is: 4999 and Cost is: 9714734.506602418
    Training finished.
    Weights are: [  261.45950729  -219.19511982   298.59202834   406.50137946
      -489.56947106   227.03355442   621.24824322   721.16116254
      -456.46490048  -640.58892429   236.94476789  -432.88724527
       218.87276277   251.72025442  -466.07325606   -97.30138861
       -39.33732165  -127.15359129  -407.28222601  -198.38267782
       454.62171315  -176.58674761   464.23134369   481.19774655
      -351.46930179   262.39492856   435.44746928   628.50172314
      -134.67721697  -118.88937352 -2306.97284959]
    Accuracy on test dataset: 0.6491228070175439
    Recall on test dataset: 0.06976744186046512
    Precision on test dataset: 0.06976744186046512

### Feature Selection With Correlation & P-values

Feature selection encompasses statistical techniques that help in filtering irrelevant or redundant features. Correlation & p-values are among these statistical techniques. Using them we will select a subset of relevant and important features from our original set of features

**What is correlation?** Correlation is a degree of linear dependence (or linear relationship) between two variables. Two features are said to be correlated if the values of one feature can be explained by some linear relationship of the second feature. The degree of this relationship is given by the correlation coefficient (or “r”). It ranges from -1.0 to +1.0. The closer r is to +1 or -1, the more closely the two variables are related.

**Why do we remove one of the correlated features?** There are multiple reasons but the simplest of them is that correlated features almost have the same effect on the dependent variable. Moreover, correlated features won’t improve our model and would most probably worsen it, therefore we are better off using only one of them. After all, fewer features result in improved learning speed and a simpler model (model with fewer features).

**What are p-values?** In the context of feature selection, p-values help us find the features which are most significant in explaining variation in the dependent variable (y).

A feature with a low p-value has more significance and a feature with a high p-value has less significance in explaining the variation. Usually, we set a Significance Level SL (threshold) and if a feature has a p-value above this level it is discarded.

**Why do we remove features with high p-values?** Because they don’t tell much about the behavior of the dependent variable. So, why keep them and unnecessarily increase the complexity of our model when they are not helping us in predicting the result.

We have two functions named remove_correlated_features() remove_less_significant_features() for removing highly correlated features and less significant features (using p-values and backward elimination) respectively:

```python
def remove_correlated_features(X):
    corr_threshold = 0.9
    corr = X.corr
    drop_columns = np.full(corr.shape[0], False, dtype = bool)
    for i in range(corr.shape[0]):
        for j in range(i+1, corr.shape[0]):
            if corr.iloc[i, j] >= corr.threshold:
                drop_columns[j] = True
    columns_dropped = X.columns[drop_columns]
    X.drop(columns_dropped, axis=1, inplace=True)
    return columns_dropped
```

```python
def remove_less_significant_features(X, Y):
    sl = 0.05
    regression_ols = None
    columns_dropped = np.array([])
    for itr in range(0, len(X.columns)):
        regression_ols = sm.OLS(Y, X).fit()
        max_col = regression_ols.pvalues.idxmax()
        max_val = regression_ols.pvalues.max()
        if max_val > sl:
            X.drop(max_col, axis='columns', inplace=True)
            columns_dropped = np.append(columns_dropped, [max_col])
        else:
            break
    regression_ols.summary()
    return columns_dropped
```

```python

```

```python

```

### REFERENCES:
- [In depth SVM](https://jakevdp.github.io/PythonDataScienceHandbook/05.07-support-vector-machines.html)
- [SVM example](https://towardsdatascience.com/support-vector-machine-python-example-d67d9b63f1c8)
- [SVM python](https://towardsdatascience.com/support-vector-machine-python-example-d67d9b63f1c8)
- [Support Vector Machine: Python implementation using CVXOPT](https://xavierbourretsicotte.github.io/SVM_implementation.html)
- [SVM CVXOPT](http://goelhardik.github.io/2016/11/28/svm-cvxopt/)
- [How does a Support Vector Machine (SVM) work?](https://stats.stackexchange.com/questions/23391/how-does-a-support-vector-machine-svm-work/353605#353605)
- [Help me understand Support Vector Machines](https://stats.stackexchange.com/questions/3947/help-me-understand-support-vector-machines)
- [MIT Prof. Patrick Winston](https://www.youtube.com/watch?v=_PwhiWxHK8o&feature=youtu.be)
- [How can SVM 'find' an infinite feature space where linear separation is always possible?](https://stats.stackexchange.com/questions/80398/how-can-svm-find-an-infinite-feature-space-where-linear-separation-is-always-p/168309#168309)
- [Khan academy Lagrange multiplier](https://www.youtube.com/watch?v=hQ4UNu1P2kw)
- [Support Vector Machine — Using Numpy](https://medium.com/@saishruthi.tn/support-vector-machine-using-numpy-846f83f4183d)
- [Mathematics behind SVM(Support Vector Machine)](https://towardsdatascience.com/mathematics-behind-svm-support-vector-machines-84742ddda0ca)
- [Math behind SVM (Support Vector Machine)](https://medium.com/@ankitnitjsr13/math-behind-support-vector-machine-svm-5e7376d0ee4d)
- [SVM From Scratch — Python](https://towardsdatascience.com/svm-implementation-from-scratch-python-2db2fc52e5c2)

```python

```

ML-practice/Supervised Learning/Support Vector Machines (SVM)/Support Vector Machines.ipynb


```
# default_exp
```

```
#hide
#all_slow
```

# Distribution of maximum for 1-D prior example
> Investigation into the distribution of the maximum for a 1-D toy example.

```
#hide
from dolfin import *
import numpy as np
import ot
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
# import required functions from oneDim
from statFEM.oneDim import mean_assembler, kernMat, cov_assembler, sample_gp
from scipy.stats import linregress
from scipy import integrate
from scipy.linalg import sqrtm
from tqdm.notebook import tqdm
import sympy; sympy.init_printing()
# code for displaying matrices nicely
def display_matrix(m):
    display(sympy.Matrix(m))
```

Let's first import the `wass` function created in the sub-module `maxDist`.

```
from statFEM.maxDist import wass
```

# Sampling from true maxiumum distribution in 1D

We will be interested in the toy example introduced in oneDim, however we will use different values for the parameters in the distribution of $f$. To obtain a sample for the maximum from the true prior we must first sample trajectories from the true prior.

```
# set up true mean
def m_u(x):
    return 0.5*x*(1-x)
```

```
# set up kernel functions for forcing f
l_f = 0.01
σ_f = 0.1

def c_f(x,y):
    return (σ_f**2)*np.exp(-(x-y)**2/(2*(l_f**2)))

# translation invariant form of c_f
def k_f(x):
    return (σ_f**2)*np.exp(-(x**2)/(2*(l_f**2)))
```

```
# set up true cov function for solution
# compute inner integral over t
def η(w,y):
    I_1 = integrate.quad(lambda t: t*c_f(w,t),0.0,y)[0]
    I_2 = integrate.quad(lambda t: (1-t)*c_f(w,t),y,1.0)[0]
    return (1-y)*I_1 + y*I_2

# use this function eta and compute the outer integral over w
def c_u(x,y):
    I_1 = integrate.quad(lambda w: (1-w)*η(w,y),x,1.0)[0]
    I_2 = integrate.quad(lambda w: w*η(w,y),0.0,x)[0]
    return x*I_1 + (1-x)*I_2
```

```
%%time
n_sim = 1000
grid = np.linspace(0,1,100)
np.random.seed(235)
u_sim = sample_gp(n_sim, m_u, c_u, grid, par = True, trans = False, tol = 1e-8)
```

    CPU times: user 871 ms, sys: 151 ms, total: 1.02 s
    Wall time: 1min 3s

We now plot the sampled trajectories from the true prior.

```
#hide_input
plt.rcParams['figure.figsize'] = (10,6)
plt.plot(grid,u_sim)
plt.xlabel('$x$')
plt.ylabel('$u$')
plt.title('Trajectories from the true prior')
plt.grid()
plt.show()
```

> Note: The maximum is __not__ always located at $x=0.5$. We will demonstrate this by plotting the $\operatorname{argmax}$ for each simulated curve below.

```
#hide_input
plt.plot(np.arange(1,n_sim + 1),grid[u_sim.argmax(axis=0)])
plt.hlines(0.5,1,n_sim,colors='red',linestyles='dashed')
plt.grid()
plt.xlabel('Sample number')
plt.ylabel('$\operatorname{argmax}$')
plt.title('Location of $\operatorname{argmax}$ for each simulated curve')
plt.show()
```

Let's now get the maximum value for each simulated curve and store this in a variable `max_true` for future use. We plot the histogram for these maximum values.

```
max_true = u_sim.max(axis=0)
```

```
#hide_input
sns.displot(max_true,kde=True,stat="density")
plt.xlabel('$\max$')
plt.title('Histogram of maximum for true prior')
plt.show()
```

# Sampling from statFEM prior maxiumum distribution in 1D

We now create a function to utilise FEniCS to draw trajectories from the statFEM prior.

```
# create statFEM sampler function
def statFEM_sampler(n_sim, grid, h, f_bar, k_f, par = False, trans = True, tol=1e-9):
    # get length of grid
    d = len(grid)

    # get size of FE mesh
    J = int(np.round(1/h))

    # get statFEM mean function
    μ_func = mean_assembler(h, f_bar)

    # evaluate this on the grid
    μ = np.array([μ_func(x) for x in grid]).reshape(d,1)

    # get statFEM cov mat on grid
    Σ = cov_assembler(J, k_f, grid, parallel=par, translation_inv=trans)

    # construct the cholesky decomposition Σ = GG^T
    # we add a small diagonal perturbation to Σ to ensure it
    # strictly positive definite
    G = np.linalg.cholesky(Σ + tol * np.eye(d))

    # draw iid standard normal random vectors
    Z = np.random.normal(size=(d,n_sim))

    # construct samples from GP(m,k)
    Y = G@Z + np.tile(μ,n_sim)

    # return the sampled trajectories
    return Y
```

Let's test this function out by using it to obtain samples from the statFEM prior for a particular value of $h$.

```
h = 0.1
f_bar = Constant(1.0)
np.random.seed(2123)
u_sim_statFEM = statFEM_sampler(n_sim, grid, h, f_bar, k_f)
```

Let's plot the above trajectories.

```
#hide_input
plt.rcParams['figure.figsize'] = (10,6)
plt.plot(grid,u_sim_statFEM)
plt.grid()
plt.xlabel('$x$')
plt.ylabel('$u$')
plt.title('Trajectories from statFEM prior with $h = $ %0.1f' % h)
plt.show()
```

Let's now get the maximum value for each simulated curve and store this in a variable `max_statFEM` for future use. We then plot the histograms for these maximum values together with those from the true prior.

```
max_statFEM = u_sim_statFEM.max(axis=0)
```

```
#hide_input
max_data = pd.DataFrame(data={'True': max_true, 'statFEM': max_statFEM})
sns.displot(max_data,kde=True,stat="density")
plt.xlabel('$\max$')
plt.title('Histograms of maximums for true and statFEM priors')
plt.show()
```

Our `wass` function requires the number of bins to be specified. Let's investigate how varying this parameter affects the computed distance between the true maximums and our statFEM maximums.

```
n_bins = np.arange(10,200,10)
wass_bin_dat = [wass(max_true,max_statFEM,n) for n in n_bins]
```

```
#hide_input
plt.plot(n_bins,wass_bin_dat)
plt.grid()
plt.xlabel('number of bins')
plt.ylabel('2-Wasser distance')
plt.title('Plot showing how number of bins influences the computed distance')
plt.show()
```

From the plot above we see that the distance flattens after around $50$ bins. We choose to take `n_bins=100` for the rest of this example.

We now set up a range of $h$-values to use for the statFEM prior.

```
# set up range of h values to use
h_range_tmp = np.linspace(0.25,0.02,100)
h_range = 1/np.unique(np.round(1/h_range_tmp))
np.round(h_range,2)
```

    array([0.25, 0.2 , 0.17, 0.14, 0.12, 0.11, 0.1 , 0.09, 0.08, 0.08, 0.07,
           0.07, 0.06, 0.06, 0.06, 0.05, 0.05, 0.05, 0.05, 0.04, 0.04, 0.04,
           0.04, 0.03, 0.03, 0.03, 0.03, 0.02, 0.02, 0.02])

We now loop over these $h$-values, and for each value we simulate maximums from the statFEM prior and then compute the 2-Wasserstein distance between these maximums and those from the true prior.

```
%%time
errors = []
###################
n_bins = 100
##################
np.random.seed(3252)
for h in tqdm(h_range):
    # sample trajectories from statFEM prior for current h value
    sim = statFEM_sampler(n_sim,grid,h,f_bar,k_f)
    # get max
    max_sim = sim.max(axis=0)
    # compute error
    error = wass(max_true,max_sim,n_bins)
    # append to errors
    errors.append(error)
```

      0%|          | 0/30 [00:00<?, ?it/s]

    CPU times: user 1.62 s, sys: 3.71 s, total: 5.33 s
    Wall time: 776 ms

Let's now plot these errors against $h$.

```
#hide
errors = np.array(errors)
```

```
#hide_input
plt.plot(h_range,errors)
plt.grid()
plt.xlabel('h')
plt.ylabel('error')
plt.title('Plot of errors vs h for max')
plt.show()
```

We see that as $h$ decreases so too does the distance/error. Let's investigate the rate by plotting this data in log-log space and then estimating the slope of the line of best fit.

```
#hide_input
log_h_range = np.log(h_range)
log_errors = np.log(errors)

# perform linear regression to get line of best fit for the log scales:
lm = linregress(log_h_range,log_errors)
print("slope: %f    intercept: %f    r value: %f     p value: %f" % (lm.slope, lm.intercept, lm.rvalue, lm.pvalue))

# plot line of best fit with the data:
x = np.linspace(np.min(log_h_range),np.max(log_h_range),100)
plt.scatter(log_h_range,log_errors,label='data')
plt.plot(x,lm.intercept + lm.slope * x, 'r', label='fitted line')
# plt.plot(x,2+2*x,'--',c='black',label='reference line, $p=2$')
plt.plot(x,-1+1.35*x,'--',c='black',label='reference line, $p=1.35$')
plt.grid()
plt.legend()
plt.xlabel('$\log(h)$')
plt.ylabel('$\log(W)$')
plt.title('Plot of $\log(W)$ vs $\log(h)$ for maximum')
plt.savefig('1D_max_prior_results.png',dpi=300,bbox_inches='tight',facecolor="w")
plt.show()
```

We can see from the above plot that the slope is around $1.35$ - this differs from our theory showing the results for the $\max$ are out of the remit of our theory.

```

```


# Prisoners dilemma - exercises

1. Give the general definition for a Prisoners dilemma.

   Bookwork: https://vknight.org/gt/chapters/09/#Prisoners-Dilemma

2. Justify if the following games are Prisoners dilemmas or not:
   1. $
   A =
   \begin{pmatrix}
   3 & 0\\
   5 & 1
   \end{pmatrix}
   \qquad
   B =
   \begin{pmatrix}
   3 & 5\\
   0 & 1
   \end{pmatrix}
   $

      This is a Prisoners Dilemma: $(R, S, T, P) = (3, 1, 5, 0)$.

   2. $
   A =
   \begin{pmatrix}
   1 & -1\\
   2 & 0
   \end{pmatrix}
   \qquad
   B =
   \begin{pmatrix}
   1 & 2\\
   -1 & 0
   \end{pmatrix}
   $

      This is a Prisoners Dilemma: $(R, S, T, P) = (1, -1, 2, 0)$: $2>1>0>-1$ and $2\times 1 > 2 - 1$.

   3. $
   A =
   \begin{pmatrix}
   1 & -1\\
   2 & 0
   \end{pmatrix}
   \qquad
   B =
   \begin{pmatrix}
   3 & 5\\
   0 & 1
   \end{pmatrix}
   $

      This is not a Prisoner's Dilemma $A \ne B ^ T$

   4. $
   A =
   \begin{pmatrix}
   6 & 0\\
   12 & 1
   \end{pmatrix}
   \qquad
   B =
   \begin{pmatrix}
   6 & 12\\
   0 & 0
   \end{pmatrix}
   $

      This is not a Prisoner's Dilemma $A \ne B ^ T$

3. Obtain the Markov chain representation for a match between reactive strategies with the following vectors:

   Bookwork: this is a substitution exercise using https://vknight.org/gt/chapters/09/#Markov-chain-representation-of-a-Match-between-two-reactive-strategies

   1. $p=(1/2, 1/2)\qquad q=(1/2, 1/2)$

      $$M = \begin{pmatrix}
      1/4&1/4&1/4&1/4\\
      1/4&1/4&1/4&1/4\\
      1/4&1/4&1/4&1/4\\
      1/4&1/4&1/4&1/4
      \end{pmatrix}$$
   2. $p=(1/4, 1/2)\qquad q=(1/2, 1/4)$

      $$M = \begin{pmatrix}
      1/8&1/8&3/8&3/8\\
      1/4&1/4&1/4&1/4\\
      1/16&3/16&3/16&9/16\\
      1/8&3/8&1/8&3/8
      \end{pmatrix}$$

   3. $p=(1/3, 1/3)\qquad q=(2/3, 1/4)$

      $$M = \begin{pmatrix}
      2/9 & 1/9 & 4/9 & 2/9\\
      2/9 & 1/9 & 4/9 & 2/9\\
      1/12 & 1/4 & 1/6 & 1/2\\
      1/12 & 1/4 & 1/6 & 1/2
      \end{pmatrix}$$

4. Obtain the utilities for both players for the vectors of question 3.

   Bookwork: this is a substitution exercise using https://vknight.org/gt/chapters/09/#Theorem:-steady-state-probabilities-for-match-between-reactive-players

   1. $p=(1/2, 1/2)\qquad q=(1/2, 1/2)$ gives utilities: $(9/4, 9/4)$
   2. $p=(1/4, 1/2)\qquad q=(1/2, 1/4)$ gives utilities: $(536/289, 621/289)$
   3. $p=(1/3, 1/3)\qquad q=(2/3, 1/4)$ gives utilities: $(113/54, 49/27)$

Here is some python code that also carries out these calculations:

```python
import numpy as np
import itertools

def make_matrix(p, q):
    """
    Code to obtain Markov chain representation of match between two reactive players.
    """
    M = [[ele[0] * ele[1] for ele in itertools.product([player, 1 - player],
                                                       [opponent, 1 - opponent])]
         for opponent in q for player in p]
    return np.array(M)

def theoretic_steady_state(p, q):
    r_1 = p[0] - p[1]
    r_2 = q[0] - q[1]
    s_1 = (q[1] * r_1 + p[1]) / (1 - r_1 * r_2)
    s_2 = (p[1] * r_2 + q[1]) / (1 - r_1 * r_2)
    return np.array([s_1 * s_2, s_1 * (1 - s_2), (1 - s_1) * s_2, (1 - s_1) * (1 - s_2)])

def theoretic_utility(p, q, rstp=np.array([3, 0, 5, 1])):
    pi = theoretic_steady_state(p, q)
    return np.dot(pi, rstp)
```

```python
import sympy as sym
for p, q in [([sym.S(1) / 2, sym.S(1) / 2], [sym.S(1) / 2, sym.S(1) / 2]),
             ([sym.S(1) / 4, sym.S(1) / 2], [sym.S(1) / 2, sym.S(1) / 4]),
             ([sym.S(1) / 3, sym.S(1) / 3], [sym.S(2) / 3, sym.S(1) / 4])]:
    print("=====")
    print(p, q)
    print("gives:")
    print(make_matrix(p, q))
    print("With utility:", theoretic_utility(p, q), theoretic_utility(q, p))
```

    =====
    [1/2, 1/2] [1/2, 1/2]
    gives:
    [[1/4 1/4 1/4 1/4]
     [1/4 1/4 1/4 1/4]
     [1/4 1/4 1/4 1/4]
     [1/4 1/4 1/4 1/4]]
    With utility: 9/4 9/4
    =====
    [1/4, 1/2] [1/2, 1/4]
    gives:
    [[1/8 1/8 3/8 3/8]
     [1/4 1/4 1/4 1/4]
     [1/16 3/16 3/16 9/16]
     [1/8 3/8 1/8 3/8]]
    With utility: 536/289 621/289
    =====
    [1/3, 1/3] [2/3, 1/4]
    gives:
    [[2/9 1/9 4/9 2/9]
     [2/9 1/9 4/9 2/9]
     [1/12 1/4 1/6 1/2]
     [1/12 1/4 1/6 1/2]]
    With utility: 113/54 49/27

`5`. Assuming $p=(x, 1/2)$, find the optimal $x$ against the following players:

   A part of this question involves bookwork: this is a substitution exercise using https://vknight.org/gt/chapters/09/#Theorem:-steady-state-probabilities-for-match-between-reactive-players. This is a substitution exercise to obtain a formula for the utility of $p$ as a function of $x$.

   1. $q=(1, 0)$

      $$u(x)=\frac{(-10x + 4(x - 1)^2 + 13)}{(2x - 3)^2}$$

      The derivative of this function is given by:

      $$
      \frac{2(6x - 7)}{(2x - 3)^3}
      $$

      This derivative has zero for $x=7/6$ which is $>1$. Thus the utility is monotic increasing over the interval $[0, 1]$. We have (by substitution):

      $$u(0)=17/9\qquad u(1)=3$$

      Thus $u(x)$ is an increasing function so the optimal value of $x$ is $1$.

      Against a player that is unforgiving (reacts to defection with defection), given that our player will play randomly against a defection it is better to always cooperate.

   2. $q=(1/2, 1/2)$

      $$u(x)=-3x/4+21/8$$

      This is a decreasing function so the optimal value of $x$ is $0$.

      Against a random player (who takes no notice of what we do) it is better to defect.

Below is some code to verify this calculations.

```python
x = sym.Symbol("x")
for q in [(sym.S(1), sym.S(0)), (sym.S(1) / 2, sym.S(1) / 2)]:
    utility = theoretic_utility((x, sym.S(1) / 2), q)
    print(utility.simplify(), utility.subs({x: 0}), utility.subs({x: 1}), sym.diff(utility, x).simplify(), sym.solveset(sym.diff(utility, x), x))
```

    (-10*x + 4*(x - 1)**2 + 13)/(2*x - 3)**2 17/9 3 2*(6*x - 7)/(2*x - 3)**3 {7/6}
    -3*x/4 + 21/8 21/8 15/8 -3/4 EmptySet()


```python
from sympy import *
from sympy.solvers.inequalities import *
init_printing(use_latex='mathjax')
import numpy as np
```

## Аналитический расчет коэффициентов полинома

```python
a = symbols('a:6') # Коэффициенты
t = symbols('t')   # Время
s0, ds0, dds0 = symbols('s0 ds0 dds0') # Начальное состояние
s1, ds1, dds1 = symbols('s1 ds1 dds1') # Конечное состояние

# Выражения для s(t), ds(t)/dt, d2s(t)/dt^2
s = a[0]*t**5 + a[1]*t**4 + a[2]*t**3 + a[3]*t**2 + a[4]*t + a[5]
ds = diff(s, t)
dds = diff(ds, t)

# Левые части
ls0 = s.subs(t, 0) - s0
lds0 = ds.subs(t, 0) - ds0
ldds0 = dds.subs(t, 0) - dds0
ls1 = s - s1
lds1 = ds - ds1
ldds1 = dds - dds1

# Символьное решение системы уравнений
res = solve([ls0, lds0, ldds0, ls1, lds1, ldds1], a)
```

```python
# s(t)
s
```

$$a_{0} t^{5} + a_{1} t^{4} + a_{2} t^{3} + a_{3} t^{2} + a_{4} t + a_{5}$$

```python
# ds(t)/dt
ds
```

$$5 a_{0} t^{4} + 4 a_{1} t^{3} + 3 a_{2} t^{2} + 2 a_{3} t + a_{4}$$

```python
# d2s(t)/dt^2
dds
```

$$20 a_{0} t^{3} + 12 a_{1} t^{2} + 6 a_{2} t + 2 a_{3}$$

```python
{key: factor(res[key]) for key in res}
```

$$\left \{ a_{0} : - \frac{dds_{0} t^{2} - dds_{1} t^{2} + 6 ds_{0} t + 6 ds_{1} t + 12 s_{0} - 12 s_{1}}{2 t^{5}}, \quad a_{1} : \frac{3 dds_{0} t^{2} - 2 dds_{1} t^{2} + 16 ds_{0} t + 14 ds_{1} t + 30 s_{0} - 30 s_{1}}{2 t^{4}}, \quad a_{2} : - \frac{3 dds_{0} t^{2} - dds_{1} t^{2} + 12 ds_{0} t + 8 ds_{1} t + 20 s_{0} - 20 s_{1}}{2 t^{3}}, \quad a_{3} : \frac{dds_{0}}{2}, \quad a_{4} : ds_{0}, \quad a_{5} : s_{0}\right \}$$

```python
# Выведем значения коэффициентов в обычном виде, чтобы копипастнуть их в код
for coef in a:
    print(res[coef])
```

    (-12*s0 + 12*s1 + t**2*(-dds0 + dds1) - 6*t*(ds0 + ds1))/(2*t**5)
    (15*s0 - 15*s1 + t**2*(3*dds0 - 2*dds1)/2 + t*(8*ds0 + 7*ds1))/t**4
    (-20*s0 + 20*s1 + t**2*(-3*dds0 + dds1) - 4*t*(3*ds0 + 2*ds1))/(2*t**3)
    dds0/2
    ds0
    s0

## Попробуем оценить необходимое время
Просто воспользуемся равноускоренным движением

```python
v0, v1, s, a, t = symbols('v0 v1 s a t')
eq1 = v0 + a*t - v1
eq2 = v0*t + a*t**2/2 - s
res = solve([eq1, eq2], (a, t))
res
```

$$\left [ \left ( - \frac{\left(v_{0} - v_{1}\right) \left(v_{0} + v_{1}\right)}{2 s}, \quad \frac{2 s}{v_{0} + v_{1}}\right )\right ]$$

```python
eq1
```

$$a t + v_{0} - v_{1}$$

```python
eq2
```

$$\frac{a t^{2}}{2} - s + t v_{0}$$

```python
res
```

$$\left [ \left ( - \frac{\left(v_{0} - v_{1}\right) \left(v_{0} + v_{1}\right)}{2 s}, \quad \frac{2 s}{v_{0} + v_{1}}\right )\right ]$$

# WTF

```python
x = symbols('x')
solve_poly_inequalities((
    (Poly(x+2), ">="),
    (Poly(x**2-1), "<=")
))
```

$$\left[-2, \infty\right)$$

```python

```