***
# HCI Week 6 - Polarimetric Differential Imaging
*Matthew Kenworthy - Leiden Observatory*
***

For the last two weeks we have been focussing on coronagraphs.
This week you will be looking at imaging data taken with a polariser.

PLEASE 1: DO NOT EMAIL BACK THE FITS IMAGES YOU MAKE, ONLY THE .ipynb NOTEBOOK

PLEASE 2: Upload is by chaning the filename to: week06_[lastname].ipynb $\leftarrow$ This saves me
    $$\# \text{ students}_{\text{not using this format}} \times \text{time}_{\text{it takes you to only upload this notebook in this format}} + \text{time}_{\text{it takes me to delete additional files}} $$

PLEASE 3: Do not forget to add axis labels, title, ect to ALL the plots that you have to make!

The data is of a star surrounded by a nearly face-on dust disk. You will carry out difference imaging to remove the unpolarised stellar flux and let the polarised flux shine through. The disk can barely be seen in the intensity image, but is clearly seen in the degree of linear polarization image.

The polarization state of photons encodes information about the scattering of the photon from dust in circumstellar disks, and from aerosols in planetary atmospheres.

The polarization data is from NaCo, an imaging camera that sits at the Nasmyth focus of the Very Large Telescope (VLT), and the data is taken at a wavelength of 2.15 microns (K band). The data has been kindly provided by Jos de Boer at Leiden Observatory.

NaCo takes two images of an astronomical object through a Wollaston prism, which splits incoming light into two orthogonal polarized images which we will refer to as Left (L) and Right (R).

A Half-Wave Plate is rotated so that **4** different polarization angles are imaged in the sequence 0, 45, 90 and 135 degrees, and two polarized images are taken each time.

The large number of images are taken so that time-varying and detector-varying effects can be removed. You will look at single differencing and double differencing.

In effect, we are determining 3 numerical values for each pixel on the detector - representing $I, Q, U$ - and we have more data than unknown parameters.

The images themselves measure quantities that are expressed by these equations:

$$ \begin{aligned}
S_0(L) &= I + Q + Q_{IP}\\
S_1(L) &= I + U + U_{IP}\\
S_2(L) &= I - Q + Q_{IP}\\
S_3(L) &= I - U + U_{IP}
\end{aligned}$$

and

$$ \begin{aligned}
S_0(R) = I - Q - Q_{IP}\\
S_1(R) = I - U - U_{IP}\\
S_2(R) = I + Q - Q_{IP}\\
S_3(R) = I + U - U_{IP}
\end{aligned}$$

...where $Q_{IP}$ and $U_{IP}$ are the instrumental polarizations - initially, you can assume that these are zero, but when considering double differencing later on, they are nonzero.

```python
%config InlineBackend.figure_format = 'retina'
import numpy as np
import matplotlib.pyplot as plt
from astropy.io import fits

def circle_mask(im, xc, yc, rcirc):
    """circle_mask - function that takes the input 2D array 'im' that evaluates the equation
            (x-x_c)^2 + (y-y_c)^2 < r^2 with circle center coordinates (x_c, y_c) and a radius 'r'
            as input parameters and return a mask array with the same shape as 'im'."""
    ny, nx = im.shape
    y,x = np.mgrid[0:nx,0:ny]
    r = np.sqrt((x-xc)*(x-xc) + (y-yc)*(y-yc))
    return ( (r < rcirc))

```

---
Question 6.1
(2 points):

****

1. **Read in the data cube from `ORD_EXT_4HWP.fits` into a numpy array. Call it `im`!**
*  **Print out the following parameters:**
   *  **The dimensions in the image contain the `L` and `R` images**
   *  **The different position angles**
   *  **The size and shape of the input images**

Tip 1: All these parameters can be obtained using Pythons `.shape` method.

Tip 2: Recall that shapes start from dimention `0`.

---

```python
# Q1 answer here
im, header = fits.getdata('ORD_EXT_4HWP.fits', header=True)
print("The shape of the fits image is: ",im.shape)
print("This stands for the dimension of the image is 4, stands for 4 different polarisations. For each polarisation, there are two images: the L and R-image.\n")
print("There are 4 position angles, namely 0, 45, 90, and 135 degrees.\n")
print("The shape of each image is: ",im[0][0].shape)
# Q1 end of answer

# print('The size of image is {:d}x{:d} pixels'.format(....))
# print('The number of orthogonal frames, both ordinary and extraordinary = {:d}'.format(....))
# print('The number of HWP positions = 0, 22.5, 45 and 67.5 degrees = {:d}'.format(....))
```

    The shape of the fits image is:  (4, 2, 100, 100)
    This stands for the dimension of the image is 4, stands for 4 different polarisations. For each polarisation, there are two images: the L and R-image.

    There are 4 position angles, namely 0, 45, 90, and 135 degrees.

    The shape of each image is:  (100, 100)

## The intensity and difference images

First, we look at the intensity image. Adding the `L` and `R` images together, and averaging over the 4 orientations of the polariser will produce the intensity image `Itot`.

---
Question 6.2
(2 points):

**Calculate `Itot` and display its log$_{10}$ with `ax.imshow`.**

---

```python
# Q2 answer here
def Itot(im):
    num_orient = im.shape[0]
    size = im[0][0].shape[0]
    img = np.zeros((num_orient,size,size))
    for i in range(num_orient):
        img[i] = np.mean(im[i],axis=0)
    return np.mean(img,axis=0)
# Q2 end of answer
```

```python
img = Itot(im)
plt.imshow(np.log10(img),origin='lower')
plt.title('$I_{tot}$')
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.show()
```

Each `L` and `R` image pair are orthogonal polarisations. From the expressions in the [*equations* given before](#equation1) we find that summing and differencing the image pairs will produce an intensity and a polarisation image respectively.

---
Question 6.3
(2 points):

1. **Create `spos` and `sneg`, which are the sum and difference of the `L` and `R` images respectively.**
*  **Write both images out to FITS images (call them `SPOS.fits` and `SNEG.fits`)**
*  **Check that the SNEG images have the butterfly pattern in them in `ds9`.**
*  **Use `imshow` with `vmin=-1000` and `vmax=1000` and display the 0 degree image in `sneg`.**

SANITY CHECK : Check that you have subtracted the correct dimensions by printing out the shape of both of these arrays. Think about what you expect the shape of the output arrays to be.

5.  **Join dimension 1 of `sneg` and `spos` together into `sdif`. Remember, you will/can use `sdif` in Question 6.13**

Tip:  Joining axes together can be done using the module np.concatenate()

---

```python
# Q3 answer here
def get_spos_sneg(im):
    return np.sum(im,axis=0),(im[0]-im[1])

size = im[0][0].shape[0]
spos = np.zeros((4,size,size))
sneg = np.zeros((4,size,size))
for i in range(4):
    spos[i],sneg[i] = get_spos_sneg(im[i])

plt.imshow(sneg[0],vmin=-1e3,vmax=1e3)
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('0 degree in SNEG')
plt.show()

hdu = fits.PrimaryHDU(spos)
hdu.writeto('SPOS.fits', overwrite=True)
hdu = fits.PrimaryHDU(sneg)
hdu.writeto('SNEG.fits', overwrite=True)

sdif = np.concatenate((spos,sneg),axis=1)
# Q3 end of answer

print("The spos image has shape {}".format(spos.shape))
print("The sneg image has shape {}".format(sneg.shape))
```

## Making a polarization image

We can now look at the nearly face-on dust disk of the star for the first time by calculating the Polarised Intensity, $I_{pol}$. The polarised itensity is given by $$I_{pol} = \sqrt{Q^2 + U^2}$$.

---
Question 6.4
(2 points):

**Calculate the polarised intensity by using the `sneg` images**

1. **Define Q to be the first dimension image of `sneg`, and U as the second dimension image of `sneg`.**

Tip: Think about the numbering of dimentions in Python arrays.. The first dimension is not [1].

2.  **Calculate the polarised intensity, and call the image `Ipol`**
*  **Display the image `Ipol` in the range $0$ to $1500$.**

**Can you identify the disk already?**

---

```python
#%run ds9 SNEG.fits
```

```python
# Q4 answer here
def Ipol(im):
    Q = im[0]
    U = im[1]
    return .5*np.sqrt(Q**2+U**2)
Ipol_img = Ipol(sneg)
# Q4 end of answer

print("The Ipol image has shape {}".format(Ipol_img.shape))
```

    The Ipol image has shape (100, 100)

```python
plt.imshow(Ipol_img,origin='lower',vmin=0,vmax=1500)
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('polarisation image')
plt.colorbar()
plt.show()
print("The disk feature can already be seen from the above plot, albeit not with clarity.")
```

## Plotting up the gain in contrast

---
Question 6.5
(2 points):

**Plot the flux of row number 50 in `Itot` as function of pixel. Overplot the same row in `Ipol` on the same graph. Make the y-axis have a logarithmic scale by using `ax=plt.gca()` to get the name of the current plotting window, and `ax.set_yscale('log')` to make it logarithmic.**

Remember :  Label the axes as "Spatial cut across image [pix]" and a suitable title and units for the y-axis. Add a legend to distinguigh the two plots.

**Describe what you see**

---

```python
# Q5 answer here

# get the flux of row no. 50 in Itot
# actually with 100*100 pixel image, the central pixel should be 49.5
# so I'm rounding down to 49 here
xcen,ycen = 49,49
Itot_50 = img[xcen]
Ipol_50 = Ipol_img[xcen]
plt.semilogy(Itot_50,label='$I_{tot}$')
plt.semilogy(Ipol_50,label='$I_{pol}$')
plt.xlabel('pixel')
plt.ylabel('Intensity')
plt.legend()
print("There is a positive correlation between Ipol and Itot, since polarisation changes the absolute intensity not the relative intensity.")
# Q5 end of answer
```

---
Question 6.6
(1 point):

**Plot the ratio of the two lines above. By doing so, you get an idea of the fractional polarisation that is measured.**

**Write down what the polarisation level is. Round your answer to whole percentages.**

---

```python
# Q6 answer here
ratio = Ipol_50/Itot_50
plt.plot(ratio)
plt.xlabel('pixel')
plt.ylabel('fractional polarisation')
plt.show()
level = ratio.mean()*1e2
# Q6 end of answer

print("The polarisation level is {:.0f} precent.".format(level))

```

## Double difference

As mentioned in the introduction of this notebook, we have more images than there are parameters to measure. So, why do we have this extra data? We are measuring polarization by comparing intensities of the images. However, there is no guarantee that the transmission of optics forming the `L` and `R` images are the same. In addition, since we have to take a series of images in time, the instrument and the atmospheric conditions may have changed as well.

---
Question 6.7
(1 points):

**Using the definitions of $S(L)$ and $S(R)$ from the top of the practicum, write out what `spos` and `sneg` are in terms of `Q` and `U`.**

---

You see that you have four independent measurements of `I` and two independent measurements of `Q` and `U` in `spos` and `sneg`.  We can average the values in `spos` together to get `I`, thus combine the $S_0$ and $S_2$ components in `spos` to get `I`. To get `U` and `Q`, we can average the values in `sneg` together.

---
Question 6.8
(3 points):

1. **With `sneg`, combine the $S_0$ and $S_2$ values to get a value for `Q`.**
* **Combine the $S_1$ and $S_3$ values to get a value for `U`.**
* **Calculate the polarisation `P` using the same formula as for `Ipol`.**

---

The angle of polarisation $\chi$ indicates the angle between the plane of polarisation and the plane of reference. This angle can be described by the formula:

$$\chi = \frac{1}{2}\arctan(\texttt{U}/\texttt{Q})$$

---
Question 6.9
(1 point):

**Calculate the angle of polarisation, $\chi$, using the formula above. Call it `X`**

Tip : ...using the np.arctan2() function.

---

### Q7 answer

$$ \begin{aligned}
S_0(L) &= I + Q + Q_{IP}\\
S_1(L) &= I + U + U_{IP}\\
S_2(L) &= I - Q + Q_{IP}\\
S_3(L) &= I - U + U_{IP}
\end{aligned}$$

and

$$ \begin{aligned}
S_0(R) = I - Q - Q_{IP}\\
S_1(R) = I - U - U_{IP}\\
S_2(R) = I + Q - Q_{IP}\\
S_3(R) = I + U - U_{IP}
\end{aligned}$$

\begin{equation}
spos = \begin{bmatrix}
S_0(L)+S_0(R)\\
S_1(L)+S_1(R)\\
S_2(L)+S_2(R)\\
S_3(L)+S_3(R)\\
\end{bmatrix}
= \begin{bmatrix}
2I\\
2I\\
2I\\
2I\\
\end{bmatrix}
\end{equation}

\begin{equation}
sneg = \begin{bmatrix}
S_0(L)-S_0(R)\\
S_1(L)-S_1(R)\\
S_2(L)-S_2(R)\\
S_3(L)-S_3(R)\\
\end{bmatrix}
= \begin{bmatrix}
2(Q+Q_{IP})\\
2(U+U_{IP})\\
2(-Q+Q_{IP})\\
2(-U+U_{IP})
\end{bmatrix}
\end{equation}
$$\implies I = \frac{1}{4}\left(spos_0+spos_2\right),Q = \frac{1}{4}\left(sneg_0-sneg_2\right),U = \frac{1}{4}\left(sneg_1-sneg_3\right)$$

```python
# Q7, Q8, Q9 answers here
I = (spos[0]+spos[2])/4
Q = (sneg[0]-sneg[2])/4
U = (sneg[1]-sneg[3])/4
P = np.sqrt(Q**2+U**2)
X = .5*np.arctan2(U,Q)
pol_ang = X.mean()/np.pi*180
print("The polarisation level calculated in this way is {:.0f} percent.".format(np.mean(P[50]/Itot_50*1e2)))
print("The angle of polarisation is {:.2f} degree.".format(pol_ang))
# Q7, Q8, Q9 end of answers
```

    The polarisation level calculated in this way is 3 percent.
    The angle of polarisation is -3.41 degree.

## Polarization image

---
Question 6.10
(1 point):

1. **Display the polarisation image with the value range (0,1500) <-- not pixel range.**
2. **Mask out a circle of radius 5 pixels in the middle of the polarisation image.**
* **Display the masked polarisation image with the same range.**

Sanity check :  You should see spiral arms in the polarisation image.

---

```python
# Q10 answer here
plt.imshow(P,origin='lower',vmin=0,vmax=1500)
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('polarisation image')
plt.colorbar()
plt.show()
msk = circle_mask(P,xcen,ycen,5)
plt.imshow(P*~msk,origin='lower',vmin=0,vmax=1500)
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('masked polarisation image')
plt.colorbar()
plt.show()

# Q10 answer here
```

## Fractional polarization

---
Question 6.11
(1 point):

**Display the fractional polarization (P/I) image from values 0 to 0.10.**

Recall:  The value of 0.10 in the fractional polarisation image stands for 10% polarisation.

---

```python
# Q11 answer here

plt.imshow(P/I,vmin=0,vmax=.1,origin='lower')
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('fractional polarisation image')
plt.colorbar()
plt.show()

# Q11 answer here
```

## Limits of signal to noise by looking at the polarization angle.

The scattered light should be at right angles to the line between any spot on the disk and the central star. Plotting the polarization angle and seeing where it degenerates into noise gives a good indication where the polarization signal lies.

---
Question 6.12
(1 point):

**Display the polarisation angle map.**

---

```python
# Q12 answer here
plt.imshow(X)
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('polarisation angle map (in radians)')
plt.colorbar()
plt.show()
# Q12 end of answer
```

---
Question 6.13
(2 points):

**Make a circular annular mask centered on the star that blocks out the star and the noisy polarisation angles at large radii (estimate by eye what outer radius to use).**

**Use this mask to display the spiral arms in the disk in the fractional polarisation image.**

```python
# Q13 answer here
msk1 = circle_mask(P,xcen,ycen,10)
msk2 = circle_mask(P,xcen,ycen,38)
plt.imshow((P/I)*msk2*~msk1,vmin=0,vmax=.1,origin='lower')
plt.xlabel('x pixel')
plt.ylabel('y pixel')
plt.title('annular masked polarisation image (in radians)')
plt.colorbar()
plt.show()

# Q13 end of answer
```

REMEMBER: to make sure your code compiles cleanly before submitting it! Do only upload the notebook to Brightspace in the correct naming format.

---

Your time has a value as well

---

### How long did this Practicum take?
Please enter the number of hours you spent on this Practicum (including the time spend besides the time spend during the Lecture) below:

Number of hours taken for this practicum ~7 hours.

```python

```


# Tutorial 2: Effects of Input Correlation
**Week 2, Day 3: Biological Neuron Models**

**By Neuromatch Academy**

__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar

__Content reviewers:__ Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Matthew Krause, Spiros Chavlis, Michael Waskom

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

*Estimated timing of tutorial: 50 minutes*

In this tutorial, we will use the leaky integrate-and-fire (LIF) neuron model (see Tutorial 1) to study how they transform input correlations to output properties (transfer of correlations). In particular, we are going to write a few lines of code to:

- inject correlated GWN in a pair of neurons

- measure correlations between the spiking activity of the two neurons

- study how the transfer of correlation depends on the statistics of the input, i.e. mean and standard deviation.

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/8djsm/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

---
# Setup

```python
# Imports

import matplotlib.pyplot as plt
import numpy as np
import time
```

```python
# @title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
# use NMA plot style
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
my_layout = widgets.Layout()
```

```python
# @title Plotting Functions

def example_plot_myCC():
  pars = default_pars(T=50000, dt=.1)

  c = np.arange(10) * 0.1
  r12 = np.zeros(10)
  for i in range(10):
    I1gL, I2gL = correlate_input(pars, mu=20.0, sig=7.5, c=c[i])
    r12[i] = my_CC(I1gL, I2gL)

  plt.figure()
  plt.plot(c, r12, 'bo', alpha=0.7, label='Simulation', zorder=2)
  plt.plot([-0.05, 0.95], [-0.05, 0.95], 'k--', label='y=x',
           dashes=(2, 2), zorder=1)
  plt.xlabel('True CC')
  plt.ylabel('Sample CC')
  plt.legend(loc='best')

# the function plot the raster of the Poisson spike train
def my_raster_Poisson(range_t, spike_train, n):
  """
  Generates poisson trains

  Args:
    range_t     : time sequence
    spike_train : binary spike trains, with shape (N, Lt)
    n           : number of Poisson trains plot

  Returns:
    Raster plot of the spike train
  """

  # find the number of all the spike trains
  N = spike_train.shape[0]

  # n should smaller than N:
  if n > N:
    print('The number n exceeds the size of spike trains')
    print('The number n is set to be the size of spike trains')
    n = N

  # plot rater
  i = 0
  while i < n:
    if spike_train[i, :].sum() > 0.:
      t_sp = range_t[spike_train[i, :] > 0.5]  # spike times
      plt.plot(t_sp, i * np.ones(len(t_sp)), 'k|', ms=10, markeredgewidth=2)
    i += 1
  plt.xlim([range_t[0], range_t[-1]])
  plt.ylim([-0.5, n + 0.5])
  plt.xlabel('Time (ms)', fontsize=12)
  plt.ylabel('Neuron ID', fontsize=12)

def plot_c_r_LIF(c, r, mycolor, mylabel):
  z = np.polyfit(c, r, deg=1)
  c_range = np.array([c.min() - 0.05, c.max() + 0.05])
  plt.plot(c, r, 'o', color=mycolor, alpha=0.7, label=mylabel, zorder=2)
  plt.plot(c_range, z[0] * c_range + z[1], color=mycolor, zorder=1)
```

```python
# @title Helper Functions
def default_pars(**kwargs):
  pars = {}

  ### typical neuron parameters###
  pars['V_th'] = -55.     # spike threshold [mV]
  pars['V_reset'] = -75.  # reset potential [mV]
  pars['tau_m'] = 10.     # membrane time constant [ms]
  pars['g_L'] = 10.       # leak conductance [nS]
  pars['V_init'] = -75.   # initial potential [mV]
  pars['V_L'] = -75.      # leak reversal potential [mV]
  pars['tref'] = 2.       # refractory time (ms)

  ### simulation parameters ###
  pars['T'] = 400. # Total duration of simulation [ms]
  pars['dt'] = .1  # Simulation time step [ms]

  ### external parameters if any ###
  for k in kwargs:
    pars[k] = kwargs[k]

  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized
                                                         # time points [ms]
  return pars

def run_LIF(pars, Iinj):
  """
  Simulate the LIF dynamics with external input current

  Args:
    pars       : parameter dictionary
    Iinj       : input current [pA]. The injected current here can be a value or an array

  Returns:
    rec_spikes : spike times
    rec_v      : mebrane potential
  """

  # Set parameters
  V_th, V_reset = pars['V_th'], pars['V_reset']
  tau_m, g_L = pars['tau_m'], pars['g_L']
  V_init, V_L = pars['V_init'], pars['V_L']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tref = pars['tref']

  # Initialize voltage and current
  v = np.zeros(Lt)
  v[0] = V_init
  Iinj = Iinj * np.ones(Lt)
  tr = 0.

  # simulate the LIF dynamics
  rec_spikes = []   # record spike times
  for it in range(Lt - 1):
    if tr > 0:
      v[it] = V_reset
      tr = tr - 1
    elif v[it] >= V_th:  # reset voltage and record spike event
      rec_spikes.append(it)
      v[it] = V_reset
      tr = tref / dt

    # calculate the increment of the membrane potential
    dv = (-(v[it] - V_L) + Iinj[it] / g_L) * (dt / tau_m)

    # update the membrane potential
    v[it + 1] = v[it] + dv

  rec_spikes = np.array(rec_spikes) * dt

  return v, rec_spikes

def my_GWN(pars, sig, myseed=False):
  """
  Function that calculates Gaussian white noise inputs

  Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    myseed     : random seed. int or boolean
                 the same seed will give the same random number sequence

  Returns:
    I          : Gaussian white noise input
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # Set random seed. You can fix the seed of the random number generator so
  # that the results are reliable however, when you want to generate multiple
  # realization make sure that you change the seed for each new realization
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate GWN
  # we divide here by 1000 to convert units to sec.
  I_GWN = sig * np.random.randn(Lt) * np.sqrt(pars['tau_m'] / dt)

  return I_GWN

def LIF_output_cc(pars, mu, sig, c, bin_size, n_trials=20):
  """ Simulates two LIF neurons with correlated input and computes output correlation

  Args:
  pars       : parameter dictionary
  mu         : noise baseline (mean)
  sig        : noise amplitute (standard deviation)
  c          : correlation coefficient ~[0, 1]
  bin_size   : bin size used for time series
  n_trials   : total simulation trials

  Returns:
  r          : output corr. coe.
  sp_rate    : spike rate
  sp1        : spike times of neuron 1 in the last trial
  sp2        : spike times of neuron 2 in the last trial
  """

  r12 = np.zeros(n_trials)
  sp_rate = np.zeros(n_trials)
  for i_trial in range(n_trials):
    I1gL, I2gL = correlate_input(pars, mu, sig, c)
    _, sp1 = run_LIF(pars, pars['g_L'] * I1gL)
    _, sp2 = run_LIF(pars, pars['g_L'] * I2gL)

    my_bin = np.arange(0, pars['T'], bin_size)

    sp1_count, _ = np.histogram(sp1, bins=my_bin)
    sp2_count, _ = np.histogram(sp2, bins=my_bin)

    r12[i_trial] = my_CC(sp1_count[::20], sp2_count[::20])
    sp_rate[i_trial] = len(sp1) / pars['T'] * 1000.

  return r12.mean(), sp_rate.mean(), sp1, sp2
```

The helper function contains the:

- Parameter dictionary: `default_pars( **kwargs)` from Tutorial 1
- LIF simulator: `run_LIF` from Tutorial 1
- Gaussian white noise generator: `my_GWN(pars, sig, myseed=False)` from Tutorial 1
- Poisson type spike train generator: `Poisson_generator(pars, rate, n, myseed=False)`
- Two LIF neurons with correlated inputs simulator: `LIF_output_cc(pars, mu, sig, c, bin_size, n_trials=20)`

---
# Section 1: Correlations (Synchrony)
Correlation or synchrony in neuronal activity can be described for any readout of brain activity. Here, we are concerned with the spiking activity of neurons.

In the simplest way, correlation/synchrony refers to coincident spiking of neurons, i.e., when two neurons spike together, they are firing in **synchrony** or are **correlated**. Neurons can be synchronous in their instantaneous activity, i.e., they spike together with some probability. However, it is also possible that spiking of a neuron at time $t$ is correlated with the spikes of another neuron with a delay (time-delayed synchrony).

## Origin of synchronous neuronal activity:
- Common inputs, i.e., two neurons are receiving input from the same sources. The degree of correlation of the shared inputs is proportional to their output correlation.
- Pooling from the same sources. Neurons do not share the same input neurons but are receiving inputs from neurons which themselves are correlated.
- Neurons are connected to each other (uni- or bi-directionally): This will only give rise to time-delayed synchrony. Neurons could also be connected via gap-junctions.
- Neurons have similar parameters and initial conditions.

## Implications of synchrony
When neurons spike together, they can have a stronger impact on downstream neurons. Synapses in the brain are sensitive to the temporal correlations (i.e., delay) between pre- and postsynaptic activity, and this, in turn, can lead to the formation of functional neuronal networks - the basis of unsupervised learning (we will study some of these concepts in a forthcoming tutorial).

Synchrony implies a reduction in the dimensionality of the system. In addition, correlations, in many cases, can impair the decoding of neuronal activity.

```python
# @title Video 1: Input & output correlations
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Bh411o7eV", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="nsAYFBcAkes", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

A simple model to study the emergence of correlations is to inject common inputs to a pair of neurons and measure the output correlation as a function of the fraction of common inputs.

Here, we are going to investigate the transfer of correlations by computing the correlation coefficient of spike trains recorded from two unconnected LIF neurons, which received correlated inputs.

The input current to LIF neuron $i$ $(i=1,2)$ is:

\begin{equation}
\frac{I_i}{g_L} =\mu_i + \sigma_i (\sqrt{1-c}\xi_i + \sqrt{c}\xi_c) \quad (1)
\end{equation}

where $\mu_i$ is the temporal average of the current. The Gaussian white noise $\xi_i$ is independent for each neuron, while $\xi_c$ is common to all neurons. The variable $c$ ($0\le c\le1$) controls the fraction of common and independent inputs. $\sigma_i$ shows the variance of the total input.

So, first, we will generate correlated inputs.

```python
# @markdown Execute this cell to get a function `correlate_input` for generating correlated GWN inputs
def correlate_input(pars, mu=20., sig=7.5, c=0.3):
  """
  Args:
    pars       : parameter dictionary
    mu         : noise baseline (mean)
    sig        : noise amplitute (standard deviation)
    c.         : correlation coefficient ~[0, 1]

  Returns:
    I1gL, I2gL : two correlated inputs with corr. coe. c
  """

  # generate Gaussian whute noise xi_1, xi_2, xi_c
  xi_1 = my_GWN(pars, sig)
  xi_2 = my_GWN(pars, sig)
  xi_c = my_GWN(pars, sig)

  # Generate two correlated inputs by Equation. (1)
  I1gL = mu + np.sqrt(1. - c) * xi_1 + np.sqrt(c) * xi_c
  I2gL = mu + np.sqrt(1. - c) * xi_2 + np.sqrt(c) * xi_c

  return I1gL, I2gL

help(correlate_input)
```

    Help on function correlate_input in module __main__:

    correlate_input(pars, mu=20.0, sig=7.5, c=0.3)
        Args:
          pars       : parameter dictionary
          mu         : noise baseline (mean)
          sig        : noise amplitute (standard deviation)
          c.         : correlation coefficient ~[0, 1]

        Returns:
          I1gL, I2gL : two correlated inputs with corr. coe. c

## Coding Exercise 1A: Compute the correlation

The _sample correlation coefficient_ between two input currents $I_i$ and $I_j$ is defined as the sample covariance of $I_i$ and $I_j$ divided by the square root of the sample variance of $I_i$ multiplied with the square root of the sample variance of $I_j$. In equation form:

\begin{align}
r_{ij} &= \frac{cov(I_i, I_j)}{\sqrt{var(I_i)} \sqrt{var(I_j)}}\\
cov(I_i, I_j) &= \sum_{k=1}^L (I_i^k -\bar{I}_i)(I_j^k -\bar{I}_j) \\
var(I_i) &= \sum_{k=1}^L (I_i^k -\bar{I}_i)^2
\end{align}

where $\bar{I}_i$ is the sample mean, k is the time bin, and L is the length of $I$.  This means that $I_i^k$ is current i at time $k\cdot dt$. Note that the equations above are not accurate for sample covariances and variances as they should be additionally divided by L-1 - we have dropped this term because it cancels out in the sample correlation coefficient formula.

The _sample correlation coefficient_ may also be referred to as the _sample Pearson correlation coefficient_. Here, is a beautiful paper that explains multiple ways to calculate and understand correlations [Rodgers and Nicewander 1988](https://www.stat.berkeley.edu/~rabbee/correlation.pdf).

In this exercise, we will create a function, `my_CC` to compute the sample correlation coefficient between two time series. Note that while we introduced this computation here in the context of input currents, the sample correlation coefficient is used to compute the correlation between any two time series - we will use it later on binned spike trains.

We then check our method is accurate by generating currents with a certain correlation (using `correlate_input`), computing the correlation coefficient using `my_CC`, and plotting the true vs sample correlation coefficients.

```python
def my_CC(i, j):
  """
  Args:
    i, j  : two time series with the same length

  Returns:
    rij   : correlation coefficient
  """
  ########################################################################
  ## TODO for students: compute rxy, then remove the NotImplementedError #
  # Tip1: array([a1, a2, a3])*array([b1, b2, b3]) = array([a1*b1, a2*b2, a3*b3])
  # Tip2: np.sum(array([a1, a2, a3])) = a1+a2+a3
  # Tip3: square root, np.sqrt()
  # Fill out function and remove
  #raise NotImplementedError("Student exercise: compute the sample correlation coefficient")
  ########################################################################

  # Calculate the covariance of i and j
  i, j = np.array(i), np.array(j)
  cov = np.sum( (i - np.mean(i)) * (j - np.mean(j)) )

  # Calculate the (biased) sample variance of i
  #var_i = np.sum( (i - np.mean(i))**2 )
  var_i = np.var(i) * (len(i) - 1)

  # Calculate the (biased) sample variance of j
  #var_j = np.sum( (j - np.mean(j))**2 )
  var_j = np.var(j) * (len(j) - 1)

  # Calculate the correlation coefficient
  rij = cov / ( np.sqrt(var_i) * np.sqrt(var_j) )

  return rij

example_plot_myCC()
```

```python
# to_remove solution
def my_CC(i, j):
  """
  Args:
    i, j  : two time series with the same length

  Returns:
    rij   : correlation coefficient
  """

  # Calculate the covariance of i and j
  cov = ((i - i.mean()) * (j - j.mean())).sum()

  # Calculate the variance of i
  var_i = ((i - i.mean()) * (i - i.mean())).sum()

  # Calculate the variance of j
  var_j = ((j - j.mean()) * (j - j.mean())).sum()

  # Calculate the correlation coefficient
  rij = cov / np.sqrt(var_i*var_j)

  return rij

with plt.xkcd():
  example_plot_myCC()
```

The sample correlation coefficients (computed using `my_CC`) match the ground truth correlation coefficient!

In the next exercise, we will use the Poisson distribution to model spike trains. Remember that you have seen the Poisson distribution used in this way in the [pre-reqs math day on Statistics](https://compneuro.neuromatch.io/tutorials/W0D5_Statistics/student/W0D5_Tutorial1.html#section-2-2-poisson-distribution). Remember that a Poisson spike train has the following properties:
- The ratio of the mean and variance of spike count is 1
- Inter-spike-intervals are exponentially distributed
- Spike times are irregular i.e. 𝐶𝑉ISI=1
- Adjacent spike intervals are independent of each other.

In the following cell, we provide a helper function `Poisson_generator` and then use it to produce a Poisson spike train.

```python
# @markdown Execute this cell to get helper function `Poisson_generator`
def Poisson_generator(pars, rate, n, myseed=False):
  """
  Generates poisson trains

  Args:
    pars       : parameter dictionary
    rate       : noise amplitute [Hz]
    n          : number of Poisson trains
    myseed     : random seed. int or boolean

  Returns:
    pre_spike_train : spike train matrix, ith row represents whether
                      there is a spike in ith spike train over time
                      (1 if spike, 0 otherwise)
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # generate uniformly distributed random variables
  u_rand = np.random.rand(n, Lt)

  # generate Poisson train
  poisson_train = 1. * (u_rand < rate * (dt / 1000.))

  return poisson_train

help(Poisson_generator)
```

    Help on function Poisson_generator in module __main__:

    Poisson_generator(pars, rate, n, myseed=False)
        Generates poisson trains

        Args:
          pars       : parameter dictionary
          rate       : noise amplitute [Hz]
          n          : number of Poisson trains
          myseed     : random seed. int or boolean

        Returns:
          pre_spike_train : spike train matrix, ith row represents whether
                            there is a spike in ith spike train over time
                            (1 if spike, 0 otherwise)

```python
# @markdown Execute this cell to visualize Poisson spike train

pars = default_pars()
pre_spike_train = Poisson_generator(pars, rate=10, n=100, myseed=2020)
my_raster_Poisson(pars['range_t'], pre_spike_train, 100)
```

## Coding Exercise 1B: Measure the correlation between spike trains

After recording the spike times of the two neurons, how can we estimate their correlation coefficient?

In order to find this, we need to bin the spike times and obtain two time series. Each data point in the time series is the number of spikes in the corresponding time bin. You can use `np.histogram()` to bin the spike times.

Complete the code below to bin the spike times and calculate the correlation coefficient for two Poisson spike trains. Note that `c` here is the ground-truth correlation coefficient that we define.

```python
# @markdown Execute this cell to get a function for generating correlated Poisson inputs (`generate_corr_Poisson`)

def generate_corr_Poisson(pars, poi_rate, c, myseed=False):
  """
  function to generate correlated Poisson type spike trains
  Args:
    pars       : parameter dictionary
    poi_rate   : rate of the Poisson train
    c.         : correlation coefficient ~[0, 1]

  Returns:
    sp1, sp2   : two correlated spike time trains with corr. coe. c
  """

  range_t = pars['range_t']

  mother_rate = poi_rate / c
  mother_spike_train = Poisson_generator(pars, rate=mother_rate,
                                         n=1, myseed=myseed)[0]
  sp_mother = range_t[mother_spike_train > 0]

  L_sp_mother = len(sp_mother)
  sp_mother_id = np.arange(L_sp_mother)
  L_sp_corr = int(L_sp_mother * c)

  np.random.shuffle(sp_mother_id)
  sp1 = np.sort(sp_mother[sp_mother_id[:L_sp_corr]])

  np.random.shuffle(sp_mother_id)
  sp2 = np.sort(sp_mother[sp_mother_id[:L_sp_corr]])

  return sp1, sp2

print(help(generate_corr_Poisson))
```

    Help on function generate_corr_Poisson in module __main__:

    generate_corr_Poisson(pars, poi_rate, c, myseed=False)
        function to generate correlated Poisson type spike trains
        Args:
          pars       : parameter dictionary
          poi_rate   : rate of the Poisson train
          c.         : correlation coefficient ~[0, 1]

        Returns:
          sp1, sp2   : two correlated spike time trains with corr. coe. c

    None

```python
def corr_coeff_pairs(pars, rate, c, trials, bins):
  """
  Calculate the correlation coefficient of two spike trains, for different
  realizations

  Args:
      pars   : parameter dictionary
      rate   : rate of poisson inputs
      c      : correlation coefficient ~ [0, 1]
      trials  : number of realizations
      bins   : vector with bins for time discretization

  Returns:
    r12      : correlation coefficient of a pair of inputs
  """

  r12 = np.zeros(n_trials)

  for i in range(n_trials):
    ##############################################################
    ## TODO for students
    # Note that you can run multiple realizations and compute their r_12(diff_trials)
    # with the defined function above. The average r_12 over trials can get close to c.
    # Note: change seed to generate different input per trial
    # Fill out function and remove
    #raise NotImplementedError("Student exercise: compute the correlation coefficient")
    ##############################################################

    # Generate correlated Poisson inputs
    sp1, sp2 = generate_corr_Poisson(pars, rate, c, myseed=2020+i)

    # Bin the spike times of the first input
    sp1_count, _ = np.histogram(sp1, bins=bins)

    # Bin the spike times of the second input
    sp2_count, _ = np.histogram(sp2, bins=bins)

    # Calculate the correlation coefficient
    r12[i] = my_CC(sp1_count, sp2_count)

  return r12

poi_rate = 20.
c = 0.2  # set true correlation
pars = default_pars(T=10000)

# bin the spike time
bin_size = 20  # [ms]
my_bin = np.arange(0, pars['T'], bin_size)
n_trials = 100  # 100 realizations

r12 = corr_coeff_pairs(pars, rate=poi_rate, c=c, trials=n_trials, bins=my_bin)
print(f'True corr coe = {c:.3f}')
print(f'Simu corr coe = {r12.mean():.3f}')
```

    True corr coe = 0.200
    Simu corr coe = 0.197

```python
# to_remove solution
def corr_coeff_pairs(pars, rate, c, trials, bins):
  """
  Calculate the correlation coefficient of two spike trains, for different
  realizations

  Args:
      pars   : parameter dictionary
      rate   : rate of poisson inputs
      c      : correlation coefficient ~ [0, 1]
      trials  : number of realizations
      bins   : vector with bins for time discretization

  Returns:
    r12      : correlation coefficient of a pair of inputs
  """

  r12 = np.zeros(n_trials)

  for i in range(n_trials):
    # Generate correlated Poisson inputs
    sp1, sp2 = generate_corr_Poisson(pars, poi_rate, c, myseed=2020+i)

    # Bin the spike times of the first input
    sp1_count, _ = np.histogram(sp1, bins=bins)

    # Bin the spike times of the second input
    sp2_count, _ = np.histogram(sp2, bins=bins)

    # Calculate the correlation coefficient
    r12[i] = my_CC(sp1_count, sp2_count)

  return r12

poi_rate = 20.
c = 0.2  # set true correlation
pars = default_pars(T=10000)
# bin the spike time
bin_size = 20  # [ms]
my_bin = np.arange(0, pars['T'], bin_size)
n_trials = 100  # 100 realizations

r12 = corr_coeff_pairs(pars, rate=poi_rate, c=c, trials=n_trials, bins=my_bin)
print(f'True corr coe = {c:.3f}')
print(f'Simu corr coe = {r12.mean():.3f}')
```

    True corr coe = 0.200
    Simu corr coe = 0.197

Sample output

```
True corr coe = 0.200
Simu corr coe = 0.197
```

---
# Section 2: Investigate the effect of input correlation on the output correlation

*Estimated timing to here from start of tutorial: 20 min*

Now let's combine the aforementioned two procedures. We first generate the correlated inputs. Then we inject the correlated inputs $I_1, I_2$ into a pair of neurons and record their output spike times. We continue measuring the correlation between the output and
investigate the relationship between the input correlation and the output correlation.

In the following, you will inject correlated GWN in two neurons. You need to define the mean (`gwn_mean`), standard deviation (`gwn_std`), and input correlations (`c_in`).

We will simulate $10$ trials to get a better estimate of the output correlation. Change the values in the following cell for the above variables (and then run the next cell) to explore how they impact the output correlation.

```python
# Play around with these parameters

pars = default_pars(T=80000, dt=1.)  # get the parameters
c_in = 0.3  # set input correlation value
gwn_mean = 55.
gwn_std = 10.
```

```python
# @title

# @markdown Do not forget to execute this cell to simulate the LIF

my_layout.width = '450px'
@widgets.interact(
    c_in=widgets.FloatSlider(0.3, min=0., max=1., step=.1,
                               layout=my_layout),
    gwn_mean=widgets.FloatSlider(10., min=10., max=50., step=1.,
                                layout=my_layout),
    gwn_std=widgets.FloatSlider(10., min=2., max=20., step=1.,
                                layout=my_layout)
)
def in_out_corr(c_in=0.3, gwn_mean=55., gwn_std=10.):
    bin_size = 10.  # ms

    starttime = time.perf_counter()  # time clock

    r12_ss, sp_ss, sp1, sp2 = LIF_output_cc(pars, mu=gwn_mean, sig=gwn_std, c=c_in,
                                            bin_size=bin_size, n_trials=10)

    # just the time counter
    endtime = time.perf_counter()
    timecost = (endtime - starttime) / 60.
    print(f"Simulation time = {timecost:.2f} min")

    print(f"Input correlation = {c_in}")
    print(f"Output correlation = {r12_ss}")

    plt.figure(figsize=(12, 6))
    plt.plot(sp1, np.ones(len(sp1)) * 1, '|', ms=20, label='neuron 1')
    plt.plot(sp2, np.ones(len(sp2)) * 1.1, '|', ms=20, label='neuron 2')
    plt.xlabel('time (ms)')
    plt.ylabel('neuron id.')
    plt.xlim(1000, 8000)
    plt.ylim(0.9, 1.2)
    plt.legend()
    plt.show()
```

    interactive(children=(FloatSlider(value=0.3, description='c_in', layout=Layout(width='450px'), max=1.0), Float…

## Think! 2: Input and Output Correlations
- Is the output correlation always smaller than the input correlation? If yes, why?
- Should there be a systematic relationship between input and output correlations?

You will explore these questions in the next figure but try to develop your own intuitions first!

Lets vary `c_in` and plot the relationship between the `c_in` and output correlation. This might take some time depending on the number of trials.

```python
#@title

#@markdown Don't forget to execute this cell!

pars = default_pars(T=80000, dt=1.)  # get the parameters
bin_size = 10.
c_in = np.arange(0, 1.0, 0.1)  # set the range for input CC
r12_ss = np.zeros(len(c_in))  # small mu, small sigma

starttime = time.perf_counter() # time clock
for ic in range(len(c_in)):
  r12_ss[ic], sp_ss, sp1, sp2 = LIF_output_cc(pars, mu=10.0, sig=10.,
                                              c=c_in[ic], bin_size=bin_size,
                                              n_trials=10)

endtime = time.perf_counter()
timecost = (endtime - starttime) / 60.
print(f"Simulation time = {timecost:.2f} min")

plt.figure(figsize=(7, 6))
plot_c_r_LIF(c_in, r12_ss, mycolor='b', mylabel='Output CC')
plt.plot([c_in.min() - 0.05, c_in.max() + 0.05],
         [c_in.min() - 0.05, c_in.max() + 0.05],
         'k--', dashes=(2, 2), label='y=x')

plt.xlabel('Input CC')
plt.ylabel('Output CC')
plt.legend(loc='best', fontsize=16)
plt.show()
```

```python
# to_remove explanation

"""
Discussion: The results above show that
- output correlation is smaller than input correlation
- output correlation varies linearly as a function of input correlation.

While the general result holds, this relationship might change depending on the neuron type.
""";
```

---
# Section 3: Correlation transfer function

*Estimated timing to here from start of tutorial: 30 min*

The above plot of input correlation vs. output correlation is called the __correlation transfer function__ of the neurons.

## Section 3.1: How do the mean and standard deviation of the Gaussian white noise (GWN) affect the correlation transfer function?

The correlations transfer function appears to be linear. The above can be taken as the input/output transfer function of LIF neurons for correlations, instead of the transfer function for input/output firing rates as we had discussed in the previous tutorial (i.e., F-I curve).

What would you expect to happen to the slope of the correlation transfer function if you vary the mean and/or the standard deviation of the GWN of the inputs ?

```python
# @markdown Execute this cell to visualize correlation transfer functions

pars = default_pars(T=80000, dt=1.) # get the parameters
no_trial = 10
bin_size = 10.
c_in = np.arange(0., 1., 0.2)  # set the range for input CC
r12_ss = np.zeros(len(c_in))   # small mu, small sigma
r12_ls = np.zeros(len(c_in))   # large mu, small sigma
r12_sl = np.zeros(len(c_in))   # small mu, large sigma

starttime = time.perf_counter()  # time clock
for ic in range(len(c_in)):
  r12_ss[ic], sp_ss, sp1, sp2 = LIF_output_cc(pars, mu=10.0, sig=10.,
                                              c=c_in[ic], bin_size=bin_size,
                                              n_trials=no_trial)
  r12_ls[ic], sp_ls, sp1, sp2 = LIF_output_cc(pars, mu=18.0, sig=10.,
                                              c=c_in[ic], bin_size=bin_size,
                                              n_trials=no_trial)
  r12_sl[ic], sp_sl, sp1, sp2 = LIF_output_cc(pars, mu=10.0, sig=20.,
                                              c=c_in[ic], bin_size=bin_size,
                                              n_trials=no_trial)
endtime = time.perf_counter()
timecost = (endtime - starttime) / 60.
print(f"Simulation time = {timecost:.2f} min")

plt.figure(figsize=(7, 6))
plot_c_r_LIF(c_in, r12_ss, mycolor='b', mylabel=r'Small $\mu$, small $\sigma$')
plot_c_r_LIF(c_in, r12_ls, mycolor='y', mylabel=r'Large $\mu$, small $\sigma$')
plot_c_r_LIF(c_in, r12_sl, mycolor='r', mylabel=r'Small $\mu$, large $\sigma$')
plt.plot([c_in.min() - 0.05, c_in.max() + 0.05],
         [c_in.min() - 0.05, c_in.max() + 0.05],
         'k--', dashes=(2, 2), label='y=x')
plt.xlabel('Input CC')
plt.ylabel('Output CC')
plt.legend(loc='best', fontsize=14)
plt.show()
```

### Think! 3.1: GWN and the Correlation Transfer Function
Why do both the mean and the standard deviation of the GWN affect the slope of the correlation transfer function?

```python
# to_remove explanation

"""
Discussion: This has got to do with which part of the input current distribution
is transferred to the spiking activity.

Intuitive understanding is difficult but this relationship arises due to non-linearities
in the neuron F-I curve.  When F-I curve is linear, output correlation is independent
of the mean and standard deviation. But this relationship arises even in neurons with
threshold-linear F-I curve.

Please see:
De La Rocha J, Doiron B, Shea-Brown E, Josić K, Reyes A. Correlation between
neural spike trains increases with firing rate. Nature. 2007 Aug;448(7155):802-6.
""";
```

## Section 3.2: What is the rationale behind varying $\mu$ and $\sigma$?
The mean and the variance of the synaptic current depends on the spike rate of a Poisson process. We can use something called [Campbell's theorem](https://en.wikipedia.org/wiki/Campbell%27s_theorem_(probability)) to estimate the mean and the variance of the synaptic current:

\begin{align}
\mu_{\rm syn} = \lambda J \int P(t) \\
\sigma_{\rm syn} = \lambda J \int P(t)^2 dt\\
\end{align}

where $\lambda$ is the firing rate of the Poisson input, $J$ the amplitude of the postsynaptic current and $P(t)$ is the shape of the postsynaptic current as a function of time.

Therefore, when we varied $\mu$ and/or $\sigma$ of the GWN, we mimicked a change in the input firing rate. Note that, if we change the firing rate, both $\mu$ and $\sigma$ will change simultaneously, not independently.

Here, since we observe an effect of $\mu$ and $\sigma$ on correlation transfer, this implies that the input rate has an impact on the correlation transfer function.

# Think!: Correlations and Network Activity

- What are the factors that would make output correlations smaller than input correlations? (Notice that the colored lines are below the black dashed line)
- What does the fact that output correlations are smaller mean for the correlations throughout a network?
- Here we have studied the transfer of correlations by injecting GWN. But in the previous tutorial, we mentioned that GWN is unphysiological. Indeed, neurons receive colored noise (i.e., Shot noise or OU process). How do these results obtained from injection of GWN apply to the case where correlated spiking inputs are injected in the two LIFs? Will the results be the same or different?

Reference
- De La Rocha, Jaime, et al. "Correlation between neural spike trains increases with firing rate." Nature (2007) (https://www.nature.com/articles/nature06028/)

- Bujan AF, Aertsen A, Kumar A. Role of input correlations in shaping the variability and noise correlations of evoked activity in the neocortex. Journal of Neuroscience. 2015 Jun 3;35(22):8611-25. (https://www.jneurosci.org/content/35/22/8611)

```python
# to_remove explanation

"""
Discussion:

1. Anything that tries to reduce the mean or variance of the input e.g. mean can
be reduced by inhibition, sigma can be reduced by the membrane time constant.
Obviously, if the two neurons have different parameters that will decorrelate them.
But more importantly, it is the slope of neuron transfer function that will affect the
output correlation.

2. These observations pose an interesting problem at the network level. If the
output correlation are smaller than the input correlation, then the network activity
should eventually converge to zero correlation. But that does not happen. So there
is something missing in this model to understand origin of synchrony in the network.

3. For spike trains, we do not have explicit control over mu and sigma but these
two variables will be tied to the firing rate of the inputs. So the
results will be qualitatively similar. But when we think of multiple spike inputs
two different types of correlations arise (see Bujan et al. 2015 for more info)
""";
```

---
# Summary

*Estimated timing of tutorial: 50 minutes*

In this tutorial, we studied how the input correlation of two LIF neurons is mapped to their output correlation. Specifically, we:

- injected correlated GWN in a pair of neurons,

- measured correlations between the spiking activity of the two neurons, and

- studied how the transfer of correlation depends on the statistics of the input, i.e., mean and standard deviation.

Here, we were concerned with zero time lag correlation. For this reason, we restricted estimation of correlation to instantaneous correlations. If you are interested in time-lagged correlation, then we should estimate the cross-correlogram of the spike trains and find out the dominant peak and area under the peak to get an estimate of output correlations.

We leave this as a future to-do for you if you are interested.

If you have time, check out the bonus video to think about responses of ensembles of neurons to time-varying input.

---
# Bonus

---
## Bonus Section 1: Ensemble Response

Finally, there is a short BONUS lecture video on the firing response of an ensemble of neurons to time-varying input. There are no associated coding exercises - just enjoy.

```python
# @title Video 2: Response of ensemble of neurons to time-varying input
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV18K4y1x7Pt", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="78_dWa4VOIo", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

```python

```


```python
import numpy as np
import numba
import matplotlib.pyplot as plt
plt.style.use('presentation')
%matplotlib notebook

def d2np(d):

    names = []
    numbers = ()
    dtypes = []
    for item in d:
        names += item
        if type(d[item]) == float:
            numbers += (d[item],)
            dtypes += [(item,float)]
        if type(d[item]) == np.float64:
            numbers += (d[item],)
            dtypes += [(item,float)]
        if type(d[item]) == np.ndarray:
            numbers += (d[item],)
            dtypes += [(item,np.float64,d[item].shape)]
    return np.array([numbers],dtype=dtypes)
```

```python
# [1] D. Ochoa and S. Martinez,
# “Fast-Frequency Response provided by DFIG-Wind Turbines and its impact on the grid,”
# IEEE Trans. Power Syst., vol. 8950, no. c, pp. 1–1, 2016.

@numba.jit(nopython=True, cache=True)
def pv_fit(struct,i,m):

    nu_w = struct[i]['nu_w']    # m/s
    omega_t = struct[i]['omega_t'] # rad/s
    beta = struct[i]['beta']       # deg

    C_1 = struct[i]['C_1']
    C_2 = struct[i]['C_2']
    C_3 = struct[i]['C_3']
    C_4 = struct[i]['C_4']
    C_5 = struct[i]['C_5']
    C_6 = struct[i]['C_6']

    #Omega_t_n = struct['Omega_t_n'] #
    Radio = struct[i]['Radio']
    Rho = struct[i]['Rho']

    lam = omega_t*Radio/nu_w
    inv_lam_i = 1/(lam + 0.08 * beta) - 0.035/(beta**3 + 1)
    c_p = C_1 * (C_2 * inv_lam_i - C_3 * beta - C_4) * np.exp(-C_5*inv_lam_i) + C_6 * lam

    p_w = 0.5*Rho*np.pi*Radio**2*nu_w**3*c_p

    tau_t = 0.001
    if omega_t>0.0:
        tau_t = p_w/omega_t

    struct[i]['p_w'] = p_w
    struct[i]['c_p'] = c_p
    struct[i]['lam'] = lam
    struct[i]['tau_t'] = tau_t

    return 0

@numba.jit(nopython=True, cache=True)
def mppt_fit(struct,i,m):

    omega_t = struct[i]['omega_t'] # rad/s

    C_1 = struct[i]['C_1']
    C_2 = struct[i]['C_2']
    C_3 = struct[i]['C_3']
    C_4 = struct[i]['C_4']
    C_5 = struct[i]['C_5']
    C_6 = struct[i]['C_6']
    lam_opt = struct[i]['lam_opt']
    beta = struct[i]['beta']

    #Omega_t_n = struct['Omega_t_n'] #
    Radio   = struct[i]['Radio']
    Rho = struct[i]['Rho']

    inv_lam_i = 1/(lam_opt + 0.08 * beta) - 0.035/(beta**3 + 1)
    c_p = C_1 * (C_2 * inv_lam_i - C_3 * beta - C_4) * np.exp(-C_5*inv_lam_i) + C_6 * lam_opt

    #lam = omega_t*R/nu_w
    nu_w = omega_t*Radio/lam_opt
    p_ref = 0.5*Rho*np.pi*Radio**2*nu_w**3*c_p

    struct[i]['p_mppt'] = p_ref
    return p_ref

@numba.jit(nopython=True, cache=True)
def wecs_mech_1(struct,i,m):

    omega_t = struct[i]['x'][0,0]  # rad/s
    tau_t   = struct[i]['tau_t']
    tau_r   = struct[i]['tau_r']

    J_t  = struct[i]['J_t']
    N_tr = struct[i]['N_tr']
    Dt   = struct[i]['Dt']

    domega_t = 1.0/J_t*(tau_t - N_tr*tau_r)

    omega_r = N_tr*omega_t

    struct[i]['f'][:,0] = domega_t

    struct[i]['omega_r'] = omega_r
    struct[i]['omega_t'] = omega_t

    return omega_t

```

```python
@numba.jit(nopython=True, cache=True)
def pitch(struct,i,m):
    Dt  = struct[i]['Dt']
    beta_ref  = struct[i]['beta_ref']
    beta      = struct[i]['beta']
    beta_rate = struct[i]['beta_rate']

    if beta<beta_ref:
        beta += beta_rate * Dt

    if beta>beta_ref:
        beta -= beta_rate * Dt

    struct[i]['beta'] = beta

    return beta
```

```python
Radio = 50.0
lam_opt = 8.0
Nu_w_n = 10.0
Omega_t_n = Nu_w_n*lam_opt/Radio*0.8
Omega_t_max = Omega_t_n*1.25
Omega_t_min = Omega_t_n*0.75
beta = 0.0
Rho = 1.225
C_1 = 0.5176
C_2 = 116.0
C_3 = 0.4
C_4 = 5.0
C_5 = 21.0
C_6 = 0.0068
inv_lam_i = 1/(lam_opt + 0.08 * beta) - 0.035/(beta**3 + 1)
c_p = C_1 * (C_2 * inv_lam_i - C_3 * beta - C_4) * np.exp(-C_5*inv_lam_i) + C_6 * lam_opt
P_t_n = 0.5*Rho*np.pi*Radio**2*Nu_w_n**3*c_p
P_t_n/1e6
```

    2.3080100601373608

```python
N = 100

#nu_w =np.linspace(0.1,15,N)
H = 4.5

# H = 0.5*J*Omega_t_n**2/S_b
S_b = 2.0e6
Omega_t_n = 1.5
J_t = 2*H*S_b/Omega_t_n**2

d = dict(Radio=Radio,
         Rho = 1.225,
         C_1 = 0.5176,
         C_2 = 116.0,
         C_3 = 0.4,
         C_4 = 5.0,
         C_5 = 21.0,
         C_6 = 0.0068,
         P_t_n = P_t_n,
         nu_w = Nu_w_n,
         omega_t = Omega_t_n,
         beta = 0.0,
         Omega_t_n = Omega_t_n,
         Omega_t_min = Omega_t_min,
         Omega_t_max = Omega_t_max,
         Nu_w_n = Nu_w_n,
         p_w = 0.0,
         c_p = 0.0,
         lam = 0.0,
         lam_opt = 8.0,
         J_t = J_t,
         N_tr = 20.0,
         x = np.ones((1,1)),
         f = np.zeros((1,1)),
         omega_r = 1.0,
         tau_t = 10.0,
         tau_r = 0.0,
         Dt = 10.0e-3,
         p_mppt = 1.0,
         beta_ref = 0.0,
         beta_rate = 0.1,
        )

struct = d2np(d)
i=0
m=2
wind_turbine_fit(struct,i,m)
mppt_fit(struct,i,m)
wecs_mech_1(struct,i,m)
pitch(struct,i,m)
```

    0.0

```python
%timeit pitch(struct,i,m)
```

    The slowest run took 15.40 times longer than the fastest. This could mean that an intermediate result is being cached.
    100000 loops, best of 3: 2 µs per loop

```python
struct = d2np(d)

fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)
betas = np.linspace(0.0,24,5)
omegas = np.linspace(0.1,3.5,100)
for beta in betas:
    C_p = []
    Lam = []
    for omega in omegas:
        struct['beta'] = beta
        struct['omega_t'] = omega
        wind_turbine_fit(struct,0)
        C_p += [float(struct['c_p'])]
        Lam += [float(struct['lam'])]
    axes.plot(np.array(Lam),np.array(C_p))
    axes.set_ylabel('$\sf C_p ( \lambda,\\beta )$')
    axes.set_xlabel('$\sf \lambda$')
    axes.set_ylim([0,0.5])
    axes.set_xlim([0,22])
    struct[0]['c_p']

fig.savefig('lam_cp.svg', bbox_inches='tight')

```

```python
struct = d2np(d)
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)
struct['beta'] = 0.0
nu_ws = np.linspace(5,10,5)
omegas = np.linspace(0.1,3.5,100)
for nu_w in nu_ws:
    P_w = []
    Omega = []
    for omega in omegas:
        struct['nu_w'] = nu_w
        struct['omega_t'] = omega
        wind_turbine_fit(struct,0)
        P_w += [float(struct['p_w'])]
        Omega += [float(struct['omega_t'])]
    axes.plot(Omega,np.array(P_w)/1e6)

omega = 1
Nu = np.linspace(2,12,20)
omegas_opt = []
p_ref_opt = []
for nu in Nu:
    struct['nu_w'] = nu
    for it in range(500):
        struct['omega_t'] = omega
        mppt_fit(struct,0)
        wind_turbine_fit(struct,0)
        p_ref = float(struct['p_mppt'])
        p_t = float(struct['p_w'])
        omega += 0.0000001*(-p_ref + p_t)/omega
        if abs(-p_ref + p_t)<0.01:
            break

    omegas_opt += [float(struct['omega_t'])]
    p_ref_opt += [float(struct['p_w'])]

axes.set_ylabel('$\sf P_t$')
axes.set_xlabel('$\sf \omega_t$')
axes.set_ylim([0,2.5])
axes.set_xlim([0,3.0])

fig.savefig('w_p_t.svg', bbox_inches='tight')
axes.plot(np.array(omegas_opt),np.array(p_ref_opt)/1e6, 'r')

fig.savefig('w_p_t_mppt.svg', bbox_inches='tight')

```

## without MPPT

```python
struct = d2np(d)
struct['nu_w'] = 5.0
for it in range(10000):
    wecs_mech_1(struct,0)
    struct[0]['x'] += 100.0e-3*struct[0]['f'][0,0]
print(struct['omega_t'])
print(struct['p_w']/1e6)
```

## with MPPT

```python
struct['omega_t'] = 1.0
struct['omega_r'] = 1.0*20
struct['nu_w'] = 8.5
t = 0.0
Omega_t = []
P_w = []
T = []
Dt = 100.0e-3
for it in range(500):
    if t>10.0:
        struct['nu_w'] = 6.0

    wecs_mech_1(struct,0)
    mppt_fit(struct,0)
    struct['tau_t'] = struct['p_w']/struct['omega_t']
    struct['tau_r'] = struct['p_mppt']/struct['omega_r']
    wind_turbine_fit(struct,0)
    struct[0]['x'] += Dt*struct[0]['f'][0,0]
    struct['omega_t'] = float(struct[0]['x'])
    Omega_t += [float(struct['omega_t'])]
    P_w += [float(struct['p_w'])]
    T += [t]
    t+= Dt
P_w = np.array(P_w)
print(struct['omega_t'])
print(struct['p_w']/1e6)
```

```python
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)

axes.plot(T,Omega_t)

```

```python
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)

axes.plot(T,P_w/1e6)
```

```python
struct['beta_ref'] = 0.0
struct['beta'] = 0.0
for it in range(1000):
    t = it*struct['Dt']
    if t>0.1:
        struct['beta_ref'] = 5.0

    pitch(struct,0)
   # print(t, struct['beta'])
```

```python
import numpy as np
import sympy as sym
from ipywidgets import interact, interactive, widgets
from IPython.display import display
import matplotlib as mpl
from matplotlib import pyplot as plt
mpl.rc('animation', html='html5')
%matplotlib notebook

fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)
fig.set_animated(True)
omegas = np.linspace(0.1,3.5,300)
p_w_line = axes.plot([], '-', ms=8)
axes.plot([struct[0]['Omega_t_max']]*2,[-10,10], '--', ms=2, color='k')
axes.plot([struct[0]['Omega_t_min']]*2,[-10,10], '--', ms=2, color='k')
axes.plot([-10,10], [struct[0]['P_t_n']/1.0e6]*2, '--', ms=2, color='k')

p_mppt_line = axes.plot(np.array(omegas_opt),np.array(p_ref_opt)/1e6)
p_mppt_point = axes.plot(0.0,0.0, 'or', ms=12)
axes.set_xlabel('$\sf \omega_t$')
axes.set_ylabel('$\sf P_t$')
axes.grid(True)

axes.set_ylim([0,2.5])
axes.set_xlim([0,3.0])
def update(beta,nu_w,court):
    P_w = []
    P_mppt =[]
    Omega = []

    struct['nu_w'] = nu_w
    for omega in omegas:
        struct['beta'] = beta
        struct['omega_t'] = omega
        wind_turbine_fit(struct,0)
        P_w += [float(struct['p_w'])]
        Omega += [float(struct['omega_t'])]
        struct[0]['beta'] = 0.0
        struct[0]['omega_t'] = omega*(court)
        mppt_fit(struct,0)
        P_mppt +=[float(struct['p_mppt'])]
    Omega = np.array(Omega)
    P_w = np.array(P_w)
    P_mppt = np.array(P_mppt)
    sh = 20
    idx = np.argmin(np.abs(P_w[sh:]-P_mppt[sh:]))
    #axes.plot(Omega,np.array(P_w)/1e6)
    p_w_line[0].set_xdata(np.array(Omega))
    p_w_line[0].set_ydata(np.array(P_w)/1.0e6)
    p_mppt_line[0].set_xdata(np.array(Omega))
    p_mppt_line[0].set_ydata(np.array(P_mppt)/1.0e6)
    print('Pt = {:2.2f} MW  speed = {:2.2f} p.u.'.format(P_w[idx+sh]/1.0e6, Omega[idx+sh]/struct[0]['Omega_t_n']))
    p_mppt_point[0].set_xdata(Omega[idx+sh])
    p_mppt_point[0].set_ydata(P_w[idx+sh]/1.0e6)

sld_beta  = widgets.FloatSlider(min=0,max=20,step=0.1,value=0.0, width=1000)
sld_nu_w  = widgets.FloatSlider(min=0.1,max=20,step=0.1,value=struct['Nu_w_n'], width=1000)
sld_court = widgets.FloatSlider(min=0.1,max=1,step=0.02,value=1, width=1000)

interaction_avr = interactive(update,
                          beta=sld_beta,
                          nu_w=sld_nu_w,
                          court=sld_court,
                          continuous_update=True, width=1000)

```

```python
display(interaction_avr)
```

```python

```

```python

```