# lattice_mc
`lattice_mc` is Python module for running (kinetic) lattice-gas Monte Carlo simulations. Simple lattices can be constructed programmatically (presently square, honeycomb, and cubic lattices). Arbitrary lattices can be generated from files that define the lattice sites and their connectivity. The algorithms used and interaction models are described in [\[1\]](#ref1). Calculated properties include tracer and “jump” diffusion coefficients; where the latter is proportional to the mobility (and hence the conductivity for charged particles) [\[2\]](#ref2); and tracer (single particle) and collective correlation factors, $f$ and $f_\mathrm{I}$ [\[3\]](#ref3). The simplest interaction model is for “non-interacting” particles, where the only restriction is volume exclusion (two particles cannot simultaneously occupy a single site) [\[4\]](#ref4). Additional interaction models include nearest-neighbour repulsion and on-site energies for inequivalent sites.

## Contents:

- Simulating the Diffusion of a Single Particle on a Square Lattice.
- Improved Averages using multiprocessing.
- Calculating correlation factors as a function of particle number.
- Interacting Particles:
    - Nearest-Neighbour Repulsion.
        - Example - The Effect of Nearest-Neighbour repulsion on a Honeycomb Lattice.
    - On-site Energies.
        - Example - The Effect of Site Inequivalence on a Honeycomb Lattice
- Defining a Lattice from a Sites File
    - Example: Lattice-Gas Monte Carlo Simulation on a Garnet Lattice.
- Running simulations for a set time, instead of a set number of jumps

```python
import lattice_mc
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.style as style
import multiprocessing
import pandas as pd
from math import sqrt, fabs

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
```

```python
# Global variables controlling simulation parallelisation and convergence.

# NPROCS sets the number of processors available for running parallel simulations
NPROCS = 32

# NSAMPLES sets the number of identical simulations averaged over for each data point
NSAMPLES = 300

# !!! Large NSAMPLES and small NPROCS will make this notebook very slow to run !!!
```

```python
# define temperature scale
kT = lattice_mc.global_vars.kT # default setting is T=298 K
```

## Simulating the diffusion of a single particle on a square lattice
The `run_simulation` function sets up a simulation with a 6×6 square lattice, using one of the functions from `lattice_mc.init_lattice`). The simulation parameters are:
lattice spacing, $a=1$.
1 atom.
10000 jumps.
Particles are non-interacting apart from volume exclusion, because we have not explicitly defined additional interactions.

```python
def run_simulation():
    a, b = 6, 6 # creates 4x4x4=64 sites
    spacing = 1.0
    n_atoms = 1
    n_jumps = 10000
    s = lattice_mc.Simulation()
    s.lattice = lattice_mc.init_lattice.square_lattice( a, b, spacing )
    s.set_number_of_atoms( n_atoms )
    s.set_number_of_jumps( n_jumps )
    s.run()
    return s

s = run_simulation()
print( s )
print( s.has_run )
```

    <lattice_mc.simulation.Simulation object at 0x7f04479fe550>
    True

Having run a simulation we can access the following information as properties of the Simulation object:

| Property.                     | symbol          | syntax.                                       |
|-------------------------------|-----------------|-----------------------------------------------|
| has the simulation run?       | N/A             | `Simulation.has_run`                          |
| correlation factor            | $f$             | `Simulation.tracer_correlation`               |
| collective correlation factor | $f_\mathrm{I}$  | `Simulation.collective_correlation`           |
| tracer diffusion coefficient  | $D^*$           | `Simulation.tracer_diffusion_coefficient`     |
| “jump” diffusion coefficient  | $D_\mathrm{J}$  | `Simulation.collective_diffusion_coefficient` |
| average site occupations      | occ($\alpha$)   | `Simulation.average_site_occupations`         |

```python
s = run_simulation()
print( "f   = {}".format( s.tracer_correlation ) )
print( "f_I = {}".format( s.collective_correlation ) )
print( "D*  = {:.3e}".format( s.tracer_diffusion_coefficient ) )
print( "D_J = {:.3e}".format( s.collective_diffusion_coefficient ) )
# Simulation.average_site_occupations returns a dictionary
for k, v in s.average_site_occupations.items():
    print( "occ({}) = {:f}".format( k, v ) )
```

    f   = 1.153
    f_I = 1.153
    D*  = 7.591e+12
    D_J = 7.591e+12
    occ(L) = 1.000000

For a single particle (i.e. in the dilute non-interacting limit) the tracer correlation factor and collective correlation factor should both be equal to 1. This is unlikely here because a single short simulation does not give good statistics.

## Improved averages using multiprocessing

Better converged results are achieved with longer simulations, or by running multiple equivalent simulations and taking averages. On a multiprocessor machine, this parallelisation over simulations can be done by wrapping `run_simulation()` in a second function.

```python
def simulation_wrapper(_):
    s = run_simulation()
    return s.tracer_correlation, s.collective_correlation, s.tracer_diffusion_coefficient, s.collective_diffusion_coefficient
```

```python
def run_simulations( simulation_function, nproc, n_samples ):
    pool = multiprocessing.Pool( processes=nproc )
    calc_pool = ( range( n_samples ) )
    f_sampled = np.array( pool.map( simulation_function, calc_pool ) )
    f = sum( f_sampled ) / n_samples
    return f
```

```python
data = run_simulations( simulation_wrapper, nproc=NPROCS, n_samples=NSAMPLES )
print( "f   = {}".format( data[0] ) )
print( "f_I = {}".format( data[1] ) )
print( "D*  = {:.3e}".format( data[2] ) )
print( "D_J = {:.3e}".format( data[3] ) )
```

    f   = 0.933027333333334
    f_I = 0.933027333333334
    D*  = 6.218e+12
    D_J = 6.218e+12

## Calculating correlation factors as a function of particle number

This example calculates the single-particle correlation factor, $f$, and collective correlation factor, $f_I$, as a function of particle concentration, for a honeycomb lattice. Here we create a lookup table for the jump probabilities, using `Simulation.setup_lookup_table`, to speed up the simulation.

```python
def honeycomb_lattice_correlation( n_atoms ):
    a, b = 4, 3
    spacing = 1.0
    n_jumps = 10000
    s = lattice_mc.Simulation()
    s.lattice = lattice_mc.init_lattice.honeycomb_lattice( a, b, spacing )
    s.set_number_of_atoms( n_atoms )
    s.set_number_of_jumps( n_jumps )
    s.setup_lookup_table() # Create a lookup table for jump probabilities to speed up the simulation
    s.run()
    return s.tracer_correlation, s.collective_correlation
```

```python
# redefine run_simulations so that it now passes n_atoms through to the simulation function
def run_simulations( n_atoms, simulation_function, nproc, n_samples ):
    pool = multiprocessing.Pool( processes=nproc )
    f_data = []
    for n in n_atoms:
        calc_pool = ( n for i in range( n_samples ) )
        f_sampled = np.array( pool.map( simulation_function, calc_pool ) )
        f = sum( f_sampled ) / n_samples
        f_data.append( [ n, *f ] )
    return f_data
```

```python
n_atoms = np.array( [ 1, 8, 16, 24, 32, 40, 47 ] )
results = run_simulations( n_atoms, honeycomb_lattice_correlation, nproc=NPROCS, n_samples=NSAMPLES )
data = pd.DataFrame( np.row_stack( results ), columns=[ 'n', 'f', 'f_I' ] )
data
```

      n
      f
      f_I

      0
      1.0
      1.015060
      1.015060

      1
      8.0
      0.886737
      0.939904

      2
      16.0
      0.728710
      0.849602

      3
      24.0
      0.609881
      1.059191

      4
      32.0
      0.519142
      0.975154

      5
      40.0
      0.423326
      1.001843

      6
      47.0
      0.351168
      1.011347

```python
# plot the data
plt.plot( data.n, data.f,   'o-', label='f'  )
plt.plot( data.n, data.f_I, 'o-', label='f_I')
plt.legend()
plt.xlabel( r"$n_\mathrm{atoms}$" )
plt.ylabel( r"$f$, $f_\mathrm{I}$" )
plt.show()

# The tracer correlation factor in the single vacancy limit is 0.33333 [Friauf J. Appl. Phys. 33, 494 (1962)]
print( "f_v = {:.4f} (exact result = 0.3333)".format( data.f.values[-1] ) )
```

The collective correlation factor, $f_I$, should equal 1 at all particle concentrations, but has much worse statistics than the single particle (tracer) correlation factor. This is because the single particle correlation factor calculation includes an average over all particles in the simulation, whereas the collective correlation factor depends on a *sum* over all particles, and does not have this additional averaging. (cf. the comparison between simulated diffusion coefficients and ionic conductivities in [\[5\]](#ref5).)

## Interacting Particles

For non-interacting particles, all allowed particle jumps have equal probability. Introducing interactions changes the relative probabilities of each jump according to the corresponding change in total energy:

\begin{equation}
  P_i \propto
  \begin{cases}
    \mathrm{exp}\left(\frac{\Delta E_i}{kT}\right),& \mathrm{if}\,\,\Delta E_i > 0 \\
    1,                                             & \mathrm{otherwise.}
  \end{cases}
\end{equation}

### Nearest-neighbour repulsion

The energy of any configuration of occupied sites, $j$ is given by
\begin{equation}
  E = \sum_j n_j^\mathrm{nn}E_\mathrm{nn},
\end{equation}
where $n^\mathrm{nn}_j$ is the number of occupied nearest neighbour sites for (occupied) site $j$.

#### Example - The Effect of Nearest-Neighbour repulsion on a Honeycomb Lattice

Nearest neighbour repulsion energies $E_\mathrm{nn}$ are set using
```python
Simulation.set_nn_energy( <energy> ).
```
In this example, the lattice is constructed using the optional argument
```python
alternating_sites=True
```
which divides the lattice into alternating sites labelled A and B.

```python
def honeycomb_lattice_correlation_nn( n_atoms, nn_repulsion ):
    a, b = 4, 3
    spacing = 1.0
    n_jumps = 10000
    n_equilibration_jumps = 100 # need to include equilibration jumps.
    s = lattice_mc.Simulation()
    # Setting alternating_sites=True assigns site labels 'A' and 'B' to alternating sites on the lattice.
    s.lattice = lattice_mc.init_lattice.honeycomb_lattice( a, b, spacing, alternating_sites=True )
    s.set_number_of_atoms( n_atoms )
    s.set_number_of_jumps( n_jumps )
    s.set_number_of_equilibration_jumps( n_equilibration_jumps )
    s.set_nn_energy( nn_repulsion * kT )
    s.setup_lookup_table() # Create a lookup table for jump probabilities to speed up the simulation
    s.run()
    # Order parameter to quantify the preference for either site A or site B.
    # Sites A and B are equivalent by symmetry, so we take the absolute difference in populations.
    order_parameter = fabs( s.average_site_occupations['A'] - s.average_site_occupations['B'] ) / 24
    return s.tracer_correlation, s.collective_correlation, order_parameter
```

```python
def run_simulations( n_atoms, nn_energy, simulation_function, nproc, n_samples ):
    pool = multiprocessing.Pool( processes=nproc )
    f_data = []
    for n in n_atoms:
        for nn in nn_energy:
            calc_pool = ( ( n, nn ) for i in range( n_samples ) )
            f_sampled = np.array( pool.starmap( simulation_function, calc_pool ) )
            f = sum( f_sampled ) / n_samples
            f_data.append( [ n, nn, *f ] )
    return f_data
```

```python
n_atoms = np.array( [ 1, 8, 16, 24, 32, 40, 47 ] )
nn_energy = [ 0.0, 1.0, 2.0, 3.0, 4.0 ]
results = run_simulations( n_atoms, nn_energy, honeycomb_lattice_correlation_nn, nproc=NPROCS, n_samples=NSAMPLES )
data = pd.DataFrame( np.row_stack( results ), columns = [ 'n_atoms', 'E_nn', 'f', 'f_I', 'order_parameter' ] )
```

```python
for nn in nn_energy:
    nn_data=data[data.E_nn==nn]
    plt.plot( nn_data.n_atoms, nn_data.f, 'o-', label="E_nn = {}".format(nn) )
    plt.legend()
    plt.xlabel( r"$n_atoms$" )
    plt.ylabel( r"$f$" )
    plt.title( 'tracer correlation factor' )
plt.show()

for nn in nn_energy:
    nn_data=data[data.E_nn==nn]
    plt.plot( nn_data.n_atoms, nn_data.f_I, 'o-', label="E_nn = {}".format(nn) )
    plt.legend()
    plt.xlabel( r"$n_atoms$" )
    plt.ylabel( r"$f_\mathrm{I}$" )
    plt.title( 'collective correlation factor' )
plt.show()

for nn in nn_energy:
    nn_data=data[data.E_nn==nn]
    plt.plot( nn_data.n_atoms, nn_data.order_parameter, 'o-', label="E_nn = {}".format(nn) )
    plt.legend()
    plt.xlabel( r"$n_atoms$" )
    plt.ylabel( r"site preference asymmetry" )
    plt.title( 'order parameter' )
plt.show()
```

As the nearest-neighbour repulsion energy increases, both $f$ and $f_I$ are decreased as the lattice occupancy approaches 50% (cf. results published in [\[6\]](#ref6)). The order parameter plot shows that this is associated with a preference for all the mobile particles to order on one subset of alternating sites.

### On-site Energies

The on-site contribution to the energy is given by
\begin{equation}
  E = \sum_j E_\mathrm{site}^j.
\end{equation}
Within the code, sites are distinguished using labels, and different on-site energies can be set for each site label.

#### Example - The Effect of Site Inequivalence on a Honeycomb Lattice

Site occupation energied $E_\mathrm{site}$  are set using `Simulation.set_site_energies( <dict> )`.
In this example, the lattice is constructed using the optional argument `alternating_sites=True`, which divides the lattice into alternating sites labelled A and B. The site energies are then set with
```python
s.set_site_energies( { 'A' : 0.0, 'B' : site_delta_E * kT } )
```

```python
def honeycomb_lattice_correlation_si( n_atoms, site_delta_E ):
    a, b = 4, 3
    spacing = 1.0
    n_jumps = 10000
    n_equilibration_jumps = 100 # need to include equilibration jumps
    s = lattice_mc.Simulation()
    s.lattice = lattice_mc.init_lattice.honeycomb_lattice( a, b, spacing, alternating_sites=True )
    s.set_number_of_atoms( n_atoms )
    s.set_number_of_jumps( n_jumps )
    s.set_number_of_equilibration_jumps( n_equilibration_jumps )
    s.set_site_energies( { 'A' : 0.0, 'B' : site_delta_E * kT } )
    s.setup_lookup_table() # Create a lookup table for jump probabilities to speed up the simulation
    s.run()
    a_site_preference = ( s.average_site_occupations['A'] - s.average_site_occupations['B'] ) / 24
    return s.tracer_correlation, s.collective_correlation, a_site_preference
```

```python
def run_simulations( n_atoms, site_delta_E, simulation_function, nproc, n_samples ):
    pool = multiprocessing.Pool( processes=nproc )
    f_data = []
    for n in n_atoms:
        for dE in site_delta_E:
            calc_pool = ( ( n, dE ) for i in range( n_samples ) )
            f_sampled = np.array( pool.starmap( simulation_function, calc_pool ) )
            f = sum( f_sampled ) / n_samples
            f_data.append( [ n, dE, *f ] )
    return f_data
```

```python
n_atoms = np.array( [ 1, 8, 16, 24, 32, 40, 47 ] )
site_delta_E = [ 0.0, 1.0, 2.0, 3.0, 4.0 ]
results = run_simulations( n_atoms, nn_energy, honeycomb_lattice_correlation_nn, nproc=NPROCS, n_samples=NSAMPLES )
data = pd.DataFrame( np.row_stack( results ), columns = [ 'n_atoms', 'delta_E_site', 'f', 'f_I', 'A site preference' ] )
```

```python
for dE in site_delta_E:
    nn_data=data[data.delta_E_site==dE]
    plt.plot( nn_data.n_atoms, nn_data.f, 'o-', label=r"$\Delta E={}$".format( dE ) )
    plt.legend()
    plt.xlabel( r"$n_\mathrm{atoms}$" )
    plt.ylabel( r"$f$" )
plt.show()

for dE in site_delta_E:
    nn_data=data[data.delta_E_site==dE]
    plt.plot( nn_data.n_atoms, nn_data.f_I, 'o-', label=r"$\Delta E={}$".format( dE ) )
    plt.legend()
    plt.xlabel( r"$n_\mathrm{atoms}$" )
    plt.ylabel( r"$f_\mathrm{I}$" )
plt.show()

for dE in site_delta_E:
    nn_data=data[data.delta_E_site==dE]
    plt.plot( nn_data.n_atoms, nn_data['A site preference'], 'o-', label=r"$\Delta E={}$".format( dE ) )
    plt.legend()
    plt.xlabel( r"$n_\mathrm{atoms}$" )
    plt.ylabel( r"site occupation asymmetry" )
plt.show()
```

Making the A and B sites inequivalent, by introducing on-site energies, has a similar effect to nearest-neighbour repulsion.

## Defining a Lattice from a Sites File

All the examples so far have used one of the built-in lattice generation routines. Arbitrary lattices can be constructed using
```python
Simulation.define_lattice_from_file( <filename>, <cell lengths> )
```
A sites file starts with the number of sites, and then has the same number of blocks with the format:
```
site: <integer>
center: <float> <float> <float>
neighbours: <integer> × m
label: <string>
(optional) energy: <float>
```

So the top of a sites file might look like
```
576

site: 5
centre: 21.4669 -1.37099999975e-05 6.133405
neighbours: -2 -3 -4 -309
label: T

site: 10
centre: 21.4669 -1.37099999975e-05 30.667
neighbours: -6 -7 -8 -311
label: T
```

`site:` assigns an integer to each site that is used to identify it. Each site must have a unique identifying integer.
`centre:` is the coordinate of the lattice site.
`neighbours:` is the list of integers identifying the sites neighbouring this site.
`label:` identifies this site as belonging to a group of sites.
`energy:` sets the site energy (optional).

The `centre` and `neighbours` entries can be British or American English spellings (`centre`|`center`) and (`neighbours`|`neighbors`).

The program does not check that your cell paramateres are consistent with the coordinates provided in the site file. Instead, when the lattice is constructed, periodic boundary conditions are applied, and site coordinates lying outside the (orthorhombic) cell are mapped to an equivalent position inside the cell.

### Example: Lattice-Gas Monte Carlo Simulation on a Garnet Lattice

This reads the lattice site data from `llzo_lattice_site_list.dat`, which contains the site coordinates and connectivity for a 2×2×2 supercell of Li7La3Zr2O12 [\[1\]](#ref1).

```python
def garnet_lattice_correlation( n_atoms ):
    n_jumps = 10000
    n_equilibration_jumps = 100
    s = lattice_mc.Simulation()
    s.define_lattice_from_file( 'llzo_lattice_site_list.dat', [ 49.0672361, 49.0672361, 49.0672361 ] )
    s.set_number_of_atoms( n_atoms )
    s.set_number_of_jumps( n_jumps )
    s.set_number_of_equilibration_jumps( n_equilibration_jumps )
    s.setup_lookup_table() # Create a lookup table for jump probabilities to speed up the simulation
    s.run()
    return s.tracer_correlation, s.collective_correlation
```

```python
def run_simulations( n_atoms, simulation_function, nproc, n_samples ):
    pool = multiprocessing.Pool( processes=nproc )
    f_data = []
    for n in n_atoms:
        calc_pool = ( n for i in range( n_samples ) )
        f_sampled = np.array( pool.map( simulation_function, calc_pool ) )
        f = sum( f_sampled ) / n_samples
        f_data.append( [ n, *f ] )
    return f_data
```

```python
n_atoms = np.array( [ 1, 96, 192, 288, 384, 480, 575 ] )
results = run_simulations( n_atoms, garnet_lattice_correlation, nproc=NPROCS, n_samples=NSAMPLES )
data = pd.DataFrame( np.row_stack( results ), columns=[ 'n', 'f', 'f_I' ] )
```

```python
# plot the data
plt.plot( data.n, data.f,   'o-', label='f'  )
plt.plot( data.n, data.f_I, 'o-', label='f_I')
plt.legend()
plt.xlabel( r"$n_\mathrm{atoms}$" )
plt.ylabel( r"$f$, $f_\mathrm{I}$" )
plt.show()
```

## Running simulations for a set time, instead of a set number of jumps

Call the Simulation `.run()` method with the optional argument `for_time=<simulation time in seconds>`.

e.g.

```python
s = lattice_mc.Simulation()
```
…
```
s.run( for_time=1.0 ) # run for 1 second.
```

# References

[1] B. J. Morgan, [*R. Soc. Open Sci.* **4**, 170824 (2017)](https://dx.doi.org/10.1098/rsos.170824).
[2] A. Van der Ven *et al.* [*Acc. Chem. Res.* **46**, 1216 (2013)](https://dx.doi.org/10.1021/ar200329r).
[3] G. E. Murch [*Sol. Stat. Ionics* **7**, 177 (1982)](https://dx.doi.org/10.1016/0167-2738%2882%2990050-9).
[4] R. Kutner [*Phys. Lett.* **81A**, 239 (1981)](https://dx.doi.org/10.1016/0375-9601%2881%2990251-6).
[5] Morgan and Madden, [*J. Phys. Condens. Matter* **24**, 275303 (2012)](http://www.iopscience.iop.org/article/10.1088/0953-8984/24/27/275303/).
 [6] G. E. Murch & R. J. Thorn, [Phil. Mag. **36** 529 (1977)](http://dx.doi.org/10.1080/14786437708239737).

```python

```


    BAYESIAN LEARNING FOR PARTIALLY OBSERVED DYNAMICAL SYSTEMS

__**Students**__

 >- ADEIKALAM Pierre
 >- CHEN Guangyue
 >- MORALES Katherine
 >- XU Kevin

 __**Professors**__
 >- DOUC Randal
 >- LECORFF Sylvain

      ON MARKOV CHAIN MONTE CARLO METHODS FOR TALL DATA

### Contents

[1. Introduction](#s1)

[2. Naive Approaches](#s2)

[3. Confidence Sampler](#s3)

[4. Conclusion](#s4)

[5. Code](#s5)

##  1. Introduction

###   1.0 The Paper

>The research paper this work is based on is "**On Markov chain Monte Carlo methods for tall data"** by [ R. Bardenet, A. Doucet and C. Holmes.](http://www.jmlr.org/papers/volume18/15-205/15-205.pdf)
>
>
> This paper aims to tackle the problem of applying Markov chain Monte Carlo (MCMC) methods for Bayesian inference on *tall* datasets where the number of observations $n$ is very large.
>
>As we will see, the quantity of observations in a dataset plays a crucial role in the speed and effectiveness of MCMC methods relying on the Metropolis-Hastings sampling algorithm, and the authors propose diverse workarounds to try and preserve the target distribution while **alleviating the computational workload** needed to implement such methods.
>
> The first aim of this notebook is to provide readers an easy to digest overview of some methods they might naïvely try in practice and give compelling arguments based on the authors' work as to why these naïve methods should not be used. Then, we will try to give an overview of a much more promising approach developed by Bardenet et. al based on a technique called confidence sampling.

###   1.1 Bayesian Inference

>Let $X = \{x_1, \dots, x_n \} \subset  \mathbb R ^d$  be a dataset and a parameter space  $\Theta$. We assume the observations are conditionally independent, with the associated average log-likelihood being given by:
>
>\begin{equation}
\tag{1}
 l(\theta) = \frac{1}{n}\sum_{i=1}^{n} \log p(x_i|\theta).
\end{equation}
>
>**Bayesian approach:**
>
>Let $p(\theta)$ be the prior distribution of  $\theta$, then the posterior distribution of $\theta$ is given by:
>
>\begin{align}
\tag{2}
 \pi(\theta) = p(\theta | X) = \frac{p(\theta) \prod_{i=1}^{n} p(x_i|\theta)}{p(X)}\; \; \propto \; \;\gamma(\theta),
\end{align}
>where $\gamma(\theta):=  p(\theta) e^{(n\; l(\theta))}$ is the unnormalized version of $\pi$.

###   1.2 Metropolis-Hastings Algorithm

> To sample from the distribution $\pi$, one can use the Metropolis-Hastings (MH) algorithm to simulate a Markov chain  $(\theta_k)_{k \geq1}$ of invariant distribution $\pi$
>
>
>---
**Algorithm 1:**  Metropolis-Hastings
>
>---
>
>**Input:**
  - $\gamma(\cdot)$: unnormalized version of $\pi$
  - $q(\cdot|\cdot)$: proposal distribution
  - $N_{iter}$: number of iterations

>
>**For** $k\leftarrow$ 1 : $N_{iter}$
  1.  $\theta \leftarrow \theta_{k-1}$
  2.  $\theta' \sim q( . | \theta)$
  3.   $\alpha(\theta, \theta ') \leftarrow \frac{\gamma(\theta ')}{\gamma(\theta)}\frac{q( \theta | \theta')}{q( \theta' | \theta)} $
  4. $ U \sim \mathcal U(0,1)$
  5.  **If** $U < \alpha(\theta, \theta ')$
    - $\theta_k \leftarrow \theta '$
  6. **else**
    - $\theta_k \leftarrow \theta$
>
>Return: $(\theta_k)_{k =1, \dots, N_{iter}}$
>

>
>---

>The observations of the dataset are independent. Thus, following equation (2), we can rewrite step 3 of the above algorithm to
>\begin{equation}
\tag{3}
 \log \alpha(\theta, \theta ')  = \log \Big [ \frac{p(\theta')}{p(\theta)}\frac{q(\theta | \theta')}{q(\theta' | \theta)} \Big] +n \big[ l(\theta ') -l(\theta) \big] .
\end{equation}

**Problem:**

>Since $l(\theta) = \frac{1}{n}\sum_{i=1}^{n} \log p(x_i|\theta)$, computing $l(\theta')$ and $l(\theta)$ requires a full pass over the dataset, which renders this algorithm unapplicable in practice as the computation needed for a single iteration will be too heavy.
>
>
> The authors propose three types of workarounds:
>
>- **Divide and conquer approaches:** Divide the data into batches, run the MH algorithm on each batch separately and combine the results.
>
>- **Exact subsampling approaches:** Use unbiased estimators of the target distribution and plug them into the computation of the acceptance ratio of the MH algorithm.
>
>- **Approximate subsampling approaches:** Instead of approximating the target distribution, directly try to approximate the acceptance ratio such that the resulting stationary distribution is close enough to the target distribution.

**Our work:**
>In this notebook, we will look at a naïve approach from each type of solution. Sadly, we will show that their most basic form do not provide simple solutions to the application of MCMC methods for  tall datasets. However, they are very helpful to understand several potential approaches to scale up MCMC on tall datasets, and each of them could be tried.

##    2. Naïve Approaches

####   2.1 Divide and conquer

> Divide-and-conquer approaches appear as a natural way of handling tall data as they divide the initial dataset into batches, run MCMC on each batch in parallel, and then combine these results to obtain an approximation of the posterior distribution. Formally, assuming the data $X$ is divided into batches $x_1, ... , x_B$,  the latter task can be written as:
>
>
>\begin{equation}
\tag{4}
\pi(\theta) = p(\theta|X) \propto \prod_{i=1}^{B}p(\theta)^{\frac{1}{B}}p(x_i|\theta) \propto \prod_{i=1}^{B} \pi_i(\theta)
\end{equation}
>
> which is justified **only if we assume the batch posteriors $\pi_i$ to be Gaussian.**
>
> Two main problems arise:
- **MCMC methods do not return posteriors as mathematical functions**, so an extra step is needed to combine the batch posteriors using this trick.
- **How to efficiently combine non-Gaussian batch posteriors?**

> The authors mention different ways of combining the batch posteriors:
>
>-  Fit a Gaussian approximation to the MCMC draws of each batch posterior and combine these approximations using $(4)$ [(Huang and Gelman, 2005)](http://www.stat.columbia.edu/~gelman/research/unpublished/comp7.pdf).
>
>-  Compute a weighted average of the MCMC draws of each batch posterior. Under Gaussian assumptions , this average should follow the same distribution as the combined batch posteriors [(Scott et al., 2013)](https://www.researchgate.net/publication/295098887_Bayes_and_Big_Data_The_Consensus_Monte_Carlo_Algorithm).
>
>-  For each batch $x_i$, target the artificial posterior $\pi_i(\theta) \propto p(\theta)^{\frac{1}{B}}p(x_i|\theta)$. Then, fit a smooth approximation to each batch posterior $\pi_i$ and multiply them [(Neiswanger
et al., 2014)](http://www.cs.cmu.edu/~epxing/papers/2014/Neiswanger_Wang_Xing_UAI14a.pdf).
>
>However, these methods are theoretically justified only when batch posteriors are Gaussian, or **when the size of each batch goes to infinity**, which defeats the purpose of this approach.
>
>**The supports of the $\pi_i$ can be disjoint and as a result the product of their approximations will yield a poor approximation of $\pi$.**

####   2.2 Pseudo-Marginal Metropolis-Hastings: The challenge of Unbiased Estimation

> Pseudo-Marginal MH consists in using an unbiased *almost surely non-negative* **estimator $\hat\gamma(\theta)$** of the unnormalized target distribution $\gamma(\theta)$ instead of evaluating $\gamma(\theta)$ at each iteration of the MH algorithm.
>
> We could then replace the evaluation of $\gamma(\theta)$ by an evaluation of $\hat\gamma(\theta)$ which should be faster to compute and still give the correct acceptance decision most of the time.
>
> We recall that :
>
> $$ \gamma(\theta) = p(\theta)e^{nl(\theta)} $$ with $$nl(\theta) = \sum_{i=1}^{n} l_i(\theta)$$
>
>
> The most naïve estimator one could build for $nl(\theta)$ is:
>
> $$n\hat l(\theta) = \frac{n}{t} \sum_{i = 1}^{t}\mathrm{log} p(x_i^* | \theta)$$
>
> which is an unbiased estimator of $nl(\theta)$.
>
> However, $\mathbb{E}[e^{n \hat l (\theta)}] \neq e^{\mathbb{E}[n\hat l(\theta)]}$ which means that $e^{n\hat l(\theta)}$ is not an unbiased estimator of $e^{nl(\theta)}$.
>
> [Jacob & Thierry (2015)](https://arxiv.org/abs/1309.6473v1)  showed that it is actually *impossible* to build an unbiased estimator of $e^{nl(\theta)}$ with just an unbiased estimator of $n\hat l(\theta)$ without making any further assumptions.

> We can build an unbiased estimator of $e^{nl(\theta)}$ if for example we were to **assume for all $i$ and for all $\theta$ that there exists $a(\theta)$ such that $l_i(\theta) \gt a(\theta)$**. Then, we can apply ([Rhee and Glynn, 2015](https://pdfs.semanticscholar.org/8a84/01f86ca70891293d41ea1c3c0d7877f7910d.pdf), Theorem 1) to build an unbiased estimator of $e^{n(l(\theta) - a(\theta))}$:
>
> *The following formulas of the estimators and inequalities are given for the sake of rigor, but the takeaway message of this section relies only on the very last inequality which highlights the inapplicability of this naïve method.*

> For $j\geq 1$, we draw independently with replacement $t$ samples $(x_{1, j}^*, ..., x_{t, j}^*)$ and define
>
> $$ D_j^* = \frac{n}{t} \sum_{i = 1}^{t}\mathrm{log} p(x_{i,j}^* | \theta) - na(\theta) = n \hat l_j(\theta) - na(\theta)$$
>
> Then, $D_j$ is an unbiased estimator of $n(l(\theta) - a(\theta)$. Let $N$ be a geometric random variable with the parameter $\frac{\epsilon}{\epsilon + 1}$, $\epsilon \gt 0$. If we take:
>
> $$ Z = e^{na(\theta)}\left[1 + \sum_{k = 1}^{N} \frac{P(N \geq k)}{k!} \prod_{j = 1}^{k}D_j^* \right] $$
>
> Then by Theorem 1 of (Rhee and Glynn, 2015), $Z$ is an unbiased estimator of $e^{n(l(\theta) - a(\theta))}$. However, it is crucial if we want to plug $\hat \gamma(\theta) = Z p(\theta) $ in the MH algorithm that we control the variance of Z.
>
> From ([Rhee and Glynn, 2015](https://pdfs.semanticscholar.org/8a84/01f86ca70891293d41ea1c3c0d7877f7910d.pdf)), we have the following lower bound on the *relative* variance of Z:
>
> $$ \frac{Var Z}{e^{2nl(\theta)}} \geq \frac{e^{-2n(l(\theta) - a(\theta)) + 2n\sqrt{(1 + \epsilon )\left[ \sigma_t(\theta)^2 + (l(\theta) - a(\theta))^2\right]}}}{n\sqrt{(1 + \epsilon)\left[ \sigma_t(\theta)^2 + (l(\theta) - a(\theta))^2\right]}} + \mathcal{O}(1)$$
>
> The important bit of this lower bound is given by $$e^{-2n(l(\theta) - a(\theta)) + 2n\sqrt{(1 + \epsilon )\left[ \sigma_t(\theta)^2 + (l(\theta) - a(\theta))^2\right]}}$$
>
> with $\sigma_t(\theta)$ being the standard deviation of $\hat l(\theta)$.
>
> We can see that for the relative variance to not explode exponentially with $n$, it is necessary for $n\sigma_t(\theta)$ to be of order $1$. From the Central Limit Theorem, we know that $\sigma_t(\theta)$ is of order $t^{-1/2}$ which means that the subsample size $t$ should be of order $n^2$ which completely defeats the purpose of this approach.

####   2.3 Naive subsampling

> Assuming we gave up on building an unbiased estimator of $e^{nl(\theta)}$, naïve subsampling is a naïve Pseudo-Marginal MH technique that consists in plugging inside MH the following unbiased estimator of the average loglikelihood:
$$\hat{l}(\theta)=\frac 1n \sum_{i=1}^{n}\frac{z_i}{\lambda}l_i(\theta)$$
>where the $z_i \sim  B(1, \lambda)$ are i.i.d.

>
> We know from the previous section that this estimator does not preserve the target distribution. However, *(Doucet et al., 2015)* have shown that under Gaussian assumptions, Pseudo-Marginal MH can still be effective if the variance of the average loglikelihood estimation is kept to 1 or 3 depending on the performance of the vanilla MH algorithm.
>
> We have chosen to illustrate the Naïve subsampling approach as the variance of the average loglikelihood can easily be written:
>
>\begin{align}
Var [\hat{l}(\theta)] &= Var \Big[\sum_{i=1}^{n} \frac{z_i}{\lambda}l_i(\theta)\Big] = \frac 1 {\lambda^2}\sum_{i=1}^{n} l_i(\theta)^2 Var[z_i] \\
&=\frac{(1-\lambda)}{\lambda} \sum_{i=1}^{n} l_i(\theta)^2
\end{align}
>
> Since $\sum_{i=1}^{n} l_i(\theta)^2$ grows with $n$, for $Var [\hat{l}(\theta)]$ to be close to 1 we need $\lambda$ to be close to 1.
> As $\lambda$ is the parameter of the Bernoulli subsampling distribution then **there is no significant advantage between using this technique  or the complete data.**

##    3. Confidence sampler

####   3.1 Concept

> If we take a step back and look at the Metropolis-Hastings algorithm, we see that if we could just approximate the *acceptance ratio* $\alpha(\theta, \theta ')$ such that we are *confident* enough that our approximation will give the same decision as the actual acceptance ratio, then the stationary target distribution of this chain should be preserved. This is what the *confidence sampling* approach is about.
>
> We recall that the acceptance ratio is given by:
> $$\alpha(\theta, \theta ') = \min\left\{\frac{\pi(y_{t+1})q(x_t,y_{t+1})}{\pi(x_t)q(y_{t+1},x_t)},1\right\} $$
>
> If we let $u \sim \mathcal{U}(0, 1)$, then the acceptance decision is given by:
>
> $$ \alpha(\theta, \theta ') \geq u \iff \mathrm{log}\alpha(\theta, \theta ') \geq \mathrm{log}(u)$$
>
> which is equivalent to:
>
> $$ \frac{1}{n}\sum_{i=1}^{n}\mathrm{log}\left(\frac{p(x_i|\theta ')}{p(x_i|\theta)}\right) \geq \frac{1}{n}\mathrm{log}u - \frac{1}{n} \mathrm{log}\left(\frac{p(\theta ')q(\theta | \theta')}{p(\theta)q(\theta' | \theta)}\right)$$
>
> If we can approximate the loglikelihood ratio on the left well enough, then the acceptance decision taken by using this approximation should be the same than the one taken with the full loglikelihood ratio. Let us now build such a procedure.
>
> For all $\theta, \theta ' \in \Theta$, we assume that there exists $C_{\theta, \theta '} \gt 0$ such that:
>
> $$ \max_{i = 1,...,n} \left|\mathrm{log}\left( \frac{p(x_i |\theta)}{p(x_i | \theta ')} \right) \right| \leq C_{\theta, \theta '}$$
>
>
> Fix $\theta, \theta ' \in \Theta$. We can use a *Bernstein* or *Hoeffding* type of **concentration inequality** to bound our approximation of the loglikelihood ratio with respect to the number of samples used. We obtain, for a subsample $(x_1^*, ... , x_t^*)$ of size $t$, that :
>
> $$\mathbb{P}\left(\left|\frac{1}{t} \sum_{i=1}^{t} \mathrm{log}\left( \frac{p(x_i^* |\theta)}{p(x_i^* | \theta ')}\right) - \frac{1}{n} \sum_{i=1}^{n} \mathrm{log}\left( \frac{p(x_i^* |\theta)}{p(x_i^* | \theta ')}\right) \right| \leq c_t(\delta) \right) \geq 1 - \delta$$
>
> where $$c_t(\delta) = C_{\theta, \theta '} \sqrt{\frac{2(1 - f_t^*)\mathrm{log}(2/\delta)}{t}}$$
>
> and $f_t^* = \frac{t-1}{n}$ is approximately the fraction of used samples.
>
> For readers that are unfamiliar with concentration inequalities, this inequality roughly translates into:
>
> $$\mathbb{P}\left(\left| approximation - real \hspace{0.1cm}value \right| \leq constant \hspace{0.1cm} depending \hspace{0.1cm} on\hspace{0.1cm} \delta \right) \geq 1 - \delta $$
>
> Let $$\psi(u, \theta, \theta') = \frac{1}{n}\mathrm{log}u - \frac{1}{n} \mathrm{log}\left(\frac{p(\theta ')q(\theta | \theta')}{p(\theta)q(\theta' | \theta)}\right)$$
> $$\Lambda_t(\theta, \theta')= \frac{1}{t} \sum_{i=1}^{t} \mathrm{log}\left( \frac{p(x_i^* |\theta)}{p(x_i^* | \theta ')}\right)$$
>
> and let us place ourselves on the event $\left\{ \left| \Lambda_t(\theta, \theta') - \Lambda_n(\theta, \theta') \right| \leq c_t(\delta)\right\}$, which is the event described by the concentration inequality.
>
> It follows from simple computation that on this event, if the event $\left\{ \left| \Lambda_t(\theta, \theta') - \psi(u, \theta, \theta') \right| \gt c_t(\delta)\right\}$ also holds, the approximated acceptance ratio will yield the correct decision. Indeed, both $\Lambda_t(\theta, \theta')$ and $\Lambda_n(\theta, \theta')$ will be on the same side of $\psi(u, \theta, \theta')$, so the decision will be the same. The idea of confidence sampling is then to sequentially increase $t$ until $\left\{ \left| \Lambda_t(\theta, \theta') - \psi(u, \theta, \theta') \right| \gt c_t(\delta)\right\}$ holds (which will eventually be the case as $c_t(\delta) \rightarrow 0$ when $t \rightarrow n$).

####   3.2 Algorithm : Confidence Sampler MH (Bardenet et al., 2014)

#####   3.2.1 Pseudocode of the algorithm

>---
**Algorithm 2:**  Confidence Sampler
>
>---
>
>**Input:**
  + $p(x|\theta)$ : A likelihood
  + $p(\theta)$ : A prior distribution
  + $q(\theta'|\theta)$ : A proposal distribution
  + $\theta_0$ : A starting point
  + $N_{iter}$ : Number of iterations
  + $\mathcal{X}$ : The dataset
  + $\delta$ : The probability for the concentration inequality
  + $C_{\theta, \theta'}$ : The range of each log likelihood ratio
  + $\gamma$ : The ratio at which the batchsize is increased geometrically
>
>**Output:**
  + a Markov chain
>
> 1. **for** $k \leftarrow 1$ **to** $N_{iter}$
> 2. $\hspace{1cm} \theta \leftarrow \theta_{k-1}$
> 3. $\hspace{1cm} \theta' \sim q(.|\theta), u \sim \mathcal{U}_{(0,1)}$,
> 4. $\hspace{1cm} \psi(u, \theta, \theta') \leftarrow \dfrac{1}{n} \log \left[ u \dfrac{p(\theta)q(\theta'|\theta)}{p(\theta')q(\theta|\theta')} \right]$
> 5. $\hspace{1cm} t \leftarrow 0$
> 6. $\hspace{1cm} t_{look} \leftarrow 0$
> 7. $\hspace{1cm} \Lambda^* \leftarrow 0$
> 8. $\hspace{1cm} \mathcal{X}^* \leftarrow \emptyset $
> 9. $\hspace{1cm} b \leftarrow 1$
> 10. $\hspace{1cm} Done \leftarrow False$
> 11. $\hspace{1cm} \textbf{while} \; Done = False \; \textbf{do}$
> 12. $\hspace{2cm} x_{t+1}^*,...,x_b^* \sim_{w/o \; repl.} \mathcal{X} \backslash \mathcal{X}^*$
> 13. $\hspace{2cm} \mathcal{X}^* \leftarrow \mathcal{X}^* \cup \{ x_{t+1}^*,...,x_b^* \}$
> 14. $\hspace{2cm} \Lambda^* \leftarrow \dfrac{1}{b} \left( t \Lambda^* + \sum_{i=t+1}^b \log \left[ \dfrac{p(x_i^*|\theta')}{p(x_i^*|\theta)} \right] \right)$
> 15. $\hspace{2cm} t \leftarrow b$
> 16. $\hspace{2cm} c \leftarrow c_t(\delta)$
> 17. $\hspace{2cm} b \leftarrow n \land [\gamma t]$
> 18. $\hspace{2cm} \textbf{if} \; |\Lambda^* - \psi(u, \theta, \theta')| \geq c \; \textbf{or} \; t=n$
> 19. $\hspace{3cm} Done \leftarrow True$
> 20. $\hspace{1cm} \textbf{if} \; \Lambda^* \geq \psi(u, \theta, \theta') $
> 21. $\hspace{2cm} \theta_k \leftarrow \theta'$ # __ACCEPT__
> 22. $\hspace{1cm} \textbf{else} \; \theta_k \leftarrow \theta \hspace{2mm}$   # __REJECT__
> 23. $\textbf{return} \; (\theta_k)_{k=1,...,N_{iter}} $

At each iteration, the algorithm first draws a candidate $\theta'$ from the proposal distribution and a uniform random number $u$ as in the MH algorithm. $\psi(u, \theta, \theta')$ is also computed. Then, we initialize the variables that will be used to find the correct $\Lambda^*$ which corresponds to an approximation of the loglikelihood ratio.

To compute $\Lambda^*$, one needs to find the subsample size $b$ such that the right MH acceptance decision is taken with probability at least $1-\delta$.
In order to achieve this, the algorithm uses a while loop for each iteration in which $b$ is increased geometrically. The points of the batch are drawn uniformly without replacement from the dataset. A confidence bound $c_t(\delta)$ is also computed for a valid concentration inequality such as Hoeffding's or Bernstein's. Finally, the while loop ends until either we have gone over the whole dataset or the confidence condition is satisfied.

Afterwards, to decide whether to accept or reject the candidate $\theta'$, the algorithm compares $\psi(u, \theta, \theta')$ and $\Lambda^*$.

#####   3.2.2 Implementation

The confidence sampler is already implemented in Python in the notebook provided by the authors of the paper. So, we did not implement it again.

The plot below is a histogram of the batchsizes used by the confidence sampler for a simple example: We fit a one-dimensional normal distribution to $10^5$ i.i.d. points drawn according to a normal distribution $\mathcal{N}(0,1)$.

We observe that almost each iteration uses batchsize equals to the total number of individual data points. That is why this confidence sampler is still computationally not efficient, even for this extremely simple example.

####   3.3 Improved confidence sampler

#####  3.3.1 Improved confidence sampler

> The authors propose a modified version of the confidence sampler. They use a proxy to replace approximated loglikelihood ratio and the same trick is used: **If we are *confident enough* in our proxy, the acceptance decision should be identical**.
>
> In an example, we will show that this new algorithm can require less than $O(n)$ likelihood evaluations per iteration.
>
> The conditions required for this algorithm to work are the following:
> For $\theta, \theta' \in \Theta$
> > 1. $\wp_i (\theta, \theta') \approx l_i(\theta') - l_i(\theta)$
> > 2. $\sum_{i=1}^n \wp_i (\theta, \theta')$ is cheap to compute
> > 3. $|l_i(\theta') - l_i(\theta) - \wp_i (\theta, \theta')|$ can be bounded uniformly for $i \in [\![1, n]\!]$ and the bound is cheap to compute.
>
> **The third condition is the one that lets us use a concentration inequality and bound the *confidence* in the approximated acceptance ratio.**
>
> These assumptions are very strong, but this method shows a very promising approach to scaling up MCMC. Let us illustrate this potential with an example.

#####   3.3.2 Example Proxy: Taylor Expansion

> We assume that around some reference value $\theta_*$, the log likelihood ratio can be approximated by:
>
> $$\hat l_i(\theta) = l_i(\theta_*) + g_{i,*}^T(\theta - \theta_*) + \frac{1}{2}(\theta - \theta_*)^T H_{i,*}(\theta - \theta_*)$$
>
> where $g_{i,*}$ is the gradient of $l_i$ at $\theta_*$ and $H_{i,*}$ is the Hessian matrix of $l_i$ at $\theta_*$.
>
> We also assume that the third order derivative of $l_i$ is uniformly bounded for all $i$.
>
> If we now define:
> $$\wp_i(\theta, \theta') = \hat l_i(\theta') - \hat l_i(\theta) \approx l_i(\theta') - l_i(\theta)$$
>
>Then, this proxy satisfies all the conditions required to be used in the improved confidence sampling algorithm.
>
> If we have already precomputed:
> $$\hat\mu = \frac{1}{n}\sum_{i = 1}^{n} g_{i,*}$$
> $$\hat S = \frac{1}{n}\sum_{i = 1}^{n} H_{i,*}$$
>
> Then, the following holds:
> $$\frac{1}{n}\sum_{i=1}^n \wp_i(\theta, \theta') = \hat\mu^T(\theta' - \theta) + \frac{1}{2}(\theta' - \theta)^T \hat S(\theta' + \theta + 2\theta_*)$$
>
> which means that $\frac{1}{n}\sum_{i=1}^n \wp_i(\theta, \theta')$ can effectively be computed in $O(1)$.

As in the last section, the improved confidence sampler with *Taylor expansions* as proxies is also already implemented in the notebook provided.

Here we used the same example as in 3.2.2 and we obtain the following histogram with the new method.

The resulting Markov chains are the same in both confidence samplers but the the difference is in the number of likelihood evaluations used at each iteration.

We note that the batchsizes are in mean drastically smaller compared to the previous confidence sampler. Each iteration uses only a really small quantity of likelihood evaluations to make the decision.

**In conclusion**, the improved confidence sampler algorithm yields far better results than all previous methods but it requires strong and specific assumptions.

#####   3.3.2 Application: Logistic Regression

> In the logistic regression model, the loglikelihood is given by:
> $$l_i(\theta) = \phi(y_i x_i^T\theta)$$
>
> where $\phi(z) = -\mathrm{log}\left( 1 + e^{-z}\right)$, $y_i$ and $x_i$ are respectively the labels and the features of $i$th observation.
>
> Then, we can use the Taylor expansion proxy with:
> $$g_{i, *} = \phi'(y_i x_i^T\theta_*)y_i x_i$$
>
> $$H_{i, *} = \phi''(y_i x_i^T\theta_*)x_i x_i^T$$
>
> We also have that $|\phi'''(z)| =\left| \frac{\mathrm{tanh}(z/2)}{\mathrm{cosh}^2(z/2)} \right| \leq \frac{1}{4}$, which means that we can uniformly bound $|l_i(\theta') - l_i(\theta) - \wp_i (\theta, \theta')|$:
> $$|l_i(\theta') - l_i(\theta) - \wp_i (\theta, \theta')| \leq \frac{1}{24} \max_{i = 1}^{n} \|x_i\|^3 (\|\theta' - \theta_*\| + \|\theta - \theta_*\|)$$
>
> Therefore, this proxy satisfies the conditions required.
>
> One last detail remains however: **the choice of $\theta_*$**.
>
> The authors propose that if one sets $\theta_*$ to be the state of the Markov chain every $\alpha$ MH iteration. We will need to do a full pass over the dataset to compute $\hat\mu$ and $\hat S$ but this cost is still overall negligeable.

##   4. Conclusion

> Sadly, unlike for gradient descent there are no "plug-and-play" approaches to scaling up MCMC methods to tall datasets:
> - Divide and conquer approaches are unreliable as they have no theoretical backing for non-Gaussian settings.
> - Pseudo-Marginal approximations are difficult to control and as a result are unpredictible *as is*.
> - While being theoretically justified, vanilla Confidence Sampling fails to make substantial gains in computation efficiency in practice.
>
> Thankfully, the scaling of MCMC methods to tall datasets is a very active area of research and novel approaches such as the use of proxies for an improved confidence sampling algorithm show promise and may provide a path forward to solving this problem.

##   5. Code

> In this section we present illustrative examples, the user can modify several parameters and visualize the results. Additionally, the codes contain comments for better understanding of the user.

>- **Generation of the data:** We generate $n$ i.i.d points drawn according:
>  - $X_i \sim  \mathcal N( \mu = 0, \sigma=1)$ or
>  - $X_i \sim  \log \mathcal N( \mu = 0, \sigma=1)$

>- **Parameters of MH algorithm:**
>  - priori distribution: $p(\theta)=p(\mu, \log\sigma) \propto 1$
>  - proposal distribution $\theta^*$: isotropic Gaussian random walk, whose stepsize $\lambda$ is firts set proportional to $\frac 1 {\sqrt{n}}$ and then adapted during the first 1 000 iterations so as to reach $50\%$ acceptance.
$$\theta^* = \theta + \lambda Z,$$
where $Z\sim \mathcal N( 0,I_2)$, $I_2$ is the $2\times 2$ identity matrix, and $\lambda > 0$.

See: [ON THE EFFICIENCY OF PSEUDO-MARGINAL RANDOM WALK METROPOLIS ALGORITHMS](https://arxiv.org/pdf/1309.7209.pdf)
>  - **$N_{ite}:$** number of iterations.

```python
#@title <font size='4'> Libraries</font> {display-mode: "form"}
"""""""""""""""""
Required packages
"""""""""""""""""
import numpy as np
import numpy.random as npr
import scipy.stats as sps
import scipy.special as spsp
import scipy.misc as spm
import scipy.optimize as spo
import numpy.linalg as npl
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import matplotlib.cm as cmx
import sympy as sym
import time
import seaborn as sns
import seaborn.distributions as snsd
import math as math

import warnings
mpl.style.use('seaborn') #Background
warnings.filterwarnings('ignore')

from IPython.display import HTML
from matplotlib import animation, rc
import ipywidgets as widgets
from ipywidgets import HBox, VBox
from IPython.display import display

```

```python
#@title <font size='4'> Metropolis Hastings algorithm code</font> {display-mode: "form"}

def getLogLhd(x, mu, sigma):
    """
    return an array of Gaussian log likelihoods up to a constant
    """
    return -(x-mu)**2/(2*sigma**2) - np.log(sigma)

def vanillaMH(T):
    """
    perform traditional isotropic random walk Metropolis
    T: Number of iterations
    """
    # True value mu, log(sigma^2)
    theta = np.array([realMean,np.log(realStd)])
    stepsize = .5/np.sqrt(N)
    S = np.zeros((T, 2))
    acceptance = 0.0

    for i in range(T):
        accepted = 0
        done = 0
        thetaNew = theta
        #- Proposal: isotropic random walk (definition)
        thetaP = theta + stepsize*npr.randn(2)
        u = npr.rand()# Uniform
        #- Diference between \theta' (proposal) and \theta_{t-1}  (array)
        #- to calculate alpha, we have a prior distribution equal to one
        lhds = getLogLhd(x, thetaP[0], np.exp(thetaP[1])) - getLogLhd(x, theta[0], np.exp(theta[1]))
        #- l(\theta): average log-likelihood
        Lambda = np.mean(lhds)
        #- q(\theta | \theta')/ q(\theta' | \theta) = 1 then log(.) = 0
        psi = (1./N)*np.log(u)
        if Lambda>psi:#it is equivalent to u < alpha(\theta,\theta' )
            thetaNew = thetaP
            theta = thetaP
            accepted = 1
            S[i,:] = thetaNew
        else:
            S[i,:] = theta

        if i<T/10:
            # Perform some adaptation of the stepsize in the early iterations
            stepsize *= np.exp(1./(i+1)**0.6*(accepted-0.5)) #(?)
        #- Historical acceptance rate (each iteration)
        acceptance*=i
        acceptance+=accepted
        acceptance/=(i+1)
        if np.mod(i,T/10)==0:
          print("Iteration %d and  Acceptance percentage %.2f "%(i,100.*np.round(acceptance,4)))

    return S
```

```python
#@title <font size='4'> Select the distribution and enter the number of data points :</font> {display-mode: "form"}

dist = widgets.ToggleButtons(
    options=['Normal', 'Log-normal'],
    description='Distribution:',
)

n_p=widgets.IntText(
    value=100000,
    description='n:',
    disabled=False
)
tab1 = VBox(children=[dist,
                      n_p])
tab = widgets.Tab(children=[tab1])
tab.set_title(0, 'Generation of the data')
VBox(children=[tab])
```

    VBox(children=(Tab(children=(VBox(children=(ToggleButtons(description='Distribution:', options=('Normal', 'Log…

```python
#@title <font size='4'> Data Generation</font> {display-mode: "form"}
#- Seed
npr.seed(1)
# Number of points
N = n_p.value
#- Distribution
dataType = dist.value
#- Generation of points
if dataType == "Normal":
    x = npr.randn(N)
    print('Normal data simulation')
elif dataType == "Log-normal":
    x = npr.lognormal(0,1,size=N)
    print('Log-normal data simulation')
# We store the mean and std deviation for later reference, they are also the MAP and MLE estimates in this case.
realMean = np.mean(x)
realStd = np.std(x)

print('Number of points= %d'%N)
print("Mean =", np.round(realMean,4))
print ("standard deviation =", np.round(realStd,4))
# Plot of points generates
plt.hist(x, 30, normed=True)
plt.title('Distribution of the generated data')
plt.show()
```

```python
#@title <font size='4'> Enter the number of iterations for vanilla MH:</font> {display-mode: "form"}
n_iter = widgets.IntText(
    value = 10000,
    description = 'Iterations:',
    disabled = False
)
n_iter
```

    IntText(value=10000, description='Iterations:')

```python
#@title <font size='4'> Running vanilla MH algorithm</font> {display-mode: "form"}
S = vanillaMH(n_iter.value)

#plt.figure(figsize=(15,7))
#plt.subplot(1,2,1)
#plt.plot(S[:,0],label = r"MH chain of: $\mu$")
#plt.axhline(0, color='red', label = r"$\mu = 0 $")
#plt.ylim([-0.1, 0.1])
#plt.xlabel('Iterations')
#plt.legend()
#plt.subplot(1,2,2)
#plt.plot(np.exp(S[:,0]),label = r"MH chain of: $\sigma$")
#plt.axhline(1, color='red',label = r"$\sigma = 1 $")
#plt.xlabel('Iterations')
#plt.legend()
#plt.ylim([0.90, 1.1])
#plt.show()

```

    Iteration 0 and  Acceptance percentage 0.00
    Iteration 1000 and  Acceptance percentage 50.35
    Iteration 2000 and  Acceptance percentage 50.72
    Iteration 3000 and  Acceptance percentage 50.68
    Iteration 4000 and  Acceptance percentage 50.79
    Iteration 5000 and  Acceptance percentage 50.43
    Iteration 6000 and  Acceptance percentage 50.47
    Iteration 7000 and  Acceptance percentage 50.19
    Iteration 8000 and  Acceptance percentage 50.18
    Iteration 9000 and  Acceptance percentage 49.89

```python
#@title <font size='4'> Confidence Sampler implementation</font> {display-mode: "form"}

## Concentration bounds
def ctBernsteinSerfling(N,n,a,b,sigma,delta):
    """
    Bernstein-type bound without replacement, from (Bardenet and Maillard, to appear in Bernoulli)
    """
    l5 = np.log(5/delta)
    kappa = 7.0/3+3/np.sqrt(2)
    if n<=N/2:
        rho = 1-1.0*(n-1)/N
    else:
        rho = (1-1.0*n/N)*(1+1.0/n)
    return sigma*np.sqrt(2*rho*l5/n) + kappa*(b-a)*l5/n

def ctHoeffdingSerfling(N,n,a,b,delta):
    """
    Classical Hoeffding-type bound without replacement, from (Serfling, Annals of Stats 1974)
    """
    l2 = np.log(2/delta)
    if n<=N/2:
        rho = 1-1.0*(n-1)/N
    else:
        rho = (1-1.0*n/N)*(1+1.0/n)
    return (b-a)*np.sqrt(rho*l2/2/n)

def ctBernstein(N,n,a,b,sigma,delta):
    """
    Classical Bernstein bound, see e.g. the book by Boucheron, Lugosi, and Massart, 2014.
    """
    l3 = np.log(3/delta)
    return sigma*np.sqrt(2*l3/n) + 3*(b-a)*l3/n

## Confidence MCMC (Bardenet, Doucet, and Holmes, ICML'14)
def confidenceMCMC(T):

    # Initialize
    theta = np.array([realMean,np.log(realStd)])
    stepsize = .01
    S_B = np.zeros((T,2))
    delta = .1
    acceptance = 0.0
    gamma = 1.5
    ns_B = []

    bs = []

    for i in range(T):

        npr.shuffle(x)
        accepted = 0
        done = 0
        thetaNew = theta
        thetaP = theta + stepsize*npr.randn(2)
        u = npr.rand()
        n = N/10
        cpt = 0
        lhds = getLogLhd(x, thetaP[0], np.exp(thetaP[1])) - getLogLhd(x, theta[0], np.exp(theta[1]))
        a = np.min(lhds)
        b = np.max(lhds)

        while not done and n<N:

            n = min(N,np.floor(gamma*n))
            cpt+=1
            deltaP = delta/2/cpt**2
            # The following step should be done differently to avoid recomputing previous likelihoods, but for the toy examples we keep it short
            lhds = getLogLhd(x[:int(n)], thetaP[0], np.exp(thetaP[1])) - getLogLhd(x[:int(n)], theta[0], np.exp(theta[1]))
            Lambda = np.mean(lhds)
            sigma = np.std(lhds)
            psi = np.log(u)/N
            if np.abs(Lambda-psi) > ctBernstein(N,n,a,b,sigma,deltaP):
                done = 1
        bs.append(n)

        if i>1 and ns_B[-1] == 2*N:
            ns_B.append(n) # Half of the likelihoods were computed at the previous stage
        else:
            ns_B.append(2*n) # The algorithm required all likelihoods for theta and theta', next iteration we can reuse half of them

        if Lambda>psi:
            # Accept
            theta = thetaP
            accepted = 1
            S_B[i] = thetaP
        else:
            # Reject
            S_B[i] = theta

        if i<T/10:
            # Perform some adaptation of the stepsize in the early iterations
            stepsize *= np.exp(1./(i+1)**0.6*(accepted-0.5))

        acceptance*=i
        acceptance+=accepted
        acceptance/=(i+1)
        if np.mod(i,T/10)==0:
            # Monitor acceptance and average number of samples used
            print("Iteration", i, "Acceptance", acceptance, "Avg. num evals", np.mean(ns_B), "sigma/sqrt(n)", sigma/np.sqrt(n), "R/n", (b-a)/n)

    return S_B, ns_B, bs
```

```python
#@title <font size='4'> Enter the number of iterations for Confidence Sampler:</font> {display-mode: "form"}
n_iter_conf = widgets.IntText(
    value = 10000,
    description = 'Iterations:',
    disabled = False
)
n_iter_conf
```

    IntText(value=10000, description='Iterations:')

```python
#@title <font size='4'> Running the Confidence Sampler</font> {display-mode: "form"}

S_B, ns_B, bs = confidenceMCMC(n_iter_conf.value)
```

    Iteration 0 Acceptance 0.0 Avg. num evals 200000.0 sigma/sqrt(n) 4.395225050451293e-05 R/n 1.5390382929998826e-06
    Iteration 1000 Acceptance 0.4905094905094903 Avg. num evals 148947.92507492506 sigma/sqrt(n) 2.0524631030610058e-05 R/n 5.513361838972485e-07
    Iteration 2000 Acceptance 0.5237381309345327 Avg. num evals 148457.95452273864 sigma/sqrt(n) 1.221251411125473e-05 R/n 4.979166349999433e-07
    Iteration 3000 Acceptance 0.5334888370543155 Avg. num evals 148237.14561812728 sigma/sqrt(n) 1.827195489979916e-05 R/n 4.948659768960084e-07
    Iteration 4000 Acceptance 0.5446138465383661 Avg. num evals 148095.23619095227 sigma/sqrt(n) 3.526295495338081e-06 R/n 1.7891358035839234e-07
    Iteration 5000 Acceptance 0.5406918616276746 Avg. num evals 148087.6898620276 sigma/sqrt(n) 1.0740827112220162e-05 R/n 3.9614072911498265e-07
    Iteration 6000 Acceptance 0.5384102649558408 Avg. num evals 148121.50541576403 sigma/sqrt(n) 1.1200128300823346e-05 R/n 3.608274174049231e-07
    Iteration 7000 Acceptance 0.5380659905727752 Avg. num evals 148088.48178831596 sigma/sqrt(n) 5.56185432900585e-06 R/n 1.707119093347198e-07
    Iteration 8000 Acceptance 0.5396825396825402 Avg. num evals 147965.3655793026 sigma/sqrt(n) 7.503746148951691e-06 R/n 1.9528925988161916e-07
    Iteration 9000 Acceptance 0.5364959448950124 Avg. num evals 148026.46328185758 sigma/sqrt(n) 8.007518402064551e-06 R/n 5.138271740664444e-07

```python
#@title <font size='4'> Histogram of the number of likelihood evaluations:</font> {display-mode: "form"}
plt.hist(bs)

labStr = "mean="+str(np.around(1.0*np.mean(bs)/N*100,1))+"%"
plt.axvline(np.mean(bs), linewidth = 4, color="blue", label=labStr)
plt.axvline(N, linewidth = 4, color="k", label="n")
labStr = "median="+str(np.around(1.0*np.median(bs)/N*100,1))+"%"
plt.axvline(np.median(bs), linewidth = 4, color="blue",linestyle='--', label=labStr)
plt.legend()
plt.xlabel("Batchsize")
plt.ylabel("Number of iterations")
plt.show()
```

```python
#@title <font size='4'> Implementation of the Improved Confidence Sampler with Taylor expansions </font> {display-mode: "form"}

def combineMeansAndSSQs(N1, mu1, ssq1, N2, mu2, ssq2):
    """
    combine means and sum of squares of two sets
    """
    dd = mu2 - mu1
    mu = mu1
    ssq = ssq1
    N = N1+N2
    mu += dd*N2/N
    ssq += ssq2
    ssq += (dd**2) * N1 * N2 / N
    return N, mu, ssq

### Differential functions for proxies,

### Define vectorized evaluation of gradient and Hessian
myGradientVect = lambda x_float, mu_float, sigma_float:np.array([-(2*mu_float - 2*x_float)/(2*sigma_float**2), -1/sigma_float + (-mu_float + x_float)**2/sigma_float**3]).T
myHessianVect = lambda x_float, mu_float, sigma_float:[[-1/sigma_float**2*np.ones(x_float.shape), 2*(mu_float - x_float)/sigma_float**3], [2*(mu_float - x_float)/sigma_float**3, (1 - 3*(mu_float - x_float)**2/sigma_float**2)/sigma_float**2]]

# Compute third order derivatives to bound the Taylor remainder. Symbolic differentiation is not really necessary in this simple case, but
# it may be useful in later applications
def thirdDerivatives():
    x, mu, sigma = sym.symbols('x, mu, sigma')
    L = []
    for i in range(4):
        for j in range(4):
            if i+j == 3:
                args = tuple([-(x-mu)**2/(2*sigma**2) -sym.log(sigma)] + [mu for cpt in range(i)] + [sigma for cpt in range(j)])
                L.append( sym.diff(*args) )
    return L

def evalThirdDerivatives(x_float, mu_float, logSigma_float):
    tt = thirdDerivatives()
    return [tt[i].subs('x',x_float).subs('mu',mu_float).subs('sigma',np.exp(logSigma_float)).evalf() for i in range(4)]

# Find the MAP (not really necessary here since the MAP are the mean and std deviation of the data)
f = lambda theta: -np.mean(getLogLhd(x, theta[0], np.exp(theta[1])))
thetaMAP = spo.minimize(f, np.array([realMean, np.log(realStd)])).x

### Confidence MCMC with proxy (Bardenet, Doucet, and Holmes, this submission)
def confidenceMCMCWithProxy(T):

    npr.seed(1)
    # Initialize
    theta = np.array([realMean,np.log(realStd)])
    stepsize = .01
    S_B = np.zeros((T,2))
    delta = .1
    acceptance = 0.0
    gamma = 2.
    ns_B = []

    # Compute some statistics of the data that will be useful for bounding the error and averaging the proxies
    minx = np.min(x)
    maxx = np.max(x)
    meanx = np.mean(x)
    meanxSquared = np.mean(x**2)

    # Prepare total sum of Taylor proxys
    muMAP = thetaMAP[0]
    sigmaMAP = np.exp(thetaMAP[1])
    meanGradMAP = np.array( [(meanx - muMAP)/sigmaMAP**2, (meanxSquared-2*muMAP*meanx+muMAP**2)/sigmaMAP**3 - 1./sigmaMAP] )
    meanHessMAP = np.array( [[-1./sigmaMAP**2, -2*(meanx-muMAP)/sigmaMAP**3], [-2*(meanx-muMAP)/sigmaMAP**3, -3*(meanxSquared-2*muMAP*meanx+muMAP**2)/sigmaMAP**4 + 1/sigmaMAP**2]] )

    bs = []

    for i in range(T):

        npr.shuffle(x)
        accepted = 0
        done = 0
        thetaNew = theta
        thetaP = theta + stepsize*npr.randn(2)
        u = npr.rand()
        n = 2
        t0 = 0
        cpt = 0
        Lambda = 0
        ssq = 0 # Sum of squares

        # Prepare Taylor bounds
        xMinusMuMax = np.max(np.abs([1, minx-theta[0], maxx-theta[0], minx-thetaMAP[0], maxx-thetaMAP[0], minx-thetaP[0], maxx-thetaP[0]]))
        sigmaMin = np.min(np.exp([theta[1], thetaMAP[1], thetaP[1]]))
        R = float(np.max(np.abs(evalThirdDerivatives(xMinusMuMax, 0, np.log(sigmaMin)))))
        h = np.array([theta[0]-thetaMAP[0], np.exp(theta[1])-np.exp(thetaMAP[1])])
        hP = np.array([thetaP[0]-thetaMAP[0], np.exp(thetaP[1])-np.exp(thetaMAP[1])])
        R *= 2*1./6 * max(np.sum(np.abs(h)), np.sum(np.abs(hP)))**3
        a = -R
        b = R

        # We can already compute the average proxy log likelihood ratio
        avgTotalProxy = np.dot(meanGradMAP, hP-h) + .5*np.dot( hP-h, np.dot(meanHessMAP, h+hP) )

        while not done and n<N:

            n = min(N,np.floor(gamma*n))
            cpt+=1
            deltaP = delta/2/cpt**2
            batch = x[int(t0):int(n)]
            lhds = getLogLhd(batch, thetaP[0], np.exp(thetaP[1])) - getLogLhd(batch, theta[0], np.exp(theta[1]))
            proxys = np.dot(myGradientVect(batch, muMAP, sigmaMAP), hP-h) + 0.5*np.dot(np.dot(hP-h, myHessianVect(batch,muMAP,sigmaMAP)).T, h+hP)
            if np.any(np.abs(lhds-proxys)>R):
                # Just a check that our error is correctly bounded
                print("Taylor remainder is underestimated")
            tmp, Lambda, ssq = combineMeansAndSSQs(t0, Lambda, ssq, n-t0, np.mean(lhds-proxys), (n-t0)*np.var(lhds-proxys))
            sigma = np.sqrt(1./n*ssq)
            psi = np.log(u)/N
            t0 = n
            if np.abs(Lambda-psi + avgTotalProxy) > ctBernstein(N,n,a,b,sigma,deltaP):
                done = 1
        bs.append(n)

        if i>1 and ns_B[-1] == 2*N:
            ns_B.append(n) # Half of the likelihoods were computed at the previous stage
        else:
            ns_B.append(2*n)

        if Lambda+avgTotalProxy>psi:
            # Accept
            theta = thetaP
            accepted = 1
            S_B[i] = thetaP
        else:
            # Reject
            S_B[i] = theta

        if i<T/10:
            # Perform some adaptation of the stepsize in the early iterations
            stepsize *= np.exp(1./(i+1)**0.6*(accepted-0.5))

        acceptance*=i
        acceptance+=accepted
        acceptance/=(i+1)
        if np.mod(i,T/10)==0:
            # Monitor acceptance and average number of samples used
            print("Iteration", i, "Acceptance", acceptance, "Avg. num samples", np.mean(ns_B), "Dist. to MAP", np.sum( np.abs(theta-thetaMAP) ), "sigma/sqrt(n)", sigma/np.sqrt(n), "R/n", R/n)

    return S_B, ns_B, bs
```

```python
#@title <font size='4'> Running the Improved Confidence Sampler</font> {display-mode: "form"}

S_BP, ns_BP, bs2 = confidenceMCMCWithProxy(10000)
```

    Iteration 0 Acceptance 0.0 Avg. num samples 1024.0 Dist. to MAP 0.0 sigma/sqrt(n) 3.1026756854543736e-09 R/n 6.472474098772885e-08
    Iteration 1000 Acceptance 0.49850149850149844 Avg. num samples 859.7082917082917 Dist. to MAP 0.0002493180558642231 sigma/sqrt(n) 4.594563220871523e-11 R/n 5.055738236017883e-08
    Iteration 2000 Acceptance 0.5072463768115952 Avg. num samples 954.5347326336831 Dist. to MAP 0.0061598892790331004 sigma/sqrt(n) 5.239127046418569e-09 R/n 3.4652091675663985e-08
    Iteration 3000 Acceptance 0.501166277907365 Avg. num samples 941.1449516827724 Dist. to MAP 0.010949986805270207 sigma/sqrt(n) 4.188643171210579e-08 R/n 1.8371592178874226e-07
    Iteration 4000 Acceptance 0.49337665583604184 Avg. num samples 960.5618595351162 Dist. to MAP 0.0029151686946896163 sigma/sqrt(n) 2.1925098484149743e-08 R/n 1.9431378819589596e-07
    Iteration 5000 Acceptance 0.49610077984403195 Avg. num samples 1092.4599080183964 Dist. to MAP 0.0010866662283205996 sigma/sqrt(n) 2.6516201557106773e-08 R/n 2.1103632341548628e-07
    Iteration 6000 Acceptance 0.4955840693217803 Avg. num samples 1144.0079986668889 Dist. to MAP 0.005822140513723733 sigma/sqrt(n) 5.408024042710274e-08 R/n 5.367141233132221e-07
    Iteration 7000 Acceptance 0.495072132552493 Avg. num samples 1145.1209827167547 Dist. to MAP 0.005379563166217382 sigma/sqrt(n) 4.9496544063918823e-11 R/n 1.827944262401288e-07
    Iteration 8000 Acceptance 0.4953130858642674 Avg. num samples 1119.8710161229847 Dist. to MAP 0.0037834568928888806 sigma/sqrt(n) 9.3969092335661e-09 R/n 1.357455396500425e-07
    Iteration 9000 Acceptance 0.49616709254527314 Avg. num samples 1138.8396844795022 Dist. to MAP 0.004839996649832641 sigma/sqrt(n) 1.2120683716210768e-09 R/n 7.038167027504496e-08

```python
#@title <font size='4'> Histogram of the number of likelihood evaluations:</font> {display-mode: "form"}
plt.hist(bs2)

labStr = "mean="+str(np.around(1.0*np.mean(bs2)/N*100,1))+"%"
plt.axvline(np.mean(bs2), linewidth = 4, color="blue", label=labStr)
plt.axvline(N, linewidth = 4, color="k", label="n")
labStr = "median="+str(np.around(1.0*np.median(bs2)/N*100,1))+"%"
plt.axvline(np.median(bs2), linewidth = 4, color="blue",linestyle='--', label=labStr)
plt.legend()
plt.xlabel("Batchsize")
plt.ylabel("Number of iterations")
plt.show()
```

```python

```


```python
from IPython.display import Image
from IPython.core.display import HTML
from sympy import *
Image(url= "https://i.imgur.com/IBghUMh.png")
```

```python
Image(url= "https://i.imgur.com/ZspIgwo.png")
#https://www.albert.io/blog/how-to-use-the-midpoint-rule-in-ap-calculus/
```

```python
x,a,b,n = symbols('x a b n')

f = lambda x: x**3
a  = -1.5
b = 3.5
n = 5
dx = (b-a)/n
mylist = []

i = -0.5
g2 = f((dx*i + dx*(i-1))/2) * dx
print(g2)
mylist.append(g2)

i = 0.5
g3 = f((dx*i + dx*(i-1))/2) * dx
print(g3)
mylist.append(g3)

i = 1.5
g4 = f((dx*i + dx*(i-1))/2) * dx
print(g4)
mylist.append(g4)

i = 2.5
g5 = f((dx*i + dx*(i-1))/2) * dx
print(g5)
mylist.append(g5)

i = 3.5
g6 = f((dx*i + dx*(i-1))/2) * dx
print(g6)
mylist.append(g6)

sum(mylist[0:10])
```

    -1.0
    0.0
    1.0
    8.0
    27.0

    35.0

```python
Image(url= "https://i.imgur.com/4EWJrf3.png")
```


# Tutorial 2: Matrices
**Week 0, Day 3: Linear Algebra**

**By Neuromatch Academy**

__Content creators:__ Ella Batty

__Content reviewers:__ Keith van Antwerp, Aderogba Bayo, Anoop Kulkarni, Pooya Pakarian

__Production editors:__ Siddharth Suresh, Ella Batty

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

*Estimated timing of tutorial: 1 hour, 35 minutes*

During today, we will learn the basics of linear algebra, focusing on the topics that underlie the material on future days in the NMA Computational Neuroscience course. In this tutorial, we focus on matrices: their definition, their properties & operations, and especially on a geometrical intuition of them.

By the end of this tutorial, you will be able to :
* Explain matrices as a linear transformation and relate matrix properties to properties of that linear transformation
*  Perform matrix multiplication by hand
*  Define what eigenvalues/eigenvectors are

**Code Credit:**

Some elements of this problem set are from or inspired by https://openedx.seas.gwu.edu/courses/course-v1:GW+EngComp4+2019/about. In particular, we are using their `plot_linear_transformation` and `plot_linear_transformations` functions.

 Code under BSD 3-Clause License © 2019 Lorena A. Barba, Tingyu Wang. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

---
# Setup

```python
# Imports

import numpy as np
import matplotlib.pyplot as plt
```

##  Figure settings

```python
#@title Figure settings
import ipywidgets as widgets# interactive display
from ipywidgets import fixed
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

##  Plotting functions

```python
# @title Plotting functions
import numpy
from numpy.linalg import inv, eig
from math import ceil
from matplotlib import pyplot, ticker, get_backend, rc
from mpl_toolkits.mplot3d import Axes3D
from itertools import cycle

_int_backends = ['GTK3Agg', 'GTK3Cairo', 'MacOSX', 'nbAgg',
                 'Qt4Agg', 'Qt4Cairo', 'Qt5Agg', 'Qt5Cairo',
                 'TkAgg', 'TkCairo', 'WebAgg', 'WX', 'WXAgg', 'WXCairo']
_backend = get_backend()   # get current backend name

# shrink figsize and fontsize when using %matplotlib notebook
if _backend in _int_backends:
  fontsize = 4
  fig_scale = 0.75
else:
  fontsize = 5
  fig_scale = 1

grey = '#808080'
gold = '#cab18c'   # x-axis grid
lightblue = '#0096d6'  # y-axis grid
green = '#008367'  # x-axis basis vector
red = '#E31937'    # y-axis basis vector
darkblue = '#004065'

pink, yellow, orange, purple, brown = '#ef7b9d', '#fbd349', '#ffa500', '#a35cff', '#731d1d'

quiver_params = {'angles': 'xy',
                 'scale_units': 'xy',
                 'scale': 1,
                 'width': 0.012}

grid_params = {'linewidth': 0.5,
               'alpha': 0.8}

def set_rc(func):
  def wrapper(*args, **kwargs):
    rc('font', family='serif', size=fontsize)
    rc('figure', dpi=200)
    rc('axes', axisbelow=True, titlesize=5)
    rc('lines', linewidth=1)
    func(*args, **kwargs)
  return wrapper

@set_rc
def plot_vector(vectors, tails=None):
  ''' Draw 2d vectors based on the values of the vectors and the position of their tails.

  Parameters
  ----------
  vectors : list.
    List of 2-element array-like structures, each represents a 2d vector.

  tails : list, optional.
    List of 2-element array-like structures, each represents the coordinates of the tail
    of the corresponding vector in vectors. If None (default), all tails are set at the
    origin (0,0). If len(tails) is 1, all tails are set at the same position. Otherwise,
    vectors and tails must have the same length.

  Examples
  --------
  >>> v = [(1, 3), (3, 3), (4, 6)]
  >>> plot_vector(v)      # draw 3 vectors with their tails at origin
  >>> t = [numpy.array((2, 2))]
  >>> plot_vector(v, t)   # draw 3 vectors with their tails at (2,2)
  >>> t = [[3, 2], [-1, -2], [3, 5]]
  >>> plot_vector(v, t)   # draw 3 vectors with 3 different tails

  '''
  vectors = numpy.array(vectors)
  assert vectors.shape[1] == 2, "Each vector should have 2 elements."
  if tails is not None:
    tails = numpy.array(tails)
    assert tails.shape[1] == 2, "Each tail should have 2 elements."
  else:
    tails = numpy.zeros_like(vectors)

  # tile vectors or tails array if needed
  nvectors = vectors.shape[0]
  ntails = tails.shape[0]
  if nvectors == 1 and ntails > 1:
    vectors = numpy.tile(vectors, (ntails, 1))
  elif ntails == 1 and nvectors > 1:
    tails = numpy.tile(tails, (nvectors, 1))
  else:
    assert tails.shape == vectors.shape, "vectors and tail must have a same shape"

  # calculate xlimit & ylimit
  heads = tails + vectors
  limit = numpy.max(numpy.abs(numpy.hstack((tails, heads))))
  limit = numpy.ceil(limit * 1.2)   # add some margins

  figsize = numpy.array([2,2]) * fig_scale
  figure, axis = pyplot.subplots(figsize=figsize)
  axis.quiver(tails[:,0], tails[:,1], vectors[:,0], vectors[:,1], color=darkblue,
                  angles='xy', scale_units='xy', scale=1)
  axis.set_xlim([-limit, limit])
  axis.set_ylim([-limit, limit])
  axis.set_aspect('equal')

  # if xticks and yticks of grid do not match, choose the finer one
  xticks = axis.get_xticks()
  yticks = axis.get_yticks()
  dx = xticks[1] - xticks[0]
  dy = yticks[1] - yticks[0]
  base = max(int(min(dx, dy)), 1)   # grid interval is always an integer
  loc = ticker.MultipleLocator(base=base)
  axis.xaxis.set_major_locator(loc)
  axis.yaxis.set_major_locator(loc)
  axis.grid(True, **grid_params)

  # show x-y axis in the center, hide frames
  axis.spines['left'].set_position('center')
  axis.spines['bottom'].set_position('center')
  axis.spines['right'].set_color('none')
  axis.spines['top'].set_color('none')

@set_rc
def plot_transformation_helper(axis, matrix, *vectors, unit_vector=True, unit_circle=False, title=None):
  """ A helper function to plot the linear transformation defined by a 2x2 matrix.

  Parameters
  ----------
  axis : class matplotlib.axes.Axes.
    The axes to plot on.

  matrix : class numpy.ndarray.
    The 2x2 matrix to visualize.

  *vectors : class numpy.ndarray.
    The vector(s) to plot along with the linear transformation. Each array denotes a vector's
    coordinates before the transformation and must have a shape of (2,). Accept any number of vectors.

  unit_vector : bool, optional.
    Whether to plot unit vectors of the standard basis, default to True.

  unit_circle: bool, optional.
    Whether to plot unit circle, default to False.

  title: str, optional.
    Title of the plot.

  """
  assert matrix.shape == (2,2), "the input matrix must have a shape of (2,2)"
  grid_range = 20
  x = numpy.arange(-grid_range, grid_range+1)
  X_, Y_ = numpy.meshgrid(x,x)
  I = matrix[:,0]
  J = matrix[:,1]
  X = I[0]*X_ + J[0]*Y_
  Y = I[1]*X_ + J[1]*Y_
  origin = numpy.zeros(1)

  # draw grid lines
  for i in range(x.size):
    axis.plot(X[i,:], Y[i,:], c=gold, **grid_params)
    axis.plot(X[:,i], Y[:,i], c=lightblue, **grid_params)

  # draw (transformed) unit vectors
  if unit_vector:
    axis.quiver(origin, origin, [I[0]], [I[1]], color=green, **quiver_params)
    axis.quiver(origin, origin, [J[0]], [J[1]], color=red, **quiver_params)

  # draw optional vectors
  color_cycle = cycle([pink, darkblue, orange, purple, brown])
  if vectors:
    for vector in vectors:
      color = next(color_cycle)
      vector_ = matrix @ vector.reshape(-1,1)
      axis.quiver(origin, origin, [vector_[0]], [vector_[1]], color=color, **quiver_params)

  # draw optional unit circle
  if unit_circle:
    alpha =  numpy.linspace(0, 2*numpy.pi, 41)
    circle = numpy.vstack((numpy.cos(alpha), numpy.sin(alpha)))
    circle_trans = matrix @ circle
    axis.plot(circle_trans[0], circle_trans[1], color=red, lw=0.8)

  # hide frames, set xlimit & ylimit, set title
  limit = 4
  axis.spines['left'].set_position('center')
  axis.spines['bottom'].set_position('center')
  axis.spines['left'].set_linewidth(0.3)
  axis.spines['bottom'].set_linewidth(0.3)
  axis.spines['right'].set_color('none')
  axis.spines['top'].set_color('none')
  axis.set_xlim([-limit, limit])
  axis.set_ylim([-limit, limit])
  if title is not None:
    axis.set_title(title)

@set_rc
def plot_linear_transformation(matrix, *vectors, name = None, unit_vector=True, unit_circle=False):
  """ Plot the linear transformation defined by a 2x2 matrix using the helper
  function plot_transformation_helper(). It will create 2 subplots to visualize some
  vectors before and after the transformation.

  Parameters
  ----------
  matrix : class numpy.ndarray.
    The 2x2 matrix to visualize.

  *vectors : class numpy.ndarray.
    The vector(s) to plot along with the linear transformation. Each array denotes a vector's
    coordinates before the transformation and must have a shape of (2,). Accept any number of vectors.

  unit_vector : bool, optional.
    Whether to plot unit vectors of the standard basis, default to True.

  unit_circle: bool, optional.
    Whether to plot unit circle, default to False.

  """
  figsize = numpy.array([4,2]) * fig_scale
  figure, (axis1, axis2) = pyplot.subplots(1, 2, figsize=figsize)
  plot_transformation_helper(axis1, numpy.identity(2), *vectors, unit_vector=unit_vector, unit_circle=unit_circle, title='Before transformation')
  plot_transformation_helper(axis2, matrix, *vectors, unit_vector=unit_vector, unit_circle=unit_circle, title='After transformation')
  if name is not None:
    figure.suptitle(f'Population {name}')

def plot_eig_vec_transform(W):
  classic = 'k'
  vec_names = ['a', 'b','c','d','e','f','g', 'h']

  _, vecs = np.linalg.eig(W)
  vecs = vecs.T

  fig, axes = plt.subplots(1, 2, figsize=(2, 1))
  colors = plt.rcParams['axes.prop_cycle'].by_key()['color']

  for i in range(2):
    axes[i].set(xlim=[-3.5, 3.5], ylim=[-3.5,3.5])
    axes[i].axis('Off')
    axes[i].plot([0, 0], [-3.5, 3.5], classic, alpha=.4)
    axes[i].plot([-3.5, 3.5], [0, 0], classic, alpha=.4)

  for i_vec, vec in enumerate(vecs):
    axes[0].arrow(0, 0, vec[0], vec[1], head_width=.2, facecolor=colors[i_vec], edgecolor=colors[i_vec], length_includes_head=True)
    axes[0].annotate(vec_names[i_vec], xy=(vec[0]+np.sign(vec[0])*.15, vec[1]+np.sign(vec[1])*.15), color=colors[i_vec])

    transformed_vec = np.matmul(W, vec)
    axes[1].arrow(0, 0, transformed_vec[0], transformed_vec[1], head_width=.2, facecolor=colors[i_vec], edgecolor=colors[i_vec], length_includes_head=True)
    axes[1].annotate(vec_names[i_vec], xy=(transformed_vec[0]+np.sign(transformed_vec[0])*.15, transformed_vec[1]+np.sign(transformed_vec[1])*.15), color=colors[i_vec])

  axes[0].set_title('Before')
  axes[1].set_title('After')
```

---

# Section 1: Intro to matrices

## Section 1.1: Matrices to solve systems of equations

###  Video 1: Systems of Equations

```python
# @title Video 1: Systems of Equations
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Aq4y1x7hf", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="3ecnOrMEh00", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video covers using a matrix to solve a linear system of equations.

 Click here for text recap of video

In a variety of contexts, we may encounter systems of linear equations like this one:

$$
\begin{align}
3x_1 + 2x_2 + x_3 &= y_1 \\
7x_1 + x_2 + 2x_3 &= y_2 \\
x_1 - x_2  - 2x_3 &= y_3 \\
\end{align}$$
We may know all the x's and want to solve for y's, or we may know the y's and want to solve for the x's. We can solve this in several different ways but one especially appealing way is to cast it as a matrix-vector equation:

$$\mathbf{W}\mathbf{x} = \mathbf{y}$$
where
$$\begin{align}
\mathbf{W} &= \begin{bmatrix} 3 & 2 & 1 \\ 7 & 1 & 2 \\ 1 &-1 &-2 \end{bmatrix}, \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}, \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix}\\
\end{align}$$

If we know $\mathbf{W}$ and $\mathbf{x}$, we can solve for $\mathbf{y}$ using matrix-vector multiplication. Each row of $\mathbf{y}$ is computed as the dot product of the equivalent row of $\mathbf{W}$ and $\mathbf{x}$.

If we know $\mathbf{W}$ and $\mathbf{y}$, we can sometimes solve for $\mathbf{x}$ by using the inverse of $\mathbf{W}$:
$$ \mathbf{x} = W^{-1}\mathbf{y} $$.
The reason this only sometimes works will be dived into later in this tutorial!

### Coding Exercise 1.1: Understanding neural transformations

We will look at a group of 2 LGN neurons which get input from 2 retinal neurons: we will call the population of LGN neurons population p. Below, we have the system of linear equations that dictates the neuron models for each population. $r_1$ and $r_2$ correspond to the retinal neural activities (of neuron 1 and 2). $g_{p_1}$ and  $g_{p_2}$ correspond to the responses of the LGN neurons 1 and 2 in population p.

$$\begin{align}
r_1 + 3r_2 &= g_{p_1} \\
2r_1 + r_2 &= g_{p_2} \\
\end{align}$$

1) Cast each this as a matrix-vector multiplication:

$$ \mathbf{g}_p = \mathbf{P}\mathbf{r} $$
where P is the weight matrix to population p.

2) Let's say we only recorded from the LGN cells (and know the weight matrix) and are trying to figure out how the retinal cells responded. Solve the matrix equation for the given LGN activities:

$$\mathbf{g}_p = \begin{bmatrix}
16 \\
7 \\
\end{bmatrix}$$

```python
# Create P (using np array)
P = np.array([
    [1, 3],
    [2, 1]
])

# Create g_p (using np array)
g_p = np.array([16, 7])

# Solve for r (using np.linalg.inv)
r = np.linalg.inv(P).dot(g_p)

# Print r
print(r)
```

    [1. 5.]

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_2fa7cf85.py)

You should see the output [1, 5]

You can recover how the retinal neurons respond given the weight matrix and LGN responses! You have solved the system of equations using matrices. We can't always do this though: let's say we have a different group of 2 LGN neurons -  population q - with the following weight matrix from the retinal neurons.

$$Q = \begin{bmatrix}
4 & 1 \\
8 & 2 \\
\end{bmatrix}, $$

As you can see if you uncomment and run the next code cell, we get an error if we try to invert this matrix to solve the equation. We'll find out more about this in the next sections.

```python
# g_q = np.array([16, 7])
# Q = np.array([[4, 1], [8, 2]])

# print(np.linalg.inv(Q) @ g_q)
```

## Section 1.2: Matrices as linear transformations

*Estimated timing to here from start of tutorial: 20 min*

###  Video 2: Linear Transformations

```python
# @title Video 2: Linear Transformations
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1KB4y1T7zM", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="N6UUV9tVIr8", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Matrices can be thought of as enacting linear transformations. When multiplied with a vector, they transform it into another vector. In fact, they are transforming a grid of space in a linear manner: the origin stays in place and grid lines remain straight, parallel, and evenly spaced.

### Coding Exercise 1.2: Creating matrices for transformations

Come up with a matrix $A$ for which the corresponding linear transformation is reflection through the $y$ axis (flipping across the $y$ axis). For example, $\mathbf{x} = \begin{bmatrix}
2 \\
6  \\
\end{bmatrix}$ should become $\mathbf{b} = \begin{bmatrix}
-2 \\
6  \\
\end{bmatrix}$ when multiplied with $A$.

**Remember to think about where your basis vectors should end up! Then your matrix consists of the transformed basis vectors. Drawing out what you want to happen can help**

```python
A = np.array([
    [-1, 0],
    [0, 1]
])

# Uncomment to visualize transformation
plot_linear_transformation(A)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_a86560cc.py)

*Example output:*

Come up with a matrix $A$ for which the corresponding linear transformation is projecting onto the $x$ axis. For example, $\bar{x} = \begin{bmatrix}
2 \\
3  \\
\end{bmatrix}$ should become $\bar{b} = \begin{bmatrix}
2 \\
0  \\
\end{bmatrix}$ when multiplied with $A$.

```python
A = np.array([
    [1, 0],
    [0, 0]
])

# Uncomment to visualize transformation
plot_linear_transformation(A)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_8d909295.py)

*Example output:*

## Section 1.3: Rank & Null Space

*Estimated timing to here from start of tutorial: 35 min*

###  Video 3: Rank & Null Space

```python
# @title Video 3: Rank & Null Space
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1vw411R7eA", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="ay2p5rgkMcY", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

###### This video covers properties of matrices, such as range, rank, and null space, in the context of matrices enacting a linear transformation.

 Click here for text recap of video

Square matrices always result in vectors with the same dimensions (number of components) but can alter the dimensionality of the transformed space. Let's say you have a 3 x 3 matrix. You will transform from 3-dimensional vectors to 3-dimensional vectors. You will often be transforming from all of 3D space to all of 3D space. However, this isn't always the case! You could have a 3 x 3 matrix that always results in a vector that lies along a 2D plane through 3D space. This matrix would be transforming from a 3 dimensional vector space (all of R3) to a 2 dimensional vector space (the 2D plane).

Matrices that aren't square are enacting transformations that change the dimensionality of the vectors. If you have a 4 x 5 matrix, you are transforming 5-dimensional vectors to 4-dimensional vectors. Similarly, if you have a 4 x 2 matrix, you are transforming from 4-dimensional vectors to 2-dimensional vectors.

The **range of a matrix** is the set of all possible vectors it can lead to after a transformation. The **rank of a matrix** is the dimensionality of the range.

Sometimes, a matrix will transform a non-zero vector into a zero vector (the origin). The **null space** of a matrix is the set of all vectors that will be transformed into the origin.

### Think! 1.3: Neural coding

Let's return to the setup of the previous coding exercise: we have two populations of LGN neurons, p and q, responding to retinal neurons. Visualize the linear transformations of these matrices by running the next code cell. Then discuss the following questions:

1) What are the ranks of weight matrix P and Q?

2) What does the null space of these matrices correspond to in our neuroscience setting? **Advanced:** What do you think the dimensionality of the null space is for P and Q?

3) What is the intrinsic dimensionality of the population of neural responses in population p? How about in q? The intrinsic dimensionality is the minimal number of dimensions required for a complete representation of the data.

4) If we wanted to decode retinal neural activity from the LGN activities, would we always be able to completely recover the retinal activity when looking at population p? How about population q? What does this tell us about the information loss of the neural processing?

```python
# @markdown Execute to visualize linear transformations
P = np.array([[1, 3], [2, 1]])
plot_linear_transformation(P, name = 'p')

Q = np.array([[4, 1], [8, 2]])
plot_linear_transformation(Q, name = 'q')
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_95edac8e.py)

1) P is transforming from all of 2D space to all of 2D space. The range of P is
  all of 2D space so the rank is 2. Q is squishing space from 2D to a 1D line.
  The range of Q is that 1D line so the rank is 1.

2) The null space is the patterns of retinal activity that result in 0 firing for the
   LGN neurons. The dimensionality of the null space of P is 0 (only the origin would
   be mapped to the origin). The dimensionality of the null space of Q is 1: when we
   squish 2D space onto the 1D line, a full 1D line would be squished onto the origin.

3) The intrinsic dimensionality of the responses of neurons in population p is 2
 (the possible response pairs span all of 2D space). For population q, the intrinsic dimensionality
  is 1: the possible response pairs lie along a 1D line. These neural responses could be fully described by just
  1 number if we switched to a basis for that line.

4) We could completely recover the retinal neural activity given the LGN activities in population
  p as the matrix P is full rank and invertible. There is no information loss from the
  two retinal neurons to the two LGN neurons. We cannot recover retinal activity given population
  q since we lose a dimension. There is information loss between retinal neurons and population q
 LGN neurons.

---
# Section 2: Eigenvalues & Eigenvectors

*Estimated timing to here from start of tutorial: 65 min*

##  Video 4: Eigenstuff

```python
# @title Video 4: Eigenstuff
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1KK4y1M7Ez", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="l-c7ptT7znM", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video covers eigenvalues and eigenvectors.

Eigenvectors, $\mathbf{v}$ of a matrix $\mathbf{W}$ are vectors that, when multipled by the matrix, equal a scalar multiple of themselves. That scalar multiple is the corresponding eigenvalue, $\lambda$.

$$\mathbf{W}\mathbf{v} = \lambda\mathbf{v} $$

If we have one eigenvector for a matrix, we technically have an infinite amount: every vector along the span of that eigenvector is also an eigenvector. So, we often use the unit vector in that direction to summarize all the eigenvectors along that line.

We can find the eigenvalues and eigenvectors of a matrix in numpy using `np.linalg.eig`.

## Think! 2: Identifying transformations from eigenvectors

Earlier, we learned how to think about linear transformations in terms of where the standard basis vectors end up. We can also think about them in terms of eigenvectors.

Just by looking at eigenvectors before and after a transformation, can you describe what the transformation is in words? Try for each of the two plots below.

Note that I show an eigenvector for every eigenvalue. The x/y limits do not change in before vs after (so eigenvectors are showed scaled by the eigenvalues).

Here are some transformation words to jog your memory and guide discussion: contraction, expansion, horizontal vs vertical, projection onto an axis, reflection, and rotation.

###

```python
# @title
# @markdown Execute this cell to visualize vectors

W = np.array([[3, 0], [0, 1]])
plot_eig_vec_transform(W)
```

###

```python
# @title
# @markdown Execute this cell to visualize vectors

W = np.array([[0, 1], [1, 0]])
plot_eig_vec_transform(W)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_80704181.py)

1) The first transformation is a horizontal expansion. You can see that b doesn't
  change length, while a grows longer. Since these eigenvectors are aligned
  with the axes, we know that there is no movement in the vertical direction
  (based on b) and an expansion in the horizontal direction (based on a).

2) This transformation is a reflection over the line y = x. Neither eigenvector
  changes length, a is along the line y = x and b is flipped over it.

As we saw above, looking at how just the eigenvectors change after a transformation can be very informative about what that transformation was.

---
# Section 3: Matrix multiplication

*Estimated timing to here from start of tutorial: 80 min*

##  Video 5: Matrix Multiplication

```python
# @title Video 5: Matrix Multiplication
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Rb4y1C7cE", width=730, height=410, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="OFyWfegC9Cs", width=730, height=410, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

We sometimes want to multiple two matrices together, instead of a matrix with a vector. Let's say we're multiplying matrices $\mathbf{A}$ and $\mathbf{B}$ to get $\mathbf{C}$:
$$ \mathbf{C} = \mathbf{A}\mathbf{B}$$.

We take the dot product of each row of A with each column of B. The resulting scalar is placed in the element of $\mathbf{C}$ that is the same row (as the row in A) and column (as the column in B). So the element of $\mathbf{C}$ at row 4 and column 2 is the dot product of the 4th row of $\mathbf{A}$ and the 2nd column of $\mathbf{B}$. We can write this in a formula as:

$$\mathbf{C}_{\text{row i, column j}} = \mathbf{A}_{\text{row i}} \cdot \mathbf{B}_{\text{column j}}$$

## Exercise 2: Computation corner

Break out the pen and paper - it's critical to implement matrix multiplication yourself to fully understand how it works.

Let's say we have 3 retina neurons and 2 LGN neurons. The weight matrix, $W$, between the retina and LGN neurons is:

$$W = \begin{bmatrix}
3 &  2 & 1 \\
1 & 2 & 7 \\
\end{bmatrix}, $$

We are going to look at the activity at two time steps (each time step is a column). Our retina activity matrix, $R$, is:

$$ R= \begin{bmatrix}
0 &  1  \\
2 & 4 \\
5 & 1 \\
\end{bmatrix} $$

Please compute the LGN neural activity, $G$, according to our linear model:

$$G = WR $$.

Please calculate it 1) by-hand and then 2) using code. Check that the answers match!

```python
# Compute by hand first!
```

```python
# Define R
R = np.array([
    [0, 1],
    [2, 4],
    [5, 1]
])

# Define W
W = np.array([
    [3, 2, 1],
    [1, 2, 7]
])

# Compute G
# in Python, we can use @ for matrix multiplication: matrix1 @ matrix2
G = W @ R

# Print values of G
print(G)
```

    [[ 9 12]
     [39 16]]

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W0D3_LinearAlgebra/solutions/W0D3_Tutorial2_Solution_4b71ea28.py)

---
# Summary

*Estimated timing of tutorial: 1 hour, 35 minutes*

In this tutorial, you have learned how to think about matrices from the perspective of solving a system of equations and as a linear transformation of space. You have learned:
- Properties of a matrix, such as rank & null space
- How the invertibility of matrices relates to the linear transform they enact
- What eigenvalues/eigenvectors are and why they might be useful

We will be using this knowledge in many of the days in the NMA computational neuroscience course.