#  Neuromatch Academy: Week 1, Day 1, Tutorial 2
# Model Types: "How" models
__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom

___
# Tutorial Objectives
This is tutorial 2 of a 3-part series on different flavors of models used to understand neural data. In this tutorial we will explore models that can potentially explain *how* the spiking data we have observed is produced

To understand the mechanisms that give rise to the neural data we save in Tutorial 1,  we will build simple neuronal models and compare their spiking response to real data. We will:
- Write code to simulate a  simple "leaky integrate-and-fire" neuron model
- Make the model more complicated — but also more realistic — by adding more physiologically-inspired details

```python
#@title Video 1: "How" models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='PpnagITsb3E', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=PpnagITsb3E

# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
# @title Figure Settings
import ipywidgets as widgets  # interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use(
    "https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle"
)
```

```python
# @title Helper Functions
def histogram(counts, bins, vlines=(), ax=None, ax_args=None, **kwargs):
    """Plot a step histogram given counts over bins."""
    if ax is None:
        _, ax = plt.subplots()

    # duplicate the first element of `counts` to match bin edges
    counts = np.insert(counts, 0, counts[0])

    ax.fill_between(bins, counts, step="pre", alpha=0.4,
                    **kwargs)  # area shading
    ax.plot(bins, counts, drawstyle="steps", **kwargs)  # lines

    for x in vlines:
        ax.axvline(x, color='r', linestyle='dotted')  # vertical line

    if ax_args is None:
        ax_args = {}

    # heuristically set max y to leave a bit of room
    ymin, ymax = ax_args.get('ylim', [None, None])
    if ymax is None:
        ymax = np.max(counts)
        if ax_args.get('yscale', 'linear') == 'log':
            ymax *= 1.5
        else:
            ymax *= 1.1
            if ymin is None:
                ymin = 0

    if ymax == ymin:
        ymax = None

    ax_args['ylim'] = [ymin, ymax]

    ax.set(**ax_args)
    ax.autoscale(enable=False, axis='x', tight=True)

def plot_neuron_stats(v, spike_times):
    fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(12, 5))

    # membrane voltage trace
    ax1.plot(v[0:100])
    ax1.set(xlabel='Time', ylabel='Voltage')
    # plot spike events
    for x in spike_times:
        if x >= 100:
            break
        ax1.axvline(x, color='red')

    # ISI distribution
    if len(spike_times) > 1:
        isi = np.diff(spike_times)
        n_bins = np.arange(isi.min(), isi.max() + 2) - .5
        counts, bins = np.histogram(isi, n_bins)
        vlines = []
        if len(isi) > 0:
            vlines = [np.mean(isi)]
        xmax = max(20, int(bins[-1]) + 5)
        histogram(counts,
                  bins,
                  vlines=vlines,
                  ax=ax2,
                  ax_args={
                      'xlabel': 'Inter-spike interval',
                      'ylabel': 'Number of intervals',
                      'xlim': [0, xmax]
                  })
    else:
        ax2.set(xlabel='Inter-spike interval', ylabel='Number of intervals')
    plt.show()
```

# Section 1: The Linear Integrate-and-Fire Neuron

How does a neuron spike?

A neuron charges and discharges an electric field across its cell membrane. The state of this electric field can be described by the _membrane potential_. The membrane potential rises due to excitation of the neuron, and when it reaches a threshold a spike occurs. The potential resets, and must rise to a threshold again before the next spike occurs.

One of the simplest models of spiking neuron behavior is the linear integrate-and-fire model neuron. In this model, the neuron increases its membrane potential $V_m$ over time in response to excitatory input currents $I$ scaled by some factor $\alpha$:

\begin{align}
  dV_m = {\alpha}I
\end{align}

Once $V_m$ reaches a threshold value a spike is produced, $V_m$ is reset to a starting value, and the process continues.

Here, we will take the starting and threshold potentials as $0$ and $1$, respectively. So, for example, if $\alpha I=0.1$ is constant---that is, the input current is constant---then $dV_m=0.1$, and at each timestep the membrane potential $V_m$ increases by $0.1$ until after $(1-0)/0.1 = 10$ timesteps it reaches the threshold and resets to $V_m=0$, and so on.

Note that we define the membrane potential $V_m$ as a scalar: a single real (or floating point) number. However, a biological neuron's membrane potential will not be exactly constant at all points on its cell membrane at a given time. We could capture this variation with a more complex model (e.g. with more numbers). Do we need to?

The proposed model is a 1D simplification. There are many details we could add to it, to preserve different parts of the complex structure and dynamics of a real neuron. If we were interested in small or local changes in the membrane potential, our 1D simplification could be a problem. However, we'll assume an idealized "point" neuron model for our current purpose.

#### Spiking Inputs

Given our simplified model for the neuron dynamics, we still need to consider what form the input $I$ will take. How should we specify the firing behavior of the presynaptic neuron(s) providing the inputs to our model neuron?

Unlike in the simple example above, where $\alpha I=0.1$, the input current is generally not constant. Physical inputs tend to vary with time. We can describe this variation with a distribution.

We'll assume the input current $I$ over a timestep is due to equal contributions from a non-negative ($\ge 0$) integer number of input spikes arriving in that timestep. Our model neuron might integrate currents from 3 input spikes in one timestep, and 7 spikes in the next timestep. We should see similar behavior when sampling from our distribution.

Given no other information about the input neurons, we will also assume that the distribution has a mean (i.e. mean rate, or number of spikes received per timestep), and that the spiking events of the input neuron(s) are independent in time. Are these reasonable assumptions in the context of real neurons?

A suitable distribution given these assumptions is the Poisson distribution, which we'll use to model $I$:

\begin{align}
  I \sim \mathrm{Poisson}(\lambda)
\end{align}

where $\lambda$ is the mean of the distribution: the average rate of spikes received per timestep.

### Exercise 1: Compute $dV_m$

For your first exercise, you will write the code to compute the change in voltage $dV_m$ (per timestep) of the linear integrate-and-fire model neuron. The rest of the code to handle numerical integration is provided for you, so you just need to fill in a definition for `dv` in the `lif_neuron` function below. The value of $\lambda$ for the Poisson random variable is given by the function argument `rate`.

The [`scipy.stats`](https://docs.scipy.org/doc/scipy/reference/stats.html) package is a great resource for working with and sampling from various probability distributions. We will use the `scipy.stats.poisson` class and its method `rvs` to produce Poisson-distributed random samples. In this tutorial, we have imported this package with the alias `stats`, so you should refer to it in your code as `stats.poisson`.

```python
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
    """ Simulate a linear integrate-and-fire neuron.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      rate (int): The mean rate of incoming spikes

    """
    # precompute Poisson samples for speed
    exc = stats.poisson(rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []

    for i in range(1, n_steps):

        dv = alpha * exc[i]

        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    return v, spike_times

# Set random seed (for reproducibility)
np.random.seed(12)

# Uncomment these lines after completing the lif_neuron function
with plt.xkcd():
    v, spike_times = lif_neuron()
    plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron(n_steps=1000, alpha=0.01, rate=10):
    """ Simulate a linear integrate-and-fire neuron.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      rate (int): The mean rate of incoming spikes
    """
    # precompute Poisson samples for speed
    exc = stats.poisson(rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []
    for i in range(1, n_steps):

        dv = alpha * exc[i]

        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    return v, spike_times

# Set random seed (for reproducibility)
np.random.seed(12)

v, spike_times = lif_neuron()
with plt.xkcd():
    plot_neuron_stats(v, spike_times)
```

## Interactive Demo: Linear-IF neuron
Like last time, you can now explore how various parametes of the LIF model influence the ISI distribution.

```python
# @title

# @markdown You don't need to worry about how the code works – but you do need to **run the cell** to enable the sliders.

def _lif_neuron(n_steps=1000, alpha=0.01, rate=10):
    exc = stats.poisson(rate).rvs(n_steps)
    v = np.zeros(n_steps)
    spike_times = []
    for i in range(1, n_steps):
        dv = alpha * exc[i]
        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0
    return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2, max=4),
                  alpha=widgets.FloatLogSlider(0.01, min=-2, max=-1),
                  rate=widgets.IntSlider(10, min=5, max=20))
def plot_lif_neuron(n_steps=1000, alpha=0.01, rate=10):
    v, spike_times = _lif_neuron(int(n_steps), alpha, rate)
    plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.0), FloatLogSlider(value=0.01,…

```python
# @title Video 2: Linear-IF models
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='QBD7kulhg4U', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=QBD7kulhg4U

# Section 2: Inhibitory signals

Our linear integrate-and-fire neuron from the previous section was indeed able to produce spikes. However, our ISI histogram doesn't look much like empirical ISI histograms seen in Tutorial 1, which had an exponential-like shape. What is our model neuron missing, given that it doesn't behave like a real neuron?

In the previous model we only considered excitatory behavior -- the only way the membrane potential could decrease was upon a spike event. We know, however, that there are other factors that can drive $V_m$ down. First is the natural tendency of the neuron to return to some steady state or resting potential. We can update our previous model as follows:

\begin{align}
  dV_m = -{\beta}V_m + {\alpha}I
\end{align}

where $V_m$ is the current membrane potential and $\beta$ is some leakage factor. This is a basic form of the popular Leaky Integrate-and-Fire model neuron (for a more detailed discussion of the LIF Neuron, see the Appendix).

We also know that in addition to excitatory presynaptic neurons, we can have inhibitory presynaptic neurons as well. We can model these inhibitory neurons with another Poisson random variable:

\begin{align}
I = I_{exc} - I_{inh} \\
I_{exc} \sim \mathrm{Poisson}(\lambda_{exc}) \\
I_{inh} \sim \mathrm{Poisson}(\lambda_{inh})
\end{align}

where $\lambda_{exc}$ and $\lambda_{inh}$ are the average spike rates (per timestep) of the excitatory and inhibitory presynaptic neurons, respectively.

### Exercise 2: Compute $dV_m$ with inhibitory signals

For your second exercise, you will again write the code to compute the change in voltage $dV_m$, though now of the LIF model neuron described above. Like last time, the rest of the code needed to handle the neuron dynamics are provided for you, so you just need to fill in a definition for `dv` below.

```python
def lif_neuron_inh(n_steps=1000,
                   alpha=0.5,
                   beta=0.1,
                   exc_rate=10,
                   inh_rate=10):
    """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
    and inhibitory inputs.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      beta (float): The membrane potential leakage factor
      exc_rate (int): The mean rate of the incoming excitatory spikes
      inh_rate (int): The mean rate of the incoming inhibitory spikes
    """

    # precompute Poisson samples for speed
    exc = stats.poisson(exc_rate).rvs(n_steps)
    inh = stats.poisson(inh_rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []

    for i in range(1, n_steps):

        dv = -beta * v[i - 1] + alpha * (exc[i] - inh[i])

        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    return v, spike_times

# Set random seed (for reproducibility)
np.random.seed(12)

# Uncomment these lines do make the plot once you've completed the function
with plt.xkcd():
    v, spike_times = lif_neuron_inh()
    plot_neuron_stats(v, spike_times)
```

```python
# to_remove solution
def lif_neuron_inh(n_steps=1000,
                   alpha=0.5,
                   beta=0.1,
                   exc_rate=10,
                   inh_rate=10):
    """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
    and inhibitory inputs.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      beta (float): The membrane potential leakage factor
      exc_rate (int): The mean rate of the incoming excitatory spikes
      inh_rate (int): The mean rate of the incoming inhibitory spikes
    """
    # precompute Poisson samples for speed
    exc = stats.poisson(exc_rate).rvs(n_steps)
    inh = stats.poisson(inh_rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []
    for i in range(1, n_steps):

        # TODO: check if typo is really in original and submit correction
        # error here! alpha - should be alpha * (see above for correction)
        dv = -beta * v[i - 1] + alpha - (exc[i] - inh[i])

        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    return v, spike_times

# Set random seed (for reproducibility)
np.random.seed(12)

v, spike_times = lif_neuron_inh()

with plt.xkcd():
    plot_neuron_stats(v, spike_times)
```

## Interactive Demo: LIF + inhibition neuron

```python
# @title
# @markdown **Run the cell** to enable the sliders.
def _lif_neuron_inh(n_steps=1000,
                    alpha=0.5,
                    beta=0.1,
                    exc_rate=10,
                    inh_rate=10):
    """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
    and inhibitory inputs.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      beta (float): The membrane potential leakage factor
      exc_rate (int): The mean rate of the incoming excitatory spikes
      inh_rate (int): The mean rate of the incoming inhibitory spikes
    """
    # precompute Poisson samples for speed
    exc = stats.poisson(exc_rate).rvs(n_steps)
    inh = stats.poisson(inh_rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []
    for i in range(1, n_steps):
        dv = -beta * v[i - 1] + alpha * (exc[i] - inh[i])
        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20))
def plot_lif_neuron(n_steps=1000,
                    alpha=0.5,
                    beta=0.1,
                    exc_rate=10,
                    inh_rate=10):
    v, spike_times = _lif_neuron_inh(int(n_steps), alpha, beta, exc_rate,
                                     inh_rate)
    plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.5), FloatLogSlider(value=0.5, …

```python
# @title Video 3: LIF + inhibition
from IPython.display import YouTubeVideo
video = YouTubeVideo(id='Aq7JrxRkn2w', width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

#Summary

In this tutorial we gained some intuition for the mechanisms that produce the observed behavior in our real neural data. First, we built a simple neuron model with excitatory input and saw that it's behavior, measured using the ISI distribution, did not match our real neurons. We then improved our model by adding leakiness and inhibitory input. The behavior of this balanced model was much closer to the real neural data.

# Bonus

### Why do neurons spike?

A neuron stores energy in an electric field across its cell membrane, by controlling the distribution of charges (ions) on either side of the membrane. This energy is rapidly discharged to generate a spike when the field potential (or membrane potential) crosses a threshold. The membrane potential may be driven toward or away from this threshold, depending on inputs from other neurons: excitatory or inhibitory, respectively. The membrane potential tends to revert to a resting potential, for example due to the leakage of ions across the membrane, so that reaching the spiking threshold depends not only on the amount of input ever received following the last spike, but also the timing of the inputs.

The storage of energy by maintaining a field potential across an insulating membrane can be modeled by a capacitor. The leakage of charge across the membrane can be modeled by a resistor. This is the basis for the leaky integrate-and-fire neuron model.

### The LIF Model Neuron

The full equation for the LIF neuron is

\begin{align}
C_{m}\frac{dV_m}{dt} = -(V_m - V_{rest})/R_{m} + I
\end{align}

where $C_m$ is the membrane capacitance, $R_M$ is the membrane resistance, $V_{rest}$ is the resting potential, and $I$ is some input current (from other neurons, an electrode, ...).

In our above examples we set many of these parameters to convenient values ($C_m = R_m = dt = 1$, $V_{rest} = 0$) to focus more on the general behavior of the model. However, these too can be manipulated to achieve different dynamics, or to ensure the dimensions of the problem are preserved between simulation units and experimental units (e.g. with $V_m$ given in millivolts, $R_m$ in megaohms, $t$ in milliseconds).

```python
def lif_neuron_full(n_steps=1000,
                    alpha=0.5,
                    beta=0.1,
                    exc_rate=10,
                    inh_rate=10,
                    capacitance=1,
                    membrane_res=1,
                    v_rest=0,
                    dt=1):
    """ Simulate a simplified leaky integrate-and-fire neuron with both excitatory
    and inhibitory inputs.

    Args:
      n_steps (int): The number of time steps to simulate the neuron's activity.
      alpha (float): The input scaling factor
      beta (float): The membrane potential leakage factor
      exc_rate (int): The mean rate of the incoming excitatory spikes
      inh_rate (int): The mean rate of the incoming inhibitory spikes
    """
    # precompute Poisson samples for speed
    exc = stats.poisson(exc_rate).rvs(n_steps)
    inh = stats.poisson(inh_rate).rvs(n_steps)

    v = np.zeros(n_steps)
    spike_times = []
    for i in range(1, n_steps):
        dv =  dt * (-beta * (v[i - 1] - v_rest) / membrane_res + alpha * (exc[i] - inh[i])) / capacitance
        v[i] = v[i - 1] + dv
        if v[i] > 1:
            spike_times.append(i)
            v[i] = 0

    # print(f'The time unit is {dt:.2f} ms')

    return v, spike_times

@widgets.interact(n_steps=widgets.FloatLogSlider(1000.0, min=2.5, max=4),
                  alpha=widgets.FloatLogSlider(0.5, min=-1, max=1),
                  beta=widgets.FloatLogSlider(0.1, min=-1, max=0),
                  exc_rate=widgets.IntSlider(12, min=10, max=20),
                  inh_rate=widgets.IntSlider(12, min=10, max=20),
                  capacitance=widgets.FloatSlider(1, min=0, max=10),
                  membrane_res=widgets.FloatSlider(1, min=0, max=10),
                  v_rest=widgets.FloatSlider(0, min=-1, max=1),
                  dt=widgets.FloatSlider(1, min=0.01, max=1))
def plot_lif_neuron_full(n_steps=1000,
                         alpha=0.5,
                         beta=0.1,
                         exc_rate=10,
                         inh_rate=10,
                         capacitance=1,
                         membrane_res=1,
                         v_rest=0,
                         dt=1):
    v, spike_times = lif_neuron_full(int(n_steps), alpha, beta, exc_rate,
                                     inh_rate)
    plot_neuron_stats(v, spike_times)
```

    interactive(children=(FloatLogSlider(value=1000.0, description='n_steps', min=2.5), FloatLogSlider(value=0.5, …


```python
from biasbalancer.balancer import BiasBalancer
from biasbalancer.get_compas_data import get_compas_data
```

# BiasBalancer Tutorial
*Authors:* Caroline Amalie Fuglsang-Damgaard, Elisabeth Zinck

The following notebook is a tutorial of the toolkit `BiasBalancer`, which is a toolkit for analysing potential unfairness in predictions from a binary classification model.

The BiasBalancer class is a 3-level toolkit for analysing and understanding how much unfairness is present in predictions from a given predictive algorithm. It analyses the unfairness by subgroups dictated by a sensitive attribute such as sex or race. The three levels of BiasBalancer each returns a visualization and the data behind. Each level nuances the analysis and aids the user in obtaining a deeper understanding of the predictions.

For further details about BiasBalancer, we refer to the [BiasBalancer documentation](https://elisabethzinck.github.io/Fairness-oriented-interpretability-of-predictive-algorithms/html/biasbalancer.html).

## Data
In this tutorial, we will use the canonical COMPAS dataset to demonstrate the use of BiasBalancer. The COMPAS dataset was analyzed in ProPublica's [article](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing) about machine bias in criminal sentencing.

The COMPAS dataset contains predictions from the COMPAS algorithm, which assigns a `decile_score` between 1 and 10 to a criminal. A score above 5 means that the criminal is considered to have medium(6-8) or high(9-10) risk of comitting another crime (also called recidivating) within two years. The prediction, `pred`, of the binary classification of the $\text{i}^{\text{th}}$ criminal is given by
$$
\text{pred}_i = \begin{cases}
    0 &\text{decile score}_i <= 5\\
    1 &\text{decile score}_i > 5
\end{cases}.
$$
The target variable, `two_year_recid`, is binary and denotes whether the individual actually recidivated within two years, 1 meaning that they did recidivate. The COMPAS dataset will be analyzed wrt. discrimination based on the sensitive attribute `race`.

A pre-processed version of the dataset can be loaded using the function `get_compas_data`, which was imported from the `biasbalancer` module in the beginning of this notebook.

```python
compas = get_compas_data(normalize_decile_scores = True)
```

We set `normalize_decile_scores` to True, which means the scores are divided with 10 such that they can be used as prediction scores. The dataset looks in the following way:

```python
compas.head(5)
```

      id
      sex
      age
      age_cat
      race
      decile_score
      score_text
      two_year_recid
      pred

      1
      3
      Male
      34
      25 - 45
      African-American
      0.3
      Low
      1
      False

      2
      4
      Male
      24
      Less than 25
      African-American
      0.4
      Low
      1
      False

      6
      8
      Male
      41
      25 - 45
      Caucasian
      0.6
      Medium
      1
      True

      8
      10
      Female
      39
      25 - 45
      Caucasian
      0.1
      Low
      0
      False

      10
      14
      Male
      27
      25 - 45
      Caucasian
      0.4
      Low
      0
      False

## Initialization of Class
The following section shows how the BiasBalancer class is initialized. To initialize the class one must specify:

* `data`: DataFrame containing data used for evaluation
* `y_name`: Name of target variable
* `y_hat_name`: Name of binary output variable
* `r_name`: Name of variable containing scores (scores must be within [0,1])
* `a_name`: Name of sensitive variable
* `w_fp`: False positive weight
* `model_name`: Name of the model or dataset used. Is used for plot titles. (Optional)

The false positive weight $w_{FP} \in [0,1]$ balances the severity of receiving a false positive vs. a false negative. Higher false positive weights means that a false positive is considered to be more severe than a false negative. Because it can have severe negative consequences to be classified as a possible recidivist, it is suitable to choose a high false positive weight when analyzing the COMPAS dataset, such as $w_{FP} = 0.9$. The class `BiasBalancer` was imported in the first cell of the notebook and is initialized below :

```python
compas_w_fp = 0.9
fair_compas = BiasBalancer(
    data = compas,
    y_name = "two_year_recid",
    y_hat_name = "pred",
    a_name = "race",
    r_name = "decile_score",
    w_fp = compas_w_fp,
    model_name='COMPAS Decile Scores')
```

## Level 1 - Single Unfairness Measure

The first level should be used to get a quick overview of whether there is unfairness present in the predictions and how much. For this, the maximum relative difference in percent between the sensitive groups' weighted misclassification rates (WMR) is used.

The *weighted misclassification rate* of sensitive group $a$ is defined as
$$
    \textit{WMR}_a = c(w_{FP})\cdot\frac{w_{FP} FP_a + (1-w_{FP})FN_a}{n_a},
$$
where $FP_a$ is the number of false positives in group $a$, $FN_a$ is the number of false negatives in group $a$, $n_a$ is the number of observations belonging to group $a$, and $w_{FP}$ is the false positive weight indicating the severity of a false positive. $c(w_{FP})$ is a normalization constant ensuring that $\textit{WMR} \in [0,1]$.

The *weighted misclassification quotient* (WMQ) for group $a$ is the percent increase in the weighted misclassification rate relative to the group with the minimum rate:
$$
    \textit{WMQ}_a = \frac{\textit{WMR}_a - \textit{WMR}_{min}}{\textit{WMR}_{min}+\varepsilon}\cdot100\%.
$$
where the minimum rate is defined as $\textit{WMR}_{min} = \min_{a \in A} WMR_a$ and $\varepsilon$ is a small number added to the denominator to ensure the expression is well-defined if $\textit{WMR}_{min}=0$. The overall unfairness present in the model is measured by the maximum weighted misclassification quotient across the sensitive groups.

The results from the first level of BiasBalancer is accessed using the `level_1()` method:

```python
level1_output_data = fair_compas.level_1()
```

```python
level1_output_data
```

      grp
      n
      WMR
      WMQ

      0
      African-American
      2682
      0.252796
      47.877821

      1
      Caucasian
      1829
      0.170950
      0.000000

From the illustration above it is seen that African-Americans see a 48% larger weighted misclassification rate (WMR) compared to the group with the smallest *WMR*, which suggests that the predictions by the COMPAS algorithm disfavor African-Americans. The illustration always shows the largest weighted misclassification quotient, and should be seen as a red flag highlighting the most disfavored group in terms of the *WMR*.

The table shows that Caucasians have the smallest *WMR* and therefore a weighted misclassification quotient of 0%. The table includes one row for each sensitive group.

The weighted classification quotient can be used to compare the degree of unfairness between different models.

## Level 2 - Overview of Unfairness

#### New:
Second level of BiasBalancer creates a visual overview of the unfairness in the predictions. The overview consists of three elements structured in a visualization and three data frames. It consists of absolute rates, relative rates and an unfairness barometer.

The absolute rates visualized are

* False Positive Rate $\left(FPR=\frac{FP}{N}\right)$
* False Negative Rate $\left(FNR=\frac{FN}{P}\right)$
* False Discovery Rate $\left(FDR=\frac{FP}{PP}\right)$
* False Omission Rate $\left(FDR=\frac{FN}{PN}\right)$

where *FP* are false positives, *FN* are false negatives, *N* are negatives, *P* are positves, *PN* are predicted negatives and *PP* are predicted positives. The relative rates are computed for each group.

The relative rates for group $a$ are computed by
$$
\begin{align}
RR_a(r) &= \frac{r_{a} - r_{min}}{r_{min}+\varepsilon}\cdot100\% \\
r_{min} &= \min_{a\in A} r_a,
\end{align}
$$

which is computed for rates $r \in \{\textit{FPR},~\textit{FNR},~\textit{FDR},~\textit{FOR},~\textit{WMR}\}$. Again, a small $\varepsilon$ is added to the denominator to ensure the relative rate is well-defined if $\textit{r}_{min}=0$.

The unfairness barometer indicates the level of unfairness present in the predictions according to different fairness criteria. The fairness criteria are summarized in the overview table below.

|Fairness Criterion |Enforces  |
|:----------------:|:-:|
|Independence  | $$\frac{PP_a}{N_a} = \frac{PP_b}{N_b}$$    |
|Sufficiency                      |$FOR_a = FOR_b$ and $FDR_a = FDR_b$|
|Predictive Parity $\bullet$|$FDR_a = FDR_b$                         |
|Separation                       |$FPR_a = FPR_b$ and $FNR_a = FNR_b$|
|FPR-balance $\star$        |$FPR_a = FPR_b$                         |
|Equal Opportunity $\star$  |$FNR_a = FNR_b$                          |

The quantity depicted in the unfairness barometer is called the *mean maximum relative rate (MMRR)*. Informally, this is the maximum relative rate across sensitive subgroups, if the criterion only depends on one rate. If the criterion depends on several rates it shows the mean of the different rate components instead.
Below the second level visualization is shown:

```python
rates, relative_rates, barometer = fair_compas.level_2()
```

The plot shows that African-Americans have almost twice as high false positive rates as Caucasians. This results in the unfairness barometer being the largest for FPR balance and separation since these fairness criteria are based on the false positive rate.

The false negative rate is much higher for Caucasians than for African-Americans, which results in large violations of the criteria Equal opportunity and separation.

Since $w_{FP} = 0.9$, a false positive prediction is considered to be more severe than a false negative. The WMR balance takes this trade-off into account, and hence deem African-Americans to be more disfavored than Caucasians.

Finally, the independence criterion is violated, which means that African-Americans have a higher rate of receiving the unfavorable prediction. This means that a larger proportion of African-Americans are predicted to have a high risk of recidivating.

The data shown in the plot is also returned:

```python
rates.head()
```

      rate
      grp
      rate_val
      rate_val_lwr
      rate_val_upr

      0
      FDR
      African-American
      0.434282
      0.409198
      0.459707

      1
      FDR
      Caucasian
      0.509025
      0.467478
      0.550448

      2
      FNR
      African-American
      0.285103
      0.259949
      0.311665

      3
      FNR
      Caucasian
      0.503650
      0.461909
      0.545340

      4
      FOR
      African-American
      0.276119
      0.251628
      0.302032

```python
relative_rates.head()
```

      rate
      grp
      rate_val
      relative_rate

      0
      FDR
      African-American
      0.434282
      0.000000

      1
      FDR
      Caucasian
      0.509025
      17.210811

      2
      FNR
      African-American
      0.285103
      0.000000

      3
      FNR
      Caucasian
      0.503650
      76.655488

      4
      FOR
      African-American
      0.276119
      27.555159

```python
barometer
```

      criterion
      relative_rate
      discriminated_grp

      1
      FPR balance
      92.323421
      [African-American]

      4
      Separation
      84.489454
      [Caucasian, African-American]

      2
      Independence
      81.690209
      [African-American]

      0
      Equal opportunity
      76.655488
      [Caucasian]

      6
      WMR balance
      47.877821
      [African-American]

      5
      Sufficiency
      22.382985
      [Caucasian, African-American]

      3
      Predictive parity
      17.210811
      [Caucasian]

## Level 3 - Analysis of Unfairness Sources

The third, and last, level contains methods that can be used for analysing the sources of the unfairness more in depth. The output from level 2 is useful for determining which analyses should be carried out in the third level. The available methods in level 3 are:
- `w_fp_influence`: Shows how different settings of `w_fp` influence the weighted misclassification quotients.
- `roc_curves`: ROC curves for each sensitive group. Suitable if the unfairness barometer shows high disparities in *separation*, *FPR balance* or *Equal opportunity*.
- `calibration`: Calibration curve for each sensitive group. Suitable if the unfairness barometer shows high disparities in *sufficiency* or *predictive parity*.
- `confusion_matrix`: Displays a confusion matrix for each sensitive group. Especially useful when group sizes are small.
- `independence_check`: Displays the fraction of individuals predicted to most unfavored outcome. Suitable if there are high disparities in *independence*.

In this example using COMPAS, independence and separation shows the biggest differences between groups, so the methods `roc_curves` and `independence_check` are used. The methods can be accessed using the method `level_3()` and specifying the argument `method`.

### ROC Curve Analysis
The ROC curves for the two groups are similar, but the cut-off of `decile_score = 0.5` lies very differently on the curves. This results in higher true positive rate (equivalently lower false negative rate) and a higher false positive rate for African-Americans.

```python
roc = fair_compas.level_3(method = 'roc_curves')
```

### Independence Check Analysis
The plot shows that the independence criterion was violated because African-Americans are predicted to recidivate at a much higher rate than Caucasians. The errors bars (showing 95% Wilson confidence intervals) show that this difference cannot be attributed to random fluctuations.

```python
independence = fair_compas.level_3(method = 'independence_check')
```

## Conclusion
The above analysis suggests that overall African-Americans could be discriminated by the predictions because they receive more unfavorable predictions than Caucasians. This is because African-Americans more often are predicted to recidivate, including when the prediction was correct, and more worringly also when the prediction was incorrect. The more in depth analyses showed that the threshold of `decile_score`=5 seemed to inflate the number of unfavorable predictions for African-Americans.

This notebook shows how the three levels in `BiasBalancer` can be used to gain a more nuanced analysis of the unfairness in the predictions. The results from level 1 suggested that further unfairness analysis was necessary, and the results from level 2 lead to the more in depth analyses `independence_check` and `roc_curves`. For completeness, the output from the remaining available analysis methods in level 3 are shown in the next section.

## The remaining plots in level 3

### False positive weight influence
The plots shows how the weighted misclassification quotient changes as the false positive weight, $w_{FP}$, changes. Increasing the false positive weight, indicating that false positives are more severe, increases the weighted misclassification quotient for African-Americans. If it were worse to be predicted to have a low risk of recidivating, then the weighted misclassification quotient suggests that Caucasians could be discriminated slightly by the algorithm.

```python
fair_compas.level_3(method = 'w_fp_influence', output_table = False)
```

### Calibration analysis
The plot shows how well the predicted probabilities align with true probabilities (calculated as the proportion of true outcomes in the bin). The two groups behave similarly in this plots, which is also why only small sufficiency values were seen in the unfairness barometer.

```python
cal = fair_compas.level_3(method = 'calibration', n_bins = 5)
```

### Confusion matrices
The plot shows the confusion matrix by sensitive group. The percentages denote the percentage of the individuals in the sensitive group who received the specified combination of prediction and outcome. In the marginal labels we see how there are more actual positives and predicted positives among African-Americans. This different marginal distribution of outcomes for African-Americans could be a factor in why the number of false positives is also large in the group.

```python
confusion = fair_compas.level_3(method = 'confusion_matrix', **{'cm_print_n': True})
```

```python

```


# `ADKit`
# Documentation

*****

## Introduction

Derivatives are ubiquitous in many fields such as engineering design optimization, fluid dynamics and machine learning.
There are in general three ways to calculate the derivatives: automatic differentiation, numeric differentiation, and
symbolic differentiation. Automatic Differentiation (AD) brings a family of techniques that can calculate the partial
derivatives of any function at any point efficiently and accurately. Unlike numeric differentiation, AD does not have
the problem of floating point precision errors, since it calculates the derivative of a simple function, and keeps track of
these derivatives, and there is no need of step sizes. Compared to symbolic differentiation, AD is not as memory intense,
and can be much faster in terms of the calculation. Therefore, AD is an important way to calculate derivatives in
practice.

There are two modes in Automatic Differentiation: forward mode and the reverse mode. In forward mode, the chain rule is applied to each basic operation, and both the variable's value and derivative are calculated along the way, leading to a complete derivative trace. In reverse mode, there is a forward pass, where the intermediate variables are computed and their values and partial derivatives with respect to the previous layer stored in the memory, and also a reverse pass (popularly known as backward propagation), where we propagate back the derivatives with the help of the chain rule.

The software that we design calculates the derivatives given the user’s input using the forward mode/reverse mode of automatic differentiation depending on the user's choice,
and provides the user with an easy way to solve their optimization problem using derivatives.

## Background

At the core of Automatic Differentiation is the principle that functions implemented as computer code can be broken down into elementary functions, ranging from arithmetic operations (e.g. addition, subtraction etc.) and other functions (e.g. power, exponential, sin etc.). Hence, any differentiable function can be interpreted as a composition of different functions.

For example, given a function, $f = sin^2(2x)$, it can be rewritten as:

$$ f = \phi_1(\phi_2(\phi_3(x))) $$

where $$ \phi_1(z) = z^2,   \phi_2(y) = sin(y) \text{ and } \phi_3(x) = 2x$$

In the forward mode, the chain rule can then be applied successively to each elementary component function to obtain the derivative of the function. Using the same example above, let $c$ be a real number:
$$ f'(c) =  \phi_3'(\phi_2(\phi_1(c))) \cdot \phi_2'(\phi_1(c)) \cdot \phi_1'(c)$$

Based on the example above, the derivative, $f'(c)$, can be evaluated based on the following function-derivative pairs at each stage of computing the function:

$$(\phi_1(c), \phi_1'(c))$$

$$(\phi_2(\phi_1(c)), (\phi_2'(\phi_1(c)) \cdot \phi_1'(c)))$$

$$(\phi_3(\phi_2(\phi_1(c))), \phi_3'(\phi_2(\phi_1(c)) \cdot \phi_2'(\phi_1(c)) \cdot \phi_1'(c))$$

Effectively, the forward mode computes the Jacobian-vector product, $Jp$. This decomposition can be represented via a computational graph structure of calculations, requiring initial values to be set for $x_1$, and $x'_1$:

$$x_1 \rightarrow^{\phi_3(x)} x_2 \rightarrow^{\phi_2(x)} x_3 \rightarrow^{\phi_1(x)} y $$

where $$ \phi_1(x) = x^2,   \phi_2(x) = sin(x) \text{ and } \phi_3(x) = 2x$$

At each stage of the function, the derivative of the function with respect to its argument is calculated. The exact values of the function and its derivative are used for the following function-derivative pair of values. An example of the computational trace for the equation $f = sin^2(2x)$ would look like this, for $x = \dfrac{\pi}{6}$.

| Trace    | Elementary Operation    | Derivative     | $\left(f\left(a\right),  f^{\prime}\left(a\right)\right)$                                          |
| :------: | :----------------------:               | :------------------------------: | :------------------------------: |
| $x_{1}$  | $\dfrac{\pi}{6}$                       | $1$                | $\left(\dfrac{\pi}{6}, 1\right)$ |
| $x_{2}$  | $2x_{1}$                               | $2\dot{x}_{1}$     | $\left(\dfrac{\pi}{3}, 2\right)$ |
| $x_{3}$  | $\sin(x_{2})$               | $\cos\left(x_{2}\right)\dot{x}_{2}$            | $\left(\dfrac{\sqrt{3}}{2}, 1\right)$ |
| $x_{4}$  | $x_{3}^{2}$                            | $2x_{3}\dot{x}_{3}$                   | $\left(\dfrac{3}{4}, \sqrt{3}\right)$ |

By evaluating the derivative at each step of the chain rule, we eventually obtain the value of the derivative $f'(x) = \sqrt{3}$ at $x = \dfrac{\pi}{6}$, as second entry of the final tuple in the table.

While the above illustrates the forward mode of AD (the focus of our package), AD also has a reverse mode. Without using chain rule, it first does a forward pass to store the partial derivatives, before undertaking a reverse pass, which starts from the final function to be differentiated, $y$. After fixing the derivative of the final function, it then computes the derivative of each component function with respect to its parent function recursively (using chain rule) until the derivative of the function with respect to the basic-level argument (e.g. $x_1$) can be calculated.

In terms of efficiency, the forward mode is more efficient when the number of functions to evaluate is much greater than the number of inputs, whereas the reverse mode, which computes the Jacobian-transpose-vector-product is more efficient when the number of inputs is much greater than the number of functions.

More details on the reverse mode is covered in the **Extension (Reverse Mode)** section further below in the documentation.

## How to Use

### How to install `ADKit`

ADKit can be installed through the Python Package Index using the following command in the command terminal:
pip install ADKit
Alternatively, the user may install ADKit by cloning the github repository (https://github.com/the-differentiators/cs207-FinalProject.git) or downloading as a zipped archive.

ADKit has only `numpy` (v. 1.14.3 or higher) as a pre-installation requirement. If `numpy` is not installed, this can be installed using the `requirements.txt` file included in the repository, after it is downloaded, using the following code:
pip install -r requirements.txt
### How to use `ADKit` (Forward Mode)

The following steps walk the user through a demo of how to import and use the `ADKit` package.

#### Importing `ADKit.AutoDiff` and requirements
The following code imports the forward mode variable class from ADKit.

For the purposes of this demo, we will import `numpy`, which is a requirement for the `ADKit` package, as well as the forward mode variable class from ADKit's AutoDiff module.

```python
import numpy as np
from ADKit.AutoDiff import Ad_Var
```

#### Using `ADKit` to compute derivative of a scalar function of one variable (forward mode)

Below, we have included a basic demo for a scalar function, given a single input. The function used in the demo is  $f = sin^2(2x)$, which was used for illustration in the *Background* section earlier. Our objective is to use the the `Ad_Var` class to compute the value of the derivative for this function automatically, unlike the manual computational trace drawn out earlier.

First, we create an instance of the `Ad_Var` object, with the value of $x = \dfrac{\pi}{6}$ assigned to the input variable, `val`.

```python
a = np.pi / 6

x = Ad_Var(a)
```

The user should note that the `ADKit` package assumes that for a single input, the object being initialised will have a derivative value of 1 (stored as a Class attribute `self._ders`).

Next, we create `f`, which represents the full function. The `Ad_Var` object from the previous code can be used with dunder functions and additional functions within `Ad_Var` class to construct the full function being evaluated.

```python
f = (Ad_Var.sin(2*x))**2
```

As the functions are applied to the original `Ad_Var` object `x`, the `_val` and `_ders` attributes of the object are being updated with new values. The object `f`, representing the full function, will have its `_val` and `_ders` attributes containing the actual function and derivative values respectively.

To note: the user also has the ability to manually set function and derivative values outside of instance initialization using the setter methods provided (`set_val` and `set_ders`). In this way, the user has the option to reuse the same objects after resetting the value and derivative(s).

The associated function value and derivative(s) of any `Ad_Var` instance may be retrieved through the `get_val` and `get_ders` functions as shown below:

```python
print(f.get_val(), f.get_ders())
```

    0.7499999999999999 1.7320508075688776

Also, the function value and derivative can be printed by directly printing the `Ad_Var` object associated with the function `f`.

```python
print(f)
```

    Value = 0.7499999999999999
    Derivative = 1.7320508075688776

#### Using `ADKit` to compute the gradient of a scalar multivariate function (forward mode)

If the user wants to calculate the value and the gradient vector of a scalar multivariate function, then each variable must be first instantiated as an `Ad_Var` object, with inputs `val`, the scalar value of that variable, and `ders`, a `numpy` array representing the seed vector which indicates a direction along which the directional derivative of a function will be calculated. An example is shown below:

```python
x = Ad_Var(1, np.array([1, 0, 0]))
y = Ad_Var(2, np.array([0, 1, 0]))
z = Ad_Var(3, np.array([0, 0, 1]))
```

Then, the user can define the function which consists of the instantiated `Ad_Var` variables. For example, below , we are calculating the value and the gradient of the function $f = sin^2(2x) + z^y$:

```python
f = (Ad_Var.sin(2*x))**2 + z**y
print(f)
```

    Value = 9.826821810431806
    Gradient = [-1.51360499  9.8875106   6.        ]

As we can see above, the gradient of the function `f` is a 3-dimensional vector, since `f` is a function of 3 variables. The first dimension of the gradient vector is the directional derivative of `f` along the seed vector $[1, 0, 0]$. Since, `x` was instantiated with this seed vector, the first dimension of the gradient vector corresponds to the partial derivative $\frac{\partial f}{\partial x}$ evaluated at $x=1, y=2, z=3$. Similarly, $y$ was instantiated with the seed vector $[0, 1, 0]$. In this way, the user has indicated that the second element of the gradient of `f` corresponds to $\frac{\partial f}{\partial y}$ evaluated at at $x=1, y=2, z=3$. Similarly, $z$ was instantiated with the seed vector $[0, 0, 1]$, hence the third dimension of the gradient vector corresponds to $\frac{\partial f}{\partial z}$ evaluated at $x=1, y=2, z=3$.

In summary, each variable should be instantiated with a seed vector with dimensions equal to the dimensions of the gradient vector of the target function. For each variable, the values of the seed vector should be 0 except for one value which should be the derivative of that variable such as 1. The index of the element in the seed vector which has nonzero value indicates the index of the gradient vector which stores the partial derivative value of the target function with respect to this specific variable.  For example, if a variable is initiated with a seed vector of $[1,0,0]$, then this should be interpreted as the first variable among the three variables, and its derivative is set to be 1.

#### Using `ADKit` to compute derivative of a vector-valued multivariate function (forward mode)

The user can also use `ADKit` to calculate the value and the jacobian matrix of a vector-valued function. Again the variables must be instantiated in the same way as discussed above. Then, a vector-valued function can be defined as a numpy array of functions composed of instantiated `Ad_Var` variables. An example is shown below for the vector valued function $f = \begin{bmatrix}
sin^2(2x) + z^y \\
e^x + z
\end{bmatrix}$ for $x = 1, y = 2, z = 3$:

```python
x = Ad_Var(1, np.array([1, 0, 0]))
y = Ad_Var(2, np.array([0, 1, 0]))
z = Ad_Var(3, np.array([0, 0, 1]))

f = np.array([(Ad_Var.sin(2*x))**2 + z**y, Ad_Var.exp(x) + z])
```

Then, the user can call `get_jacobian` to get the jacobian matrix of `f` evaluated at $x = 1, y = 2, z = 3$. The first argument of this method is the vector-valued function $f$ defined as a numpy array. The second argument is the dimension of the vector of the functions (in this example the vector-valued function has 2 dimensions). The third argument is the number of variables composing the vector-valued function (in this example vector-valued function is composed of 3 variables, $x,y$ and $z$).

```python
Ad_Var.get_jacobian(f, 2, 3)
```

    array([[-1.51360499,  9.8875106 ,  6.        ],
           [ 2.71828183,  0.        ,  1.        ]])

Also, the user can call `get_values` by passing `f`, to calculate the value of the vector-valued function for the given values of the variables.

```python
Ad_Var.get_values(f)
```

    array([9.82682181, 5.71828183])

Alternatively, the vector valued function can also be defined as a numpy array of other already instantiated functions, as shown below:

```python
g = (Ad_Var.sin(2*x))**2 + z**y
h = Ad_Var.exp(x) + z
f = np.array([g, h])
```

```python
Ad_Var.get_jacobian(f, 2, 3)
```

    array([[-1.51360499,  9.8875106 ,  6.        ],
           [ 2.71828183,  0.        ,  1.        ]])

```python
Ad_Var.get_values(f)
```

    array([9.82682181, 5.71828183])

#### Using `ADKit` to compute the derivatives of any type of function on a grid of points (forward mode)

In the above examples, the derivative/gradient/jacobian of a function is evaluated at a single point which is defined by the value with which each variable is instantiated. `ADKit`, however, can be used to evaluate the derivative/gradient/jacobian of a function on a grid of points defined by the user. The first step to do this is again to instantiate the variables with any value (please note that the default value of an `Ad_Var` variable is 1 so the value argument can be skipped).

```python
x = Ad_Var(ders = np.array([1, 0, 0]))
y = Ad_Var(ders = np.array([0, 1, 0]))
z = Ad_Var(ders = np.array([0, 0, 1]))
```

Then, the user needs to define the function as a string using the same standard syntax used in any of the examples above. For example, if function is $f = sin^2(2x) + z^y$:

```python
f_string = "(Ad_Var.sin(2*x))**2 + z**y"
```

Then, the user can call `grid_eval` to calculate the gradient and the value of the given function on a grid of points. The first argument passed is the function string. The second argument is a list of strings where each string represents one of the variables used in the function string. The third argument is the list of the already instantiated `Ad_Var` objects which are referenced in the function string. The last argument is a list of lists defining the grid of all possible points that the user wants to calculate the gradient and the value of the function for. For example, below the function and its gradient are evaluated for all possible combinations of $(x, y, z)$ where $x \in \{1, 2\}, y \in \{2, 3\}, z=4$. The function returns a dictionary where each key is one of the points of the grid and the value is a tuple. The first element of the tuple is the value of the function at this point and the second element of the tuple is the gradient of the function evaluated at this point.

```python
Ad_Var.grid_eval(f_string, ['x', 'y', 'z'], [x, y, z], [[1, 2], [2,3], [4]])
```

    {(1, 2, 4): (16.826821810431806,
      array([-1.51360499, 22.18070978,  8.        ])),
     (1, 3, 4): (64.82682181043181,
      array([-1.51360499, 88.72283911, 48.        ])),
     (2, 2, 4): (16.57275001690431,
      array([ 1.97871649, 22.18070978,  8.        ])),
     (2, 3, 4): (64.57275001690431,
      array([ 1.97871649, 88.72283911, 48.        ]))}

The function `grid_eval` can also be used to evaluate the jacobian for vector valued functions at different points. In this case, the string representation of the vector-valued function must be written as a list of functions referencing the already instantiated `Ad_Var` variables. Please note that in this case the string representation corresponds to a list of functions and not a numpy array of functions. For example, if the user wants to evaluate the jacobian of the vector-valued function $f = \begin{bmatrix}
sin^2(2x) + z^y \\
e^x + z
\end{bmatrix}$ at different points, the function string should be defined as follows:

```python
f_string = "[(Ad_Var.sin(2*x))**2 + z**y, Ad_Var.exp(x) + z]"
```

Then, by calling `grid_eval` on this function string, a dictionary is returned where each key is one of the points of the grid and the value is a tuple. The first element of the tuple is the value of the function at this point and the second element of the tuple is the jacobian of the vector-valued function evaluated at this point.

```python
Ad_Var.grid_eval(f_string, ['x', 'y', 'z'], [x, y, z], [[1, 2], [2,3], [4]])
```

    {(1, 2, 4): (array([16.82682181,  6.71828183]),
      array([[-1.51360499, 22.18070978,  8.        ],
             [ 2.71828183,  0.        ,  1.        ]])),
     (1, 3, 4): (array([64.82682181,  6.71828183]),
      array([[-1.51360499, 88.72283911, 48.        ],
             [ 2.71828183,  0.        ,  1.        ]])),
     (2, 2, 4): (array([16.57275002, 11.3890561 ]),
      array([[ 1.97871649, 22.18070978,  8.        ],
             [ 7.3890561 ,  0.        ,  1.        ]])),
     (2, 3, 4): (array([64.57275002, 11.3890561 ]),
      array([[ 1.97871649, 88.72283911, 48.        ],
             [ 7.3890561 ,  0.        ,  1.        ]]))}

### How to use `ADKit` (Reverse Mode)

As part of the extension to the minimum requirements, we have implemented the reverse mode of Automatic Differentiation. Using a separate class `rAd_Var`, the user is able to use ADKit to implement the reverse mode. The value of using the reverse mode over the forward mode is the increase in efficiency when the number of inputs is much greater than the number of functions.

The user should note that usage of the `rAd_Var` class differs from that of the `Ad_Var` class in the following ways:
* The initialization of a `rAd_Var` instance does not allow for the input of a derivative value. The implementation necessitates that the derivative of the instance is initialized as None. There is the option for the user to manually set the derivative using the `set_ders` function, if they so wish to.

* The derivatives of the `rAd_Var` object obtained using the `get_ders` method will be returned as a numpy array of partial derivative(s) of the input variables, unlike the forward mode, where the final derivative at the given value of the input variables is calculated using the chain rule.

* To obtain a Jacobian matrix for vector functions with multiple real scalar inputs, e.g. $f = \begin{bmatrix}
xy \\
y \\
ln(x^y)\\
\end{bmatrix}$, the user will need to define the functions first before passing them (as Python functions) as arguments for the `get_jacobian` method, together with the variable names and the given values for the inputs, as demonstrated below.

*Note: In defining the functions, the user should only include variables used in the function as arguments. Adding additional variables would lead to errors in the calculation.*

This difference in the implementation of the Jacobian matrix is because `rAd_Var` objects can only be defined in the context of one function. Hence, feeding all the functions into the `get_jacobian` method allows for the Jacobian to obtain the respective partial derivatives for each function separately, before combining them and returning them in a single Jacobian matrix.

#### Using `ADKit` to compute derivative of a scalar function of one variable (reverse mode)

As a demo, we will again find the value and the derivative of $f = sin^2(2x)$ at $x = \dfrac{\pi}{6}$, similar to the demo for `Ad_Var` above.

```python
from ADKit.AutoDiff import rAd_Var
```

```python
a = np.pi / 6

x = rAd_Var(a)

f = (rAd_Var.sin(2*x))**2

print(f)
```

    Value = 0.7499999999999999
    Partial Derivative(s) = [1.73205081]

#### Using `rAd_Var` to compute derivative, values of a vector-valued multivariate function (reverse mode)

We will use a similar function as shown above to obtain the Jacobian matrix of a vector-valued function. The functions in the vector must be defined as Python functions as discussed above. A basic example is shown below for the vector valued function $f = \begin{bmatrix}
xy \\
y \\
ln(x^y)\\
\end{bmatrix}$ at the points where $x = 1, y = 2$.

```python
def f1(x, y):
    return x * y

def f2(y):
    return y

def f3(x, y):
    return rAd_Var.log(x ** y)

rAd_Var.get_jacobian([f1, f2, f3], ["x","y"], [1, 2])
```

    array([[2., 1.],
           [0., 1.],
           [2., 0.]])

In the event that more variables are defined in the constructor but not used in the functions, their partial derivative is defined in the Jacobian matrix as 0, as shown below with the inclusion of the variable `z`. This gives the user flexibility in adjusting the vector functions to include/exclude a variable where needed. Note: In the arguments passed into the Python functions, the user should only include variables used in the function as arguments (e.g. for `f2`, including only `y` and not both `x` and `y`).

```python
def f1(x, y):
    return x * y

def f2(y):
    return y

def f3(x, y):
    return rAd_Var.log(x ** y)

rAd_Var.get_jacobian([f1, f2, f3], ["x","y","z"], [1, 2, 3])
```

    array([[2., 1., 0.],
           [0., 1., 0.],
           [2., 0., 0.]])

The `get_values` method is then used to return an array of computed values for the functions passed into the method.

```python
rAd_Var.get_values([f1, f2, f3], ["x","y","z"], [1, 2, 3])
```

    array([2., 2., 0.])

### Comparison of `rAd_Var` vs. `Ad_var` in obtaining the Jacobian matrix

While the implementation of the Jacobian matrix for the reverse mode is different, it will return a similar result as the forward mode (implemented in the `Ad_Var` class). We recommend that the user uses the `rAd_Var` class, in the event that the number of inputs is significantly greater than the number of functions, where it will perform more efficiently.

Below we compare the output from the `get_jacobian` method from both methods, based on the more complicated equation $f = \begin{bmatrix}
sin^2(2x) + z^y \\
e^x + z
\end{bmatrix}$ for $x = 1, y = 2, z = 3$, used in the earlier demo.

```python
x = Ad_Var(1, np.array([1, 0, 0]))
y = Ad_Var(2, np.array([0, 1, 0]))
z = Ad_Var(3, np.array([0, 0, 1]))

f = np.array([(Ad_Var.sin(2*x))**2 + z**y, Ad_Var.exp(x) + z])
Ad_Var.get_jacobian(f, 2, 3)
```

    array([[-1.51360499,  9.8875106 ,  6.        ],
           [ 2.71828183,  0.        ,  1.        ]])

```python
def f1(x, z, y):
    return rAd_Var.sin(2*x)**2 + z**y

def f2(x, z):
    return rAd_Var.exp(x) + z

rAd_Var.get_jacobian([f1, f2], ["x","y","z"], [1, 2, 3])
```

    array([[-1.51360499,  9.8875106 ,  6.        ],
           [ 2.71828183,  0.        ,  1.        ]])

## Software Organization

### Directory structure

Our intended directory structure is as follows:
```
cs207-FinalProject/
                   ADKit/
                        test/
                            test_autodiff.py
                            test_autodiff_reverse.py
                        AutoDiff.py
                   demo/
                        Forward_Mode_demo1.ipynb
                        Forward_Mode_demo2.ipynb
                        Reverse_Demo.ipynb
                   docs/
                        documentation.ipynb
                        Milestone 1.pdf
                        Milestone 2.ipynb
                   LICENSE
                   README.md
                   requirements.txt
                   setup.cfg
                   setup.py

```

### Modules

The primary module is a single `AutoDiff.py` file. Contained within it are two classes - the `Ad_Var` class and `rAd_Var` class.

Instances of these two classes, through interaction with other objects of the same class, are able to compute the value of a function as well as the value of that function's derivative with respect to any input variable. The `AutoDiff` module is powerful enough to handle both forward and reverse mode of Automatic Differentiation of any function comprised of the following elementary functions:

* Fundamental arithmetic operators (addition, subtraction, multiplication, and division)
* Logarithm (of any base)
* Negation
* Exponentiation ($e^x$ for an `Ad_Var` instance $x$)
* Power and root functions ($x^n$ for some real $n$)
* Trigonometric functions ($\sin(x)$, $\cos(x)$, $\tan(x)$)
* Inverse trigonometric functions ($\arcsin(x)$, $\arccos(x)$, $\arctan(x)$)

Each instance of the `Ad_Var` and `rAd_Var` class in the `AutoDiff` module represents the definition of a set of variables at a particular evaluation point. Through manipulations of these instances (either through fundamental arithmetic operations or built-in methods representing additional elementary functions described earlier), a user has the capability of representing any continuous differentiable function, be it scalar or vector. This was shown earlier via the code demo.

The other modules in the package are stored in the `test` folder and make up the test-suite for `AutoDiff.py`, with more details in the *Testing and Coverage* section below.

### Testing and Coverage
In the `test` folder, there are two separate Python modules `test_autodiff.py` and `test_autodiff_reverse.py`, which together consist of the test-suite for `AutoDiff.py`.

`test_autodiff.py` will contain tests for the methods in the `Ad_Var` class and `test_autodiff_reverse.py` for the `rAd_Var` class, to ensure that the elementary functions return the desired output. Tests are run using pytest. The tests are linked to Travis CI and CodeCov, which will manage continuous integration and code coverage respectively.

###  Installation and Distribution of package
The user must ensure that the package's requirements (`numpy`) are installed, or install them manually.

Following this, the user may install the package through PyPI (see above "How to Install ADKit"), or by cloning the GitHub repository directly.

## Implementation Details

### Class Implementation and Core Attributes

* There are two classes used for forward mode, as well as the extension (reverse mode): `Ad_Var` class and `rAd_Var` class respectively.

* The choice of keeping them as two separate classes is based on the fact that there is limited resuability of code between both implementations - the forward mode determines the derivatives of the variables using the chain rule whereas reverse mode traverses the computational graph in the forward pass and stores both parent-child relationships and the respective partial derivatives of the variables without doing the chain rule.

#### `Ad_Var` Class (Reverse Mode)

* The `Ad_Var`  class will represent the variables that are used in the forward mode of Automatic Differentiation process. In the case of a single input, the instance should be initialized with, `val`, a scalar value of that variable to be evaluated on when calculating both the function and derivative values (as shown in the demo above)

* In the case of multiple inputs, each input will be initialized as an `Ad_Var` object, with inputs `val`, a scalar value of that variable and `ders`, a `numpy` array representing the derivative of the input with regards to the other variables. An example is shown below:

```python
x1 = Ad_Var(1, np.array([1, 0, 0]))
x2 = Ad_Var(2, np.array([0, 1, 0]))
x3 = Ad_Var(3, np.array([0, 0, 1]))
```

* Dunder methods such as `__add__` and `__mul__`, and other elementary functions are implemented under this class. More information on this is covered below in the *Class Methods* section.

* As part of the class methods, we have included two static methods, `get_jacobian` and `get_values`, which respectively compute the Jacobian matrix and an array of function values for an array of `Ad_Var` objects. Also, a static method `grid_eval` is included which evaluates the function and its derivative/gradient/jacobian on a grid of points.

* In our implementation, we will also use the try-except method to catch unexpected input types: for example, if the user initializes the variable value of the `Ad_Var` instance with a value of type string, which is not a valid input type.

#### `Ad_Var`: Core Attributes

* `_val`: float value, indicating the function value of the Ad_Var object evaluated at the given point
* `_ders` (for single input): float value, indicating the derivative value of Ad_Var object evaluated at the given point
* `_ders` (for multiple inputs): 1-D array of floats, representing the value of the derivatives of the multiple inputs evaluated at the given point

`_val` and `_ders` attributes are made pseudoprivate to prevent users from manually setting function and derivative values outside of instance initialization

#### `rAd_Var` Class (Reverse Mode)

* The `rAd_Var`  class will represent the variables that are used in the reverse mode of Automatic Differentiation process. In the case of a single input, the instance should be initialized with, `val`, a scalar value of that variable to be evaluated on when calculating both the function and derivative values (as shown in the demo above)

* The initialization of a `rAd_Var` instance does not allow for the input of a derivative value. The implementation necessitates that the derivative of the instance is initialized as None. There is the option for the user to manually set the derivative using the `set_ders` function, if they so wish to.

* The derivatives of the `rAd_Var` object obtained using the `get_ders` method will be returned as a numpy array of partial derivative(s) of the input variables with respective to the final function

* As part of the class methods, we have included two static methods, `get_jacobian` and `get_values`, which respectively compute the Jacobian matrix and an array of function values for an array of `rAd_Var` objects.

* To obtain a Jacobian matrix for vector functions, the user will need to define the functions first before passing them (as Python functions) as arguments for the `get_jacobian` method, together with the variable names and the given values for the inputs, as shown in the demo above.

* This difference in the implementation of the Jacobian matrix is because `rAd_Var` objects can only be defined in the context of one function. Hence, feeding all the functions into the `get_jacobian` method allows for the Jacobian to obtain the respective partial derivatives for each function separately, before combining them and returning them in a single Jacobian matrix.

* In the case of multiple inputs, each input will be initialized as an `rAd_Var` object, with inputs `val`.

#### `rAd_Var`: Core Attributes

* `_val`: float value, indicating the function value of the rAd_Var object evaluated at the given point
* `_ders`: instantiated as None and updated via the `get_ders` method which sets the derivative of the final function (with respect to itself) as 1 before using the `get_gradient` helper method to recursively go through all children of the input variables and updating `_ders` with their partial derivative value
* `parents`: lists containing the parent node(s) of the rAd_Var object; initialized as an empty list and populated at every computation (via class methods)
* `children`: lists of tuples containing the children node(s) of the rAd_Var object; initialized as an empty and populated at every computation (via class methods)
* `visited`: boolean value used to track if a node has been traversed in the reverse pass

### Core Data Structures

In both classes, the following core data structures were used:

* **`numpy` arrays**: 1-D `numpy` arrays will be used to keep the gradient vectors as the entire trace is evaluated. `numpy`
provides vectorized operations which will make the overloading of elementary functions much more efficient for
multivariate functions. If a vector function is provided, 2-D `numpy` arrays will be used to hold the Jacobian matrix.

* **Dictionaries**: In the `Ad_Var` class, dictionaries are used to keep the results of `grid_eval` function call. Particularly, the keys of the dictionary are points on the grid defined by the user and the corresponding values are the function value and its derivative/gradient/jacobian at the corresponding point. In the `rAd_Var` class, dictionaries are used extensively in the `get_jacobian` method to keep and track the variable inputs and the partial derivatives for each variable separately.

Specifically, in the `rAd_Var` class, where a node is defined as the point at which the computation of a new `rAd_Var` instance is performed, lists are used to store the the parent-child relationships of each node.

* **Lists**: At the every computation of a new `rAd_Var` instance, the new object and the partial derivative of the new function with respect to the input variable are stored as a tuple. This simulates the forward pass of the reverse mode. The tuples in the list are then accessed in the reverse pass of the reverse mode using the `get_ders` method.

### External Dependencies
* `numpy` for implementation of the elementary functions (e.g. sin, sqrt, log and exp), by overloading `numpy` implementations for these functions
* `pytest` and `doctest` for testing
* TravisCI and CodeCov used to manage continuous integration and code coverage

### Elementary Functions and Class Methods

While both the forward mode and reverse mode support a same set of basic operations, comparison operators and elementary functions, the computation of the attributes of the object returned by the functions is vastly different.

At each computation, the reverse mode stores both parent-child relationships and the respective partial derivatives of the variables to each other without doing the chain rule via the attributes `parents` and `children`, covered above. Below are the functions supported by both classes - for more details on how the partial derivatives of the variables are calculated for `rAd_Var`, refer to the code.

#### Elementary Functions / Operators supported by both classes
* `__add__(self, other)` and `__radd__(self, other)`:
    * Other can be a float, int or an `AutoDiff` object
    * Returns an `Ad_Var` or `rAd_Var` object when calculating self + other or other + self

* `__sub__(self, other)` and `__rsub__(self, other)`:
    * Other can be a float, int or an `AutoDiff` object
    * Returns an `Ad_Var` or `rAd_Var` object when calculating self - other or other - self

* `__mul__(self, other)` and `__rmul__(self, other)`:
    * Other can be a float, int or an `AutoDiff` object
    * Returns an `Ad_Var` or `rAd_Var` object when calculating self * other or other * self

* `__truediv__(self, other)` and `__rtruediv__(self, other)`:
    * Other can be a float, int or an `AutoDiff` object
    * Returns an `Ad_Var` or `rAd_Var` object when calculating self / other or other / self

* `__pow__(self, other)` and `__rpow__(self, other)`:
    * `other` can be a float, int or `Ad_Var` object
    * `__rpow__` will require `other` to be a numeric type, otherwise, it will raise a TypeError
    * Returns an `Ad_Var` or `rAd_Var` object when calculating self ** other

* `__neg__(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object when calculating - self

* `__eq__(self, other)`:
    * Returns True if `self._val` == `other._val` and `self._ders` == `other._ders`, returns False otherwise

* `__ne__(self, other)`:
    * Returns True if `self._val` != `other._val` or `self._ders` != `other._ders`, returns False otherwise

* `__repr__(self)`:
    * Returns a string representing the value of `self._val` (Value) and the value of `self._ders` (Gradient)

*  `sqrt(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object by calling the __pow__ method using self**0.5

* `exp(self)`:
    * `Ad_Var`: Returns an `Ad_Var` object with `self._val = np.exp(self._val)` and `self._ders = np.exp(self._val) * self._ders`
    * `rAd_Var`: Returns an `rAd_Var` object with `self._val = np.exp(self._val)` and the `parent` and `children` attributes for both `self` and the new `rAd_Var` object updated accordingly

* `log(self, logbase=np.e)`:
    * Optional argument for `logbase` (can be a float or int). By default, `logbase` is set to the exponential.
    * `Ad_Var`: Returns an `Ad_Var` object with `self._val = np.log(self._val) / np.log(logbase)` and `self._ders = self._ders / (self._val * np.log(logbase))`.
    * `rAd_Var`: Returns an `rAd_Var` object with `self._val = np.log(self._val) / np.log(logbase)` and the `parent` and `children` attributes for both `self` and the new `rAd_Var` object updated accordingly

* `sin(self)` and `cos(self)` and `tan(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object with the respective class attributes updated accordingly based on the given trigonometric function

* `arcsin(self)` and `arccos(self)` and `arctan(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object with respective class attributes updated accordingly based on the given inverse trigonometric function

* `sinh(self)` and `cosh(self)` and `tanh(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object with respective class attributes updated accordingly based on the given hyperbolic function

* `logistic(self)`:
    * Returns an `Ad_Var` or `rAd_Var` object with respective class attributes updated accordingly based on the logistic (sigmoid) function

* `set_val(self, value)`:
    * Set the value of the private attribute `self._val` with `value`

* `set_ders(self, derivatives)`:
    * Set the value of the private attribute `self._ders` with `derivatives`

* `get_val(self)`:
    * Returns the value of the attribute `self._val`

#### Specific `Ad_Var` class methods
* `__init__(self, val, ders=1)`:
    * Sets `self._val` to the argument `val`
    * Sets `self._ders` to the argument `ders`

* `get_ders(self)`:
    * Returns the value of the attribute `self._ders`

* `get_jacobian(functions_array, functions_dim, vars_dim)`:
    * Static method that returns the Jacobian matrix for a given array of `Ad_Var` objects

* `get_values(functions_array)`:
    * Static method that returns an array of function values for a given array of `Ad_Var` objects

* `grid_eval(func_string, vars_strings, Ad_Vars_list, grid)`:
    * Static method that evaluates a function and its derivative/gradient/jacobian on a grid of points. A dictionary is returned where each key is a point of the grid and the value is a tuple with the first element being the value of the function at this point and second element is the derivative/gradient/jacobian evaluated at this point.

#### Specific `rAd_Var` class methods
* `__init__(self, val, ders=1)`:
    * Sets `self._val` to the argument `val`
    * Initializes `self._ders` as `None`
    * Initializes `self.parents` as an empty list
    * Initializes `self.children` as an empty list
    * Initializes `self.visited` as `False`

* `get_ders(self)`:
    * Sets the derivative of the final function (with respect to itself) as 1 before using the `get_gradient` helper method, which recursively traverses all children of the input variables and updating `_ders` with their partial derivative value
    * Returns `gradient_matrix`, a `numpy` array consisting of the partial derivatives of input variables used to compute the final function

* `get_jacobian(functions_array, var_list, var_values)`:
    * Static method that returns the Jacobian matrix for a vector of Python functions, with given variable names and values for the variables used as arguments in these functions
    * Instantiation of `rAd_Var` objects is done within this method based on the functions and variables passed in

* `get_values(functions_array, var_list, var_values)`:
    * Static method that returns an array of function values for a vector of Python functions, with given variable names and values for the variables used as arguments in these functions
    * Instantiation of `rAd_Var` objects is done within this method based on the functions and variables passed in

## Extension (Reverse Mode)

### Description

As part of an extension from the earlier milestone, we have implemented the reverse mode of Automatic Differentiation using the `rAd_Var` class, with details on the class (data structures, attributes and methods) elaborated and demonstrated above.

In terms of efficiency, the forward mode is more efficient when the number of functions to evaluate is much greater than the number of inputs, whereas the reverse mode, which computes the Jacobian-transpose-vector-product is more efficient when the number of inputs is much greater than the number of functions.

Having the `rAd_Var` class allows the user of this package the flexibility to choose between the two modes of automatic differentiation, depending on the vector functions and variables that they will carry out Automatic Differentiation on.

### Implementation Details

Here, instead of propagating the chain rule forwards from the innermost expressions outwards, we begin by constructing an evaluation tree for the entire expression, seed the outermost one with a derivative of one, and work from the outside inwards until we have computed the derivatives for everything down to the input variables. This method of automatic differentiation can be faster in many circumstances, but at the cost of storage.

This space overhead is due to the need to keep track of the evaluation tree; reverse mode would not be possible without being able to traverse the structure of an expression. Each stage in computation must store its children, the intermediate variables that constitute itself, and its parents, any intermediates that make use of it. To this end, we've implemented the `rAd_Var` class, which is largely identical to the forward-mode `Ad_Var` class with several key differences.

Most significantly, `rAd_Var` keep track of both parents and children, enabling backwards traversal of the evaluation. This means that an individual `rAd_Var` instance is larger than an `Ad_Var` instance, and can be many times more so depending on the structure of the expression being differentiated. In addition, computing the derivatives of an expression necessitates changing each `rAd_Var` object's `_ders` attribute. As a result, `rAd_Var` instances may only be used once; an input variable cannot be used in two different expressions, even if they have the same value.

### Additional information / background

Unlike the forward mode of Automatic Differentiation, the reverse mode of Automatic Differentiation consists of two stages - 1) forward pass, followed by the 2) reverse pass (also popularly known as backward propagation).

With a function $f$ which takes in $n$ inputs, $x_1, x_2, ..., x_n$ and produces a single output, the reverse mode will return derivatives of the function $\dfrac{\partial f}{\partial x_{i}}$ for all $i$.

The following references and builds on [CS207 2019 Lecture 12 materials](https://harvard-iacs.github.io/2019-CS207/lectures/lecture12/notebook/) (Sondak):

**Forward pass**

In the forward pass, the evaluation of the function is carried out at each elementary function, $f_i$, of the entire function, $f$. What this means is that for $i = n + 1, n + 2, ..., N$, where $\pi(i)$ denotes the set of "parents" of $x_i$.

$$x_i = f_i(x_\pi(i))$$

E.g. for $x_{3} = x_{1}x_{2}$, where the elementary function, $f_3$, is a multiplication of two input variables, thus forming a node in the computational graph. Here, $\pi(3) = (x_1, x_2)$. This is important in the forward pass, which stores the partial derivatives of every node with respect to its "parents", as denoted by $\pi(i)$. In the example above, where $x_{3} = x_{1}x_{2}$, we store $\dfrac{\partial x_{3}}{\partial x_{1}}$ and $\dfrac{\partial x_{3}}{\partial x_{2}}$.

This can be best illustrated using an example where $$ f (x, y) = 2xy - \exp(xy)$$ at the point $x = 1, y = 2$

Here, to generate the forward trace, we calculate the partial derivatives of a node with regards to its children:

| Node   | Current Value      | Numerical Value           | $\partial_{1}$       | $\partial_{1}$ Value  | $\partial_{2}$ | $\partial_{2}$ Value |
| :---: | :----------------------------: | :-----------: | :----------------------------: | :-----------------:  | :-----------------: | :-----------------: |
| $x_{1}$ | $x$                      | $1$         | $1$                        | $1$     | $0$     | $0$   |
| $x_{2}$ | $y$                      | $2$         | $0$                        | $0$     | $1$     | $1$   |
| $x_{3}$ | $x_{1}x_{2}$             | $2$         | $x_{2}$                    | $2$     | $x_{1}$ | $1$   |
| $x_{4}$ | $2x_{3}$                 | $4$         | $2$                        | $2$     | $-$     | $-$   |
| $x_{5}$ | $\exp\left(x_{3}\right)$ | $e^{2}$     | $\exp\left(x_{3}\right)$   | $e^{2}$ | $-$     | $-$   |
| $x_{6}$ | $-x_5$                   | $-e^{2}$    | $-1$                       | $-1$    | $-$     | $-$   |
| $x_{7}$ | $x_{4} + x_{6}$          | $4 - e^{2}$ | $1$                        | $1$     | $1$     | $1$   |

**Reverse pass**

Following the forward pass, the reverse pass starts from the final function to be differentiated, $f_N$, setting $\overline{x}_{N} = \dfrac{\partial f}{\partial x_{N}} = 1$ (since $f = x_{N}$).

Then, using the chain rule, it traverses the computational graph to obtain values for the partial derivative of every variable in the computational graph, $x_i$:

\begin{align}
  \overline{x}_{i} = \dfrac{\partial f}{\partial x_{i}} = \sum_{\text{j a child of i}}{\dfrac{\partial f}{\partial x_{j}}\dfrac{\partial x_{j}}{\partial x_{i}}}.
\end{align}

This is done recursively until the partial derivatives of the function with respect to the $n$ inputs, $x_1, x_2, ..., x_n$ are computed.

Using the same example above, we recursively go through every variable($x_i$) in the computational trace shown above. The computation of the gradient of each variable accesses the partial derivative with respect to its children that has been previously computed and stored during the forward pass.

$$\overline{x}_{7} = \dfrac{\partial f}{\partial x_{7}} = 1$$

$$\overline{x}_{6} = \dfrac{\partial f}{\partial x_{7}}\dfrac{\partial x_{6}}{\partial x_{7}} = 1 \cdot 1 = 1$$

$$\overline{x}_{5} = \dfrac{\partial f}{\partial x_{6}}\dfrac{\partial x_{6}}{\partial x_{5}} = 1 \cdot (-1) = -1$$

$$\overline{x}_{4} = \dfrac{\partial f}{\partial x_{7}}\dfrac{\partial x_{7}}{\partial x_{4}} = 1 \cdot 1 = 1$$

$$\overline{x}_{3} = \dfrac{\partial f}{\partial x_{5}}\dfrac{\partial x_{5}}{\partial x_{3}} + \dfrac{\partial f}{\partial x_{4}}\dfrac{\partial x_{4}}{\partial x_{3}}= (-1 \cdot e^{2}) + (1\cdot 2) = 2 - e^{2}$$

$$\overline{x}_{2} = \dfrac{\partial f}{\partial x_{3}}\dfrac{\partial x_{3}}{\partial x_{2}} = (2 - e^{2}) \cdot 1 = 2 - e^{2}$$

$$\overline{x}_{1} = \dfrac{\partial f}{\partial x_{3}}\dfrac{\partial x_{3}}{\partial x_{1}} = (2 - e^{2}) \cdot 2 = 4 - 2e^{2}$$

This gives us the gradient of:

\begin{align}
  \nabla f &=
  \begin{bmatrix}
    4 - 2e^{2} \\
    2 - e^{2}
  \end{bmatrix}
\end{align}

which is identical to what we would have calculated using symbolic differentiation for the function $f (x, y) = 2xy - \exp(xy)$:

$$\nabla f = \begin{bmatrix} 2y - \exp\left(xy\right)y \\ 2x - \exp\left(xy\right)x \end{bmatrix}$$

Reverse mode of Automatic Differentiation is supported by the `rAd_Var` class in the `ADKit` package. Further details on using the class is covered in the **How to use ADKit (Reverse Mode)** section of the Documentation

#### References

* [A Hitchhiker’s Guide to Automatic Differentiation](https://link.springer.com/article/10.1007/s11075-015-0067-6)
* [A simple explanation of reverse-mode automatic differentiation](https://justindomke.wordpress.com/2009/03/24/a-simple-explanation-of-reverse-mode-automatic-differentiation/)
* Harvard CS207 2019 course materials


```javascript
%%javascript
MathJax.Hub.Config({
    TeX: { equationNumbers: { autoNumber: "AMS" } }
});
```

    <IPython.core.display.Javascript object>

## Exercise 2.15 MLE minimizes KL divergence to the empirical distribution
Let $p_{emp}(x)$ be the empirical distribution, and let $q(x|\theta)$ be some model. Show that $\mathrm{argmin}_q\mathbb{KL}(p_{emp}||q)$
is obtained by $q(x) = q(x; \theta)$, where $\theta$ is the MLE. Hint: use non-negativity of the KL divergence.

### Notes
We will prove that the MLE minimises the KL divergence between the empirical distribution and some model. What does this mean? Suppose we get access to some data $\mathcal{D}$ generated by some unknown distribution and we are asked to estimate this distribution based on the data.

In this situation, there are two approaches we could take: the **nonparametric** and the **parametric**. On the former approach, we could make an estimation based purely on the data we have at hand. This estimation could be the empirical distribution $p_{emp}(x)$, defined as

$$
p_{emp}(x) = \sum_{i=1}^N w_i\delta_{xi}(x)
$$

where we require $0\le w_i < 1$ and $\sum_{i=1}^N w_i = 1$.

On the latter approach, we could make some reasonable assumptions about the data pattern and create a parametric model $q(x|\theta)$ for it. The model is based on the prior we have about the process behind the data and it dictates the general shape of the distribution (e.g. Gaussian). One of the most popular ways of estimating the parameters is the Maximum Likelihood Estimator (MLE). The MLE is the parameter value $\theta^*$ which solves the following optimization problem:

\begin{equation}
\theta^* = \mathrm{argmax}_\theta\prod_{x\in\mathcal{D}}p(x|\theta)
\end{equation}

The RHS of (1) is called the likelihood of the dataset and it is a function of $\theta$. Note that most of the times it's most convenient to optimize the log-likelihood, which gives the same result as optimizing the likelihood.

$$
\theta^* = \mathrm{argmax}_{\theta}\prod_{x\in\mathcal{D}}p(x|\theta) = \mathrm{argmax}_\theta\sum_{x\in\mathcal{D}}\log(p(x|\theta))
$$

Given two such different scenarios, this exercise proposes we find a bridge between them. We will find the parametric model $q(x|\theta)$ that is the most similar to the empirical distribution $p_{emp}(x)$. As you can imagine, one great way to do this is to express the similarity by the KL-divergence and minimize this value.

### Solution

\begin{equation}
\mathbb{KL}(p_{emp}||q) = \sum p_{emp}\log(\frac{p_{emp}}{q}) = \sum p_{emp}\log(p_{emp}) - \sum p_{emp}\log q
\end{equation}

Since $p_{emp}$ is fixed given the dataset $\mathcal{D}$, the first term of (1) is constant and we only need to focus on the second term.

\begin{aligned}
\mathrm{argmin}_\theta\mathbb{KL}(p_{emp}||q(\theta)) & = -\mathrm{argmin}_\theta\sum p_{emp}\log q \\
& = \mathrm{argmax}_\theta\sum \log(q^{p_{emp}}) \\
& = \mathrm{argmax}_\theta\log\left(\prod_{x\in\mathcal{D}} q^{p_{emp}}\right)
\end{aligned}

For the final step, note that the empirical distribution is defined as $p_{emp}(x_i) = N_i/N$, where $N_i$ is the number of occurrences of $x_i$ and $N$ is the size of our dataset.

Thus:

\begin{aligned}
\mathrm{argmin}_\theta\mathbb{KL}(p_{emp}||q(\theta)) & = \mathrm{argmax}_\theta \log(\prod_{x\in\mathcal{D}}q^{p_{emp}}) \\
& = \mathrm{argmax}_\theta\frac{1}{N}\log\left(\prod_{x\in\mathcal{D}}q(x_i|\theta)^{N_i}\right)
\end{aligned}

This is the same expression as the log-likelihood, with the exception of the $1/N$ term. Therefore the solution for this problem is the MLE parameter $\theta^*$.

### Conclusion
In this exercise, we saw that the MLE of a parametric model is the most similar distribution to the empirical distribution, amongst all the models considered $(\theta\in\Theta)$. This makes a lot of sense - as the name tells, the maximum likelihood estimator is the parameter that maximizes the likelihood that our given dataset could occur. So its optimization is strongly based on the data (not taking into account a probability prior, like Bayesian models. The only prior is the shape of the distribution, like a Gaussian, which is given by the model). Therefore, as both the MLE and $p_{emp}$ have such a strong relationship with the empirical data, it makes sense that they have a strong relationship between themselves.