# Mg diffusion in plagioclase model
## Finite Difference Method Approach to the Diffusion Equation in 1D
*Jordan Lubbers*

*College of Earth, Ocean, and Atmospheric Sciences*

*Oregon State University*

Utilizes the forward difference scheme to the finite difference method for Fick's $2^{nd}$ Law

While the diffusion coefficient of Mg in plagioclase are compositionally dependent (i.e. the rate at which it diffuses within the plagioclase crystal lattice decreases with increasing An content), the partitioning of trace elements into plagioclase is also An dependent and follows an Arrhenius relationship (e.g. Bindeman et al., 1998; Dohmen and Blundy, 2014; Nielsen et al., 2017). Costa and others (2003; Eq. 7) derive a solution to the diffusion equation that incorporates this:
\begin{equation}\label{costa_eq}
\frac{\delta C}{\delta t} = \left(\frac{\delta D}{\delta x}\frac{\delta C}{\delta x}+D\frac{\delta^2C}{\delta x^2}\right) - \frac{A}{RT}\left(D\frac{\delta C}{\delta x}\frac{\delta X_{An}}{\delta x}+C\frac{\delta D}{\delta x}\frac{\delta X_{An}}{\delta x}+DC\frac{\delta^2X_{An}}{\delta x^2} \right)
\end{equation}
\
We can see here that the first part of the equation deals with the diffusion of matter, and the second part of the equation pertains to the Arrhenius partitioning relationship. Discretizing this equation we get the following:
\begin{equation}\label{plag_FD}
C_{i,j+1} = C_{i,j} + \Delta t\Bigg\{\left[\left(\frac{D_{i+1,j}-D_{i,j}}{\Delta x}\right)\left(\frac{C_{i+1,j}-C_{i,j}}{\Delta x}\right)+D_{i,j}\left(\frac{C_{i+1,j}-2C_{i,j}+C_{i-1,j}}{\Delta x^2}\right)\right] - \frac{A}{RT}\left[ D_{i,j}\left(\frac{C_{i+1,j}-C_{i,j}}{\Delta x}\cdot\frac{An_{i+1,j}-An_{i,j}}{\Delta x}\right)+C_{i,j}\left(\frac{D_{i+1,j}-D_{i,j}}{\Delta x}\cdot\frac{An_{i+1,j}-An_{i,j}}{\Delta x}\right)+D_{i,j}C_{i,j}\left(\frac{An_{i+1,j}-2An_{i,j}+An_{i-1,j}}{\Delta x^2}\right)\right]\Bigg\}
\end{equation}

- $C_{i,j+1}$ is the concentration of a trace element at a point in space $i$ and time $j$
- $D_{i,j+1}$ is the diffusion coefficient of a trace element at a point in space $i$ and time $j$
- $An_{i,j}$ is the anorthite of the plag a point in space $i$ and time $j$
- $\Delta t$ is the time spacing between $j$ steps
- $\Delta x$ is the distance spacing between $i$ steps.
- $A$ is the constant from Nielsen et al., (2017) for $Mg$ partitioning in plagioclase
- $R$ and $T$ are the universal gas constant $8.314 \frac{J}{mol\cdot K}$ and temperature in Kelvin, respectively

## Import dependencies

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from tqdm.notebook import tqdm
import scipy.interpolate as interp
import seaborn as sns
from scipy import stats
```

## Import data
This is how I format my data: each column is a variable (e.g., analyte) and each row is an individual observation (e.g., concentration of said analyte at a given spot in a transect). For the code below to work there should be one column that denotes which grain a transect came from. You can call it whatever you want but ultimately this helps delineate between different transects.

```python
filepath = "path/to/mydata.xlsx"
sheet_name = "plagioclase trace profiles"
mydata = pd.read_excel(filepath, sheet_name=sheet_name,)
mydata.head()
```

      index
      grain
      sample
      spot
      location
      pumice type
      An
      Li
      Mg
      Si
      ...
      Si_se
      Ca_se
      Ti_se
      Fe_se
      Sr_se
      Ba_se
      La_se
      Ce_se
      Eu_se
      Pb_se

      0
      272
      MQ2
      MQ
      MQ2-LT-readyspot0
      rim
      banded
      0.428469
      14.182287
      59.847871
      264729.067093
      ...
      4612.456707
      948.287042
      19.816264
      40.177845
      10.361437
      5.428762
      0.311744
      0.285490
      0.073922
      0.953166

      1
      273
      MQ2
      MQ
      MQ2-LT-readyspot1
      rim
      banded
      0.402673
      12.186312
      54.506839
      267569.625241
      ...
      4683.859089
      889.788913
      18.311831
      40.254044
      9.886144
      5.375988
      0.305544
      0.298061
      0.085942
      0.989139

      2
      274
      MQ2
      MQ
      MQ2-LT-readyspot2
      rim
      banded
      0.373036
      13.505803
      56.124201
      271426.082654
      ...
      4424.407389
      822.603271
      17.202320
      38.423881
      9.401607
      5.101791
      0.291372
      0.266983
      0.078428
      1.086601

      3
      275
      MQ2
      MQ
      MQ2-LT-readyspot3
      rim
      banded
      0.389422
      14.483002
      56.472904
      269206.011004
      ...
      4189.466220
      859.746452
      17.768755
      38.537895
      9.393451
      5.203899
      0.287544
      0.296493
      0.070561
      1.113580

      4
      276
      MQ2
      MQ
      MQ2-LT-readyspot4
      rim
      banded
      0.395573
      13.779702
      55.497519
      268430.006433
      ...
      4274.631948
      873.690346
      17.768394
      38.664305
      9.597920
      5.127188
      0.279626
      0.303730
      0.074639
      1.084794

5 rows × 31 columns

# Set Index
This allows us to better access our data by which transect it belongs to
1. Set the plag dataframe index by grain. This then allows us to index that dataframe by transect
2. Get list of grains so you can index in the next step

```python
plag_bygrain = mydata.set_index("grain")
plag_grains = list(plag_bygrain.index.unique())
print(plag_grains)
```

    ['MQ2', 'MQ3', 'MQ6core', 'MQ6rim', 'MQ7', 'MQ8', 'MQ9', 'MQ11', 'MQ12', 'MQ13', 'MQ14rim', 'MQ15', 'MQ17core', 'MQ17rim', 'MQ18core', 'MQ18rim', 'MQ19', 'MQ20core', 'MQ20rim', 'MQ21rim', 'MQ22', 'MQ23', 'MQ24core', 'MQ24int', 'MQ24rim', 'MQ25', 'MQ26', 'MQ27int', 'MQ27rim', 'MQ29', 'MQ31', 'MQ32', 'MQ33', 'MQ34', 'LT1', 'LT2', 'LT5', 'LT8', 'LT9', 'LT10', 'LT12', 'LT17', 'LT18', 'LT19', 'LT22']

## Set up initial Model range
Here you can choose which element you want to model (choose Mg because that's what the diffusion model is set up to do), it's uncertainty, and which grain (transect) you want to model it for.
- The ```start``` and ```stop``` variables are for the index of the profile you want to start and stop the model at. This is reflected by the green markers in the plot

```python
element = "Mg"
unc = "Mg_se"
grain = "MQ2"
# the domain you wish to model diffusion over
start = 0
stop = 0
# spatial resolution in microns
res = 5

# measured trace element information
te_all = plag_bygrain.loc[grain, element].to_numpy()
te_unc_all = plag_bygrain.loc[grain, unc].to_numpy()
An_all = plag_bygrain.loc[grain, "An"].to_numpy()

# distance data
dist_all = np.arange(0, len(plag_bygrain.loc[grain, :])) * res

# this is if you want to clip any erroneous points from the ends of your data
clip_start = 10

clip_stop = len(te_all) - 1

dist_all = dist_all[clip_start:clip_stop]
te_all = te_all[clip_start:clip_stop]
te_unc_all = te_unc_all[clip_start:clip_stop]
An_all = An_all[clip_start:clip_stop]

te = te_all[start : len(te_all) - stop]
te_unc = te_unc_all[start : len(te_all) - stop]
dist = dist_all[start : len(te_all) - stop]

# plot observed data
fig, ax = plt.subplots(figsize=(6, 4))
# observed profile and subset
ax.errorbar(
    dist_all,
    te_all,
    yerr=te_unc_all,
    fmt="o",
    mfc="w",
    mec="k",
    ecolor="k",
    capsize=3,
    label="measured prof",
)
ax.errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt="o",
    mfc="g",
    mec="k",
    ecolor="k",
    capsize=3,
    label="diffusion prof",
)
ax.legend(loc="best")
ax.set_ylabel("{} (ppm)".format(element), fontsize=20)
ax.set_xlabel("Distance ($\mu$m)", fontsize=20)
ax.minorticks_on()
```

## Generating random points based off your measured values and the uncertainty at each point in the profile
Because your analysis value is technically the mean value associated with a normally distributed uncertainty, it has a chance to be a range of values with the probability of a given value governed by a normal distribution centered around your analysis value. To better reflect this, we can generate a 'random profile' where new points are picked at each analysis point according to the reported uncertainty at each point.

In order to generate this random point based off our measured value and analytical uncertainty we can use the [```numpy.random.normal```](https://numpy.org/doc/stable/reference/random/generated/numpy.random.normal.html)function that generates a normally distributed random number if given the mean and standard deviation. This is a viable function to use becaue if our measurements are subjected to small random errors and negligible systematic errors, the distribution of measurements taken will be normal centered around the mean.

We will use this later in our Monte Carlo simulation to estimate the uncertainty in our diffusion model

```python
# random profile from your y and yerr
def random_profile(x, y, yerr):
    """random_profile generates a random observed profile for diffusion modeling that is based of the measured value
    and its corresponding uncertainty at each point.
            Inputs:
            x = x data
            y = y data
            yerr = 1 sigma uncertainty on y data measurements. If your data has 2 sigma uncertainties, just
            create a new variable that divides it by 2 so you have 1 standard deviation uncertainties

            Returns:
            yrand = random y data points who's liklihood is normally distributed around the mean"""
    import numpy as np

    # np.random.normal(mean,std deviation)
    yrand = np.random.normal(loc=y, scale=yerr)
    return yrand

yrand = random_profile(dist, te, te_unc)
fig, ax = plt.subplots(figsize=(6, 4))
# observed profile
ax.errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt="o",
    mfc="g",
    mec="k",
    ecolor="k",
    capsize=3,
    label="measured prof",
)
# random profile
ax.plot(
    dist, yrand, ":o", c="lawngreen", mec="k", label="random profile",
)
ax.legend(loc="best")
ax.set_ylabel("Sr (ppm)")
ax.set_xlabel("Distance ($\mu$m)")
ax.minorticks_on()
```

## Constants, equilibrium trace element profile, diffusion coefficient profile
Here we define
- temperature of our model
- universal gas constant (R)
- activation energy for the diffusion coefficient of Mg in plag
- thermodynamic parameters from Nielsen et al 2017 for Mg partitioning in plagioclase
- Mg glass composition. THIS IS SPECIFIC TO YOUR SYSTEM

We then use the partitioning parameters, temperature, and glass composition to calculate what the profile would look like if it were fully equilibrated with a magma that has a composition identical to the glass values we used. Since this usually comes from pumice, we're justified in this being the most accurate representation of the magma chemistry at the time of eruption. We also set up an An array from 0 to 1 and use taht combined with the partitioning parameters to calculate a range of compositions that, based on An, our Mg data should fall into if it is equilibrated. The slope of this curve is very important, as it shows that if our profile is equilibrated, Mg should be increasing with decreasing An. If it is not equilibrated then it will have the opposite relationship

```python
# Constants
T_K = 750 + 273.15  # temp in K
# Ea = 266e3  # J/mol
R = 8.314  # gas constant in J/mol*K

# anorthite
An = An_all[start : len(te_all) - stop]

# Thermodynamic constants for Mg partitioning in plagiocase [DO NOT CHANGE]
# [average,max,min] 1sigma
a_Mg = np.array([-10000, -10000 + 3300, -10000 - 3300])  # from Nielsen et al., 2017
b_Mg = np.array([-35000, -35000 + 2100, -35000 - 2100])

An_partition = np.arange(
    0.1, 1.1, 0.1
)  # range of anorthite compositions to calculate equilibrium curves

# glass compositions [ppm] averages
# this is specific to your magma system!
Mg_gl = 450

# Mg partition coefficients [DO NOT CHANGE]
D_Mg_ave = np.exp(((a_Mg[0] * An_partition) + b_Mg[0]) / (R * T_K))
D_Mg_max = np.exp(((a_Mg[1] * An_partition) + b_Mg[1]) / (R * T_K))
D_Mg_min = np.exp(((a_Mg[2] * An_partition) + b_Mg[2]) / (R * T_K))

# Calculated Mg equilibrium  [DO NOT CHANGE]
Eq_Mg_ave = Mg_gl * D_Mg_ave
Eq_Mg_max = Mg_gl * D_Mg_max
Eq_Mg_min = Mg_gl * D_Mg_min

# Mg equilibrium profile based off the relationships found in Nielsen et al., (2017)
# partition coefficient at each point based on An concentration
Kd = np.exp((a_Mg[0] * An_all + b_Mg[0]) / (R * T_K))

# if we assume that the rim most point is in equilibrium with the liquid:
# concentration of liquid
Cl = te[-1] / Kd[-1]
Eq_prof_all = Cl * Kd
Eq_prof = Eq_prof_all[start : len(te_all) - stop]
```

## Plot up your equilibrium, anorthite, and trace element profiles
This is just plotting the stuff defined and calculated above

```python
fig, ax = plt.subplots(1, 3, figsize=(18, 6))
fig.patch.set_facecolor("white")
# entire trace element profile
ax[0].errorbar(
    dist_all,
    te_all,
    yerr=te_unc_all,
    fmt="o",
    mfc="w",
    mec="k",
    c="gray",
    ecolor="k",
    capsize=3,
    label="Measured profile",
)
# part of the profile you are modeling
ax[0].errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt=":o",
    mfc="g",
    mec="k",
    c="g",
    ecolor="k",
    capsize=3,
    label="section to model",
)
# entire equilibrium profile
ax[0].plot(dist_all, Eq_prof_all, ":b", mec="k", label="Equilibrium Profile")

ax[0].set_xlabel("Distance ($\mu m$)", fontsize=16)
ax[0].set_ylabel("Concentration (ppm)", fontsize=16)
ax[0].legend(loc="best")
ax[0].minorticks_on()
ax[0].grid()

# entire trace element profile
ax[1].errorbar(
    dist_all, te_all, yerr=te_unc_all, fmt="o", mfc="w", mec="k", ecolor="k", capsize=3,
)
# part of the profile you are modeling
ax[1].errorbar(
    dist, te, yerr=te_unc, fmt=":o", mfc="g", mec="k", c="g", ecolor="k", capsize=3
)
ax[1].set_xlabel("Distance ($\mu m$)", fontsize=16)
ax[1].minorticks_on()
# ax[1].set_ylim(top = 60)

# secondary axis for An profile
ax3 = ax[1].twinx()
color3 = "orange"
ax3.plot(dist_all, An_all, "-", c=color3, label="An profile")
ax3.tick_params(axis="y", colors=color3)
ax3.set_ylabel("An", fontsize=16, color=color3)
ax3.yaxis.label.set_color(color3)
ax3.legend(loc="best")

# An vs Mg for your entire profile with error bars
ax[2].errorbar(
    An_all,
    te_all,
    yerr=te_unc_all,
    fmt="o",
    mec="k",
    ecolor="k",
    color="gray",
    capsize=3,
    label="Measured values",
    zorder=1,
)
# plotting An vs Mg but colored for distance so that you can
# have errorbars and this info on the same plot. This is needed
# because plt.errorbar is based off plt.plot not plt.scatter
s = ax[2].scatter(
    An_all, te_all, marker="o", c=dist_all, cmap="viridis", ec="none", zorder=2
)
cbar = fig.colorbar(s, ax=ax[2],)
cbar.set_label("Distance from rim ($\mu m$)", fontsize=16)

# fill between the equilibrium range bounds
ax[2].fill_between(
    An_partition,
    Eq_Mg_min,
    Eq_Mg_max,
    fc="gray",
    zorder=0,
    alpha=0.2,
    label="Equilibrium range \n(Bindeman et al., 1998)",
)
# plot the equilibrium range lines
ax[2].plot(An_partition, Eq_Mg_min, "k--")
ax[2].plot(An_partition, Eq_Mg_max, "k--")
ax[2].plot(An_partition, Eq_Mg_ave, "k-", lw=2)

ax[2].legend(loc="best")
ax[2].set_xlabel("$X_{An}$", fontsize=20)
ax[2].set_xlim(np.min(mydata["An"]) * 0.9, np.max(mydata["An"]) * 1.2)
# ax[2].set_ylim(np.min(Eq_Mg_min),np.max(Eq_Mg_max))

fig.tight_layout(pad=2)
```

## Setting up Diffusion model
Here is where we start to establish boundary conditions for our diffusion model. But first we must tweak a couple things. Namely, we are going to interpolate our profiles so that things are spaced closer together. This allows us a couple advantages:
1. Numerical stability that is defined as $\frac{D\Delta t}{\Delta x^2}$
2. Smoothing out our An profile a little bit

Per the advice of Sumit Chakraborty,
> Anorthite gradients are often very sharp, and particularly if one analytical point falls off the trend, it has a huge (negative) impact on the calculations - usually, that error multiplies with each iteration and leads to unusual numbers. A trick is the following - "diffuse"  and smoothen the Anortite gradient a little bit artificially

First we create some interpolating functions for our distance, An, trace element, and random trace element profiles. We then apply those interpolating functions to an x-grid that has ```multiplier```x as many points as our observed distance spacing.

We then create an array that will be the length of total time we want the diffusion model to run for spaced by the $\Delta t$ we want. With Mg I usually have a $\Delta t$ spaced by 'days'. This is actually the amount of seconds in a day. Since our diffusion coefficient is in units of $\frac{um^2}{s}$, our model time needs to also be in seconds. We can convert this to a meaningful time later. The total duration of the diffusion model is specified by:
```python
iterations = int(3.65e3)
sinyear = 60*60*24*365.25
days = sinyear/365.25

t = np.arange(0,iterations*days+1,days)
```
and states that we are going to run the model for ```3.65e3``` years, and spaced by the amount of seconds in one day.

Next we need to set up our initial boundary condition. We can do this by creating some containers:
```python
#containers for each iteration
#unknown at current iteration
u = np.zeros(nx)
#u at previous iteration
u_n = np.zeros(nx)
```
```u_n``` in the beginning will be our boundary condition, so we then fill that with values that we think the profile may start at. Because An diffuses so slowly in plagioclase relative to Mg, this can be a good reference for the width and height of your initial profile. This code is set up to do up two double sided step functions. If you believe you have that in your profile just un-comment the part of the code that would define the second double sided step function. To define our initial profile:
```python
#these numbers are the actual distance in microns
#we convert them to their index in the array later
stepstart = 60 - np.min(dist)
stepstop = 95 - np.min(dist)
```
These values are where you think your step should start on either side. If you only have a one sided step function, then just make the step stop value larger than your maximum distance value.
```python
#boundary conditions in ppm for left, middle, and right
Cl = 55
Cm = 73
Cr = 50
```
These are the Mg values for the boundary conditions
```python
step_begin = stepstart
step_begin_idx = int(step_begin/dx)
step_stop = stepstop
step_stop_idx = int(step_stop/dx)
```
This part finds the index that step starts and stops at in your profile. This is needed to later fill in the ```u_n``` container with boundary condition values.

```python
u_n[:step_begin_idx] = Cl
u_n[step_begin_idx:step_stop_idx] = Cm
u_n[step_stop_idx:] = Cr
```
This now uses the indices you just found for where your step is to say that "everything left of the step fill with the left boundary condition, everything between the step fill with the middle boundary condition, and everything right of the step fill with the right boundary condition". We now have created our initial condition for diffusion to start at! There is also the option to interpolate between two points defined as left and right boundary conditions to make an initial boundary condition that is not a "step function". For that we just uncomment the ```Cm``` line to be:

```python

# first step function
u_n[:step_begin_idx] = Cl
u_n[step_begin_idx:step_stop_idx] = np.interp(
    x[step_begin_idx:step_stop_idx],
    np.array((x[step_begin_idx], x[step_stop_idx])),
    np.array((Cl, Cr)),
)
# u_n[step_begin_idx:step_stop_idx] = Cm
u_n[step_stop_idx:] = Cr
```

We then can repeat this for as many steps as we want by just copy and pasting the steps that define our ```Cl, Cm, Cr```. We just have to name them something different (e.g., ```Cl2, Cm2, Cr2```.

Because ```u_n``` will get re-written at each step of the model we make a copy of it to save as our initial step function:
```python
step_func = u_n.copy()
```
We then diffuse the An profile slightly using an arbitrary ```r``` value of .2. For each iteration of the diffusion equation we save the value in the container ```an_curves```. This will allow us to access the values when the loop is done. We choose the third diffusion step so that the corners are rounded slightly but not too much so as to still keep the same shape.

Once we have the smoothed out An profile we can calculate the diffusion coefficient for each step in the interpolated grid. We defined a function to do this above called ```diffusivity```, so we just use that here.

For more information about how to code up the diffusion equation in Python, see [Numerical Modeling of the Diffusion Equation Walkthrough](https://drive.google.com/file/d/1Tig0Ex6ZiVMGUX5Xusm2lfVL8LBtROBb/view?usp=sharing)

```python
# interpolated profiles to 'smooth out' An profile steepness that may cause errors in the
# diffusion...per the advice of Sumit Chakraborty

# this multiplies the amount of of x points you have effectively 'filling in' your x grid
multiplier = 2
# setting up your finite difference with a specified number of x points
# here we are saying to add 5 times as many interpolated points as observed points

# number of points in your xgrid
nx = len(dist) * multiplier
# creating your xgrid
x = np.linspace(np.min(dist), np.max(dist), nx)
# x spacing
dx = x[1] - x[0]

# interpolated An profile
f_An = interp.interp1d(dist, An)
An_interp = f_An(x)

# interpolated te profile
f_te = interp.interp1d(dist, te)
te_interp = f_te(x)

# interpolated te profile
f_teunc = interp.interp1d(dist, te_unc)
teunc_interp = f_teunc(x)

f_rand = interp.interp1d(dist, yrand)
rand_interp = f_rand(x)

# since our Di is in um^2/s we need to factor that into our time grid
# by spacing out the points accordingly.
iterations = int(10 * 18250)
sinyear = 60 * 60 * 24 * 365.25
tenthsofyear = sinyear / 10
days = sinyear / 365.25
sinday = 60 * 60 * 24

# creating a time grid that is spaced by years
t = np.arange(0, iterations * tenthsofyear + 1, tenthsofyear)
dt = t[1] - t[0]
nt = len(t)

# containers for each iteration
# unknown at current iteration
u = np.zeros(nx)
# u at previous iteration
u_n = np.zeros(nx)

# initial condition as a step function
# these numbers are the actual distance in microns
# we convert them to their index in the array later
stepstart = 110 - np.min(dist)
stepstop = 160 - np.min(dist)

# boundary conditions in ppm for left, middle, and right
Cl = 55
Cm = 71
Cr = 56

step_begin = stepstart
step_begin_idx = int(step_begin / dx)
step_stop = stepstop
step_stop_idx = int(step_stop / dx)

# # IF THERE ARE TWO DOUBLE SIDED STEP FUNCTIONS IN ONE PROFILE
# stepstart2 = 160 - np.min(dist)
# stepstop2 = 190 - np.min(dist)
# Cl2 = Cr
# Cm2 = 70
# Cr2 = 62

# step2_begin = stepstart2
# step2_begin_idx = int(step2_begin/dx)
# step2_stop = stepstop2
# step2_stop_idx = int(step2_stop/dx)

# first step function
u_n[:step_begin_idx] = Cl
# u_n[step_begin_idx:step_stop_idx] = np.interp(
#     x[step_begin_idx:step_stop_idx],
#     np.array((x[step_begin_idx], x[step_stop_idx])),
#     np.array((Cl, Cr)),
# )
u_n[step_begin_idx:step_stop_idx] = Cm
u_n[step_stop_idx:] = Cr

# if there are two step functions use this bit instead of the Cr
# above
# u_n[step_stop_idx:step2_begin_idx] = Cr
# # u_n[step2_begin_idx:step2_stop_idx] = np.interp(
# #     x[step2_begin_idx:step2_stop_idx],
# #     np.array((x[step2_begin_idx], x[step2_stop_idx])),
# #     np.array((Cl2, Cr2)),
# # )
# u_n[step2_begin_idx:step2_stop_idx] = Cm2
# u_n[step2_stop_idx:] = Cr2

step_func = u_n.copy()

# creating a container to put all of your curve iterations
curves = np.zeros((nt, nx))

# diffuse the An profile a bit to get rid of the sharp corners
# Advice from Sumit Chakraborty himself...
# left and right boundary conditions
an_t = 100

An_l = An_interp[0]
An_r = An_interp[-1]

# arbitrary r value for diffusion. Just need to round the corners
# a small bit
r_an = 0.2

# unknown at current iteration
an_u = np.zeros(nx)

# make initial profile defined above our starting point for
# diffusion
an_u_n = An_interp.copy()

# creating a container to put all of your curve iterations
an_curves = np.zeros((an_t, nx))

# iterating over the length of nt(each iteration is a time step)
for n in range(0, int(an_t)):

    # diffusion equation with constant D. Runs over length of nt for each
    # r value
    an_u[1 : nx - 1] = an_u_n[1 : nx - 1] + r_an * (
        an_u_n[0 : nx - 2] - 2 * an_u_n[1 : nx - 1] + an_u_n[2:nx]
    )

    # fixing your boundary conditions with above values
    an_u[0] = An_l
    an_u[-1] = An_r
    # saving your iteration to your curve container
    an_curves[n, :] = an_u
    # switch your variables before the next iteration
    # makes your current u vals the u_n vals in the next loop
    an_u_n[:] = an_u
An_smooth = an_curves[2]

# Van Orman et al 2013 diffusion coefficient
# diffusivity in (um^2/s)
D_interp = np.exp(-6.06 - 7.96 * An_interp - 287e3 / (R * T_K)) * 1e12

r = (D_interp * dt) / dx ** 2

fig, ax = plt.subplots(1, 3, figsize=(18, 4))
ax[0].plot(x, r, "k-")
ax[0].set_ylabel("r", fontsize=20)
ax[0].set_xlabel("Distance $\mu m$", fontsize=20)
ax[0].set_title("Checking the stability of our diffusion model")

ax[1].plot(x, te_interp, "k-", label="Interpolated prof")
ax[1].errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt="ro",
    mec="k",
    ecolor="k",
    capsize=3,
    label="measured prof",
)
ax[1].plot(x, step_func, "k--", label="initial profile")
# ax[1].plot(x,mg_init,
#            'b--',
#            label = 'mg_init'
# )

ax[1].set_ylabel("{} (ppm)".format(element), fontsize=20)
ax[1].set_xlabel("Distance $\mu m$", fontsize=20)
ax[1].set_title("Interpolated vs. original Sr profile")
ax[1].legend(loc="best")

ax[2].plot(x, An_interp, "k-", label="Interpolated prof")
ax[2].plot(
    dist, An, "ro", mec="k", label="measured prof",
)
ax[2].plot(
    x, An_smooth, "go", mec="k", ms=5, label="smoothed An",
)
ax[2].set_ylabel("An ", fontsize=20)
ax[2].set_xlabel("Distance $\mu m$", fontsize=20)
ax[2].set_title("Interpolated vs. original An profile")
ax[2].legend(loc="best")

t20 = (0.24 * (((dist[1] - dist[0]) * 1e-6) ** 2) / (D_interp / 1e12)) / sinyear

ax[0].text(
    0.1,
    0.1,
    "t$_{{20}}$ = {} yrs".format(np.round(t20.max(), 2)),
    transform=ax[0].transAxes,
    fontsize=14,
)
```

## Diffusion equation part
This is the coded up discretization of the equation at the top and follows the logic in in this [walkthrough](https://drive.google.com/file/d/1Tig0Ex6ZiVMGUX5Xusm2lfVL8LBtROBb/view?usp=sharing).

```python
# iterating over the length of Nt(each iteration is a time step)
for n in tqdm(range(0, int(nt))):
    # this is long...
    u[1 : nx - 1] = u_n[1 : nx - 1] + dt * (
        ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
        * ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
        + D_interp[1 : nx - 1]
        * ((u_n[2:nx] - 2 * u_n[1 : nx - 1] + u_n[0 : nx - 2]) / dx ** 2)
        - (a_Mg[0] / (R * T_K))
        * (
            D_interp[1 : nx - 1]
            * (
                ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
                * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
            )
            + u_n[1 : nx - 1]
            * (
                ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
                * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
            )
            + D_interp[1 : nx - 1]
            * u_n[1 : nx - 1]
            * (
                (An_smooth[2:nx] - 2 * An_smooth[1 : nx - 1] + An_smooth[0 : nx - 2])
                / dx ** 2
            )
        )
    )
    # letting the most 'core-ward' boundary condition diffuse according to the Costa 2003 equation
    u[0] = u_n[0] + dt * (
        ((D_interp[1] - D_interp[0]) / dx) * ((u_n[1] - u_n[0]) / dx)
        + D_interp[0] * ((u_n[1] - 2 * u_n[0] + u_n[1]) / dx ** 2)
        - (a_Mg[0] / (R * T_K))
        * (
            D_interp[0]
            * (((u_n[1] - u_n[0]) / dx) * ((An_smooth[1] - An_smooth[0]) / dx))
            + u_n[0]
            * (
                ((D_interp[1] - D_interp[0]) / dx)
                * ((An_smooth[1] - An_smooth[0]) / dx)
            )
            + D_interp[0]
            * u_n[0]
            * ((An_smooth[1] - 2 * An_smooth[0] + An_smooth[1]) / dx ** 2)
        )
    )

    # fix the most 'rim-ward' concentration (infinite reservoir assumption) based on observed data
    u[-1] = te_interp[-1]

    # #     infinite reservoir based on boundary condition fixed based on boundary conditions
    #     u[-1] = step_func[-1]

    # not infinite reservoir assumption. Let's everything diffuse and does not keep it fixed
    # potentially useful for grains that are not at the rim
    #     u[-1] = u_n[-1] +  dt*(((D_interp[-2] - D_interp[-1])/dx)*((u_n[-2]-u_n[-1])/dx) + D_interp[-1]*((u_n[-2]-2*u_n[-1]+u_n[-2])/dx**2)-(a_Mg[0]/(R*T_K))*(D_interp[-1]*(((u_n[-2]-u_n[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + u_n[-1]*(((D_interp[-2]-D_interp[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + D_interp[-1]*u_n[-1]*((An_smooth[-2]-2*An_smooth[-1]+An_smooth[-2])/dx**2)))

    # saving your iteration to your curve container
    curves[n, :] = u
    # switch your variables before the next iteration
    # makes your current u vals the u_n vals in the next loop
    u_n[:] = u

# Chi- squared minimization for best fit
chi2 = abs(np.sum((te_interp[None, :] - curves) ** 2 / (te_interp[None, :]), axis=1))

# find the minimum value
chi2_min = np.min(chi2)

# find where in the array it is (e.g., it's position)
fit_idx = np.argwhere(chi2 == chi2_min)

# Get that array index
fit_idx = fit_idx[0].item()

# add one because python starts counting at 0
bf_time = fit_idx + 1
```

      0%|          | 0/182501 [00:00<?, ?it/s]

## Plot it all up!
This is just plotting up all of your results:
1. Observed data with initial condition and best fit curve
2. Best fits vs. time
3. Observed data and An on same plot to show justification for intial boundary conditions
4. Mg vs. An data plotted up against equilibrium range based on assumed liquid composition and temperature

```python
# these are just comparison time steps. I have them set for years but you can do it for whatever makes sense
# to your problem

# This converts your number of iterations to years
# eg if you are spacing by "sinyear", makeyears = 1
# if you are spacing by "tenthsofyear", makeyears = 10
# if you are spacing by "days", makeyears = 365

makeyears = 10
compare = np.array([50 * makeyears, 500 * makeyears,])

# color list for plotting
colors = ["darkblue", "darkviolet", "orange", "gray", "g"]

# make iterations to years we have to divide by 365 since we iterate in days

fig, ax = plt.subplots(
    2,
    2,
    figsize=(14, 10),
    gridspec_kw={"width_ratios": [2, 1], "height_ratios": [1, 1]},
)
# modeled section of profile
ax[0, 0].errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt="ro",
    mec="k",
    ecolor="darkslategray",
    lw=0.5,
    capsize=0,
    label="measured prof",
)

# best fit curve
ax[0, 0].plot(
    x,
    curves[bf_time],
    "-",
    c="c",
    mec="k",
    label="best fit : t = {} yrs".format(np.round(bf_time / makeyears, 2)),
    lw=1.5,
)

# comparison curves. Omit this if you don't want them
for i in range(0, len(compare)):

    ax[0, 0].plot(
        x,
        curves[compare[i]],
        c=colors[i],
        label="t = {} yrs".format(compare[i] / makeyears),
        lw=0.5,
    )
# equilibrium profile
ax[0, 0].plot(dist, Eq_prof, "b--", label="Equilibrium")
# boundary conditions
ax[0, 0].plot(x, step_func, "-k", lw=2, label="initial profile")

# legend
# ax[0,0].legend(loc = 'best',
#                title='CGI plagioclase: {}\_{}\-{}'.format(grain.split('_')[0],grain.split('_')[1].split('-')[0],grain.split('_')[1].split('-')[1]),
#                prop={'size': 10},ncol = 2
# )
ax[0, 0].legend(
    loc="best", title="Toba plagioclase: {}".format(grain), prop={"size": 10}, ncol=2
)

ax[0, 0].set_ylabel("{} (ppm)".format(element), fontsize=20)
# ax[0,0].set_xlim(left = 0, right = np.max(x))
# ax[0,0].set_ylim(bottom = .95*Eq_prof.min(),top = 1.1*step_func.max())

# chi-squared plot
# convert to days
tdays = t / dt
# chi-squared curve
# ax[0,1].plot(np.array(count_list)/makeyears,chi2_list,'-k',lw =2, )

ax[0, 1].plot(
    tdays / makeyears, chi2, "-k", lw=2,
)

# vertical line at best fit value
ax[0, 1].axvline(
    bf_time / makeyears,
    color="g",
    label="best fit : t = {} yrs".format(np.round(bf_time / makeyears, 2)),
)
ax[0, 1].set_xlabel("time (yrs)", fontsize=16)
ax[0, 1].set_ylabel("$\sum{\chi^2} $", fontsize=16)
ax[0, 1].set_xscale("log")
ax[0, 1].legend(loc="best", prop={"size": 14})
ax[0, 1].set_yscale("log")

# observed data and An profile subplot
# entire profile but alpha'd to show where the model
# is
ax[1, 0].errorbar(
    dist_all,
    te_all,
    yerr=te_unc_all,
    fmt="o",
    mfc="r",
    mec="k",
    ecolor="darkslategray",
    capsize=0,
    alpha=0.3,
    label="measured profile",
)
# modeled profile in same color scheme but no alpha
ax[1, 0].errorbar(
    dist,
    te,
    yerr=te_unc,
    fmt="ro",
    mec="k",
    ecolor="darkslategray",
    capsize=0,
    lw=0.5,
    label="model profile",
)
# connect the dots on your modeled profile so its easier to see the shape of
# the profile and compare to An profile
ax[1, 0].plot(dist, te, "k-")
ax[1, 0].set_xlabel("distance ($\mu m$)", fontsize=20)
ax[1, 0].set_ylabel("{} (ppm)".format(element), fontsize=20)
ax[1, 0].legend(loc="best", fontsize=14)

# An profile on same x axis but different y
ax3 = ax[1, 0].twinx()
# entire profile
ax3.plot(dist_all, An_all, marker="o", c="g", mec="k", alpha=0.3)
# modeled portion of profile
ax3.plot(dist, An, marker="o", c="g", mec="k")
ax3.set_ylabel("An", fontsize=20, color="g")
ax3.tick_params(axis="y", colors="g")

# comparison with equilibrium concentration range from
# Nielsen et al. 2017
# observed data errorbars but no fill
ax[1, 1].errorbar(
    An_all,
    te_all,
    yerr=te_unc_all,
    fmt="o",
    mec="k",
    ecolor="k",
    color="k",
    capsize=0,
    label="Measured values",
    zorder=1,
)
# observed data scatter colored for distance in profile
# goes on top of errorbar above
s = ax[1, 1].scatter(An_all, te_all, marker="o", c=dist_all, ec="none", zorder=2)
# colorbar
cbar = fig.colorbar(s, ax=ax[1, 1],)
cbar.set_label("Distance ($\mu m$)", fontsize=16)
# shade in the equilibrium area from Nielsen et al 2017
ax[1, 1].fill_between(
    An_partition,
    Eq_Mg_min,
    Eq_Mg_max,
    fc="gray",
    zorder=0,
    alpha=0.2,
    label="Equilibrium range \n(Nielsen et al. (2017)",
)
# lines for the shaded region boundaries and mean
ax[1, 1].plot(An_partition, Eq_Mg_min, "k--")
ax[1, 1].plot(An_partition, Eq_Mg_max, "k--")
ax[1, 1].plot(An_partition, Eq_Mg_ave, "k-", lw=2)
# for i in range(1,len(compare)):

#     ax[1,1].scatter(An_interp[::5],curves[compare[i]][::5],
#                     marker ='s',
#                     c = colors[i],
#                     ec = 'k',
#                     label = 't = {} yrs'.format(compare[i]/makeyears),
#                     lw = .5
#     )
ax[1, 1].legend(loc="best")
ax[1, 1].set_xlabel("$X_{An}$", fontsize=20)
ax[1, 1].set_xlim(plag_bygrain["An"].min(), plag_bygrain["An"].max())
# ax[1,1].set_ylim(.8*te.min(),1.2*te.max())
fig.tight_layout(pad=1)
# save your figure
```

##### Determining Uncertainties on your model
### Define the Monte_Carlo_FD function
This part uses the analytical uncertainties in your Mg profile to generate a random profile for each iteration. It then uses the same boundary conditions and An profile you defined above to establish your diffusion model and run it to find the best fit iteration. The main difference here is that the observed profile is changing each time randomly (based on the analytical uncertainty). You then end up with a list: ```best_fits``` that represents the best fit from each random profile generated. We can then do some statistics on that distribution.

Instead of running for a set amount of time, which can increase the amount of time actually needed if the best fit diffusion curve is near the middle or beginning of our time range, s we use a ```while``` loop that essentially says "As long as the $\chi^2$ goodness of fit value for the entire curve is less than the previous iteration, keep iterating because we haven't found the minimum yet. As soon as the $\chi^2$ goodness of fit value for the entire curve is greater than the previous iteration, stop."

To keep track of iterations we create a counter. To keep track of the counter and the $\chi^2$ values, we create some containers to hold the values in the form of a list:
```python
count_list = []
chi2_list = []
```
Then in each iteration we can just append the values from that iteration to the list container. Because the ```while``` loop breaks at the best fit, the final ```count``` value is our best fit time.

**This only works if there are no local minima in our $\chi^2$ data (e.g., upper right plot in our panel above). Otherwise, use the ```Monte_Carlo_FD_long``` function that just runs for a set number of iterations**

```python
def Monte_Carlo_FD(n, limit, init_prof):
    """
    runs a monte carlo simulation that generates a random profile
    for each iteration of the diffusion model

    inputs:
    n : int
    number of iterations to run the monte carlo simulation for

    limit : int
    the maximum amount of diffusion iterations to search for the best time

    init_prof : array-like
    the initial boundary conditions for the diffusion model each iteration of the
    monte carlo simulation

    returns:
    best_fits: list
    list of best fit times for the diffusion model. Units are in the same as
    the original diffusion model specified above

    """
    best_fits = [] * n
    x = np.linspace(np.min(dist), np.max(dist), nx)

    # containers for each iteration
    # unknown at current iteration
    #     u = np.zeros(len(u_n))

    for i in tqdm(range(0, n)):
        u_n = init_prof.copy()

        yrand = random_profile(dist, te, te_unc)
        f_rand = interp.interp1d(dist, yrand)
        rand_interp = f_rand(x)

        # initial chi squared values. Should be high enough to never be an issue
        chi2_p = 10000
        chi2_c = 9999
        count = 0
        count_list = []
        chi2_list = []

        while chi2_c < chi2_p:
            count += 1
            chi2_p = chi2_c
            # this is long...
            u[1 : nx - 1] = u_n[1 : nx - 1] + dt * (
                ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
                * ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
                + D_interp[1 : nx - 1]
                * ((u_n[2:nx] - 2 * u_n[1 : nx - 1] + u_n[0 : nx - 2]) / dx ** 2)
                - (a_Mg[0] / (R * T_K))
                * (
                    D_interp[1 : nx - 1]
                    * (
                        ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
                        * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
                    )
                    + u_n[1 : nx - 1]
                    * (
                        ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
                        * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
                    )
                    + D_interp[1 : nx - 1]
                    * u_n[1 : nx - 1]
                    * (
                        (
                            An_smooth[2:nx]
                            - 2 * An_smooth[1 : nx - 1]
                            + An_smooth[0 : nx - 2]
                        )
                        / dx ** 2
                    )
                )
            )
            # letting the most 'core-ward' boundary condition diffuse according to the Costa 2003 equation
            u[0] = u_n[0] + dt * (
                ((D_interp[1] - D_interp[0]) / dx) * ((u_n[1] - u_n[0]) / dx)
                + D_interp[0] * ((u_n[1] - 2 * u_n[0] + u_n[1]) / dx ** 2)
                - (a_Mg[0] / (R * T_K))
                * (
                    D_interp[0]
                    * (((u_n[1] - u_n[0]) / dx) * ((An_smooth[1] - An_smooth[0]) / dx))
                    + u_n[0]
                    * (
                        ((D_interp[1] - D_interp[0]) / dx)
                        * ((An_smooth[1] - An_smooth[0]) / dx)
                    )
                    + D_interp[0]
                    * u_n[0]
                    * ((An_smooth[1] - 2 * An_smooth[0] + An_smooth[1]) / dx ** 2)
                )
            )

            # fix the most 'rim-ward' concentration (infinite reservoir assumption)
            u[-1] = te_interp[-1]

            # #     infinite reservoir based on boundary condition fixed based on boundary conditions
            #     u[-1] = step_func[-1]

            # not infinite reservoir assumption. Let's everything diffuse and does not keep it fixed
            # potentially useful for grains that are not at the rim
            #     u[-1] = u_n[-1] +  dt*(((D_interp[-2] - D_interp[-1])/dx)*((u_n[-2]-u_n[-1])/dx) + D_interp[-1]*((u_n[-2]-2*u_n[-1]+u_n[-2])/dx**2)-(a_Mg[0]/(R*T_K))*(D_interp[-1]*(((u_n[-2]-u_n[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + u_n[-1]*(((D_interp[-2]-D_interp[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + D_interp[-1]*u_n[-1]*((An_smooth[-2]-2*An_smooth[-1]+An_smooth[-2])/dx**2)))

            # saving your iteration to your curve container
            curves[n, :] = u
            # switch your variables before the next iteration
            # makes your current u vals the u_n vals in the next loop
            u_n[:] = u

            chi2_c = np.sum((u - rand_interp) ** 2 / rand_interp,)

            if count == limit:
                break

        bf_time_mc = count
        best_fits.append(bf_time_mc)
    #         print(bf_time_mc)

    return best_fits

def Monte_Carlo_FD_long(n, limit):
    """runs a monte carlo simulation that generates a random profile
    for each iteration of the diffusion model

    inputs:
    n : int
    number of iterations to run the monte carlo simulation for

    limit : int
    the amount of diffusion iterations to search for the best time

    returns:
    best_fits: list
    list of best fit times for the diffusion model. Units are in the same as
    the original diffusion model specified above
    """
    best_fits = [] * n
    x = np.linspace(np.min(dist), np.max(dist), nx)

    for i in tqdm(range(0, n)):
        u_n[:step_begin_idx] = Cl
        u_n[step_begin_idx:step_stop_idx] = Cm
        u_n[step_stop_idx:] = Cr

        yrand = random_profile(dist, te, te_unc)
        f_rand = interp.interp1d(dist, yrand)
        rand_interp = f_rand(x)

        u_n[:step_begin_idx] = Cl
        u_n[step_begin_idx:step_stop_idx] = Cm
        u_n[step_stop_idx:] = Cr

        # creating a container to put all of your curve iterations
        curves = np.zeros((nt, nx))

        for n in range(0, int(limit)):
            # this is long...
            u[1 : nx - 1] = u_n[1 : nx - 1] + dt * (
                ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
                * ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
                + D_interp[1 : nx - 1]
                * ((u_n[2:nx] - 2 * u_n[1 : nx - 1] + u_n[0 : nx - 2]) / dx ** 2)
                - (a_Mg[0] / (R * T_K))
                * (
                    D_interp[1 : nx - 1]
                    * (
                        ((u_n[2:nx] - u_n[1 : nx - 1]) / dx)
                        * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
                    )
                    + u_n[1 : nx - 1]
                    * (
                        ((D_interp[2:nx] - D_interp[1 : nx - 1]) / dx)
                        * ((An_smooth[2:nx] - An_smooth[1 : nx - 1]) / dx)
                    )
                    + D_interp[1 : nx - 1]
                    * u_n[1 : nx - 1]
                    * (
                        (
                            An_smooth[2:nx]
                            - 2 * An_smooth[1 : nx - 1]
                            + An_smooth[0 : nx - 2]
                        )
                        / dx ** 2
                    )
                )
            )
            # letting the most 'core-ward' boundary condition diffuse according to the Costa 2003 equation
            u[0] = u_n[0] + dt * (
                ((D_interp[1] - D_interp[0]) / dx) * ((u_n[1] - u_n[0]) / dx)
                + D_interp[0] * ((u_n[1] - 2 * u_n[0] + u_n[1]) / dx ** 2)
                - (a_Mg[0] / (R * T_K))
                * (
                    D_interp[0]
                    * (((u_n[1] - u_n[0]) / dx) * ((An_smooth[1] - An_smooth[0]) / dx))
                    + u_n[0]
                    * (
                        ((D_interp[1] - D_interp[0]) / dx)
                        * ((An_smooth[1] - An_smooth[0]) / dx)
                    )
                    + D_interp[0]
                    * u_n[0]
                    * ((An_smooth[1] - 2 * An_smooth[0] + An_smooth[1]) / dx ** 2)
                )
            )

            # fix the most 'rim-ward' concentration (infinite reservoir assumption)
            u[-1] = te_interp[-1]

            # #     infinite reservoir based on boundary condition fixed based on boundary conditions
            #     u[-1] = step_func[-1]

            # not infinite reservoir assumption. Let's everything diffuse and does not keep it fixed
            # potentially useful for grains that are not at the rim
            #     u[-1] = u_n[-1] +  dt*(((D_interp[-2] - D_interp[-1])/dx)*((u_n[-2]-u_n[-1])/dx) + D_interp[-1]*((u_n[-2]-2*u_n[-1]+u_n[-2])/dx**2)-(a_Mg[0]/(R*T_K))*(D_interp[-1]*(((u_n[-2]-u_n[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + u_n[-1]*(((D_interp[-2]-D_interp[-1])/dx)*((An_smooth[-2]-An_smooth[-1])/dx)) + D_interp[-1]*u_n[-1]*((An_smooth[-2]-2*An_smooth[-1]+An_smooth[-2])/dx**2)))

            # saving your iteration to your curve container
            curves[n, :] = u
            # switch your variables before the next iteration
            # makes your current u vals the u_n vals in the next loop
            u_n[:] = u

        chi2 = abs(
            np.sum(
                (rand_interp[None, :] - curves) ** 2 / (rand_interp[None, :]), axis=1
            )
        )
        chi2_min = np.min(chi2)
        fit_idx = np.argwhere(chi2 == chi2_min)
        fit_idx = fit_idx[0].item()
        bf_time_mc = fit_idx + 1

        best_fits.append(bf_time_mc)

    return best_fits
```

## Run the Monte Carlo simulation
Choose the number of iterations you want to run for the monte carlo simulation and run it!

```python
# run the monte carlo simulation a desired number of times. Put an upper limit on it.
# This is really only an issue if you have large uncertainties in your data which may cause erroneous
# profiles

# number of iterations
limit = 20 * 18250

best_fits = Monte_Carlo_FD(1000, limit, step_func)
# best_fits = Monte_Carlo_FD_long(500,limit)
```

      0%|          | 0/1000 [00:00<?, ?it/s]

## Plot up the orignal distribution
```python
x = np.array(best_fits)/makeyears
```
This turns the list into an array and then converts it from iterations to a value that resembles years

```python
# convert our iterations to years
xx = np.array(best_fits) / makeyears

# if you need to trim some extremely erroneous tails. Sometimes large error bars on one singular
# point will mess up the random profiles.
# you can just filter the array like this:
# x = x[x>100]

# plot up original distribution
fig, ax = plt.subplots(figsize=(10, 4))
sns.histplot(xx, bins=75, kde=False, color="gray", ec="k", element="step")
ax.set_title("Monte Carlo Diffusion model results: Plag diffusion model")
ax.set_xlabel("Best Fit Diffusion time (yrs)", fontsize=16)
ax.set_ylabel("Frequency")
ax.text(0.6, 0.9, "n = {}".format(len(xx)), transform=ax.transAxes, fontsize=16)
ax.text(
    0.6,
    0.7,
    "$T_{{mean}}$ = {} yrs".format(np.round(np.mean(xx), 2)),
    transform=ax.transAxes,
    fontsize=16,
)
ax.text(
    0.6,
    0.6,
    "$T_{{95}} = \pm$ {} yrs".format(np.round(1.96 * np.std(xx), 2)),
    transform=ax.transAxes,
    fontsize=16,
)
ax.text(
    0.6,
    0.5,
    "$T_{{med}}$ = {} yrs".format(np.round(np.median(xx), 2)),
    transform=ax.transAxes,
    fontsize=16,
)
# ax.set_xlim(0,8000)
ax.minorticks_on()
# this is manually input after your run the diffusion model based on the measured profile not the random
ax.text(
    0.6,
    0.4,
    "$T_{{actual}}$ = {} yrs".format(np.round(bf_time / makeyears, 2)),
    transform=ax.transAxes,
    fontsize=16,
)
```

## Transform the data to make it normal
In order for things like the mean and standard deviation to be useful for uncertainties, the data must be normally distributed. To accomplish this we can transform it. Commonly this is either a log transform or square root transform. [This](https://www.statisticssolutions.com/transforming-data-for-normality/) link can help with some explanation on which transform to choose and [this](http://www.biostathandbook.com/transformation.html) link can help with an explanation on back transforms (which can be a little tricky if you log transform your data). I think that both log and sqrt transforms work fairly well here, but the sqrt transform makes the data fit a normal distribution (plotted over the transform for reference) better. Neither are perfect, but they are MUCH better than the original distribution.

In the below plot,the mean, median, 95% confidence limits, and best fit time from above are displayed.

You now have your best fit diffusion time with calculated uncertainties!

```python
bin_n = 75

# SQUARE ROOT TRANSFORM
# Transforming your data to make it normally distributed
# transform = np.sqrt(x)
# transform_std = np.std(transform)
# transform_mean = np.mean(transform)
# transform_median = np.median(transform)

# #Back calculate mean and standard deviation
# back_all = transform**2
# back_mean = transform_mean**2
# back_median = transform_median**2
# back_std_l = (transform_mean - 1.96*transform_std)**2
# back_std_u = (transform_mean + 1.96*transform_std)**2

# LOG TRANSFORM
# Transforming your data to make it normally distributed
transform = np.log(xx)
transform_std = np.std(transform)
transform_mean = np.mean(transform)
transform_median = np.median(transform)

# Back calculate mean and standard deviation
back_all = np.exp(transform)
back_mean = np.exp(transform_mean)
back_median = np.exp(transform_median)
back_std_l = np.exp(transform_mean - 1.96 * transform_std)
back_std_u = np.exp(transform_mean + 1.96 * transform_std)

fig, ax = plt.subplots(1, 2, figsize=(14, 4), gridspec_kw={"width_ratios": [1.5, 1.1],})

sns.histplot(
    back_all, bins=bin_n, color="darkgray", edgecolor="k", element="step", ax=ax[0]
)

ax[0].text(
    0.55,
    0.9,
    "$T_{{actual}}$ = {} yrs".format(np.round(bf_time / makeyears, 2)),
    transform=ax[0].transAxes,
    fontsize=16,
)
ax[0].text(
    0.55,
    0.8,
    "$T_{{mean}}$ = {} yrs".format(np.round(back_mean, 2)),
    transform=ax[0].transAxes,
    fontsize=16,
)
ax[0].text(
    0.55,
    0.7,
    "$T_{{95}}$ {} ; {} yrs".format(np.round(back_std_l, 2), np.round(back_std_u, 2)),
    transform=ax[0].transAxes,
    fontsize=16,
)
ax[0].text(
    0.55,
    0.6,
    "$T_{{median}}$ = {} yrs".format(np.round(back_median, 2)),
    transform=ax[0].transAxes,
    fontsize=16,
)

ax[0].set_ylabel("frequency", fontsize=20)
ax[0].set_xlabel("best fit diffusion times", fontsize=16)
ax[0].set_title("Original Monte Carlo Distribution n = {}".format(len(xx)), fontsize=20)
ax[0].minorticks_on()

# histogram for the transformed data

# Goes into the height array and finds the value that is neares the transform mean
# To get one singular number, just change the value that gets multiplied by "transform_mean"
#'e.g.' .99*transform_mean
# histogram for the transformed data
h2, bins2, patches2 = ax[1].hist(
    transform, bins=bin_n, color="darkgray", edgecolor="k", ls=":"
)

multiply = h2[
    np.where(
        np.logical_and(bins2 > 0.995 * transform_mean, bins2 < 1.0 * transform_mean)
    )
]
# create normal distribution based on your transform mean, std dev, and the bin locations as the x array
normal_dist = (1 / transform_std * np.sqrt(2 * np.pi)) * np.exp(
    -0.5 * ((bins2 - transform_mean) / transform_std) ** 2
)

# scale the normal distribution to match the height of your histogram
normal_dist_scaled = normal_dist * multiply / np.max(normal_dist)

ax[1].plot(bins2, normal_dist_scaled, c="r", lw=3, label="normal distribution")
# ax[1].set_xlabel('$\sqrt{bf_{time}}$',fontsize = 16)
ax[1].set_xlabel("$\log{bf_{time}}$", fontsize=16)

ax[1].minorticks_on()
# ax[1].set_title('$\sqrt{x}$ Transform',fontsize = 20)
ax[1].set_title("$\log{x}$ Transform", fontsize=20)

ax[1].legend(loc="upper right", fontsize=9)
fig.suptitle("Diffusion Model Uncertainty Toba: {}".format(grain), y=1.1, fontsize=28)

```

```python

```


# 4/ Exercise solutions

```python
# helper code needed for running in colab
if 'google.colab' in str(get_ipython()):
    print('Downloading plot_helpers.py to util/ (only neded for colab')
    !mkdir util; wget https://raw.githubusercontent.com/minireference/noBSLAnotebooks/master/util/plot_helpers.py -P util
```

```python
from sympy import *
init_printing()

%matplotlib inline
import matplotlib.pyplot as mpl
from util.plot_helpers import plot_augmat, plot_plane, plot_point, plot_line, plot_vec, plot_vecs

Vector = Matrix  # define alias Vector so I don't have to explain this during video
Point = Vector   # define alias Point for Vector since they're the same thing
```

### E4.5

```python
# Given point r
r = Point([1,3,0])

# line L
pL = Point([0,0,2])
vL = Vector([1,-1,0])  # note this changed in v2.2 of LA book

# plane P
nP = Vector([1,1,1])
dP = 1

# compute the following closest distances:
```

```python
# a)  d(r,O) = ||r||
r.norm()
```

```python
# b)   d(L,O) = || pL - proj_{L}(pL) ||
(pL - (pL.dot(vL)/vL.norm()**2)*vL).norm()
```

```python
# c)   d(P,O) = || proj_{nP}(pP) ||
pP = Matrix([1,0,0])
( (nP.dot(pP)/nP.norm()**2)*nP).norm()
```

```python
# d)   u=pL-r   d(r,L)=||u - proj_{L}(u)||
u = pL - r
(u - (u.dot(vL)/vL.norm()**2)*vL).norm()
```

```python
# e)     v=pP-r   d(r,P)=||proj_{nP}(v)||
pP = Matrix([1,0,0])
v = pP - r
( (nP.dot(v)/nP.norm()**2)*nP).norm()
```

```python
#f)  d(L,P)
# STEP1: check relative orientation of L and P
print('vL·nP =', vL.dot(nP), 'so line is parallel to plane. OK to proceed...')
# STEP2: find vector w=pP-pL arbitrary points on plane nad line,
pP = Matrix([1,0,0])
w = pP - pL
# then compute component of w in perp. to the plane    d(L,P)=||proj_{nP}(w)||
( (nP.dot(w)/nP.norm()**2)*nP).norm()
```

```python
# # debug viz to make sure L parallel to P
# plot_vec(5*vL, at=pL, color='b')
# plot_vec(-5*vL, at=pL, color='b')
# plot_vec(w, color='r', at=pL)
# plot_plane(nP,dP)
```

```python

```

```python

```

### E4.7

```python
# Setup the variables of the exercise:
p = Point([10,10,10])
pL1 = Point([3,0,5])   # an arbitrary point on L1
pL2 = Point([6,0,0])   # an arbitrary point on L2
d = Vector([1,-2,0])   # direction vector of L1 and L2

```

a) Projection of $p$ onto $\ell_1$

```python
# define a vector from a point on L1 to p:
v1 = p - pL1

# proj_{L1}(p) = proj_{L1}(v1) + pL1
p_proj_L1 = (d.dot(v1)/d.norm()**2)*d + pL1
p_proj_L1
```

^ This is the point on the line $\ell_1$ that is closes to the point $p$

b) (shortest) distance form $p$ to $\ell_1$

```python
# d(p, L1) = subtract from v1 the part that is perp to L1 and compute the length:
(v1 - (d.dot(v1)/d.norm()**2)*d).norm()
```

```python
# ... or compute the distance directly:
(p_proj_L1-p).norm()
```

```python
(p_proj_L1-p).norm().n()  # numeric approx.
```

```python
plot_line(d, pL1)  # Line L1

# vector v1 and it's decomposition into parallel-to-L1 and perp-to-L1 components
plot_vec(v1, at=pL1)
plot_vec(p_proj_L1-pL1, at=pL1,     color='b')
plot_vec(p-p_proj_L1, at=p_proj_L1, color='r')

ax = mpl.gca()
mpl.xlim([0,10])
mpl.ylim([0,10])
ax.set_zlim([0,10])
ax.grid(True,which='both')
```

Answer to a) is the tip of the blue vector; answer to b) is the length of the red vector.

```python

```

Use a similar approach for c) and d)

```python
# define a vector from a point on L2 to p:
v2 = p - pL2
```

```python
# p_proj_L2 =
(d.dot(v2)/d.norm()**2)*d + pL2
```

```python
# d(p, L2) =
(v2 - (d.dot(v2)/d.norm()**2)*d).norm()
```

```python
(v2 - (d.dot(v2)/d.norm()**2)*d).norm().n()
```

```python

```

```python

```

e) distance $\ell_1$ to $\ell_2$

```python
# first define a vector from a point on L1 to a point on L2:
v3 = pL2 - pL1
v3
```

```python
# d(L1, L2) =
d_L1L2 = (v3 - (d.dot(v3)/d.norm()**2)*d).norm()
d_L1L2
```

```python
d_L1L2.n()
```

```python

```

```python

```

```python

```


# Estimating MI by Probabilistic Classifier Method (PCM)

## Setup

The following cell imports the necessary packages:

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import os
import numpy as np
import matplotlib.pyplot as plt
import argparse
from torch.autograd import Variable
import itertools
from tqdm import tqdm
from data.mix_gaussian import MixedGaussian
from data.gaussian import Gaussian
from model.utils import *
from torch.utils.tensorboard import SummaryWriter

cuda = True if torch.cuda.is_available() else False
FloatTensor = torch.cuda.FloatTensor if cuda else torch.FloatTensor
LongTensor = torch.cuda.LongTensor if cuda else torch.LongTensor
```

Use GPU if GPU is available.

```python
cuda = True if torch.cuda.is_available() else False
FloatTensor = torch.cuda.FloatTensor if cuda else torch.FloatTensor
LongTensor = torch.cuda.LongTensor if cuda else torch.LongTensor
torch.set_default_tensor_type(FloatTensor)

name = './results/PCM' # filename
chkpt_name = name+'.pt'                         # checkpoint
writer = SummaryWriter('./results/log/PCM')
```

Set random seed for reproducibility.

```python
SEED = 0
torch.manual_seed(SEED)
torch.cuda.manual_seed(SEED)
np.random.seed(SEED)
```

Specify the hyper-parameters:

```python
parser = argparse.ArgumentParser()
parser.add_argument("--rho", type=float, default=0.9, help="coefficient of Gaussian")
parser.add_argument("--d", type=int, default=6, help="dimension of X & Y")
parser.add_argument("--sample_size", type=int, default=400, help="sample size")
parser.add_argument("--gamma", type=float, default=1e-8, help="clipping parameter")
parser.add_argument("--b1", type=float, default=0.5, help="adam: decay of first order momentum of gradient")
parser.add_argument("--b2", type=float, default=0.999, help="adam: decay of first order momentum of gradient")
parser.add_argument("--n_iters", type=int, default=60000, help="number of epochs of training")
parser.add_argument("--batch_size", type=int, default=100, help="size of the batches")
parser.add_argument("--lr", type=float, default=1e-4, help="adam: learning rate")
parser.add_argument("--hidden_dim", type=int, default=100, help="Hidden dimension")
parser.add_argument("--c_0_1_ratio", type=float, default=1, help="Ratio of samples with label 0 and samples with label 1 ")
parser.add_argument("--ma_rate", type=float, default=0.1, help="move average rate")
parser.add_argument("--ma_ef", type=float, default=1, help="move average ef")
parser.add_argument("--alpha", type=float, default=1e-4, help="smooth parameter")

opt, unknown = parser.parse_known_args()
```

Specify the distribution to generate the data samples:

```python
# Two choices here: 'Gaussian' and 'MixedGaussian
density = 'Gaussian'
```

## Model

Define the function `generate_data` for generating Gaussian or Mixed Gaussian distributions.

- When generating Gaussian distrition, ${X}$ and ${Y}$ are distributed as
\begin{align}
p_{XY} = \prod_{i=1}^d \mathcal{N} \left( \mathbf{0}, \begin{bmatrix}1 & \rho \\ \rho & 1\\ \end{bmatrix} \right)
\end{align}
- When generating Mixed Gaussian distributions, ${X}$ and ${Y}$ are distributed as
 \begin{align}
 p_{XY} = \frac{1}{2} \prod_{i=1}^d \mathcal{N} \left( \mathbf{0}, \begin{bmatrix}1 & \rho \\ \rho & 1\\ \end{bmatrix} \right) + \frac{1}{2} \prod_{i=1}^d \mathcal{N} \left( \mathbf{0}, \begin{bmatrix}1 & {-} \rho \\ {-} \rho & 1\\ \end{bmatrix} \right)
 \end{align}

```python
def generate_data(distribution='Gaussian', rho=0.9):
    # rho is the covariance for generating distributions
    # mu1 and mu2 are means for generating Mixed Gaussian distribution
    mu1 = 0
    mu2 = 0

    # mg is an object of Class Gaussian or MixedGaussian
    if distribution=='Gaussian':
        mg = Gaussian(sample_size=opt.sample_size,rho=rho)
    else:
        mg = MixedGaussian(sample_size=opt.sample_size, mean1=mu1, mean2=mu2,rho1=rho, rho2=-rho)

    # Calculate the ground truth MI between X and Y for (X, Y) from mg
    mi = mg.ground_truth * opt.d

    # Create X, Y for storing generated samples
    X = np.zeros((opt.sample_size,opt.d))
    Y = np.zeros((opt.sample_size,opt.d))

    # Generate samples of random variable X,Y and XY
    for j in range(opt.d):
        # In each iteration, mg.data will generate samples of two dimensions where one dimension for X and another for Y respectively
        data = mg.data
        X[:,j] = data[:,0]
        Y[:,j] = data[:,1]
    X = torch.Tensor(X)
    Y = torch.Tensor(Y)
    XY = torch.cat((X, Y), dim=1)

    return XY, X, Y, mi
```

Define the neural network.

```python
class Net(nn.Module):
    # Inner class that defines the neural network architecture
    def __init__(self, input_size=2, hidden_size=100, sigma=0.02):
        super().__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, 1)
        nn.init.normal_(self.fc1.weight, std=sigma)
        nn.init.constant_(self.fc1.bias, 0)
        nn.init.normal_(self.fc2.weight, std=sigma)
        nn.init.constant_(self.fc2.bias, 0)
        nn.init.normal_(self.fc3.weight, std=sigma)
        nn.init.constant_(self.fc3.bias, 0)

    def forward(self, input):
        output = F.elu(self.fc1(input))
        output = F.elu(self.fc2(output))
        output = F.sigmoid(self.fc3(output))
        return output
```

The function `_resample` below is for resampling the given data samples for training the neural network.

```python
def _resample(data, batch_size, replace=False):
    # Resample the given data sample.
    index = np.random.choice(
        range(data.shape[0]), size=batch_size, replace=replace)
    batch = data[index]
    return batch

```

The neural network `discriminator` below implements and outputs $p_{{C} | \tilde{X}\tilde{Y}}(1|x,y)$ before clipping:

```python
discriminator = Net(input_size=opt.d*2, hidden_size=100)
# move NN model GPU if GPU is available
if cuda:
    discriminator.cuda()

# Adam opetimizer
optimizer_D = torch.optim.Adam(discriminator.parameters(), lr=opt.lr, betas=(opt.b1, opt.b2))

```

Load previous results.

```python
load_available = True # set to False to prevent loading previous results
if load_available and os.path.exists(chkpt_name):
    checkpoint = torch.load(
        chkpt_name, map_location='cuda' if torch.cuda.is_available() else 'cpu')
    mi_list = checkpoint['mi_list']
    model_state = checkpoint['model_state']
    discriminator.load_state_dict(model_state)
    print('Previous results loaded.')
else:
    mi_list = [] # storing the mi estimation of each iteration
```

    Previous results loaded.

## Training

The estimate of MI given by Probabilistic Classifier Method (PCM) is as follows:
\begin{align}
I(X, Y) = E \left[ \ln r(X, Y) \right],
\end{align}
where
\begin{align}
r(X, Y) := \frac{p_{C}(0)}{p_{C}(1)} \cdot \frac{p_{{C} | \tilde{X}\tilde{Y}} (1|x,y)}{p_{{C} | \tilde{X}\tilde{Y}} (0|x,y)}.
\end{align}

To obtain MI estimate of given samples by Probabilistic Classifier Method (PCM), we train the neural network $\hat{p}_{\theta}$ to implement $p_{{C} | \tilde{X}\tilde{Y}}(1|x,y)$ by minimizing the cross-entropy of $\hat{p}_{\theta}$ relative to $p_{{C} | \tilde{X}\tilde{Y}}$.

The output of the neural network $\hat{p}_{\theta}$ may be clipped to a certain range before used for calculating MI estimate.

The following cell train the neural network using the data samples.

```python
XY, X, Y, Ground_truth = generate_data(distribution=density, rho=opt.rho)

continue_train = False  # set to True to continue to train
if continue_train:
    for i in range(opt.n_iters):
        data_joint = _resample(XY, batch_size=opt.batch_size)
        X_ref = resample(X, batch_size=opt.batch_size)
        Y_ref = resample(Y, batch_size=opt.batch_size)
        data_margin = torch.cat((X_ref, Y_ref), dim=1)
        valid = Variable(torch.Tensor(opt.batch_size, 1).fill_(1.0), requires_grad=False)
        fake = Variable(torch.Tensor(opt.batch_size, 1).fill_(0.0), requires_grad=False)
        train_data = torch.cat((data_joint, data_margin), dim=0)
        labels = torch.cat((valid, fake), dim=0)
        pred_label = discriminator(train_data)
        alpha = data_margin.shape[0]/data_joint.shape[0]

        optimizer_D.zero_grad()
        # PCM loss
        loss = torch.nn.BCELoss()(pred_label, labels)

        loss.backward()
        optimizer_D.step()
        with torch.no_grad():
            mi_est = mi_estimate(discriminator, XY, opt.gamma, alpha)
            if torch.isinf(mi_est):
                mi_est = torch.from_numpy(np.array(mi_list[-1]))
            mi_list.append(mi_est.item())
        writer.add_scalar('mi_list', mi_est.item(), i)
        writer.add_scalar('loss', loss, i)
        if i%500==0:
            print("Iternation: %d, loss: %f, mi_est: %f"%(i, loss.item(), mi_est))
writer.add_graph(discriminator, (XY,))
```

## MI estimates

Use moving average to smooth the estimated MI in each training iteration.

```python
ma_rate = 0.01
mi_copy = mi_list.copy()
for k in range(1,len(mi_list)):
    mi_copy[k] = (1-ma_rate) * mi_copy[k-1] + ma_rate * mi_copy[k]

```

Plot the MI estimation curve against the training iteration, together the ground truth.

```python
plt.plot(mi_copy, label='MI estimate')
plt.axhline(Ground_truth,label='ground truth',linestyle='--',color='red')
for t in range(len(mi_copy)):
    if (mi_copy[t]>.8*Ground_truth):
        plt.axvline(t,label='80% reached',linestyle=':',color='green')
        break
plt.xlabel('number of iterations')
plt.ylabel('MI estimation')
plt.legend()

```

Save the model.

```python
overwrite = True  # set to True to overwrite previously stored results
if overwrite or not os.path.exists(chkpt_name):
    model_state = discriminator.state_dict()
    torch.save({
        'mi_list': mi_list,
        'mi_copy': mi_copy,
        'model_state': model_state
    }, chkpt_name)
    writer.close()
    print('Current results saved.')
```

    Current results saved.

```python

```


# K-Means Clustering

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from tqdm import tqdm_notebook
from sklearn.cluster import KMeans
%matplotlib inline
```

Let's use the online news popularity dataset again.

```python
df = pd.read_csv('./data/OnlineNewsPopularity/OnlineNewsPopularity.csv')
```

```python
df.columns = list(map(str.strip, df.columns))
```

```python
df.head()
```

      url
      timedelta
      n_tokens_title
      n_tokens_content
      n_unique_tokens
      n_non_stop_words
      n_non_stop_unique_tokens
      num_hrefs
      num_self_hrefs
      num_imgs
      ...
      min_positive_polarity
      max_positive_polarity
      avg_negative_polarity
      min_negative_polarity
      max_negative_polarity
      title_subjectivity
      title_sentiment_polarity
      abs_title_subjectivity
      abs_title_sentiment_polarity
      shares

      0
      http://mashable.com/2013/01/07/amazon-instant-...
      731.0
      12.0
      219.0
      0.663594
      1.0
      0.815385
      4.0
      2.0
      1.0
      ...
      0.100000
      0.7
      -0.350000
      -0.600
      -0.200000
      0.500000
      -0.187500
      0.000000
      0.187500
      593

      1
      http://mashable.com/2013/01/07/ap-samsung-spon...
      731.0
      9.0
      255.0
      0.604743
      1.0
      0.791946
      3.0
      1.0
      1.0
      ...
      0.033333
      0.7
      -0.118750
      -0.125
      -0.100000
      0.000000
      0.000000
      0.500000
      0.000000
      711

      2
      http://mashable.com/2013/01/07/apple-40-billio...
      731.0
      9.0
      211.0
      0.575130
      1.0
      0.663866
      3.0
      1.0
      1.0
      ...
      0.100000
      1.0
      -0.466667
      -0.800
      -0.133333
      0.000000
      0.000000
      0.500000
      0.000000
      1500

      3
      http://mashable.com/2013/01/07/astronaut-notre...
      731.0
      9.0
      531.0
      0.503788
      1.0
      0.665635
      9.0
      0.0
      1.0
      ...
      0.136364
      0.8
      -0.369697
      -0.600
      -0.166667
      0.000000
      0.000000
      0.500000
      0.000000
      1200

      4
      http://mashable.com/2013/01/07/att-u-verse-apps/
      731.0
      13.0
      1072.0
      0.415646
      1.0
      0.540890
      19.0
      19.0
      20.0
      ...
      0.033333
      1.0
      -0.220192
      -0.500
      -0.050000
      0.454545
      0.136364
      0.045455
      0.136364
      505

5 rows × 61 columns

```python
predictors = df.columns[1:-1]
```

```python
predictors
```

    Index(['timedelta', 'n_tokens_title', 'n_tokens_content', 'n_unique_tokens',
           'n_non_stop_words', 'n_non_stop_unique_tokens', 'num_hrefs',
           'num_self_hrefs', 'num_imgs', 'num_videos', 'average_token_length',
           'num_keywords', 'data_channel_is_lifestyle',
           'data_channel_is_entertainment', 'data_channel_is_bus',
           'data_channel_is_socmed', 'data_channel_is_tech',
           'data_channel_is_world', 'kw_min_min', 'kw_max_min', 'kw_avg_min',
           'kw_min_max', 'kw_max_max', 'kw_avg_max', 'kw_min_avg', 'kw_max_avg',
           'kw_avg_avg', 'self_reference_min_shares', 'self_reference_max_shares',
           'self_reference_avg_sharess', 'weekday_is_monday', 'weekday_is_tuesday',
           'weekday_is_wednesday', 'weekday_is_thursday', 'weekday_is_friday',
           'weekday_is_saturday', 'weekday_is_sunday', 'is_weekend', 'LDA_00',
           'LDA_01', 'LDA_02', 'LDA_03', 'LDA_04', 'global_subjectivity',
           'global_sentiment_polarity', 'global_rate_positive_words',
           'global_rate_negative_words', 'rate_positive_words',
           'rate_negative_words', 'avg_positive_polarity', 'min_positive_polarity',
           'max_positive_polarity', 'avg_negative_polarity',
           'min_negative_polarity', 'max_negative_polarity', 'title_subjectivity',
           'title_sentiment_polarity', 'abs_title_subjectivity',
           'abs_title_sentiment_polarity'],
          dtype='object')

```python
X = df[predictors]
```

Let's do some clustering!

```python
# default number of cluster is 8
km = KMeans(random_state=123)
```

```python
km.fit(X)
```

    KMeans(algorithm='auto', copy_x=True, init='k-means++', max_iter=300,
           n_clusters=8, n_init=10, n_jobs=None, precompute_distances='auto',
           random_state=123, tol=0.0001, verbose=0)

**Important step**: Explain clusters using their cluster centers

```python
pd.DataFrame(km.cluster_centers_, columns=predictors, index=[f'cluster_{i}' for i in range(km.n_clusters)])
```

      timedelta
      n_tokens_title
      n_tokens_content
      n_unique_tokens
      n_non_stop_words
      n_non_stop_unique_tokens
      num_hrefs
      num_self_hrefs
      num_imgs
      num_videos
      ...
      avg_positive_polarity
      min_positive_polarity
      max_positive_polarity
      avg_negative_polarity
      min_negative_polarity
      max_negative_polarity
      title_subjectivity
      title_sentiment_polarity
      abs_title_subjectivity
      abs_title_sentiment_polarity

      cluster_0
      312.532490
      10.474285
      637.205812
      0.514725
      0.979677
      0.667342
      11.508067
      3.590160
      4.768607
      0.772440
      ...
      0.351817
      0.090790
      0.772173
      -0.255333
      -0.544298
      -0.098654
      0.269734
      0.067475
      0.341026
      0.148424

      cluster_1
      701.075981
      9.825432
      477.826060
      0.562130
      0.990895
      0.705377
      9.249922
      3.243328
      3.481633
      1.380220
      ...
      0.367362
      0.100495
      0.770791
      -0.253593
      -0.482222
      -0.114428
      0.284196
      0.098022
      0.344088
      0.165438

      cluster_2
      260.382867
      10.604942
      484.208237
      0.540988
      0.958814
      0.673785
      10.807743
      3.132125
      4.590280
      2.007414
      ...
      0.354090
      0.099166
      0.740957
      -0.269538
      -0.519875
      -0.114768
      0.300977
      0.070144
      0.343373
      0.166063

      cluster_3
      239.278199
      10.641706
      579.094123
      0.593189
      1.080664
      0.731937
      11.657820
      3.217156
      5.273270
      0.973555
      ...
      0.358234
      0.096926
      0.767177
      -0.267543
      -0.546387
      -0.108111
      0.283421
      0.065954
      0.338656
      0.155495

      cluster_4
      616.295149
      9.713263
      499.539825
      0.555388
      0.991946
      0.698625
      10.294612
      3.373904
      3.695185
      0.862180
      ...
      0.357643
      0.096745
      0.763352
      -0.251214
      -0.485760
      -0.111387
      0.261819
      0.074259
      0.351109
      0.142987

      cluster_5
      308.187500
      10.549107
      426.375000
      0.548780
      0.937500
      0.683813
      7.464286
      2.285714
      2.241071
      2.700893
      ...
      0.344292
      0.099235
      0.708658
      -0.250590
      -0.480960
      -0.114012
      0.327289
      0.089509
      0.324973
      0.182003

      cluster_6
      345.552632
      10.298246
      583.675439
      0.541461
      1.000000
      0.697670
      14.210526
      6.324561
      2.315789
      3.947368
      ...
      0.364190
      0.083847
      0.779647
      -0.299456
      -0.569846
      -0.128740
      0.300746
      0.081838
      0.326942
      0.158885

      cluster_7
      244.601113
      10.759184
      368.749165
      0.496784
      0.842301
      0.608682
      8.720594
      2.571058
      3.952876
      3.027829
      ...
      0.320771
      0.092167
      0.659715
      -0.246313
      -0.461191
      -0.107880
      0.324640
      0.072925
      0.334441
      0.181327

8 rows × 59 columns

### Pattern/Group Discovery

```python
new_X = X.copy()
new_X['cluster_assignment'] = km.predict(X)
new_X.insert(0, 'url', df.url)
new_X.insert(new_X.shape[1], 'shares', df.shares)
```

```python
new_X[['url', 'cluster_assignment', 'shares']].head()
```

      url
      cluster_assignment
      shares

      0
      http://mashable.com/2013/01/07/amazon-instant-...
      1
      593

      1
      http://mashable.com/2013/01/07/ap-samsung-spon...
      1
      711

      2
      http://mashable.com/2013/01/07/apple-40-billio...
      1
      1500

      3
      http://mashable.com/2013/01/07/astronaut-notre...
      1
      1200

      4
      http://mashable.com/2013/01/07/att-u-verse-apps/
      1
      505

An example of what you can explore using these information: **Average number of shares by cluster**

```python
new_X.groupby('cluster_assignment').shares.mean()
```

    cluster_assignment
    0     2914.458151
    1     3129.574568
    2     4232.111111
    3     3177.414889
    4     3199.874709
    5     3196.093750
    6    11389.885965
    7     4761.953247
    Name: shares, dtype: float64

## How many clusters is the good number of clusters?

Unless you have a SME telling you some insights on how many to expect, you sually don't know this piece of information. However, we can use metrics to figure out what is a good number.

### Goodness of Cluster Fit - Intra Distance Score

A good clustering result should produce clusters that have

    1) clusters that are as far as each other (max inter distance)
    2) clusters that are tight as possible (min intra distance)

You can calculate a score as such

\begin{equation}
    score=\frac{IntraDistance}{InterDistance}=\frac{\sum_{i=0}^n a_i^2}{\sum_{i=0}^n b_i^2}
\end{equation}

where a is the distance between each instance and its cluster centroid and b is the distance of each centroid's distance.

In `Scikit-Learn`, a score called `inertia` is calculated, which is within-cluster sum-of-square. Read more [here](https://scikit-learn.org/stable/modules/clustering.html#k-means). In another word, the tipping point of this score can be an indicator of the number of clusters we should use for the final result.

```python
km.inertia_
```

    152497198933272.78

**An example of searching number of clusters**

```python
search_range = list(range(2, 21, 2))
search_range
```

    [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

```python
scores = []
for n_cluster in tqdm_notebook(search_range):
    km = KMeans(n_clusters=n_cluster, random_state=123)
    km.fit(X)
    scores.append(km.inertia_)
```

    /Users/tli/Desktop/talks/data-science-for-developers/venv/lib/python3.6/site-packages/ipykernel_launcher.py:2: TqdmDeprecationWarning: This function will be removed in tqdm==5.0.0
    Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`

    HBox(children=(IntProgress(value=0, max=10), HTML(value='')))

As you can see above, the more clusters there are, the longer it takes for the algorithm to converge.

```python
plt.plot(search_range, scores)
```

This is also called an "elbow plot", where we use to find the elbow point or the turning point indicating the score is becoming plateau. Let's go with `k=2` for now.

```python
km = KMeans(n_clusters=2, random_state=123)
```

```python
km.fit(X)
```

    KMeans(algorithm='auto', copy_x=True, init='k-means++', max_iter=300,
           n_clusters=2, n_init=10, n_jobs=None, precompute_distances='auto',
           random_state=123, tol=0.0001, verbose=0)

```python
pd.DataFrame(km.cluster_centers_, columns=predictors, index=[f'cluster_{i}' for i in range(km.n_clusters)])
```

      timedelta
      n_tokens_title
      n_tokens_content
      n_unique_tokens
      n_non_stop_words
      n_non_stop_unique_tokens
      num_hrefs
      num_self_hrefs
      num_imgs
      num_videos
      ...
      avg_positive_polarity
      min_positive_polarity
      max_positive_polarity
      avg_negative_polarity
      min_negative_polarity
      max_negative_polarity
      title_subjectivity
      title_sentiment_polarity
      abs_title_subjectivity
      abs_title_sentiment_polarity

      cluster_0
      324.256843
      10.448873
      552.499205
      0.547004
      0.996955
      0.687765
      11.025783
      3.29763
      4.636486
      1.238521
      ...
      0.352642
      0.095004
      0.755492
      -0.260042
      -0.525401
      -0.106897
      0.282200
      0.069104
      0.341642
      0.155249

      cluster_1
      700.957313
      9.825173
      478.033271
      0.562082
      0.990898
      0.705313
      9.257690
      3.24796
      3.487445
      1.379787
      ...
      0.367362
      0.100495
      0.770863
      -0.253600
      -0.482385
      -0.114407
      0.284107
      0.097991
      0.344136
      0.165386

2 rows × 59 columns

### Importance of `Random State`

A random state is a seed/starting point in generating random numbers. It tells the computer where to start. If we don't set it, the result of KMeans and any other algorithms that involve random processes will change every time we execute the algorithm.

**With Random State Set**

```python
km = KMeans(n_clusters=2, random_state=123)
km.fit(X)
pd.DataFrame(km.cluster_centers_, columns=predictors, index=[f'cluster_{i}' for i in range(km.n_clusters)])
```

      timedelta
      n_tokens_title
      n_tokens_content
      n_unique_tokens
      n_non_stop_words
      n_non_stop_unique_tokens
      num_hrefs
      num_self_hrefs
      num_imgs
      num_videos
      ...
      avg_positive_polarity
      min_positive_polarity
      max_positive_polarity
      avg_negative_polarity
      min_negative_polarity
      max_negative_polarity
      title_subjectivity
      title_sentiment_polarity
      abs_title_subjectivity
      abs_title_sentiment_polarity

      cluster_0
      324.256843
      10.448873
      552.499205
      0.547004
      0.996955
      0.687765
      11.025783
      3.29763
      4.636486
      1.238521
      ...
      0.352642
      0.095004
      0.755492
      -0.260042
      -0.525401
      -0.106897
      0.282200
      0.069104
      0.341642
      0.155249

      cluster_1
      700.957313
      9.825173
      478.033271
      0.562082
      0.990898
      0.705313
      9.257690
      3.24796
      3.487445
      1.379787
      ...
      0.367362
      0.100495
      0.770863
      -0.253600
      -0.482385
      -0.114407
      0.284107
      0.097991
      0.344136
      0.165386

2 rows × 59 columns

**Without Random State**

Might need to increase the number of clusters to see more obvious change.

```python
km = KMeans(n_clusters=5)
km.fit(X)
pd.DataFrame(km.cluster_centers_, columns=predictors, index=[f'cluster_{i}' for i in range(km.n_clusters)])
```

      timedelta
      n_tokens_title
      n_tokens_content
      n_unique_tokens
      n_non_stop_words
      n_non_stop_unique_tokens
      num_hrefs
      num_self_hrefs
      num_imgs
      num_videos
      ...
      avg_positive_polarity
      min_positive_polarity
      max_positive_polarity
      avg_negative_polarity
      min_negative_polarity
      max_negative_polarity
      title_subjectivity
      title_sentiment_polarity
      abs_title_subjectivity
      abs_title_sentiment_polarity

      cluster_0
      277.592180
      10.555504
      610.109796
      0.553030
      1.029220
      0.699121
      11.573725
      3.420542
      4.996551
      0.863920
      ...
      0.354802
      0.093644
      0.769729
      -0.261092
      -0.545369
      -0.103165
      0.276235
      0.066924
      0.339818
      0.151644

      cluster_1
      701.075981
      9.825432
      477.826060
      0.562130
      0.990895
      0.705377
      9.249922
      3.243328
      3.481633
      1.380220
      ...
      0.367362
      0.100495
      0.770791
      -0.253593
      -0.482222
      -0.114428
      0.284196
      0.098022
      0.344088
      0.165438

      cluster_2
      616.089161
      9.710315
      497.429895
      0.555867
      0.991958
      0.698852
      10.259441
      3.362937
      3.689860
      0.908392
      ...
      0.357740
      0.097130
      0.762617
      -0.251707
      -0.486138
      -0.111403
      0.261645
      0.073320
      0.351674
      0.143067

      cluster_3
      305.330396
      10.541850
      422.008811
      0.544106
      0.929515
      0.678237
      7.387665
      2.268722
      2.215859
      2.674009
      ...
      0.341127
      0.098364
      0.701495
      -0.248526
      -0.476365
      -0.113239
      0.324946
      0.088547
      0.325304
      0.179818

      cluster_4
      250.957146
      10.660261
      454.089132
      0.527180
      0.925337
      0.654065
      10.305059
      2.992490
      4.443782
      2.298763
      ...
      0.344591
      0.096802
      0.718631
      -0.263109
      -0.504161
      -0.112730
      0.308219
      0.071112
      0.340056
      0.170856

5 rows × 59 columns

**Bonus**

Other metric and clustering methods too look into:
- Metric: [Silhousette Score](https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html)
- Clustering method: [Other implemented algorithms](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.cluster)

```python

```


# Newton-Cotes uniform node interpolation.

These are rough notes. If you don't understand what's going on here and you want to understand it, feel free to get in touch and ask me to explain better.

In some time-stepping settings, it's nice to use a uniform grid for discretization since the grid spacing affects stability. For example, the [Courant-Friedrichs-Lewy](https://en.wikipedia.org/wiki/Courant–Friedrichs–Lewy_condition).

There are two experiments here:
1. What happens if we interpolate on a uniform grid and then integrate exactly. This is like the implementation of a normal BIE method since we can adaptively use upsampling to accurately evaluate integrals.
2. What happens if we interpolate on a uniform grid *and* integrate on that same uniform grid. This is simply a demonstration that Newton-Cotes *alone* would work, though not super well.

```python
import quadpy
import numpy as np
import sympy as sp
import matplotlib.pyplot as plt

ncq = []
ps = np.arange(2, 21, 3)
for i in ps:
    ncq.append(quadpy.c1.newton_cotes_open(i))
```

# Using uniform node interpolation to represent the solution to a BIE.

This first experiment uses a very high order quadrature rule to remove all quadrature error and focus on the error coming from different choices of nodes for interpolating the density. This is most similar to what I am doing in practice.

```python
# We use a 1000-th order Gauss rule to examine the case where integrals
# are computed exactly. This will isolate error coming from our
# representation of the density.
gq = quadpy.c1.gauss_legendre(1000)
gqx = gq.points
gqw = gq.weights

spx = sp.var('x')
for A in [1, 2, 3]:
    for C in [2, sp.Rational(1,5)]:
        print(f'running for A={A}, C={C}')

        # The exact density function
        gsp = sp.cos(A * spx)

        # The symbolic kernel
        fsp = 1 / (spx ** 2 + C ** 2)
        g = sp.lambdify((spx,), gsp)
        f = sp.lambdify((spx,), fsp)

        # Integrate symbolically to get the exact value.
        exact = float(sp.integrate(gsp * fsp, (spx, -1, 1)))

        gauss_err = []
        for i in range(ps.shape[0]):
            # Barycentric interpolation at the Gauss nodes
            I = quadpy.c1.gauss_legendre(ps[i])
            Ix = I.points
            Iw = I.weights
            dist = Ix[:, None] - Ix[None, :]
            np.fill_diagonal(dist, 1.0)
            interp_weights = 1.0 / np.prod(dist, axis=1)
            dist = gqx[:,None] - Ix[None,:]
            kernel = interp_weights[None, :] / dist
            interp_g = kernel.dot(g(Ix)) / np.sum(kernel, axis=1)

            # Integrate with the density interpolated at the gauss nodes: interp_g
            # and the kernel evaluated at the quadrature nodes
            est = np.sum(interp_g * f(gqx) * gqw)
            gauss_err.append(est - exact)

        nc_err = []
        for i in range(ps.shape[0]):
            # Barycentric interpolation at the Newton-Cotes nodes
            I = ncq[i]
            Ix = I.points
            Iw = I.weights
            dist = Ix[:, None] - Ix[None, :]
            np.fill_diagonal(dist, 1.0)
            interp_weights = 1.0 / np.prod(dist, axis=1)
            dist = gqx[:,None] - Ix[None,:]
            kernel = interp_weights[None, :] / dist
            interp_g = kernel.dot(g(Ix)) / np.sum(kernel, axis=1)

            # Integrate with the density interpolated at the gauss nodes: interp_g
            # and the kernel evaluated at the quadrature nodes
            est = np.sum(interp_g * f(gqx) * gqw)
            nc_err.append(est - exact)

        plt.title(r'convergence for $\int_{-1}^{1}[\frac{' + str(A) + r'cos(x)}{x^2 + ' + str(C) + r'^2}]dx$')
        plt.plot(ps, np.log10(np.abs(gauss_err)), 'k-o', label='gauss nodes')
        plt.plot(ps, np.log10(np.abs(nc_err)), 'r-o', label='newton-cotes nodes')
        plt.xticks(np.arange(2,21,2))
        plt.ylim([-17, 0])
        plt.xlabel('quadrature order')
        plt.ylabel('log_10 error')
        plt.legend()
        plt.show()
```

```python

```

# Newton-Cotes vs Gauss for a hypersingular integral

Lets integrate

$$
\int_{-1}^{1} \frac{1}{x^2 + C^2} dx
$$
for different values of $C$.

Looking at the figures below, it's clear that:
1. Newton-Cotes is pretty terrible compared to Gaussian quadrature for small $C$.
2. It's also bad for large $C$. But, not horrible.

In a BIE method, $C$ is controllable by setting the distance at which we refine source panels. If a source panel is in a position such that $C$ would be large, by dividing that source panel in half $C$ will be made approximately twice as large. So, this isn't a disaster for Newton-Cotes.

```python
for C in [1, 2, 3]:
    f = lambda x: 1 / (x ** 2 + C ** 2)
    exact = float(sp.integrate(1 / (spx ** 2 + C ** 2), (spx, -1, 1)))
    gauss_err = []
    for i in ps:
        q = quadpy.c1.gauss_legendre(i)
        qx = q.points
        qw = q.weights
        est = np.sum(f(qx) * qw)
        gauss_err.append(est - exact)

    nc_err = []
    for i in range(ps.shape[0]):
        q = ncq[i]
        qx = q.points
        qw = q.weights
        est = np.sum(f(qx) * qw)
        nc_err.append(est - exact)

    plt.title(r'convergence for $\int_{-1}^{1}\frac{1}{x^2 + C^2}dx$ with C = ' + str(C))
    plt.plot(ps, np.log10(np.abs(gauss_err)), 'k-o', label='gauss')
    plt.plot(ps, np.log10(np.abs(nc_err)), 'r-o', label='newton-cotes')
    plt.ylim([-17, 0])
    plt.legend()
    plt.show()
```


# juliaのSymbolicsでやってみる

```julia
using Symbolics
```

```julia
include("./kinematics.jl")
using .Kinematics
```

    WARNING: replacing module Kinematics.
    WARNING: using Kinematics.locals in module Main conflicts with an existing identifier.

```julia
@variables l1_1, l1_2, l1_3, l2_1, l2_2, l2_3
@variables ξ1, ξ2

N = 2
Q = [l1_1, l1_2, l1_3, l2_1, l2_2, l2_3]
Ξ = [ξ1, ξ2]
```

\begin{equation}
\left[
\begin{array}{c}
{\xi}1 \\
{\xi}2 \\
\end{array}
\right]
\end{equation}

```julia
Ps = locals(N, Q, Ξ)
```

```julia
Z = [l1_1 l1_2; l2_1 l2_2]
```

\begin{equation}
\left[
\begin{array}{cc}
l1_{1} & l1_{2} \\
l2_{1} & l2_{2} \\
\end{array}
\right]
\end{equation}


# How to define a compartment population model in Compartor
$$
\def\n{\mathbf{n}}
\def\x{\mathbf{x}}
\def\N{\mathbb{\mathbb{N}}}
\def\X{\mathbb{X}}
\def\NX{\mathbb{\N_0^\X}}
\def\C{\mathcal{C}}
\def\Jc{\mathcal{J}_c}
\def\DM{\Delta M_{c,j}}
\newcommand\diff{\mathop{}\!\mathrm{d}}
\def\Xc{\mathbf{X}_c}
\def\Yc{\mathbf{Y}_c}
\newcommand{\muset}[1]{\dot{\{}#1\dot{\}}}
$$

Whenever using Compartor in a Jupyter notebook, run the following commands:

```python
# initialize sympy printing (for latex output)
from sympy import init_printing, Symbol
init_printing()

# import functions and classes for compartment models
from compartor import *
```

## Usage of the constructor TransitionClass

The population dynamics are specified in Compartor through a set of transition classes. These are stoichiometric-like equations whose left-hand and right-hand sides specify how some `Compartments` are modified by the occurrence of a transition.

To define a compartment $[\x]$, it is first necessary to define some `Content` variables $\x \in \N_0^D$ that Compartor can interpret as symbols on which to perform symbolic computation. For instance,

```python
x = Content('x')
y = Content('y')
Compartment(x)
```

Content variables are $D$-dimensional, with `x[d]` denoting the copy number of chemical species `d`, for $d=0,1,...,D-1$ .

Once some content variables have been defined, the fastest way to define a transition class is the constructor `TransitionClass`. For instance,

```python
Exit = TransitionClass( [x] -to> {}, 'k_E', name='E')
display(Exit)
```

defines a transition class that randomly removes one compartment from the population with rate $k_E$. In particular:

* The first argument of `TransitionClass` is the compartment stoichiometry, where lelf-hand side and right-hand side are separated by the keyword `-to>`. The notation `[x]` denotes a compartment of content `x`, while `{}` denotes the empty set.
* The second argument assignes a name to the rate constant
* The optional parameter `name` defines the subscript of the transition propensity

Similarly, we can define a transition class that randomly fuses two compartments as follows

```python
Fusion  = TransitionClass( [x] + [y] -to> [x+y], 'k_F', name='F')
display(Fusion)
```

Note that the population dependency of the propensity `h` is automatically inferred with the law of mass action.

Note that in the compartment notation we can use compound expressions inside compartment brackets. In the above example, we have used `x+y` to denote the content formed by adding content vectors `x` and `y`.
Content vectors can be also notated explicitly as $D$-tuples, listing the copy number of each chemical species $d=0,1,...,D-1$.
For example, in the expression `[x + (-1, 0)]`, the tuple `(-1, 0)` denotes a change by $-1$ in chemical species $d=0$ (in a model with $D=2$ species). The expression could be equivalently written as `[(x[0]-1,x[1])]`. We will see more examples of this notation below.

### Propensities with content dependency
It is possible to tune the propensity as a function of the compartment contents by providing a third argument to `TransitionClass`, such as for the following chemical events given in the example of the paper:

```python
Conversion  = TransitionClass( [x] -to> [x + (-1,1)], 'k_c', x[0], name='c')
Degradation = TransitionClass( [x] -to> [x + (0,-1)], 'k_d', x[1], name='d')
display(Conversion, Degradation)
```

The Conversion class transforms the first chemical species (indexed by `0`) to the second type with propensity $k_cx_0$ in any compartment across the population. The Degradation class, instead, removes one molecule of the second chemical species with rate $k_dx_1$, for a given compartment.

Some transition classes involve compartments on the product side (i.e. right-hand side) whose content is drawn in probabilistic fashion with respect to the reactant compartments. In such cases, a conditional distribution can be passed as optional argument `pi` in `TransitionClass`.

The type of $\pi$ is `OutcomeDistribution`, which is a class comprising
* an expression or symbol to use for displaying $\pi$ in compound expressions
* a function `expectation` that takes an expression over reactant contents, and returns its expectation over all product compartment variables.

There are generators for several predefined outcome distributions. If nothing is specified, as in the above "Exit" transition example, `OutcomeDistribution.Identity()` is used by default. Instead, when the content of product compartments follows a distribution, other generators can be used or created.

Compartor currently includes the following `OutcomeDistribution` generators
* `Poisson()`
* `NegativeBinomial()`
* `Uniform()`

For example, the model in the paper has an "Intake" transition class where new compartments are created with Poisson-distributed content

```python
from sympy import Symbol
pi_I = OutcomeDistribution.Poisson(Symbol('\pi_{I}(y; \lambda)'),y[0],Symbol('\lambda'))

Intake = TransitionClass( {} -to> [(y[0],0)], 'k_I', pi=pi_I, name='I')
display(Intake)
```

## Model definition
The declaration of a model consists in defining a list of transition classes. We provide some examples of model declaration here below.

### Example: case study shown in the paper

```python
x = Content('x')
y = Content('y')

# Intake Distribution
pi_I = OutcomeDistribution.Poisson(Symbol('\pi_{I}(y; \lambda)'),y[0],Symbol('\lambda'))

Intake      = TransitionClass( {}        -to> [(y[0],0)],   'k_I', pi=pi_I, name='I')
Fusion      = TransitionClass( [x] + [y] -to> [x+y],        'k_F',          name='F')
Conversion  = TransitionClass( [x]       -to> [x + (-1,1)], 'k_c', x[0],    name='c')
Degradation = TransitionClass( [x]       -to> [x + (0,-1)], 'k_d', x[1],    name='d')

transitions = [ Intake, Fusion, Conversion, Degradation]
```

The transition classes stored into the variable `transitions` can be displayed with the function `display_transition_classes()` as follows

```python
display_transition_classes(transitions)
```

$\displaystyle \begin{align} \emptyset&\overset{h_{I}}{\longrightarrow}\left[\left( {y}_{0}, \  0\right)\right] && h_{I} = \pi_{I}(y; \lambda) k_{I}\\\left[x\right] + \left[y\right]&\overset{h_{F}}{\longrightarrow}\left[x + y\right] && h_{F} = \frac{k_{F} \left(n{\left(y \right)} - \delta_{x y}\right) n{\left(x \right)}}{\delta_{x y} + 1}\\\left[x\right]&\overset{h_{c}}{\longrightarrow}\left[\left( -1, \  1\right) + x\right] && h_{c} = k_{c} n{\left(x \right)} {x}_{0}\\\left[x\right]&\overset{h_{d}}{\longrightarrow}\left[\left( 0, \  -1\right) + x\right] && h_{d} = k_{d} n{\left(x \right)} {x}_{1} \end{align}$

### Example: nested birth-death process

```python
x = Content('x')
y = Content('y')

# Intake
pi_I = OutcomeDistribution.NegativeBinomial(Symbol('\pi_{NB}(y; \lambda)'), y[0],Symbol('r'),Symbol('p'))

Intake = TransitionClass( {}  -to> [y],   'k_I', pi=pi_I, name='I')
Exit   = TransitionClass( [x] -to> {},    'k_E',          name='E')
Birth  = TransitionClass( [x] -to> [x+1], 'k_b',          name='b')
Death  = TransitionClass( [x] -to> [x-1], 'k_d', x[0],    name='d')

transitions = [Intake, Exit, Birth, Death]
display_transition_classes(transitions)
```

$\displaystyle \begin{align} \emptyset&\overset{h_{I}}{\longrightarrow}\left[y\right] && h_{I} = \pi_{NB}(y; \lambda) k_{I}\\\left[x\right]&\overset{h_{E}}{\longrightarrow}\emptyset && h_{E} = k_{E} n{\left(x \right)}\\\left[x\right]&\overset{h_{b}}{\longrightarrow}\left[\left( 1\right) + x\right] && h_{b} = k_{b} n{\left(x \right)}\\\left[x\right]&\overset{h_{d}}{\longrightarrow}\left[\left( -1\right) + x\right] && h_{d} = k_{d} n{\left(x \right)} {x}_{0} \end{align}$

### Example: coagulation-fragmentation system with intake and export

```python
x = Content('x')
y = Content('y')

pi_I = OutcomeDistribution.Poisson(Symbol("\pi_{Poiss}(y; \lambda)"), y[0], Symbol("\lambda"))
pi_F = OutcomeDistribution.Uniform(Symbol("\pi_F(y|x)"), y[0], 0, x[0])

Intake        = TransitionClass( {}        -to> [y],         'k_I',         pi=pi_I, name='I')
Exit          = TransitionClass( [x]       -to> {},          'k_E',                  name='E')
Coagulation   = TransitionClass( [x] + [y] -to> [x+y],       'k_C',                  name='C')
Fragmentation = TransitionClass( [x]       -to> [y] + [x-y], 'k_F', g=x[0], pi=pi_F, name='F')

transitions = [Intake, Exit, Coagulation, Fragmentation]
display_transition_classes(transitions)
```

$\displaystyle \begin{align} \emptyset&\overset{h_{I}}{\longrightarrow}\left[y\right] && h_{I} = \pi_{Poiss}(y; \lambda) k_{I}\\\left[x\right]&\overset{h_{E}}{\longrightarrow}\emptyset && h_{E} = k_{E} n{\left(x \right)}\\\left[x\right] + \left[y\right]&\overset{h_{C}}{\longrightarrow}\left[x + y\right] && h_{C} = \frac{k_{C} \left(n{\left(y \right)} - \delta_{x y}\right) n{\left(x \right)}}{\delta_{x y} + 1}\\\left[x\right]&\overset{h_{F}}{\longrightarrow}\left[x - y\right] + \left[y\right] && h_{F} = \pi_F(y|x) k_{F} n{\left(x \right)} {x}_{0} \end{align}$