## Note: the example below requires Cantera 2.5.0a2 or newer

If the stable release of 2.5.0 is not yet available, you might consider [building from source](https://cantera.org/install/compiling-install.html). Note also that 2.5.0, once released, is more recent than and supercedes 2.5.0a2, which is an alpha release.

```python
import cantera as ct
print('Runnning Cantera version: ' + ct.__version__)
```

    Runnning Cantera version: 2.5.0a2

# Lithium Ion Battery Example
## Open circuit calculations as a function of anode and cathode lithium content.

In this example we will illustrate how to calculate the open circuit voltage (voltage when the external applied current is 0) for a lithium ion battery.  The thermodynamics are based on a graphite anode and a LiCoO2 cathode (the typical/standard active materials for commercial batteries, as of 2019), and are modeled using the 'BinarySolutionTabulatedThermo' class.

For the sake of simplicity, we're going to assume that the anode and cathode capacities are perfectly balanced (i.e. if the cathode lithium content is X percent of it's max possible (i.e. its capacity), then we will assume that the anode is at 1-X percent.  Without loss of generality, we will define the anode composition:

The routine below returns the cell voltage (in Volt) of a lithium-ion cell for a given cell current and active material lithium stoichiometries.

Note that the function 'E_cell' below has even greater capabilities than what we use, here. It calculates the steady state cell voltage, at a given composition and cell current, for a given electrolyte ionic resistance.  This functionality is presented in greater detail in the reference (which also describes the derivation of the BinarySolutionTabulatedThermo class): M. Mayur, S. DeCaluwe, B. L. Kee, W. G. Bessler, "Modeling thermodynamics and kinetics of intercalation phases for lithium-ion batteries in open source software", currently under review at _Electrochimica Acta_.  In the future, this example may be developed further to demonstrate simulating charge-discharge of the lithium ion battery.

Other than the typical Cantera dependencies, plotting functions require that you have matplotlib installed, and finding the electrode potentials uses Scipy's fsolve. See https://matplotlib.org/ and https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html for additional info.

The open circuit (i.e. equilibrium) voltage, here, is calculated via two means: kinetically and thermodynamically.

The system here, can be thought of as consisting of two particles--anode and cathode--connected by a liquid electrolyte:

### Kinetic equilibrium calculations
In the kinetic equilibrium problem, steady state is achieved when the faradaic current at each electrode (anode and cathode) surface and the ionic current through the electrolyte are all zero.  There are, essentially, four relevant electric potentials which must be determined:

- $\phi_{\rm anode}$: electric potential of graphite anode.
- $\phi_{\rm elyte,\, anode}$: electric potential of electrolyte at anode interface.
- $\phi_{\rm elyte,\, cathode}$: electric potential of electrolyte at cathode interface.
- $\phi_{\rm cathode}$: electric potential of LCO cathode.

Setting one of these four to the reference potential of zero (because it is the difference in electric potential which drives currents, the actual potential values are irrelevant. Let's assume the anode electric potential is zero), there is only one distribution of electric potentials across the cell such that the current is invariant across the cell.  I.e. we want the potentials such that:

\begin{equation}
i_{\rm Far,\, anode} = i_{\rm ionic} = i_{\rm Far,\,cathode}= i_{\rm app}
\end{equation}

where $i_{\rm app}$ is the user input for the applied current.  For this example, we assume an applied current of 0, to calculate the equilibrium voltage.

#### Faradaic current
The Faradaic current for this model is calculated using Butler-Volmer kinetics.  For a Li-ion battery, this is:

\begin{equation}
i = S_{\rm elde}i_\circ\left[\exp\left(\frac{F\beta\eta}{RT}\right) - \exp\left(-\frac{F\beta\eta}{RT}\right) \right]
\end{equation}

where $S_{\rm elde}$ is the specific surface area of the electrode in question, $F$ is Faraday's constant, $\beta$ is the charge-transfer symmetry parameter, $R$ the universal gas constant, $T$ the temperature, and $\eta$ the overpotential, which is the electric potential difference between the electrode and electrolyte, $\Delta \phi = \phi_{\rm elde} - \phi_{\rm elyte}$, relative to that at equilibrium, $\Delta \phi_{\rm eq}$:

\begin{equation}
\eta = \Delta \phi - \Delta \phi_{\rm eq}
\end{equation}

$i_\circ$ is known as the "exchange current density," which is equal to the rate of the forward and reverse current at equilibrium (which are equal).  $i_\circ$ and $\beta$ are provided as user inputs in the cti file.  At any particular state, (user-specified electric potentials, pressure, temperature, and chemical compositions), Cantera calculates $\eta$ as part of the routine to evaluate reaction rates of progress $\left(\dot{q} = \frac{i_{\rm Far}}{F}\right)$. The user simply sets the state values mentioned above.

#### Ionic current
The electrolyte is modeled as a resistor with user-defined ionic resistance $R_{\rm io}$, and hence the ionic current is calculated as:

\begin{equation}
i_{\rm ionic} = \frac{\phi_{\rm elyte,\,ca} - \phi_{\rm elyte,\,an}}{R_{\rm io}}
\end{equation}

where positive current is defined as delivering Li$^+$ to the anode interface.  Given $i_{\rm app}$, this equation can be inverted, to calculate the electric potential of the electrolyte at the cathode interface, relative to that at the anode interface:

\begin{equation}
\phi_{\rm elyte,\,ca} = \phi_{\rm elyte,\,an} + R_{\rm io}i_{\rm app}
\end{equation}

Again: in this example, $i_{\rm app} = 0$ and hence the two electric potential values in the electrolyte are equal.

#### Numerical routine
For the kinetic routine, there are three processes to determine the cell voltage $\phi_{\rm cathode} - \phi_{\rm anode}$ which corresponds to the user-provided $i_{\rm app}$:
1. Determine the $\phi_{\rm elyte,\,anode}$ value which corresponds to $i_{\rm app}$, given $X_{\rm Li, anode}$, the percentage of Li in the anode active material.
2. Determine $\phi_{\rm elyte,\,cathode}$, given $\phi_{\rm elyte,\,anode}$ and $i_{\rm app}$.
3. Determine the $\phi_{\rm cathode}$ which corresponds to $i_{\rm app}$, given $phi_{\rm elyte,\,cathode}$ and $X_{\rm Li, anode}$, the percentage of Li in the anode active material.

The routines below are written generally such that an interested user may set $i_{\rm app}$ to any value of interest.

#### Import necessary packages:

```python
from __future__ import division
from __future__ import print_function

import numpy as np

from scipy.optimize import fsolve

# Used for timing our calculations:
import time

# Plotting:
import matplotlib.pyplot as plt
import matplotlib as mpl

plt.rcParams['axes.labelsize'] = 16
plt.rcParams['xtick.labelsize'] = 12
plt.rcParams['ytick.labelsize'] = 12
plt.rcParams['figure.autolayout'] = True
```

### Define the phases

The phase thermodynamics are defined according to experimentally-measured open circuit voltage values, as desscribed in the reference provided above:

```python
inputCTI = 'lithium_ion_battery.cti'
anode = ct.Solution(inputCTI, 'anode');
cathode = ct.Solution(inputCTI,'cathode');
# The 'elde' electrode phase is needed as a source/sink for electrons:
elde = ct.Solution(inputCTI,'electron');
elyte = ct.Solution(inputCTI,'electrolyte');
anode_interface = ct.Interface(inputCTI, 'edge_anode_electrolyte', [anode, elde, elyte]);
cathode_interface = ct.Interface(inputCTI, 'edge_cathode_electrolyte', [cathode, elde, elyte]);
```

### Define battery conditions : temperature, pressure, stoichiometry, electrolyte resistance:

Inputs are:
- Stoichiometries X_Li_ca and X_Li_an [-] (can be vectors)
- Temperature T [K]
- Pressure P [Pa]
- Externally-applied current i_app [A]
- Electrolyte resistance R_elyte [Ohm]
- Anode total surface area S_an [m^2]
- Cathode total surface area S_ca [m^2]

```python
# Array of lithium mole fractions in the anode:
X_Li_an = np.arange(0.005, 0.995, 0.02)
# Assume that the cathode and anode capacities are balanced:
X_Li_ca = 1. - X_Li_an;

# I_app = 0: Open circuit
I_app = 0.;

# At zero current, electrolyte resistance is irrelevant:
R_elyte = 0.;

# Temperature and pressure
T = 300 # K
P = ct.one_atm

F = ct.faraday

S_ca = 1.1167; # [m^2] Cathode total active material surface area
S_an = 0.7824; # [m^2] Anode total active material surface area
```

### Set phase temperatures and pressures:

```python
phases = [anode, elde, elyte, cathode, anode_interface, cathode_interface];
for ph in phases:
    ph.TP = T,P

```

### Helper Functions:

```python
def anode_curr(phi_l,I_app,phi_s,X_Li_an):

    # Set the active material mole fraction
    anode.X = 'Li[anode]:'+str(X_Li_an)+', V[anode]:'+str(1-X_Li_an)

    # Set the electrode and electrolyte potential
    elde.electric_potential = phi_s
    elyte.electric_potential = phi_l

    # Get the net product rate of electrons in the anode (per m2^ interface)
    r_elec = anode_interface.get_net_production_rates(elde)

    anCurr = r_elec*ct.faraday*S_an;
    diff = I_app + anCurr

    return diff

def cathode_curr(phi_s,I_app,phi_l,X_Li_ca):

    # Set the active material mole fractions
    cathode.X = 'Li[cathode]:'+str(X_Li_ca)+', V[cathode]:'+str(1-X_Li_ca)

    # Set the electrode and electrolyte potential
    elde.electric_potential = phi_s
    elyte.electric_potential = phi_l

    # Get the net product rate of electrons in the cathode (per m2^ interface)
    r_elec = cathode_interface.get_net_production_rates(elde)

    caCurr = r_elec*ct.faraday*S_an;
    diff = I_app - caCurr

    return diff
```

### Run the calculations for all stoichiometries:

```python
#Tic
t0 = time.time()

# Initialize array of OCVs:
E_cell_kin = np.zeros_like(X_Li_ca)

for i,X_an in enumerate(X_Li_an):
    #Set anode electrode potential to 0:
    phi_s_an = 0
    E_init = 3.0

    phi_l_an = fsolve(anode_curr,E_init,args=(I_app,phi_s_an,X_an))

    # Calculate electrolyte potential at cathode interface:
    phi_l_ca = phi_l_an + I_app*R_elyte;

    # Calculate cathode electrode potential
    phi_s_ca = fsolve(cathode_curr,E_init,args=(I_app,phi_l_ca,X_Li_ca[i]))

    # Calculate cell voltage
    E_cell_kin[i] = phi_s_ca - phi_s_an

#Toc
t1 = time.time()
print('{:d} cell voltages calculated in {:3.2f} seconds.'.format(i,t1-t0))
```

    49 cell voltages calculated in 0.36 seconds.

### Plot cell voltage, as a function of the cathode stoichiometry:

```python
plt.figure()
plt.plot(100*X_Li_ca, E_cell_kin,color='b',linewidth=2.5)
plt.ylim([2.5,4.3])
plt.xlabel('Li Fraction in Cathode (%)',fontname='Times New Roman',fontsize=18)
plt.ylabel('Open Circuit Potential (V)',fontname='Times New Roman',fontsize=18)

ax = plt.gca()

for tick in ax.xaxis.get_major_ticks():
    tick.label1.set_fontsize(14)
    tick.label1.set_fontname('Times New Roman')
for tick in ax.yaxis.get_major_ticks():
    tick.label1.set_fontsize(14)
    tick.label1.set_fontname('Times New Roman')

plt.show()

```

### Thermodynamic Equilibrium Calculation
For the I_app = 0 case, we can also calcualte the voltage using thermodynamics.  At equilibrium, the net electrochemical potential change of the reaction must be zero:

\begin{equation}
\sum_k\nu_k\tilde{\mu}_k = 0
\end{equation}

where $\tilde{\mu}_k = \mu_k + z_kF\Phi_k$, where, in turn $\mu_k = \frac{\partial g_k}{\partial n_k}$ is the chemical potential, $\nu_k$ the net stoichiometric coefficient, $z_k$ the net elementary charge, and $\Phi_k$ the phase electric potential for species $k$.

From this, we can calculate the equilibrium electric potential difference $\Delta \Phi_{\rm eq} = \left(\Phi_{\rm elde} - \Phi_{\rm elyte}\right)_{\rm eq}$ as:

\begin{equation}
\Delta \Phi_{\rm eq} = -\frac{\Delta g_{\rm rxn}}{n_{\rm charge}F}
\end{equation}

where $\Delta g_{\rm rxn} = \sum_k \nu_k\mu_k$ is the chemical potential of the reaction and and $n_{\rm charge} = \sum_{k,\,{\rm elde}} \nu_k z_k$ is the net elementary charge transferred from the electrolyte to the electrode.

```python
#Tic
t0 = time.time()

# Initialize array of OCVs:
E_cell_therm = np.zeros_like(X_Li_ca)

for i,X_an in enumerate(X_Li_an):
    #Set anode electrode potential to 0:
    anode.X = 'Li[anode]:'+str(X_an)+', V[anode]:'+str(1-X_an)
    dG_an = anode_interface.delta_gibbs[0]
    n_charge = -1.
    E_eq_an = -dG_an/n_charge/ct.faraday

    cathode.X = 'Li[cathode]:'+str(1.-X_an)+', V[cathode]:'+str(X_an)
    dG_ca = cathode_interface.delta_gibbs[0]
    n_charge = 1.
    E_eq_ca = -dG_ca/n_charge/ct.faraday

    E_cell_therm[i] = E_eq_ca - E_eq_an

#Toc
t1 = time.time()
print('{:d} cell voltages calculated in {:3.2f} seconds.'.format(i,t1-t0))
```

    49 cell voltages calculated in 0.01 seconds.

### Plot thermodynamic OCV, and compare to results from kinetic method

```python
plt.figure()
plt.plot(100*X_Li_ca, E_cell_therm,color='b',linewidth=2.5)
plt.plot(100*X_Li_ca, E_cell_kin,linewidth=0.,marker='o',markerfacecolor='none',markeredgecolor='r')
plt.ylim([2.5,4.3])
plt.xlabel('Li Fraction in Cathode (%)',fontname='Times New Roman',fontsize=18)
plt.ylabel('Open Circuit Potential (V)',fontname='Times New Roman',fontsize=18)
plt.legend(['Thermodynamic','Kinetic'])

ax = plt.gca()

for tick in ax.xaxis.get_major_ticks():
    tick.label1.set_fontsize(14)
    tick.label1.set_fontname('Times New Roman')
for tick in ax.yaxis.get_major_ticks():
    tick.label1.set_fontsize(14)
    tick.label1.set_fontname('Times New Roman')

plt.show()
```

As one would expect, the two approaches give identical results.  While both methods are incredibly fast, the thermodynamic method is roughly 30 times faster.

A large part of this is that the thermodynamic approach is an analytical approach (i.e. the answer is known from theory), while the kinetic approach relies on the root-finding fzero method to fit the correct voltage.  Note also that the kinetic method, because of the use of Butler-Volmer kinetics, calculates the thermodynamic voltage, in order to calculate the overpotential $\eta = \Delta \Phi - \Delta \Phi_{\rm eq}$.

However, it is at last important to note that, while slower, the kinetic method is of course more robust, and can be used to find results away from equilibrium.  The thermodynamic method is only applicable at equilibrium (zero current).

```python

```


# 1D Alfven Wave `GiRaFFEfood` Initial Data for `GiRaFFE`

## This module provides another initial data option for `GiRaFFE`, drawn from [this paper](https://arxiv.org/abs/1310.3274) .

**Notebook Status:**  Self-Validated

**Validation Notes:** This tutorial notebook has been confirmed to be self-consistent with its corresponding NRPy+ module, as documented [below](#code_validation). **Additional validation tests may have been performed, but are as yet, undocumented. (TODO)**

### NRPy+ Source Code for this module: [GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests.py](../edit/GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests.py)

## Introduction:

### Alfvén Wave:

 This is a flat-spacetime test with initial data
\begin{align}
A_x &= 0 \\
A_y &= \left \{ \begin{array}{lll}\gamma_\mu x - 0.015 & \mbox{if} & x \leq -0.1/\gamma_\mu \\
1.15 \gamma_\mu x - 0.03g(x) & \mbox{if} & -0.1/\gamma_\mu \leq x \leq 0.1/\gamma_\mu \\
1.3 \gamma_\mu x - 0.015 & \mbox{if} & x \geq 0.1/\gamma_\mu \end{array} \right. , \\
 A_z = &\ y - \gamma_\mu (1-\mu)x ,
\end{align}
which generates the magnetic field in the wave frame,
\begin{align}
B'^{x'}(x') = &\ 1.0,\ B'^y(x') = 1.0, \\
B'^z(x') = &\ \left \{ \begin{array}{lll} 1.0 & \mbox{if} & x' \leq -0.1 \\
				1.0+0.15 f(x') & \mbox{if} & -0.1 \leq x' \leq 0.1 \\
				1.3 & \mbox{if} & x' \geq 0.1 \end{array} \right. .
\end{align}
The electric field in the wave frame is then given by
$$E'^{x'}(x') = -B'^z(0,x') \ \ , \ \ E'^y(x') = 0.0 \ \ , \ \ E'^z(x') = 1.0  .$$

These are converted to the grid frame by
\begin{align}
  B^x(0,x) = &\ B'^{x'}(\gamma_\mu x) , \\
  B^y(0,x) = &\ \gamma_\mu [ B'^y(\gamma_\mu x) - \mu E'^z(\gamma_\mu x) ] , \\
  B^z(0,x) = &\ \gamma_\mu [ B'^z(\gamma_\mu x) + \mu E'^y(\gamma_\mu x) ] ,
\end{align}
and
\begin{align}
  E^x(0,x) = &\ E'^{x'}(\gamma_\mu x) , \\
  E^y(0,x) = &\ \gamma_\mu [ E'^y(\gamma_\mu x) + \mu B'^z(\gamma_\mu x) ] ,\\
  E^z(0,x) = &\ \gamma_\mu [ E'^z(\gamma_\mu x) - \mu B'^y(\gamma_\mu x) ],
\end{align}
and the velocity is given by $$\mathbf{v} = \frac{\mathbf{E} \times \mathbf{B}}{B^2}$$ in flat spacetime. Additionally, $f(x)=1+\sin (5\pi x)$, $-1<\mu<1$ is the wave speed relative to the grid frame and $\gamma_\mu = (1-\mu^2)^{-1/2}$, and $g(x) = \cos (5\pi \gamma_\mu x)/\pi$.

For the eventual purpose of testing convergence, any quantity $Q$ evolves as $Q(t,x) = Q(0,x-\mu t)$

See the [Tutorial-GiRaFFEfood_NRPy](Tutorial-GiRaFFEfood_NRPy.ipynb) tutorial notebook for more general detail on how this is used.

# Table of Contents:
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Import core NRPy+ modules and set NRPy+ parameters
1. [Step 2](#vector_ak): Set the vector $A_k$
1. [Step 3](#vectors_for_velocity): Set the vectors $B^i$ and $E^i$ for the velocity
1. [Step 4](#vi): Calculate $v^i$
1. [Step 5](#code_validation): Code Validation against `GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests` NRPy+ module
1. [Step 6](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Import core NRPy+ modules and set NRPy+ parameters \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

Here, we will import the NRPy+ core modules and set the reference metric to Cartesian, set commonly used NRPy+ parameters, and set C parameters that will be set from outside the code eventually generated from these expressions. We will also set up a parameter to determine what initial data is set up, although it won't do much yet.

```python
# Step 0: Add NRPy's directory to the path
# https://stackoverflow.com/questions/16780014/import-file-from-parent-directory
import os,sys
nrpy_dir_path = os.path.join("..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

# Step 0.a: Import the NRPy+ core modules and set the reference metric to Cartesian
from outputC import *            # NRPy+: Core C code output module
import finite_difference as fin  # NRPy+: Finite difference C code generation module
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import grid as gri               # NRPy+: Functions having to do with numerical grids
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support

import reference_metric as rfm   # NRPy+: Reference metric support
par.set_parval_from_str("reference_metric::CoordSystem","Cartesian")
rfm.reference_metric()

# Step 1a: Set commonly used parameters.
thismodule = "GiRaFFEfood_NRPy_1D"
# Set the spatial dimension parameter to 3.
par.set_parval_from_str("grid::DIM", 3)
DIM = par.parval_from_str("grid::DIM")

```

#####

# Step 2: Set the vector $A_k$ \[Back to [top](#toc)\]
$$\label{vector_ak}$$

The vector potential is given as
\begin{align}
A_x &= 0 \\
A_y &= \left \{ \begin{array}{lll}\gamma_\mu x - 0.015 & \mbox{if} & x \leq -0.1/\gamma_\mu \\
1.15 \gamma_\mu x - 0.03g(x) & \mbox{if} & -0.1/\gamma_\mu \leq x \leq 0.1/\gamma_\mu \\
1.3 \gamma_\mu x - 0.015 & \mbox{if} & x \geq 0.1/\gamma_\mu \end{array} \right. , \\
A_z &= y - \gamma_\mu (1-\mu)x .
\end{align}

However, to take full advantage of NRPy+'s automated function generation capabilities, we want to write this without the `if` statements, replacing them with calls to `fabs()`. To do so, we will use rewrite these functions in terms of maxima and minima. Critically, we will define these functions in the following way:
\begin{align}
\min(a,b) &= \tfrac{1}{2} \left( a+b - \lvert a-b \rvert \right) \\
\max(a,b) &= \tfrac{1}{2} \left( a+b + \lvert a-b \rvert \right). \\
\end{align}
For real numbers, these operate exactly as expected. In the case $a>b$,
\begin{align}
\min(a,b) &= \tfrac{1}{2} \left( a+b - (a-b) \right) = b \\
\max(a,b) &= \tfrac{1}{2} \left( a+b + (a-b) \right) = a, \\
\end{align}
and in the case $a<b$, the reverse holds:
\begin{align}
\min(a,b) &= \tfrac{1}{2} \left( a+b - (b-a) \right) = a \\
\max(a,b) &= \tfrac{1}{2} \left( a+b + (b-a) \right) = b, \\
\end{align}

In code, we will represent this as:
```
min_noif(a,b) = sp.Rational(1,2)*(a+b-nrpyAbs(a-b))
max_noif(a,b) = sp.Rational(1,2)*(a+b+nrpyAbs(a-b))
```
Note the `_noif` suffix to avoid conflicts with other functions, and the use of `nrpyAbs()` function, which will always be interpreted as the C function `fabs()` (Sympy `Abs()` may get interpreted as $\sqrt{zz^*}$, for instance).

For convenience, we define two new positions, $x^+ = x+0.1/\gamma_\mu$ and $x^- = x-0.1/\gamma_\mu$. With these tools, we rewrite the $y$-component of the vector potential:
\begin{align}
A_y =& 1.15 \gamma_\mu x - 0.15 \gamma_\mu \left( \min(x^-,0)-\max(x^+,0) \right) \\
     &+ g(x) \frac{\max(x^-,0) \min(x^+,0)}{\max(x^-,\epsilon) \min(x^+,\epsilon)} \\\
     &- 0.015 \frac{\min(x^-,0) \max(x^+,0)}{\min(x^-,\epsilon) \max(x^+,\epsilon)}
\end{align}

First, however, we must set $$\gamma_\mu = (1-\mu^2)^{-1/2}$$ and $$g(x) = \cos (5\pi \gamma_\mu x)/\pi$$.

```python
mu_AW = par.Cparameters("REAL",thismodule,["mu_AW"], -0.5) # The wave speed
M_PI  = par.Cparameters("#define",thismodule,["M_PI"], "")

gammamu = 1/sp.sqrt(1-mu_AW**2)

# We'll use reference_metric.py to define x and y
x = rfm.xxCart[0]
y = rfm.xxCart[1]

g_AW = sp.cos(5*M_PI*gammamu*x)/M_PI
```

Now, we can define the vector potential. We will create three copies of this variable, because the potential is uniquely defined in three zones. Data for $x \leq -0.1/\gamma_\mu$ shall be referred to as "left", data for $-0.1/\gamma_\mu \leq x \leq 0.1/\gamma_\mu$ as "center", and data for $x \geq 0.1/\gamma_\mu$ as "right".

Starting on the left,
\begin{align}
A_x &= 0 \\
A_y &= \gamma_\mu x - 0.015 \\
A_z &= y - \gamma_\mu (1-\mu)x .
\end{align}

```python
AD = ixp.register_gridfunctions_for_single_rank1("EVOL","AD")
AleftD = ixp.zerorank1()

AleftD[0] = sp.sympify(0)
AleftD[1] = gammamu*x-0.015
AleftD[2] = y-gammamu*(1-mu_AW)*x
```

In the center,
\begin{align}
A_x &= 0 \\
A_y &= 1.15 \gamma_\mu x - 0.03g(x) \\
A_z &= y - \gamma_\mu (1-\mu)x .
\end{align}

```python
AcenterD = ixp.zerorank1()

AcenterD[0] = sp.sympify(0)
AcenterD[1] = 1.15*gammamu*x-0.03*g_AW
AcenterD[2] = y-gammamu*(1-mu_AW)*x
```

And on the right,
\begin{align}
A_x &= 0 \\
A_y &= 1.3 \gamma_\mu x - 0.015 \\
A_z &= y - \gamma_\mu (1-\mu)x .
\end{align}

```python
ArightD = ixp.zerorank1()

ArightD[0] = sp.sympify(0)
ArightD[1] = 1.3*gammamu*x-0.015
ArightD[2] = y-gammamu*(1-mu_AW)*x
```

# Step 3: Set the vectors $B^i$ and $E^i$ for the velocity \[Back to [top](#toc)\]
$$\label{vectors_for_velocity}$$

Now, we will set the magnetic and electric fields that we will need to define the initial velocities. First, we need to define $$f(x)=1+\sin (5\pi x);$$ note that in the definition of $B^i$, we need $f(x')$ where $x'=\gamma_\mu x$.

```python
xprime = gammamu*x
f_AW = 1.0 + sp.sin(5.0*M_PI*xprime)
print(f_AW)
```

    sin(5.0*M_PI*xx0/sqrt(1 - mu_AW**2)) + 1.0

We will now set the magnetic field in the wave frame:
\begin{align}
B'^{x'}(x') = &\ 1.0,\ B'^y(x') = 1.0, \\
B'^z(x') = &\ \left \{ \begin{array}{lll} 1.0 & \mbox{if} & x' \leq -0.1 \\
				1.0+0.15 f(x') & \mbox{if} & -0.1 \leq x' \leq 0.1 \\
				1.3 & \mbox{if} & x' \geq 0.1 \end{array} \right. .
\end{align}

```python
BleftpU = ixp.zerorank1()
BleftpU[0] = sp.sympify(1.0)
BleftpU[1] = sp.sympify(1.0)
BleftpU[2] = sp.sympify(1.0)

BcenterpU = ixp.zerorank1()
BcenterpU[0] = sp.sympify(1.0)
BcenterpU[1] = sp.sympify(1.0)
BcenterpU[2] = 1.0 + 0.15*f_AW

BrightpU = ixp.zerorank1()
BrightpU[0] = sp.sympify(1.0)
BrightpU[1] = sp.sympify(1.0)
BrightpU[2] = sp.sympify(1.3)

```

Now, we will set the electric field in the wave frame:
\begin{align}
E'^{x'}(x') &= -B'^z(0,x'), \\
E'^y(x') &= 0.0, \\
E'^z(x') &= 1.0  .
\end{align}

```python
EleftpU = ixp.zerorank1()
EleftpU[0] = -BleftpU[2]
EleftpU[1] = sp.sympify(0.0)
EleftpU[2] = sp.sympify(1.0)

EcenterpU = ixp.zerorank1()
EcenterpU[0] = -BcenterpU[2]
EcenterpU[1] = sp.sympify(0.0)
EcenterpU[2] = sp.sympify(1.0)

ErightpU = ixp.zerorank1()
ErightpU[0] = -BrightpU[2]
ErightpU[1] = sp.sympify(0.0)
ErightpU[2] = sp.sympify(1.0)

```

Next, we must transform the fields into the grid frame. We'll do the magnetic fields first.
\begin{align}
  B^x(0,x) = &\ B'^{x'}(\gamma_\mu x) , \\
  B^y(0,x) = &\ \gamma_\mu [ B'^y(\gamma_\mu x) - \mu E'^z(\gamma_\mu x) ] , \\
  B^z(0,x) = &\ \gamma_\mu [ B'^z(\gamma_\mu x) + \mu E'^y(\gamma_\mu x) ] ,
\end{align}

```python
BleftU = ixp.zerorank1()
BleftU[0] = BleftpU[0]
BleftU[1] = gammamu*(BleftpU[1]-mu_AW*EleftpU[2])
BleftU[2] = gammamu*(BleftpU[2]+mu_AW*EleftpU[1])

BcenterU = ixp.zerorank1()
BcenterU[0] = BcenterpU[0]
BcenterU[1] = gammamu*(BcenterpU[1]-mu_AW*EcenterpU[2])
BcenterU[2] = gammamu*(BcenterpU[2]+mu_AW*EcenterpU[1])

BrightU = ixp.zerorank1()
BrightU[0] = BrightpU[0]
BrightU[1] = gammamu*(BrightpU[1]-mu_AW*ErightpU[2])
BrightU[2] = gammamu*(BrightpU[2]+mu_AW*ErightpU[1])

```

And now the electric fields:
\begin{align}
  E^x(0,x) = &\ E'^{x'}(\gamma_\mu x) , \\
  E^y(0,x) = &\ \gamma_\mu [ E'^y(\gamma_\mu x) + \mu B'^z(\gamma_\mu x) ] ,\\
  E^z(0,x) = &\ \gamma_\mu [ E'^z(\gamma_\mu x) - \mu B'^y(\gamma_\mu x) ],
\end{align}

```python
EleftU = ixp.zerorank1()
EleftU[0] = EleftpU[0]
EleftU[1] = gammamu*(EleftpU[1]+mu_AW*BleftpU[2])
EleftU[2] = gammamu*(EleftpU[2]-mu_AW*BleftpU[1])

EcenterU = ixp.zerorank1()
EcenterU[0] = EcenterpU[0]
EcenterU[1] = gammamu*(EcenterpU[1]+mu_AW*BcenterpU[2])
EcenterU[2] = gammamu*(EcenterpU[2]-mu_AW*BcenterpU[1])

ErightU = ixp.zerorank1()
ErightU[0] = ErightpU[0]
ErightU[1] = gammamu*(ErightpU[1]+mu_AW*BrightpU[2])
ErightU[2] = gammamu*(ErightpU[2]-mu_AW*BrightpU[1])

```

# Step 4: Calculate $v^i$ \[Back to [top](#toc)\]
$$\label{vi}$$

Now, we calculate $$\mathbf{v} = \frac{\mathbf{E} \times \mathbf{B}}{B^2},$$ which is equivalent to $$v^i = [ijk] \frac{E^j B^k}{B^2},$$ where $[ijk]$ is the Levi-Civita symbol and $B^2 = \gamma_{ij} B^i B^j$ is a trivial dot product in flat space.

```python
import WeylScal4NRPy.WeylScalars_Cartesian as weyl
LeviCivitaSymbolDDD = weyl.define_LeviCivitaSymbol_rank3()

Bleft2 = BleftU[0]*BleftU[0] + BleftU[1]*BleftU[1] + BleftU[2]*BleftU[2]
Bcenter2 = BcenterU[0]*BcenterU[0] + BcenterU[1]*BcenterU[1] + BcenterU[2]*BcenterU[2]
Bright2 = BrightU[0]*BrightU[0] + BrightU[1]*BrightU[1] + BrightU[2]*BrightU[2]

ValenciavU = ixp.register_gridfunctions_for_single_rank1("AUX","ValenciavU")

ValenciavleftU = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            ValenciavleftU[i] += LeviCivitaSymbolDDD[i][j][k] * EleftU[j] * BleftU[k] / Bleft2

ValenciavcenterU = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            ValenciavcenterU[i] += LeviCivitaSymbolDDD[i][j][k] * EcenterU[j] * BcenterU[k] / Bcenter2

ValenciavrightU = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            ValenciavrightU[i] += LeviCivitaSymbolDDD[i][j][k] * ErightU[j] * BrightU[k] / Bright2
```

# Step 5: Code Validation against `GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests` NRPy+ module \[Back to [top](#toc)\]
$$\label{code_validation}$$

Here, as a code validation check, we verify agreement in the SymPy expressions for the `GiRaFFE` Aligned Rotator initial data equations  we intend to use between
1. this tutorial and
2. the NRPy+ [`GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests.py`](../edit/GiRaFFEfood_NRPy/GiRaFFEfood_NRPy_1D_tests.py) module.

```python
# Reset the list of gridfunctions, as registering a gridfunction
#   twice will spawn an error.
gri.glb_gridfcs_list = []

import GiRaFFEfood_NRPy.GiRaFFEfood_NRPy_1D_tests as gfho
gfho.GiRaFFEfood_NRPy_1D_tests()

print("Consistency check between GiRaFFEfood_NRPy tutorial and NRPy+ module: ALL SHOULD BE ZERO.")

for i in range(DIM):

    print("ValenciavleftU["+str(i)+"] - gfho.ValenciavleftU["+str(i)+"] = " + str(ValenciavleftU[i] - gfho.ValenciavleftU[i]))
    print("AleftD["+str(i)+"] - gfho.AleftD["+str(i)+"] = " + str(AleftD[i] - gfho.AleftD[i]))
    print("ValenciavcenterU["+str(i)+"] - gfho.ValenciavcenterU["+str(i)+"] = " + str(ValenciavcenterU[i] - gfho.ValenciavcenterU[i]))
    print("AcenterD["+str(i)+"] - gfho.AcenterD["+str(i)+"] = " + str(AcenterD[i] - gfho.AcenterD[i]))
    print("ValenciavrightU["+str(i)+"] - gfho.ValenciavrightU["+str(i)+"] = " + str(ValenciavrightU[i] - gfho.ValenciavrightU[i]))
    print("ArightD["+str(i)+"] - gfho.ArightD["+str(i)+"] = " + str(ArightD[i] - gfho.ArightD[i]))
```

    Consistency check between GiRaFFEfood_NRPy tutorial and NRPy+ module: ALL SHOULD BE ZERO.
    ValenciavleftU[0] - gfho.ValenciavleftU[0] = 0
    AleftD[0] - gfho.AleftD[0] = 0
    ValenciavcenterU[0] - gfho.ValenciavcenterU[0] = 0
    AcenterD[0] - gfho.AcenterD[0] = 0
    ValenciavrightU[0] - gfho.ValenciavrightU[0] = 0
    ArightD[0] - gfho.ArightD[0] = 0
    ValenciavleftU[1] - gfho.ValenciavleftU[1] = 0
    AleftD[1] - gfho.AleftD[1] = 0
    ValenciavcenterU[1] - gfho.ValenciavcenterU[1] = 0
    AcenterD[1] - gfho.AcenterD[1] = 0
    ValenciavrightU[1] - gfho.ValenciavrightU[1] = 0
    ArightD[1] - gfho.ArightD[1] = 0
    ValenciavleftU[2] - gfho.ValenciavleftU[2] = 0
    AleftD[2] - gfho.AleftD[2] = 0
    ValenciavcenterU[2] - gfho.ValenciavcenterU[2] = 0
    AcenterD[2] - gfho.AcenterD[2] = 0
    ValenciavrightU[2] - gfho.ValenciavrightU[2] = 0
    ArightD[2] - gfho.ArightD[2] = 0

# Step 6: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-GiRaFFEfood_NRPy_1D_tests.pdf](Tutorial-GiRaFFEfood_NRPy_1D_tests.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx --log-level='WARN' Tutorial-GiRaFFEfood_NRPy_1D_tests.ipynb
!pdflatex -interaction=batchmode Tutorial-GiRaFFEfood_NRPy_1D_tests.tex
!pdflatex -interaction=batchmode Tutorial-GiRaFFEfood_NRPy_1D_tests.tex
!pdflatex -interaction=batchmode Tutorial-GiRaFFEfood_NRPy_1D_tests.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode


```python
import itertools
import pathlib

import matplotlib.pyplot as plt
import pandas as pd
from myst_nb import glue

from src.commons import NUM_EXPERIMENTS
from src.decorators import repeat, get_from_cache_or_run
from src.utils import build_plots_dir_path, build_cache_dir_path
from src.visualization import PLOT_DPI

plt.ioff()  # turn off interactive plotting
plt.style.use('../../../src/phd.mplstyle')

root_dir = pathlib.Path().cwd().parent.parent.parent
cwd_dir = pathlib.Path().cwd()

plot_dir = build_plots_dir_path(root_dir) / cwd_dir.name
cache_dir = build_cache_dir_path(root_dir) / cwd_dir.name

glue('num_experiments', NUM_EXPERIMENTS, display=False)
```

# Statistical verification of results

In order to assess the significance and performance of obtained results, two statistical approaches were used. The majority of metrics were compared using the Bayesian estimation (BEST) method {cite}`kruschke2013bayesian`, while the other straightforward metrics were just averaged.

## Bayesian analysis
The Bayesian approach towards comparing data from multiple groups was used instead of traditional methods of _null hypothesis significance testing_ (NHST). They are more intuitive than the calculation and interpretation of _p-value_ scores, provides complete information about credible parameter values and allow more coherent inferences from data {cite}`dienes2011bayesian`.

Benavoli depicted the perils of the frequentist NHST approach when comparing machine learning classifiers in {cite}`benavoli2017time`, which is particularly suited for this work. He points out the following reasons against using the NHST methods:
- it does not estimate the probability of hypotheses,
- point-wise null hypotheses are practically always false,
- the _p-value_ does not separate between the effect size and the sample size,
- it ignores magnitude and uncertainty,
- it yields no information about the null hypothesis,
- there is no principled way to decide the $\alpha$ level.

Additionally, in 2016 the _[American Statistical Association](https://www.amstat.org/)_ made a statement against _p-values_ {cite}`wasserstein2016asa` which might be a motivation for other disciplines to pursue the Bayesian approach.

````{margin}
```{admonition} T-distribution
Like the normal distribution, the T-distribution is bell-shaped and symmetric, but it has heavier tails, which means it tends to produce values that fall far from its mean.

Tail heaviness is determined by a parameter called _degrees of freedom_ $\nu$ with smaller values giving heavier tails and higher values, making the T-distribution resembles a standard normal distribution with a mean of 0 and a standard deviation of 1.
```
````

In this work, we focus on establishing a descriptive mathematical model of the data $D$ using the Bayes' theorem deriving the posterior probability as a consequence of two antecedents: a prior probability and a "likelihood function" derived from a statistical model for the observed data {eq}`bayesian_approach`.

```{math}
:label: bayesian_approach
\underbrace{p(\mu, \sigma, \nu|D)}_{\text{posterior}} = \underbrace{p(D|\mu, \sigma, \nu)}_{\text{likelihood}} \times \underbrace{p(\mu, \sigma, \nu)}_{\text{prior}} \big/ \underbrace{p(D)}_{\text{evidence}}
```

Each experiment is performed {glue:}`num_experiments` times, generating independent samples, which according to the Central Limit Theorem, should be enough to consider it is approximating the normal distribution {cite}`kwak2017central` {cite}`islam2018sample`. To further provide a robust solution towards dealing with potential outliers, the _Student t-distribution_ is chosen. The prior distribution, described with three parameters - $\mu$ (expected mean value), $\sigma$ (standard deviation) and $\nu$ (degrees of freedom) is presented using the Equation {eq}`bayesian_prior`.  The standard deviation $\sigma$ parameter uniformly covers a vast possible parameter space. The degrees of freedom follows a shifted exponential distribution controlling the normality of the data. When $\nu>30$, the Student-t distribution is close to a normal distribution. However, if $\nu$ is small, Student t-distributions have a heavy tail. Therefore, value of $\nu \sim \mathrm{Exp}(\frac{1}{29})$ allows the model to be more tolerant for potential outliers.

```{math}
:label: bayesian_prior
\begin{align}
    \mu &\sim \mathrm{N}(\mu_D, \sigma^2_D) \\
    \sigma &\sim \mathrm{U}(\frac{1}{100}, 1000) \\
    \nu &\sim \mathrm{Exp}(\frac{1}{29})
\end{align}
```
The posterior distribution is approximated arbitrarily high accuracy by generating a large representative sample using _Markov chain Monte Carlo_ (MCMC) methods. Its sample provides thousands of combinations of parameter values $<\mu, \sigma, \nu>$. Each such combination of values is representative of credible parameter values that simultaneously accommodate the observed data and the prior distribution. From the MCMC sample, one can infer credible parameter values like the mean or standard deviation.

### Example
To show an example of this technique, the performance of an ACS2 algorithm operating in a multistep, toy-problem - Simple Maze environment {cite}`gerard2000yacs` (see Figure {numref}`{number} <simple-maze-env>`) will be discussed using three methods - the summary statistics, frequentist approach and the Bayesian estimation.

:::{figure-md} simple-maze-env

The Simple Maze environment is a _Partially Observable Markov Decision Problem_ where the agent is placed in the starting location (denoted as "S"), at the beginning of each new trial, and the goal is to reach the final state "F" by executing four possible actions – moving north, east, south or west. The bolded lines represent the walls, and the goal can be reached optimally in seven successive steps.
:::

```python
import gym
import lcs.agents.acs2 as acs2

SIMPLE_MAZE_TRIALS = 500
LEARNING_RATE = 0.1
DISCOUNT_FACTOR = 0.8

def simple_maze_env_provider():
    import gym_yacs_simple_maze  # noqa: F401
    return gym.make('SimpleMaze-v0')

def custom_metrics(agent, env):
    population = agent.population
    return {
        'pop': len(population),
        'reliable': len([cl for cl in population if cl.is_reliable()])
    }

@get_from_cache_or_run(cache_path=f'{cache_dir}/simple-maze/acs2.dill')
@repeat(num_times=NUM_EXPERIMENTS, use_ray=True)
def run_acs2(env_provider):
    simple_maze_env = env_provider()

    cfg = acs2.Configuration(
        classifier_length=4,
        number_of_possible_actions=4,
        beta=LEARNING_RATE,
        gamma=DISCOUNT_FACTOR,
        do_ga=False,
        metrics_trial_frequency=1,
        user_metrics_collector_fcn=custom_metrics)

    agent = acs2.ACS2(cfg)
    return agent.explore(simple_maze_env, SIMPLE_MAZE_TRIALS)

@get_from_cache_or_run(cache_path=f'{cache_dir}/simple-maze/acs2ga.dill')
@repeat(num_times=NUM_EXPERIMENTS, use_ray=True)
def run_acs2ga(env_provider):
    simple_maze_env = env_provider()

    cfg = acs2.Configuration(
        classifier_length=4,
        number_of_possible_actions=4,
        beta=LEARNING_RATE,
        gamma=DISCOUNT_FACTOR,
        do_ga=True,
        metrics_trial_frequency=1,
        user_metrics_collector_fcn=custom_metrics)

    agent = acs2.ACS2(cfg)
    return agent.explore(simple_maze_env, SIMPLE_MAZE_TRIALS)

# run computations
acs2_runs_metrics = run_acs2(simple_maze_env_provider)
acs2ga_runs_metrics = run_acs2ga(simple_maze_env_provider)

# build dataframes
acs2_metrics_df = pd.DataFrame(itertools.chain(*acs2_runs_metrics))
acs2ga_metrics_df = pd.DataFrame(itertools.chain(*acs2ga_runs_metrics))

# extract population size in last trial
acs2_population_counts = acs2_metrics_df.query(f'trial == {SIMPLE_MAZE_TRIALS - 1}')['reliable']
acs2ga_population_counts = acs2ga_metrics_df.query(f'trial == {SIMPLE_MAZE_TRIALS - 1}')['reliable']

# glue variables
glue('stats_simple_maze_trials', SIMPLE_MAZE_TRIALS, display=False)
```

The ACS2 agent will be tested in two variants - with and without the genetic algorithm modification. All other settings are identical. Both agents in each experiment will execute {glue:}`stats_simple_maze_trials` trials, each time randomly selecting an action. Finally, each such experiment will be independently repeated {glue:}`num_experiments` times.

````{admonition} Null hypothesis
The classifier population sizes obtained by ACS2 and ACS2 GA in the last trial $x$ are equal.
````

**Descriptive statistics**

The first option is to understand the data and extract basic summary statistics. Figure {numref}`{number} <summary-stats>` presents a whisker plot showing basic data aggregations like the minimum, first quartile, median, third quartile, maximum values alongside the histogram visualization. While the values look similar, it is still unclear whether they can be considered the same.

```python
def plot_summary_statistics(data, labels, plot_filename=None):
    fig = plt.figure(figsize=(14, 8))
    colors = ['lightblue', 'lightgreen']

    # boxplot
    ax1 = fig.add_subplot(121)

    bp = ax1.boxplot(data,
                     patch_artist=True,
                     labels=labels)

    for patch, color in zip(bp['boxes'], colors):
        patch.set_facecolor(color)

    ax1.set_title('Whisker plot', pad=20)
    ax1.set_ylabel('Observed population size values')
    ax1.grid(which='major', axis='y', linestyle='--')

    # histogram
    ax2 = fig.add_subplot(122)
    for d, l, c in zip(data, labels, colors):
        ax2.hist(d, bins=10, label=l, color=c, edgecolor='black', alpha=.8, linewidth=1)

    ax2.set_title('Histogram', pad=20)
    ax2.grid(which='major', axis='y', linestyle='--')
    ax2.legend()

    # common settings
    fig.suptitle('ACS2 and ACS2 GA Comparison', fontweight='bold')
    fig.subplots_adjust(top=0.85)

    if plot_filename:
        fig.savefig(plot_filename, dpi=PLOT_DPI)

    return fig

summary_plot_fig = plot_summary_statistics(
    [acs2_population_counts.to_numpy(), acs2ga_population_counts.to_numpy()],
    ['ACS2', 'ACS2 GA'],
    plot_filename=f'{plot_dir}/summary-stats.png')
```

:::{figure-md} summary-stats
:class: full-width

Descriptive statistics depicting population size obtained after executing two versions of the ACS2 agent in the Simple Maze environment.
:::

**Frequentist approach**

For the frequentist approach first two hypotheses about data distribution are formed:

```{math}
\begin{align*}
&H_0: \text{The classifier population sizes obtained by ACS2 and ACS2 GA are equal}\\
&H_1: \text{The classifier population sizes obtained by ACS2 and ACS2 GA are different}
\end{align*}
```

In traditional NHST workflow, the first step would be to apply normality tests to verify whether the data is normally distributed. However, looking at the histograms from the Figure {numref}`{number} <summary-stats>` we might assume that the data follow the Gaussian distribution and skip it.

If the $H_0$ hypothesis is rejected, there is no significant difference between the means. To do so, a _p-value_ will be calculated and compared with a certain threshold $\alpha \leq 0.05$. If the _p-value_ falls below the threshold, it means that the null hypothesis $H_0$ can be rejected and **there is 95% confidence that both means are significantly different**.

```python
from scipy.stats import ttest_ind

# don't assume equal variances
value, pvalue = ttest_ind(acs2_population_counts, acs2ga_population_counts, equal_var=False)

glue('stats-pvalue', round(pvalue, 2), display=False)
```

In this case, after running the two-sided t-Test, the calculated _p-value_ is {glue:}`stats-pvalue`, which indicates strong evidence for the $H_0$ hypothesis, meaning that it is retained. Remember that NHST does not accept the null hypothesis; it can be only rejected or failed to be rejected. Stating that it is correct implies 100% certainty, which is not valid in this methodology.

**Bayesian estimation**

Throughout this work a [PyMC3](https://docs.pymc.io/en/v3/) open-sourced Python framework is used for the probabilistic programming. Figure {numref}`{number} <stats_bayes_params_fig>` depicts two separate Student-t posterior distribution hyper-parameters estimated by using the MCMC method.

```python
from src.bayes_estimation import bayes_estimate

@get_from_cache_or_run(cache_path=f'{cache_dir}/simple-maze/bayes-acs2.dill')
def bayes_estimate_acs2(data):
    return bayes_estimate(data)

@get_from_cache_or_run(cache_path=f'{cache_dir}/simple-maze/bayes-acs2ga.dill')
def bayes_estimate_acs2ga(data):
    return bayes_estimate(data)

# run estimations
bayes_acs2_trace = bayes_estimate_acs2(acs2_population_counts.to_numpy())
bayes_acs2ga_trace = bayes_estimate_acs2ga(acs2ga_population_counts.to_numpy())

def plot_bayes_parameters(data, labels, plot_filename=None):
    colors = ['lightblue', 'lightgreen']
    hist_kwargs = {'bins': 20, 'edgecolor': 'black', 'alpha': .8, 'linewidth': 1}

    fig, axs = plt.subplots(2, 3, figsize=(16, 8))

    for i, (d, l, c) in enumerate(zip(data, labels, colors)):
        axs[i][0].hist(d['mu'], color=c, label=l, **hist_kwargs)
        axs[i][0].set_title(f'$\mu_{{{l}}}$')

        axs[i][1].hist(d['std'], color=c, label=l, **hist_kwargs)
        axs[i][1].set_title(f'$\sigma_{{{l}}}$')

        axs[i][2].hist(d['nu'], color=c, label=l, **hist_kwargs)
        axs[i][2].set_title(fr'$\nu_{{{l}}}$')

    # share axes
    axs[0][0].sharex(axs[1][0])
    axs[0][1].sharex(axs[1][1])
    axs[0][2].sharex(axs[1][2])

    # common
    handles, labels = [(a + b) for a, b in zip(axs[0][0].get_legend_handles_labels(), axs[1][0].get_legend_handles_labels())]
    fig.suptitle('Posterior distribution of ACS2 and ACS2 GA Student-t parameters', fontweight='bold')
    fig.legend(handles, labels, loc='lower center', ncol=2, bbox_to_anchor=(0.5, -0.1))
    fig.tight_layout()

    if plot_filename:
        fig.savefig(plot_filename, dpi=PLOT_DPI)

    return fig

bayes_parameters_fig = plot_bayes_parameters(
    [bayes_acs2_trace, bayes_acs2ga_trace],
    ['ACS2', 'ACS2 GA'],
    plot_filename=f'{plot_dir}/stats-bayes-parameters.png')

glue('stats_bayes_params_fig', bayes_parameters_fig, display=False)
```

```{glue:figure} stats_bayes_params_fig
:name: "stats_bayes_params_fig"
Parameter distributions of Student-t estimation of population size data for two agents.
```

````{margin}
```{admonition} Credible values
The credible values are defined by the _highest density interval_ **HDI**, which is a useful summary of where the bulk of the most credible values falls.

By definition, every value inside the HDI has a higher probability density than any value outside the HDI, and the total mass of points inside the 95%  HDI is 95% of the distribution.
```
````

The posterior distribution allows assessing the null value more credibly. Contrary to NHST, it can also accept $H_0$. To do so, a researcher defines a _region of practical equivalence_ (ROPE) {cite}`kruschke2011bayesian`, enclosing the values of the parameter that are deemed to be negligibly different from the null values for practical purposes. When nearly all the credible values fall within the ROPE, the null value is said to be accepted for practical purposes.

Figure {numref}`{number} <stats_bayes_delta_fig>` shows the graphical example of verifying the null hypothesis using the difference between two posterior distributions of population means $\mu_{ACS2}-\mu_{ACS2GA}$. The ROPE region (red sector) was assumed to be $[-1, 1]$, which means that the difference of one classifier is acceptable to consider two populations equal. Blue lines represent the area of credible values of the previously calculated difference. Since the credible value region does not fall into the ROPE region, the $H_0$ is rejected (while the NHST approach retained it).

```python
import arviz
import numpy as np
from scipy.stats import gaussian_kde

def plot_kde(data_diff, rope=None, hdi_prob=0.95, plot_filename=None):
    if rope is None:
        rope = [-1, 1]

    fig = plt.figure(figsize=(16, 8))
    ax = fig.add_subplot(111)

    x = np.linspace(min(data_diff), max(data_diff), 1000)
    gkde = gaussian_kde(data_diff)
    ax.plot(x, gkde(x), color='orange', label='$\mu_{ACS2}-\mu_{ACS2GA}$')
    ax.fill_between(x, gkde(x), alpha=0.1, color='orange')

    # no difference vertical line
    ax.axvline(0, color='black', alpha=0.5, linestyle='--')

    # high density interval
    hdi = arviz.hdi(data_diff, hdi_prob=0.95)
    ax.axvline(x=hdi[0], linestyle='--', color='blue', label=f'{int(hdi_prob * 100)}% HDI')
    ax.axvline(x=hdi[1], linestyle='--', color='blue')

    # ROPE
    ax.axvline(x=rope[0], linestyle='--', color='red', label='ROPE')
    ax.axvline(x=rope[1], linestyle='--', color='red')
    ax.fill_between(np.arange(rope[0], rope[1], 0.01), 0.5, color='red', alpha=0.05)

    ax.set_ylim(0, 0.5)
    fig.suptitle(r'Assessing $H_0$ using Bayes estimation', fontweight='bold')

    plt.legend()

    if plot_filename:
        fig.savefig(plot_filename, dpi=PLOT_DPI)

    return fig

data_diff = bayes_acs2_trace['mu'] - bayes_acs2ga_trace['mu']

bayes_delta_fig = plot_kde(data_diff, plot_filename=f'{plot_dir}/stats-bayes-delta.png')

glue('stats_bayes_delta_fig', bayes_delta_fig, display=False)
```

```{glue:figure} stats_bayes_delta_fig
:name: "stats_bayes_delta_fig"

Assessing $H_0$ using the probabilistic density function of $\mu$ parameter difference of two examined populations. The ROPE interval was chosen to $[-1, 1]$ meaning that a difference of one classifier can be neglected in terms of population equality. Even though the mean difference is less than one, the $H_0$ is rejected because the HDI region is not contained within the ROPE.
```


# Problem set 1: Financial Frictions, Liquidity and the Business Cycle.

This notebook sums up relevant information on the setup for the exercises. If further suggests what to think about, before going to exercise classes. The solution follows from "PS1.ipynb".

# Exercise 3.5 in JT (The Theory of Corporate Finance)

Consider the continuous the continuous investment model with decreasing returns to scale outlined in chapter 3 of JT. The core of the model is as follows:
- Let $I\in[0, \infty)$ be the level of investment in a given project.
- Entrepreneur proposes a project. If successful the project generates income $R(I)$; if not the project generates $0$.
- The probability of success depends on the behavior of the entrepreneur. If E behaves $(b)$ then probability of success is $p_H\in(0,1)$. If E does not behave $(nb)$ then the probability of success is $p_L$ where $0\leq p_L<p_H$.
- The entrepreneur has an incentive to not behave $(nb)$, as he receives $BI$ in private benefits in this case.
- The entrepreneur is endowed with A in assets.
- No investment project is profitable, if entrepreneur chooses not to behave.
- The entrepreneur's technology obeys the following conditions:
    1. Positive, but decreasing returns to investments: $R'(I)>0, R''(I)<0$.
    2. *Regularity condition 1:* Under perfect information a positive investment level would be optimal, i.e. $R'(0)>1/p_H$.
    3. *Regularity condition 2:* Under perfect information a finite level of investment is optiaml, i.e. $ lim_{I\rightarrow \infty} R'(I) < 1/p_H$.
- Assume perfect competition between lenders.
- We will consider loan agreements where the entrepreneur *pledges income* $R(I)-R_b(I)$, to the lender. This leaves the entrepreneur with $R_b(I)$ if the project is successful.

### Suggestions before exercise class:

1. Write up the utility for an entrepreneur, in the case where he behaves $(u_b)$ and in the case where he does not $(u_{nb})$.
2. Write up the *incentive compatibility constraint* (IC) stating what level of $R_b(I)$ is needed, for the entrepreneur to choose to behave.
3. Write up the *individual rationality constraint* (IR) for the lender, ensuring that he will agree to the loan contract.
4. What does the *perfect competition* amongst lenders imply, for the contract the entrepreneur will offer?
5. Given that lenders are profit maximizing, what does this imply for the level $R_b(I)$ in the loan contract? (Hint: Think about the (IC) constraint).

# Exercise 6.1 in JT: Privately known private benefit and market breakdown

Let us start with a brief outline of the setup (from section 6.2), compared to exercise 3.5 (a lot of is the same, will not be repeated here):

* Two types of entrepreneurs: Good and bad types with private benefits of not behaving $B_H>B_L$. (good type has $B_L$)
* No equity (A=0),
* Investment is not continuous, but either 0 or I.
* Investment is either successfull (return R) or not (return 0).
* Capital markets put probability $\alpha\in(0,1)$ on type 'good' and $1-\alpha$ on type 'bad'.
* Regularity conditions:

$$ \begin{align}
    p_H\left(R-\dfrac{B_H}{\Delta p}\right)<I<p_H\left(R-\dfrac{B_L}{\Delta p}\right), && \text{and} && p_LR<I.
\end{align} $$

Recall that the IC condition for an entrpreneur was **behave if**: $\Delta p R_b \geq B$.

The regularity conditions thus state that:
1. It is not profitable for lenders to invest in project with 'bad' entrepreneurs **and** making sure that they behave. (first inequality)
2. It is profitable to invest in 'good type' entrepreneur **and** making sure that he behaves (second inequality).
3. It is not profitable to invest in **any** project, where the entrepreneur mis-behaves (third inequality).

### Suggestions before exercise class:

1. Interpret the three regularity conditions (the three inequalities).
2. If the first inequality holds and $R_b=B_H\Delta p$, what will the two entrepreneurs do (behave/not behave)? When negotiating contract with lender, what will the two types have an incentive to reveal about their type?
3. Same scenario for the second inequality?

# Exercise 6.2 in JT: More on pooling in credit markets.

Compared to before, alter the setup as follows:

* Continuum of types instead of good/bad. For entrepreneur $i$ the value $B_i$ is distributed according to the CDF function $H(B)$, with support on $[0, \bar{B}]$
* Monopoly lender offers credit to borrowers (entrepreneurs).

The lender offers $R_b$ for a successfull investment, otherwise 0. Borrower $i$ then behaves **if** the IC constraint holds: $B_i\leq \Delta p R_b$. The expected profits from offering the contract is then defined by:

$$\begin{align}
    \pi(R_b) = H\big(\Delta p R_b\big)p_H(R-R_b)+\left[1-H\big(\Delta p R_b\big)\right] p_L(R-R_b)-I, \tag{Profits}
\end{align} $$

where $H(\Delta p R_b)$ measures the share of borrowers that behave, i.e. with $B_i<\Delta R_b$. From this note:

* The share of high-quality borrowers increase with $R_b$ (bad types start to behave).
* Same dynamics as before: Adverse selection reduces quality of lending, induces cross-subsidies between types.

### Suggestions before exercise class:

Make sure you understand the profit function. You can experiment with the profit function by changing the parameters in the cell below and run (shift+Enter) that cell along with the next one with the graphs.

```python
LowerBound = 0
UpperBound = 10
N = 100
x = np.linspace(LowerBound, UpperBound, N)
pH = 0.9
pL = 0.5
R = 10
I = 2
rv = stats.uniform(loc=LowerBound, scale=UpperBound-LowerBound)
Exp_profits = rv.cdf(x)*pH*(R-x)+(1-rv.cdf(x))*pL*(R-x)-I
def prof_func(y):
    return rv.cdf(y)*pH*(R-y)+(1-rv.cdf(y))*pL*(R-y)-I
# For what value of B does the profits cross zero?
Zero_profits = optimize.newton(prof_func, 0)
print(Zero_profits)
```

    7.5

```python
fig = plt.figure(frameon=False, figsize=(8, 6), dpi=100)
ax = fig.add_subplot(1, 1, 1)

ax.plot(x, rv.cdf(x))
ax.plot(x, rv.pdf(x))
ax.set_xlim([LowerBound, UpperBound])
ax.set_ylim([0, 1])

# Labels:
ax.set_xlabel('$B$')
#ax.set_ylabel('Share of entrepreneurs with $B_i\leq B$')
plt.legend(('Share of entrepreneurs with $B_i\leq B$', '$dH/dB_i$'),
           loc='upper left')

# Add a cool layout
fig.tight_layout()

fig2 = plt.figure(frameon=False, figsize=(8, 6), dpi=100)
ax2 = fig2.add_subplot(1, 1, 1)
ax2.plot(x,Exp_profits)
ax2.set_xlim([LowerBound, UpperBound])
ax2.set_ylim([math.floor(min(Exp_profits)), math.ceil(max(Exp_profits))])
ax2.set_xlabel('$B$')
ax2.set_ylabel('Expected profits')
plt.axvline(x=Zero_profits, color='k', linestyle='--')
plt.axhline(y=0, color='k')

plt.legend(('Expected profits' , 'Level of B implying zero profits' ),
          loc='lower left')
plt.text(Zero_profits+0.25, (math.ceil(max(Exp_profits))-math.floor(min(Exp_profits)))/2, 'Zero profits at B='+str(Zero_profits), bbox=dict(facecolor='darkblue', alpha=0.5))

fig2.tight_layout()
```

# Exercise 13.1 JT: Improved governance

The setup in this exercise:

* Two period model, $t=0,1$.
* Continuum of entrepreneurs with initial wealth $(A)$ distributed with the CDF $G(A)$ with support on $[0,I]$.
* At time $t=0$: Entrepreneurs can invest $I$ and borrow $(I-A)$ or invest in other firms.
* Entrepreneurs are risk neutral and thus have utility $u_e=c_0+c_1$.
* As before: Receive $R$ is investment is successful, with probability $p_H$ if entrepreneur behaves or $p_L$ if he shirks (misbehaves). Shirking yields private benefits of $B$.
* Now add consumers: Behavior results in an increasing savings function $S(r)$, with $S(r)=0$ for $r<0$.

### Suggestions before exercise class:

State (similarly to other exercises) (1) the (IC) constraint, (2) the profits when the (IC) constraint exactly holds and (3) the (IR) constraint for the lender.

# Exercise from PS: Adverse selection with heterogenous ability

## The setup:

* Continuum of risk-neutral entrepreneurs (same initial $A$)
* Investment cost $I$ (where $I>A$) yields $R$ in return w. probability $p$. Otherwise $0$ return.
* Agents differ in the probability of success $p$. Is distributed on $[0,1]$ according to the pdf function $f(p_i)$. (As usual $F(p)$ is the share of agents with $p_i\leq p$. $f(p)$ is the marginal change in that probability around $p_i=p$.)
* Total supply of loanable funds is simply presented by the increasing function $S(r)$.
* This $S(r)$ is supplied by a competetive market (e.g. banks).
* Assume perfect information: This means that loan contracts **can depend on the probability of sucess p**.
* Compared to before: No assumption of private benefits from not behaving.

### Suggestions before exercise class:

The same basic things:
1. What is the (IR) constraint of an entrepreneur that has the option of (1) entering into agreement recceiving $R-R_b(p)$ if the project suceeds (zero otherwise) or (2) depositing the wealth $A$ and getting a *normal return* of r?
2. What is the zero profits condition for a bank that receives the $pR_b(p)$ if the project is successful (0 otherwise), by investing $I-A$?