# Exercise session nº 6 - Part II
---
# Theory of epithelial sheet morphology in three dimensions

__*Sacha Ichbiah, 07/03/22, ENS Paris*__

This subject is extracted from :
> Edouard Hannezo et al., *Theory of epithelial sheet morphology in three dimensions*, PNAS, 2013. \
> https://doi.org/10.1073/pnas.1312076111

Morphogenesis during embryo development requires the coordination of mechanical forces to generate the macroscopic shapes of organs. To understand these processes, a full 3D description of the cells is needed. In this session, we will study a theoretical model, based on cell adhesion and actomyosin contractility, which describes the various shapes of epithelial cells and the bending and buckling of epithelial sheets, as well as the relative stability of cellular tubes and spheres. The morphologies observed in vivo can be understood as stable points of mechanical equations and the transitions between them are either continuous or discontinuous. From this model, will study the equilibrium configuration of the cells, and derive simple scaling laws, that can be verified experimentally.
The authors also studied the buckling instability of epithelia, driven by a flattening of individual cell shapes.

---

# I - Morphologies of planar epithelial cells
We model epithelial cells as hexagonal prisms of base length r and height h. We consider only cohesive sheets and, at first, we do not implement any constraint on the total area of the sheet. Considering all cells as identical, we write the force balance equation on a single cell, which we consider planar in this section.

Although the tissue is an out-of-equilibrium system, it is convenient to write the mechanical equilibrium at steady state as the minimization of a work function, or effective energy $\mathcal{F}$, where the nonequilibrium aspect is hidden in the tensions.  We define a cell as columnar when $\dfrac{h}{r}>>1$, cuboidal when $\dfrac{h}{r} \approx 1$ and squamous when $\dfrac{h}{r}<<1$.

Epithelial cells display an apico-basal polarity: the adhesion with the substrate defines the basal surface, whereas a contractile actomyosin belt often forms on the upper, apical surface.
We thus consider the following contributions to the effective energy:
- a cell–substrate energy, proportional to the basal area, $\gamma_b \mathcal{A}_{basal}$.
- a cell–cell lateral energy, proportional to the lateral area, $\alpha_l \mathcal{A}_{lat}$.
- an energy associated to the tension of the apical actomyosin belt $\Lambda_a$, proportional to the apical perimeter, $\Lambda_a \mathcal{P}_{ap}$.
- the confinement energy of the cytoplasmic components (nucleus, intermediate filanements..) : $A\left( \frac{1}{h^2} + 2 \frac{1}{r^2} \right)$ (confinement of Gaussian polymers to a thickness h or a thickness r, depending on the regime).

The lateral tension $\alpha_l$ is the sum of several contributions of opposite signs : a positive contribution from contractile forces of the actomyosin lateral cortex, which tends to minimize the lateral surface, and a negative contribution from the adhesion with the neighboring cells, which tends to maximize the lateral surface. The tensions could also have contributions from external stresses.

Thus the energy takes the following form :

$\mathcal{F} = \gamma_b \mathcal{A}_{basal} - \alpha_l \mathcal{A}_{lat} + \Lambda_a \mathcal{P}_{ap} + A\left( \frac{1}{h^2} + 2 \frac{1}{r^2} \right)$

The volume of each cell $V_0$ is conserved, leading to a geometrical relation between h and r.

#### **Question 0 :**
> Determine this geometrical relation $r = r(h,V_0)$ and express $\mathcal{A}_{basal}, \mathcal{A}_{lat}, \mathcal{P}_{ap}$ and eventually the energy $\mathcal{F}$ as a function of r

__Correction__

An hexagon is composed of six isosceles triangles, of height $\frac{r}{2}$ and of base length $d = 2\text{tan}(\frac{\pi}{6})\frac{r}{2} = \frac{r}{\sqrt{3}}$.

The area of each triangle is thus $\mathcal{A}_{triangle} = \frac{\frac{r}{2}d}{2}=\frac{r^2}{4\sqrt{3}}$, thus : $\mathcal{A}_{basal} = 6 \times \mathcal{A}_{triangle} = \frac{\sqrt{3}}{2} r^2 $

$V_0 = h \mathcal{A}_{basal} = \frac{\sqrt{3}}{2} r^2 h \Rightarrow h = \frac{2V_0}{\sqrt{3}r^2} $

$\mathcal{A}_{lat} = 6 \times dh = 6 \frac{r}{\sqrt{3}} h = 2 \sqrt{3} rh = 4 \frac{V_0}{r}$.

$\mathcal{P}_{ap} = 6 \times d = 2\sqrt{3} r $

Eventually :

$\mathcal{F} = \gamma_b \frac{\sqrt{3}}{2} r^2 - \alpha_l 4 \frac{V_0}{r} + 2 \sqrt{3} \Lambda_a r + A\left( \frac{3r^4}{4 V_0^2} + \frac{2}{r^2} \right)$

We can adimensionalize the previous energy by normalizing parameters, to obtain the following energy :

$\mathcal{F} = \gamma_b r^2 - \dfrac{\alpha_l}{r} + \Lambda_a r + r^4 + \dfrac{2}{r^2}$.

The mechanical equilibrium is attained when $\dfrac{\partial\mathcal{F}}{\partial r} = 0$

In our model, cell–cell lateral adhesion $\alpha_l>0$ and apical belt tension favor tall columnar cells, whereas cell–cell contractile forces $\alpha_l<0$ and cell–substrate adhesion $\gamma_b <0$ favor squamous cells, in agreement with the experimental observations that squamous cells down-regulate E-cadherin and Fas2/3 (cell–cell adhesion), whereas columnar cells up-regulate E-cadherin expression and downregulate cell–matrix adhesion.

#### **Question 1 :**
> Compute the energy derivative. Then, from scaling arguments, describe the aspect ratios encountered in the following situations :
> - $|\gamma_b|>>1, \gamma_b<0 $
> - $|\alpha_l|>>1, \alpha_l>0 $
> - $|\alpha_l|>>1, \alpha_l<0 $
> - $|\Lambda_a|>>1, \Lambda_a>0 $

__Correction__ :

$\dfrac{\partial\mathcal{F}}{\partial r} = 2 \gamma_b r + \dfrac{\alpha_l}{r^2} + \Lambda_a + 4r^3 - \dfrac{4}{r^3}$

####  $\bullet  \:  \boxed{ \:  |\gamma_b|>>1, \gamma_b<0 }$ :

$\mathcal{F} \approx \gamma_b r^2$, thus $r>>1$ (Squamous cells).

$\Rightarrow \dfrac{\partial\mathcal{F}}{\partial r} \approx 2 \gamma_b r + 4r^3 \Rightarrow r \propto -(\dfrac{\gamma_b}{2})^{1/2}$

####  $\bullet  \:  \boxed{ \:  |\alpha_l|>>1, \alpha_l>0 }$  :

$\mathcal{F} \approx -\dfrac{\alpha_l}{r}$, thus $r<<1$ (Columnar cells).

$\Rightarrow \dfrac{\partial\mathcal{F}}{\partial r} \approx \dfrac{\alpha_l}{r^2} - \dfrac{4}{r^3} \Rightarrow r \propto \dfrac{4}{\alpha_l}$

#### $\bullet  \:  \boxed{ \: |\alpha_l|>>1, \alpha_l<0 }$  :

$\mathcal{F} \approx -\dfrac{\alpha_l}{r}$, thus $r>>1$ (Squamous cells).

$\Rightarrow \dfrac{\partial\mathcal{F}}{\partial r} \approx \dfrac{\alpha_l}{r^2} + 4 r^3 \Rightarrow r \propto (-\dfrac{\alpha_l}{4})^{1/5}$

####  $\bullet  \:  \boxed{ \: |\Lambda_a|>>1, \Lambda_a>0}$  :

$\mathcal{F} \approx \Lambda_a r $, thus $r<<1$ (Columnar cells).

$\Rightarrow \dfrac{\partial\mathcal{F}}{\partial r} \approx \Lambda_a - \dfrac{4}{r^3} \Rightarrow r = (\dfrac{4}{\Lambda_a})^{1/3}$

#### **Question 2 :**
> Plot the Energy and its derivative for $\gamma_b = -15, \alpha_l \in \{4,-9\}$ and $\Lambda_a \in \{0,25,50\}$. Describe qualitatively the configurations encountered.

**Correction** :

```python
import numpy as np
import matplotlib.pyplot as plt

fig, axes = plt.subplots(4,3,figsize=(20,20))
r_values = np.linspace(0.1,4,100)
yb = -15
def F(r, alpha_l, lambda_a):
    return(yb*(r**2) - alpha_l/r + lambda_a*r + r**4 + 2/(r**2))

def dF(r, alpha_l, lambda_a):
    return(2*yb*r + alpha_l/(r**2) + lambda_a + 4*r**3 - 4/(r**3))

for i,lambda_a in enumerate([0,25,50]):
    for j,alpha_l in enumerate([4,-9]):
        func = lambda r : F(r, alpha_l, lambda_a)
        func_derivative = lambda r : dF(r, alpha_l, lambda_a)

        axes[2*j,i].plot(r_values,func(r_values))
        axes[2*j,i].set_title("Energy : Lambda_a="+str(lambda_a) + "  alpha_l="+str(alpha_l))

        axes[2*j+1,i].plot(r_values,func_derivative(r_values),color="green")
        axes[2*j+1,i].set_title("Derivative : Lambda_a="+str(lambda_a) + "  alpha_l="+str(alpha_l))
        axes[2*j+1,i].set_ylim(-5,5)
        axes[2*j+1,i].axhline(0,color = 'red')

axes[0,1].set_xlim(0,3)
axes[0,1].set_ylim(0,15)
axes[1,1].set_xlim(0,3)
```

```python
from mpl_toolkits.axes_grid1 import make_axes_locatable
from scipy.signal import argrelextrema

r_values = np.linspace(0.1,4,500)
l_values = np.linspace(0,50,500)

X, Y = np.meshgrid(r_values, l_values)

fig, axes = plt.subplots(1,2,figsize=(30,10))

for i, alpha_l in enumerate([4,-9]):
    Z = np.zeros((len(l_values),len(r_values)))
    for j,lambda_a in enumerate(l_values) :
        func = lambda r : F(r, alpha_l, lambda_a)
        array = func(r_values)
        array-= np.amin(array)

        args = argrelextrema(array, np.less)
        maximum = np.amax(array)

        for arg in args :
            if array[arg[0]]>0 :
                array[arg[0]]=np.amax(array)/2

        array[np.argmin(array)]=maximum

        Z[j]=(array.copy())/np.amax(array)

    mappable = axes[i].pcolormesh(r_values, l_values, Z,shading="auto")
    divider = make_axes_locatable(axes[i])
    cax = divider.append_axes('right', size='5%', pad=0.05)

    fig.colorbar(mappable, cax=cax, orientation='vertical')
    axes[i].set_title("Energy landscape, alpha_l={alpha_l:.1f}".format(alpha_l = alpha_l))
    axes[i].set_xlabel("radius")
    axes[i].set_ylabel("Lambda")

```

If contractile forces dominate $\alpha_l$, there is only one energy minimum to columnar aspect ratios. If lateral adhesion is large enough ($\alpha_l$ negative), two minima, cells can be in a metastable state and “jump” from squamous to columnar aspect ratios.

# Epithelial Sheet Bending - The buckling instability

We consider the confinement of an epithelium to an area different from its equilibrium value. If this occurs, cells can accommodate the decreased area by increasing their height. However, the area accessible to cells can also increase to its value dictated by mechanical equilibrium through an out-of-plane deformation of the entire sheet. This occurs if the energetic cost of bending the sheet is larger than the energetic cost of compression and is analogous to the Eulerian buckling of an elastic sheet under compression. For example, during Drosophila wing development, the columnar epithelium forms several folds. Moreover, local mutations affecting the actin belt cause the epithelium to collapse to a lower height and a bigger area and thus to form additional folds.

<sub> *Cell confinement and buckling. When a tissue is confined by external forces to an area lower than the area dictated by its mechanical equilibrium, it can either be homogeneously compressed or buckle to relieve the stress.*

We consider the case of columnar cells in the case $\alpha_l >> \Lambda_a >0 $ (Cell–cell adhesion is dominant). We thus have (cf Q.0) : $\alpha_l = \dfrac{4}{r_0}$.

In each case, confining a cell to a new base length r < r0 costs an effective energy that depends of the geometrical properties of the tissue. We will compute the energy needed for $\Delta r = r_0 - r << r_0$, in the case of confinement and buckling.

## Case I - Confinement

#### **Question 3 :**
> Compute the energy cost $\mathcal{F}_{confinement}$ of the confinement to a length $r$ (right case on the sketch).

**Correction** :

We compute the derivatives of the energy to do a Taylor expansion :

$\mathcal{F}(r) \approx -\dfrac{\alpha_l}{r} + \dfrac{2}{r^2} $

$\dfrac{\partial \mathcal{F}}{\partial r} \approx \dfrac{\alpha_l}{r^2} - \dfrac{4}{r^3}$

$\dfrac{\partial^2 \mathcal{F}}{\partial r^2} \approx -\dfrac{2 \alpha_l}{r^3} + \dfrac{12}{r^4}$

$\bullet$ As $\Delta r << r_0$ we can expand the energy around $r_0$:

$\mathcal{F}(r) = \mathcal{F}(r_0 - \Delta r) =  \mathcal{F}(r_0) - \Delta r \dfrac{\partial \mathcal{F}}{\partial r}(r_0) + \dfrac{1}{2}(\Delta r)^2 \dfrac{\partial^2 \mathcal{F}}{\partial r^2}(r_0) + o(\Delta r^2)$.

We know that $r_0$ is the length corresponding to an equilibrium configuration, thus $\dfrac{\partial \mathcal{F}}{\partial r}(r_0) = 0 $.

Thus :

$\begin{align}
\mathcal{F}(r) &=\mathcal{F}(r_0) +  \dfrac{1}{2}(\Delta r)^2 \dfrac{\partial^2 \mathcal{F}}{\partial r^2}(r_0) + o(\Delta r^2) \newline
&= \mathcal{F}(r_0) + \dfrac{1}{2}(\Delta r)^2 ( -\dfrac{2 \alpha_l}{r_0^3} + \dfrac{12}{r_0^4}) +  o(\Delta r^2) \newline
&= \mathcal{F}(r_0) + \dfrac{1}{2}(\Delta r)^2 ( \alpha_l^4 \dfrac{1}{4^3}) + o(\Delta r^2) \newline
\end{align}$

$\boxed{\mathcal{F}_{confinement}(\Delta r) = \dfrac{1}{128}(\Delta r)^2 \alpha_l^4 + o(\Delta r^2)}$

## Case II : Buckling

### Geometrical relationships

In a buckled sheet, parameterized by the definition $l(z) = u \cos(qz)$, cells are forced to adopt a lampshade shape that has an energetic cost, but the amplitude $u$ and inverse wavelength $q$ are such as to accommodate cells to their equilibrium wavelength. We have thus the following relationship :

$r_0 = r\left(1 + \frac{u^2q^2}{2} \right)$ for $uq<<r_0$.

#### **Question 4 :**
> Express u from this relationship

**Correction** :
$u = \frac{1}{q}\sqrt{\frac{2(r_0 - r)}{r}} = \frac{1}{q}\sqrt{\frac{2\Delta r}{r}} $

#### **Question 5 :**
> Compute the mean curvature of the sheet  $C_{sheet}$ from the expression of $l(z)$.

**Correction** :

The curvature of a parametric curve $l(z)=u\cos(qz)$ at one point is $\gamma(z) = ||l^{''}(z)||= uq^2 |\cos(qz)|$. \
Its mean curvature is $\overline{\gamma} = \frac{q}{2\pi} \int_0^{2\pi/q} \gamma(z) \mathrm{d}z = uq^2 \frac{4}{2\pi} \approx uq^2$

We define $\delta r$ such that the elongated base length of a cell is $r_0 +\delta r$ and the shortened base length of a cell is $r_0 -\delta r$. We assume that the spontaneous curvature of a cell  $C_{cell}$ is equal to the mean curvature of a sheet  $C_{sheet}$. The cell curvature $C_{cell}$ equals $r_0 \delta r$.

#### **Question 6 :**
> Express $\delta r$ in terms of $q,\alpha \text{ and } \Delta r$.

**Correction** :

$C_{cell} = C_{sheet} \Rightarrow r_ 0 \delta r = uq^2$,

Thus
$\delta _r = q \sqrt{\dfrac{2 \Delta r}{r r_0^2}} \approx q \sqrt{\dfrac{2 \Delta r}{r_0^3}} = q \sqrt{\dfrac{2 \alpha_l^3 \Delta r }{4^3}} =  q \sqrt{\dfrac{\alpha_l^3 \Delta r }{32}} $

### Energetic cost

A curved epithelium has two radii $r_1$ and $r_2$ instead of one, and the energy writes :

$\mathcal{F}_s = -\dfrac{\alpha_l}{2}\dfrac{r_1+r_2}{r_1^2 + r_2^2 + r_1r_2} + \dfrac{1}{9}(r_1^2 + r_2^2 + r_1r_2)^2 + \dfrac{2}{r_1r_2} + \gamma_b r_1^2 + \Lambda_a r_2$.

As we are in the columnar setting $\alpha_l >> \Lambda_a$, we can write the energy :

$\mathcal{F}_s(r_1,r_2) =  -\dfrac{\alpha_l}{2}\dfrac{r_1+r_2}{r_1^2 + r_2^2 + r_1r_2} + \dfrac{2}{r_1r_2}$

#### **Question 7 :**
> Compute the buckling energetic cost $\mathcal{F}_{buckling}(\Delta r)$.

**Correction** :

$\begin{align}
\mathcal{F}_s(\delta r) &\approx -\dfrac{3 \alpha_l}{2} \dfrac{r_1 + r_2}{r_1^2 + r_2^2 + r_1 r_ 2} + 2\dfrac{1}{r_1 r_2}
\newline
&=- 3\alpha_l\dfrac{r_0}{3r_0^2 + \delta r ^2} + \dfrac{2}{r_0^2 - \delta r^2}
\newline
&= -\dfrac{\alpha_l}{r_0} \dfrac{1}{1 + \frac{\delta r^2}{3 r_0 ^2}} + \dfrac{2}{r_0^2}\dfrac{1}{1 - \frac{\delta r^2}{r_0^2}} + o(\delta r^2)
\newline
&=-\dfrac{\alpha_l}{r_0} (1-\dfrac{\delta r^2}{3 r_0 ^2}) + \dfrac{\alpha_l}{2r_0} (1 + \dfrac{\delta r^2}{r_0^2})     + o(\delta r^2)
\newline
&= \mathcal{F}_s(0) + \alpha_l \left( \dfrac{\delta r^2}{3 r_0 ^3} +  \dfrac{\delta r^2}{2r_0^3} \right)    + o(\delta r^2)
\newline
&= \mathcal{F}_s(0) + \alpha_l \dfrac{5}{3} \dfrac{\delta r ^2}{2r_0^3}    + o(\delta r^2)
\newline
\end{align}
$

$\boxed{\mathcal{F}_{buckling}(\Delta r) = \dfrac{5}{3} \dfrac{q^2 \alpha_l^7 \Delta r }{4^6}+ o(\Delta r) }$

Therefore, the confinement energy depends quadratically on the confinement Δr, whereas the effective buckling energy is linear in Δr. This means that small confinements are always accommodated by a uniform squeezing of the sheet, until a critical threshold where the sheet buckles. Not surprisingly, large wavelengths (i.e $q \to 0$) are favored and reduce the buckling energy, because less bending is required of each individual cell. The preferred wavelength is then the length of the sheet L, with a critical confinement threshold

$\Delta r_c \propto \dfrac{\alpha_l^3}{L^2}$

The threshold increases with $\alpha_l$, which means that cell–cell adhesion increases the bending rigidity of a sheet.

# Conclusion

Taking into account the 3-dimensionnal geometry of the cells allows for a wide range of new behaviour in epithelial sheets, where out-of-plane effects appears. In recent years, experimentalists have demonstrated curling and buckling behaviours using in-vitro experiments, that can also be observed in the embryonic development. Such bending and folding of epithelial sheets seems to be key to create complex shapes from a single cell. The control and regulation of such mechanisms is an active topic of research, linking biology and mechanical modeling.

_Additional references :_
> Fouchard et al., *Curling of epithelial monolayers reveals coupling between active bending and tissue tension*, PNAS, 2020. https://doi.org/10.1073/pnas.1917838117

> C. Collinet and T. Lecuit, *Programmed and self-organized flow of information during morphogenesis* , Nature, 2021. https://doi.org/10.1038/s41580-020-00318-6


# Mutual Information Estimation

$\def\abs#1{\left\lvert #1 \right\rvert}
\def\Set#1{\left\{ #1 \right\}}
\def\mc#1{\mathcal{#1}}
\def\M#1{\boldsymbol{#1}}
\def\R#1{\mathsf{#1}}
\def\RM#1{\boldsymbol{\mathsf{#1}}}
\def\op#1{\operatorname{#1}}
\def\E{\op{E}}
\def\d{\mathrm{\mathstrut d}}$

```python
# init
import ipywidgets as widgets
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import tensorboard as tb
import torch
import torch.optim as optim
from IPython import display
from torch import Tensor, nn
from torch.nn import functional as F
from torch.utils.tensorboard import SummaryWriter

%load_ext tensorboard
%matplotlib inline

def plot_samples_with_kde(df, **kwargs):
    p = sns.PairGrid(df, **kwargs)
    p.map_lower(sns.scatterplot, s=2)  # scatter plot of samples
    p.map_upper(sns.kdeplot)  # kernel density estimate for pXY
    p.map_diag(sns.kdeplot)  # kde for pX and pY
    return p

SEED = 0

# set device
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

# create samples
XY_rng = np.random.default_rng(SEED)
rho = 1 - 0.19 * XY_rng.random()
mean, cov, n = [0, 0], [[1, rho], [rho, 1]], 1000
XY = XY_rng.multivariate_normal(mean, cov, n)

XY_ref_rng = np.random.default_rng(SEED)
cov_ref, n_ = [[1, 0], [0, 1]], n
XY_ref = XY_ref_rng.multivariate_normal(mean, cov_ref, n_)

Z = Tensor(XY).to(DEVICE)
Z_ref = Tensor(XY_ref).to(DEVICE)

ground_truth = -0.5 * np.log(1 - rho ** 2)
```

**How to estimate MI via KL divergence?**

In this notebook, we will introduce a few methods of estimating the mutual information ({prf:ref}`mi-estimation`) via KL divergence.

We first introduce the Mutual Information Neural Estimation (MINE) method in {cite}`belghazi2018mine`.

## MINE

The idea is to obtain MI {eq}`MI` from KL divergence {eq}`D` as follows:

$$
\begin{align*}
I(\R{X}\wedge \R{Y}) = D(\underbrace{P_{\R{X},\R{Y}}}_{P_{\R{Z}}}\| \underbrace{P_{\R{X}}\times P_{\R{Y}}}_{P_{\R{Z}'}}).
\end{align*}
$$

and then apply the DV formula {eq}`avg-DV` to estimate the divergence:

---

**Definition** MINE
:label: MINE

The idea of mutual information neural estimation (MINE) is to estimate $I(\R{X}\wedge\R{Y})$ by

$$
\begin{align}
\sup_{t_{\theta}: \mc{Z} \to \mathbb{R}} \overbrace{\frac1n \sum_{i\in [n]} t_{\theta}(\R{X}_i,\R{Y}_i) - \frac1{n'}\sum_{i\in [n']} e^{t_{\theta}(\R{X}'_i,\R{Y}'_i)}}^{\R{I}_{\text{MINE}}(\theta):=}
\end{align}
$$ (MINE)

where

- the supremum is over $t_{\theta}$ representable by a neural network with trainable/optimizable parameters $\theta$,
- $P_{\R{X}',\R{Y}'}:=P_{\R{X}}\times P_{\R{Y}}$, and
- $(\R{X}'_i,\R{Y}'_i\mid i\in [n'])$ is the sequence of i.i.d. samples of $P_{\R{X}',\R{Y}'}$.

---

The above is not a complete description of MINE because there are some implementation details yet to be filled in.

### Resampling

**How to obtain the reference samples ${\R{Z}'}^{n'}$, i.e., ${\R{X}'}^{n'}$ and ${\R{Y}'}^{n'}$?**

We can approximate the i.i.d. sampling of $P_{\R{X}}\times P_{\R{Y}}$ using samples from $P_{\R{X},\R{Y}}$ by a re-sampling trick:

$$
\begin{align}
P_{\R{Z}'^{n'}} &\approx P_{((\R{X}_{\R{J}_i},\R{Y}_{\R{K}_i})\mid i \in [n'])}
\end{align}
$$ (resample)

where $\R{J}_i$ and $\R{K}_i$ for $i\in [n']$ are independent and uniformly random indices

$$
P_{\R{J},\R{K}} = \op{Uniform}_{[n]\times [n]}
$$

and $[n]:=\Set{1,\dots,n}$.

MINE {cite}`belghazi2018mine` uses the following implementation that samples $(\R{J},\R{K})$ but without replacement. You can change $n'$ using the slider for `n_`.

```python
rng = np.random.default_rng(SEED)

def resample(data, size, replace=False):
    index = rng.choice(range(data.shape[0]), size=size, replace=replace)
    return data[index]

@widgets.interact(n_=widgets.IntSlider(n, 2, n, continuous_update=False))
def plot_resampled_data_without_replacement(n_):
    XY_ = np.block([resample(XY[:, [0]], n_), resample(XY[:, [1]], n_)])
    resampled_data = pd.DataFrame(XY_, columns=["X'", "Y'"])
    plot_samples_with_kde(resampled_data)
    plt.show()
```

In the above, the function `resample`
- uses `choice` to uniformly randomly select a choice
- from a `range` of integers going from `0` to
- the size of the first dimension of the `data`.

Note however that the sampling is without replacement.

**Is it a good idea to sample without replacement?**

**Exercise**

Sampling without replacement has an important restriction $n'\leq n$. Why?

YOUR ANSWER HERE

**Exercise** To allow $n>n'$, complete the following code to sample with replacement and observe what happens when $n \gg n'$.

```python
@widgets.interact(n_=widgets.IntSlider(n, 2, 10 * n, continuous_update=False))
def plot_resampled_data_with_replacement(n_):
    # YOUR CODE HERE
    raise NotImplementedError()
    resampled_data = pd.DataFrame(XY_, columns=["X'", "Y'"])
    plot_samples_with_kde(resampled_data)
    plt.show()
```

**Exercise**

Explain whether the resampling trick gives i.i.d. samples $(\R{X}_{\R{J}_i},\R{Y}_{\R{K}_i})$ for the cases with replacement and without replacement respectively?

YOUR ANSWER HERE

### Smoothing

To improve the stability of the training, MINE applies additional smoothing to the gradient calculation.

**How to train the neural network?**

{eq}`MINE` can be solved iteratively by minibatch gradient descent using the loss function:

$$
\begin{align}
\R{L}_{\text{MINE}}(\theta) &:= \overbrace{- \frac{1}{\abs{\R{B}}} \sum_{i\in \R{B}}  t_{\theta} (\R{X}_i, \R{Y}_i) }^{\R{L}(\theta):=} + \log \overbrace{\frac{1}{\abs{\R{B}'}} \sum_{i\in \R{B}'}  e^{t_{\theta} (\R{X}'_i, \R{Y}'_i)}}^{\R{L}'(\theta):=}
\end{align}
$$

where $\R{B}$ and $\R{B}'$ are batches of uniformly randomly chosen indices from $[n]$ and $[n']$ respectively.

The gradient expressed in terms of $\R{L}$ and $\R{L}'$ is:

$$
\begin{align}
\nabla \R{L}_{\text{MINE}}(\theta) &= \nabla \R{L}(\theta) + \frac{\nabla \R{L}'(\theta)}{\R{L}'(\theta)}.
\end{align}
$$

Variation in $\nabla \R{L}'(\theta)$ leads to the variation of the overall gradient especially when $\R{L}'(\theta)$ is small. With minibatch gradient descent, the sample average is over a small batch and so the variance can be quite large.

**How to reduce the variation in the gradient approximation?**

To alleviate the variation, MINE replaces the denominator $\R{L}'(\theta)$ by its moving average:

$$
\theta_{j+1} := \theta_j - s_j \underbrace{\left(\nabla \R{L}_j (\theta_j) + \frac{\nabla \R{L}'_j(\theta_j)}{\overline{\R{L}'}_j}\right)}_{\text{(*)}}
$$ (MINE:update)

where $\R{L}_j$ and $\R{L}'_j$ are the

$$
\overline{\R{L}'}_j =  \beta \overline{\R{L}'}_{j-1} + (1-\beta) \R{L}'(\theta_j)
$$ (MINE:L2bar)

for some smoothing factor $\beta\in [0,1]$.

**How to implement the smoothing?**

Instead of implementing a new optimizer, a simpler way is to redefine the loss at each step $i$ as follows {cite}`belghazi2018mine`:

$$
\R{L}_{\text{MINE},j}(\theta) = \R{L}_j(\theta) + \frac{\R{L}'_j(\theta)}{\overline{\R{L}'}_j}
$$ (MINE:mv)

where $\overline{\R{L}'}_j$ in {eq}`MINE:L2bar` is regarded as a constant in calculating the gradient. It is easy to verify that $\nabla \R{L}_{\text{MINE},j}(\theta_j)$ gives the desired gradient (*) in {eq}`MINE:update`.

In summary:

---

**Definition**

MINE estimates the mutual information $I(\R{X}\wedge\R{Y})$ as $\R{I}_{\text{MINE}}(\theta_j)$ {eq}`MINE` where $\theta_j$ is updated by descending along the gradient of $\R{L}_{\text{MINE},j}$ {eq}`MINE:mv` iteratively after $j$ steps.

---

### Implementation

Consider implementing MINE for the jointly gaussian $\R{X}$ and $\R{Y}$:

```python
SEED = 0

# set device
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

# create samples
XY_rng = np.random.default_rng(SEED)
rho = 1 - 0.19 * XY_rng.random()
mean, cov, n = [0, 0], [[1, rho], [rho, 1]], 1000
XY = XY_rng.multivariate_normal(mean, cov, n)

X = Tensor(XY[:, [0]]).to(DEVICE)
Y = Tensor(XY[:, [1]]).to(DEVICE)
ground_truth = -0.5 * np.log(1 - rho ** 2)
X.shape, Y.shape, ground_truth
```

The tensors `X` and `Y` correspond to samples of $\R{X}$ and $\R{Y}$ respectively. The first dimension indices the different samples. The `ground_truth` is the actual mutual information $I(\R{X}\wedge\R{Y})$.

**Exercise**

Complete the definition of `forward` so that the neural network $t_{\theta}$ is a vectorized function that takes samples `x` and `y` of $\R{X}$ and $\R{Y}$ as input and approximates the density ratio $\frac{P_{\R{X},\R{Y}}}{P_{\R{X}}\times P_{\R{Y}}}$ at `(x, y)`:

- Use `torch.cat` to concatenate tensors `x` and `y` in the last dimension.
- Use `F.elu` for the activation function $\sigma$.

```python
class Net(nn.Module):
    def __init__(self, input_size=2, hidden_size=100, sigma=0.02):
        super().__init__()
        # fully-connected (fc) layers
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)  # layer 2
        self.fc3 = nn.Linear(hidden_size, 1)  # layer 3
        nn.init.normal_(self.fc1.weight, std=sigma)  #
        nn.init.constant_(self.fc1.bias, 0)
        nn.init.normal_(self.fc2.weight, std=sigma)
        nn.init.constant_(self.fc2.bias, 0)
        nn.init.normal_(self.fc3.weight, std=sigma)
        nn.init.constant_(self.fc3.bias, 0)

    def forward(self, x, y):
        """
        Vectorized function that implements the neural network t(x,y).

        Parameters:
        -----------
        x, y: 2D Tensors where the first dimensions index different samples.
        """
        # YOUR CODE HERE
        raise NotImplementedError()
        return t

    def plot(self, xmin=-5, xmax=5, ymin=-5, ymax=5, xgrids=50, ygrids=50, ax=None):
        """Plot a heat map of a neural network net. net can only have two inputs."""
        x, y = np.mgrid[xmin : xmax : xgrids * 1j, ymin : ymax : ygrids * 1j]
        with torch.no_grad():
            z = (
                net(
                    Tensor(x.reshape(-1, 1)).to(DEVICE),
                    Tensor(y.reshape(-1, 1)).to(DEVICE),
                )
                .reshape(x.shape)
                .cpu()
            )
        if ax is None:
            ax = plt.gca()
        im = ax.pcolormesh(x, y, z, cmap="RdBu_r", shading="auto")
        ax.figure.colorbar(im)
        ax.set(xlabel=r"$x$", ylabel=r"$y$", title=r"Heatmap of $t(x,y)$")
```

To create and plot the neural network:

```python
torch.manual_seed(SEED)
net = Net().to(DEVICE)
net.plot()
```

**Exercise**

To implement a neural network trainer for MINE similar to `DVTrainer`, completes the definition of `loss` according to {eq}`MINE:mv`:

```python
class MINETrainer:
    def __init__(
        self, X, Y, net, n_iters_per_epoch, writer_params={}, m=1, beta=0.1, **kwargs
    ):
        """
        Neural estimator for Mutual Information based on MINE.

        Estimate I(X;Y) using samples of X and Y by training a network t to maximize
            avg(t(X,Y)) - avg(e^t(X',Y')) / m
        where samples of (X',Y') approximates P_X * P_Y using the resampling trick, and
        m is obtained by smoothing avg(e^t(X',Y')) with the factor beta.

        parameters:
        -----------

        X, Y : Tensors with first dimensions of the same size indicing the samples.
        net  : The neural network t that takes X and Y as input and output a real number for each a real number for each sample.
        n_iters_per_epoch : Number of iterations per epoch.
        m    : initial value for the moving average
        beta : Smoothing/forgetting factor between [0,1]
        writer_params     : Parameters to be passed to SummaryWriter for logging.
        """
        self.X = X
        self.Y = Y
        self.n = min(X.shape[0], Y.shape[0])
        self.beta = beta
        self.m = m
        self.net = net
        self.n_iters_per_epoch = n_iters_per_epoch

        # set optimizer
        self.optimizer = optim.Adam(self.net.parameters(), **kwargs)

        # logging
        self.writer = SummaryWriter(**writer_params)
        self.n_iter = self.n_epoch = 0

    def step(self, epochs=1):
        """
        Carries out the gradient descend for a number of epochs and returns the MI estimate evaluated over the entire data.

        Loss for each epoch is recorded into the log, but only one MI estimate is computed/logged using the entire dataset.
        Rerun the method to continue to train the neural network and log the results.

        Parameters:
        -----------
        epochs : number of epochs
        """
        for i in range(epochs):
            self.n_epoch += 1

            # random indices for selecting samples for all batches in one epoch
            idx = torch.randperm(self.n)

            # resampling to approximate the sampling of (X',Y')
            idx_X = torch.randperm(self.n)
            idx_Y = torch.randperm(self.n)

            for j in range(self.n_iters_per_epoch):  # loop through multiple batches
                self.n_iter += 1
                self.optimizer.zero_grad()

                # obtain a random batch of samples
                batch_X = self.X[idx[j : self.n : self.n_iters_per_epoch]]
                batch_Y = self.Y[idx[j : self.n : self.n_iters_per_epoch]]
                batch_X_ref = self.X[idx_X[j : self.n : self.n_iters_per_epoch]]
                batch_Y_ref = self.Y[idx_Y[j : self.n : self.n_iters_per_epoch]]

                # define the loss
                # YOUR CODE HERE
                raise NotImplementedError()
                loss.backward()
                self.optimizer.step()

            self.writer.add_scalar("Loss/train", loss.item(), global_step=self.n_iter)

        with torch.no_grad():
            idx_X = torch.randperm(self.n)
            idx_Y = torch.randperm(self.n)
            X_ref = self.X[idx_X]
            Y_ref = self.Y[idx_Y]
            estimate = (
                self.net(self.X, self.Y).mean()
                - self.net(X_ref, Y_ref).logsumexp(0)
                + np.log(self.n)
            ).item()
            self.writer.add_scalar("Estimate", estimate, global_step=self.n_epoch)
            return estimate
```

To create the trainer for MINE:

```python
trainer = MINETrainer(X, Y, net, n_iters_per_epoch=10)
```

To train the neural network:

```python
if input("Train? [Y/n]").lower() != "n":
    for i in range(10):
        print('MI estimate:', trainer.step(10))
    net.plot()
```

```python
%tensorboard --logdir=runs
```

**Exercise**

See if you can get an estimate close to the ground truth by training the neural network repeated as shown below.

To clean-up:

```python
if input('Delete logs? [y/N]').lower() == 'y':
    !rm -rf ./runs
```

```python
tb.notebook.list() # list all the running TensorBoard notebooks.
while (pid := input('pid to kill? (press enter to exit)')):
    !kill {pid}
```

## MI-NEE

**Is it possible to generate i.i.d. samples for ${\R{Z}'}^{n'}$?**

Consider another formula for mutual information:

---

**Proposition**
:label: MI-3D

$$
\begin{align}
I(\R{X}\wedge \R{Y}) &= D(P_{\R{X},\R{Y}}\|P_{\R{X}'}\times P_{\R{Y}'}) - D(P_{\R{X}}\|P_{\R{X}'}) - D(P_{\R{Y}}\|P_{\R{Y}'})
\end{align}
$$ (MI-3D)

for any product reference distribution $P_{\R{X}'}\times P_{\R{Y}'}$ for which the divergences are finite.

---

---

**Corollary**
:label: MI-ub

$$
\begin{align}
I(\R{X}\wedge \R{Y}) &= \inf_{\substack{P_{\R{X}'}\in \mc{P}(\mc{X})\\ P_{\R{Y}'}\in \mc{P}(\mc{Y})}} D(P_{\R{X},\R{Y}}\|P_{\R{X}'}\times P_{\R{Y}'}).
\end{align}
$$ (MI-ub)

where the optimal solution is $P_{\R{X}'}\times P_{\R{Y}'}=P_{\R{X}}\times P_{\R{Y}}$, the product of marginal distributions of $\R{X}$ and $\R{Y}$.

---

---

**Proof**

{eq}`MI-ub` follows from {eq}`MI-3D` directly since the divergences are non-negative. To prove the proposition:

$$
\begin{align}
I(\R{X}\wedge \R{Y}) &= H(\R{X}) + H(\R{Y}) - H(\R{X},\R{Y})\\
&= E\left[-\log dP_{\R{X}'}(\R{X})\right] - D(P_{\R{X}}\|P_{\R{X}'})\\
&\quad+E\left[-\log dP_{\R{Y}'}(\R{Y})\right] - D(P_{\R{Y}}\|P_{\R{Y}'})\\
&\quad-E\left[-\log d(P_{\R{X}'}\times P_{\R{Y}'})(\R{X},\R{Y})\right] + D(P_{\R{X},\R{Y}}\|P_{\R{X}'}\times P_{\R{Y}'})\\
&= D(P_{\R{X},\R{Y}}\|P_{\R{X}'}\times P_{\R{Y}'}) - D(P_{\R{X}}\|P_{\R{X}'}) - D(P_{\R{Y}}\|P_{\R{Y}'})
\end{align}
$$

---

*Mutual Information Neural Entropic Estimation (MI-NEE)* {cite}`chan2019neural` uses {eq}`MI-3D` to estimate MI by estimating the three divergences.

Applying {eq}`avg-DV` to each divergence in {eq}`MI-3D`:

$$
\begin{align}
I(\R{X}\wedge \R{Y}) &\approx \sup_{t: \mc{Z} \to \mathbb{R}} \frac1n \sum_{i\in [n]} t_{\R{X},\R{Y}}(\R{X}_i,\R{Y}_i) - \frac1{n'}\sum_{i\in [n']} e^{t_{\R{X},\R{Y}}(\R{X}'_i,\R{Y}'_i)}\\
&\quad - \sup_{t: \mc{Z} \to \mathbb{R}} \frac1n \sum_{i\in [n]} t_{\R{X}}(\R{X}_i) - \frac1{n'}\sum_{i\in [n']} e^{t_{\R{X}}(\R{X}'_i)} \\
&\quad - \sup_{t: \mc{Z} \to \mathbb{R}} \frac1n \sum_{i\in [n]} t_{\R{Y}}(\R{Y}_i) - \frac1{n'}\sum_{i\in [n']} e^{t_{\R{Y}}(\R{Y}'_i)}
\end{align}
$$ (MI-NEE)

$P_{\R{X}'}$ and $P_{\R{Y}'}$ are known distributions and so arbitrarily many i.i.d. samples can be drawn from them directly without using the resampling trick {eq}`resample`.

Indeed, since the choice of $P_{\R{X}'}$ and $P_{\R{Y}'}$ are arbitrary, we can also also train neural networks to optimize them. In particular, {eq}`MI-ub` is a special case where we can train neural networks to approximate $P_{\R{X}}$ and $P_{\R{Y}}$.


# PIT on simplified linear Nomoto for many runs

# Purpose
Run PIT on a series of tests the find the best average parameters

# Methodology
* Load time series an concatenate into a large y and X.
* Run OLS regression.

# Setup

```python
# %load imports.py
%matplotlib inline
%load_ext autoreload
%autoreload 2

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

```

## Load tests

```python
df_runs = mdl.runs()
```

```python
mask = ~df_runs['sailing']
df_tests = df_runs.loc[mask]
```

```python
df = pd.DataFrame()

for id, row in df_tests.iterrows():

    df_ = mdl.load_run(id=id, dir_path='../data/processed/kalman/')
    df_['time'] = df_.index
    df_['id'] = id
    df = df.append(df_, ignore_index=True, )

```

```python
px.line(df, x='y0', y='x0', width=1000, height=1000, line_group='id', color='id')
```

# First order Nomoto model for ship maneuvering dynamics
The first order Nomoto model can be written as:

```python
K, delta, T_1, T_2 = sp.symbols('K delta T_1 T_2')
r = dynamicsymbols('r')

eq_nomoto = sp.Eq(-K*delta,
                 r + T_1*r1d + T_2*r1d.diff())
Math(vlatex(eq_nomoto))
```

where $r$ is yaw rate with its time derivatives and $\delta$ is the rudder angle. $K$, $T_{1}$
 and $T_{1}$ are the coefficients describing the hydrodynamics of the ship.

For slow manoeuvres this equation can be further simplified by removing the $\ddot{r}$ term:

```python
eq_nomoto_simple = eq_nomoto.subs(r1d.diff(),0)
Math(vlatex(eq_nomoto_simple))
```

```python
eq_r1 = sp.Eq(r1d,
             sp.solve(eq_nomoto_simple, r1d)[0])
r1d_lambda = lambdify(sp.solve(eq_nomoto_simple, r1d)[0])
Math(vlatex(eq_r1))

```

```python
def nomoto(t,states,df_control,params):
    """
    Simulation model for heading and yaw rate using Nomoto with only K and T_1
    """
    # states:
    psi = states[0]
    r = states[1]

    index = df_control.index.get_loc(t, method='nearest')
    delta = float(df_control['delta'].iloc[index])
    r1d_ = r1d_lambda(K=params['K'], T_1=params['T_1'], delta=delta, r=r)
    d_states_dt = np.array([r,r1d_])
    return d_states_dt
```

## Regress nomoto parameters

```python
eq_r1
```

```python
eq_r1.rhs
```

```python
y = df['r1d']
X = df[['delta','r']]

#y = df_control['r1d_numerical']
#X = df_control[['delta','r_numerical']]
#X.rename(columns={'r_numerical': 'r'}, inplace=True)

model = sm.OLS(y,X)
results = model.fit()
results.summary()
```

```python
df_pred = df.copy()
df_pred['r1d_pred'] = results.predict(X)
prstd, iv_l, iv_u = wls_prediction_std(results, exog=X, alpha=0.10)
df_pred['iv_l'] = iv_l
df_pred['iv_u'] = iv_u

px.line(data_frame=df_pred.resample('1S').mean(), y=['r1d_pred','iv_l','iv_u'], width=1400, height=300,)

```

```python
params={}
params['T_1'] = -1/results.params['r']
params['K'] = -results.params['delta']*params['T_1']
```

```python
def simulate(df, params):

    states_0 = np.array([df.iloc[0]['psi'],df.iloc[0]['r']])

    df_sim = df.copy()

    t = df_sim.index

    sol = solve_ivp(fun = nomoto,t_span=[t[0],t[-1]],t_eval=t, y0 = states_0, args=(df_sim,params))
    df_sim['psi'] = sol.y[0,:]
    df_sim['r'] = sol.y[1,:]

    return df_sim

```

```python
def plot_sim(df, df_sim):

    fig,ax=plt.subplots()
    df['psi_deg'] = np.rad2deg(df['psi'])
    df.plot(y='psi_deg', label='model test', ax=ax)

    df['-delta_deg'] = -np.rad2deg(df['delta'])
    df.plot(y='-delta_deg', label=r'$-\delta$', ax=ax)

    df_sim['psi_deg'] = np.rad2deg(df_sim['psi'])
    df_sim.plot(y='psi_deg', label='simulation', ax=ax)

    ax.grid(True)
    ax.set_title('Resimulation with Nomoto model')
    ax.set_ylabel('$\psi$ [deg]')
    ax.set_xlabel('time [s]');
    ax.legend();
    return ax

```

```python
y_lim_min = (-10,10)

for id, test in df.groupby(by='id'):

    meta_data = df_tests.loc[id]

    test.index = pd.TimedeltaIndex(test['time']).total_seconds()

    df_sim = simulate(df=test, params=params)
    ax = plot_sim(df=test, df_sim=df_sim)
    ax.set_title(f'test : {id} {meta_data["test_type"]} {meta_data["comment"]}')

    y_lim = ax.get_ylim()
    ax.set_ylim(np.min([y_lim_min[0], y_lim[0]]),
                np.max([y_lim_min[1], y_lim[1]]))

```

```python

```


# Inference

## An inferential problem: The Gender Wage Gap

In the previous lab, we already analyzed data from the March Supplement of the U.S. Current Population Survey (2015) and answered the question how to use job-relevant characteristics, such as education and experience, to best predict wages. Now, we focus on the following inference question:

**What is the difference in predicted wages between men and women with the same job-relevant characteristics?**

Thus, we analyze if there is a difference in the payment of men and women (gender wage gap). The gender wage gap may partly reflect discrimination against women in the labor market or may partly reflect a selection effect, namely that women are relatively more likely to take on occupations that pay somewhat less (for example, school teaching).

To investigate the gender wage gap, we consider the following log-linear regression model

\begin{align} \log(Y) &= \beta'X + \epsilon\\ &= \beta_1 D + \beta_2' W + \epsilon, \end{align}
where $D$ is the indicator of being female ($1$ if female and $0$ otherwise) and the $W$'s are controls explaining variation in wages. Considering transformed wages by the logarithm, we are analyzing the relative difference in the payment of men and women.

---

```python
import numpy as np
import pandas as pd
import statsmodels.api as sm
import sklearn.linear_model as lm
import statsmodels.formula.api as smf
from sklearn.preprocessing import PolynomialFeatures
```

## Data Analysis

We consider the same subsample of the U.S. Current Population Survey (2015) as in the previous lab. Let us load the data set.

```python
file_dir = '/Users/jaumevives/Desktop/MIT 2/14.388/wage2015_subsample_inference.csv'
df = pd.read_csv(file_dir)
```

To start our (causal) analysis, we compare the sample means given gender:

```python
df.head()
```

```python
table = pd.DataFrame()

cols = ["lwage","sex","shs","hsg","scl","clg","ad","ne","mw","so","we","exp1"]

table['Variable'] = ["Log Wage","Sex","Less then High School","High School Graduate","Some College",
                      "Gollage Graduate","Advanced Degree", "Northeast","Midwest","South","West","Experience"]

table['All'] = df[cols].mean().values

table['Men'] = df[df['sex']==0][cols].mean().values
table['Women'] = df[df['sex']==1][cols].mean().values

# Show results
table
```

```python
df.describe()
```

```python
# print to Latex
table.to_latex()
```

In particular, the table above shows that the difference in average logwage between men and women is equal to  0,038.

```python
# compute difference
table.loc[0, 'Women'] - table.loc[0, 'Men']
```

Thus, the unconditional gender wage gap is about  $3,8\%$ for the group of never married workers (women get paid less on average in our sample). We also observe that never married working women are relatively more educated than working men and have lower working experience.

This unconditional (predictive) effect of gender equals the coefficient  $\beta$  in the univariate ols regression of  $Y$  on  $D$ :

$$
log(Y)=\beta D+\epsilon.
$$

We verify this by running an ols regression.

```python
nocontrol_fit = smf.ols("lwage ~ sex", data=df).fit()
nocontrol_est = nocontrol_fit.params['sex']
nocontrol_se = nocontrol_fit.HC0_se['sex']

print("The estimated gender coefficient is "\
      + str(nocontrol_est) + " and the corresponding robust standard error is " + str(nocontrol_se))
```

Note that the standard error is computed to be robust to heteroskedasticity.

Next, we run an ols regression of  $Y$  on  $(D,W)$  to control for the effect of covariates summarized in  $W$ :

$$
log(Y)=\beta_1 D + \beta_2' W + \epsilon.
$$

Here, we are considering the flexible model from the previous lab. Hence,  $W$ controls for experience, education, region, and occupation and industry indicators plus transformations and two-way interactions.

Let us run the ols regression with controls.

```python
# Ols regression with controls

flex = "lwage ~ sex + (exp1+exp2+exp3+exp4)*(shs+hsg+scl+clg+C(occ2)+C(ind2)+mw+so+we)"

#   Note that ()*() operation in formula objects in statsmodels creates a formula of the sort:
#  (exp1+exp2+exp3+exp4)+ (shs+hsg+scl+clg+occ2+ind2+mw+so+we) +  (exp1+exp2+exp3+exp4)*(shs+hsg+scl+clg+occ2+ind2+mw+so+we)
#  This is not intuitive at all, but that's what it does.

control_fit = smf.ols(flex, data=df).fit()
control_est = control_fit.params['sex']
control_se = control_fit.HC0_se['sex']

control_fit.summary()
```

```python
control_est
```

The estimated regression coefficient  $\beta_1≈-0.06955$  measures how our linear prediction of wage changes if we set the gender variable  $D$  from 0 to 1, holding the controls  $W$  fixed. We can call this the predictive effect (PE), as it measures the impact of a variable on the prediction we make. Overall, we see that the unconditional wage gap of size  4 \% for women increases to about  7 \% after controlling for worker characteristics.

Next, we are using the Frisch-Waugh-Lovell theorem from the lecture partialling-out the linear effect of the controls via ols.

```python
# Partialling-Out using ols

# models
flex_y = "lwage ~  (exp1+exp2+exp3+exp4)*(shs+hsg+scl+clg+C(occ2)+C(ind2)+mw+so+we)" # model for Y
flex_d = "sex ~ (exp1+exp2+exp3+exp4)*(shs+hsg+scl+clg+C(occ2)+C(ind2)+mw+so+we)"   # model for D

# partialling-out the linear effect of W from Y
t_Y = smf.ols(flex_y, data=df).fit().resid
# partialling-out the linear effect of W from D
t_D = smf.ols(flex_d, data=df).fit().resid

# regression of Y on D after partialling-out the effect of W
partial_fit = sm.OLS(t_Y, t_D).fit()
partial_est = partial_fit.params['x1']

print("Coefficient for D via partialling-out " + str(partial_est))

# standard error
partial_se = partial_fit.HC0_se['x1']

# confidence interval
print("2.5 - 97.5 % CI: " + str(partial_fit.conf_int().values[0]))
```

Again, the estimated coefficient measures the linear predictive effect (PE) of  $D$  on  $Y$  after taking out the linear effect of  $W$  on both of these variables. This coefficient equals the estimated coefficient from the ols regression with controls.

We know that the partialling-out approach works well when the dimension of $W$ is low in relation to the sample size  $n$ . When the dimension of  $W$  is relatively high, we need to use variable selection or penalization for regularization purposes.

In the following, we illustrate the partialling-out approach using lasso instead of ols.

```python
# Partialling-Out using lasso

# models
X_d =  smf.ols(flex_d, data=df).data.exog
y_d = smf.ols(flex_d, data=df).data.endog

X_y =  smf.ols(flex_y, data=df).data.exog
y_y = smf.ols(flex_y, data=df).data.endog

# partialling-out the linear effect of W from Y
t_Y_lasso = y_y - lm.LassoCV(cv=10, fit_intercept=False,
                      alphas=np.logspace(-6,2, 100)).fit(X_y,y_y).predict(X_y)

# partialling-out the linear effect of W from D
t_D_lasso = y_d - lm.LassoCV(cv=10, fit_intercept=False,
                      alphas=np.logspace(-6,2, 100)).fit(X_d,y_d).predict(X_d)

# regression of Y on D after partialling-out the effect of W
partial_lasso_fit = sm.OLS(t_Y_lasso, t_D_lasso).fit()
partial_lasso_est = partial_lasso_fit.params[0]

print("Coefficient for D via partialling-out using lasso " + str(partial_lasso_est))

# standard error
partial_lasso_se = partial_lasso_fit.HC0_se[0]

# confidence interval
print("2.5 - 97.5 % CI: " + str(partial_lasso_fit.conf_int()[0]))
```

Using lasso for partialling-out here provides similar results as using ols.

```python
table2 = pd.DataFrame()

table2['Model'] = ["Without controls", "full reg", "partial reg", "partial reg via lasso"]

table2['Estimate'] = [nocontrol_est, control_est, partial_est, partial_lasso_est]

table2['Std. Error'] = [nocontrol_se, control_se, partial_se, partial_lasso_se]

# Show results
table2
```

```python
# print to Latex
table2.to_latex()
```

It it worth to notice that controlling for worker characteristics increases the gender wage gap from less that 4\% to 7\%. The controls we used in our analysis include 5 educational attainment indicators (less than high school graduates, high school graduates, some college, college graduate, and advanced degree), 4 region indicators (midwest, south, west, and northeast); a quartic term (first, second, third, and fourth power) in experience and 22 occupation and 23 industry indicators.

Keep in mind that the predictive effect (PE) does not only measures discrimination (causal effect of being female), it also may reflect selection effects of unobserved differences in covariates between men and women in our sample.

Next we try "extra" flexible model, where we take interactions of all controls, giving us about 1000 controls.

```python
extraflex = "lwage ~ sex + (exp1+exp2+exp3+exp4+shs+hsg+scl+clg+C(occ2)+C(ind2)+mw+so+we)**2"

covs = ["exp1", "exp2", "exp3", "exp4", "shs", "hsg", "scl", "clg", "occ2", "ind2", "mw", "so", "we"]

# we use sklearn polynomial features
poly = PolynomialFeatures(interaction_only=True, degree=4)
#X_poly = poly.fit_transform(df[covs].values)
#X = np.concatenate([df[['sex']].values, X_poly], axis=1)

X = smf.ols(extraflex, data=df).data.exog
y = smf.ols(extraflex, data=df).data.endog
```

```python
# model
extraflex = "lwage ~ sex + (exp1+exp2+exp3+exp4+shs+hsg+scl+clg+C(occ2)+C(ind2)+mw+so+we)**2"

# covs = ["exp1", "exp2", "exp3", "exp4", "shs", "hsg", "scl", "clg", "occ2", "ind2", "mw", "so", "we"]
# alternatively e use sklearn polynomial features
# poly = PolynomialFeatures(interaction_only=True, degree=4)
# X_poly = poly.fit_transform(df[covs].values)
# X = np.concatenate([df[['sex']].values, X_poly], axis=1)

control_fit_extra = smf.ols(extraflex, data=df).fit()
control_extra_est = control_fit_extra.params['sex']

n = df.shape[0]
p = control_fit_extra.params.shape[0]

control_extra_se = control_fit_extra.HC0_se[0]*np.sqrt(n/(n-p))
# crude adjustment for the effect of dimensionality on OLS standard errors, motivated by Cattaneo, Jannson, and Newey (2018)
# for really correct way of doing this, we need to implement Cattaneo, Jannson, and Newey (2018)'s procedure.

print("Number of Extra-Flex Controls " + str( p- 1))

print("Coefficient for OLS with extra flex controls " +  str(control_extra_est))
```

```python
# Design matrices
X = smf.ols(extraflex, data=df).data.exog
y = smf.ols(extraflex, data=df).data.endog

# partialling-out the linear effect of W from Y
t_Y_lasso_extra = y_y - lm.LassoCV(cv=10, fit_intercept=False,
                      alphas=np.logspace(-6,2, 100)).fit(X,y).predict(X)

# partialling-out the linear effect of W from D
t_D_lasso_extra = y_d - lm.LassoCV(cv=10, fit_intercept=False,
                      alphas=np.logspace(-6,2, 100)).fit(X,y).predict(X)

# regression of Y on D after partialling-out the effect of W
partial_lasso_fit_extra = sm.OLS(t_Y_lasso_extra, t_D_lasso_extra).fit()
partial_lasso_est_extra = partial_lasso_fit.params[0]
partial_lasso_se_extra = partial_lasso_fit_extra.HC0_se[0]

print("Coefficient for D via partialling-out using lasso " + str(partial_lasso_est_extra))

```

```python
table3 = pd.DataFrame()

table3['Model'] = ["full reg", "partial reg via lasso"]

table3['Estimate'] = [control_extra_est, partial_lasso_est_extra]

table3['Std. Error'] = [control_extra_se, partial_lasso_se_extra]

# Show results
table3
```

```python
# print to Latex
table3.to_latex()
```

In this case $p/n = 20\%$, that is  $p/n$  is no longer small and we start seeing the differences between unregularized partialling out and regularized partialling out with lasso (double lasso). The results based on double lasso have rigorous guarantees in this non-small $p/n$ regime under approximate sparsity. The results based on OLS still have guarantees in $p/n< 1$ regime under assumptions laid out in Cattaneo, Newey, and Jansson (2018), without approximate sparsity, although other regularity conditions are needed.


# Implementing a basic neural network
By Brett Naul (UC Berkeley)

In this exercise we'll implement and train a simple single-layer neural network classifier using `numpy`. First, let's create some example data: we'll sample points from along interlocking spirals like we saw in the [TensorFlow playground](http://playground.tensorflow.org/#dataset=spiral).

*Based on Stanford CS 231n "Neural Network Case Study" exercise.*

```python
# Imports / plotting configuration
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_context('poster')
plt.rcParams['image.interpolation'] = 'nearest'  # hard classification boundaries
plt.rcParams['image.cmap'] = 'viridis'
np.random.seed(13)
```

```python
# Generate spiral sample data
def spiral_data(N, K=3, sigma=0.1):
    X = np.zeros((N * K, 2))
    y = np.zeros(N * K, dtype='int')

    for j in range(K):
        ix = range(N * j, N * (j + 1))
        r = np.linspace(0.0, 1, N)  # radius
        theta = 2 * np.pi * j / K + np.linspace(0, 3 * np.pi, N) + np.random.randn(N) * sigma
        X[ix] = np.c_[r * np.sin(theta), r * np.cos(theta)]
        y[ix] = j

    return X, y
```

```python
N = 100
K = 3
X, y = spiral_data(N, K, 0.1)

# Visualize the generated data
fig = plt.figure()
plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap='viridis')
plt.xlim([-1, 1])
plt.ylim([-1, 1]);
```

## Setup
Our neural network will take an $\mathbf{x} = (x_1, x_2)$ vector as input and output a $K$-dimensional vector $\mathbf{p}=(p_1,\dots,p_K)$ of class probabilities. For simplicity we'll focus on a single choice of activation function, the ReLU function $f(x) = \max(x, 0)$.

We'll follow the `scikit-learn` model API and construct a simple model with `fit` and `predict` methods. The exercises below will step you through the necessary steps to implement a fully-functional neural network classifier.

```python
from sklearn.base import BaseEstimator, ClassifierMixin

class SingleLayerReLU(BaseEstimator, ClassifierMixin):
    """Skeleton code for single-layer multi-class neural network classifier w/ ReLU activation.
    NOTE: Whenever you change the code below, you need to re-run this cell AND re-initialize
    your model (`model = SingleLayerNet(...)`) in order to update your specific `model` object.
    """
    def __init__(self, hidden_size, num_classes, sigma_init=0.01):
        """Initialize weights with Gaussian noise scaled by `sigma_init` and
        biases with zeros.
        """
        self.hidden_size = hidden_size
        self.num_classes = num_classes
        self.W1 = sigma_init * np.random.randn(hidden_size, 2)
        self.W2 = sigma_init * np.random.randn(num_classes, hidden_size)
        self.b1 = np.zeros(hidden_size)
        self.b2 = np.zeros(num_classes)

    def loss(self, y, P):
        """Compute total softmax loss.
        Inputs:  y -> (N,) array of true (integer) labels
                 p -> (N, K) array of predicted probabilities
        Outputs: L -> total loss value
        """
        return -np.sum(np.log(P[range(len(P)), y]))

    def dloss(self, X, y):
        """Compute gradient of softmax loss with respect to network weights.
        Note: this function takes the entire dataset as input, not a single example.
        Inputs:  X -> (N, 2) array of network inputs
                 y -> (N,) array of true labels
        Outputs: dW1 -> (hidden_size, 2) array of weight derivatives
                 dW2 -> (hidden_size, 2) array of weight derivatives
                 db1 -> (hidden_size,) array of bias derivatives
                 db2 -> (hidden_size,) array of bias derivatives
        """
        H = np.maximum(0, X @ self.W1.T + self.b1)  # ReLU activation
        Z = H @ self.W2.T + self.b2
        P = np.exp(Z) / np.sum(np.exp(Z), axis=1, keepdims=True)

        dZ = P
        dZ[range(len(X)), y] -= 1

        dW2 = (H.T @ dZ).T
        db2 = np.sum(dZ, axis=0)

        dH = dZ @ self.W2
        dH[H <= 0] = 0  # backprop ReLU  activation

        dW1 = (X.T @ dH).T
        db1 = np.sum(dH, axis=0)

        return (dW1, dW2, db1, db2)

    def predict_proba(self, X):
        """Compute forward pass for all input values.
        Inputs:  X -> (N, 2) array of network inputs
        Outputs: P -> (N, K) array of class probabilities
        """
        pass

    def predict(self, X):
        """Compute most likely class labels for all input values.
        Inputs:  X -> (N, 2) array of network inputs
        Outputs: P -> (N, K) array of class probabilities
        """
        pass

    def fit(self, X, y, step_size=3e-3, n_iter=10000):
        """Optimize model parameters W1, W2, b1, b2 via gradient descent.
        Inputs:  X -> (N, 2) array of network inputs
                 y -> (N,) array of true labels
                 step_size -> gradient descent step size
                 n_iter -> number of gradient descent steps to perform
        Outputs: losses -> (n_iter,) array of loss values after each step
        """
        pass
```

## Part 1: Forward pass
The "forward pass" step (i.e., computing the output for a given input) can be written as
$$
\begin{align}
\mathbf{h} &= f(W_1 \mathbf{x} + \mathbf{b_1}) \\
\mathbf{z} &= W_2 \mathbf{h} + \mathbf{b_2} \\
\mathbf{p} &= \operatorname{softmax}(\mathbf{z})
\end{align}
$$
where $f(\cdot)$ is an activation function, $W_1$ and $W_2$ are weight matrices, and $\mathbf{b}_1$ and $\mathbf{b}_2$ are bias vectors. The softmax function is given by
$$
\operatorname{softmax}(\mathbf{z}) = \exp\left(\mathbf{z}\right) / \left(\sum_{k=1}^K \exp\left( z_k\right)\right)
$$
and the ReLU activation is
$$
f(x) = \max(x, 0).
$$

### 1a) Implement `predict` function
Your function should loop over the input points (i.e., the rows of $X$) and compute the estimated class probabilities for each. The model parameters $W_1, W_2, b_1, b_2$ are treated as fixed.

### 1b) Examine predictions
The function `visualize_predictions` below can be used to inspect the decision boundaries for a given model, dataset, and class labels. Initialize a network and see what the classifications look like. Try varying the number of hidden units and see how the decision boundary shapes change.

```python
def visualize_predictions(model, X, y, step=0.02):
    x_min, x_max = X[:, 0].min(), X[:, 0].max()
    y_min, y_max = X[:, 1].min(), X[:, 1].max()
    xx, yy = np.meshgrid(np.arange(x_min, x_max, step), np.arange(y_min, y_max, step))
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    plt.contourf(xx, yy, Z, alpha=0.8)
    plt.scatter(X[:, 0], X[:, 1], c=y, s=40)
    plt.xlim(xx.min(), xx.max())
    plt.ylim(yy.min(), yy.max())
    plt.title("Accuracy: {}%".format(100 * np.mean(y == model.predict(X))))

# Test `predict` function for part 1b:
pass
```

## Part 2: Training
Obviously generating classifications at random isn't the way to go. Now we'll use gradient descent to iteratively improve the weights of our network and hopefully improve our classification accuracy.

If $y$ is some true label and $\mathbf{p}$ are the estimated class probabilities from our network, then our loss function for a single training example can be written as
$$L(y, \mathbf{p}) = -\log p_y,$$
that is, the negative log of the predicted probability for the true class: 0 if our predicted probability was 1, and $+\infty$ if it was 0.
In order to perform gradient descent, we'll need the derivatives of the loss with respect to all of our model parameters*. We can compute these derivatives via the backpropagation (effectively just the chain rule)**:
$$
\begin{aligned}
\frac{\partial L}{\partial z_k}   &= p_k - \text{\{1 if $y=k$, 0 otherwise\}} \\
\frac{\partial L}{\partial W_2}          &= \mathbf{h} \left(\frac{\partial L}{\partial \mathbf{z}}\right)^T \\
\frac{\partial L}{\partial \mathbf{b}_2} &= \frac{\partial L}{\partial \mathbf{z}}\\
\frac{\partial L}{\partial \mathbf{h}}   &= W_2 \frac{\partial L}{\partial \mathbf{z}} \circ f'(\mathbf{h}) \\
\frac{\partial L}{\partial W_1}          &= \mathbf{x} \left(\frac{\partial L}{\partial \mathbf{h}}\right)^T\\
\frac{\partial L}{\partial \mathbf{b}_1} &= \frac{\partial L}{\partial \mathbf{h}}\\
\end{aligned}
$$
Even for very deep or complicated networks, the gradient of the loss function can always be decomposed in this way, one connection at a time.

The actual quantity we wish to minimize is the sum of this loss value over all the samples in our data set. The gradient of this total loss is just the sum of the gradient for each sample.

### 2a) Implement `fit` function
Our gradient descent training consists of the following steps:
- Compute loss gradient with respect to model parameters using `dloss`. Note: for the sake of efficiency, this function returns the sum of the gradient of over all samples, so you should pass in the entire dataset.
- Update model parameters as $W_i \leftarrow W_i - \alpha\frac{\partial{L}}{\partial W_i}$ and $\mathbf{b}_i \leftarrow \mathbf{b}_i - \alpha\frac{\partial{L}}{\partial \mathbf{b}_i}$, where $\alpha$ is some chosen step size.
- Repeat for `num_iter` iterations.

Implement these steps in the `fit` function above. Check that the loss function is decreasing, either by occasionaly printing the loss value or by returning the losses from each step and plotting the trajectory.

### 2b) Examine predictions for fitted model
Train a model and examine the predictions using `visualize_predictions`. Repeat for a few combinations hidden layer sizes, gradient descent step sizes, and numbers of training iterations. What's the best accuracy you can achieve? How does the loss curve change with the step size?

```python
# Initialize model, train on sample data, and visualize predictions for part 2b:
pass
```

### 2c) (optional) Numerically check the gradient implementation in `dloss`
This is an extremely important step when coding derivatives by hand: check the gradients returned by `dloss` by perturbing a single weight value by $\epsilon$ and comparing the resulting change in loss to what you'd expect from the gradient.

\* This is a bit of an abuse of notation since we're taking some derivatives with respect to vectors/matrices; this just means a vector/matrix of derivatives with respect to each element.
\*\* For an excrutiating amount of detail about backprop, see e.g. [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/chap2.html).

```python
# Check gradient for part 2c:
pass
```

## Part 3: Compare with `scikit-learn`
The `scikit-learn` package does contain a minimal implementation of a neural network classifier in `scikit-learn.neural_networks.MLPClassifier`. This classifier is nowhere near as powerful or fully-featured as the neural networks provided by other packages such as `tensorflow`, `theano`, etc., but it still can represent complicated non-linear relationships within small datasets.

### 3a) Implement a simple neural network classifier using `scikit-learn`
What parameter values give a network comparable to the one we implemented above? Note that the results may not be equivalent since `scikit-learn` version could reach a different set of (locally) optimal parameters.

Hint: try using `solver='lbfgs'` (a more complicated optimization method) for better results.

### 3b) Experiment with different activations/network sizes
How do the decision boundaries change for different models?

### 3c) Experiment with different numbers of classes and noise levels
Go back to where we initialized the data and change the parameters `K` and `sigma`. Can a single- or multi-layer perceptron still represent more complicated patterns?

```python
# Train `scikit-learn` multi-layer perceptron
from sklearn.neural_network import MLPClassifier

?MLPClassifier
```

## Part 4: Visualize neuron activations (optional)
Interpreting neural network classifiers can be quite difficult, especially when the network contains many layers. One way of attempting to make sense of the model is by visualizing the activations of individual neurons: by plotting the activation of a neuron over the entire input space, we can see what patterns the neurons are learning to represent.

Try below to generate plots that show the activation of an individual neuron. `scikit-learn` doesn't make this immediately available, but we can make it work: try setting the last layer's weights (in `coefs_[-1]`) to zero for everything except a single unit, and setting that unit's weights to 1 (so that the activation just passes straight through). You can re-use most of the code from `visualize_predictions` to generate a contour map of activation strengths.

```python
# Visualize neuron activations for part 4:
pass
```


# ***Introduction to Radar Using Python and MATLAB***
## Andy Harrison - Copyright (C) 2019 Artech House
<br/>

# Barker Ambiguity Function
***

Referring to Section 8.7.1, a common set of binary phase codes for radar applications is Barker codes, which have several unique properties.  The time sidelobe level of the normalized ambiguity function is $1/N$, where $N$ is the length of the code.  The ambiguity function has a length of $2N\tau_n$, the main lobe has a width of $2\tau_n$, and there are $(N-1)/2$ sidelobes on either side of the main lobe.  There are only seven known Barker codes, and these are given in Table 8.1.

While these are the only known Barker codes, much research has been performed to find longer binary phase codes with low time sidelobe levels.  Although the time sidelobe levels are not $1/N$ as with the Barker codes, some codes have been found to have quite small sidelobe levels.  A Barker code of length $4$ is illustrated in Figure 8.20.  Another approach to obtain longer codes is to combine or embed one code within another.  For example, a Barker code of length $5$ may be used with a Barker code of length $3$ to give (Equation 8.71)

\begin{equation}
    B_{MN} = [\; \; 0\;\; 0\;\; 0\;\; \pi\;\; 0\,,   \; 0\;\; 0\;\; 0\;\; \pi\;\; 0\,,   \; \pi\;\; \pi\;\; \pi\;\; 0\;\; \pi\;].
\end{equation}

The compression ratio of the combined Barker code is $MN$.  However, the sidelobe level of the combined code is not $1/MN$.  When using combined Barker codes, a number of sidelobes may be reduced to zero if the matched filter is followed by a linear transversal filter.
***

Begin by getting the library path

```python
import lib_path
```

Set the chip width (s) and the code length (2, 3, 4, 5, 7, 11, 13)

```python
chip_width = 0.1

code_length = 5
```

Set the Barker code

```python
if code_length == 2:

    code = [1, -1]

elif code_length == 3:

    code = [1, 1, -1]

elif code_length == 4:

    code = [1, 1, -1, 1]

elif code_length == 5:

    code = [1, 1, 1, -1, 1]

elif code_length == 7:

    code = [1, 1, 1, -1, -1, 1, -1]

elif code_length == 11:

    code = [1, 1, 1, -1, -1, -1, 1, -1, -1, 1, -1]

elif code_length == 13:

    code = [1, 1, 1, 1, 1, -1, -1, 1, 1, -1, 1, -1, 1]
```

Calculate the ambiguity function for the Barker code

```python
from Libs.ambiguity.ambiguity_function import phase_coded_wf

ambiguity, time_delay, doppler_frequency = phase_coded_wf(code, chip_width)
```

Display the zero-Doppler cut, the zero-range cut, and the 2D contour plot using the `matplotlib` routines

```python
from matplotlib import pyplot as plt

# Set the figure size

plt.rcParams["figure.figsize"] = (15, 10)

# Plot the ambiguity function

plt.plot(time_delay, ambiguity[round(len(doppler_frequency) / 2)], '')

# Set the time axis limits

plt.xlim(-len(code) * chip_width, len(code) * chip_width)

# Set the x and y axis labels

plt.xlabel("Time (s)", size=12)

plt.ylabel("Relative Amplitude", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Barker Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

Create the zero-range cut

```python
 # Plot the ambiguity function

plt.plot(doppler_frequency,  ambiguity[:, round(len(time_delay) / 2)], '')

# Set the x and y axis labels

plt.xlabel("Doppler (Hz)", size=12)

plt.ylabel("Relative Amplitude", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Barker Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

Create the two-dimensional contour plot

```python
from numpy import meshgrid

# Create the grid

t, f = meshgrid(time_delay, doppler_frequency)

# Plot the ambiguity function

plt.contour(t, f, ambiguity, 30, cmap='jet', vmin=-0.2, vmax=1.0)

# Set the time axis limits

plt.xlim(-len(code) * chip_width, len(code) * chip_width)

# Set the x and y axis labels

plt.xlabel("Time (s)", size=12)

plt.ylabel("Doppler (Hz)", size=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the plot title and labels

plt.title('Barker Code Ambiguity Function', size=14)

# Set the tick label size

plt.tick_params(labelsize=12)
```

```python

```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, G.F. Forsyth, I. Hawke. Partly based on content by David Ketcheson, also under CC-BY.

# Full phugoid model

This is the third IPython Notebook of the series on the _phugoid model of glider flight_, our first learning module of the course ["Practical Numerical Methods with Python"](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about). In the first notebook, we described the physics of the trajectories known as phugoids obtained from an exchange of potential and kinetic energy in an idealized motion with no drag. We gave you a neat little code to play with and plot various phugoid curves.

In the second notebook, we looked at the equation representing small perturbations on the straight-line phugoid, resulting in simple harmonic motion. This is a second-order ordinary differential equation, and we solved it numerically using **Euler's method**: the simplest numerical method of all. We learned about convergence and calculated the error of the numerical solution, comparing with an analytical solution. That is a good foundation!

Now, let's go back to the dynamical model, and take away the idealization of no-drag. Let's remind ourselves of the forces affecting an aircraft, considering now that it may be accelerating, with an instantaneous upward trajectory. We use the designation $\theta$ for the angle, and consider it positive upwards.

#### Figure 1. Forces with a positive trajectory angle.

In Figure 1, $L$ is the lift, $W$ is the weight, $D$ is the drag, and $\theta$ the positive angle of the trajectory, instantaneously.

In lesson 1, we wrote the force balance in the directions perpendicular and parallel to the trajectory for a glider in _equilibrium_. What if the forces are _not_ in balance? Well, there will be acceleration terms in the equations of motion, and we would have in that case:

\begin{align}
m \frac{dv}{dt} & = - W \sin\theta - D \\
m v \, \frac{d\theta}{dt} & = - W \cos\theta + L
\end{align}

We can use a few little tricks to make these equations more pleasing. First, use primes to denote the time derivatives and divide through by the weight:

\begin{align}
 \frac{v'}{g} & = - \sin\theta - D/W \\
\frac{v}{g} \, \theta' & = - \cos\theta + L/W
\end{align}

Recall, from our first lesson, that the ratio of lift to weight is known from the trim conditions—$L/W=v^2/v_t^2$— and also from the definitions of lift and drag,

$$\begin{eqnarray}
L &=& C_L S \times \frac{1}{2} \rho v^2 \\
D &=& C_D S \times \frac{1}{2} \rho v^2
\end{eqnarray}$$

we see that $L/D=C_L/C_D$. The system of equations can be re-written:

\begin{align}
 v' & = - g\, \sin\theta - \frac{C_D}{C_L} \frac{g}{v_t^2} v^2 \\
 \theta' & = - \frac{g}{v}\,\cos\theta + \frac{g}{v_t^2}\, v
\end{align}

It is very interesting that the first equation has the factor $C_D/C_L$, which is the inverse of a measure of the aerodynamic efficiency of the aircraft. It turns out, this is the term that contributes damping to the phugoid model: if drag is zero, there is no damping. Drag is never zero in real life, but as engineers design more aerodynam-ically efficient aircraft, they make the phugoid mode more weakly damped. At altitude, this is nothing but a slight bother, but vertical oscillations are unsafe during final approach to land, so this is something to watch out for!

## The initial value problem

If we want to visualize the flight trajectories predicted by this model, we are going to need to integrate the spatial coordinates, which depend on both the forward velocity (tangential to the trajectory) and the trajectory angle. The position of the glider on a vertical plane will be designated by coordinates $(x, y)$ with respect to an inertial frame of reference, and are obtained from:

\begin{align}
x'(t) & = v \cos(\theta) \\
y'(t) & = v \sin(\theta).
\end{align}

Augmenting our original two differential equations by the two equations above, we have a system of four first-order differential equations to solve. We will use a time-stepping approach, like in the previous lesson. To do so, we do need *initial values* for every unknown:

$$
v(0) = v_0 \quad \text{and} \quad \theta(0) = \theta_0\\
x(0) = x_0 \quad \text{and} \quad y(0) = y_0
$$

## Solve with Euler's method

We know how to apply Euler's method from the previous lesson. We replace each of the time derivatives by an approximation of the form:

$$v'(t) \approx \frac{v^{n+1} - v^n}{\Delta t},$$

where we are now using a superscript $n$ to indicate the $n$-th value in the time iterations. The first differential equation, for example, gives:

$$\frac{v^{n+1} - v^n}{\Delta t} = - g\, \sin\theta^n - \frac{C_D}{C_L} \frac{g}{v_t^2} (v^n)^2$$

Alright, we know where this is going. At each time iteration $t^n$, we want to evaluate all the known data of our system to obtain the state at $t^{n+1}$—the next time step. We say that we are _stepping in time_ or _time marching_.

The full system of equations discretized with Euler's method is:

\begin{align}
v^{n+1} & = v^n + \Delta t \left(- g\, \sin\theta^n - \frac{C_D}{C_L} \frac{g}{v_t^2} (v^n)^2 \right) \\
\theta^{n+1} & = \theta^n + \Delta t \left(- \frac{g}{v^n}\,\cos\theta^n + \frac{g}{v_t^2}\, v^n \right) \\
x^{n+1} & = x^n + \Delta t \, v^n \cos\theta^n \\
y^{n+1} & = y^n + \Delta t \, v^n \sin\theta^n.
\end{align}

As we've learned before, the system of differential equations can also be written as a vector equation:

$$u'(t) = f(u)$$

where

\begin{align}
u & = \begin{pmatrix} v \\ \theta \\ x \\ y \end{pmatrix} & f(u) & = \begin{pmatrix} - g\, \sin\theta - \frac{C_D}{C_L} \frac{g}{v_t^2} v^2 \\ - \frac{g}{v}\,\cos\theta + \frac{g}{v_t^2}\, v \\ v\cos\theta \\ v\sin\theta \end{pmatrix}.
\end{align}

It's a bit tricky to code the solution using a NumPy array holding all your independent variables. But if you do, a function for the Euler step can be written that takes any number of simultaneous equations. It simply steps in time using the same line of code:

```Python
def euler_step(u, f, dt):
    return u + dt * f(u)
```

This function can take a NumPy array `u` with any number of components. All we need to do is create an appropriate function `f(u)` describing our system of differential equations. Notice how we are passing a _function_ as part of the arguments list to `euler_step()`. Neat!

### And solve!

As always, we start by loading the modules and libraries that we need for this problem. We'll need a few transcendental functions, including the $\log$ for a convergence study later on. And remember: the line `%matplotlib inline` is a magic function that tells Matplotlib to give us the plots in the notebook (the default behavior of Matplotlib is to open a pop-up window). In addition, we are importing the module `rcParams` to define notebook-wide plotting parameters: font family and size. Here we go!

```python
from math import sin, cos, log, ceil
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

Next, we need to set things up to start our numerical solution: the parameter
values and the _initial values_. You know what the acceleration of gravity is: 9.8 m/s$^2$, but what are good values for $C_D/C_L$, the inverse of the aerodynamic efficiency? Some possible values are given on a table in the Wikipedia entry for [lift-to-drag ratio](http://en.wikipedia.org/wiki/Lift-to-drag_ratio): a modern sailplane can have $L/D$ of 40 to 60, depending on span (and, in case you're interested, a flying squirrel has $L/D$ close to 2).

For the _trim velocity_, the speed range for typical sailplanes is between 65 and 280 km/hr, according to Wikipedia (it must be right!). Let's convert that to meters per second: 18 to 78 m/s. We'll pick a value somewhere in the middle of this range.

Here's a possible set of parameters for the simulation, but be sure to come back and change some of these, and see what happens!

```python
# model parameters:
g = 9.8      # gravity in m s^{-2}
v_t = 30.0   # trim velocity in m s^{-1}
C_D = 1/40  # drag coefficient --- or D/L if C_L=1
C_L = 1   # for convenience, use C_L = 1

### set initial conditions ###
v0 = v_t     # start at the trim velocity (or add a delta)
theta0 = 0 # initial angle of trajectory
x0 = 0     # horizotal position is arbitrary
y0 = 1000  # initial altitude
```

We'll define a function `f()` to match the right-hand side of Equation (15), the full differential system in vector form. This function assumes that we have available the parameters defined above. If you re-execute the cell above with different parameter values, you can just run the solution without re-executing the function definition.

```python
def f(u):
    """Returns the right-hand side of the phugoid system of equations.

    Parameters
    ----------
    u : array of float
        array containing the solution at time n.

    Returns
    -------
    dudt : array of float
        array containing the RHS given u.
    """

    v = u[0]
    theta = u[1]
    x = u[2]
    y = u[3]
    return numpy.array([-g*sin(theta) - C_D/C_L*g/v_t**2*v**2,
                      -g*cos(theta)/v + g/v_t**2*v,
                      v*cos(theta),
                      v*sin(theta)])
```

Compare the code defining function `f(u)` with the differential equations, and convince yourself that it's right!

\begin{align}
u & = \begin{pmatrix} v \\ \theta \\ x \\ y \end{pmatrix} & f(u) & = \begin{pmatrix} - g\, \sin\theta - \frac{C_D}{C_L} \frac{g}{v_t^2} v^2 \\ - \frac{g}{v}\,\cos\theta + \frac{g}{v_t^2}\, v \\ v\cos\theta \\ v\sin\theta \end{pmatrix} \nonumber
\end{align}

Now, Euler's method is implemented in a simple function `euler_step()`:

```python
def euler_step(u, f, dt):
    """Returns the solution at the next time-step using Euler's method.

    Parameters
    ----------
    u : array of float
        solution at the previous time-step.
    f : function
        function to compute the right hand-side of the system of equations.
    dt : float
        time-increment.

    Returns
    -------
    u_n_plus_1 : array of float
        approximate solution at the next time step.
    """

    return u + dt * f(u)
```

After defining a final time for the solution, and the time step $\Delta t$, we can construct the grid in time using the NumPy function [`linspace()`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.linspace.html). Make sure you study the decisions we made here to build the time grid: why do we add 1 to the definition of `N`, for example?

Look at the code below, and make sure you understand the following aspects of it.

* The NumPy array `u` contains the solution at every time-step,  consisting of the velocity, angle and location of the glider.
* The first element of the array `u` is set to contain the initial conditions.
* In the `for`-loop, the function `euler_step()` is called to get the solution at time-step $n+1$.

```python
T = 100                          # final time
dt = 0.1                           # time increment
N = int(T/dt) + 1                  # number of time-steps

# initialize the array containing the solution for each time-step
u = numpy.empty((N, 4))
u[0] = numpy.array([v0, theta0, x0, y0])# fill 1st element with initial values

# time loop - Euler method
for n in range(N-1):

    u[n+1] = euler_step(u[n], f, dt)
```

## Plot the trajectory

In order to plot the path of the glider, we need the location (`x`, `y`) with respect to time. That information is already contained in our NumPy array containing the solution; we just need to pluck it out.

Make sure you understand the indices to `u`, below, and the use of the colon notation. If any of it is confusing, read the Python documentation on [Indexing](http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html).

```python
# get the glider's position with respect to the time
x = u[:,2]
y = u[:,3]
```

Time to plot the path of the glider and get the distance travelled!

```python
# visualization of the path
pyplot.figure(figsize=(8,6))
pyplot.grid(True)
pyplot.xlabel(r'x', fontsize=18)
pyplot.ylabel(r'y', fontsize=18)
pyplot.title('Glider trajectory, flight time = %.2f' % T, fontsize=18)
pyplot.plot(x,y, 'k-', lw=2);
```

## Grid convergence

Let's study the convergence of Euler's method for the phugoid model. In the previous lesson, when we studied the straight-line phugoid under a small perturbation, we looked at convergence by comparing the numerical solution with the exact solution. Unfortunately, most problems don't have an exact solution (that's why we compute in the first place!). But here's a neat thing: we can use numerical solutions computed on different grids to study the convergence of the method, even without an analytical solution.

We need to be careful, though, and make sure that the fine-grid solution is resolving all of the features in the mathematical model. How can we know this? We'll have a look at that in a bit. Let's see how this works first.

You need a sequence of numerical solutions of the same problem, each with a different number of time grid points.

Let's create a NumPy array called `dt_values` that contains the  time-increment of each grid to be solved on. For each element `dt_values[i]`, we will compute the solution `u_values[i]` of the glider model using Euler's method. If we want to use five different values of $\Delta t$, we'll have five solutions: we put them in an array ... but each one is also an array! We'll have an array of arrays. How meta is that?

We have one more trick up our sleeve: `enumerate()`. To get all the numerical solutions—each with its value of $\Delta t$—done in one fell swoop, we will loop over the elements of the array `dt_values`. Within the loop, we need to access both `dt_values[i]` and the index `i`. It turns out, `enumerate()` is a built-in Python function that will give us consecutive `index, value` pairs just like we need.

Read the code below carefully, and remember: you can get a help pane on any function by entering a question mark followed by the function name. For example, add a new code cell below and type:  `?numpy.empty_like`.

```python
dt_values = numpy.array([0.1, 0.05, 0.01, 0.005, 0.001])

u_values = numpy.empty_like(dt_values, dtype=numpy.ndarray)

for i, dt in enumerate(dt_values):

    N = int(T/dt) + 1    # number of time-steps

    # initialize the array containing the solution for each time-step
    u = numpy.empty((N, 4))
    u[0] = numpy.array([v0, theta0, x0, y0])

    # time loop
    for n in range(N-1):

        u[n+1] = euler_step(u[n], f, dt)   ### call euler_step() ###

    # store the value of u related to one grid
    u_values[i] = u
```

In [Lesson 2](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/01_phugoid/01_02_Phugoid_Oscillation.ipynb), we compared our numerical result to an analytical solution, but now we will instead compare numerical results from different grids.

For each solution, we'll compute the difference relative to the finest grid. You will be tempted to call this an _"error"_, but be careful: the solution at the finest grid is _not the exact_ solution, it is just a reference value that we can use to estimate grid convergence.

To calculate the difference between one solution `u_current` and the solution at the finest grid, `u_finest`, we'll use the $L_1$-norm, but any norm will do.

There is a small problem with this, though.  The coarsest grid, where $\Delta t = 0.1$, has 1001 grid points, while the finest grid, with $\Delta t = 0.001$ has 100001 grid points.  How do we know which grid points correspond to the same location in two numerical solutions, in order to compare them?

If we had time grids of 10 and 100 steps, respectively, this would be relatively simple to calculate.  Each element in our 10-step grid would span ten elements in our 100-step grid.

Calculating the _ratio_ of the two grid sizes will tell us how many elements in our fine-grid will span over one element in our coarser grid.

Recall that we can _slice_ a NumPy array and grab a subset of values from it.  The syntax for that is

```Python
my_array[3:8]
```

An additional slicing trick that we can take advantage of is the "slice step size."  We add an additional `:` to the slice range and then specify how many steps to take between elements. For example, this code

```Python
my_array[3:8:2]
```

will return the values of `my_array[3]`, `my_array[5]` and `my_array[7]`

With that, we can write a function to obtain the differences between coarser and finest grids. Here we go ...

```python
def get_diffgrid(u_current, u_fine, dt):
    """Returns the difference between one grid and the fine one using L-1 norm.

    Parameters
    ----------
    u_current : array of float
        solution on the current grid.
    u_finest : array of float
        solution on the fine grid.
    dt : float
        time-increment on the current grid.

    Returns
    -------
    diffgrid : float
        difference computed in the L-1 norm.
    """

    N_current = len(u_current[:,0])
    N_fine = len(u_fine[:,0])

    grid_size_ratio = ceil(N_fine/N_current)

    diffgrid = dt * numpy.sum( numpy.abs(\
            u_current[:,2]- u_fine[::grid_size_ratio,2]))

    return diffgrid
```

Now that the function has been defined, let's compute the grid differences for each solution, relative to the fine-grid solution. Call the function `get_diffgrid()` with two solutions, one of which is always the one at the finest grid. Here's a neat Python trick: you can use negative indexing in Python!  If you have an array called `my_array` you access the _first_ element with

`my_array[0]`

But you can also access the _last_ element with

`my_array[-1]`

and the next to last element with

`my_array[-2]`

and so on.

```python
# compute difference between one grid solution and the finest one
diffgrid = numpy.empty_like(dt_values)

for i, dt in enumerate(dt_values):
    print('dt = {}'.format(dt))

    ### call the function get_diffgrid() ###
    diffgrid[i] = get_diffgrid(u_values[i], u_values[-1], dt)
```

    dt = 0.1
    dt = 0.05
    dt = 0.01
    dt = 0.005
    dt = 0.001

Time to create a plot of the results! We'll create a *log-log* plot with the Matplotlib function [`loglog()`](http://matplotlib.org/api/pyplot_api.html?highlight=loglog#matplotlib.pyplot.loglog). Remember to skip the difference of the finest-grid solution with itself, which is zero.

```python
# log-log plot of the grid differences
pyplot.figure(figsize=(6,6))
pyplot.grid(True)
pyplot.xlabel('$\Delta t$', fontsize=18)
pyplot.ylabel('$L_1$-norm of the grid differences', fontsize=18)
pyplot.axis('equal')
pyplot.loglog(dt_values[:-1], diffgrid[:-1], color='k', ls='-', lw=2, marker='o');
```

### Order of convergence

The order of convergence is the rate at which the numerical solution approaches the exact one as the mesh is refined. Considering that we're not comparing with an exact solution, we use 3 grid resolutions that are refined at a constant ratio $r$ to find the *observed order of convergence* ($p$), which is given by:

$$
\begin{equation}
p = \frac{\log \left(\frac{f_3-f_2}{f_2-f_1} \right) }{\log(r)}
\end{equation}
$$

where $f_1$ is the finest mesh solution, and $f_3$ the coarsest.

```python
r = 2
h = 0.001

dt_values2 = numpy.array([h, r*h, r**2*h])

u_values2 = numpy.empty_like(dt_values2, dtype=numpy.ndarray)

diffgrid2 = numpy.empty(2)

for i, dt in enumerate(dt_values2):

    N = int(T/dt) + 1   # number of time-steps

    # initialize the array containing the solution for each time-step
    u = numpy.empty((N, 4))
    u[0] = numpy.array([v0, theta0, x0, y0])

    # time loop
    for n in range(N-1):

        u[n+1] = euler_step(u[n], f, dt)         ### call euler_step() ###

    # store the value of u related to one grid
    u_values2[i] = u

#calculate f2 - f1
diffgrid2[0] = get_diffgrid(u_values2[1], u_values2[0], dt_values2[1])

#calculate f3 - f2
diffgrid2[1] = get_diffgrid(u_values2[2], u_values2[1], dt_values2[2])

# calculate the order of convergence
p = (log(diffgrid2[1]) - log(diffgrid2[0])) / log(r)

print('The order of convergence is p = {:.3f}'.format(p));
```

    The order of convergence is p = 1.014

See how the observed order of convergence is close to 1? This means that the rate at which the grid differences decrease match the mesh-refinement ratio. We say that Euler's method is of *first order*, and this result is a consequence of that.

# Paper airplane challenge

Suppose you wanted to participate in a paper-airplane competition, and you want to use what you know about the phugoid model to improve your chances. For a given value of $L/D$ that you can obtain in your design, you want to know what is the best initial velocity and launch angle to fly the longest distance from a given height.

Using the phugoid model, write a new code to analyze the flight of a paper airplane, with the following conditions:

*  Assume $L/D$ of 5.0 (a value close to measurements in Feng et al. 2009)
*  For the trim velocity, let's take an average value of 4.9 m/s.
*  Find a combination of launch angle and velocity that gives the best distance.
*  Think about how you will know when the flight needs to stop ... this will influence how you organize the code.
*  How can you check if your answer is realistic?

## References

* Feng, N. B. et al. _"On the aerodynamics of paper airplanes"_, AIAA paper 2009-3958, 27th AIAA Applied Aerodynamics Conference, San Antonio, TX. [PDF](http://arc.aiaa.org/doi/abs/10.2514/6.2009-3958)

* Simanca, S. R. and Sutherland, S. _"Mathematical problem-solving with computers,"_ 2002 course notes, Stony Brook University, chapter 3: [The Art of Phugoid](https://www.math.sunysb.edu/~scott/Book331/Art_Phugoid.html). (Note that there is an error in the figure: sine and cosine are switched.)

---

###### The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "PT Mono";
    font-size: 90%;
}

</style>