# A toy example

**Purpose of this demo**: Motivate that abstract notions, such as sparse projection, are useful in practice.

+ Disclaimer: I'm not expert in Python - I use Python/Matlab as tools to validate algorithms and theorems.
+ Thus, my implementations are not the most efficient ones + there might be bugs

**Problem definition: Linear regression**.

\begin{align}
  y = A x^\star + w
\end{align}

+ $A \in \mathbb{R}^{n \times p}$
+ $x^\star \in \mathbb{R}^p$
+ $w \in \mathbb{R}^n$

Assume $n = p$, and $A$ is in general position.
Given $y$ and $A$:

\begin{equation*}
\begin{aligned}
& \underset{x \in \mathbb{R}^p}{\text{min}}
& & f(x) \triangleq \|y - A x\|_2^2
\end{aligned}
\end{equation*}

```python
%matplotlib inline

import warnings
warnings.filterwarnings('ignore')

import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
import random
from scipy import stats
from scipy.optimize import fmin

from matplotlib import rc
#rc('font',**{'family':'sans-serif','sans-serif':['Helvetica']})
## for Palatino and other serif fonts use:
rc('font',**{'family':'serif','serif':['Palatino']})
rc('text', usetex=True)

from PIL import Image

import random
from numpy import linalg as la

p = 100  # Ambient dimension
n = 100  # Number of samples

# Generate a p-dimensional zero vector
x_star = np.random.randn(p)
# Normalize
x_star = (1 / la.norm(x_star, 2)) * x_star
```

```python
# Plot
xs = range(p)
markerline, stemlines, baseline = plt.stem(xs, x_star, '-.')
plt.setp(markerline, 'alpha', 0.3, 'ms', 6)
plt.setp(markerline, 'markerfacecolor', 'b')
plt.setp(baseline, 'color', 'r', 'linewidth', 1, 'alpha', 0.3)
plt.rc('text', usetex=True)
#plt.rc('font', family='serif')
plt.xlabel('Index')
plt.ylabel('Amplitude')
plt.show()
```

 How would you solve this problem?

Closed form solution using matrix inverse

\begin{align}
\widehat{x} = A^{-1} y
\end{align}

```python
A = np.random.randn(n, p)
y = A.dot(x_star)

A_inv = la.inv(A)
widehat_x = A_inv.dot(y)
# Plot
xs = range(p)
markerline, stemlines, baseline = plt.stem(xs, widehat_x, '-.')
plt.setp(markerline, 'alpha', 0.3, 'ms', 6)
plt.setp(markerline, 'markerfacecolor', 'b')
plt.setp(baseline, 'color', 'r', 'linewidth', 1, 'alpha', 0.3)
plt.xlabel('Index')
plt.ylabel('Amplitude')
plt.show()

print('\|x^\star - x\|_2 = {0}'.format(la.norm(x_star - widehat_x)))
```

**Problem definition: Sparse linear regression**.

\begin{align}
  y = A x^\star + w
\end{align}

+ $A \in \mathbb{R}^{n \times p}$, but now $n \ll p$
+ $x^\star \in \mathbb{R}^p$ but $k$-sparse, where $k \ll p$
+ $w \in \mathbb{R}^n$

Assume $n = p$, and $A$ is in general position.
Given $y$ and $A$:

\begin{equation*}
\begin{aligned}
& \underset{x \in \mathbb{R}^p}{\text{min}}
& & f(x) \triangleq \|y - A x\|_2^2
\end{aligned}
\end{equation*}

 Would a similar technique solve the problem?

```python
p = 100  # Ambient dimension
n = 40   # Number of samples
k = 5    # Sparsity level

# Generate a p-dimensional zero vector
x_star = np.zeros(p)
# Randomly sample k indices in the range [1:p]
x_star_ind = random.sample(range(p),  k)
# Set x_star_ind with k random elements from Gaussian distribution
x_star[x_star_ind] = np.random.randn(k)
# Normalize
x_star = (1 / la.norm(x_star, 2)) * x_star

# Plot
xs = range(p)
markerline, stemlines, baseline = plt.stem(xs, x_star, '-.')
plt.setp(markerline, 'alpha', 0.3, 'ms', 6)
plt.setp(markerline, 'markerfacecolor', 'b')
plt.setp(baseline, 'color', 'r', 'linewidth', 1, 'alpha', 0.3)
plt.xlabel('Index')
plt.ylabel('Amplitude')
plt.show()
```

We will use the pseudo-inverse of $A$:

\begin{align}
A^\dagger = A^\top (AA^\top)^{-1}
\end{align}

```python
A = np.random.randn(n, p)
y = A.dot(x_star)

A_inv = la.pinv(A)
widehat_x = A_inv.dot(y)
# Plot
xs = range(p)
markerline, stemlines, baseline = plt.stem(xs, widehat_x, '-.')
plt.setp(markerline, 'alpha', 0.3, 'ms', 6)
plt.setp(markerline, 'markerfacecolor', 'b')
plt.setp(baseline, 'color', 'r', 'linewidth', 1, 'alpha', 0.3)
plt.xlabel('Index')
plt.ylabel('Amplitude')
plt.show()

la.norm(x_star - widehat_x)
```

+ The reconstruction of $x^\star$ from $y$ is an ill-posed problem since $n < p$ and there is no hope in finding the *true vector* without ambiguity.
+ Additional prior information is needed.
+ We might want to use the fact that $\|x\|_0 \leq k$ where $k \ll p$ and $\|\cdot\|_0$ is the $\ell_0$-"norm".
+ It turns out that, under proper assumptions on the sensing matrix $A$ and the sparsity level $k$, one can still recover $x^\star$!

### Why sparsity?

Let us consider the following practical case: image processing.

```python
%matplotlib inline
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
import random
from scipy import stats
from scipy.optimize import fmin

from PIL import Image

# Open image using Image package
x_mush_orig = Image.open("./SupportFiles/mushroom.png").convert("L")
# Transform to a np array
x_mush_star = np.fromstring(x_mush_orig.tobytes(), np.uint8)
# Set the shape of np array
x_mush_star.shape = (x_mush_orig.size[1], x_mush_orig.size[0])
# Show the image
plt.imshow(x_mush_star, interpolation = "nearest", cmap = plt.cm.gray)
```

Obviously, this is a simple image case: the "mushroom" image is sparse by itself (do you see the black pixels? Yes, they are zeros). To see this more clearly, let's sort the true coeffients in decreasing order.

```python
from bokeh.plotting import figure, show, output_file
from bokeh.palettes import brewer

# Get the absolute value of a flatten array (vectorize)
x_mush_abs = abs(x_mush_star.flatten())
# Sort the absolute values (ascending order)
x_mush_abs.sort()
# Descending order
x_mush_abs_sort = np.array(x_mush_abs[::-1])

plt.style.use('bmh')
fig, ax = plt.subplots()
# Generate an array with elements 1:len(...)
xs = np.arange(len(x_mush_abs_sort))
# Fill plot - alpha is transparency (might take some time to plot)
ax.fill_between(xs, 0, x_mush_abs_sort, alpha = 0.2)
# Plot - alpha is transparency (might take some time to plot)
ax.plot(xs, x_mush_abs_sort, alpha = 0.8)
plt.show()
```

For this 64 x 64 image, the total number of pixels sums up to 4096. As you can observe, by default almost half of the pixels are zero, which constitutes "mushroom" image sparse (but still the sparsity level is quite high: more than half the ambient dimension).

Since this seems to be a "cooked"-up example, let us consider a more *realistic* scenario: a brick house. (*Does anyone know where is this house?*)

```python
x_house_orig = Image.open("./SupportFiles/house128.png").convert("L")
x_house_star = np.fromstring(x_house_orig.tobytes(), np.uint8)
x_house_star.shape = (x_house_orig.size[1], x_house_orig.size[0])
plt.imshow(x_house_star, interpolation = "nearest", cmap = plt.cm.gray)
```

...and here is the bar plot of the coefficients.

```python
x_house_abs = abs(x_house_star.flatten())
x_house_abs.sort()
x_house_abs_sort = np.array(x_house_abs[::-1])

plt.style.use('bmh')
fig, ax = plt.subplots()
xs = np.arange(len(x_house_abs_sort))
ax.fill_between(xs, 0, x_house_abs_sort, alpha = 0.2)
plt.plot(xs, x_house_abs_sort, alpha=0.8)
plt.show()
```

+ All the coefficients are non-zero! Is there anything we can do in this case?

+ However: under proper orthonormal transformations, natural images become sparse.

```python
import pywt

x_house_orig = Image.open("./SupportFiles/house.png").convert("L")
x_house_star = np.fromstring(x_house_orig.tobytes(), np.uint8)
x_house_star.shape = (x_house_orig.size[1], x_house_orig.size[0])

# Defines a wavelet object - 'db1' defines a Daubechies wavelet
wavelet = pywt.Wavelet('db1')

# Multilevel decomposition of the input matrix
coeffs = pywt.wavedec2(x_house_star, wavelet, level=2)
cA2, (cH2, cV2, cD2), (cH1, cV1, cD1) = coeffs

# Concatenate the level-2 submatrices into a big one and plot
x_house_star_wav = np.bmat([[cA2, cH2], [cV2, cD2]])
plt.imshow(np.flipud(x_house_star_wav), origin='image', interpolation="nearest", cmap=plt.cm.gray)
plt.title("Wavelet representation of image", fontsize=10)
plt.tight_layout()
```

After wavelet transformation, let's plot the wavelet coefficients.

```python
# Flatten and show the histogram
x_house_abs_wav = abs(x_house_star_wav.flatten())
x_house_abs_wav.sort()
x_house_abs_wav.flatten()
x_house_abs_wav_sort = np.array(x_house_abs_wav[::-1])

plt.style.use('bmh')
fig, ax = plt.subplots()
xs = np.arange(len(x_house_abs_wav_sort.flatten()))

ax.fill_between(xs, 0, np.flipud(x_house_abs_wav_sort.flatten()), alpha = 0.2)
plt.plot(xs, np.flipud(x_house_abs_wav_sort.transpose()), alpha = 0.8)
plt.show()
```

It is obvious that much less number of coefficients are non-zero! (...and this holds generally for naturally images.)

```python
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.cm as cm

fig = plt.figure()
ax = fig.add_subplot(111, projection = '3d')
for c, z, zi in zip(['r', 'g', 'b', 'y'], ['./SupportFiles/house128.png', './SupportFiles/peppers128.png', './SupportFiles/man128.png', './SupportFiles/pedro128.png'], [4, 3, 2, 1]):
    y = Image.open(z).convert("L")
    y_star = np.fromstring(y.tobytes(), np.uint8)
    y_star.shape = (y.size[1], y.size[0])

    # Multilevel decomposition of the input matrix
    y_coeffs = pywt.wavedec2(y_star, wavelet, level=2)
    y_cA2, (y_cH2, y_cV2, y_cD2), (y_cH1, y_cV1, y_cD1) = y_coeffs

    # Concatenate the level-2 submatrices into a big one and plot
    y_star_wav = np.bmat([[y_cA2, y_cH2], [y_cV2, y_cD2]])

    y_abs_wav = abs(y_star_wav.flatten())
    y_abs_wav.sort()
    y_abs_wav.flatten()
    y_abs_wav_sort = np.array(y_abs_wav[::-1])

    xs = np.arange(len(y_abs_wav_sort.flatten()))
    cs = c
    ys = [zi] * len(xs)
    ys = np.array(ys)

    ax.plot(xs, ys = ys.flatten(), zs = np.flipud(y_abs_wav_sort.flatten()), zdir = 'z', color = cs, alpha = 0.5)

ax.set_xlabel('X axis')
ax.set_ylabel('Y axis')
ax.set_zlabel('Z axis')
plt.show

```

In the above picture, the y values (1.0 to 4.0) correspond to four different image cases (for chanity check, observe that the red curve is the same curve for the house.png case, presented above).

One can observe that most of the coeffs are close to zero and only few of them (compared to the ambient dimension) are significantly large. This has led to the observation that keeping only the most important coefficients (even truncating the non-zero entries further) leads to a significant compression of the image. At the same time, only these coefficients can lead to a pretty good reconstruction of the original image.

### Using sparse projections

```python
import math

# Generate sensing matrix
A = (1 / math.sqrt(n)) * np.random.randn(n, p)

# Observation model
y = A @ x_star
```

**Gradient descent with sparse projections[7-8]**. Solve the criterion
\begin{align}
    \min_{x} ~ ~f(x) := \frac{1}{2}\|y - Ax\|_2^2 \quad \text{s.t.} \quad \|x\|_0 \leq k
\end{align}

             *The IHT method*

       1:   Choose initial guess $x_0$

       2:   for i = 0, 1, 2, ... do

       3:         Compuete $\nabla f(x_i) = -A^\top \cdot (y - A x_i)$

       4:         $\widehat{x}_{i+1} = x_i - \nabla f(x_i)$

       5:         $x_{i+1} = \arg \min_{x \text{ is } k-\text{sparse}} \|\widehat{x}_{i+1} - x\|_2$

       5:    end for

Let's use this algorithm and see how it performs in practice.

```python
from numpy import linalg as la

# Hard thresholding function
def hardThreshold(x, k):
    p = x.shape[0]
    t = np.sort(np.abs(x))[::-1]
    threshold = t[k-1]
    j = (np.abs(x) < threshold)
    x[j] = 0
    return x

# Returns the value of the objecive function
def f(y, A, x):
    return 0.5 * math.pow(la.norm(y - A @ x, 2), 2)
```

```python
def IHT(y, A, k, iters, epsilon, verbose, x_star):
    p = A.shape[1]   # Length of original signal
    n = A.shape[0]   # Length of measurement vector

    x_new = np.zeros(p)    # Initial estimate
    At = np.transpose(A)   # Transpose of A

    x_list, f_list = [1], [f(y, A, x_new)]

    for i in range(iters):
        x_old = x_new

        # Compute gradient
        grad = -At @ (y - A @ x_new)

        # Perform gradient step
        x_temp = x_old - 0.5 * grad

        # Perform hard thresholding step
        x_new = hardThreshold(x_temp, k)

        if (la.norm(x_new - x_old, 2) / la.norm(x_new, 2)) < epsilon:
            break

        # Keep track of solutions and objective values
        x_list.append(la.norm(x_new - x_star, 2))
        f_list.append(f(y, A, x_new))

        if verbose:
            print("iter# = "+ str(i) + ", ||x_new - x_old||_2 = " + str(la.norm(x_new - x_old, 2)))

    print("Number of steps:", len(f_list))
    return x_new, x_list, f_list

```

```python
# Run algorithm
epsilon = 1e-6                # Precision parameter
iters = 100

x_IHT, x_list, f_list = IHT(y, A, k, iters, epsilon, True, x_star)

# Plot
plt.rc('text', usetex=True)
plt.rc('font', family='serif')

xs = range(p)
markerline, stemlines, baseline = plt.stem(xs, x_IHT, '-.x')
plt.setp(markerline, 'alpha', 0.3, 'ms', 6)
plt.setp(markerline, 'markerfacecolor', 'b')
plt.setp(baseline, 'linewidth', 1, 'alpha', 0.3)
plt.xlabel('Index')
plt.ylabel('Amplitude')
#plt.title(r"$\|x^\star - \widehat{x}\|_2 = %s$" %(la.norm(x_star - x_IHT, 2)), fontsize=16)

# Make room for the ridiculously large title.
plt.subplots_adjust(top=0.8)
plt.show()
```

This is great! IHT finds $\mathbf{x}^\star$ fast and 'accurately'. How fast? Let's create a convergence plot.

```python
# Plot
plt.rc('text', usetex=True)
plt.rc('font', family='serif')

xs = range(len(x_list))
plt.plot(xs, x_list, '-o', color = '#3399FF', linewidth = 4, alpha = 0.7, markerfacecolor = 'b')
plt.yscale('log')
plt.xlabel('Iterations')
plt.ylabel(r"$\|x^\star - \widehat{x}\|_2$")

# Make room for the ridiculously large title.
plt.subplots_adjust(top=0.8)
plt.show()
```


# Trading Lists and Trading Trajectories

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) provided a solution to the optimal liquidation problem by assuming the that stock prices follow a discrete arithmetic random walk, and that the permanent and temporary market impact functions are linear functions of the trading rate.

Almgren and Chriss showed that for each value of risk aversion there is a unique optimal execution strategy. This optimal execution strategy is determined by a trading trajectory and its associated trading list. The optimal trading trajectory is given by:

\begin{equation}
x_j = \frac{\sinh \left( \kappa \left( T-t_j\right)\right)}{ \sinh (\kappa T)}X, \hspace{1cm}\text{ for } j=0,...,N
\end{equation}

and the associated trading list is given by:

\begin{equation}
n_j = \frac{2 \sinh \left(\frac{1}{2} \kappa \tau \right)}{ \sinh \left(\kappa T\right) } \cosh \left(\kappa \left(T - t_{j-\frac{1}{2}}\right)\right) X, \hspace{1cm}\text{ for } j=1,...,N
\end{equation}

where $t_{j-1/2} = (j-\frac{1}{2}) \tau$.

Given some initial parameters, such as the number of shares, the liquidation time, the trader's risk aversion, etc..., the trading list will tell us how many shares we should sell at each trade to minimize our transaction costs.

In this notebook, we will see how the trading list varies according to some initial trading parameters.

## Visualizing Trading Lists and Trading Trajectories

Let's assume we have 1,000,000 shares that we wish to liquidate. In the code below, we will plot the optimal trading trajectory and its associated trading list for different trading parameters, such as trader's risk aversion, number of trades, and liquidation time.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Plot the trading list and trading trajectory. If show_trl = True, the data frame containing the values of the
# trading list and trading trajectory is printed
utils.plot_trade_list(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, show_trl = True)
```

# Implementing a Trading List

Once we have the trading list for a given set of initial parameters, we can actually implement it. That is, we can sell our shares in the stock market according to the trading list and see how much money we made or lost. To do this, we are going to simulate the stock market with a simple trading environment. This simulated trading environment uses the same price dynamics and market impact functions as the Almgren and Chriss model. That is, stock price movements evolve according to a discrete arithmetic random walk and the permanent and temporary market impact functions are linear functions of the trading rate. We are going to use the same environment to train our Deep Reinforcement Learning algorithm later on.

We will describe the details of the trading environment in another notebook, for now we will just take a look at its default parameters. We will distinguish between financial parameters, such the annual volatility in stock price, and the parameters needed to calculate the trade list using the Almgren and Criss model, such as the trader's risk aversion.

```python
import utils

# Get the default financial and AC Model parameters
financial_params, ac_params = utils.get_env_param()
```

### Default Financial Parameters

```python
financial_params
```

Financial Parameters

  Annual Volatility:	  12%	   Bid-Ask Spread:    	     0.125

  Daily Volatility:	  0.8%	   Daily Trading Volume:	 5,000,000

### Parameters for the Almgren and Chriss Model

```python
ac_params
```

Almgren and Chriss Model Parameters

  Total Number of Shares to Sell:	                  1,000,000	   Fixed Cost of Selling per Share:	 $0.062

  Starting Price per Share:	                         $50.00	     Trader's Risk Aversion:	           1e-06

  Price Impact for Each 1% of Daily Volume Traded:	 $2.5e-06	    Permanent Impact Constant:	       2.5e-07

  Number of Days to Sell All the Shares:	              60	       Single Step Variance:	             0.144

  Number of Trades:	                                   60	       Time Interval between trades:	      1.0

The code below implements the trading list resulting from different trading parameters, such as trader's risk aversion, number of trades, and liquidation time. All other parameters, such as total number shares to sell, are taken from the simulated trading environment (see above).

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the random seed
sd = 0

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Implement the trading list for the given parameters
utils.implement_trade_list(seed = sd, lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
```

```python

```


# Generative models - variational auto-encoders

### Author: Philippe Esling (esling@ircam.fr)

In this course we will cover
1. A [quick recap](#recap) on simple probability concepts
2. A formal introduction to [Variational Auto-Encoders](#vae) (VAEs)
3. An explanation of the [implementation](#implem) of VAEs
4. Some [modifications and tips to improve the reconstruction](#improve) of VAEs **(exercise)**

## Quick recap on probability

The field of probability aims to model random or uncertain events. Hence, a random variable $X$ denotes a quantity that is uncertain, such as the result of an experiment (flipping a coin) or the measurement of an uncertain property (measuring the temperature). If we observe several occurrences of the variable $\{\mathbf{x}_{i}\}_{i=1}$, it might take different values on each occasion, but some values may occur more often than others. This information is captured by the _probability distribution_ $p(\mathbf{x})$ of the random variable.

To understand these concepts graphically, we will rely on the `Pytorch Probability` package.

```python
import torch
import torch.nn as nn
import torch.distributions as distrib
import torchvision
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
from helper_plot import hdr_plot_style
hdr_plot_style()
```

### Probability distributions

#### Discrete distributions

Let $\mathbf{x}$ be a discrete random variable with range $R_{X}=\{x_1,\cdots,x_n\}$ (finite or countably infinite). The function
\begin{equation}
    p_{X}(x_{i})=p(X=x_{i}), \forall i\in\{1,\cdots,n\}
\end{equation}
is called the probability mass function (PMF) of $X$.

Hence, the PMF defines the probabilities of all possible values for a random variable. The above notation allows to express that the PMF is defined for the random variable $X$, so that $p_{X}(1)$ gives the probability that $X=1$. For discrete random variables, the PMF is also called the \textit{probability distribution}. The PMF is a probability measure, therefore it satisfies all the corresponding properties
- $0 \leq p_{X}(x_i) < 1, \forall x_i$
- $\sum_{x_i\in R_{X}} p_{X}(x_i) = 1$
- $\forall A \subset R_{X}, p(X \in A)=\sum_{x_a \in A}p_{X}(x_a)$

A very simple example of discrete distribution is the `Bernoulli` distribution. With this distribution, we can model a coin flip. If we throw the coin a very large number of times, we hope to see on average an equal amount of _heads_ and _tails_.

```python
bernoulli = distrib.Bernoulli(0.5)
samples = bernoulli.sample((10000,))
plt.figure(figsize=(10,8))
sns.distplot(samples)
plt.title("Samples from a Bernoulli (coin toss)")
plt.show()
```

However, we can also _sample_ from the distribution to have individual values of a single throw. In that case, we obtain a series of separate events that _follow_ the distribution

```python
vals = ['heads', 'tails']
samples = bernoulli.sample((10,))
for s in samples:
    print('Coin is tossed on ' + vals[int(s)])
```

#### Continuous distributions

The same ideas apply to _continuous_ random variables, which can model for instance the height of human beings. If we try to guess the height of someone that we do not know, there is a higher probability that this person will be around 1m70, instead of 20cm or 3m. For the rest of this course, we will use the shorthand notation $p(\mathbf{x})$ for the distribution $p(\mathbf{x}=x_{i})$,  which expresses for a real-valued random variable $\mathbf{x}$, evaluated at $x_{i}$, the probability that $\mathbf{x}$ takes the value $x_i$.

One notorious example of such distributions is the Gaussian (or Normal) distribution, which is defined as
\begin{equation}
    p(x)=\mathcal{N}(\mu,\sigma)=\frac{1}{\sqrt{2\pi\sigma^{2}}}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}
\end{equation}

Similarly as before, we can observe the behavior of this distribution with the following code

```python
normal = distrib.Normal(loc=0., scale=1.)
samples = normal.sample((10000,))
plt.figure(figsize=(10,8))
sns.distplot(samples)
plt.title("Samples from a standard Normal")
plt.show()
```

### Comparing distributions (KL divergence)
$
\newcommand{\R}{\mathbb{R}}
\newcommand{\bb}[1]{\mathbf{#1}}
\newcommand{\bx}{\bb{x}}
\newcommand{\by}{\bb{y}}
\newcommand{\bz}{\bb{z}}
\newcommand{\KL}[2]{\mathcal{D}_{\text{KL}}\left[#1 \| #2\right]}$
Originally defined in the field of information theory, the _Kullback-Leibler (KL) divergence_ (usually noted $\KL{p(\bx)}{q(\bx)}$) is a dissimilarity measure between two probability distributions $p(\bx)$ and $q(\bx)$. In the view of information theory, it can be understood as the cost in number of bits necessary for coding samples from $p(\bx)$ by using a code optimized for $q(\bx)$ rather than the code optimized for $p(\bx)$. In the view of probability theory, it represents the amount of information lost when we use $q(\bx)$ to approximate the true distribution $p(\bx)$. %that explicit the cost incurred if events were generated by $p(\bx)$ but charged under $q(\bx)$

Given two probability distributions $p(\bx)$ and $q(\bx)$, the Kullback-Leibler divergence of $q(\bx)$ _from_ $p(\bx)$ is defined to be
\begin{equation}
    \KL{p(\bx)}{q(\bx)}=\int_{\R} p(\bx) \log \frac{p(\bx)}{q(\bx)}d\bx
\end{equation}

Note that this dissimilarity measure is \textit{asymmetric}, therefore, we have
\begin{equation}
    \KL{p(\bx)}{q(\bx)}\neq \KL{q(\bx)}{p(\bx)}
\end{equation}
This asymmetry also describes an interesting behavior of the KL divergence, depending on the order to which it is evaluated. The KL divergence can either be a _mode-seeking_ or _mode-coverage} measure.

## Variational auto-encoders

As we have seen in the previous AE course, VAEs are also a form generative models. However, they are defined from a more sound probabilistic perspective. to find the underlying probability distribution of the data $p(\mathbf{x})$ based on a set of examples in $\mathbf{x}\in\mathbb{R}^{d_{x}}$. To do so, we consider *latent variables* defined in a lower-dimensional space $\mathbf{z}\in\mathbb{R}^{d_{z}}$ ($d_{z} \ll d_{x}$) with the joint probability distribution $p(\mathbf{x}, \mathbf{z}) = p(\mathbf{x} \vert \mathbf{z})p(\mathbf{z})$. Unfortunately, for complex distributions this integral is too complex and cannot be found in closed form.

### Variational inference

The idea of *variational inference* (VI) allows to solve this problem through *optimization* by assuming a simpler approximate distribution $q_{\phi}(\mathbf{z}\vert\mathbf{x})\in\mathcal{Q}$ from a family $\mathcal{Q}$ of approximate densities. Hence, the goal is to minimize the difference between this approximation and the real distribution. Therefore, this turns into the optimization problem of minimizing the Kullback-Leibler (KL) divergence between the parametric approximation and the original density

$$
q_{\phi}^{*}(\mathbf{z}\vert \mathbf{x})=\text{argmin}_{q_{\phi}(\mathbf{z} \vert \mathbf{x})\in\mathcal{Q}} \mathcal{D}_{KL} \big[ q_{\phi}\left(\mathbf{z} \vert \mathbf{x}\right) \parallel p\left(\mathbf{z} \vert \mathbf{x}\right) \big]
\tag{2}
$$

By developing this KL divergence and re-arranging terms (the detailed development can be found in [3](#reference1)), we obtain

$$
\log{p(\mathbf{x})} - D_{KL} \big[ q_{\phi}(\mathbf{z} \vert \mathbf{x}) \parallel p(\mathbf{z} \vert \mathbf{x}) \big] =
\mathbb{E}_{\mathbf{z}} \big[ \log{p(\mathbf{x} \vert \mathbf{z})}\big] - D_{KL} \big[ q_{\phi}(\mathbf{z} \vert \mathbf{x}) \parallel p(\mathbf{z}) \big]
\tag{3}
$$

This formulation describes the quantity we want to maximize $\log p(\mathbf{x})$ minus the error we make by using an approximate $q$ instead of $p$. Therefore, we can optimize this alternative objective, called the *evidence lower bound* (ELBO)

$$
\begin{equation}
\mathcal{L}_{\theta, \phi} = \mathbb{E} \big[ \log{ p_\theta (\mathbf{x|z}) } \big] - \beta \cdot D_{KL} \big[ q_\phi(\mathbf{z|x}) \parallel p_\theta(\mathbf{z}) \big]
\end{equation}
\tag{4}
$$

We can see that this equation involves $q_{\phi}(\mathbf{z} \vert \mathbf{x})$ which *encodes* the data $\mathbf{x}$ into the latent representation $\mathbf{z}$ and a *decoder* $p(\mathbf{x} \vert \mathbf{z})$, which allows generating a data vector $\mathbf{x}$ given a latent configuration $\mathbf{z}$. Hence, this structure defines the *Variational Auto-Encoder* (VAE).

The VAE objective can be interpreted intuitively. The first term  increases the likelihood of the data generated given a configuration of the latent, which amounts to minimize the *reconstruction error*. The second term represents the error made by using a simpler posterior distribution $q_{\phi}(\mathbf{z} \vert \mathbf{x})$ compared to the true prior $p_{\theta}(\mathbf{z})$. Therefore, this *regularizes* the choice of approximation $q$ so that it remains close to the true posterior distribution [3].

### Reparametrization trick

Now, while this formulation has some very interesting properties, it involves sampling operations, where we need to draw the latent point $\mathbf{z}$ from the distribution $q_{\phi}(\mathbf{z}\vert\mathbf{x})$.  The simplest choice for this variational approximate posterior is a multivariate Gaussian with a diagonal covariance structure (which leads to independent Gaussians on every dimension, called the *mean-field* family) so that
$$
\text{log}q_\phi(\mathbf{z}\vert\mathbf{x}) = \text{log}\mathcal{N}(\mathbf{z};\mathbf{\mu}^{(i)},\mathbf{\sigma}^{(i)})
\tag{5}
$$
where the mean $\mathbf{\mu}^{(i)}$ and standard deviation $\mathbf{\sigma}^{(i)}$ of the approximate posterior are different for each input point and are produced by our encoder parametrized by its variational parameters $\phi$. Now the KL divergence between this distribution and a simple prior $\mathcal{N}(\mathbf{0}, \mathbf{I})$ can be very simply obtained with
$$
D_{KL} \big[ q_\phi(\mathbf{z|x}) \parallel \mathcal{N}(\mathbf{0}, \mathbf{I}) \big] = \frac{1}{2}\sum_{j=1}^{D}\left(1+\text{log}((\sigma^{(i)}_j)^2)+(\mu^{(i)}_j)^2+(\sigma^{(i)}_j)^2\right)
\tag{6}
$$

While this looks convenient, we will still have to perform gradient descent through a sampling operation, which is non-differentiable. To solve this issue, we can use the *reparametrization trick*, which takes the sampling operation outside of the gradient flow by considering $\mathbf{z}^{(i)}=\mathbf{\mu}^{(i)}+\mathbf{\sigma}^{(i)}\odot\mathbf{\epsilon}^{(l)}$ with $\mathbf{\epsilon}^{(l)}\sim\mathcal{N}(\mathbf{0}, \mathbf{I})$

## VAE implementation

As we have seen, VAEs can be simply implemented by decomposing the above series of operations into an `encoder` which represents the distribution $q_\phi(\mathbf{z}\vert\mathbf{x})$, from which we will sample some values $\tilde{\mathbf{z}}$ (using the reparametrization trick) and compute the Kullback-Leibler (KL) divergence. Then, we use these values as input to a `decoder` which represents the distribution $p_\theta(\mathbf{x}\vert\mathbf{z})$ so that we can produce a reconstruction $\tilde{\mathbf{x}}$ and compute the reconstruction error.

Therefore, we can define the VAE based on our previous implementation of the AE that we recall here

```python
class AE(nn.Module):
    def __init__(self, encoder, decoder, encoding_dim):
        super(AE, self).__init__()
        self.encoding_dims = encoding_dim
        self.encoder = encoder
        self.decoder = decoder

    def forward(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded
```

In order to move to a probabilistic version, we need to add the latent space sampling mechanism, and change the behavior of our `call` function. This process is implemented in the following `VAE` class.

Note that we purposedly rely on an implementation of the `encode` function where the `encoder` first produces an intermediate representation of size `encoder_dims`. Then, this representation goes through two separate functions for encoding $\mathbf{\mu}$ and $\mathbf{\sigma}$. This provides a clearer implementation but also the added bonus that we can ensure that $\mathbf{\sigma} > 0$

```python
class VAE(AE):

    def __init__(self, encoder, decoder, encoding_dims, latent_dims):
        super(VAE, self).__init__(encoder, decoder, encoding_dims)
        self.latent_dims = latent_dims
        self.mu = nn.Sequential(nn.Linear(self.encoding_dims, self.latent_dims), nn.ReLU())
        self.sigma = nn.Sequential(nn.Linear(self.encoding_dims, self.latent_dims), nn.Softplus())

    def encode(self, x):

        ######################
        # YOUR CODE GOES HERE
        ######################

        return mu, sigma

    def decode(self, z):
        return self.decoder(z)

    def forward(self, x):
        # Encode the inputs
        z_params = self.encode(x)
        # Obtain latent samples and latent loss
        z_tilde, kl_div = self.latent(x, z_params)
        # Decode the samples
        x_tilde = self.decode(z_tilde)
        return x_tilde.reshape(-1, 1, 28, 28), kl_div

    def latent(self, x, z_params):

        ######################
        # YOUR CODE GOES HERE
        ######################

        return z, kl_div
```

Now the interesting aspect of VAEs is that we can define any parametric function as `encoder` and `decoder`, as long as we can optimize them. Here, we will rely on simple feed-forward neural networks, but these can be largely more complex (with limitations that we will discuss later in the tutorial).

```python
def construct_encoder_decoder(nin, n_latent = 16, n_hidden = 512, n_classes = 1):
    # Encoder network
    encoder = nn.Sequential(
          nn.Flatten(),
          nn.Linear(nin, n_hidden), nn.ReLU(),
          nn.Linear(n_hidden, n_hidden), nn.ReLU(),
          nn.Linear(n_hidden, n_hidden), nn.ReLU(),
    )
    # Decoder network
    decoder = nn.Sequential(
          nn.Linear(n_latent, n_hidden), nn.ReLU(),
          nn.Linear(n_hidden, n_hidden), nn.ReLU(),
          nn.Linear(n_hidden, nin * n_classes), nn.Sigmoid()
    )
    return encoder, decoder
```

### Evaluating the error

In the definition of the `VAE` class, we directly included the computation of the $D_{KL}$ term to regularize our latent space. However, remember that the complete loss of equation (4) also contains a *reconstruction loss* which compares our reconstructed output to the original data.

While there are several options to compare the error between two elements, there are usually two preferred choices among the generative literature depending on how we consider our problem
1. If we consider each dimension (pixel) to be a binary unit (following a Bernoulli distribution), we can rely on the `binary cross entropy` between the two distributions
2. If we turn our problem to a set of classifications, where each dimension can belong to a given set of *intensity classes*, then we can compute the `multinomial loss` between the two distributions

In the following, we define both error functions and regroup them in the `reconstruction_loss` call (depending on the `num_classes` considered). However, as the `multinomial loss` requires a large computational overhead, and for the sake of simplicity, we will train all our first models by relying on the `binary cross entropy`

```python
# Reconstruction criterion
recons_criterion = torch.nn.MSELoss(reduction='sum')

def compute_loss(model, x):

    ######################
    # YOUR CODE GOES HERE
    ######################

    return full_loss

def train_step(model, x, optimizer):
    # Compute the loss.
    loss = compute_loss(model, x)
    # Before the backward pass, zero all of the network gradients
    optimizer.zero_grad()
    # Backward pass: compute gradient of the loss with respect to parameters
    loss.backward()
    # Calling the step function to update the parameters
    optimizer.step()
    return loss
```

### Optimizing a VAE on a real dataset

For this tutorial, we are going to take a quick shot at a real-life problem by trying to train our VAEs on the `FashionMNIST` dataset. This dataset can be natively used in PyTorch by relying on the `torchvision.datasets` classes as follows

```python
dataset_dir = './data'
# Going to use 80%/20% split for train/valid
valid_ratio = 0.2
# Load the dataset for the training/validation sets
train_valid_dataset = torchvision.datasets.FashionMNIST(root=dataset_dir, train=True, transform=torchvision.transforms.ToTensor(), download=True)
# Split it into training and validation sets
nb_train = int((1.0 - valid_ratio) * len(train_valid_dataset))
nb_valid =  int(valid_ratio * len(train_valid_dataset))
train_dataset, valid_dataset = torch.utils.data.dataset.random_split(train_valid_dataset, [nb_train, nb_valid])
# Load the test set
test_dataset = torchvision.datasets.FashionMNIST(root=dataset_dir, transform=torchvision.transforms.ToTensor(),train=False)
# Prepare
num_threads = 4     # Loading the dataset is using 4 CPU threads
batch_size  = 128   # Using minibatches of 128 samples
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=batch_size, num_workers=num_threads)
valid_loader = torch.utils.data.DataLoader(dataset=valid_dataset, batch_size=batch_size, shuffle=False, num_workers=num_threads)
test_loader = torch.utils.data.DataLoader(dataset=test_dataset,batch_size=batch_size,shuffle=False,num_workers=num_threads)
```

The `FashionMNIST` dataset is composed of simple 28x28 black and white images of different items of clothings (such as shoes, bags, pants and shirts). We put a simple function here to display one batch of the test set (note that we keep a fixed batch from the test set in order to evaluate the different variations that we will try in this tutorial).

```python
print("The train set contains {} images, in {} batches".format(len(train_loader.dataset), len(train_loader)))
print("The validation set contains {} images, in {} batches".format(len(valid_loader.dataset), len(valid_loader)))
print("The test set contains {} images, in {} batches".format(len(test_loader.dataset), len(test_loader)))
nsamples = 10
classes_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal','Shirt', 'Sneaker', 'Bag', 'Ankle boot']
imgs_test, labels = next(iter(test_loader))
fig = plt.figure(figsize=(20,5))
for i in range(nsamples):
    ax = plt.subplot(1,nsamples, i+1)
    plt.imshow(imgs_test[i, 0, :, :], vmin=0, vmax=1.0, cmap=matplotlib.cm.gray)
    ax.set_title("{}".format(classes_names[labels[i]]), fontsize=15)
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
```

Now based on our proposed implementation, the optimization aspects are defined in a very usual way

```python
# Using Bernoulli or Multinomial loss
num_classes = 1
# Number of hidden and latent
n_hidden = 512
n_latent = 2
# Compute input dimensionality
nin = imgs_test.shape[2] * imgs_test.shape[3]
# Construct encoder and decoder
encoder, decoder = construct_encoder_decoder(nin, n_hidden = n_hidden, n_latent = n_latent, n_classes = num_classes)
# Build the VAE model
model = VAE(encoder, decoder, n_hidden, n_latent)
# Construct the optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
```

Now all that is left to do is train the model. We define here a `train_vae` function that we will reuse along the future implementations and variations of VAEs and flows. Note that this function is set to run for only a very few number of `epochs` and also most importantly, *only considers a subsample of the full dataset at each epoch*. This option is just here so that you can test the different models very quickly on any CPU or laptop.

```python
def generate_and_save_images(model, epoch, test_sample):
    predictions, _ = model(test_sample)
    fig = plt.figure(figsize=(4, 4))
    for i in range(predictions.shape[0]):
        plt.subplot(4, 4, i + 1)
        plt.imshow(predictions[i, 0, :, :].detach(), cmap='gray')
        plt.axis('off')
    # Tight_layout minimizes the overlap between 2 sub-plots
    #plt.savefig('image_at_epoch_{:04d}.png'.format(epoch))
    plt.show()

epochs=50
test_sample = imgs_test[0:16, :, :, :]
for epoch in range(1, epochs + 1):
    full_loss = torch.Tensor([0])
    # Forward pass: compute predicted y by passing x to the model.
    for i, (x, _) in enumerate(train_loader):
        full_loss += train_step(model, x, optimizer)
    #for i, (x, _) in enumerate(valid_loader):
    #    train_step(model, x, optimizer)
    print('Epoch: {}, Test set ELBO: {}'.format(epoch, full_loss))
    generate_and_save_images(model, epoch, test_sample)
```

### Evaluating generative models

In order to evaluate our upcoming generative models, we will rely on the computation of the Negative Log-Likelihood. This code for the following `evaluate_nll_bpd` is inspired by the [Sylvester flow repository](https://github.com/riannevdberg/sylvester-flows)

```python
from scipy.special import logsumexp

def evaluate_nll_bpd(data_loader, model, batch = 500, R = 5):
    # Set of likelihood tests
    likelihood_test = []
    # Go through dataset
    for batch_idx, (x, _) in enumerate(data_loader):
        for j in range(x.shape[0]):
            a = []
            for r in range(0, R):
                cur_x = x[j].unsqueeze(0)
                # Repeat it as batch
                x = cur_x.expand(batch, *cur_x.size()[1:]).contiguous()
                x = x.view(batch, -1)
                x_tilde, kl_div = model(x)
                rec = reconstruction_loss(x_tilde, x, average=False)
                a_tmp = (rec + kl_div)
                a.append(- a_tmp.cpu().data.numpy())
            # calculate max
            a = np.asarray(a)
            a = np.reshape(a, (a.shape[0] * a.shape[1], 1))
            likelihood_x = logsumexp(a)
            likelihood_test.append(likelihood_x - np.log(len(a)))
    likelihood_test = np.array(likelihood_test)
    nll = - np.mean(likelihood_test)
    # Compute the bits per dim (but irrelevant for binary data)
    bpd = nll / (np.prod(nin) * np.log(2.))
    return nll, bpd
```

Now we can evaluate our VAE model more formally as follows.

```python
# Plot final loss
plt.figure()
plt.plot(losses_kld[:, 0].numpy());
# Evaluate log-likelihood and bits per dim
nll, _ = evaluate_nll_bpd(test_loader, model)
print('Negative Log-Likelihood : ' + str(nll))
```

We can also evaluate the latent space of our model, which should be organized (being the overall point of using a VAE instead of a common AE).

```python
x = np.linspace(-3, 3, 8)
y = np.linspace(-3, 3, 8)
fig = plt.figure(figsize=(10, 8))
for i in range(8):
    for j in range(8):
        plt.subplot(8, 8, (i * 8) + j + 1)
        final_tensor = torch.zeros(2)
        final_tensor[0] = x[i]
        final_tensor[1] = y[j]
        plt.imshow(model.decode(final_tensor).detach().reshape(28, 28), cmap='gray')
        plt.axis('off')
```

### Limitations of VAEs - (**exercise**)

Although VAEs are extremely powerful tools, they still have some limitations. Here we list the three most important and known limitations (all of them are still debated and topics of active research).
1. **Blurry reconstructions.** As can be witnessed directly in the results of the previous vanilla VAE implementation, the reconstructions appear to be blurry. The precise origin of this phenomenon is still debated, but the proposed explanation are
    1. The use of the KL regularization
    2. High variance regions of the latent space
    3. The reconstruction criterion (expectation)
    4. The use of simplistic latent distributions
2. **Posterior collapse.** The previous *blurry reconstructions* issue can be mitigated by using a more powerful decoder. However, relying on a decoder with a large capacity causes the phenomenon of *posterior collapse* where the latent space becomes useless. A nice intuitive explanation can be found [here](https://ermongroup.github.io/blog/a-tutorial-on-mmd-variational-autoencoders/)
3. **Simplistic Gaussian approximation**. In the derivation of the VAE objective, recall that the KL divergence term needs to be computed analytically. Therefore, this forces us to rely on quite simplistic families. However, the Gaussian family might be too simplistic to model real world data

In the present tutorial, we show how normalizing flows can be used to mostly solve the third limitation, while also adressing the two first problems. Indeed, we will see that normalizing flows also lead to sharper reconstructions and also act on preventing posterior collapse

## Improving the quality of VAEs

As we discussed in the previous section, several known issues have been reported when using the vanilla VAE implementation. We listed some of the major issues as being
1. **Blurry reconstructions.**
2. **Posterior collapse.**
3. **Simplistic Gaussian approximation**.

Here, we discuss some recent developments that were proposed in the VAE literature and simple adjustments that can be made to (at least partly) alleviate these issues. However, note that some more advanced proposals such as PixelVAE [5](#reference1) and VQ-VAE [6](#reference1) can lead to wider increases in quality

### Reducing the bluriness of reconstructions

In this tutorial, we relied on extremely simple decoder functions, to show how we could easily define VAEs and normalizing flows together. However, the capacity of the decoder obviously directly influences the quality of the final reconstruction. Therefore, we could address this issue naively by using deep networks and of course convolutional layers as we are currently dealing with images.

First you need to construct a more complex encoder and decoder

```python
def construct_encoder_decoder_complex(nin, n_latent = 16, n_hidden = 512, n_params = 0, n_classes = 1):

    ######################
    # YOUR CODE GOES HERE
    ######################

    # Encoder network
    encoder = ...
    # Decoder network
    decoder = ...
    return encoder, decoder
```

### Preventing posterior collapse with Wasserstein-VAE-MMD (InfoVAE)

As we discussed earlier, the reason behind posterior collapse mostly relates to the KL divergence criterion (a nice intuitive explanation can be found [here](https://ermongroup.github.io/blog/a-tutorial-on-mmd-variational-autoencoders/). This can be mitigated by relying on a different criterion, such as regularizing the latent distribution by using the *Maximum Mean Discrepancy* (MMD) instead of the KL divergence. This model was independently proposed as the *InfoVAE* and later also as the *Wasserstein-VAE*.

Here we provide a simple implementation of the `InfoVAEMMD` class based on our previous implementations.

```python
######################
# YOUR CODE GOES HERE
######################

def compute_kernel(x, y):
    return ...

def compute_mmd(x, y):
    return ...

class InfoVAEMMD(VAE):

    def __init__(self, encoder, decoder):
        super(InfoVAEMMD, self).__init__(encoder, decoder)

    def latent(self, x, z_params):
        return ...
```

### Putting it all together

Here we combine all these ideas (except for the MMD, which is not adequate as the flow definition already regularizes the latent space without the KL divergence) to perform a more advanced optimization of the dataset. Hence, we will rely on the complex encoder and decoder with gated convolutions, the multinomial loss and the normalizing flows in order to improve the overall quality of our reconstructions.

```python
# Size of latent space
n_latent = 16
# Number of hidden units
n_hidden = 256
# Rely on Bernoulli or multinomial
num_classes = 128

######################
# YOUR CODE GOES HERE
######################

# Construct encoder and decoder
encoder, decoder = ...
# Create VAE or (InfoVAEMMD - WAE) model
model_flow_p = ...
# Create optimizer algorithm
optimizer = ...
# Add learning rate scheduler
scheduler = ...
# Launch our optimization
losses_flow_param = ...
```

*NB*: It seems that the multinomial version have a hard time converging. Although I only let this run for 200 epochs and only for a subsampling of 5000 examples, it might need more time, but this might also come from a mistake somewhere in my code ... If you spot something odd please let me know :)

### References

[1] Rezende, Danilo Jimenez, and Shakir Mohamed. "Variational inference with normalizing flows." _arXiv preprint arXiv:1505.05770_ (2015). [link](http://arxiv.org/pdf/1505.05770)

[2] Kingma, Diederik P., Tim Salimans, and Max Welling. "Improving Variational Inference with Inverse Autoregressive Flow." _arXiv preprint arXiv:1606.04934_ (2016). [link](https://arxiv.org/abs/1606.04934)

[3] Kingma, D. P., & Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. (2013). [link](https://arxiv.org/pdf/1312.6114)

[4] Rezende, D. J., Mohamed, S., & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082. (2014). [link](https://arxiv.org/pdf/1401.4082)

[5] Gulrajani, I., Kumar, K., Ahmed, F., Taiga, A. A., Visin, F., Vazquez, D., & Courville, A. (2016). Pixelvae: A latent variable model for natural images. arXiv preprint arXiv:1611.05013. [link](https://arxiv.org/pdf/1611.05013)

[6] Van den Oord, A., & Vinyals, O. (2017). Neural discrete representation learning. In NIPS 2017 (pp. 6306-6315). [link](http://papers.nips.cc/paper/7210-neural-discrete-representation-learning.pdf)

### Inspirations and resources

https://blog.evjang.com/2018/01/nf1.html
https://github.com/ex4sperans/variational-inference-with-normalizing-flows
https://akosiorek.github.io/ml/2018/04/03/norm_flows.html
https://github.com/abdulfatir/normalizing-flows
https://github.com/riannevdberg/sylvester-flows


# Calculate near surface RH% using ERA-interim fields

* 2-m dew point
* 2-m temperature
* surface pressure

Once the yearly RH files are made, merge these data into a single file and put into the era merged time directory, then, regrid that file and place in the common grid file, such that this newly created variable has the same handling as variables processed by Python/format_raw_era_data.py

NOTE: RH was calculated from raw, native (0.75 x 0.75 deg grid) era-interim nc data. These data live in a different project directory (metSpreadData).

Link to documentations and instructions on how to calculate near surface relative humidity using ecmwf era-interim:
https://www.ecmwf.int/en/faq/do-era-datasets-contain-parameters-near-surface-humidity

\begin{align}
RH=100\frac{es(T_{d})}{es(T)}
\end{align}

### Saturation Specific Humidity

Saturation specific humidity is expressed as a function of saturation water vapor pressure as:

\begin{align}
q_{sat} & = \frac{e_{sat}(T)\frac{R_{dry}}{R_{vapor}}}{p-(1-\frac{R_{dry}}{R_{vapor}})e_{sat}(T)}
\end{align}

where the saturation water vopor pressure ($e_{sat}$) is expressed with the Teten's formula:
\begin{align}
e_{sat}(T) & = a_{1}exp(a_{3}\frac{T - T_{0}}{T-a_{4}})\\
\\
a_{1} & = 611.21Pa \\
a_{3} & = 17.502 \\
a_{4} & = 32.19 \\
T_{0} & = 273.16
\end{align}

```python
import numpy as np

def calculat_svp(T, P):
    """Calculates and returns saturation vapor pressure (svp)"""
    # constants for saturation over water
    a1 = 611.21 # Pa
    a3 = 17.502
    a4 = 32.19 # K
    To = 273.16 # K
    R_dry = 287. # J/kg/K gas constant of dry air
    R_vap = 461. # J/kg/K gas constant for water vapor

    e_sat = a1 * np.exp( a3 * (T-To)/(T-a4) )

    # Saturation specific humidity (eqn 7.4)
    top = e_sat * R_dry / R_vap
    bottom = P - (1. - R_dry / R_vap) * e_sat
    q_sat = top / bottom

    return q_sat
```

### Create 3D RH arrays

```python
from netCDF4 import Dataset
import os
from matplotlib import pylab as plt
dataDir = os.path.join("..","..","metSpreadData","ERA-Interim")

def get_nc(var, year):
    """
    Loads an era-interim netcdf file. Very simple.
    """
    loadFile = os.path.join(dataDir, var + "_" + str(year) + ".nc")
    nc = Dataset(loadFile)
    vals = nc.variables[var][:]
    t = nc.variables["time"][:]
    lon = nc.variables["longitude"][:]
    lat = nc.variables["latitude"][:]
    nc.close()
    return vals, t, lon, lat

def write_RH_nc(RH, t, x, y, year, dataDir):
    """
    Writes a relative humidity netcdf file.
    """

    outputFile = os.path.join(dataDir, "RH_" + str(year) + ".nc")

    ncFile = Dataset(outputFile, 'w', format='NETCDF4')
    ncFile.description = 'Relative Humidity (saturation pressure relative to water, Tetons eq.)'
    ncFile.location = 'Global'
    ncFile.createDimension('time',  len(t) )
    ncFile.createDimension('latitude', len(y) )
    ncFile.createDimension('longitude', len(x) )

    VAR_ = ncFile.createVariable("RH", 'f4',('time', 'latitude','longitude'))
    VAR_.long_name = "Relative Humidity"
    VAR_.units = "%"

    # Create time variable
    time_ = ncFile.createVariable('time', 'i4', ('time',))
    time_.units = "hours since 1900-01-01 00:00:0.0"
    time_.calendar = "gregorian"

    # create lat variable
    latitude_ = ncFile.createVariable('latitude', 'f4', ('latitude',))
    latitude_.units = 'degrees_north'

    # create longitude variable
    longitude_ = ncFile.createVariable('longitude', 'f4', ('longitude',))
    longitude_.units = 'degrees_east'

    # Write the actual data to these dimensions
    VAR_[:]       = RH
    time_[:]      = t
    latitude_[:]  = y
    longitude_[:] = x

    ncFile.close()
```

Generate the RH yearly files!

```python
years = range(1983, 2017+1)
for year in years :
    print("Making RH file for %i " % year)
    # Get the grids needed to calculate specific humidity
    t2m,t,x,y = get_nc("t2m", year) # 2 meter temperature
    d2m,t,x,y = get_nc("d2m", year) # 2 meter dew point
    sp,t,x,y  = get_nc("sp", year)  # Surface pressure

    RH = calculat_svp(d2m, sp) / calculat_svp(t2m, sp) * 100.
    write_RH_nc(RH, t, x, y, year, dataDir)
```

    Making RH file for 1983
    Making RH file for 1984
    Making RH file for 1985
    Making RH file for 1986
    Making RH file for 1987
    Making RH file for 1988
    Making RH file for 1989
    Making RH file for 1990
    Making RH file for 1991
    Making RH file for 1992
    Making RH file for 1993
    Making RH file for 1994
    Making RH file for 1995
    Making RH file for 1996
    Making RH file for 1997
    Making RH file for 1998
    Making RH file for 1999
    Making RH file for 2000
    Making RH file for 2001
    Making RH file for 2002
    Making RH file for 2003
    Making RH file for 2004
    Making RH file for 2005
    Making RH file for 2006
    Making RH file for 2007
    Making RH file for 2008
    Making RH file for 2009
    Making RH file for 2010
    Making RH file for 2011
    Making RH file for 2012
    Making RH file for 2013
    Making RH file for 2014
    Making RH file for 2015
    Making RH file for 2016
    Making RH file for 2017

Show the last month for the last year output as an example of what these values and output look like.
Make sure the dry places in the world have lower RH values.

```python
plt.figure(dpi=150)
plt.pcolor(x,y,RH[0,:,:])
plt.title("Example of RH% output for January 2017")
plt.colorbar(extend="both", label="RH%")
plt.xlabel("Longitude")
plt.ylabel("Latitude")
plt.show()
```

Merge yearly files and regrid
- TODO: the code below is not working in a notebook, something strange with the call to Cdo()
- These commands were issued at the command line using cdo on 11/24/2018.
import globfrom cdo import *   # python version
cdo = Cdo()common_grid_txt  = os.path.join(dataDir, 'COMMON_GRID.txt')
files_to_merge   = glob.glob(os.path.join(dataDir, "RH_*"))
merged_time_file = os.path.join(dataDir, 'merged_time', 'RH_1983-2017.nc')
common_grid_out  = os.path.join(dataDir, 'merged_t_COMMON_GRID', 'RH_1983-2017.nc')cdo.mergetime(input=" ".join(files_to_merge), output=merged_time_file, options="-b F64")cdo.remapbil(common_grid_txt, input=merged_time_file, output=common_grid_out, options="-b F64")


# Simulation study for the Functional CLT for TDA on graphs

We illustrate the functional CLT at hand of two prototypical examples, namely the [Gilbert graph](https://projecteuclid.org/euclid.aoms/1177706098) and the [Directed spanning tree](https://projecteuclid.org/euclid.aoap/1171377186).

For the simulation study, we rely on a Poisson point process with intensity 2 in a $15 \times 5$ observation window.

```R
rm(list = ls())

library(ggplot2)
library(spatstat)
library(TDA)
```

    Loading required package: spatstat.data

    Loading required package: nlme

    Loading required package: rpart

    spatstat 1.62-2       (nickname: ‘Shape-shifting lizard’)
    For an introduction to spatstat, type ‘beginner’

    Note: spatstat version 1.62-2 is out of date by more than 3 months; we recommend upgrading to the latest version.

```R
seed <- 1
set.seed(seed)

width = 20
height = 10
options(repr.plot.width = width,
        repr.plot.height = height)

#number of samples
nsim <- 1e3

win = owin(c(0,15), c(0,5))

ppps <- rpoispp(2,
                    win = win,
                    nsim = nsim)

ppp <- ppps[[1]]
qplot(x = ppp$x,
      y = ppp$y,
      size = I(2.5)) +
        theme_void() +
        theme(plot.margin = margin(10, 10, 10, 10)) +
        coord_fixed() +
        theme(panel.border = element_rect(fill = NA))
```

## Gilbert graph

First, we start by discussing the results for the Gilbert graph. This corresponds to the standard choice of using the Čech complex for growing disks. To begin with, we take one realization andaillustrate the persistence diagrams for 0- and 1-dimensional features.

```R
library(grid)
library(gridExtra)

#0-features
plots0 <- lapply( list(ppp),
                function(pp){
                    pd <- alphaComplexDiag(cbind(pp$x, pp$y))$diag
                            ggplot(data.frame(pd[pd[,'dimension'] == 0,
                                                 2:3][-1,]),
                            aes(x = Birth, y = Death)) +
                            geom_violin() +
                            geom_point(size = 1) +
                            scale_y_continuous(limits = c(0, 0.5)) +
                            theme_bw()
                    })

#1-features
plots1 <- lapply(list(ppp),
                function(pp){
                    pd <- alphaComplexDiag(cbind(pp$x, pp$y))$diag
                            ggplot(data.frame(pd[pd[,'dimension'] == 1, 2:3]),
                            aes(x = Birth, y = Death)) +
                            geom_point(size = 1) +
                            scale_x_continuous(limits = c(0, .8)) +
                            scale_y_continuous(limits = c(0, 1.3)) +
                            theme_bw()
                    })

grid.arrange(grobs = list(plots0[[1]],
                           plots1[[1]]),
            ncol = 2, nrow = 1)
```

### Test statistics

The functional limit theorem shows that continuous functionals of the persistence diagram are asymptotically Gaussian. We illustrate this effect at hand of two specific statistics for 0- and 1-dimensional features, that were also considered in the setting of [$M$-bounded features](https://arxiv.org/abs/1906.07608).

For features in dimension 0, we look at the integrated number of cluster deaths until  a time $r_{\mathsf c}$, i.e.,
$$\int_0^{r_{\mathsf c}} \mathsf{PD}^0(\mathcal P_n)([0, d]) {\rm d}d.$$
Since it is a continuous functional of the persistence diagram, it is asymptotically normal, so that we deduce from the functional CLT that it becomes Gaussian in large windows. When dealing with data, the intensity needs to be estimated and [it was found](https://projecteuclid.org/euclid.ejs/1582534816) that the following intensity-adapted variant leads to superior test powers:
\begin{align}
        \label{t0Def}
        T_{\mathsf C} = \frac1{\sqrt{\hat\lambda}|W|}\int_0^{r_{\mathsf C} / \sqrt{\hat\lambda}} \mathsf{PD}^0(\mathcal P_n)([0, d]) {\rm d}d.
\end{align}

```R
t_vals0 <- seq(0, .299, by = 1e-3)

tda0_raw <- function(coords){
    #' Death times of a point pattern
    #'
    #' @param coords coordinates of point pattern

    diag <- alphaComplexDiag(coords)$diagram
    diag_df <-  data.frame(dimension = diag[,1],
                           death = diag[,3])
    sort(diag_df[diag_df$dimension == 0, 'death'])
}

tda0 <- function(X,
                 r = 0,
                 tx = t_vals0,
                 correction = NULL){
    #' Adapter of death times for spatstat scaled by number of points
    #'
    #' @param X point pattern
    #' @param r time point
    #' @param t_vals time values for which the accumulated death count is computed
    #' @param correction edge correction
    dt <- tda0_raw(cbind(X$x, X$y))

    #intensity
    intens <- X$n / area.owin(win)

    #rescale integration domain
    tx <- tx * sqrt(2 / intens)

    fv(data.frame(r = tx,
                  trans = findInterval(tx, dt) / (sqrt(intens) * area.owin(win))),
       valu = 'trans',
       labl = c('r', 'lab'),
       fname='Diag')
}
```

For 1-dimensional features, we rely on the [accumulated persistence function](https://www.tandfonline.com/doi/abs/10.1080/10618600.2019.1573686?journalCode=ucgs20). That is, we aggregate the life times of all loops with birth times in a time interval $[0, r_{\mathsf L}]$ with $r_{\mathsf L} \le r_{\mathsf f}$:
$$\int_{[0, r_{\mathsf L}] \times [0, r_{\mathsf f}]} (d - b) \mathsf{PD}^1(\mathcal P_n)({\rm d}b, {\rm d}d).$$

As before, adapting to the intensity results in the loop-based statistic
\begin{align}
        \label{t1Def}
        T_{\mathsf L} =  \frac1{\sqrt{\lambda}|W|}\int_{[0, r_{\mathsf L} / \sqrt \lambda] \times [0, r_{\mathsf f} / \sqrt \lambda]} (d - b) \mathsf{PD}^1(\mathcal P_n)({\rm d}b, {\rm d}d)
\end{align}

```R
t_vals1 <- seq(0, .499, by=5e-3)
ty1 <- seq(0, .299, by=1e-2)

tda1_raw <- function(coords,
                     max_bt = 10){
    #' accumulated persistence function of a point pattern
    #'
    #' @param coords coordinates of point pattern
    #' @param max_bt add maximum birth time to avoid out of bound error

    diag <- alphaComplexDiag(coords)$diagram
    diag_df <- data.frame(dimension = diag[,1],
                           birth = diag[,2],
                           life = (diag[,3] - diag[,2]))

    #only 1-dimensional features
    diag_df <- diag_df[diag_df$dimension == 1, 2:3]
    diag_df <- diag_df[order(diag_df$birth),]
    rownames(diag_df) <- c()
    rbind(diag_df, c(max_bt, max_bt))
}

tda1 <- function(X,
                 r = 0,
                 tx = t_vals1,
                 correction = NULL){
    #' Adapter of the accumulated persistence function for spatstat
    #'
    #' @param X point pattern
    #' @param r time point
    #' @param tx times at which the APF is computed

    tda1_vals <- tda1_raw(cbind(X$x, X$y))
    tda1_vals <- cbind(tda1_vals[,1], cumsum(tda1_vals$life))
    birth_locs = findInterval(tx, tda1_vals[,1])

    #intensity
    intens <- X$n / area.owin(win)

    #rescale integration domain
    tx <- tx * sqrt(2 / intens)

    fv(data.frame(r = tx,
                  trans = tda1_vals[1 + birth_locs, 2] / (sqrt(intens) * area.owin(win))),
      valu = 'trans',
      labl = c('r', 'lab'),
      fname='Diag')
}
```

### Asymptotic Gaussianity

By the functional CLT, the cluster- and loop-based statistics $T_{\mathsf C}$ and $T_{\mathsf L}$ are asymptotically Gaussian. We now illustrate that this distributional convergence becomes already clearly apparant on bounded sampling windows. To that end, we first compute the statistics on the iid samples of the Poisson process.

```R
library(parallel)

#compute summary statistics
funs_sum <- list(function(X) mean((tda0(X)$trans[1:100])),
                 function(X) mean(tda1(X)$trans))

null_stats <- lapply(funs_sum,
                    function(f) as.numeric(mclapply(ppps,
                                         function(X) f(X),
                                                    mc.cores = 36))
                    )

#normalize
null_means <- lapply(null_stats, mean)
null_sds <- lapply(null_stats, sd)
null_norms <- lapply(1:2,
                    function(i) data.frame(x = (null_stats[[i]] - null_means[[i]]) / null_sds[[i]])
                    )
```

The histograms and Q-Q-plots for the normalized statistics illustrate impressively that the Gaussian distribution is already clearly visible on a bounded sampling window.

```R
hists <- lapply(null_norms, function(norm) ggplot(norm, aes(x = x)) +
        geom_histogram(aes(y = ..density..),
                       color = 'black',
                       fill="white") +
        stat_function(fun = dnorm,
                      colour = "red",
                      size = 1.5) +
        theme_bw()
                )

grid.arrange(grobs = list(hists[[1]],
                           hists[[2]]),
            ncol = 2, nrow = 1)
```

```R
qplots <- lapply(null_norms,
                 function(norm) qplot(sample = x, data = norm) +
                    stat_qq_line() +
                    theme_bw()
                 )
grid.arrange(grobs = list(qplots[[1]],
                           qplots[[2]]),
            ncol = 2, nrow = 1)
```

### Goodness of fit tests

After illustrating that the test statistics $T_{\mathsf C}$ and $T_{\mathsf L}$ are asymptotically Gaussian, we now sketch how to derive goodness of fit tests. For this purpose, we introduce point patterns that are more clustered or more repulsive than the Poisson point process. To be more precise, the Matérn cluster process $\mathsf{MatC}(2, 0.5, 1)$ features a Poisson parent process with intensity 2 and generates a $\mathsf{Poi}(1)$ number of offspring uniformly in a disk of radius $0.1$ around each parent. The Strauss process $\mathsf{Str}(4.0, 0.6, 0.5)$ has interaction parameter $0.6$ and interaction radius $0.5$, respectively.

```R
seed <- 48
set.seed(seed)
nsim <- 1e3

time <- Sys.time()

ppps_val <- rpoispp(2,
                    win = win,
                    nsim = nsim)

mcps <- rMatClust(2, .5, 1,
                  win = win,
                  nsim = nsim)

strs <- rStrauss(4.0, .6, .5,
                 W = win,
                 nsim = nsim)
Sys.time() - time
```

    Time difference of 18.27153 secs

```R
library(gridExtra)

real <- 5

samples <-list(ppps_val[[real]],
               mcps[[real]],
               strs[[real]])

plots <- lapply(samples,
                function(pp) qplot(x = pp$x,
                                   y = pp$y,
                                   size = I(1.5)) +
                    theme_void() +
                    theme(plot.margin = margin(10, 10, 10, 10)) +
                    theme(panel.border = element_rect(fill = NA))
                )
grid.arrange(plots[[1]],
             plots[[2]],
             plots[[3]],
             clip = F,
             nrow = 3, ncol = 3)
```

We see that when drawing samples from the Poisson null model, the actual type 1 error is close to the nominal asymptotic 5 \% level. When moving to the type 2 errors, we see that the TDA-based tests often succeed in detecting deviations from the null hypothesis. More precisely, for the attractive point pattern, the null hypothesis is rejected in 44.4\% and 36.6\% of the samples by the tests based on $T_{\mathsf C}$ and $T_{\mathsf L}$. For the repulsive point pattern, the corresponding rates are 50.5\% and 47.4\%, respectively.

```R
p_val <- function(X,
                  stat,
                  null_mean,
                  null_sd){
    #' p-value for deviation test
    #'
    #' @param pp point pattern to be tested
    #' @param stat test statistics
    #' @param null_mean mean under the null model
    #' @param null_sd standard deviation under the null model
    z_val <- abs(stat(X) - null_mean) / null_sd
    2 * (1 - pnorm(z_val))
}
```

```R
alpha <- .05
pps_list <- list(ppps_val, mcps, strs)

do.call(rbind.data.frame,
        lapply(seq(length(funs_sum)),
               function(i) sapply(pps_list,
                                  function(pp_list) sum(
                                      mclapply(pp_list,
                                               function(pp) p_val(pp,
                                                                  funs_sum[[i]],
                                                                  null_means[[i]],
                                                                  null_sds[[i]]),
                                               mc.cores = 36
                                              ) < alpha)
                                  )
               )
       )
```

A data.frame: 2 × 3

c.53L..42L.	c.444L..505L.	c.366L..474L.

<int>	<int>	<int>

53	444	366

42	505	474

## Directed spanning tree

The random geometric graph from the previous section corresponds to the standard situation for building the Čech complex for a union of growing balls. Next, we illustrate the asymptotic normality for a prototypical example of a geometric tree network, namely the [directed spanning tree](https://projecteuclid.org/euclid.aoap/1171377186).

In this directed network model, the nodes are given by a spatial point process $X = \{X_i\}_{i \ge 1}$ and every node $X_i$ has precisely one outgoing arrow. More precisely, among all nodes to the right of $X_i$ we select the one at the smallest Euclidean distance.

```R
dsf <- function(X){
    #' Directed spanning forest
    #'
    #' @param X point pattern

    #order increasing in x-direction
    x_ord <- X$x[order(X$x)]
    y_ord <- X$y[order(X$x)]
    points_ord <-cbind(x_ord, y_ord)
    len <- length(x_ord)

    #distance matrix
    dists <- pairdist(points_ord)

    #outgoing edge
    neighbs <- sapply(1:(X$n - 1),
                      function(i) i + which.min(dists[i, (i+1):X$n])
                      )

    #ingoing edges
    in_neighbs <- lapply(1:X$n, function(x) list())
    for(i in 1:(X$n - 1))
    in_neighbs[[neighbs[i]]] <- append(in_neighbs[[neighbs[i]]],
                                                i)

    neighbs <- c(neighbs, X$n)
    list(points_ord,
         neighbs,
         in_neighbs)
}
```

Now, we illustrate the DSF on a Poisson point process.

```R
real <- 3
dsf_samp <- dsf(ppps[[real]])

pp_ord <- dsf_samp[[1]]
neighb_list <- dsf_samp[[2]]

df = data.frame(x = pp_ord[,1],
                y = pp_ord[,2],
                xend = pp_ord[,1][neighb_list],
                yend = pp_ord[,2][neighb_list]
                )

ggplot() +
geom_segment(data=df,
             mapping=aes(x=x, y=y, xend=xend, yend=yend),
             arrow=arrow(),
             size=1) +
    geom_point(data=df,
               mapping=aes(x=x, y=y),
               size=4) +
    theme_void()
```

Export the drawing to tikz.

```R
pic_tikz <- file("./long_time.tex")
pt_size <- 1
digits <- 2
width <-  15
height <- 5
x_offset <- 15

squeeze_height <- .3
y_offset <- squeeze_height * nrow(df) / 2 - height / 2

code <- c()

#order by birth
df <- df[order(df[,1]),]

for(i in seq(1, nrow(pp_ord), by = 5)){

code <- c(code, '\\begin{tikzpicture}')

    #bounding box
    pt <- paste0('\\draw[white] (', x_offset + width,
                 ',', squeeze_height * (nrow(df) + 1),
                 ') -- (', x_offset + width,
                 ',', -1,
                 ');')
    code <- c(code, pt)

    x_cur <- pp_ord[i, 1]

#dsf
for(j in 1:nrow(pp_ord)){
    ##pts
    pt <- paste0('\\fill (', format(round(pp_ord[j,1], digits), nsmall = digits),
                 ',', format(round(y_offset + pp_ord[j,2], digits), nsmall = digits),
                 ') circle (',pt_size,'pt);')
    code <- c(code, pt)

    ##arrows
    arr <- paste0('\\draw (', format(round(pp_ord[j,1], digits), nsmall = digits),
                 ',', format(round(y_offset + pp_ord[j,2], digits), nsmall = digits),
                 ') -- (', format(round(pp_ord[neighb_list[j],1], digits), nsmall = digits),
                 ',', format(round(y_offset + pp_ord[neighb_list[j],2], digits), nsmall = digits),
                 ');')

    code <- c(code, arr)
    }

##scanner
    cur_pt <- paste0('\\fill[red] (',format(round( x_cur, digits), nsmall = digits),
                 ',', format(round(y_offset + pp_ord[i,2], digits), nsmall = digits),
                 ') circle (', 3 * pt_size,'pt);')
    code <- c(code, cur_pt)

    #left dash
    dash <- paste0('\\draw[dashed] (', format(round( x_cur, digits), nsmall = digits),
                 ', ',y_offset,') -- (',format(round( x_cur, digits), nsmall = digits),
                 ',', y_offset + height,
                 ');' )
    code <- c(code, dash)

    #barcode

    ##births
    df_birth <- df[df[,1]  <= x_cur,]
    if(nrow(df_birth)>0){
    for(j in 1:nrow(df_birth)){
        bar <- paste0('\\draw[very thick] (', format(round(x_offset + df_birth[j,1], digits), nsmall = digits),
                 ',', squeeze_height *(nrow(df) - j),
                 ') -- (', format(round(x_offset + min(x_cur, df_birth[j,2]), digits), nsmall = digits),
                 ',', squeeze_height *(nrow(df) - j),
                 ');')
        code <- c(code, bar)
    }
        }

    #right dash
    dash_bar <- paste0('\\draw[dashed] (', format(round(x_offset + x_cur, digits), nsmall = digits),
                 ', ',squeeze_height * nrow(df),') -- (', format(round(x_offset + x_cur, digits), nsmall = digits),
                 ',', squeeze_height * (nrow(df) - nrow(df_birth)),
                 ');' )
    code <- c(code, dash_bar)

    code <- c(code, '\\end{tikzpicture}')

    }

#barcode

writeLines(code, pic_tikz)
close(pic_tikz)
```

Now, we compute the persistence diagram associated with the sub-level filtration of merging branches when scanning in the $x$-direction.

```R
pd_tree <- function(nodes,
                    nbs_in){
    #' Persistence diagram of tree. Input data should be ordered.
    #'
    #' @nodes node coordinates
    #' @param nbs_in in-neighbors

    comps <- matrix(,
                    nrow = 0,
                    ncol = 2)

    #birth and death times
    b_d <-  matrix(,
                    nrow = 0,
                    ncol = 2)

    for(i in 1:(length(nbs_in) )){
        #birth
        if(length(nbs_in[[i]])==0)
            comps <- rbind(comps, c(i,i))
        #death
        else{
            #which clusters are merged?
            merge_idxs <- match(c(nbs_in[[i]]), comps[,2])
            #oldest cluster
            idx_old <- which.min(comps[merge_idxs,1])

            #update surving component
            idx_surv <- merge_idxs[idx_old]
            comps[idx_surv,2] <-i

            #kill other components
            if(length(merge_idxs) > 1){
                bd_inc <- matrix(,
                                 nrow = length(merge_idxs[-idx_old]),
                                 ncol = 2)
                bd_inc[,1] <- comps[merge_idxs[-idx_old],1]
                bd_inc[,2] <- i
                b_d <- rbind(b_d, bd_inc)

                comps <- comps[-(merge_idxs[-idx_old]),,drop=FALSE]
                }
            }
        }
    #kill initial component
    b_d <- rbind(b_d, c(1, length(nodes)/2))
    cbind(nodes[b_d[,1],1],
          nodes[b_d[,2],1])
    }
```

Now, we plot the persistence diagram for the Poisson null model

```R
library(gridExtra)

points <- dsf(ppps[[3]])[[1]]
neighbs <- dsf(ppps[[3]])[[3]]

df <- data.frame(pd_tree(points,
                         neighbs))
colnames(df) <- c('Birth', 'Death')
dsf_plot <- ggplot(df, aes(x = Birth, y = Death)) +
            geom_abline(intercept = 0, slope = 1) +
            geom_point(size = 1) +
            scale_x_continuous(limits = c(0, 15)) +
            scale_y_continuous(limits = c(0, 15)) +
            theme_bw()
grid.arrange(dsf_plot ,
             clip = F,
             nrow = 3, ncol = 6)
```

To compute the the aggregated persistence function, we adapt the method from the Čech complex.

```R
t_vals0_net <- seq(0, 15.9, by=1e-1)
ty0_net <- seq(0, 9.99, by=1e-1)

tda0_raw_net <- function(tree,
                         max_lt = 5,
                         max_bt = 30){
    #' accumulated persistence function of a tree
    #'
    #' @param tree tree under consideration
    #' @param max_lt maximum lifetime to be considered
    #' @param max_bt add maximum birth time to avoid out of bound error

    diag <- pd_tree(tree[[1]], tree[[3]])
    diag_df <- data.frame(birth = diag[,1],
                           life = (diag[,2] - diag[,1]))

    #only 1-dimensional features
    diag_df <- diag_df[diag_df$life < max_lt,]
    diag_df <- diag_df[order(diag_df$birth),]
    rownames(diag_df) <- c()
    rbind(diag_df, c(max_bt, max_bt))
}

tda0_net <- function(X,
                 r = 0,
                 tx = t_vals0_net){
    #' Accumulated persistence function for spatstat
    #'
    #' @param X point pattern
    #' @param r time point
    #' @param tx times at which the APF is computed

    tree <- dsf(X)
    tda0_vals <- tda0_raw_net(tree)
    tda0_vals <- cbind(tda0_vals[,1], cumsum(tda0_vals$life))
    birth_locs = findInterval(tx, tda0_vals[,1])

    #intensity
    intens <- X$n / area.owin(win)

    #rescale integration domain
    tx <- tx * sqrt(2 / intens)

    fv(data.frame(r = tx,
                  trans = tda0_vals[1 + birth_locs, 2]),
      valu = 'trans',
      labl = c('r', 'lab'),
      fname='Diag')
}
```

### Asymptotic Gaussianity

Next, we illustrate that the FCLT the persistence diagram on the sublevel filtration is already accurate for moderately-sized sampling windows, by considering the aggregated persistence function defined above. We compute the standardized scalar statistics for the null model.

```R
library(parallel)

#compute summary statistics
null_stats <-  as.numeric(mclapply(ppps,
                                         function(X) mean(tda0_net(X)$trans,
                                                    mc.cores = 36)))
#normalize
null_means <- mean(null_stats)
null_sds <- sd(null_stats)
null_norms <-  data.frame(x = (null_stats - null_means) / null_sds)
```

Both the histogram and the QQ-plot illustrate that the Gaussian approximation is already accurate on the considered sampling window.

```R
hists <- ggplot(null_norms, aes(x = x)) +
        geom_histogram(aes(y = ..density..),
                       color = 'black',
                       fill="white") +
        stat_function(fun = dnorm,
                      colour = "red",
                      size = 1.5) +
        theme_bw()

grid.arrange(hists,
            ncol = 2, nrow = 1)
```

```R
qplots <-  qplot(sample = x, data = null_norms) +
                    stat_qq_line() +
                    theme_bw()

grid.arrange(qplots,
            ncol = 2, nrow = 1)
```

### Goodness of fit tests

Finally, we discuss how suitable the test statistic is from the point of view of goodness-of-fit tests. When evaluated at the null model, the actual type 1 error is close to the nominal 5 \% level, as we would expect from the Gaussian approximation. However, relying only on the network-based statistics, it is difficult to detect subtle deviations from the null hypothesis of the Poisson process. Indeed, the test powers for the Matérn-cluster and Strauss process are 6.0\% and 8.3\%, respectively.

```R
alpha <- .05
pps_list <- list(ppps_val, mcps, strs)

do.call(rbind.data.frame,
     lapply(pps_list,
                                  function(pp_list) sum(
                                      mclapply(pp_list,
                                               function(pp) p_val(pp,
                                                                  function(X) mean(tda0_net(X)$trans),
                                                                  null_means,
                                                                  null_sds),
                                               mc.cores = 36
                                              ) < alpha)
                                  )
       )
```

A data.frame: 3 × 1

c.54L..60L..83L.

<int>

54

60

83

```R

```


---
title: Triple Pendulum Example
type: submodule
---

```python
%matplotlib inline
```

Try running with this variable set to true and to false and see the difference in the resulting equations of motion

```python
use_constraints = False
```

Import all the necessary modules

```python
# -*- coding: utf-8 -*-
"""
Written by Daniel M. Aukes
Email: danaukes<at>gmail.com
Please see LICENSE for full license.
"""

import pynamics
from pynamics.frame import Frame
from pynamics.variable_types import Differentiable,Constant
from pynamics.system import System
from pynamics.body import Body
from pynamics.dyadic import Dyadic
from pynamics.output import Output,PointsOutput
from pynamics.particle import Particle
from pynamics.constraint import AccelerationConstraint
import pynamics.integration
import numpy
import sympy
import matplotlib.pyplot as plt
plt.ion()
from math import pi
```

The next two lines create a new system object and set that system as the global system within the module so that other variables can use and find it.

```python
system = System()
pynamics.set_system(__name__,system)
```

## Parameterization

### Constants

Declare constants and seed them with their default value.  This can be changed at integration time but is often a nice shortcut when you don't want the value to change but you want it to be represented symbolically in calculations

```python
lA = Constant(1,'lA',system)
lB = Constant(1,'lB',system)
lC = Constant(1,'lC',system)

mA = Constant(1,'mA',system)
mB = Constant(1,'mB',system)
mC = Constant(1,'mC',system)

g = Constant(9.81,'g',system)
b = Constant(1e1,'b',system)
k = Constant(1e1,'k',system)

preload1 = Constant(0*pi/180,'preload1',system)
preload2 = Constant(0*pi/180,'preload2',system)
preload3 = Constant(0*pi/180,'preload3',system)

Ixx_A = Constant(1,'Ixx_A',system)
Iyy_A = Constant(1,'Iyy_A',system)
Izz_A = Constant(1,'Izz_A',system)
Ixx_B = Constant(1,'Ixx_B',system)
Iyy_B = Constant(1,'Iyy_B',system)
Izz_B = Constant(1,'Izz_B',system)
Ixx_C = Constant(1,'Ixx_C',system)
Iyy_C = Constant(1,'Iyy_C',system)
Izz_C = Constant(1,'Izz_C',system)

torque = Constant(0,'torque',system)
freq = Constant(3e0,'freq',system)
```

### Differentiable State Variables

Define your differentiable state variables that you will use to model the state of the system.  In this case $qA$, $qB$, and $qC$ are the rotation angles of a three-link mechanism

```python
qA,qA_d,qA_dd = Differentiable('qA',system)
qB,qB_d,qB_dd = Differentiable('qB',system)
qC,qC_d,qC_dd = Differentiable('qC',system)
```

### Initial Values
Define a set of initial values for the position and velocity of each of your state variables.  It is necessary to define a known.  This code create a dictionary of initial values.

```python
initialvalues = {}
initialvalues[qA]=0*pi/180
initialvalues[qA_d]=0*pi/180
initialvalues[qB]=0*pi/180
initialvalues[qB_d]=0*pi/180
initialvalues[qC]=0*pi/180
initialvalues[qC_d]=0*pi/180
```

These two lines of code order the initial values in a list in such a way that the integrator can use it in the same order that it expects the variables to be supplied

```python
statevariables = system.get_state_variables()
ini = [initialvalues[item] for item in statevariables]
```

## Kinematics

### Frames
Define the reference frames of the system

```python
N = Frame('N',system)
A = Frame('A',system)
B = Frame('B',system)
C = Frame('C',system)
```

### Newtonian Frame

It is important to define the Newtonian reference frame as a reference frame that is not accelerating, otherwise the dynamic equations will not be correct

```python
system.set_newtonian(N)
```

Rotate each successive frame by amount q<new> from the last.  This approach can produce more complex equations but is representationally simple (Minimal Representation)

```python
A.rotate_fixed_axis(N,[0,0,1],qA,system)
B.rotate_fixed_axis(A,[0,0,1],qB,system)
C.rotate_fixed_axis(B,[0,0,1],qC,system)
```

### Vectors
Define the vectors that describe the kinematics of a series of connected lengths

* pNA - This is a vector with position at the origin.
* pAB - This vector is length $l_A$ away from the origin along the A.x unit vector
* pBC - This vector is length $l_B$ away from the pAB along the B.x unit vector
* pCtip - This vector is length $l_C$ away from the pBC along the C.x unit vector

Define my rigid body kinematics

```python
pNA=0*N.x
pAB=pNA+lA*A.x
pBC = pAB + lB*B.x
pCtip = pBC + lC*C.x
```

## Centers of Mass

It is important to define the centers of mass of each link.  In this case, the center of mass of link A, B, and C is halfway along the length of each

```python
pAcm=pNA+lA/2*A.x
pBcm=pAB+lB/2*B.x
pCcm=pBC+lC/2*C.x
```

## Calculating Velocity

The angular velocity between frames, and the time derivatives of vectors are extremely useful in calculating the equations of motion and for determining many of the forces that need to be applied to your system (damping, drag, etc).  Thus, it is useful, once kinematics have been defined, to take or find the derivatives of some of those vectors for calculating  linear or angular velocity vectors

### Angular Velocity
The following three lines of code computes and returns the angular velocity between frames N and A (${}^N\omega^A$), A and B (${}^A\omega^B$), and B and C (${}^B\omega^C$).  In other cases, if the derivative expression is complex or long,  you can supply pynamics with a given angular velocity between frames to speed up computation time.

```python
wNA = N.get_w_to(A)
wAB = A.get_w_to(B)
wBC = B.get_w_to(C)
```

### Vector derivatives
The time derivatives of vectors may also be

vCtip = pCtip.time_derivative(N,system)

### Define Inertias and Bodies
The next several lines compute the inertia dyadics of each body and define a rigid body on each frame.  In the case of frame C, we represent the mass as a particle located at point pCcm.

```python
IA = Dyadic.build(A,Ixx_A,Iyy_A,Izz_A)
IB = Dyadic.build(B,Ixx_B,Iyy_B,Izz_B)
IC = Dyadic.build(C,Ixx_C,Iyy_C,Izz_C)

BodyA = Body('BodyA',A,pAcm,mA,IA,system)
BodyB = Body('BodyB',B,pBcm,mB,IB,system)
BodyC = Body('BodyC',C,pCcm,mC,IC,system)
#BodyC = Particle(pCcm,mC,'ParticleC',system)
```

## Forces and Torques
Forces and torques are added to the system with the generic ```addforce``` method.  The first parameter supplied is a vector describing the force applied at a point or the torque applied along a given rotational axis.  The second parameter is the  vector describing the linear speed (for an applied force) or the angular velocity(for an applied torque)

```python
system.addforce(torque*sympy.sin(freq*2*sympy.pi*system.t)*A.z,wNA)
```

    <pynamics.force.Force at 0x290477b2250>

### Damper

```python
system.addforce(-b*wNA,wNA)
system.addforce(-b*wAB,wAB)
system.addforce(-b*wBC,wBC)
```

    <pynamics.force.Force at 0x290476084f0>

### Spring Forces

Spring forces are a special case because the energy stored in springs is conservative and should be considered when calculating the system's potential energy.  To do this, use the ```add_spring_force``` command.  In this method, the first value is the linear spring constant.  The second value is the "stretch" vector, indicating the amount of deflection from the neutral point of the spring.  The final parameter is, as above, the linear or angluar velocity vector (depending on whether your spring is a linear or torsional spring)

In this case, the torques applied to each joint are dependent upon whether qA, qB, and qC are absolute or relative rotations, as defined above.

```python
system.add_spring_force1(k,(qA-preload1)*N.z,wNA)
system.add_spring_force1(k,(qB-preload2)*A.z,wAB)
system.add_spring_force1(k,(qC-preload3)*B.z,wBC)
```

    (<pynamics.force.Force at 0x290477aef10>,
     <pynamics.spring.Spring at 0x290477ae400>)

### Gravity
Again, like springs, the force of gravity is conservative and should be applied to all bodies.  To globally apply the force of gravity to all particles and bodies, you can use the special ```addforcegravity``` method, by supplying the acceleration due to gravity as a vector.  This will get applied to all bodies defined in your system.

```python
system.addforcegravity(-g*N.y)
```

## Constraints
Constraints may be defined that prevent the motion of certain elements.  Try turning on the constraints flag at the top of the script to see what happens.

```python
if use_constraints:
    eq = []
    eq.append(pCtip)
    eq_d=[item.time_derivative() for item in eq]
    eq_dd=[item.time_derivative() for item in eq_d]
    eq_dd_scalar = []
    eq_dd_scalar.append(eq_dd[0].dot(N.y))
    constraint = AccelerationConstraint(eq_dd_scalar)
    system.add_constraint(constraint)
```

## F=ma
This is where the symbolic expressions for F and ma are calculated.  This must be done after all parts of the system have been defined.  The ```getdynamics``` function uses Kane's method to derive the equations of motion.

```python
f,ma = system.getdynamics()
```

    2021-07-22 13:50:52,813 - pynamics.system - INFO - getting dynamic equations

```python
f
```

    ⎡          g⋅lA⋅mA⋅cos(qA)                                       g⋅lB⋅mB⋅sin(q
    ⎢-b⋅qA_d - ─────────────── - g⋅lA⋅mB⋅cos(qA) - g⋅lA⋅mC⋅cos(qA) + ─────────────
    ⎣                 2                                                         2

    A)⋅sin(qB)   g⋅lB⋅mB⋅cos(qA)⋅cos(qB)
    ────────── - ─────────────────────── + g⋅lB⋅mC⋅sin(qA)⋅sin(qB) - g⋅lB⋅mC⋅cos(q
                            2

                 g⋅lC⋅mC⋅(-sin(qB)⋅sin(qC) + cos(qB)⋅cos(qC))⋅cos(qA)   g⋅lC⋅mC⋅(-
    A)⋅cos(qB) - ──────────────────────────────────────────────────── - ──────────
                                          2

    sin(qB)⋅cos(qC) - sin(qC)⋅cos(qB))⋅sin(qA)
    ────────────────────────────────────────── - k⋅(-preload₁ + qA) + torque⋅sin(2
                   2

                          g⋅lB⋅mB⋅sin(qA)⋅sin(qB)   g⋅lB⋅mB⋅cos(qA)⋅cos(qB)
    ⋅freq⋅π⋅t), -b⋅qB_d + ─────────────────────── - ─────────────────────── + g⋅lB
                                     2                         2

                                                    g⋅lC⋅mC⋅(-sin(qB)⋅sin(qC) + co
    ⋅mC⋅sin(qA)⋅sin(qB) - g⋅lB⋅mC⋅cos(qA)⋅cos(qB) - ──────────────────────────────
                                                                             2

    s(qB)⋅cos(qC))⋅cos(qA)   g⋅lC⋅mC⋅(-sin(qB)⋅cos(qC) - sin(qC)⋅cos(qB))⋅sin(qA)
    ────────────────────── - ────────────────────────────────────────────────────
                                                      2

                                    g⋅lC⋅mC⋅(-sin(qB)⋅sin(qC) + cos(qB)⋅cos(qC))⋅c
    - k⋅(-preload₂ + qB), -b⋅qC_d - ──────────────────────────────────────────────
                                                             2

    os(qA)   g⋅lC⋅mC⋅(-sin(qB)⋅cos(qC) - sin(qC)⋅cos(qB))⋅sin(qA)
    ────── - ──────────────────────────────────────────────────── - k⋅(-preload₃ +
                                      2

        ⎤
     qC)⎥
        ⎦

```python
ma
```

    ⎡
    ⎢                                                                        2
    ⎢                                                                      lA ⋅mA⋅
    ⎢Izz_A⋅qA_dd + Izz_B⋅(qA_dd + qB_dd) + Izz_C⋅(qA_dd + qB_dd + qC_dd) + ───────
    ⎣                                                                           4

                                                       2
    qA_dd     2              2            lA⋅lB⋅mB⋅qA_d ⋅sin(qB)   lA⋅lB⋅mB⋅qA_dd⋅
    ───── + lA ⋅mB⋅qA_dd + lA ⋅mC⋅qA_dd + ────────────────────── + ───────────────
                                                    2                        2

                                    2
    cos(qB)   lA⋅lB⋅mB⋅(qA_d + qB_d) ⋅sin(qB)                2
    ─────── - ─────────────────────────────── + lA⋅lB⋅mC⋅qA_d ⋅sin(qB) + lA⋅lB⋅mC⋅
                             2

                                          2                 ⎛lB⋅qA_dd   lB⋅qB_dd⎞
    qA_dd⋅cos(qB) - lA⋅lB⋅mC⋅(qA_d + qB_d) ⋅sin(qB) + lA⋅mB⋅⎜──────── + ────────⎟⋅
                                                            ⎝   2          2    ⎠

                                                       ⎛
                                                       ⎜  lC⋅mC⋅(qA_d + qB_d + qC_
    cos(qB) + lA⋅mC⋅(lB⋅qA_dd + lB⋅qB_dd)⋅cos(qB) + lA⋅⎜- ────────────────────────
                                                       ⎝                   2

      2                                                      ⎞              ⎛
    d) ⋅sin(qC)      ⎛lC⋅qA_dd   lC⋅qB_dd   lC⋅qC_dd⎞        ⎟              ⎜  lC⋅
    ─────────── + mC⋅⎜──────── + ──────── + ────────⎟⋅cos(qC)⎟⋅cos(qB) + lA⋅⎜- ───
                     ⎝   2          2          2    ⎠        ⎠              ⎝

                           2
    mC⋅(qA_d + qB_d + qC_d) ⋅cos(qC)      ⎛lC⋅qA_dd   lC⋅qB_dd   lC⋅qC_dd⎞
    ──────────────────────────────── - mC⋅⎜──────── + ──────── + ────────⎟⋅sin(qC)
                  2                       ⎝   2          2          2    ⎠

    ⎞                                 2                                        2
    ⎟           lB⋅lC⋅mC⋅(qA_d + qB_d) ⋅sin(qC)   lB⋅lC⋅mC⋅(qA_d + qB_d + qC_d) ⋅s
    ⎟⋅sin(qB) + ─────────────────────────────── - ────────────────────────────────
    ⎠                          2                                    2

                   ⎛lB⋅qA_dd   lB⋅qB_dd⎞
             lB⋅mB⋅⎜──────── + ────────⎟
    in(qC)         ⎝   2          2    ⎠                                       ⎛lC
    ────── + ─────────────────────────── + lB⋅mC⋅(lB⋅qA_dd + lB⋅qB_dd) + lB⋅mC⋅⎜──
                          2                                                    ⎝

    ⋅qA_dd   lC⋅qB_dd   lC⋅qC_dd⎞           lC⋅mC⋅(lB⋅qA_dd + lB⋅qB_dd)⋅cos(qC)
    ────── + ──────── + ────────⎟⋅cos(qC) + ─────────────────────────────────── +
     2          2          2    ⎠                            2

          ⎛lC⋅qA_dd   lC⋅qB_dd   lC⋅qC_dd⎞
    lC⋅mC⋅⎜──────── + ──────── + ────────⎟      ⎛          2
          ⎝   2          2          2    ⎠   lC⋅⎝lA⋅mC⋅qA_d ⋅sin(qB) + lA⋅mC⋅qA_dd
    ────────────────────────────────────── + ─────────────────────────────────────
                      2                                                2

            ⎞              ⎛            2                              ⎞
    ⋅cos(qB)⎠⋅cos(qC)   lC⋅⎝- lA⋅mC⋅qA_d ⋅cos(qB) + lA⋅mC⋅qA_dd⋅sin(qB)⎠⋅sin(qC)
    ───────────────── - ────────────────────────────────────────────────────────,
                                                   2

                                                                         2
                                                            lA⋅lB⋅mB⋅qA_d ⋅sin(qB)
    Izz_B⋅(qA_dd + qB_dd) + Izz_C⋅(qA_dd + qB_dd + qC_dd) + ──────────────────────
                                                                      2

       lA⋅lB⋅mB⋅qA_dd⋅cos(qB)                2
     + ────────────────────── + lA⋅lB⋅mC⋅qA_d ⋅sin(qB) + lA⋅lB⋅mC⋅qA_dd⋅cos(qB) +
                 2

                          2                                        2           lB⋅
    lB⋅lC⋅mC⋅(qA_d + qB_d) ⋅sin(qC)   lB⋅lC⋅mC⋅(qA_d + qB_d + qC_d) ⋅sin(qC)
    ─────────────────────────────── - ────────────────────────────────────── + ───
                   2                                    2

       ⎛lB⋅qA_dd   lB⋅qB_dd⎞
    mB⋅⎜──────── + ────────⎟
       ⎝   2          2    ⎠                                       ⎛lC⋅qA_dd   lC⋅
    ──────────────────────── + lB⋅mC⋅(lB⋅qA_dd + lB⋅qB_dd) + lB⋅mC⋅⎜──────── + ───
              2                                                    ⎝   2

                                                                            ⎛lC⋅qA
                                                                      lC⋅mC⋅⎜─────
    qB_dd   lC⋅qC_dd⎞           lC⋅mC⋅(lB⋅qA_dd + lB⋅qB_dd)⋅cos(qC)         ⎝   2
    ───── + ────────⎟⋅cos(qC) + ─────────────────────────────────── + ────────────
    2          2    ⎠                            2

    _dd   lC⋅qB_dd   lC⋅qC_dd⎞
    ─── + ──────── + ────────⎟      ⎛          2                              ⎞
             2          2    ⎠   lC⋅⎝lA⋅mC⋅qA_d ⋅sin(qB) + lA⋅mC⋅qA_dd⋅cos(qB)⎠⋅co
    ────────────────────────── + ─────────────────────────────────────────────────
          2                                                2

               ⎛            2                              ⎞
    s(qC)   lC⋅⎝- lA⋅mC⋅qA_d ⋅cos(qB) + lA⋅mC⋅qA_dd⋅sin(qB)⎠⋅sin(qC)
    ───── - ────────────────────────────────────────────────────────, Izz_C⋅(qA_dd
                                       2

                                              2
                        lB⋅lC⋅mC⋅(qA_d + qB_d) ⋅sin(qC)   lC⋅mC⋅(lB⋅qA_dd + lB⋅qB_
     + qB_dd + qC_dd) + ─────────────────────────────── + ────────────────────────
                                       2                                   2

                        ⎛lC⋅qA_dd   lC⋅qB_dd   lC⋅qC_dd⎞
                  lC⋅mC⋅⎜──────── + ──────── + ────────⎟      ⎛          2
    dd)⋅cos(qC)         ⎝   2          2          2    ⎠   lC⋅⎝lA⋅mC⋅qA_d ⋅sin(qB)
    ─────────── + ────────────────────────────────────── + ───────────────────────
                                    2

                          ⎞              ⎛            2
     + lA⋅mC⋅qA_dd⋅cos(qB)⎠⋅cos(qC)   lC⋅⎝- lA⋅mC⋅qA_d ⋅cos(qB) + lA⋅mC⋅qA_dd⋅sin(
    ─────────────────────────────── - ────────────────────────────────────────────
       2                                                         2

                ⎤
       ⎞        ⎥
    qB)⎠⋅sin(qC)⎥
    ────────────⎥
                ⎦

## Solve for Acceleration

The next line of code solves the system of equations F=ma plus any constraint equations that have been added above.  It returns one or two variables.  func1 is the function that computes the velocity and acceleration given a certain state, and lambda1(optional) supplies the function that computes the constraint forces as a function of the resulting states

There are a few ways of solveing for a.  The below function inverts the mass matrix numerically every time step.  This can be slower because the matrix solution has to be solved for, but is sometimes more tractable than solving the highly nonlinear symbolic expressions that can be generated from the previous step.  The other options would be to use ```state_space_pre_invert```, which pre-inverts the equations symbolically before generating a numerical function, or ```state_space_post_invert2```, which adds Baumgarte's method for intermittent constraints.

```python
func1,lambda1 = system.state_space_post_invert(f,ma,return_lambda = True)
```

    2021-07-22 13:50:53,430 - pynamics.system - INFO - solving a = f/m and creating function
    2021-07-22 13:50:53,599 - pynamics.system - INFO - substituting constrained in Ma-f.
    2021-07-22 13:50:53,762 - pynamics.system - INFO - done solving a = f/m and creating function
    2021-07-22 13:50:53,762 - pynamics.system - INFO - calculating function for lambdas

## Integration Tolerance
Specify the precision of the integration

```python
tol = 1e-5
```

### Time
Define variables for time that can be used throughout the script.  These get used to create the t array, a list of every time value that is solved for during integration

```python
tinitial = 0
tfinal = 10
fps = 30
tstep = 1/fps
t = numpy.r_[tinitial:tfinal:tstep]
```

## Integrate

The next line of code integrates the function calculated

```python
states=pynamics.integration.integrate(func1,ini,t,rtol=tol,atol=tol, args=({'constants':system.constant_values},))
```

    2021-07-22 13:50:53,947 - pynamics.integration - INFO - beginning integration
    2021-07-22 13:50:53,947 - pynamics.system - INFO - integration at time 0000.00
    2021-07-22 13:50:54,163 - pynamics.integration - INFO - finished integration

## Outputs

The next section simply calculates and plots a variety of data from the previous simulation
### States

```python
plt.figure()
artists = plt.plot(t,states[:,:3])
plt.legend(artists,['qA','qB','qC'])
```

### Energy

```python
KE = system.get_KE()
PE = system.getPEGravity(pNA) - system.getPESprings()
energy_output = Output([KE-PE],system)
energy_output.calc(states,t)
energy_output.plot_time()
```

### Constraint Forces

This line of code computes the constraint forces once the system's states have been solved for.

```python
if use_constraints:
    lambda2 = numpy.array([lambda1(item1,item2,system.constant_values) for item1,item2 in zip(t,states)])
    plt.figure()
    plt.plot(t, lambda2)
```

### Motion

```python
points = [pNA,pAB,pBC,pCtip]
points_output = PointsOutput(points,system)
y = points_output.calc(states,t)
points_output.plot_time(20)
```

#### Motion Animation
in normal Python the next lines of code produce an animation using matplotlib
points_output.animate(fps = fps,movie_name = 'triple_pendulum.mp4',lw=2,marker='o',color=(1,0,0,1),linestyle='-')
To plot the animation in jupyter you need a couple extra lines of code...
from matplotlib import animation, rc
from IPython.display import HTML
HTML(points_output.anim.to_html5_video())