# TALENT Course 11
## Learning from Data: Bayesian Methods and Machine Learning
### York, UK, June 10-28, 2019
$% Some LaTeX definitions we'll use.
\newcommand{\pr}{\textrm{p}}
$

## Model selection (I)

### Bayesian evidence:
Please see the full version of the lecture notes here in [html](pub/model_selection-bs.html) and [pdf](pub/model_selection-minted.pdf) formats. The notes contain a somewhat adapted version of Ch. 4.1 in Sivia's book: "The story of Dr A and Prof B", which is extremely well written. It also contains a summary of Laplace's method for approximating evidence factors.

### Import of modules

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from scipy import optimize

# Not really needed, but nicer plots
import seaborn as sns
sns.set()
sns.set_context("talk")
```

## What order polynomial?

Throughout the rest of this section, we will use data that was generated from a "true model" where x and y satisfy the following:
$$
y_i = x_i \sin(x_i)+\epsilon_i,
$$
where $0 \leq x_i \leq 3$ and the noise is drawn from a normal distribution $\epsilon_i \sim \mathcal{N}(0, \sigma_0)$. The values for 20 regularly spaced points with $\sigma_0=0.1$ are shown below.

```python
#------------------------------------------------------------
# Define our functional form
def true_func(x):
    return np.sin(x) * x

def func(x, dy=0.1):
    return np.random.normal(true_func(x), dy)

#------------------------------------------------------------
# select the (noisy) data
np.random.seed(0)
num_data = 20
x_max = 3
x = np.linspace(0, x_max, num_data+2)[1:-1]
sig0 = 0.1  # try 0.5 or higher or 0.01
y = func(x, sig0)
```

```python
fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(1,1,1)
ax.errorbar(x, y, sig0, fmt='o');
ax.plot(x, true_func(x), color='red')
ax.set(xlabel='x',ylabel='y')
fig.tight_layout()
```

Assume that we have a multiple of models, with varying degree of sophistication. In this example we use polynomials of different orders to represent models of increasing complexity and with an increasing number of model parameters.

> Our task is to find which model finds the most support in the given data.

It is clear that a more complicated model with more free parameters should be able to fit the data much more closely. But what is the evidence in the data for such a complicated model? Finding the answer to this question is a task for a Bayesian, and the problem is generally known as *Model selection*.

Below, we will use an approximate way of computing the Bayesian evidence, namely the Laplace method. In some cases one can also use conjugate priors to simplify the computation of the evidence factor. Or one can use certain sampling methods to compute the evidence numerically. The highlight will be the comparison of different models using the evidences to extract odds-ratios.

### The Model

See previous lecture notes, in particular [Parameter estimation III](../bayesian-parameter-estimation/Lecture_Th1a_rjf.pdf) for some more details.

In general, we're fitting a $M$-degree polynomial to data,

$$
y_M(x) = \sum_{i=0}^M \theta_i x^i
$$

where we use $\theta$ to denote our parameter vector of length $M$.

Assuming all the points are independent, we can find the full log likelihood by adding the individual likelihoods together:
$$
\begin{align}
\log p(D\mid\theta, I) &= -\frac{1}{2}\sum_{i=1}^N\left(\log(2\pi\sigma_0^2) + \frac{\left[ y_i - y_M(x_i;\theta)\right]^2}{\sigma_0^2}\right) \\
&= \text{constant} - \sum_{i=1}^N \frac{\left[ y_i - y_M(x_i;\theta)\right]^2}{2 \sigma_0^2}
\end{align}
$$

We often define the residuals
$$
R_i = \left[ y_i - y_M(x_i;\theta) \right]/\sigma_0,
$$
so that the relevant chi-square sum reads $- \sum_{i=1}^N R_i^2 / 2$.

```python
def residuals(theta, x=x, y=y, sigma0=sig0):
    dy = y - np.polyval(theta,x)
    return dy / sigma0

# Standard likelihood with Gaussian errors as specified
# uniform prior for theta
def log_likelihood(theta):
    return -0.5 * np.sum(residuals(theta)**2)
```

### Max likelihood fits

We can maximize the likelihood to find $\theta$ within a frequentist paradigm. Let us start with a linear fit:

```python
degree = 1
theta_hat = np.polyfit(x, y, degree)
x_fit = np.linspace(0, x_max, 1000)
y_fit = np.polyval(theta_hat, x_fit)
```

Rather than just plotting this fit, we will compare several different models in the figure below.

```python
def fit_degree_n(degree, ax):
    """Fit a polynomial of order 'degree', return the chi-squared, and plot in axes 'ax'."""
    theta_hat = np.polyfit(x, y, degree)
    x_fit = np.linspace(0, x_max, 1000)
    y_fit = np.polyval(theta_hat, x_fit)

    ax.errorbar(x, y, sig0, fmt='o');
    ax.plot(x_fit, true_func(x_fit), color='red', alpha=.5)

    ax.text(0.03, 0.96, f"d = {degree}", transform=plt.gca().transAxes,
            ha='left', va='top',
            bbox=dict(ec='k', fc='w', pad=10))

    ax.plot(x_fit, y_fit, '-k')
    ax.set_xlabel('$x$')
    ax.set_ylabel('$y$');

    return -2. * log_likelihood(theta_hat)  # chi_squared

#------------------------------------------------------------
# First figure: plot points with a linear fit
nrows=1; ncols=2;
fig = plt.figure(figsize=(6*ncols, 6*nrows))

num_plots = nrows * ncols
degrees = np.zeros(num_plots, dtype=int)
chi_sqs_dof = np.zeros(num_plots)
print('degree  chi^2/dof')
for i in range(num_plots):
    ax = fig.add_subplot(nrows, ncols, i+1)
    degrees[i] = i
    dof = len(x) - (degrees[i])
    chi_sqs_dof[i] = fit_degree_n(i, ax) / dof
fig.tight_layout()
```

### Questions

* Change the degree of the polynomial that is used for the fit. Plot the fits next to each other in a multi-panel figure (e.g., make it 3 by 3).
* Compute the chi-squared value per degree of freedom (the ingredients are already included in the script) and plot that as a function of the degree of the polynomial. Is it decreasing, or is there a peak?
* For which degree polynomials would you say that you're underfitting the data?
* For which degree polynomials would you say that you're overfitting the data?

### Cross validation
This section will introduce the frequentist tool of cross-validation. This approach is used extensively within machine-learning as a way to handle overfitting and underfitting, bias and variance.

```python
# Select the cross-validation points
ncross=5
index_cv = np.random.choice(range(len(x)), ncross, replace=False)

x_cv=x[index_cv]
y_cv=y[index_cv]
```

```python
# The training data is then
x_train = np.delete(x,index_cv)
y_train = np.delete(y,index_cv)
```

```python
# Plot training and CV errors as a function of polynomial degree d
degree_max = 13
d = np.arange(0, degree_max+1)
training_err = np.zeros(d.shape)
crossval_err = np.zeros(d.shape)

fig,ax = plt.subplots(figsize=(8, 6))
for i in range(len(d)):
    p = np.polyfit(x_train, y_train, d[i])
    training_err[i] = np.sqrt(np.sum((np.polyval(p, x_train) - y_train) ** 2)
                              / len(y_train))
    crossval_err[i] = np.sqrt(np.sum((np.polyval(p, x_cv) - y_cv) ** 2)
                              / len(y_cv))

ax.plot(d, crossval_err, '--k', label='cross-validation')
ax.plot(d, training_err, '-k', label='training')
ax.plot(d, sig0 * np.ones(d.shape), ':k')

ax.set_xlim(0, degree_max)
# You might need to change the y-scale if you make modifications to the training data
ax.set_ylim(0, 0.8)

ax.set_xlabel('polynomial degree')
ax.set_ylabel('rms error')
ax.legend(loc='best');

```

### Questions

* Can you see the transition from underfit to overfit in this figure?
* What would you say is the degree of polynomial that is supported by the data?
* Try changing the size of the cross-validation and training sets. Does the conclusions become more/less clear?
* Does the results change between different runs with the same number of CV samples? If so, why?
* K-fold cross validation is a popular variant of CV. It addresses some issues with the sensitivity to the actual choice of which data is used for training and validation.  What do you think that it means, and what is the possible drawback if you have a computational expensive model?
* Leave-one-out is another variant. For linear regression problems, this type of cross-validation can actually be performed without having to do multiple fits. What do you think that it means?
* It is common to divide the data into a training set, a cross-validation set, and a test set. What do you think is the purpose of having three different sets?

## Bayesian evidence

Let us try the Bayesian approach and actually compute the evidence for these different models. We will use the Laplace method for computing the norm of the posterior distribution (i.e. approximating it as a single Gaussian).

We use simple uniform priors for the model parameters:
$$
p(\theta_i|I) = \left\{
\begin{array}{ll}
\frac{1}{\theta_\mathrm{max} - \theta_\mathrm{min}} & \text{for } \theta_\mathrm{min} \leq \theta_i \leq \theta_\mathrm{max}, \\
0 & \text{otherwise},
\end{array}
\right.
$$
which means that the posterior will be
$$
p(\theta | D, I) = \frac{1}{(\theta_\mathrm{max} - \theta_\mathrm{min})^K} \frac{1}{\sqrt{(2\pi)\sigma_0^2}^N} \exp\left( -\chi^2 / 2\right),
$$
within the allowed prior region for the $K$ parameters and zero elsewhere.

Assuming that the peak of the Gaussian is located at $\theta^*$, well inside the prior region; we can easily approximate the integral
$$
Z_p = \int d^K \theta p(\theta | D, I),
$$
using Laplace's method (see lecture notes here in [html](pub/model_selection-bs.html) and [pdf](pub/model_selection-minted.pdf) formats). With this particular choice of prior, and again under the assumption that the cut at the edges does not change the integral over the multidimensional integral, we get
$$
Z_p \approx \frac{1}{(\theta_\mathrm{max} - \theta_\mathrm{min})^K} \exp\left( -\chi^2(\theta^*) / 2\right) \frac{\sqrt{(2\pi)^K}}{\sqrt{\det(\Sigma^{-1})}},
$$
where $\Sigma^{-1}_{ij} = \partial^2\chi^2/\partial \theta_i \partial \theta_j$ (i.e. the Hessian) evaluated at the maximum $\theta^*$. Note that we removed the constant factor $\sqrt{(2\pi)\sigma_0}^N$ since it will be the same for all models.

Note that for this linear regression problem we can get all these quantities ($\theta^*$, $\Sigma$) via linear algebra. See, e.g., Dick's [lecture notes](https://github.com/NuclearTalent/Bayes2019/blob/master/topics/bayesian-parameter-estimation/Lecture_Th1a_rjf.pdf) or Hogg's nice paper: [Data analysis recipes: Fitting a model to data](https://arxiv.org/abs/1008.4686). Below, we will use `numpy.polyfit` to extract the relevant quantities.

```python
# We use a uniform prior for all parameters in [-10,10]
theta_max = 10
theta_min = -10
prior_range = theta_max - theta_min
```

```python
degree_max = 6
evidence = np.zeros(degree_max+1)
print("Degree  P*          Best fit parameters: ")
for ideg,deg in enumerate(range(degree_max+1)):
    theta_hat, Cov = np.polyfit(x, y, deg,cov='unscaled')
    if not (np.all(theta_hat < theta_max) and np.all(theta_hat > theta_min)):
        print("Outside of prior range")
    P_star = np.exp(log_likelihood(theta_hat))
    H=np.linalg.inv(Cov)
    evidence[ideg] = P_star * np.sqrt((2*np.pi)**deg / np.linalg.det(H)) / prior_range**deg
    print (f'   {deg}    {P_star:.2e}  ',('{:5.2f} '*len(theta_hat)).format(*theta_hat))
```

    Degree  P*          Best fit parameters:
       0    1.76e-151    1.13
       1    1.18e-76    0.50  0.38
       2    2.69e-22   -0.59  2.26 -0.54
       3    6.15e-03   -0.49  1.61 -0.44  0.21
       4    6.44e-03   -0.02 -0.35  1.35 -0.25  0.18
       5    7.62e-03    0.06 -0.48  0.88 -0.10  0.43  0.09
       6    7.62e-03    0.00  0.03 -0.38  0.72  0.03  0.39  0.09

```python
d = np.arange(0, degree_max+1)

fig,ax = plt.subplots(figsize=(8, 6))
ax.plot(d,evidence,'o-')

ax.set_xlabel('polynomial degree')
ax.set_ylabel('evidence');
```

### Questions

* Can you see the transition from underfit to overfit in this figure?
* What would you say is the degree of polynomial that is supported by the data?

```python
# Odds ratio table

```

### Questions

* What happens when you change the number of the generated data?
* What happens when you change the range of the generated data?
* What happens when you change the error of the generated data?

#### Odds-ratios
Quoting the well-known paper by Trotta:
[Bayes in the sky: Bayesian inference and model selection in cosmology](https://arxiv.org/abs/0803.4089) we can quantify an empirical scale for evaluating the strength of evidence when comparing two models:

Here, the ratio of the evidences of model $M_0$ and $M_1$ is given by,
\begin{equation}
 \label{eq:Bayes_factor}
 B_{01} = \frac{p(\mathrm{data} | M_0)}{p(\mathrm{data} | M_1)} \; ,
\end{equation}
which is also called _Bayes factor_. That means $|\ln B_{01}| \equiv |\ln p(\mathrm{data} | M_0) - \ln p(\mathrm{data} | M_1)|$ is the relevant quantity for estimating the strength of evidence of the two models (see first and last column of the table).

### Questions

* Create a table of odds-ratios to select between pairs of the different-order polynomial models, given that the ratio of prior probabilities for the different models is unity.

```python

```


# Block 2: Linear programming: The min-cost flow problem
### Alfred Galichon (NYU)
## `math+econ+code' masterclass on matching models, optimal transport and applications
© 2018-2019 by Alfred Galichon. Support from NSF grant DMS-1716489 is acknowledged. Pauline Corblet, James Nesbit, and Lucas Vernet contributed.

### Learning Objectives

* Basic concepts of directed networks

* The min-cost flow problem

* Duality, optimality and equilibrium

### References

* [OTME], Ch.8

* Tolstoi (1930). Methods of finding the minimal total kilometrage in cargo transportation planning in space. *Transportation Planning* [in Russian]

* Koopmans (1949). Optimum utilization of the transportation system. *Econometrica*.

* Schrijver (2002). On the History of the Transportation and Maximum Flow Problems. *Mathematical programming*.

## Motivation

### Motivation: network flow problems

In 1930 Tolstoi, a Russian engineer, has to optimize the shipping of cement from factories to cities in the Soviet union through railway.

* each factory produces a fixed number of tons

* each city needs a fixed number of tons -- for now, we'll assume total production=total consumption

* each factory is connected by rail with a few cities, and the corresponding distance is given

* how to ship in order to minimize the total distance travelled?

This problem belongs to the class of *min-cost flow problems*, an important class of linear programming problems, which are the focus of today's lecture. A decade before the invention of linear programming and the work of Kantorovich, Koopmans and Dantzig, Tolstoi described a heuristic method for solving the problem, which led to the optimal solution. We'll solve the problem using modern tools (Gurobi), and will see that his solution was right.

The *shortest path problem*, or how to find the path of minimal distance from a point to another in a network, also belongs in this class.

```R
# Lets load up libraries now
library('gurobi')
library('Matrix')
```

## Data

We will be looking at 3 types of data:

### Sovet Planning Problem

Tolstoi's data, collected by Schrijver (2002). There are 10 factories and 68 Soviet cities, and 155 links connecting a factory to a city. This yields a sparse $68\times10$ matrix. Two vectors listing the demand of each city and the supply of each factory are also specified. This is stored in a $69\times11$ matrix, where we have appended the demand/supply vectors to the right and to the botton of the distance matrix.

```R
thePath = paste0(getwd(),"/../data_mec_optim/networks_sovietplanning")
spp_data = as.matrix(read.csv(paste0(thePath, "/distances.csv"), sep = ",", header = TRUE))  # loads the data
head(spp_data)
```

A matrix: 6 × 12 of type chr

X	Arkhangelsk	Yaroslavl.	Murom	Balakhonikha	Dzerzhinsk	Kishert.	Sverdlovsk	Artemovsk	Iledzhk	Dekonskaya	demand.

Agryz      	NA	NA	NA	 709	1064	 693	NA	NA  	NA	NA  	  2.0

Aleksandrov	NA	NA	NA	NA  	 397	NA  	NA	1180	NA	NA  	  4.0

Almaznaya  	NA	NA	NA	NA  	NA  	NA  	NA	  81	NA	  65	  1.5

Alchevskaya	NA	NA	NA	NA  	NA  	NA  	NA	 106	NA	 114	  4.0

Baku       	NA	NA	NA	NA  	NA  	NA  	NA	1554	NA	1563	 10.0

Barybino   	NA	NA	NA	NA  	NA  	NA  	NA	 985	NA	 968	  2.0

### Subway

Paris (NYC) subway network data: the network data are available on www.ratp.fr/opendata (http://www.mta.info). This data is made of two files. The file `nodes.csv` lists the stations: each station is indexed by the line number; each line has the name of the station, and its spatial coordinates. The file `arcs.csv` lists the links between stations: each link specifies the index of the origin, the index of the destination, and the length of the segment.

```R
city = "Paris"
head = FALSE
sep = ","
# city = 'NYC'
#if (city == "Paris") {
#    sep = ";"
#    head = FALSE
#} else {
#    sep = ","
#    head = TRUE
#}

thePath = paste0(getwd() ,"/../data_mec_optim/networks_subway")

sub_arcs = as.matrix(read.csv(paste0(thePath, "/", city, "/arcs.csv"), sep = sep,
    header = head))  # loads the arc data
sub_nodes = as.matrix(read.csv(paste0(thePath, "/", city, "/nodes.csv"), sep = sep,
    header = head))  # loads the nodes data
head(sub_arcs) # the colums are origin node index, destination node index and distance
head(sub_nodes) # name,
```

A matrix: 6 × 3 of type dbl

V1	V2	V3

1	  2	1027.280873

1	319	   9.062544

2	  1	1027.280873

2	  3	 785.939117

3	  2	 785.939117

3	  4	1010.642024

A matrix: 6 × 6 of type chr

V1	V2	V3	V4	V5	V6

La Défense (Grande Arche)           	14367265	2.237883	48.89193	 535	1282

Esplanade de la Défense             	 9992248	2.250442	48.88784	 603	1373

Pont de Neuilly                     	 6823045	2.259892	48.88451	 687	1456

Les Sablons (Jardin d'acclimatation)	 5500065	2.272539	48.88090	 767	1539

Porte Maillot                       	 9089205	2.281836	48.87796	 863	1633

Argentine                           	 2947071	2.289322	48.87553	 998	1766

```R
dim(sub_arcs)
dim(sub_nodes)
```

866

3

366

6

### NYC Roads

The NYC roads data is available,is available as a `shapefile`. The dataset is very similar in construction to the subway data-except that the network is much larger, as there are $67,316$ nodes and $169,399$ arcs. We won't load up this dataset yet, as it is quite large.

## Shortest path problems

### Topology of Networks

We start by defining the directed graph on which transportation takes place.

---
**Definition**

A (directed) *graph* $\left(\mathcal{X},\mathcal{A}\right)$ is a set of *nodes* (cities) $\mathcal{X}$, along with a set of *arcs* $\mathcal{A}\subseteq\mathcal{X}^{2}$ which are pairs $\left(x,y\right)$ where $x,y\in\mathcal{X}$.

---
Our definition allows an arc to have the same origin and destination. Note that for a dense network, $\mathcal{A}=\mathcal{X}^{2}$. For a line, $\left\vert \mathcal{A}\right\vert =\left\vert \mathcal{X}\right\vert -1$.

Next, we define the gradient matrix.

---
**Definition**

We define the *gradient matrix* (also called an `edge-node matrix') as the matrix with general term $\nabla_{ax}$, $a\in\mathcal{A}$, $x\in \mathcal{X}$, such that

\begin{align*}
\nabla_{ax}  &  =-1\text{ if }a\text{ is out of }x\text{,}\\
&  =+1\text{ if }a\text{ is into }x\text{,}\\
&  =0\text{ else.}
\end{align*}

---

Hence, if $f\in\mathbb{R}^{\mathcal{X}}$, then $\nabla f\in\mathbb{R}^{\mathcal{V}}$, and $\left(  \nabla f\right)  _{xy}=f_{y}-f_{x}$.

We shall denote $\nabla^{\intercal}$ the transpose of the gradient matrix. It is the network analog of the $-\operatorname{div}$ differential operator.

Next, we define paths and loops

---
**Definition**

Given two nodes $x$ and $y$, a *path* from $x$ to $y$ is a sequence $x_{1},x_{2},...,x_{K}$ in $\mathcal{X}$ where $x_{1}=x$, $x_{K}=y$, and for every $1\leq k\leq K-1$, $\left(  x_{k},x_{k+1}\right)  \in\mathcal{A}$.

A *loop* (also called `cycle') is a path from a node $x$ to itself.

---

A vector $c\in\mathbb{R}^{\mathcal{V}}$ defined transportation costs. That is, for $xy\in\mathcal{V}$, $c_{xy}$ is the transportation cost associated to arc $xy$. $c$ can also be thought of as the length of arc $xy$. The cost of moving the good from node $x$ to node $y$ along path $x_{1},x_{2},...,x_{K}$ is

\begin{align*}
\sum_{k=1}^{K-1}c_{x_{k}x_{k+1}}.
\end{align*}

No arbitrage conditions: there is no loop of negative cost:

---
**Assumption** [No profitable loop]

There is no profitable loop, which means that there is no sequence $x_{1},...,x_{K}$ in $\mathcal{X}$ such that $x_{K}=x_{1}$, $\left(  x_{k},x_{k+1}\right)  \in\mathcal{A}$, and $\sum_{k=1}^{K-1}c_{x_{k}x_{k+1}}<0$.

---

In particular, there is no profitable loop if $c\geq0$.

Let $n_{x}$ be the *net demand*, which is the flow of goods disappearing from the graph. The set of nodes defined by

\begin{align*}
\mathcal{X}_{0}=\left\{  x\in\mathcal{X}:n_{x}<0\right\}  \text{, and}\mathcal{X}_{1}=\left\{  x\in\mathcal{X}:n_{x}>0\right\}
\end{align*}

are called the *supply nodes* and *demand nodes* respectively. A node which is neither a supply node, neither a demand node is called a *transit node*.

Total supply is $-\sum_{x\in\mathcal{X}_{0}}n_{x}$, total demand is $\sum_{y\in\mathcal{X}_{1}}n_{y}$.

---
**Assumption** [Balancedness]

Assume that total supply equals total demand on the network, that is

\begin{align*}
\sum_{x\in\mathcal{X}_{0}}n_{x}+\sum_{y\in\mathcal{X}_{1}}n_{y}=0.
\end{align*}

---

---
**Assumption** [Connectedness]

Assume the set of supply nodes $\mathcal{X}_{0}$ is strongly connected to the set of demand nodes $\mathcal{X}_{1}$, i.e. for every $x\in\mathcal{X}_{0}$ and $y\in \mathcal{X}_{1}$, there is a path from $x$ to $y$.

---

The specification of the graph, the net demand vector, and the surplus vector
defines a network.

---
**Definition**

A directed graph $\left(\mathcal{X},\mathcal{A}\right)$ endowed with a net demand vector $\left(n_{z}\right)_{z \in \mathcal{X}}$ and a cost vector $\left(c_{a}\right)_{a\in\mathcal{A}}$ is called a *network* $\left(\mathcal{X}, \mathcal{A}, n, c\right)$. If Assumptions [no profitable loop](#noprofitloop), [balancedness](#balancedness) and [connectedness](#connectedness) all hold, the network is called *regular*.

---

Without mention of the contrary we shall assume that the network under consideration is regular.

The flow of mass disappearing at $x$ equals the flow arriving from other nodes minus the flow shipping to other nodes

\begin{align*}
n_{x}=\sum_{z:\left(  z,x\right)  \in\mathcal{A}}\pi_{zx}-\sum_{z:\left(x,z\right)  \in\mathcal{A}}\pi_{xz}
\end{align*}

and this equation can be rewritten as $\nabla^{\intercal}\pi=n$. This motivates the following definition:

---
**Definition**

The set of feasible flows, denoted $\mathcal{M}\left(  n\right)$, or $\mathcal{M}$ when there is no ambiguity, is defined as the set of flows $\pi\geq0$ that verify conservation equation

\begin{align*}
    \nabla^{\intercal}\pi=n.
\end{align*}

---

Let $\phi_{x}$ be the price of the commodity at $x$. Consider the strategy which consists in purchasing the good at $x$, shipping to $y$, and selling at $y$. The profit of this strategy is

\begin{align*}
\phi_{y}-\phi_{x}-c_{xy}=\left(  \nabla\phi-c\right)_{xy}
\end{align*}

and hence there is no arbitrage opportunity if $\phi_{y}-\phi_{x}-c_{xy}\leq0$ for every arc $xy$, that is

\begin{align*}
\nabla\phi\leq c.
\end{align*}

Consider the *minimum cost flow problem*

\begin{align*}
&  \min_{\pi\geq0}\sum_{\left(  x,y\right)  \in\mathcal{A}}\pi_{xy}%
c_{xy} \\
&  s.t.~\nabla^{\intercal}\pi=n
\end{align*}

which is a Linear Programming problem whose dual is

\begin{align*}
&  \max_{\phi\in\mathbb{R}^{\mathcal{X}}}\sum_{x\in\mathcal{X}}n_{x}\phi
_{x}\\
&  s.t.~\nabla\phi\leq c.
\end{align*}

---
**Proposition**
1. Under Assumption [No profitable loop](#noprofitloop), the [dual problem](#dual) is feasible, which means that there is a vector $\phi\in\mathbb{R}^{\mathcal{X}}$ such that $\nabla\phi\leq c$; and the value of [primal problem](#primal) is strictly less than $+\infty$.

2. Under Assumptions [balancedness](#balandedness) and [connectedness](#connectedness), the [primal problem](#primal) is feasible, which means that there is a flow $\pi\geq0$ such that $\nabla^{\intercal}\pi=n$; and the value of [dual problem](#dual) is strictly greater than $-\infty$.

---

Assume that $\left(\mathcal{X},\mathcal{A},n,c\right)$ is a regular network. Then the value of the [primal problem](#primal) coincides with the value of its [dual](#dual), and both problems have solutions. Further, if $\pi$ is a solution to the primal and $\phi$ is a solution to the dual, then $\pi_{xy}>0$ implies $\phi_{y}-\phi_{x}=c_{xy}$.

### Special cases

#### Shortest path problem

Assume there is only one source node $s \in \mathcal{X}$ and one target node $t\in\mathcal{X}$, each associated with unit flow. That is, $n_{x}=1\left\{  x=t\right\}  -1\left\{  x=s\right\}$. Then the problem boils down to how to push one unit of mass from $s$ to $t$. If we interpret $c_{xy}$ as the distance along arc $xy$, the solution of this problem corresponds to the shortest path from $s$ to $t$. This is why this problem is called shortest path problem. (More generally, this problem extends to the case when $c$ does not have a negative loop).

The dual problem is then

\begin{align*}
\max_{\phi}  &  ~\phi_{t}-\phi_{s}\\
s.t.  &  ~\phi_{y}-\phi_{x}\leq c_{xy}~\forall xy\in\mathcal{A}
\end{align*}

and we can impose normalization $\phi_{s}=0$, so that along the travelled path, $\phi_{x}$ interprets as the distance travelled thus far.

We have advocated for the use of Gurobi as a black box in this problem,but there exists a direct method to find out the minimal cost path by dynamic programming.

* The crucial remark here is that if there is a minimal cost path from $s$ to $t$, then there is one of length at most $\left\vert \mathcal{X}\right\vert$.

For $z\in\mathcal{X}$, and $t\in\mathbb{N}$, let $C_{sz}^{k}$ be the minimal cost of the path from $s$ to $z$ among paths of length at most $k$, with the convention that $C_{sz}^{k}=+\infty$ if there is no such path. One has:

* $C_{ss}^{0}=0$ and $C_{sz}^{0}=+\infty$ for all $z\neq s$, and

* for $t\geq1$, $C_{sz}^{k}$ satisfies the Bellman equation:

\begin{align*}
C_{sz}^{k}=\min\left\{  C_{sz}^{k},\min_{x\in\mathcal{Z}:sz\in\mathcal{A}%
}\left\{  c_{sx}+C_{xz}^{k-1}\right\}  \right\}  .
\end{align*}

It is easy to see that $C_{st}^{\left\vert \mathcal{X}\right\vert }$ is the minimal cost of any path from $s$ to $t$. Shortest-paths algorithms (Dijikstra and Bellman-Ford) implement this idea.

#### Transportation problem

Assume the problem is bipartite, that is $\mathcal{X}=\mathcal{S}\cup\mathcal{T}$ and $\mathcal{A}\subseteq\mathcal{S}\times\mathcal{T}$. That is, there are no intermediate nodes, and an arc can only go directly from a source to a target.

Note that any min-cost flow problem can be recast in this form, by taking the shortest distance between any source node and any target node.

The dual problem is then

\begin{align*}
\max_{\phi}  &  ~\sum_{x\in\mathcal{X}}n_{x}\phi_{x}\\
s.t.  &  ~\phi_{t}-\phi_{s}\leq c_{st}~\forall s\in\mathcal{S},t\in\mathcal{T}
\end{align*}

which is our first encounter with optimal transport (more on this tomorrow).

## Soviet Planning Problem

The problem that we are trying to solve is

\begin{align*}
\min_{\pi \geq 0} \sum_{(x,y) \in \mathcal{A}} \pi_{xy} c_{xy} \\
\text{s.t. } \nabla^T \pi = n
\end{align*}

Suppose that the network has `nbNodes` nodes and `nbArcs` arcs.
* $\pi$ is our object of interest,
* $c$ a vector of length `nbArcs` such that $c_a$ is the transportation cost at arc $a$,
* $\nabla$ is an matrix of size `nbArcs` $\times$ `nbNodes`. If the first arc has $i$ as the origin node and $j$ as the the destination node then $\nabla_{1i} = -1$ and $\nabla_{1j} = 1$,
* $n$, a vector of length `nbNodes` such that $n_x$ is the net demand at node $x$.

The first thing we will try and construct is the [gradient matrix](#grad_mat). It is a `nbArcs` $\times$ `nbNodes` matrix with $-1$ in the origin node of an arc and $1$ in the destination node. We will construct the gradient matrix as a sparse matrix.

```R
?sparseMatrix
```

```R
library("gurobi")
library("Matrix")

# Data cleaning
nsources = dim(spp_data)[1] - 1
ndests = dim(spp_data)[2] - 2
dists = matrix(as.numeric(spp_data[1:68, 2:11]), 68, 10)
demand = matrix(as.numeric(spp_data[1:68, 12]))
supply = matrix(as.numeric(spp_data[69, 2:11]))
nameNodes = c(spp_data[2:(nsources + 1)], dimnames(spp_data)[[2]][2:11])
nonzeros = which(!is.na(dists), arr.ind = TRUE)  # find nonzero entries

nbNodes = nsources+ndests
nbArcs = length(nonzeros[,1])

rows = nonzeros[, 1]
cols = nonzeros[, 2]
distances = dists[nonzeros]
arcs = cbind(rows, cols + nsources, distances)  # we add nsources to cols to generate a new `ID` for the destinations

head(arcs)
```

A matrix: 6 × 3 of type dbl

rows		distances

13	69	 634

16	69	 815

27	69	1237

42	69	1749

43	69	1394

 7	70	 135

We have constructed our matrix of arcs, now we can easily construct our $\nabla$ matrix, as the first column of `arcs` are the indices of the origin node (which will have $-1$ entry in $\nabla$) and the second column is are the indices of the destination nodes (which will have $+1$ entry).

```R
Nabla = sparseMatrix(i = 1:nbArcs, j = arcs[, 1], dims = c(nbArcs, nbNodes), x = -1) + sparseMatrix(i = 1:nbArcs, j = arcs[, 2], dims = c(nbArcs, nbNodes), x = 1)
```

$\nabla^\intercal$ is a `nbNodes` $\times$ `nbArcs` matrix, with the origins ordered first and then the destinations. So the net demand vector $n$ will just involve stacking the net demand of first the origins, then the destinations.

```R
n = c(-demand, supply)
```

Lastly, the cost vector is just the vector of distances.

```R
c = distances
```

Now let's solve using Gurobi:
* `A` = $\nabla^T$
* `obj` = $c$
* `sense` = '$=$'
* `rhs` = $n$
* `modelsense` = '$\min$'.

```R
result = gurobi(list(A = t(Nabla), obj = c, modelsense = "min", rhs = n, sense = "=",
    start = matrix(0, nbArcs, 1)), params = NULL)
pi = result$x
distance = result$objval

print(distance)
```

    Optimize a model with 78 rows, 155 columns and 310 nonzeros
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+01, 3e+03]
      Bounds range     [0e+00, 0e+00]
      RHS range        [5e-01, 3e+02]
    Presolve removed 48 rows and 48 columns
    Presolve time: 0.00s
    Presolved: 30 rows, 107 columns, 214 nonzeros

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
           0    1.6658785e+05   4.252320e+02   0.000000e+00      0s
          37    3.9505200e+05   0.000000e+00   0.000000e+00      0s

    Solved in 37 iterations and 0.00 seconds
    Optimal objective  3.950520000e+05
    [1] 395052

## Subway Problem

We are trying to solve the exact same problem as the subway problem:
\begin{align}
\min_{\pi \geq 0} \sum_{(x,y) \in \mathcal{A}} \pi_{xy} c_{xy} \\
\text{s.t. } \nabla^T \pi = n
\end{align}

Once again let's first construct our $\nabla$ matrix. This will be pretty straightforward as the data is already in the correct format.

```R
sub_nbNodes = max(as.numeric(sub_arcs[, 1]))
sub_nbArcs = dim(sub_arcs)[1]
sub_namesNodes = sub_nodes[, 1]

sub_Nabla = sparseMatrix(i = 1:sub_nbArcs, j = as.numeric(sub_arcs[, 1]), dims = c(sub_nbArcs,
    sub_nbNodes), x = -1) + sparseMatrix(i = 1:sub_nbArcs, j = as.numeric(sub_arcs[,
    2]), dims = c(sub_nbArcs, sub_nbNodes), x = 1)
```

The net demand vector will just be $-1$ in our origin node and $1$ in our destination node.

```R
originNode = 84  #saint-germain des pres
destinationNode = 116  #trocadero
sub_n = rep(0, sub_nbNodes)  # construct vector of exiting flow, net demand is zero
sub_n[c(originNode, destinationNode)] = c(-1, 1)  # except for our origin and destination
```

And finally our cost vector

```R
sub_c = sub_arcs[, 3]
```

Now let's solve using Gurobi:
* `A` = $\nabla^T$
* `obj` = $c$
* `sense` = '$=$'
* `rhs` = $n$
* `modelsense` = '$\min$'.

```R
result = gurobi(list(A = t(sub_Nabla), obj = as.numeric(sub_c), sense = "=", rhs = sub_n,
    modelsense = "min", start = matrix(0, sub_nbArcs, 1)), params = NULL)
pi = result$x
distance = result$objval
```

    Optimize a model with 366 rows, 866 columns and 1732 nonzeros
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [6e+00, 6e+03]
      Bounds range     [0e+00, 0e+00]
      RHS range        [1e+00, 1e+00]
    Presolve removed 301 rows and 602 columns
    Presolve time: 0.00s
    Presolved: 65 rows, 264 columns, 528 nonzeros

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
           0    0.0000000e+00   2.000000e+00   0.000000e+00      0s
          25    5.3943935e+03   0.000000e+00   0.000000e+00      0s

    Solved in 25 iterations and 0.00 seconds
    Optimal objective  5.394393536e+03

Let's deduce what the minimal distance path is

```R
pi = as.logical(pi)
eqpath = which(pi > 0)
sub_arcs[pi, 1:2]
```

A matrix: 10 × 2 of type dbl

V1	V2

 81	238

 82	 81

 83	 82

 84	 83

117	116

118	117

119	118

120	119

237	120

238	237

Which isn't very illuminating...

```R
# Some plotting stuff
library("magick")
library("igraph")
geoCoordinates = sub_nodes[, 3:4]
class(geoCoordinates) = "numeric"

# Function to plot network
plotCurrentNetwork = function(network, curNode) {
    sizeNodes = rep(1, sub_nbNodes)
    sizeNodes[originNode] = 4
    sizeNodes[destinationNode] = 4
    sizeNodes[curNode] = 4
    labelNodes = rep(NA, sub_nbNodes)
    labelNodes[originNode] = sub_namesNodes[originNode]
    labelNodes[destinationNode] = sub_namesNodes[destinationNode]
    labelNodes[curNode] = sub_namesNodes[curNode]
    plot.igraph(network, vertex.label = labelNodes, vertex.label.cex = 1, vertex.size = sizeNodes,
        edge.arrow.size = 0, layout = geoCoordinates)
}

# Begin animation instance
frames = image_graph(width = 1600, height = 1600, res = 200)

thegraph = graph_from_edgelist(sub_arcs[, 1:2])

cont = TRUE
i = originNode
writeLines(paste("Starting from",paste0(sub_namesNodes[i], " (#", i, ")")))
rank = 0
while (cont) {
    rank = rank + 1
    leavingi = which(sub_Nabla[, i] == -1)
    a = intersect(eqpath, leavingi)[1]
    j = which(sub_Nabla[a, ] == 1)[1]
    plotCurrentNetwork(thegraph, j)
    writeLines(paste0(rank, ": ", sub_namesNodes[j], " (#", j, ")"))
    i = j
    if (j == destinationNode) {
        cont <- FALSE
    }
}
# done with plotting
dev.off()
```

    Starting from Saint-Germain des Prés (#84)
    1: Saint-Sulpice (#83)
    2: Saint-Placide (#82)
    3: Montparnasse-Bienvenue (#81)
    4: Duroc (#238)
    5: Ségur (#237)
    6: La Motte-Picquet-Grenelle (#120)
    7: Dupleix (#119)
    8: Bir-Hakeim (Grenelle) (#118)
    9: Passy (#117)
    10: Trocadéro (#116)

png: 2

```R
# animate
animation = image_animate(frames, 1)
image_write(animation, "subway.gif")
print(animation)
```

       format width height colorspace matte filesize density
    1     gif  1600   1600       sRGB  TRUE        0   72x72
    2     gif  1600   1600       sRGB  TRUE        0   72x72
    3     gif  1600   1600       sRGB  TRUE        0   72x72
    4     gif  1600   1600       sRGB  TRUE        0   72x72
    5     gif  1600   1600       sRGB  TRUE        0   72x72
    6     gif  1600   1600       sRGB  TRUE        0   72x72
    7     gif  1600   1600       sRGB  TRUE        0   72x72
    8     gif  1600   1600       sRGB  TRUE        0   72x72
    9     gif  1600   1600       sRGB  TRUE        0   72x72
    10    gif  1600   1600       sRGB  TRUE        0   72x72

## NYC Roads

Again we are trying to solve the shortest path problem.

```R
library("rgdal")
library("rdist")

thePath = paste0(getwd() ,"/../data_mec_optim/networks_NYCroads")
load(paste0(thePath, "/DataNYC.Rdata"))
```

    Loading required package: sp
    rgdal: version: 1.4-4, (SVN revision 833)
     Geospatial Data Abstraction Library extensions to R successfully loaded
     Loaded GDAL runtime: GDAL 2.2.3, released 2017/11/20
     Path to GDAL shared files: C:/Users/jmcgn/OneDrive/Documents/R/win-library/3.6/rgdal/gdal
     GDAL binary built with GEOS: TRUE
     Loaded PROJ.4 runtime: Rel. 4.9.3, 15 August 2016, [PJ_VERSION: 493]
     Path to PROJ.4 shared files: C:/Users/jmcgn/OneDrive/Documents/R/win-library/3.6/rgdal/proj
     Linking to sp version: 1.3-1

```R
shortestPath = function(arcs, nodes, originNode, destinationNode) {
    nbNodes = dim(nodes)[1]
    nbArcs = dim(arcs)[1]
    n = rep(0, nbNodes)
    n[c(originNode, destinationNode)] = c(-1, 1)
    Nabla = sparseMatrix(i = 1:nbArcs, j = arcs[, 1], dims = c(nbArcs, nbNodes),
        x = -1) + sparseMatrix(i = 1:nbArcs, j = arcs[, 2], dims = c(nbArcs, nbNodes),
        x = 1)
    Phi <- -arcs[, 3]

    result = gurobi(list(A = t(Nabla), obj = Phi, modelsense = "max", rhs = n, sense = "=",
        start = matrix(0, nbArcs, 1)), params = NULL)
    pi = result$x
    distance = -result$objval

    cont = TRUE
    i = originNode
    eqpath = which(pi > 0)
    rank = 0
    nodespath = c(0)
    nodespath[rank + 1] = i
    while (cont) {
        rank = rank + 1
        leavingi = which(Nabla[, i] == -1)
        a = intersect(eqpath, leavingi)[1]
        j = which(Nabla[a, ] == 1)[1]
        i = j
        nodespath[rank + 1] = i
        if (j == destinationNode) {
            cont <- FALSE
        }
    }

    return(nodespath)

    plot(shpNYC)
    points(nodes[nodespath, ], col = "red")
}
```

To get coordinates associated with an address:
https://www.gps-coordinates.net/

```R
startlat = 40.70102
startlong = -73.90414
finishlat = 40.7290094
finishlong = -73.9952367

StartGPS = matrix(c(startlong, startlat), ncol = 2)
FinishGPS = matrix(c(finishlong, finishlat), ncol = 2)
StartNAD83 = project(StartGPS, "+proj=lcc +lat_1=40.66666666666666 +lat_2=41.03333333333333 +lat_0=40.16666666666666 +lon_0=-74 +x_0=300000 +y_0=0 +datum=NAD83 +units=us-ft +no_defs")
FinishNAD83 = project(FinishGPS, "+proj=lcc +lat_1=40.66666666666666 +lat_2=41.03333333333333 +lat_0=40.16666666666666 +lon_0=-74 +x_0=300000 +y_0=0 +datum=NAD83 +units=us-ft +no_defs")
StartNode = which.min(cdist(StartNAD83, nodes, metric = "euclidean"))
FinishNode = which.min(cdist(FinishNAD83, nodes, metric = "euclidean"))
nodespath = shortestPath(arcs, nodes, StartNode, FinishNode)
```

    Optimize a model with 67316 rows, 169399 columns and 338796 nonzeros
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [5e+00, 1e+04]
      Bounds range     [0e+00, 0e+00]
      RHS range        [1e+00, 1e+00]

    Concurrent LP optimizer: dual simplex and barrier
    Showing barrier log only...

    Presolve removed 17773 rows and 28823 columns
    Presolve time: 0.54s
    Presolved: 49543 rows, 140576 columns, 281146 nonzeros

    Ordering time: 0.00s

    Barrier statistics:
     AA' NZ     : 8.984e+04
     Factor NZ  : 8.256e+05 (roughly 80 MBytes of memory)
     Factor Ops : 1.873e+07 (less than 1 second per iteration)
     Threads    : 3

                      Objective                Residual
    Iter       Primal          Dual         Primal    Dual     Compl     Time
       0  -4.22816139e+07  3.57483804e+02  5.30e+00 3.84e+03  8.76e+03     1s
       1  -3.80006823e+07 -2.24488020e+03  1.24e-01 2.51e+03  2.13e+03     1s
       2  -2.88697338e+07 -3.02933993e+03  6.99e-03 2.57e+02  3.64e+02     1s
       3  -6.13984474e+06 -5.15559077e+03  2.33e-04 1.35e+01  4.64e+01     1s
       4  -7.72223691e+05 -1.47773761e+04  2.35e-05 5.75e-01  5.43e+00     1s
       5  -3.32665358e+05 -2.65673557e+04  8.72e-06 1.54e-01  2.19e+00     1s
       6  -1.65534906e+05 -3.02040135e+04  3.64e-06 1.03e-01  9.65e-01     1s
       7  -9.77877154e+04 -3.18335966e+04  1.73e-06 2.04e-01  4.70e-01     1s
       8  -4.05263884e+04 -3.22636316e+04  1.98e-07 7.39e-02  5.89e-02     1s
       9  -3.24930927e+04 -3.24397597e+04  9.41e-10 2.86e-03  3.82e-04     1s
      10  -3.24453581e+04 -3.24453003e+04  2.36e-11 3.52e-12  4.11e-07     1s
      11  -3.24453076e+04 -3.24453075e+04  2.34e-13 3.47e-12  4.11e-10     2s

    Barrier solved model in 11 iterations and 1.56 seconds
    Optimal objective -3.24453076e+04

    Crossover log...

           0 DPushes remaining with DInf 6.9903905e-10                 2s

           0 PPushes remaining with PInf 0.0000000e+00                 2s

      Push phase complete: Pinf 0.0000000e+00, Dinf 6.9903905e-10      2s

    Iteration    Objective       Primal Inf.    Dual Inf.      Time
       49435   -3.2445308e+04   0.000000e+00   0.000000e+00      2s

    Solved with barrier
    Solved in 49435 iterations and 2.16 seconds
    Optimal objective -3.244530753e+04

Plot the result (this will take a while, so don't worry).

```R
plot(shpNYC)
points(nodes[nodespath, ], col = "red")
```


**Review 2**

**程$
% text size
% tiny scriptsize small normalsize large Large LARGE huge Huge
% color
% Aquamarine, black, blue, brown, cyan, darkgray, gray, green, lightgray, lime, magenta, olive, orange, pink, purple, red, teal, violet, white, yellow
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\bbspace}{\;\;\;\;\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\given}[1]{\left. #1 \right|}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N}\left( #1 \right)}
\newcommand{\Exp}[1]{\mathrm{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Avar}[1]{\mathrm{Avar}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left( #1 \right)}
\newcommand{\Corr}[1]{\mathrm{Corr}\left( #1 \right)}
\newcommand{\ExpH}{\mathrm{E}}
\newcommand{\VarH}{\mathrm{Var}}
\newcommand{\AVarH}{\mathrm{Avar}}
\newcommand{\CovH}{\mathrm{Cov}}
\newcommand{\CorrH}{\mathrm{Corr}}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\wp}{\text{with probability }}
\newcommand{\FSD}{\text{FSD}}
\newcommand{\SSD}{\text{SSD}}$远星$^\dagger$**

## Question 1

Consider the claim which pays $S_{T_1}/S_{T_0}$ at time $T_1 > T_0 > 0$. What's the value of the claim at time $T_0$? What is the price of the claim at time zero?

>To calculate $\ExpH_{T_0}^\QQ\SB{e^{-r\P{T_1-T_0}} \ffrac{S_{T_1}}{S_{T_0}}}$, note that under $\QQ$, $e^{rt}S_t$ is a martingale. Thus
>
>$$\ExpH_{T_0}^\QQ\SB{e^{-rT_1}S_{T_1}} = \ExpH_{T_0}^\QQ\SB{e^{-rT_0}S_{T_0}} \Rightarrow \ExpH_{T_0}^\QQ\SB{e^{-r\P{T_1-T_0}} \ffrac{S_{T_1}}{S_{T_0}}} = 1$$
>
>Then at time $0$, $\ExpH_{0}^\QQ\SB{e^{-r\P{T_1-0}} \ffrac{S_{T_1}}{S_{T_0}}} =\ExpH_{0}^\QQ\SB{\ExpH_{T_0}^\QQ \SB{e^{-rT_1} \ffrac{S_{T_1}}{S_{T_0}}}} = e^{-rT_0}$
>***
>Or, if you cannot find that $e^{rt}S_t$ is a martingale, you can still solve is alternatively. Under $\QQ$, we have $\dd S_t = r S_t \dd t + \sigma S_t \dd W_t$ and thus we write
>
>$$S_{T_1} = S_{T_0} \exp\CB{\P{r-\ffrac{\sigma^2}{2}}\P{T_1-T_0} +\sigma \P{W_{T_1} - W_{T_0}}}$$
>
>Then, $\ExpH_{T_0}^\QQ\SB{e^{-r\P{T_1-T_0}} \ffrac{S_{T_1}}{S_{T_0}}} = \ExpH_{T_0}^\QQ\SB{\exp\CB{\P{-\ffrac{\sigma^2}{2}}\P{T_1-T_0} +\sigma \P{W_{T_1} - W_{T_0}}}}=1$

## Question 1.extended

Calculate $\ExpH_0^\QQ\SB{e^{-rT_1}\P{S_{T_1} - kS_{T_0}}^+}$, especially when $k = 1$, it becomes the forward start option's price.

>Actually, we are computing $\ExpH_0^\QQ\SB{e^{-rT}\P{S_{T} - kS_{T_1}}^+}$, and that's what a forward start option looks like. However here we keep the "wrong" notation.
>
>$$
\begin{align}
\ExpH_0^\QQ\SB{e^{-rT_1}\P{S_{T_1} - kS_{T_0}}^+} &= \ExpH_0^\QQ\SB{e^{-rT_0} \ExpH_{T_0}^\QQ\SB{e^{-r\P{T_1-T_0}}\P{S_{T_1} - kS_{T_0}}^+}}\\
&= \ExpH_0^\QQ\SB{e^{-rT_0} c\P{T_0,S_{T_0}, T_1,kS_{T_0}} }\\
&= \ExpH_0^\QQ\SB{\ffrac{S_{T_0}}{e^{rT_0}} c\P{T_0,1, T_1,k} },\bspace\text{martingale}\\
&= S_0 c\P{T_0,1,T_1,k} = c\P{T_0,S_0,T_1,kS_0}\\
&= S_0 \N{d_1'} - kS_0e^{-r\P{T_1 - T_0}}\N{d_2'}\\
d_{1,2}' &= \ffrac{1}{\sigma\sqrt{T_1-T_0}}\P{-\log k + \P{r \pm \ffrac{\sigma^2}{2}}\P{T_1-T_0}}
\end{align}$$

## Question 2

Under the Black-Scholes model, calculate the value of an *Asian call option*, with strike price $K$, in which the *arithmetic average* of the stock price is calculated on the basis of just two sampling times, $0$ and $T$, where $T$ is the maturity time of the contract. Find an expression for the value of the corresponding contract when there are three sampling times, $0$, $T/2$ and $T$.

>$$\begin{align}
\ExpH_0^\QQ\SB{e^{-rT}\P{\ffrac{S_0+S_T}{2}-K}^+} &= \ffrac{1}{2}\ExpH_0^\QQ\SB{e^{-rT}\P{S_T - \P{2K-S_0}}^+}\\
&= \ffrac{1}{2}\P{S_0\N{d_1'} - \P{2K-S_0}e^{-rT}\N{d_2'}}\\
d_{1,2}' &= \ffrac{1}{\sigma\sqrt{T-t}}\P{\log\ffrac{S_t}{2K-S_0}+\P{r\pm\ffrac{\sigma^2}{2}}\P{T-t}}
\end{align}$$
>
>And the preceding formula requires that $2K-S_0>0$. If not, then $2K - S_0 \leq 0$ then
>
>$$\begin{align}
\ExpH_0^\QQ\SB{e^{-rT}\P{\ffrac{S_0+S_T}{2}-K}^+} &= \ffrac{1}{2}\ExpH_0^\QQ\SB{e^{-rT}\P{S_T - \P{2K-S_0}}}\\
&= \ffrac{1}{2}\ExpH_0^\QQ\SB{e^{-rT} S_T} - \ffrac{1}{2}e^{-rT}\P{2K-S_0},\bspace\text{martingale}\\
&= \ffrac{1}{2}S_0 - \ffrac{1}{2}e^{-rT}\P{2K-S_0}
\end{align}$$
>
>And when there are three sampling time, the corresponding Asian call option price is
>
>$$\ExpH_0^\QQ\SB{e^{-rT}\P{\ffrac{S_0+S_T+S_{T/2}}{3}-K}^+}$$

## Question 3

An *European call option* on a certain stock has a strike price of $\$30$, a time to maturity of one year, and an implied volatility of $30\%$. An *European put option* on the same stock has a strike price of $\$30$, a time to maturity of one year, and an implied volatility of $33\%$. What is the arbitrage opportunity open to a trader? Does the arbitrage work only when the *lognormal assumption* underlying Black-Scholes-Merton holds? Explain carefully the reasons for your answer.

>Put-call parity implies that European put and call options have the *same implied volatility*.
>
>$$\left.\begin{align}
p_{\text{BS}} + S_0 e^{-qT} &= c_{\text{BS}} + Ke^{-rT}\\
p_{\text{imp}} + S_0 e^{-qT} &= c_{\text{imp}} + Ke^{-rT}\\
\end{align}\right\}\Rightarrow p_{\text{BS}} - p_{\text{imp}} = c_{\text{BS}}- c_{\text{imp}}$$
>
>Now the put has a higher implied volatility, meaning that its actual value is higher than the theoretical value, since ${\large\nu} = \ffrac{\partial c}{\sigma} = \ffrac{\partial p}{\sigma}>0$. Thus the artibtrage opportunity is to *buy the call*, *sell the put* and *short the stock*. This does not depend on the lognormal assumption underlying Black-Scholes-Merton. Put-call parity is true for any set of assumptions.

## Question 4

Consider a *European call* and a *European put* with the *same* strike price and time to maturity. Show that they change in value by the same amount when the volatility increases from a level, $\sigma_1$, to a new level, $\sigma_2$ within a short period of time.

>Define $c_1$ and $p_1$ as the values of the call and the put when the volatility is $\sigma_1$. Define $c_2$ and $p_2$ as the values of the call and the put when the volatility is $\sigma_2$. From put-call parity, we have:
>
>$$\left.\begin{align}
p_1 + S_0 e^{-qT} &= c_1 + Ke^{-rT}\\
p_2 + S_0 e^{-qT} &= c_2 + Ke^{-rT}
\end{align}\right\}\Rightarrow p_1 - p_2 = c_1 - c_2$$

## Question 5

Suppose that the stock price today is $S_t = 10$, the interest rate is $r = 0\%$, and the time to maturity is $0.5$ years. Consider an option whose Black-Scholes price is given by the function

$$c\P{t,S_t} = S_t^2 e^{2\P{T-t}}$$

where time is in annual terms. If your portfolio is long $10$ of these options and long or short $N$ shares of the stock, what should the value of $N$ be so that the portfolio is delta-neutral?

>The $\Delta$ of the portfolio is
>
>$$\ffrac{\partial \P{10c\P{t,S_t} - NS_t}}{\partial S_t} = 20 S_t e^{2\P{T-t}} - N = 200e-N$$
>
>Thus Delta-Neutral requires to short $200e$ shares of stock.

## Question 6

In the previous problem suppose that another option with the same maturity is available with the Black-Scholes price given by the function $\tilde c\P{t,S_t} = S_t^3 e^{6\P{T-t}}$ and time is in annual terms. If you still hold $10$ units of the first option, how many options of the second type and how many shares of the stock would you buy or sell to make a portfolio both delta-neutral and gamma-neutral?

>Suppose we need to short $N$ shares of the stock and buy $M$ units of option of the second type. Then:
>
>$$\begin{align}
\Delta &= \ffrac{\partial \P{10c\P{t,S_t} + M\tilde c\P{t,S_t} - NS_t}}{\partial S_t} = 20 S_t e^{2\P{T-t}} + 3S_t^2M e^{6\P{T-t}}- N = 200e+300Me^3-N\\
\Gamma &= \ffrac{\partial \Delta}{\partial S_t} = 20e^{2\P{T-t}} + 6S_tM e^{6\P{T-t}} = 20e + 60Me^3
\end{align}$$
>
>Let the two be zero we obtain that $M = \ffrac{-1}{3e^3}$, $N = 100e$. Therefore, we should short $100e$ shares of the stock and sell $\ffrac{1}{3e^3}$ units of the second type of options so that the portfolio is delta-neutral and gamma-neutral.

## Question 7

Let $c\P{t, S_t }$ and $p\P{t, S_t }$ denote the values of a European call and put option with the same exercise price $K$ and expiry time $T$. Show that $c\P{t, S_t }-p\P{t, S_t }$ also satisfies the Black-Scholes equation with the final data $c\P{T, S_T }-p\P{T, S_T } = S_T-K$. Deduce that $S_t-Ke^{-r\P{T-t}}$ is also a solution of the Black-Scholes equation. Interpret these results financially.

>Notice that for call, $\ffrac{\partial c}{\partial t}+rS_t \ffrac{\partial c}{\partial S_t}+\ffrac{1}{2}\sigma^2S_t^2\ffrac{\partial^2 c}{\partial S_t^2}-rc = 0$ subject to $c\P{T,S_T} = \P{S_T-K}^+$ and for put, $\ffrac{\partial p}{\partial t}+rS_t \ffrac{\partial p}{\partial S_t}+\ffrac{1}{2}\sigma^2S_t^2\ffrac{\partial^2 p}{\partial S_t^2}-rp = 0$ subject to $p\P{T,S_T} = \P{K-S_T}^+$. Hence,
>
>$$\ffrac{\partial \P{c-p}}{\partial t}+rS_t \ffrac{\partial \P{c-p}}{\partial S_t}+\ffrac{1}{2}\sigma^2S_t^2\ffrac{\partial^2 \P{c-p}}{\partial S_t^2}-r\P{c-p} = 0$$
>
>subject to $c\P{T,S_T} - p\P{T,S_T} = S_T-K$. And that's the equation, however we observe that
>
>$$\ffrac{\partial \P{S_t-Ke^{-r\P{T-t}}}}{\partial t}+rS_t \ffrac{\partial \P{S_t-Ke^{-r\P{T-t}}}}{\partial S_t}+\ffrac{1}{2}\sigma^2S_t^2\ffrac{\partial^2 \P{S_t-Ke^{-r\P{T-t}}}}{\partial S_t^2}-r\P{S_t-Ke^{-r\P{T-t}}} = 0$$
>
>And when $t=T$, $S_T-Ke^{-r\P{T-T}} = S_T-K$. Thus, $c\P{t, S_t }-p\P{t, S_t } = S_t-Ke^{-r\P{T-t}}$. Financially, this means to hold a European call is equivalent to holding a European put and a forward.

## Question 8

Consider the exchange option which entities the holder the right but not the obligation to exchange risky asset $S^\P1$ for another risky asset $S^\P2$ at maturity $T$. Let the price dynamics of $S^\P1$ and $S^\P2$ under the risk neutral measure be governed by, for $i=1,2$,

$$\ffrac{\partial S_t^\P{ i}}{S_t^\P{i}} = \P{r-q_i}\dd t + \sigma_i \dd W_t^\P i$$

The correlation of the two brownian motion has instantaneous correlation $\rho$, such that $\Exp{W_t^\P1 W_t^\P2} = \rho t$. Let $V\P{t,S_t^\P1,S_t^\P2}$ denote the price function of the exchange option, whose terminal payoff is $V\P{T,S_T^\P1,S_T^\P2} = \P{S_T^\P2 - S_T^\P1}^+$. Solve it.

> First consider the wealth process. Write the total value of holding $1$ share of stock as $G_t^\P i\equiv S_t + \d{\int_0^t q_iS_u\;\dd u}$, then
>
>$$\dd V_t = \ffrac{V_t - \P{\pi_t^\P 1 + \pi_t^\P2}}{B_t} \dd B_t + \ffrac{\pi_t^\P1}{G_t^\P1}\dd G_t^\P1 + \ffrac{\pi_t^\P2}{G_t^\P2}\dd G_t^\P2\\
\dd V_t = \P{rV_t + \pi_t^\P1\P{\mu_1 \color{brown}{+ q_1} - r} + \pi_t^\P2\P{\mu_2 \color{brown}{+ q_2} - r}}\dd t + \pi_t^\P1\sigma_1 \dd W_t^\P1 + \pi_t^\P2\sigma_2 \dd W_t^\P2$$
>
>Using **two-dimensional Itô's rule** yields
>
>$$\begin{align}\dd V &= \ffrac{\partial V}{\partial t}\dd t + \ffrac{\partial V}{\partial S_t^\P 1} \dd S_t^\P 1 + \ffrac{\partial V}{\partial S_t^\P 2} \dd S_t^\P 2 + \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P1} \P{\dd S_t^\P 1}^2 + \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P2\partial S_t^\P2} \P{\dd S_t^\P 2}^2\\
&\bspace+ \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P2}\dd S_t^\P 1\dd S_t^\P 2\\
&= \ffrac{\partial V}{\partial t}\dd t + \ffrac{\partial V}{\partial S_t^\P 1} S_t^\P 1 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1} + \ffrac{\partial V}{\partial S_t^\P 2} S_t^\P 2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}\\
&\bspace + \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P1} \P{S_t^\P 1}^2 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1}^2 + \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P2\partial S_t^\P2} \P{S_t^\P 2}^2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}^2\\
&\bspace + \ffrac{1}{2}\ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P2}S_t^\P 1 \SB{\mu_1 \dd t + \sigma_1 \dd W_t^\P 1}S_t^\P 2 \SB{\mu_2 \dd t + \sigma_2 \dd W_t^\P 2}
\end{align}$$
>
>Since $\dd t\dd t = 0$, $\dd W_t^\P 1 \dd W_t^\P 1 = \dd t$, $\dd W_t^\P 1 \dd t = 0$, and $\dd W_t^\P 1 \dd W_t^\P 2 = \rho \dd t$, the preceding it equal to

>$$\begin{align}\dd V &= \Big[ \ffrac{\partial V}{\partial t} + \mu_1 S_t^\P 1 \ffrac{\partial V}{\partial S_t^\P 1} + \mu_2 S_t^\P 2 \ffrac{\partial V}{\partial S_t^\P 2} + \ffrac{1}{2}\P{\sigma_1S_t^\P1}^2 \ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P1} + \ffrac{1}{2}\P{\sigma_2S_t^\P2}^2 \ffrac{\partial^2 V}{\partial S_t^\P2\partial S_t^\P2}\\
&\bspace+ \rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P2}\Big]\dd t + \sigma_1 S_t^\P1 \ffrac{\partial V}{\partial S_t^\P1} \dd W_t^\P 1 + \sigma_2 S_t^\P2 \ffrac{\partial V}{\partial S_t^\P2} \dd W_t^\P 2
\end{align}$$
>
>Thus to replicate, $V\P{t,S_t^\P1,S_t^\P2} = V_t$. And similar to the one-asset case, comparing the $\dd t$ term and the $\dd W_t^\P 1$, $\dd W_t^\P 2$ terms, we have the ***Black-Scholes equation for two assets***:
>
>$$\begin{align}
\ffrac{\partial V}{\partial t} + \ffrac{1}{2}\P{\sigma_1S_t^\P1}^2 \ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P1} + \ffrac{1}{2}\P{\sigma_2S_t^\P2}^2 \ffrac{\partial^2 V}{\partial S_t^\P2\partial S_t^\P2} + \rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \ffrac{\partial^2 V}{\partial S_t^\P1\partial S_t^\P2}&\\
+ \P{r\color{brown}{- q_1}}S_t^\P1 \ffrac{\partial V}{\partial S_t^\P1} + \P{r\color{brown}{- q_2}}S_t^\P2 \ffrac{\partial V}{\partial S_t^\P2} - rV &= 0
\end{align}$$
>
>with the **replicating portfolio** $\ffrac{\pi_t^\P1}{S_t^\P 1} = \ffrac{\partial V}{\partial S_t^\P1}$ and $\ffrac{\pi_t^\P2}{S_t^\P 2} = \ffrac{\partial V}{\partial S_t^\P2}$ and **boundary condition**: $V\P{T,S_T^\P 1,S_T^\P2} = \P{S_T^\P 2 - S_T^\P1}^+ = S_T^\P1\P{\ffrac{S_T^\P 2}{S_T^\P1} - 1}^+$.
>
>Then our conjecture is $V\P{t,S_t^\P1,S_t^\P2} = S_t^\P1 h\P{t,Z_t}$, where $Z_t = \ffrac{S_t^\P2}{S_t^\P1}$. We now derive the governing equation for $h$:
>
>$$\begin{align}
0=\;&S_t^\P 1 \ffrac{\partial h}{\partial t} + \ffrac{1}{2} \P{\sigma_1 S_t^\P1}^2 \color{brown}{S_t^\P1 \ffrac{\partial^2 h}{\partial Z_t^2}\P{\ffrac{\partial Z_t}{\partial S_t^\P1}}^2} + \ffrac{1}{2} \P{\sigma_2 S_t^\P2}^2 S_t^\P1\ffrac{\partial^2 h}{\partial Z_t^2}\P{\ffrac{\partial Z_t}{\partial S_t^\P2}}^2\\
&\bspace+\rho \sigma_1 \sigma_2 S_t^\P1 S_t^\P2 \color{brown}{S_t^\P1\ffrac{\partial^2 h}{\partial Z_t^2}\ffrac{\partial Z_t^2}{\partial S_t^\P1\partial S_t^\P2}}\\
&\bspace+ \P{r\color{brown}{- q_1}}S_t^\P1 \color{brown}{\P{S_t^\P1\ffrac{\partial h}{\partial Z_t}\ffrac{\partial Z_t}{\partial S_t^\P1} + h\P{t,Z_t}}} + \P{r\color{brown}{- q_2}}S_t^\P2 S_t^\P1\ffrac{\partial h}{\partial Z_t}\ffrac{\partial Z_t}{\partial S_t^\P2} - rS_t^\P1 h\P{t,Z_t}\\
\Rightarrow 0=\;& h_t + \ffrac{1}{2} \P{\sigma_1^2 + \sigma_2^2 - 2 \rho\sigma_1\sigma_2}Z_t^2\ffrac{\partial^2 h}{\partial Z_t^2} \color{brown}{+\P{q_1-q_2}Z_t\ffrac{\partial h}{\partial Z_t} - q_1 h\P{t,Z_t}}
\end{align}$$
>
>And indeed, we can use **Feynman–Kac Formula**, with its **boundary condition**: $h\P{T,Z_T} = \P{Z_T-1}^+$ (with "strike price" $K=1$) and $r = q_1$, $\mu=\P{q_1-q_2}\color{brown}{Z_t}$, and $\sigma = \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2} \color{brown}{\cdot Z_t}$. So the preceding *PDE and its boundary condition* is equivalent to the following *SDE* (stochastic differential equation): $\dd Z_t = \P{q_1-q_2}Z_t \dd t + \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2} \cdot Z_t \dd \hat{W}_t$. Or in stardard **Black-Schole equation** form, $\mu=q_1-q_2$ and $\sigma = \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2}$:
>
>$$\ffrac{\dd Z_t}{Z_t} = \P{q_1-q_2} \dd t + \sqrt{\sigma_1^2 + \sigma_2^2-2\rho\sigma_1\sigma_2} \dd \hat{W}_t$$
>
>And similar to regular **BS formula with dividend**, we have:
>
>$$\begin{align}h\P{t,Z_t} &= Z_t \color{brown}{e^{-q_{\void_2}\P{T-t}}}\N{\hat d_1}-1\cdot\color{brown}{e^{-q_{\void_1}\P{T-t}}}\N{\hat d_2}\\
V\P{t,S_t^\P1,S_t^\P2} &= S_t^\P2 \color{brown}{e^{-q_{\void_2}\P{T-t}}}\N{\hat d_1} - S_t^\P1 \color{brown}{e^{-q_{\void_1}\P{T-t}}}\N{\hat d_2}\\
&= e^{-r\P{T-t}}\P{S_t^\P2 e^{\color{brown}{\P{r-q_{\void_2}}}\P{T-t}}\N{\hat d_1} - S_t^\P1 e^{\color{brown}{\P{r-q_{\void_1}}}\P{T-t}}\N{\hat d_2}}\\
\hat d_{1,2} &= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\P{\ffrac{S_t^{\P2}}{S_t^{\P1}}}+ \P{\color{brown}{q_1 - q_2} \pm \ffrac{\sigma^2}{2}}\P{T-t}}\\
&= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\P{\ffrac{S_t^{\P2}}{S_t^{\P1}}}+ \P{\color{brown}{\P{r-q_2} - \P{r-q_1}} \pm \ffrac{\sigma^2}{2}}\P{T-t}}
\end{align}$$

## Question 9

Based on the above pricing formula for exchange options,

$\P{\text a}$ Deduce the European put price formula with the underlying process $S_t^\P2$ and strike $K$.

>Equivalently, this is to use $S_t^\P2$ in exchange of $K$. So first we rewrite the pricing formula for using $S_t^\P2$ in exchange of $S_t^\P1$:
>
>$$\begin{align}
V\P{t,S_t^\P1,S_t^\P2} &= e^{-r\P{T-t}}\P{S_t^\P1 e^{\color{brown}{\P{r-q_{\void_1}}}\P{T-t}}\N{\hat d_1} - S_t^\P2 e^{\color{brown}{\P{r-q_{\void_2}}}\P{T-t}}\N{\hat d_2}}\\
\hat d_{1,2} &= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\P{\ffrac{S_t^{\P1}}{S_t^{\P2}}}+ \P{\color{brown}{q_2 - q_1} \pm \ffrac{\sigma^2}{2}}\P{T-t}}\\
&= \ffrac{1}{\sigma\sqrt{T-t}} \P{\log\P{\ffrac{S_t^{\P1}}{S_t^{\P2}}}+ \P{\color{brown}{\P{r-q_1} - \P{r-q_2}} \pm \ffrac{\sigma^2}{2}}\P{T-t}}
\end{align}$$
>
>Now $S_t^\P1 = K$ and the corresponding $q_1 = r$, $\sigma_1 = 0$ then the preceding is updated as
>
>$$\begin{align}
p\P{t,S_t^\P1,S_t^\P2} &= e^{-r\P{T-t}}\P{K\N{\hat d_1} - S_t^\P2 e^{\color{brown}{\P{r-q_{\void_2}}}\P{T-t}}\N{\hat d_2}}\\
\hat d_{1,2} &= \ffrac{1}{\sigma_2\sqrt{T-t}} \P{\log\P{\ffrac{K}{S_t^{\P2}}}+ \P{\color{brown}{ - \P{r-q_2}} \pm \ffrac{\sigma_2^2}{2}}\P{T-t}}
\end{align}$$

$\P{\text b}$ Show how we can derive the European call price formula from the European put price formula using the put-call symmetry relation.

>The **put-call symmetry relation** with **dividend** is
>
>$$c + Ke^{-rT} = p + S_0e^{-qT}$$
>
>Thus here
>
>$$\begin{align}
c\P{t,S_t^\P1,S_t^\P2} &= e^{-r\P{T-t}}\P{K\N{\hat d_1} - S_t^\P2 e^{\color{brown}{\P{r-q_{\void_2}}}\P{T-t}}\N{\hat d_2}} + S_t^\P2 e^{-q_{\void_2}\P{T-t}} - Ke^{-r\P{T-t}}\\
&= S_t^\P2 e^{-q_{\void_2}\P{T-t}}\P{1-\N{\hat d_2}} - Ke^{-r\P{T-t}}\P{1-\N{\hat d_1}}\\
&= e^{-r\P{T-t}}\P{S_t^\P2 e^{\P{r-q_{\void_2}}\P{T-t}}\N{-\hat d_2} - K\N{-\hat d_1}}\\
\hat d_{1,2} &= \ffrac{1}{\sigma_2\sqrt{T-t}} \P{\log\P{\ffrac{K}{S_t^{\P2}}}+ \P{\color{brown}{ - \P{r-q_2}} \pm \ffrac{\sigma_2^2}{2}}\P{T-t}}
\end{align}$$
>***
>Or directly regard the call as using $K$ in exchange of $S_t^\P2$
>
>$$\begin{align}
c\P{t,S_t^\P1,S_t^\P2} &=  e^{-r\P{T-t}}\P{S_t^\P2 e^{\color{brown}{\P{r-q_{\void_2}}}\P{T-t}}\N{\hat d_1} - K \N{\hat d_2}}\\
\hat d_{1,2} &= \ffrac{1}{\sigma_2\sqrt{T-t}} \P{\log\P{\ffrac{S_t^{\P2}}{K}}+ \P{\color{brown}{\P{r-q_2}} \pm \ffrac{\sigma_2^2}{2}}\P{T-t}}
\end{align}$$

## Question 10

Consider a *future* on an underlying asset which pays $N$ discrete dividends between $t$ and $T$ and let $D_i$ denote the amount of the $i$th dividend paid on the ex-dividend date $t_i$, $i = 1, 2 \cdots, N$. Show that the futures price is given by

$$F\P{t,S_t} = S_te^{r\P{T-t}} - \sum_{i=1}^N D_i e^{r\P{T-t_{\void_i}}}$$

where $S_t$ is the current asset price and $r$ is the riskless interest rate.

>This special future is not applicable to the put call parity. So here we use risk-neutral pricing method. Consider at time $t$, we sell $S_t$ units of bonds, then buy an asset, and short this forward. At time $T$, our payoff is
>
>$$K+\sum_{i=1}^N D_i e^{r\P{T-t_{\void_i}}} - S_te^{r\P{T-t}} = 0 \Rightarrow F\P{t,S_t} = K = S_te^{r\P{T-t}} - \sum_{i=1}^N D_i e^{r\P{T-t_{\void_i}}}$$

$Remark$

>The forward price is not its value. Its value is
>
>$$f\P{t,S_t} = e^{-r\P{T-t}}\ExpH^\QQ_t\SB{S_T-K} = S_t - Ke^{-r\P{T-t}} = \sum_{i=1}^N D_i e^{r\P{t-t_{\void_i}}}$$

## Question 11

Consider a contingent claim whose value at maturity $T$ is given by $\min\P{S_{T_0},S_T}$ where $T_0<T$ is some intermediate time before maturity, and $S_T$ and $S_{T_0}$ are the asset price at $T$ and $T_0$ respectively. Show that the value of the contingent claim at time $t$ is given by

$$\begin{align}
V\P{t,S_t} &= S_t\SB{1-\N{d_1}+e^{-r\P{T-T_0}}\N{d_2}}\\
d_{1,2} &= \ffrac{\P{r-q}\pm\sigma^2/2}{\sigma}\sqrt{T-T_0}
\end{align}$$

>$$\begin{align}
V\P{t,S_t}&= \ExpH^\QQ_t\SB{e^{-r\P{T-t}}\min\P{S_{T_0} - S_T}}\\
&= e^{-r\P{T-t}}\ExpH^\QQ_t\SB{S_T-\P{S_T-S_{T_0}}^+}\\
&= e^{rt}\ExpH^\QQ_t\SB{\ffrac{S_T}{e^{rT}}} - \ExpH^\QQ_t\SB{e^{-r\P{T-t}}\P{S_T-S_{T_0}}^+}
\end{align}$$
>
>The second term is a forward start option discounted to time $t$, thus
>
>$$\begin{align}
V\P{t,S_t}&= S_t - S_t\N{d_1'} + S_te^{-r\P{T-t}}\N{d_2'}\\
&= S_t\P{1-\N{d_1'}+e^{-r\P{T-t}}\N{d_2'}}\\
d_{1,2}' &= \ffrac{\sqrt{T-T_0}}{\sigma}\P{r\pm\ffrac{\sigma^2}{2}}
\end{align}$$
>
>And if there's dividend,
>
>$$\begin{align}
V\P{t,S_t}&= \ExpH^\QQ_t\SB{e^{-r\P{T-t}}\min\P{S_{T_{\void_0}} - S_T}}\\
&= \ffrac{e^{rt}}{e^{qT}}\ExpH^\QQ_t\SB{\ffrac{S_Te^{qT}}{e^{rT}}} - \ExpH^\QQ_t\SB{e^{-r\P{T_{\void_0}-t}}\ExpH^\QQ_{T_{\void_0}} \SB{e^{ -r\P{T-T_{\void_0}}}\P{S_T-S_{T_{\void_0}}}^+}}\\
&= \ffrac{e^{rt}}{e^{qT}}\ffrac{S_te^{qt}}{e^{rt}}-\ExpH^\QQ_t\SB{e^{-r\P{T_{\void_0}-t}} \P{S_{T_{\void_0}}e^{-q\P{T-T_{\void_0}}}\N{d_1'} - S_{T_{\void_0}}e^{-r\P{T-T_{\void_0}}}\N{d_2'}}}\\
&=S_te^{-q\P{T-t}} - \ffrac{e^{rt}}{e^{qT}}\ExpH^\QQ_t\SB{\ffrac{S_{T_{\void_0}}e^{qT_{\void_0}}}{e^{rT_{\void_0} }}}\N{d_1'} + \ffrac{e^{rT_{\void_0}}}{e^{qT_{\void_0}}}\ExpH^\QQ_t\SB{\ffrac{S_{T_{\void_0}}e^{qT_{\void_0}}}{e^{rT_{\void_0} }}}e^{-r\P{T-t}}\N{d_2'}\\
&= S_te^{-q\P{T-t}} - S_t\N{d_1'}e^{-q\P{T-t}} + e^{-q\P{T-t}}e^{q\P{T-T_{\void_0}}}S_t  e^{-r\P{T-T_{\void_0}}} \N{d_2'}\\
&= S_te^{-q\P{T-t}}\P{1-\N{d_1'} + \N{d_2'}e^{-\P{r-q}\P{T-T_{\void_0}}}}\\
d_{1,2}' &= \ffrac{\sqrt{T-T_0}}{\sigma}\P{r-q\pm\ffrac{\sigma^2}{2}}
\end{align}$$

***

FinMath/Models and Pricing of Financial Derivatives/Review_2.ipynb


---
## 01. Optimization. Linear Fit

Eduard Larrañaga (ealarranaga@unal.edu.co)

---

### About this notebook

In this notebook we present the method to define a straight line that fits a dataset.

---

Empirical relationships (e.g., the $M-\sigma_*$ relation for galaxies)
are typically established by taking experimental/observational data
and fitting an analytic function to them.

In this section, we will
introduce the most common curve fitting methods.

Figure from:
[J. E. Greene & L. C. Ho, ApJ, 641:L21-L24, (2006)](https://ui.adsabs.harvard.edu/abs/2006ApJ...641L..21G/abstract)

---

### Some Statistical Concepts of Interest

We will begin by defining some important statistical concepts.

#### Standard Deviation

Consider a dataset with $N$ datapoints denoted by $\{ x_i \}$. The **standard deviation**, $\sigma$, is a function that shows how much variation or *dispersion* from the average exists in the dataset. It is defined by

\begin{equation}
\sigma = \sqrt{\frac{ \sum_{i=1} ^N (x_i - \mu)^2}{N}}
\end{equation}

where

\begin{equation}
\mu = \frac{\sum_{i=1} ^N x_i}{N}.
\end{equation}

A low standard deviation indicates that the data points tend to be very close to the mean (expected value)

A high standard deviation indicates that the data points are spread out over a large range of values.

**Datapoints with different probabilities**

If the data points have different probabilities, for example probability $p_i$ for the point $x_i$, the standard deviation is calculated as

\begin{equation}
\sigma = \sqrt{\frac{ \sum_{i=1} ^N p_i(x_i - \mu)^2}{N}}
\end{equation}

where

\begin{equation}
\mu = \sum_{i=1} ^N p_i x_i .
\end{equation}

**Continuous real-valued data**

The standard deviation of a continuous real-valued variable $x$ with probability density function $f(x)$ is
\begin{equation}
\sigma = \sqrt{\int_{-\infty}^{\infty} (x-\mu)^2 f(x) dx}
\end{equation}

where

\begin{equation}
\mu = \int_{-\infty} ^{\infty} x f(x) dx.
\end{equation}

#### Variance

The variance is the squared standard deviation,

\begin{equation}
\text{var}(x_i) = \sigma^2 = \frac{ \sum_{i=1} ^N (x_i - \mu)^2}{N}
\end{equation}

**Properties of the Variance**

- Variance is non-negative,
\begin{equation}
\text{var}(x_i) \geq 0
\end{equation}

- Variance is invariant with respect to changes in the location parameter,
\begin{equation}
\text{var}(x_i + a) =\text{var}(x_i)
\end{equation}

- If values are scaled by a constant, the variance is scaled by the square of that constant,
\begin{equation}
\text{var}(a x_i) =a^2 \text{var}(x_i)
\end{equation}

- If values $x_i$ are statistically independent (i.e. uncorrelated),
\begin{equation}
\text{var}\left(\sum_i x_i \right) = \sum_i \text{var}(x_i)
\end{equation}

#### Variance of the Mean

From the properties above, it is easy to show that the variance of the mean $\bar{x} = \frac{1}{N} \sum_i x_i$ is

\begin{align}
\text{var} (\bar{x}) = &\text{var} \left( \frac{1}{N} \sum_i x_i \right) \\
= &\frac{1}{N^2} \text{var} \left( \sum_i x_i \right) \\
= &\frac{1}{N^2}   \sum_i \text{var} \left( x_i \right) \\
= & \frac{1}{N^2}   \sum_i \sigma^2\\
= & \frac{1}{N^2}   N \sigma^2\\
= & \frac{\sigma^2}{N}.
\end{align}

This result implies that the variance of the mean decreases when $N$ increases.

#### Covariance

Consider a sample with $N$ datapoints with the form $(x_i, y_i)$ with $i=1,2,...,N$.

The covariance is a measure of the joint variability of two variables and it is defined as

\begin{equation}
\text{cov} (x,y) = \frac{1}{N} \sum_{i=1} ^N (x_i - \mu_x)(y_i - \mu_y)
\end{equation}

where

\begin{align}
\mu_x =& \frac{\sum_{i=1} ^N x_i}{N}\\
\mu_y =& \frac{\sum_{i=1} ^N y_i}{N}.
\end{align}

This expression can be re-written without directly referring to the means as

\begin{equation}
\text{cov} (x,y) = \frac{1}{N^2} \sum_{i=1} ^N \sum_{j=1} ^N \frac{1}{2}(x_i - x_j)(y_i - y_j) = \frac{1}{N^2} \sum_{i} \sum_{j>1}  (x_i - x_j)(y_i - y_j)
\end{equation}

Graphically, the covariance is illustrated as follows

The sign of the covariance shows the tendency in the "linear relationship" between the variables. However, the magnitude of the covariance is not easy to interpret because it is not normalized and hence, it depends on the magnitudes of the variables. The normalized version of the covariance is called the **(Pearson) correlation coefficient**

**Covariance with Itself**

 The covariance in which the two variables are identical, corresponds to the variance,

\begin{equation}
\text{cov}(x,x) = \frac{1}{N} \sum_{i=1} ^N (x_i - \mu_x)(x_i - \mu_x) = \frac{1}{N} \sum_{i=1} ^N (x_i - \mu_x)^2 = \sigma_x^2 = \text{var}(x_i)
\end{equation}

**Other Properties of the Covariance**

Considering $a$ and $b$ as constants,

- $\text{cov}(x,a) = 0 $
- $\text{cov}(x,y) = \text{cov}(y,x)$
- $\text{cov}(a x, by) = ab\text{cov}(x,y)$
- $\text{cov}(x + a, y + b) = \text{cov}(x,y)$

#### Pearson Correlation

Correlation describes the strength of the linear association between two variables and is equivalent to the normalized version of the covariance.

The correlation coefficient is usually denoted as $R$, $r$ or $\rho$ and it is defined as
\begin{equation}
\text{corr}(x,y) = r = \frac{\text{cov}(x,y)}{\sigma_x \sigma_y}
\end{equation}

Using the definitions above, we have that

\begin{align}
r =  &\left( \sqrt{\frac{ \sum_{i=1} ^N (x_i - \mu_x)^2}{N}}
 \right)^{-1} \left( \sqrt{\frac{ \sum_{i=1} ^N (x_i - \mu_x)^2}{N}}
 \right)^{-1} \frac{1}{N} \sum_{i=1} ^N (x_i - \mu_x)(y_i - \mu_y)\\
 r = &\frac{\sum_{i=1} ^N (x_i - \mu_x)(y_i - \mu_y)}{\sqrt{\sum_{i=1} ^N (x_i - \mu_x)^2}\sqrt{\sum_{i=1} ^N (y_i - \mu_y)^2}}
\end{align}

Finally, using the expressions

\begin{align}
\mu_x =& \frac{\sum_{i=1} ^N x_i}{N}\\
\mu_y =& \frac{\sum_{i=1} ^N y_i}{N}.
\end{align}

we get

\begin{equation}
r = \frac{N\sum x_i y_i - \sum x_i \sum y_i}{\sqrt{N\sum x_i^2 - \left(\sum x_i \right)^2}\sqrt{N\sum y_i^2 - \left(\sum y_i \right)^2}}
\end{equation}

In the following figure, it is shown some of the values of the correlation coefficient for some datapoints distributions,

**Some Properties of the Correlation Coefficient**

- The sign of the correlation coefficient indicates the direction of association.

- The correlation coefficient is always between $-1$ (perfect negative linear association) and $+1$ (perfect positive linear association).

- The value $r=0$ indicates no lienar relationship.

- $\text{corr}(x,y)=\text{corr}(y,x)$

- The correlation coefficient is **sensitive to outliers**.

---
## Curve Fitting Criteria

$N$ data points $(x_i,y_i)$

We will fit a general function $Y(x,\{a_j\})$, depending on M-parameters $\{a_j\}$ with $j=1,2,3,...,M$

### Least squares fit

\begin{equation}
\Delta_i = Y(x_i,\{a_j\}) - y_i\,\,,
\end{equation}

In the *least squares fit*, the goal is to minimize the function

\begin{equation}
\Delta(\{a_j\}) = \sum_{i=1}^N \Delta_i^2 = \sum_{i=1}^N \left( Y(x_i,\{a_j\}) - y_i \right)^2\,\,.
\end{equation}

The square of the difference is used because negative
and positive variations would otherwise partially or fully cancel out, leading to a wrong result.

### Data with y-uncertainties

Observational data having an estimated error as $y_i \pm \sigma_i$

Now, the function to minimize is $\chi^2$, defined as

\begin{equation}
\chi^2 (\{a_j \}) = \sum_{i=1}^N \left(\frac{\Delta_i}{\sigma_i} \right)^2 =
\sum_{i=1}^N \left( \frac{Y(x_i,\{a_j\}) - y_i}{\sigma_i} \right)^2
\end{equation}

---
## Linear Regression

The simplest curve to fit some data is a stright
line (**linear regression**). The function to fit has only two parameters: $a_1$ and $a_2$,

\begin{equation}
Y(x, \{a_1,a_2\}) = a_1 + a_2 x\,\,,
\end{equation}

The objective is to determine $a_1$ and $a_2$ such that

\begin{equation}
\chi^2(a_1,a_2) = \sum_{i=1}^N \frac{1}{\sigma_i^2} (a_1 + a_2 x_i - y_i)^2
\end{equation}

is minimized. Therefore, differentiating this equation and setting the result to zero gives the equations

\begin{equation}
\begin{aligned}
\frac{\partial \chi^2}{\partial a_1} &= 2 \sum_{i=1}^N \frac{1}{\sigma_i^2}(a_1 + a_2x_i - y_i) = 0\,\,,\\
\frac{\partial \chi^2}{\partial a_2} &= 2 \sum_{i=1}^N \frac{1}{\sigma_i^2}(a_1 + a_2 x_i - y_i) x_i = 0\,\,.
\end{aligned}
\end{equation}

This system can be written as

\begin{equation}
\begin{aligned}
a_1 S + a_2 \Sigma x - \Sigma y &= 0\,\,,\\
a_1\Sigma x + a_2 \Sigma x^2 - \Sigma xy &=0\,\,,
\end{aligned}
\end{equation}

with

\begin{align}
S =& \sum_{i=1}^N \frac{1}{\sigma_i^2}\\
\Sigma x =& \sum_{i=1}^N \frac{x_i}{\sigma_i^2}\\
\Sigma y =& \sum_{i=1}^N \frac{y_i}{\sigma_i^2}\\
\Sigma x^2 =& \sum_{i=1}^N \frac{x_i^2}{\sigma_i^2}\\
\Sigma xy =& \sum_{i=1}^N \frac{x_i y_i}{\sigma_i^2}\\
\end{align}

Solving for the two unknowns $a_1$ and $a_2$,

\begin{equation}
a_1 = \frac{\Sigma y\Sigma x^2 - \Sigma x \Sigma xy}{S\Sigma x^2 - (\Sigma x)^2}
\,\,,\hspace{2em} a_2 = \frac{S\Sigma xy - \Sigma y \Sigma x}{S\Sigma x^2 - (\Sigma x)^2}\,\,.
\label{eq:analysis_chi23}
\end{equation}

**Notes**
- If all $\sigma_i$ are identical, they will cancel out of the
above equations and $a_1$ and $a_2$ will be independent of them.

- If the $\sigma_i$ are unknown, then one can still use the $\chi^2$ method and
just sets $\sigma_i = 1$.

---
### Data with x- and y-uncertainties

Observational data having an estimated error as $y_i \pm \sigma_i$ and $x_i \pm \sigma^x_i$

The uncertainty in $x_i$ can be incorporated in the $\chi^2$ fit by relating $\sigma^x_i$ into an additional error $\sigma^{\text{extra}}_i$ in $y_i$.

To first order, this can be done by writing

\begin{equation}
\sigma_{i,\text{extra}} = \left|\frac{\partial y}{\partial x} \right|_i \sigma^x_i\,\,,
\end{equation}

where one needs an approximation for the slope $\frac{\partial y}{\partial x}$.

- If both $\sigma_i$ and $\sigma_{i,\text{extra}}$
contribute significantly, one simply adds their squares:
$\sigma_{i,\text{total}}^2$ = $\sigma_i^2$ + $\sigma_{i,\text{extra}}^2$.

- If the error in $x_i$ or $y_i$ is asymmetric about $(x_i,y_i)$ one
could weigh by the maximum of the left and right error, or use
advanced techniques to incorporate this information.

---
### Associated Error to the fit parameters $a_j$

Associated error bar  $\sigma_{a_j}^2$  for the
curve fit parameter $a_j$.

Using first-order error propagation,
we have

\begin{equation}
\sigma^2_{a_j} = \sum_{i=1}^N \left(\frac{\partial a_j}{\partial y_i}
\right)^2 \sigma_i^2\,\,,
\end{equation}

from which we obtain

\begin{equation}
\sigma_{a_1} = \sqrt{\frac{\Sigma x^2}{S\Sigma x^2 - (\Sigma x)^2}}\,\,,
\hspace{2em} \sigma_{a_2} = \sqrt{\frac{S}{S\Sigma x^2 - (\Sigma x)^2}}\,\,.
\end{equation}

---
If the data set doesn't have an associated set of error bars, the error
 $\sigma_{a_j} = \sigma_0$ is estimated from the sample variance of the data,

\begin{equation}
\sigma_0^2 = \frac{1}{N-2} \sum_{i=1}^N \left(y_i - (a_1 + a_2 x_i) \right)^2\,\,.
\end{equation}

The normalization factor $N-2$ of the variance is due to the fact that
we have taken out two parameters ($a_1$ and $a_2$) from the
data.

## Non-Linear Fit

Many non-linear fitting problems may be transformed to linear
problems by a simple change of variables.

**Example**

Consider a power law

\begin{equation}
Z(t, \{\alpha,\beta\}) = \alpha t^{\beta}\,\,.
\end{equation}

This may be rewritten as $Y(x, \{a_1,a_2\}) = a_1 + a_2 x$ with

\begin{equation}
Y = \log Z\,\,,\hspace{1em} x = \log t\,\,\hspace{1em} a_1 = \log
\alpha\,\,,\hspace{1em} a_2 = \beta\,\,.
\end{equation}

**Example**

Consider an exponential

\begin{equation}
Z(t, \{\alpha, \beta\}) = \alpha e^{\beta x}\,\,.
\end{equation}

This may be rewritten as $Y(x, \{a_1,a_2\}) = a_1 + a_2 x$ with

\begin{equation}
Y = \ln Z\,\,,\hspace{1em} a_1 = \ln
\alpha\,\,,\hspace{1em} a_2 = \beta\,\,.
\end{equation}


# Introducing Mechanical Vibrations with a Book Oscillating on a Coffee Mug

## Course Learning Objectives

The primary course objectives are (succinctly) that students will be able to:

- characterize vibrations based on measurements from real and simulated dynamic systems
- design vibrating systems with desired characteristics
- create predictive mathematical and computational models of real vibrating systems

In this notebook, you will get a taste of each of these three using a simple real system we can examine in class. Each class session will follow a similar pattern but with new and more complicated systems as we progress. You will be able to complete each step in the entire modeling-analysis-design iterative loop when the course is over.

## Prerequesites

- Familarity using an interpreted scientific programming language (Python, Matlab, R, etc)
- Basic Python skills will be helpful: how to write a function, assign variables, import things
- Intro dynamics: F=ma, ODEs, energy

## Execution Instructions

Execute code cells by clicking on the cell and pressing the "Run" button in the toolbar or by simultaneously pressing the "Shift" and "Enter" keys on your keyboard.

# Introduction

This notebook introduces a single [degree of freedom](https://en.wikipedia.org/wiki/Degrees_of_freedom_(mechanics) vibratory system: a textbook balancing on a cylindrical coffee mug. The system is implemented as a computational model that you can interact with in order to visualize the free response and compare the computer simulation to a demonstration in the classroom. The video below shows the real system in action:

```python
from IPython.display import YouTubeVideo
YouTubeVideo('B12HbAOKnqI', width=600, height=480)
```

## What are vibrations?

Here we will study a simple vibratory system. A vibrating mechanical system is typically defined as a collection of rigid and flexible objects that interact in a closed envelope. Thinking about the book and the mug, it will oscillate if initially displaced at a small non-horizontal angle and let go. Note that it oscillates about a horizontal position. This position is called an **equilibrium point**, **equilibrium state**, or **equilibrium configuration** which is a natural position the system comes to when there is no motion. Vibration is formally defined as an **oscillation about an equilibrium** and occurs if there is a moving inertial object which has **restoring forces** acting on it.

Vibrations require:

- inertial object that can move
- restoring forces
- equilibrium configuration

During this class, we will examine and explore many different vibratory systems, such as this simple book on a mug system. We will have some live demos, as we are showing now, but in general we will work with computational representations of systems to experiment and learn about the nature of vibration.

## Python imports and setup

This is boiler plate import code that makes all of the necessary libraries available for use and sets up some display setting for the notebook. In Python, all special functionality comes from importing different Python *modules* and *packages*.

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
# makes matplotlib plots show up in notebook and be interactive
%matplotlib widget
```

# Model Description

For the purposes of the introduction we will assume you have come up with a simplified version of the conceptual mechanics that govern the system's motion. In this case there are several assumptions made.

### Exercise

If we want to learn specifically about how the book oscillates on the cup, what might be some good assumptions to make to simplify reality so that we can create a mathematical model?

Here are the main assumptions I started with:

- The book and the cup surfaces are perfectly shaped geometric entities.
- The book rolls without slip on the cup's surface.
- The book is a uniformly dense rectangle that only has planar motion.

The essence of modeling is coming up with appropriate assumptions that let you describe the system in the simplest way such that the model's behavior is sufficient to answer your questions about the system. It is a skill that takes a lot of practice and you will get to practice this a fair amount in this class. A model may be good or it may be bad, depending on what we are hoping to predict. We will check its "goodness" in the analysis section.

Below you will find a pre-determined **free body diagram** that can likely capture the essential vibratory motion of this system.

- $l$: length of the book in meters
- $d$: thickness of the book in meters
- $r$: radius of the cup in meters
- $m$: mass of the book in kilograms
- $h$: height of the mass center above the cup's center plane
- $\theta(t)$: angle of the book in radians

# Objective 1: Numerical Analysis of the Motion

In this section we will make use of a premade **system** to handle the numerical evaluation of the equations of motion. The following cells load this system and name it `sys` so we can make use of it.

```python
from book_balance import BookCupSystem
```

```python
sys = BookCupSystem()
```

`sys` is now a new system object that you can interact with. This system has many variables and functions associated with it. You can see what the system has and can do by using the Python dot notation. Type `sys.` and press the tab key to see a list of all the variables and functions that are associated with this system.

## Analysis Step 1: Visualize the System's Configuration

It is often very helpful to visualize a system's configuration. In this case we need a two dimensional drawing similar to the diagram above. The `plot_configuration()` function let's us see a simple visualization.

```python
sys.plot_configuration();
```

### System constants

One thing that systems have are different constants, for example this system has geometry, such as the book's thickness and length and the cup's radius. The book also has a mass and, in this case, an underlying assumption is that the book is uniformly dense. Note that all of these do not change with time, i.e. they are constant. You can view all of the constants, which are stored in a Python dictionary by typing:

```python
sys.constants
```

A Python dictionary maps keys, in this case the constant's names, to values, the numerical values you'd like to assign to the constant. For example the key `'l'`, for "length", is associated with a value `0.029`. An individual constant's value can be accessed by using square brackets:

```python
sys.constants['l']
```

You can set the values of the constants as such:

```python
sys.constants['l'] = 0.1  # meters
sys.constants
```

```python
sys.plot_configuration();
```

Set the length constant back to it's original value before moving on.

```python
sys.constants['l'] = 0.238  # meters
```

### Coordinates

There are other system values of interest too. Another very important type are those that vary with time.

There are are an infinite number of time varying parameters, but it is often preferable to choose a uniquely simple set of time varying parameters, often called *generalized coordinates*. These coordinates define the **configuration** of the system. In our case, the vertical and horizontal location of the book's mass center could uniquely describe the configuration of the system (if the book can't slip on the cup). But a better choice would be to use the single time varying angle of the books surface relative to horizontal to define the configuration.

The angle of the book is thus a generalized coordinate because no fewer number of time varying parameters can possibly be used to describe the configuration. For simple systems, the number of generalized coordinates corresponds to the number of *degrees of freedom* of a system. The degrees of freedom are the number of independent parameters that define the configuration. The non-slipping book on a cup has 1 degree of freedom which is described by the single generalized coordinate, the book's angle. The system's generalized coordinates can be accessed as such:

```python
sys.coordinates['theta']
```

```python
sys.coordinates['theta'] = np.deg2rad(10)  # radians
```

```python
sys.plot_configuration();
```

### Exercise

Change the system's `constants` and `coordinates` and plot the configuration to see how the coordinates relate to the visual depiction.

*Note that you will be responsible for ensuring that the units are consistent and that all angles should be in radians. Best to stick with SI units.*

```python
# write your answer here
```

## Free Response

Now that we have a system with a defined equation of motion. There are two ways to create this motion: apply perturbing forces to the system or set the coordinate to an initial angle other than the equilibrium angle. We will do the later here. The resulting motion is called the **free response** of the system or the solution to the **initial value problem**, meaning that no external forces are causing the motion. To simulate the free response of the system, some values of time are needed. In this case a final time value, effectively the duration, is passed into the `free_response()` function. First, set the initial angle of the book and then call `free_response()`, storing the returned result in a variable named `trajectories` :

```python
sys.coordinates['theta'] = np.deg2rad(1)
```

```python
trajectories = sys.free_response(5.0)
```

This creates what is called a [data frame](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html). Data frames are defined in the [Pandas](http://pandas.pydata.org/) Python package and are one of the most common Python data types. They are essentially a 2D table with labels for each column and an index for each row. In our case the index is the time value and the columns are the values of the coordinates and the measurements at that particular time:

```python
type(trajectories)
```

```python
trajectories
```

```python
trajectories.index
```

```python
trajectories.theta
```

The data frames have useful plotting functions associated with them, so it is rather easy to plot the various coordinates and measurements versus time:

```python
trajectories.plot(subplots=True);
```

### Exercise

What is the period of oscillation if the book's length, $l$, is $0.1$ m and $0.3$ m? Simulate the system and use the zoom and cross hair feature in the plot to find out.

```python
# write answer here
```

### Exercise

Investigate each constant individually by varying the values. Determine which constants affect the amplitude and period of oscillation and how they are affected. If you increase/decrease a constant do you see increase/decreases in amplitude and/or period?

```python
# write answer here
```

## Analysis Step 3: Animating The Motion

Now that we we have a time varying response, we can animate the configuration figure to visualize how the system moves. Use the `animate_configuration()` function to do so:

```python
sys = BookCupSystem()
sys.coordinates['theta'] = np.deg2rad(1)
sys.free_response(5.0)
sys.animate_configuration(fps=10, repeat=False)
```

### Exercise

Try out changing a constant, generating a new free response, and then animating the motion.

```python
# write answer here
```

## Analysis Step 4: Validation Of the Simulation

Now that we have some data produced from the simulation we can see how it compares to what we can measure in real life.

### Exercise

There are some books of various sizes and some coffee cups available. Select one with some other students, measure its dimensions, the book's mass, and time its period of oscillation. If you set your constants to those that you measuured, does the free response give a good estimate of the period?

```python
# write answer here
```

# Objective 2: Design

When designing vibrating systems, the desired outcome is a specific motion (including the possibilty of no motion) when the system is subjected to some initial conditions and some externally applied forces. The designer has the freedom to manipulate many aspects of the system, for example: degrees of freedom, kinematic relationships, mass/inertia, stiffness, damping, geometry, etc. Given an existing system, one approach to design is to manipulate the constants (mass, stiffness, geometry) to obtain a desired motion.

In general, vibrating systems can be **stable** or **unstable**. A system is stable if it osciallates about its equilibrium and the amplitude stays constant or decreases over time. An unstable system's apmplitude grows unbounded over time. It is almost always desirable for a system to be stable. The goal in this section will be to determine how the parameters of the book/cup system relate to the system's stability.

Imagine glueing three books together in a stack and balancing them on the standard coffee mug. If you give an initial angle of 5 degrees what happens?

```python
sys = BookCupSystem()
sys.constants
```

```python
sys.constants['d'] = 3 * 0.029  # m
```

```python
sys.coordinates['theta'] = np.deg2rad(5)
```

```python
traj = sys.free_response(5.0)
```

```python
traj.plot(subplots=True);
```

```python
animation = sys.animate_configuration(fps=10, repeat=False)
```

### Exercise

Is this model a valid model in this scenario?

write answer here

### Exercise

Is there of a cup radius that ensures the three books vibrates stably on the cup (i.e. doesn't fall off)?

Execute the following cells and a slider will appear, allowing you to see how changing the cup's radius affects the angle trajectory. See if you can find any values of radius that ensure a stable oscillation.

Secondly, find the radius of the cup that will cause the period of oscillation to be 1 second, them maybe we can use this as a clock.

Feel free to animate the system if you happen to find a value.

```python
from ipywidgets import interact, FloatSlider
```

```python
fig, ax = plt.subplots(1, 1)

line = ax.plot(traj.index, np.rad2deg(traj.theta))[0]

ax.set_xlabel('Time [s]')
ax.set_ylabel(r'$\theta$ [deg]')

ax.set_ylim((-90, 90))

def update_line(r=0.042):
    sys.constants['r'] = r
    traj = sys.free_response(5.0)
    line.set_data(traj.index, np.rad2deg(traj.theta))
```

```python
widget = interact(update_line, r=FloatSlider(value=0.042, min=0.0, max=0.1, step=0.005, readout_format='.3f'))
```

```python
# write your answer here
```

The mug radius can be considered a "design parameter", i.e. something an engineering could change, to get the desired behavior. This is a simple example of meeting a design criteria, "stability", with a predictive model.

# Objective 3: Modeling

*Modeling* is the development of a simple abstraction of reality that can be used for predictive purposes. Models are often mathematical but come in many forms. We will work with mathematical and computational models here.

## Modeling Step 1: Make Assumptions and Free Body Diagram

We have given this to you above. More on this step will be developed in class.

## Modeling Step 2: Write the kinetic and potential energy of the system

Once we have a conceptual model, the next task is to form the mathematical equations of motion of the system. These equations most often differential equations that follow [Newton's Laws of Motion](https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion) and are typically expressed in the form of the linear (or translational) equation:

$$ \sum F = ma $$

$$ \sum \textrm{forces} = \textrm{mass} \cdot \textrm{acceleration} $$

and a rotational equation (Euler's equation):

$$ \sum T = I\alpha $$

$$ \sum \textrm{torque} = \textrm{inertia} \cdot \textrm{angular acceleration} $$

We will use **[Lagrange's method](https://en.wikipedia.org/wiki/Lagrangian_mechanics)** of forming the equations of motion of vibrating systems in this class. Lagrange's method is equivalent to the Newton-Euler approach of forming $F=ma$ and $T=I\alpha$ but is often a simpler approach due to only having to determine the total **kinetic and potential energy** of the system.

### Kinetic energy

The total kinetic energy is the sum of the translational kinetic energy:

$$
T_{tran} = \frac{1}{2}mv^2
$$

where:

- $m$: total mass of the moving object
- $v$: magnitude of the translational velocity of the mass center of the object

and rotational kinetic energy:

$$
T_{rot} = \frac{1}{2}I\dot{\theta}^2
$$

- $I$: moment of inertia of the moving object with respect to the mass center
- $\dot{\theta}$: the magnitude of the angular velocity of the moving object

$$
T = T_{tran} + T_{rot}
$$

### Potential energy

The potential energy relates to a conservative restoring forces. In our case, only the force due to gravity creates potential energy. The higher the mass center of the object raises, the more potential energy it has.

$$
U = mgh
$$

where:

- $m$: total mass of the moving object
- $g$: acceleration due to gravity
- $h$: the change in height with respect to some stationary reference point

### Finding the velocity and the height

The book oscillates at an angular rate, $\dot{\theta}$, and the magnitude of the velocity of the center of mass of the book can be shown to be:

$$v = \frac{1}{2} \sqrt{(d^2 + 4 r^2 \theta^2) \dot{\theta}^{2}}$$

The height above the center of the circle can be shown to be:

$$h = \left(r + \frac{d}{2}\right) \cos\theta + r \theta \sin\theta$$

Given the expressions for the magnitude of the translational and angular velocities and the height of the mass center of mass of the moving book we will write symbolic expressions using SymPy which can be accessed via `sm.` using Python's dot notation. For example, you will need `sm.sin()` and `sm.cos()`, the symbolic sine and cosine functions.

First create a function of time like so:

```python
import sympy as sm
sm.init_printing()  # makes SymPy display as pretty math
```

```python
from book_balance import print_eq, eom_in_first_order_form
```

```python
t = sm.symbols('t')
```

```python
theta = sm.Function('theta')(t)
```

Create symbols for the system's constants (mass, geometry, etc.):

```python
d, l, r, m, g = sm.symbols('d, l, r, m, g')
```

Now using SymPy functions, i.e. `sm.<function name>()`, you can create and display mathematical equations. Notice that you can take time derivatives with `sm.diff()`:

```python
v = sm.sqrt((d**2 + 4 * r**2 * theta**2) * sm.diff(theta, t)**2) / 2
```

```python
print_eq('v', v)
```

```python
h = (r + d/2) * sm.cos(theta) + r * theta * sm.sin(theta)
```

```python
print_eq('h', h)
```

### Exercise

Look up the moment of inertia of a rectangle with respect to its geometric center (also it's center of mass under our uniformly dense assumption) and rotation about its longest axis:

https://en.wikipedia.org/wiki/List_of_moments_of_inertia

Create a symbolic expression in terms of our variables (see free body diagram) and assign it to the variable `I`.

**Note**: In Python use `**` to raise a value to a power, not `^`.

```python
# write your answer here, I = ...
```

```python
I = m / 12 * (d**2 + l**2)
```

```python
print_eq('I', I)
```

### Exercise

Write a symbolic expression for the sum of the translational and rotational kinetic energy and store it in the variable `T`.

```python
# write your answer here, T = ...
```

```python
T = m * v**2 / 2 + I * sm.diff(theta, t)**2 / 2
```

```python
print_eq('T', T)
```

### Exercise

Write a symbolic expression for the potential energy and store it in the variable `U`.

```python
# Write your answer here, U = ...
```

```python
U = m * g * h
```

```python
print_eq('U', U)
```

## Modeling Step 3: Form the equations of motion

The Newton-Euler equations of motion are in second order form, i.e. second time derivative of the angular position. We will need them to be in explicit first order form. To put the systems in to explicit first order form, we will need to introduce a new variable called a generalized speed. We define this variable as:

$$\omega = \frac{d\theta}{dt}$$

and it becomes the first of our two differential equations of motion.

```python
omega = sm.Function('omega')(t)
```

The following function is a bit "magic" in the sense that all you know is that it takes the kinetic and potential energy and returns the equations of motion. We will learn how this happens further into the class.

```python
eom_in_first_order_form(T, U, theta, omega, t)
```

The above mathematical equation represents a set of **coupled explicit first order nonlinear ordinary differential equations**. The left hand side has the derivatives of the two state variables $\theta$ and $\omega$ and the right hand side are the expressions one can use to calculate the derivatives given the constants (mass, geometry, etc) and the state variables at a given instance of time.

This is a **mathematical model** that can predict the motion of the system and is derived from our conceptual model which is described with the **free body diagram**.

```python
# write solution here
```

### Exercise

Write a Python function that computes the derivatives of the two **state variables**, $\theta$ and $\omega$. These expressions should come directly from the symbolic equations of motion we formed above.

```python
def rhs(theta, omega, d, l, r, g):
    thetadot = # write your expression here
    omegadot = # write your expression here
    return thetadot, omegadot
```

**Note**: Make sure to use NumPy functions because this should work with floating point numbers. You will likely need `np.sin()` and `np.cos()`. For example:

```python
np.sin(5.0)
```

```python
# write answer here
```

```python
def rhs(theta, omega, d, l, r, g):
    thetadot = omega
    omegadot = (6 * d * g * np.sin(theta) - 12 * g * r * theta * np.cos(theta) -
                12 * r**2 * omega**2 * theta) / (4 * d**2 + l**2 + 12 * r**2 * theta**2)
    return thetadot, omegadot
```

You can test your function to see if it computes with floating point numbers:

```python
rhs(5.0, 2.1, 6.0, 7.0, 12.0, 9.8)
```

If you used all NumPy compatible functions it should also work with arrays:

```python
rhs(5.0 * np.ones(2), 3.6 * np.ones(2), 6.0, 7.0, 12.0, 9.8)
```

# Summary

In this notebook you:

- characterized the books motion by its simulated and real period of oscillation
- validated that the model matched reality and was a good predictor
- found the value of the radius, a design parameter, that governed good

This is a simple of example of what you will be doing in this class. Real systems offer many more interesting dynamics and vibratory motion, especially when there is more than one moving part. We will work with a new system each couple of class periods and get deeper and deeper into the analysis, modeling, and design of vibrating mechanical systems.