# 09 Differentiation
See *Computational Physics* (Landau, Páez, Bordeianu), Chapter 5.1 – 5.6.

These slides include material from  *Computational Physics. eTextBook Python 3rd Edition.* Copyright © 2012 Landau, Rubin, Páez. Used under the Creative-Commons Attribution-NonCommerical-ShareAlike 3.0 Unported License.

## Theory

Taking numerical *derivatives* is based on the elementary definition

$$
\frac{dy(t)}{dt} := \lim_{h\rightarrow 0} \frac{y(t+h) - y(t)}{h}
$$

Problem $h\rightarrow 0$:
* subtractive cancelations in numerator
* overflow `number / epsilon_m`

### Forward difference
Expand $y(t)$ in Taylor series

$$
y(t+h) = y(t) + h y^{(1)}(t) + \frac{h^2}{2!} y^{(2)}(t) + \frac{h^3}{3!} y^{(3)}(t) + \cdots
$$

(with the derivatives $y^{(n)} \equiv \frac{d^{n}y}{dt^n}$)

$$
\frac{y(t+h) - y(t)}{h} = y^{(1)}(t) + \frac{h}{2!} y^{(2)}(t) + \frac{h^2}{3!} y^{(3)}(t) + \cdots
$$

#### Forward difference algorithm

Neglect $\mathcal{O}(h^2)$ terms:

$$
\frac{dy(t)}{dt} \simeq D_\text{fd} y(t) \equiv \frac{y(t+h) - y(t)}{h}
$$

$$
D_\text{fd} y(t) \equiv \frac{y(t+h) - y(t)}{h}
$$

Image from Computational Physics. eTextBook Python 3rd Edition. Copyright © 2012 Landau, Rubin, Páez. Used under the Creative-Commons Attribution-NonCommerical-ShareAlike 3.0 Unported License.

#### Approximation error in the forward difference algorithm
Substitute the Taylor series into $D_\text{fd} y(t)$

$$
D_\text{fd} y(t) = y^{(1)}(t) - \frac{h}{2} y^{(2)}(t) + \cdots
$$
so the error is **first order in $h$**:
$$
y^{(1)}(t) - D_\text{fd} y(t) = \frac{1}{2} h y^{(2)}(t) + \mathcal{O}(h^2)
$$

#### Example for $D_\text{fd}$

Quadratic function
$$
y(t) = a + bt^2 \quad \text{and}\quad y^{(1)}(t) = 2bt
$$
Forward difference approximation
$$
D_\text{fd} y(t) = \frac{y(t+h) - y(t)}{h} = 2bt + bh
$$
is only good for small $h \ll \frac{1}{b}$.

### Central difference
Improve over the (rather bad) error in the *forward difference algorithm* by using the *central difference*.

Image from Computational Physics. eTextBook Python 3rd Edition. Copyright © 2012 Landau, Rubin, Páez. Used under the Creative-Commons Attribution-NonCommerical-ShareAlike 3.0 Unported License.

#### Central difference algorithm
$$
D_\text{cd} y(t) \equiv \frac{y\Big(t + \frac{h}{2}\Big) - y\Big(t - \frac{h}{2}\Big)}{h}
$$

Applied to the parabola $y(t) = a + bt^2$ yields the exact result:
$$
D_\text{cd} y(t)= 2bt
$$

#### Approximation error in $D_\text{cd}$
Insert Taylor expansion...

$$
y^{(1)}(t) - D_\text{cd} y(t) = \frac{1}{24} h^2 y^{(3)}(t) + \mathcal{O}(h^4)
$$

The error is **second order in $h$**.

This is generally *much better* than the forward difference.

"Generally" means "when $y(t)$ is sufficiently smooth", i.e.
\begin{align}
y^{(3)} \frac{h^2}{24} &\ll y^{(2)} \frac{h}{2} \quad\text{or}\\
12\frac{y^{(2)}}{y^{(3)}} &\gg h.
\end{align}

Other difference algorithms exist with even better error behavior (see homework) but all have problems with data that are not smooth. In this case you might have to smooth the data in advance by *interpolation* or *kernel density estimates* (KDE).

### Error assessment
$$
\epsilon_\text{tot} = \epsilon_\text{app} + \epsilon_\text{ro}
$$

**Round-off error of differentiation**: difference of two similar numbers cannot be better than machine precision, always divided by small number $h$:

$$
\epsilon_\text{ro} \approx \frac{\epsilon_m}{h}
$$

**Forward difference algorithmic error**
$$
\epsilon_\text{app}^\text{fd} = y^{(2)} \frac{h}{2}
$$

What is the *most accurate value of $h$* that we can choose for $D_\text{fd}$?

Cross-over from approximation error to round-off error around

$$
\epsilon_\text{app} = \epsilon_\text{ro}
$$

For the forward difference
$$
y^{(2)} \frac{h_\text{fd}}{2} = \frac{\epsilon_m}{h_\text{fd}}\\
h_\text{fd}^2 = \frac{2\epsilon_m}{y^{(2)}}
$$

For rough estimate, assume $y^{(1)} \approx y^{(2)} \approx y^{(3)} \approx 1$:

```python
import numpy as np
def h_fd(eps_m, y2=1):
    return np.sqrt(2*eps_m/y2)
def eps_fd(eps_m, y2=1.):
    h = h_fd(eps_m, y2=y2)
    return 2*eps_m/h
```

```python
eps_m = 1e-15
print("h_fd = {0:.2e}".format(h_fd(eps_m, y2=1)))
print("eps_fd = {0:.2e}".format(eps_fd(eps_m, y2=1)))
```

    h_fd = 4.47e-08
    eps_fd = 4.47e-08

For double precision ($\epsilon_m = 10^{-15}$)
$$
h_\text{fd} \approx 4\times 10^{-8}\\
\epsilon_\text{tot}^\text{fd} \approx 2\epsilon_\text{ro} = \frac{2\epsilon_m}{h_\text{fd}} =  h_\text{fd} = 4 \times 10^{-8}
$$

(Note: simplified because $y^{(2)} \approx 1$.)

Do you expect the "best" $h_\text{cd}$ for the *central difference algorithm* to be larger or smaller than $h_\text{fd}$?

$$
h_\text{cd}^3 = \frac{24\epsilon_m}{y^{(3)}}
$$

Generally $h_\text{cd} > h_\text{fd}$ because the better algorithm allows you to use a bigger step size for a smaller error.

## Student Problem: Finite Difference Operators (Solution)
- implement the three finite difference operators as functions
- test implementations
- `git pull` the course resources and work with the notebook **[09_differentiation/09-differentiation-students.ipynb](http://nbviewer.jupyter.org/github/ASU-CompMethodsPhysics-PHY494/PHY494-resources/blob/master/09_differentiation/09-differentiation-students.ipynb)**.

### Implementation in Python

\begin{align}
D_\text{fd} y(t) &\equiv \frac{y(t+h) - y(t)}{h} \\
D_\text{cd} y(t) &\equiv \frac{y\Big(t + \frac{h}{2}\Big) - y\Big(t - \frac{h}{2}\Big)}{h}
\end{align}

and also the *extended difference algorithm*

\begin{align}
D_\text{ed} y(t) &\equiv \frac{4 D_\text{cd}y(t, h/2) - D_\text{cd}y(t, h)}{3} \\
  &= \frac{8\big(y(t+h/4) - y(t-h/4)\big) - \big(y(t+h/2) - y(t-h/2)\big)}{3h}
\end{align}

```python
def D_fd(y, t, h):
    """Forward difference"""
    return (y(t + h) - y(t))/h

def D_cd(y, t, h):
    """Central difference"""
    # implement

def D_ed(y, t, h):
    """Extended difference"""
    # implement
```

#### Test your implementations
Test function: $y(t) = \cos t$
1. What is the analytical derivative $\frac{d\cos(t)}{dt}$?
1. Calculate the derivative of $y(t) = \cos t$ at $t=0.1, 1, 100$.
1. Print derivative and relative error $E = \frac{D y(t) - y^{(1)}(t)}{y^{(1)}(t)}$ (finite difference value $D y(t)$ compared to the analystical value $y^{(1)}(t)$– use numpy functions for "exact" values) as function of $h$.
1. Reduce $h$ until you reach machine precision, $h \approx \epsilon_m$
1. Plot $\log_{10} |E(h)|$ against $\log_{10} h$.

Try to do the above for all three algorithms

#### Function definitions (Solution)

```python
def D_fd(y, t, h):
    """Forward difference"""
    return (y(t + h) - y(t))/h

def D_cd(y, t, h):
    """Central difference"""
    return (y(t + h/2) - y(t - h/2))/h

def D_ed(y, t, h):
    """Extended difference"""
    return (8*(y(t + h/4) - y(t - h/4)) - (y(t + h/2) - y(t - h/2)))/(3*h)
```

```python
import numpy as np
# test function: np.cos

# Analytical derivative
def y1(t):
    return -np.sin(t)

t_values = np.array([0.1, 1, 100], dtype=np.float64)
```

Use numpy functions for everything because then you can operate on all `t_values` at once.

#### Evaluate the finite difference derivatives
Note that we pass *a function* `y` to the forward difference function `D_fd` and we can also pass a whole array of `t_values`!

```python
D_fd(np.cos, t_values, 0.1)
```

    array([-0.14937587, -0.86706184,  0.46244209])

```python
D_fd(np.cos, t_values, 1e-12)
```

    array([-0.09980905, -0.84154905,  0.50359716])

#### Evaluate the exact derivatives
Compute the exact derivatives (again, operate on all $t$ together... start thinking in numpy arrays!)

```python
y1(t_values)
```

    array([-0.09983342, -0.84147098,  0.50636564])

Calculation of the **absolute error**: subtract the two arrays that you got previously:

```python
D_fd(np.cos, t_values, 0.1) - y1(t_values)
```

    array([-0.04954246, -0.02559086, -0.04392355])

#### Calculate the relative error $E$

```python
def error(Dxx, y, y1, t, h):
    """Relative error.

    Note: will fail for exact 0.
    """
    y1_val = y1(t)
    # return (Dxx(y, t, h) - y1_val/y1_val
    return Dxx(y, t, h)/y1_val - 1
```

Note that we pass again a general function for the difference operator so that we can use `error()` with `D_fd()`, `D_cd()` and `D_ep()`.

```python
error(D_fd, np.cos, y1, t_values, 0.1)
```

    array([ 0.49625125,  0.03041205, -0.08674275])

```python
error(D_fd, np.cos, y1, t_values, 1e-12)
```

    array([-2.44073917e-04,  9.27754603e-05, -5.46734793e-03])

#### Plot $|E|$
Plot $\log_{10} |E(h)|$ against $\log_{10} h$.

```python
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
matplotlib.style.use('ggplot')
```

```python
h_values = 10**(np.arange(-15, -1, 0.1))
abs_errors = np.abs(error(D_fd, np.cos, y1, 0.1, h_values))
```

```python
plt.loglog(h_values, abs_errors, label="t=0.1")
plt.xlabel(r"$h$")
plt.ylabel(r"$|E_\mathrm{fd}|$");
```

##### Forward Difference $D_\text{fd}$
Plot the three different $t$ values together in one plot:

```python
for t in t_values:
    abs_errors = np.abs(error(D_fd, np.cos, y1, t, h_values))
    plt.loglog(h_values, abs_errors, label=r"$t={}$".format(t))
ax = plt.gca()
ax.legend(loc="best")
ax.set_xlabel(r"$h$")
ax.set_ylabel(r"$|E_\mathrm{fd}|$");
```

* error behavior depends on $t$ and on cancellation of errors (e.g. for $t=1$
* algorithmic error decreases for decreasing $h$ until the round of error starts dominating

##### Central Difference $D_\text{cd}$
Plot the three different $t$ values together in one plot:

```python
for t in t_values:
    abs_errors = np.abs(error(D_cd, np.cos, y1, t, h_values))
    plt.loglog(h_values, abs_errors, label=r"$t={}$".format(t))
ax = plt.gca()
ax.legend(loc="best")
ax.set_xlabel(r"$h$")
ax.set_ylabel(r"$|E_\mathrm{cd}|$");
```

##### Extended Difference $D_\text{ed}$
Plot the three different $t$ values together in one plot:

```python
for t in t_values:
    abs_errors = np.abs(error(D_ed, np.cos, y1, t, h_values))
    plt.loglog(h_values, abs_errors, label=r"$t={}$".format(t))
ax = plt.gca()
ax.legend(loc="best")
ax.set_xlabel(r"$h$")
ax.set_ylabel(r"$|E_\mathrm{ed}|$");
```

```python

```


```python
import numpy
from sympy import *
```

```python
def telescoping_method(function, point, n_t, precision):
    x = Symbol('x')
    a = point - 1
    b = point + 1
    polynomial = series(function, x, x0=0, n=n_t).removeO()
    chebyshev = LC(polynomial)*(b - a)**n_t/2**(2*n_t-1)*chebyshevt(n_t, (2*x+a+b)/(b-a))
    chebyshevN = lambdify(x, chebyshev, 'numpy')(point)
    while abs(chebyshevN) < precision:
        polynomial = polynomial - chebyshev
        n_t -= 1
        chebyshev = LC(polynomial)*(b - a)**n_t/2**(2*n_t-1)*chebyshevt(n_t, (2*x+a+b)/(b-a))
        chebyshevN = lambdify(x, chebyshev, 'numpy')(point)
    polynomialN = lambdify(x, polynomial, 'numpy')(point)
    return {"value": polynomialN, "degree": n_t}
```

```python
def telescoping_method_list(function, points, n_t, precision):
    return [telescoping_method(function, point, n_t, precision)["value"] for point in points]
```

```python
x = Symbol('x')
function = ln(1+x)
point = 0
n_t = 15
precision = 0.0001
telescope = telescoping_method(function, point, n_t, precision)
telescope
```

    {'value': -2.6157924107142856e-05, 'degree': 10}

```python
import matplotlib.pyplot as plt
import numpy as np
space = np.linspace(-0.99, 1, 20)
plt.figure(figsize=(12.8, 9.6))
plt.title('Сравнение точной функции и построенного приближения')
plt.plot(space, telescoping_method_list(function, space, n_t, precision), label='Приближение')
plt.plot(space, lambdify(x, ln(1+x), 'numpy')(space), label='Точная функция')
plt.legend()
plt.show()
```


```python
from sympy import init_printing; init_printing();
from sympy import symbols, sin, cos, tan, acos, sqrt, Eq, Function, Array
from einsteinpy.symbolic import MetricTensor, ChristoffelSymbols, RiemannCurvatureTensor

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all" # display all expression in one cell instead of the last one
```

## Spherical Coordinates

Transform Relation from $(r, \theta, \phi)$ to $(x, y, z)$:

\begin{equation}
\left\{\begin{matrix}
x =& r \sin\theta & \cos\phi \\
y =& r \sin\theta & \sin\phi \\
z =& r \cos\theta &
\end{matrix}\right.
\end{equation}

Transform Relation from $(x, y, z)$ to $(r, \theta, \phi)$:
\begin{equation}
\left\{\begin{matrix}
 & r^2 =& x^2+y^2+z^2 & \\
\tan & \phi  =& y/x & \\
& \theta =& \arccos (z/r) & = \arccos (\frac{z}{\sqrt{x^2+y^2+z^2}})
\end{matrix}\right.
\end{equation}

```python
r, theta, phi = symbols('r theta phi', real=True)
x, y, z = [f(r,theta,phi) for f in symbols('x, y, z', real=True, cls=Function)]
```

```python

eq_list = [
    Eq(x, r * sin(theta) * cos(phi)),
    Eq(y, r * sin(theta) * sin(phi)),
    Eq(z, r * cos(theta))]
for eq in eq_list: eq
diffed_eq_list = []
for eq in eq_list:
    for var in [r, theta, phi]:
        diffed_eq_list.append(
            Eq(eq.lhs.diff(var),
               eq.rhs.diff(var)))# .subs(x**2+y**2+z**2, r**2).simplify() )
```

```python
for diffed_eq in diffed_eq_list: diffed_eq
```

\begin{equation}
\vec{e}_{1} = \frac{\partial \vec{r}}{\partial r} = \frac{\partial (x\hat{x}+y\hat{y}+z\hat{z})}{\partial r},\quad
\vec{e}_{2} = \frac{\partial \vec{r}}{\partial \theta} = \frac{\partial (x\hat{x}+y\hat{y}+z\hat{z})}{\partial \theta},\quad
\vec{e}_{3} = \frac{\partial \vec{r}}{\partial \phi} = \frac{\partial (x\hat{x}+y\hat{y}+z\hat{z})}{\partial \phi}
\end{equation}

```python
from einsteinpy.symbolic.vector import GenericVector

e_i = [None, None, None] # The three components would be
for i, var in enumerate([r, theta, phi]):
    e_i_temp = Array([x.diff(var), y.diff(var), z.diff(var)])
    for eq in diffed_eq_list:
        e_i_temp = e_i_temp.subs(eq.lhs, eq.rhs)
    e_i[i] = e_i_temp
```

```python
from sympy import trigsimp
# define the metric for 3d spherical coordinates
metric = [[0 for i in range(3)] for i in range(3)]
for i in range(3):
    for j in range(3):
        metric[i][j] = (lambda vec1, vec2:
                        vec1[0]*vec2[0] + vec1[1]*vec2[1] + vec1[2]*vec2[2])(e_i[i], e_i[j])
        metric[i][j] = trigsimp(metric[i][j])
        # metric[i][j]
metric
```

```python
# define the metric for 3d spherical coordinates
metric = [[0 for i in range(3)] for i in range(3)]
metric[0][0] = 1
metric[1][1] = r**2
metric[2][2] = (r**2)*(sin(theta)**2)
# creating metric object
m_obj = MetricTensor(metric, (r, theta, phi), config='ll')
m_obj.tensor()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & r^{2} & 0\\0 & 0 & r^{2} \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

```python
ch = ChristoffelSymbols.from_metric(m_obj)
ch.config
```

    'ull'

```python
# Calculating Riemann Tensor from Christoffel Symbols
rm1 = RiemannCurvatureTensor.from_christoffels(ch)
rm1.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\end{matrix}\right]$

```python
# Calculating Riemann Tensor from Metric Tensor
rm2 = RiemannCurvatureTensor.from_metric(m_obj)
rm2.tensor()
```

$\displaystyle \left[\begin{matrix}\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\\\left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right] & \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]\end{matrix}\right]$

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


# Almgren and Chriss Model For Optimal Execution of Portfolio Transactions

### Introduction

We consider the execution of portfolio transactions with the aim of minimizing a combination of risk and transaction costs arising from permanent and temporary market impact. As an example, assume that you have a certain number of stocks that you want to sell within a given time frame. If you place this sell order directly to the market as it is, transaction costs may rise due to temporary market impact. On the other hand, if you split up into pieces in time, cost may rise due to volatility in the stock price.

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) provided a solution to this problem by assuming the permanent and temporary market impact functions are linear functions of the rate of trading, and that stock prices follow a discrete arithmetic random walk.

In this notebook, we will take a look at the model used by Almgren and Chriss to solve the optimal liquidation problem. We will start by stating the formal definitions of *trading trajectory*, *trading list*, and *trading strategy* for liquidating a single stock.

### Trading Trajectory, Trading List, and Trading Strategy

We define trading trajectory, trading list, and trading strategy just as Almgren and Chriss did in their [paper](https://cims.nyu.edu/~almgren/papers/optliq.pdf). Suppose we hold $X$ shares of a stock that we want to liquidate before time $T$. Divide $T$ into $N$ intervals of length $\tau=\frac{T}{N}$ and define:

- $t_k = k\tau$ to be discrete times, where $k = 0,..,N$.

- A **trading trajectory** to be the list $(x_0,..,x_N)$, where $x_k$ is the number of shares we plan to hold at time $t_k$. We require that our initial position $x_0 = X$, and that at liquidation time $T$, $x_N = 0$.

- A **trading list** to be $(n_1,..,n_N)$, $n_k =  x_{k-1} - x_k$ as the number of shares that we will sell between times $t_{k-1}$ and $t_k$.

- A **trading strategy** as a rule for determining $n_k$ from the information available at time $t_{k-1}$.

Below, we can see a visual example of a trading trajectory, for $N = 12$.

## Price Dynamics

We will assume that the stock price evolves according to a discrete arithmetic random walk:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k
\end{equation}

for $k = 1,..,N$ and where:

\begin{equation}
S_k = \text{ stock price at time $k$}\hspace{21.6cm}\\
\sigma = \text{ standard deviation of the fluctuations in stock price}\hspace{16.3cm}\\
\tau = \text{ length of discrete time interval}\hspace{20.2cm}\\
\xi_k = \text{ draws from independent random variables}\hspace{17.8cm}
\end{equation}

We will denote the initial stock price as $S_0$. The role of $\xi_k$ is to simulate random price fluctuations using random numbers drawn from a Normal Gaussian distribution with zero mean and unit variance. The code below shows us what this price model looks like, for an initial stock price of $S_0 =$ \$50 dollars, a standard deviation of price fluctuations of $\sigma = 0.379$, and a discrete time interval of $\tau = 1$.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the number of days to follow the stock price
n_days = 100

# Plot the stock price as a function of time
utils.plot_price_model(seed = 0, num_days = n_days)
```

## Market Impact

As we learned previously the price of a stock is affected by market impact that occurs every time we sell a stock. In their model, Almgren and Chriss distinguish between two types of market impact, permanent and temporary market impact. We will now add these two factors into our price model.

### Permanent Impact

Permanent market impact refers to changes in the equilibrium price of a stock as a direct function of our trading. Permanent market impact is called *permanent* because its effect persists for the entire liquidation period, $T$. We will denote the permanent price impact as $g(v)$, and will add it to our price model:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k - \tau g\left(\frac{n_k}{\tau}\right)
\end{equation}

Here, we assumed the permanent impact function, $g(v)$, is a linear function of the trading rate, $v = n_k / \tau$. We will take $g(v)$ to have the form:

\begin{equation}
g(v) = \gamma \left(\frac{n_k}{\tau}\right)
\end{equation}

where $\gamma$ is a constant and has units of (\$/share${}^2$). Replacing this in the above equation we get:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k - \gamma n_k
\end{equation}

With this form, we can see that for each $n$ shares that we sell, we will depress the stock price permanently by $n\gamma$, regardless of the time we take to sell the stocks.

### Temporary Impact

Temporary market impact refers to temporary imbalances in supply and demand caused by our trading. This leads to temporary price movements away from equilibrium. Temporary market impact is called *temporary* because its effect
dissipates by the next trading period. We will denote the temporary price impact as $h(v)$. Given this, the actual stock price at time $k$ is given by:

\begin{equation}
\tilde{S_k} = S_{k-1} - h\left(\frac{n_k}{\tau}\right)
\end{equation}

Where, we have again assumed the temporary impact function, $h(v)$, is a linear function of the trading rate, $v = n_k / \tau$. We will take $h(v)$ to have the form:

\begin{equation}
h(v) = \epsilon \mbox{ sign}(n_k) + \eta \left(\frac{n_k}{\tau}\right)
\end{equation}

where $\epsilon$ and $\eta$ are constants with units (\$/share) and (\$ time/share${}^2$), respectively. It is important to note that $h(v)$ does not affect the price $S_k$.

## Capture

We define the **Capture** to be the total profits resulting from trading along a particular trading trajectory, upon completion of all trades. We can compute the capture via:

\begin{equation}
\sum\limits_{k=1}^{N} n_k \tilde{S_k} = X S_0 + \sum\limits_{k=1}^{N} \left(\sigma \tau^{1/2} \xi_k - \tau g\left(\frac{n_k}{\tau}\right)\right) x_k - \sum\limits_{k=1}^{N} n_k h\left(\frac{n_k}{\tau}\right)
\end{equation}

As we can see this is the sum of the product of the number of shares $n_k$ that we sell in each time interval, times the effective price per share $\tilde{S_k}$ received on that sale.

## Implementation Shortfall

We define the **Implementation Shortfall** as the total cost of trading and is given by:

\begin{equation}
I_s = X S_0 - \sum_{k = 1}^N n_k \tilde{S_k}
\end{equation}

This is what we seek to minimize when determining the best trading strategy!

Note that since $\xi_k$ is random, so is the implementation shortfall. Therefore, we have to frame the minimization problem in terms of the expectation value of the shortfall and its corresponding variance. We'll refer to $E(x)$ as the expected shortfall and $V(x)$ as the variance of the shortfall. Simplifying the above equation for $I_s$, is easy to see that:

\begin{equation}
E(x) = \sum\limits_{k=1}^{N} \tau x_k g\left(\frac{n_k}{\tau}\right) + \sum\limits_{k=1}^{N} n_k h\left(\frac{n_k}{\tau}\right)
\end{equation}

and

\begin{equation}
V(x) = \sigma^2 \sum\limits_{k=1}^{N} \tau {x_k}^2
\end{equation}

The units of $E(x)$ are dollars and the units of $V(x)$ are dollars squared. So now, we can reframe our minimization problem in terms of $E(x)$ and $V(x)$.
For a given level of variance of shortfall, $V(x)$, we seek to minimize the expectation of shortfall, $E(x)$. In the next section we will see how to solve this problem.

## Utility Function

Our goal now is to find the strategy that has the minimum expected shortfall $E(x)$ for a given maximum level of variance $V(x) \ge 0$. This constrained optimization problem can be solved by introducing a Lagrange multiplier $\lambda$. Therefore, our problem reduces to finding the trading strategy that minimizes the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

The parameter $\lambda$ is referred to as **trader’s risk aversion** and controls how much we penalize the variance relative to the expected shortfall.

The intuition of this utility function can be thought of as follows. Consider a stock which exhibits high price volatility and thus a high risk of price movement away from the equilibrium price. A risk averse trader would prefer to trade a large portion of the volume immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. Alternatively, if the price is expected to be stable over the liquidation period, the trader would rather split the trade into smaller sizes to avoid price impact. This trade-off between speed of execution and risk of price movement is ultimately what governs the structure of the resulting trade list.

# Optimal Trading Strategy

Almgren and Chriss solved the above problem and showed that for each value
of risk aversion there is a uniquely determined optimal execution strategy. The details of their derivation is discussed in their [paper](https://cims.nyu.edu/~almgren/papers/optliq.pdf). Here, we will just state the general solution.

The optimal trajectory is given by:

\begin{equation}
x_j = \frac{\sinh \left( \kappa \left( T-t_j\right)\right)}{ \sinh (\kappa T)}X, \hspace{1cm}\text{ for } j=0,...,N
\end{equation}

and the associated trading list:

\begin{equation}
n_j = \frac{2 \sinh \left(\frac{1}{2} \kappa \tau \right)}{ \sinh \left(\kappa T\right) } \cosh \left(\kappa \left(T - t_{j-\frac{1}{2}}\right)\right) X, \hspace{1cm}\text{ for } j=1,...,N
\end{equation}

where $t_{j-1/2} = (j-\frac{1}{2}) \tau$. The expected shortfall and variance of the optimal trading strategy are given by:

In the above equations $\kappa$ is given by:

\begin{align*}
&\kappa = \frac{1}{\tau}\cosh^{-1}\left(\frac{\tau^2}{2}\tilde{\kappa}^2 + 1\right)
\end{align*}

where:

\begin{align*}
&\tilde{\kappa}^2 = \frac{\lambda \sigma^2}{\tilde{\eta}} = \frac{\lambda \sigma^2}{\eta \left(1-\frac{\gamma \tau}{2 \eta}\right)}
\end{align*}

# Trading Lists and Trading Trajectories

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) provided a solution to the optimal liquidation problem by assuming the that stock prices follow a discrete arithmetic random walk, and that the permanent and temporary market impact functions are linear functions of the trading rate.

Almgren and Chriss showed that for each value of risk aversion there is a unique optimal execution strategy. This optimal execution strategy is determined by a trading trajectory and its associated trading list. The optimal trading trajectory is given by:

\begin{equation}
x_j = \frac{\sinh \left( \kappa \left( T-t_j\right)\right)}{ \sinh (\kappa T)}X, \hspace{1cm}\text{ for } j=0,...,N
\end{equation}

and the associated trading list is given by:

\begin{equation}
n_j = \frac{2 \sinh \left(\frac{1}{2} \kappa \tau \right)}{ \sinh \left(\kappa T\right) } \cosh \left(\kappa \left(T - t_{j-\frac{1}{2}}\right)\right) X, \hspace{1cm}\text{ for } j=1,...,N
\end{equation}

where $t_{j-1/2} = (j-\frac{1}{2}) \tau$.

Given some initial parameters, such as the number of shares, the liquidation time, the trader's risk aversion, etc..., the trading list will tell us how many shares we should sell at each trade to minimize our transaction costs.

In this notebook, we will see how the trading list varies according to some initial trading parameters.

## Visualizing Trading Lists and Trading Trajectories

Let's assume we have 1,000,000 shares that we wish to liquidate. In the code below, we will plot the optimal trading trajectory and its associated trading list for different trading parameters, such as trader's risk aversion, number of trades, and liquidation time.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Plot the trading list and trading trajectory. If show_trl = True, the data frame containing the values of the
# trading list and trading trajectory is printed
utils.plot_trade_list(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, show_trl = True)
```

# Implementing a Trading List

Once we have the trading list for a given set of initial parameters, we can actually implement it. That is, we can sell our shares in the stock market according to the trading list and see how much money we made or lost. To do this, we are going to simulate the stock market with a simple trading environment. This simulated trading environment uses the same price dynamics and market impact functions as the Almgren and Chriss model. That is, stock price movements evolve according to a discrete arithmetic random walk and the permanent and temporary market impact functions are linear functions of the trading rate. We are going to use the same environment to train our Deep Reinforcement Learning algorithm later on.

We will describe the details of the trading environment in another notebook, for now we will just take a look at its default parameters. We will distinguish between financial parameters, such the annual volatility in stock price, and the parameters needed to calculate the trade list using the Almgren and Criss model, such as the trader's risk aversion.

```python
import utils

# Get the default financial and AC Model parameters
financial_params, ac_params = utils.get_env_param()
print(financial_params)
print(ac_params)
```

                                  Financial Parameters
    ================================================================================
    Annual Volatility:                12%   Bid-Ask Spread:                    0.125
    Daily Volatility:                0.8%   Daily Trading Volume:          5,000,000
    ================================================================================
                                               Almgren and Chriss Model Parameters
    =========================================================================================================================
    Total Number of Shares to Sell:                           1,000,000   Fixed Cost of Selling per Share:             $0.062
    Starting Price per Share:                                    $50.00   Trader's Risk Aversion:                       1e-06
    Price Impact for Each 1% of Daily Volume Traded:           $2.5e-06   Permanent Impact Constant:                  2.5e-07
    Number of Days to Sell All the Shares:                           60   Single Step Variance:                         0.144
    Number of Trades:                                                60   Time Interval between trades:                   1.0
    =========================================================================================================================

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the random seed
sd = 0

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Implement the trading list for the given parameters
utils.implement_trade_list(seed = sd, lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
```

# The Efficient Frontier of Optimal Portfolio Transactions

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) showed that for each value of risk aversion there is a unique optimal execution strategy. The optimal strategy is obtained by minimizing the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

where $E(x)$ is the **Expected Shortfall**, $V(x)$ is the **Variance of the Shortfall**, and $\lambda$ corresponds to the trader’s risk aversion. The expected shortfall and variance of the optimal trading strategy are given by:

In this notebook, we will learn how to visualize and interpret these equations.

# The Expected Shortfall

As we saw in the previous notebook, even if we use the same trading list, we are not guaranteed to always get the same implementation shortfall due to the random fluctuations in the stock price. This is why we had to reframe the problem of finding the optimal strategy in terms of the average implementation shortfall and the variance of the implementation shortfall. We call the average implementation shortfall, the expected shortfall $E(x)$, and the variance of the implementation shortfall $V(x)$. So, whenever we talk about the expected shortfall we are really talking about the average implementation shortfall. Therefore, we can think of the expected shortfall as follows. Given a single trading list, the expected shortfall will be the value of the average implementation shortfall if we were to implement this trade list in the stock market many times.

To see this, in the code below we implement the same trade list on 50,000 trading simulations. We call each trading simulation an episode. Each episode will consist of different random fluctuations in stock price. For each episode we will compute the corresponding implemented shortfall. After all the 50,000 trading simulations have been carried out we calculate the average implementation shortfall and the variance of the implemented shortfalls. We can then compare these values with the values given by the equations for $E(x)$ and $V(x)$ from the Almgren and Chriss model.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the liquidation time
l_time = 60

# Set the number of trades
n_trades = 60

# Set trader's risk aversion
t_risk = 1e-6

# Set the number of episodes to run the simulation
episodes = 10

utils.get_av_std(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, trs = episodes)

# Get the AC Optimal strategy for the given parameters
ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

# Extreme Trading Strategies

Because some investors may be willing to take more risk than others, when looking for the optimal strategy we have to consider a wide range of risk values, ranging from those traders that want to take zero risk to those who want to take as much risk as possible. Let's take a look at these two extreme cases. We will define the **Minimum Variance** strategy as that one followed by a trader that wants to take zero risk and the **Minimum Impact** strategy at that one followed by a trader that wants to take as much risk as possible. Let's take a look at the values of $E(x)$ and $V(x)$ for these extreme trading strategies. The `utils.get_min_param()` uses the above equations for $E(x)$ and $V(x)$, along with the parameters from the trading environment to calculate the expected shortfall and standard deviation (the square root of the variance) for these strategies. We'll start by looking at the Minimum Impact strategy.

```python
import utils

# Get the minimum impact and minimum variance strategies
minimum_impact, minimum_variance = utils.get_min_param()
```

### Minimum Impact Strategy

This trading strategy will be taken by trader that has no regard for risk. In the Almgren and Chriss model this will correspond to having the trader's risk aversion set to $\lambda = 0$. In this case the trader will sell the shares at a constant rate over a long period of time. By doing so, he will minimize market impact, but will be at risk of losing a lot of money due to the large variance. Hence, this strategy will yield the lowest possible expected shortfall and the highest possible variance, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of \$197,000 dollars but has a very big standard deviation of over 3 million dollars.

```python
minimum_impact
```

<table class="simpletable">
<caption>AC Optimal Strategy for Minimum Impact</caption>
<tr>
  <th>Number of Days to Sell All the Shares:</th>     <td>250</td>     <th>  Initial Portfolio Value:</th>         <td>$50,000,000.00</td>
</tr>
<tr>
  <th>Half-Life of The Trade:</th>                <td>1,284,394.9</td> <th>  Expected Shortfall:</th>                <td>$197,000.00</td>
</tr>
<tr>
  <th>Utility:</th>                               <td>$197,000.00</td> <th>  Standard Deviation of Shortfall:</th>  <td>$3,453,707.55</td>
</tr>
</table>

### Minimum Variance Strategy

This trading strategy will be taken by trader that wants to take zero risk, regardless of transaction costs. In the Almgren and Chriss model this will correspond to having a variance of $V(x) = 0$. In this case, the trader would prefer to sell the all his shares immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. This strategy will yield the smallest possible variance, $V(x) = 0$, and the highest possible expected shortfall, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of over 2.5 million dollars but has a standard deviation equal of zero.

```python
minimum_variance
```

<table class="simpletable">
<caption>AC Optimal Strategy for Minimum Variance</caption>
<tr>
  <th>Number of Days to Sell All the Shares:</th>       <td>1</td>       <th>  Initial Portfolio Value:</th>         <td>$50,000,000.00</td>
</tr>
<tr>
  <th>Half-Life of The Trade:</th>                     <td>0.2</td>      <th>  Expected Shortfall:</th>               <td>$2,562,500.00</td>
</tr>
<tr>
  <th>Utility:</th>                               <td>$2,562,500.00</td> <th>  Standard Deviation of Shortfall:</th>      <td>$0.00</td>
</tr>
</table>

# The Efficient Frontier

The goal of Almgren and Chriss was to find the optimal strategies that lie between these two extremes. In their paper, they showed how to compute the trade list that minimizes the expected shortfall for a wide range of risk values. In their model, Almgren and Chriss used the parameter $\lambda$ to measure a trader's risk-aversion. The value of $\lambda$ tells us how much a trader is willing to penalize the variance of the shortfall, $V(X)$, relative to expected shortfall, $E(X)$. They showed that for each value of $\lambda$ there is a uniquely determined optimal execution strategy. We define the **Efficient Frontier** to be the set of all these optimal trading strategies. That is, the efficient frontier is the set that contains the optimal trading strategy for each value of $\lambda$.

The efficient frontier is often visualized by plotting $(x,y)$ pairs for a wide range of $\lambda$ values, where the $x$-coordinate is given by the equation of the expected shortfall, $E(X)$, and the $y$-coordinate is given by the equation of the variance of the shortfall, $V(X)$. Therefore, for a given a set of parameters, the curve defined by the efficient frontier represents the set of optimal trading strategies that give the lowest expected shortfall for a defined level of risk.

In the code below, we plot the efficient frontier for $\lambda$ values in the range $(10^{-7}, 10^{-4})$, using the default parameters in our trading environment. Each point of the frontier represents a distinct strategy for optimally liquidating the same number of stocks. A risk-averse trader, who wishes to sell quickly to reduce exposure to stock price volatility, despite the trading costs incurred in doing so, will likely choose a value of $\lambda = 10^{-4}$. On the other hand, a trader
who likes risk, who wishes to postpones selling, will likely choose a value of $\lambda = 10^{-7}$. In the code, you can choose a particular value of $\lambda$ to see the expected shortfall and level of variance corresponding to that particular value of trader's risk aversion.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Plot the efficient frontier for the default values. The plot points out the expected shortfall and variance of the
# optimal strategy for the given the trader's risk aversion. Valid range for the trader's risk aversion (1e-7, 1e-4).
utils.plot_efficient_frontier(tr_risk = 1e-6)
```

```python

```

```python

```

```python

```


```python
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import sympy

style_name = 'bmh' #bmh
mpl.style.use(style_name)
np.set_printoptions(precision=4, linewidth =150)

style = plt.style.library[style_name]
style_colors = [ c['color'] for c in style['axes.prop_cycle'] ]

sympy.init_printing()
```

```python
from matplotlib.patches import ConnectionPatch
from matplotlib import animation, rc
from IPython.display import HTML

plt.rcParams["figure.figsize"] = (5,7)

def f(x) :
    return  x**3 - 5*x

def fdot(x):
    return 3*x**2 - 5

fig = plt.figure()
fig.set_tight_layout(True)

ax1 = fig.add_subplot(2, 1, 1)
ax2 = fig.add_subplot(2, 1, 2)

step = 60
x = np.linspace(-3, 3, step)
slope_points = np.linspace(-2.5, 2.5, step)

fdotx = fdot(x)
m_idx = np.where(fdotx<0)[0]
p_idx = np.where(fdotx>=0)[0]

ax1.plot(x, f(x), color=style_colors[0], lw=3)
ax2.plot(x, fdotx, color=style_colors[1], lw=3)
ax2.plot(x[m_idx], fdotx[m_idx], color=style_colors[7], lw=3)

#numerical deriv.
method = 'center'
eps = 0.01
itv=0.6
x = slope_points[0]
diff = (f(x+eps/2) - f(x-eps/2))/eps

img1_cp,    = ax1.plot(x, f(x), 'o', markersize=10, color=style_colors[0], zorder=3)
img1_slope, = ax1.plot([x-itv, x+itv], [f(x)-diff*itv, f(x)+diff*itv], lw=2, color=style_colors[1])
img1_text  = ax1.text(x + 0.1 ,f(x)+0.2, "Slope {:.2f}".format(diff), fontsize=14)
ax1.set_title(r"$f(x)=x^3 - 5x$", fontsize=16)

img2_cp,    = ax2.plot(x, fdot(x), 'o', markersize=10, color=style_colors[1], zorder=3)
con = ConnectionPatch(xyA=(x, fdot(x)), xyB=(x, f(x)), ls='--', coordsA="data", coordsB="data",
              axesA=ax2, axesB=ax1, color=style_colors[3], lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
ax2.add_artist(con)
img2_deriv = ax2.text(x + 0.1 ,fdot(x)+0.2, "Value {:.2f}".format(fdot(x)), fontsize=14)
ax2.set_title(r"Derivative")

def update(i):
    global con

    x = slope_points[i]
    diff = (f(x+eps/2) - f(x-eps/2))/eps

    if diff > 0 :
        color = style_colors[1]
    else :
        color = style_colors[7]

    img1_cp.set_color(color)
    img1_cp.set_data(x, f(x))
    img1_slope.set_color(color)
    img1_slope.set_data([x-itv, x+itv], [f(x)-diff*itv, f(x)+diff*itv])
    img1_text.set_position((x+0.1, f(x)+0.2))
    img1_text.set_text("Slope {:.2f}".format(diff))

    img2_cp.set_color(color)
    img2_cp.set_data(x, fdot(x))
    con.remove()
    con = ConnectionPatch(xyA=(x, fdot(x)+0.5), xyB=(x, f(x)-0.5), ls='--', coordsA="data", coordsB="data",
              axesA=ax2, axesB=ax1, color=color, lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
    ax2.add_artist(con)
    img2_deriv.set_position((x+0.1, fdot(x)+0.2))
    img2_deriv.set_text("Value {:.2f}".format(fdot(x)))

    return img1_cp, img1_slope, img1_text, img2_cp, img2_deriv,

ani = animation.FuncAnimation(fig, update, frames=np.arange(step), interval=30)
#ani.save('deriv-ani.gif', dpi=80, writer='imagemagick')

rc('animation', html='html5')
HTML(ani.to_html5_video())
```

```python
from matplotlib.patches import ConnectionPatch
from matplotlib import animation, rc
from IPython.display import HTML

plt.rcParams["figure.figsize"] = (8,6)

step = 60

x = np.linspace(-4, 4, step)
def f(x):
    return x**3 - 15*x + 30

fig = plt.figure()
fig.set_tight_layout(True)
ax1 = fig.add_subplot(221)

p1x = x[0]; p1y = f(p1x);
p3x = p1y;  p3y = np.log(p3x)

# 원래 그래프, x, y를 바꿔서 그린다.
ax1.plot(f(x), x, lw=3, label="f(x)", color=style_colors[0])
P1, = ax1.plot(p1y, p1x, 'o')
anno = ax1.annotate("", xy=np.array([6, p1x]), xytext=(p1y, p1x),
                fontsize=20,color=style_colors[1],
                arrowprops=dict(arrowstyle="<|-", color=style_colors[0], lw=2,
                                fc="w", ls="--", alpha=0.6, connectionstyle="arc3,rad=0.0",),
                )
ax1.set_title(r"$y=x^3 - 15x + 30$", fontsize=18)
ax1.set_xlabel("$y$", fontsize=18)
ax1.set_ylabel(r"$x$", fontsize=18)

#같이 그린 그래프
ax2 = fig.add_subplot(222)
ax2.plot(x, f(x), lw=3, label="f(x)", color=style_colors[0])
ax2_1 = ax2.twinx()
ax2_1.plot(x, np.log(f(x)), lw=3, label="log f(x)", color=style_colors[1])
ax2.set_title(r'$f(x)$ and $\logf(x)$', fontsize=18)

# 로그 그래프
ax3 = fig.add_subplot(223)
logx = np.linspace(*ax1.get_xlim(), step)
ax3.set_xlim(ax1.get_xlim())
ax3.plot(logx, np.log(logx), lw=3, label="log(x)", color=style_colors[1])
P3, = ax3.plot(p3x, p3y, 'o', color=style_colors[1])
ax3.set_title(r"$\log(y)$", fontsize=18)
ax3.set_xlabel("$y$", fontsize=18)
ax3.set_ylabel(r"$\log(y)$", fontsize=18)

# 로그 씌운 그래프
ax4 = fig.add_subplot(224, sharey=ax3)
ax4.plot(x, np.log(f(x)), lw=3, label="log f(x)", color=style_colors[1])
P4, = ax4.plot(p1x, p3y, 'o', color=style_colors[1])
ax4.set_xlabel("$x$", fontsize=18)
ax4.set_ylabel(r"$\logf(x)$", fontsize=18)

con1 = ConnectionPatch(xyA=(p3x, p3y), xyB=(p1y, p1x), ls='--', coordsA="data", coordsB="data",
              axesA=ax3, axesB=ax1, color=style_colors[0], lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
ax3.add_artist(con1)

con2 = ConnectionPatch(xyA=(p1x, p3y), xyB=(p3x, p3y), ls='--', coordsA="data", coordsB="data",
              axesA=ax4, axesB=ax3, color=style_colors[1], lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
ax4.add_artist(con2)

plt.subplots_adjust(hspace=0.5)

def update(i):
    global con1, con2, step
    x = np.linspace(-4, 4, step)

    p1x = x[i]; p1y = f(p1x);
    p3x = p1y;  p3y = np.log(p3x)

    P1.set_data(p1y, p1x)
    anno.xy     = [6, p1x]
    anno.set_position([p1y, p1x])

    P3.set_data(p3x, p3y)
    P4.set_data(p1x, p3y)

    con1.remove()
    con1 = ConnectionPatch(xyA=(p3x, p3y), xyB=(p1y, p1x), ls='--', coordsA="data", coordsB="data",
              axesA=ax3, axesB=ax1, color=style_colors[0], lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
    ax3.add_artist(con1)

    con2.remove()
    con2 = ConnectionPatch(xyA=(p1x, p3y), xyB=(p3x, p3y), ls='--', coordsA="data", coordsB="data",
              axesA=ax4, axesB=ax3, color=style_colors[1], lw=2, alpha=0.6,
              arrowstyle="<|-", mutation_scale=20, fc="w", zorder=2)
    ax4.add_artist(con2)

    return P1, P3, P4

ani = animation.FuncAnimation(fig, update, frames=np.arange(len(x)), interval=30)
#ani.save('log-extrema.gif', dpi=80, writer='imagemagick')

rc('animation', html='html5')
HTML(ani.to_html5_video())

#plt.show()
```

```python
###########################################
# 미분 = 기울기를 나타내는 애니메이션
###########################################

from matplotlib import animation, rc
from IPython.display import HTML

plt.rcParams["figure.figsize"] = (6,6)

x = np.linspace(0,5)

def f(x):
    return 2*x**3 + 5*x**2

a , b = 3, 5
d  = 0.01

fig = plt.figure()
fig.set_tight_layout(True)

plt.plot(x, f(x), lw=3)
avg, = plt.plot( (a,b), (f(a), f(b)), lw=2) #평균변화율
mp,  = plt.plot(b, f(b), 'ro')
dx,  = plt.plot( (a,b), (f(a), f(a)), lw=2) #dx
dy,  = plt.plot( (b,b), (f(a), f(b)), lw=2) #dy
dx2, = plt.plot( (a,a), (0, f(a)), '--', lw=2 )
dy2, = plt.plot( (b,b), (0, f(a)), '--', lw=2 )
img1_text  = plt.text(1, 300, "Slope {:.2f}".format((f(b)-f(a))/(b-a)), fontsize=14)

plt.xticks( [a, b], ['a', 'b'] );
plt.xticks(fontsize=20)
plt.yticks(fontsize=15)

def update(i):
    global b

    if b - a < d/100 :
        return
    else :
        b -= d

    #a, f(a)를 지나고 기울기 (f(b)-f(a))/(b-a)인 직선을 x=1,x=5를 양끝으로 하는 직선
    l = ((f(b)-f(a))/(b-a))*(1-a) + f(a)
    r = ((f(b)-f(a))/(b-a))*(5-a) + f(a)
    avg.set_data([1,5], [l,r])
    mp.set_data(b, f(b))
    img1_text.set_text("Slope {:.2f}".format((r-l)/(5-1)))

    dx.set_data((a,b), (f(a), f(a)))
    dx2.set_data((a,a), (0, f(a)))

    dy.set_data((b,b), (f(a), f(b)))
    dy2.set_data((b,b), (0, f(a)))
    plt.xticks( [a, b], ['a', 'b'] );

    return avg

ani = animation.FuncAnimation(fig, update, frames=np.arange(250), interval=20)
#rc('animation', html='html5')
#HTML(ani.to_html5_video())

ani.save('imgs/diff-ani.gif', dpi=80, writer='imagemagick', fps=60)
#plt.show()
```

```python
###########################################
# 미분 가능을 나타내는 애니메이션
###########################################

from matplotlib import animation, rc
from IPython.display import HTML

plt.rcParams["figure.figsize"] = (6,6)

x = np.linspace(0,5)

def f(x):
    return 2*x**3 + 5*x**2

c, a , b = 1, 3, 5
d  = 0.01

fig = plt.figure()
fig.set_tight_layout(True)

plt.plot(x, f(x), lw=3)
avg_r, = plt.plot( (a,b), (f(a), f(b)), lw=2) #평균변화율
avg_l, = plt.plot( (c,a), (f(c), f(a)), lw=2)

mp_r, = plt.plot(b, f(b), 'ro')
mp_l, = plt.plot(c, f(c), 'bo')

img1_text  = plt.text(1, 300, "Right slope {:.2f}".format((f(b)-f(a))/(b-a)), fontsize=14)
img2_text  = plt.text(1, 280, "Left  slope {:.2f}".format((f(a)-f(c))/(a-c)), fontsize=14)

plt.xticks( [c, a, b], ['c', 'a', 'b'] );
plt.xticks(fontsize=20)
plt.yticks(fontsize=15)

def update(i):
    global b
    global c

    if b - a < (d / 100) :
        return
    else :
        b -= d
        c += d

    #a, f(a)를 지나고 기울기 (f(b)-f(a))/(b-a)인 직선을 x=1,x=5를 양끝으로 하는 직선
    l = ((f(b)-f(a))/(b-a))*(1-a) + f(a)
    r = ((f(b)-f(a))/(b-a))*(5-a) + f(a)
    avg_r.set_data([1,5], [l,r])
    img1_text.set_text("Right slope {:.2f}".format((r-l)/(5-1)))

    l = ((f(c)-f(a))/(c-a))*(1-a) + f(a)
    r = ((f(c)-f(a))/(c-a))*(5-a) + f(a)
    avg_l.set_data([1,5], [l,r])
    img2_text.set_text("Left  slope {:.2f}".format((r-l)/(5-1)))

    mp_r.set_data(b, f(b))
    mp_l.set_data(c, f(c))

    #dx.set_data((a,b), (f(a), f(a)))
    #dx2.set_data((a,a), (0, f(a)))

    #dy.set_data((b,b), (f(a), f(b)))
    #dy2.set_data((b,b), (0, f(a)))

    plt.xticks( [c, a, b], ['c', 'a', 'b'] );

    #return avg

ani = animation.FuncAnimation(fig, update, frames=np.arange(250), interval=20)
#rc('animation', html='html5')
#HTML(ani.to_html5_video())

ani.save('imgs/diff-ani-psb.gif', dpi=80, writer='imagemagick', fps=60)
#plt.show()
```

```python
###########################################
# 미분 불가능을 나타내는 애니메이션
###########################################

from matplotlib import animation, rc
from IPython.display import HTML

plt.rcParams["figure.figsize"] = (6,6)

x = np.linspace(0,5)

def f(x):
    return (2*x**3 + 5*x**2) + 20*np.abs(x-3)

c, a , b = 1, 3, 5
d  = 0.01

fig = plt.figure()
fig.set_tight_layout(True)

plt.plot(x, f(x), lw=3)
avg_r, = plt.plot( (a,b), (f(a), f(b)), lw=2) #평균변화율
avg_l, = plt.plot( (c,a), (f(c), f(a)), lw=2)

mp_r, = plt.plot(b, f(b), 'ro')
mp_l, = plt.plot(c, f(c), 'bo')

img1_text  = plt.text(1, 300, "Right slope {:.2f}".format((f(b)-f(a))/(b-a)), fontsize=14)
img2_text  = plt.text(1, 280, "Left  slope {:.2f}".format((f(a)-f(c))/(a-c)), fontsize=14)

plt.xticks( [c, a, b], ['c', 'a', 'b'] );
plt.xticks(fontsize=20)
plt.yticks(fontsize=15)

def update(i):
    global b
    global c

    if b - a < (d / 100) :
        return
    else :
        b -= d
        c += d

    #a, f(a)를 지나고 기울기 (f(b)-f(a))/(b-a)인 직선을 x=1,x=5를 양끝으로 하는 직선
    l = ((f(b)-f(a))/(b-a))*(1-a) + f(a)
    r = ((f(b)-f(a))/(b-a))*(5-a) + f(a)
    avg_r.set_data([1,5], [l,r])
    img1_text.set_text("Right slope {:.2f}".format((r-l)/(5-1)))

    l = ((f(c)-f(a))/(c-a))*(1-a) + f(a)
    r = ((f(c)-f(a))/(c-a))*(5-a) + f(a)
    avg_l.set_data([1,5], [l,r])
    img2_text.set_text("Left  slope {:.2f}".format((r-l)/(5-1)))

    mp_r.set_data(b, f(b))
    mp_l.set_data(c, f(c))

    #dx.set_data((a,b), (f(a), f(a)))
    #dx2.set_data((a,a), (0, f(a)))

    #dy.set_data((b,b), (f(a), f(b)))
    #dy2.set_data((b,b), (0, f(a)))

    plt.xticks( [c, a, b], ['c', 'a', 'b'] );

    #return avg

ani = animation.FuncAnimation(fig, update, frames=np.arange(250), interval=20)
#rc('animation', html='html5')
#HTML(ani.to_html5_video())

ani.save('imgs/diff-ani-impsb.gif', dpi=80, writer='imagemagick', fps=60)
#plt.show()
```

```python
%%html
<link href='https://fonts.googleapis.com/earlyaccess/notosanskr.css' rel='stylesheet' type='text/css'>
<!--https://github.com/kattergil/NotoSerifKR-Web/stargazers-->
<link href='https://cdn.rawgit.com/kattergil/NotoSerifKR-Web/5e08423b/stylesheet/NotoSerif-Web.css' rel='stylesheet' type='text/css'>
<!--https://github.com/Joungkyun/font-d2coding-->
<link href="http://cdn.jsdelivr.net/gh/joungkyun/font-d2coding/d2coding.css" rel="stylesheet" type="text/css">
<style>
    h1     { font-family: 'Noto Sans KR' !important; color:#348ABD !important;   }
    h2     { font-family: 'Noto Sans KR' !important; color:#467821 !important;   }
    h3, h4 { font-family: 'Noto Sans KR' !important; color:#A60628 !important;   }
    p:not(.navbar-text) { font-family: 'Noto Serif KR', 'Nanum Myeongjo'; font-size: 12pt; line-height: 200%;  text-indent: 10px; }
    li:not(.dropdown):not(.p-TabBar-tab):not(.p-MenuBar-item):not(.jp-DirListing-item):not(.p-CommandPalette-header):not(.p-CommandPalette-item):not(.jp-RunningSessions-item)
            { font-family: 'Noto Serif KR', 'Nanum Myeongjo'; font-size: 12pt; line-height: 200%; }
    table  { font-family: 'Noto Sans KR' !important;  font-size: 11pt !important; }
    li > p  { text-indent: 0px; }
    li > ul { margin-top: 0px !important; }
    sup { font-family: 'Noto Sans KR'; font-size: 9pt; }
    code, pre  { font-family: D2Coding, 'D2 coding' !important; font-size: 12pt !important; line-height: 130% !important;}
    .code-body { font-family: D2Coding, 'D2 coding' !important; font-size: 12pt !important;}
    .ns        { font-family: 'Noto Sans KR'; font-size: 15pt;}
    .summary   {
                   font-family: 'Georgia'; font-size: 12pt; line-height: 200%;
                   border-left:3px solid #FF0000;
                   padding-left:20px;
                   margin-top:10px;
                   margin-left:15px;
               }
    .green { color:#467821 !important; }
    .comment { font-family: 'Noto Sans KR'; font-size: 10pt; }
</style>
```

<link href='https://fonts.googleapis.com/earlyaccess/notosanskr.css' rel='stylesheet' type='text/css'>
<!--https://github.com/kattergil/NotoSerifKR-Web/stargazers-->
<link href='https://cdn.rawgit.com/kattergil/NotoSerifKR-Web/5e08423b/stylesheet/NotoSerif-Web.css' rel='stylesheet' type='text/css'>
<!--https://github.com/Joungkyun/font-d2coding-->
<link href="http://cdn.jsdelivr.net/gh/joungkyun/font-d2coding/d2coding.css" rel="stylesheet" type="text/css">
<style>
    h1     { font-family: 'Noto Sans KR' !important; color:#348ABD !important;   }
    h2     { font-family: 'Noto Sans KR' !important; color:#467821 !important;   }
    h3, h4 { font-family: 'Noto Sans KR' !important; color:#A60628 !important;   }
    p:not(.navbar-text) { font-family: 'Noto Serif KR', 'Nanum Myeongjo'; font-size: 12pt; line-height: 200%;  text-indent: 10px; }
    li:not(.dropdown):not(.p-TabBar-tab):not(.p-MenuBar-item):not(.jp-DirListing-item):not(.p-CommandPalette-header):not(.p-CommandPalette-item):not(.jp-RunningSessions-item)
            { font-family: 'Noto Serif KR', 'Nanum Myeongjo'; font-size: 12pt; line-height: 200%; }
    table  { font-family: 'Noto Sans KR' !important;  font-size: 11pt !important; }
    li > p  { text-indent: 0px; }
    li > ul { margin-top: 0px !important; }
    sup { font-family: 'Noto Sans KR'; font-size: 9pt; }
    code, pre  { font-family: D2Coding, 'D2 coding' !important; font-size: 12pt !important; line-height: 130% !important;}
    .code-body { font-family: D2Coding, 'D2 coding' !important; font-size: 12pt !important;}
    .ns        { font-family: 'Noto Sans KR'; font-size: 15pt;}
    .summary   {
                   font-family: 'Georgia'; font-size: 12pt; line-height: 200%;
                   border-left:3px solid #FF0000;
                   padding-left:20px;
                   margin-top:10px;
                   margin-left:15px;
               }
    .green { color:#467821 !important; }
    .comment { font-family: 'Noto Sans KR'; font-size: 10pt; }
</style>


# Tools for Scientific Python

# scipy

General-purpose scientific toolbox, including signal processing, statistics, optimization, linear algebra, etc.

http://docs.scipy.org/doc/scipy/reference/

```python
import scipy.stats

# probability density at x=2.5 for Gaussian RV with μ=0 and σ=1
scipy.stats.norm.pdf(2.5, 0, 1)
```

    0.01752830049356854

# Matplotlib
Plotting library. Great gallery and example code

http://matplotlib.org

# Pyraf
- Interface between Python and IRAF
- Use Ureka (not Anaconda installation): http://ssb.stsci.edu/ureka/
- Great talk on development of Pyraf by Perry Greenfield: https://www.youtube.com/watch?v=uz53IV1V_Xo

```python
import pyraf
from pyraf import iraf

iraf.imhead('spectrum.fits[0]', longheader = 'yes')
```

# statsmodels

Statistical modeling package

http://statsmodels.sourceforge.net/

```python
import numpy as np
import statsmodels.api as sm

# Generate artificial data (2 regressors + constant)
nobs = 100
X = np.random.random((nobs, 2))
X = sm.add_constant(X)
beta = [1, .1, .5]
e = np.random.random(nobs)
y = np.dot(X, beta) + e

# Fit regression model
results = sm.OLS(y, X).fit()

# Inspect the results
results.summary()
```

OLS Regression Results

  Dep. Variable:	            y	          R-squared:         	    0.161

  Model:	                   OLS	         Adj. R-squared:    	    0.144

  Method:	             Least Squares	    F-statistic:       	    9.303

  Date:	             Fri, 03 Jul 2015	   Prob (F-statistic):	 0.000201

  Time:	                 12:53:13	       Log-Likelihood:    	  -24.917

  No. Observations:	         100	        AIC:               	    55.83

  Df Residuals:	              97	        BIC:               	    63.65

  Df Model:	                   2

  Covariance Type:	      nonrobust

    	       coef	     std err	      t	      P>|t|	 [95.0% Conf. Int.]

  const	     1.5522	     0.081	    19.220	  0.000	     1.392     1.713

  x1	        0.0545	     0.107	     0.509	  0.612	    -0.158     0.267

  x2	        0.4451	     0.104	     4.278	  0.000	     0.239     0.652

  Omnibus:	       52.704	   Durbin-Watson:     	    1.846

  Prob(Omnibus):	  0.000	   Jarque-Bera (JB):  	    7.045

  Skew:	           0.074	   Prob(JB):          	   0.0295

  Kurtosis:	       1.708	   Cond. No.          	     5.11

## scikit-learn

Machine learning in Python

http://scikit-learn.org/stable/

```python
from sklearn import datasets
from sklearn import svm

# Load digits dataset
digits = datasets.load_digits()

# Create a support vector machine classifier and fit it to the digits data
clf = svm.SVC(gamma=0.001, C=100.)
clf.fit(digits.data[:-1], digits.target[:-1])

# Classify one of the digits
print("True digit: " + str(digits.target[-1]))
print("Prediction: " + str(int(clf.predict(digits.data[-1]))))
```

    True digit: 8
    Prediction: 8

# sympy

Symbolic math

http://www.sympy.org/en/index.html

```python
import sympy

# Enable pretty printing
sympy.init_printing()

# Define a variable, x
x = sympy.symbols('x')

# Define an integral over x
a = sympy.Integral(sympy.cos(x) * sympy.exp(x), x)

# Compute the value of the integral
sympy.Eq(a, a.doit())
```

# scikit-image

Another image processing library, focusing more on computer vision.

http://scikit-image.org/

```python
from skimage import data, io, filters

# Load the data (can be any NumPy array)
image = data.coins()

# Find the edges
edges = filters.sobel(image)

# Display the result
io.imshow(edges)
```

# numba

Just-In-Time compiler to improve speed and performance of your python programs.

http://numba.pydata.org/

```python
from numba import jit
from numpy import arange

# jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when function is called.
@jit
def sum2d(arr):
    M, N = arr.shape
    result = 0.0
    for i in range(M):
        for j in range(N):
            result += arr[i,j]
    return result

a = arange(9).reshape(3,3)
print(sum2d(a))
```

# cython

Compile Python programs to C

http://cython.org/

```python
%load_ext Cython
```

```python
def pyfib(n):
    a, b = 0, 1
    for i in range(n):
        a, b = b, a + b
    return b
```

```cython
%%cython -a

def cfib(int n):
    cdef int a = 0
    cdef int b = 1
    for i in range(n):
        a, b = b, a + b
    return b
```

<!DOCTYPE html>

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Generated by Cython 0.22

 1:

+2: def cfib(int n):

/* Python wrapper */
static PyObject *__pyx_pw_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_1cfib(PyObject *__pyx_self, PyObject *__pyx_arg_n); /*proto*/
static PyMethodDef __pyx_mdef_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_1cfib = {"cfib", (PyCFunction)__pyx_pw_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_1cfib, METH_O, 0};
static PyObject *__pyx_pw_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_1cfib(PyObject *__pyx_self, PyObject *__pyx_arg_n) {
  int __pyx_v_n;
  PyObject *__pyx_r = 0;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("cfib (wrapper)", 0);
  assert(__pyx_arg_n); {
    __pyx_v_n = __Pyx_PyInt_As_int(__pyx_arg_n); if (unlikely((__pyx_v_n == (int)-1) && PyErr_Occurred())) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L3_error;}
  }
  goto __pyx_L4_argument_unpacking_done;
  __pyx_L3_error:;
  __Pyx_AddTraceback("_cython_magic_8032ac3b541f08ba99f383847d102ad6.cfib", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __Pyx_RefNannyFinishContext();
  return NULL;
  __pyx_L4_argument_unpacking_done:;
  __pyx_r = __pyx_pf_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_cfib(__pyx_self, ((int)__pyx_v_n));
  int __pyx_lineno = 0;
  const char *__pyx_filename = NULL;
  int __pyx_clineno = 0;

  /* function exit code */
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}

static PyObject *__pyx_pf_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_cfib(CYTHON_UNUSED PyObject *__pyx_self, int __pyx_v_n) {
  int __pyx_v_a;
  int __pyx_v_b;
  CYTHON_UNUSED int __pyx_v_i;
  PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
  __Pyx_RefNannySetupContext("cfib", 0);
/* … */
  /* function exit code */
  __pyx_L1_error:;
  __Pyx_XDECREF(__pyx_t_5);
  __Pyx_AddTraceback("_cython_magic_8032ac3b541f08ba99f383847d102ad6.cfib", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r = NULL;
  __pyx_L0:;
  __Pyx_XGIVEREF(__pyx_r);
  __Pyx_RefNannyFinishContext();
  return __pyx_r;
}
/* … */
  __pyx_tuple_ = PyTuple_Pack(5, __pyx_n_s_n, __pyx_n_s_n, __pyx_n_s_a, __pyx_n_s_b, __pyx_n_s_i); if (unlikely(!__pyx_tuple_)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_tuple_);
  __Pyx_GIVEREF(__pyx_tuple_);
/* … */
  __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_46_cython_magic_8032ac3b541f08ba99f383847d102ad6_1cfib, NULL, __pyx_n_s_cython_magic_8032ac3b541f08ba99); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_1);
  if (PyDict_SetItem(__pyx_d, __pyx_n_s_cfib, __pyx_t_1) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;

+3:     cdef int a = 0

  __pyx_v_a = 0;

+4:     cdef int b = 1

  __pyx_v_b = 1;

+5:     for i in range(n):

  __pyx_t_1 = __pyx_v_n;
  for (__pyx_t_2 = 0; __pyx_t_2 < __pyx_t_1; __pyx_t_2+=1) {
    __pyx_v_i = __pyx_t_2;

+6:         a, b = b, a + b

    __pyx_t_3 = __pyx_v_b;
    __pyx_t_4 = (__pyx_v_a + __pyx_v_b);
    __pyx_v_a = __pyx_t_3;
    __pyx_v_b = __pyx_t_4;
  }

+7:     return b

  __Pyx_XDECREF(__pyx_r);
  __pyx_t_5 = __Pyx_PyInt_From_int(__pyx_v_b); if (unlikely(!__pyx_t_5)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 7; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_5);
  __pyx_r = __pyx_t_5;
  __pyx_t_5 = 0;
  goto __pyx_L0;

```python
%timeit cfib(100)
```

    The slowest run took 49.17 times longer than the fastest. This could mean that an intermediate result is being cached
    10000000 loops, best of 3: 101 ns per loop

```python
%timeit pyfib(100)
```

    100000 loops, best of 3: 7.98 µs per loop

# astropy

http://www.astropy.org/

Variety of astronomy tools for observers and theorists

#References
* Effective Computation in Physics http://physics.codes

# File Formats

* **CSV**
  * [built in](https://docs.python.org/3/library/csv.html)
* **HDF5**
  * http://www.pytables.org/
  * http://www.h5py.org/
* **NetCDF**
  * http://unidata.github.io/netcdf4-python/
* **JSON**
  * [built in](https://docs.python.org/3/library/json.html)
* **XML**
  * http://lxml.de/
* **GIS** (e.g. GeoJSON)
  * http://toblerity.org/fiona/

# Installing Packages

Most of these packages are included in Anaconda

For those that aren't...

## Conda

http://conda.pydata.org/

Included with Anaconda

Doesn't have everything, but installs binary packages

    conda search package_name
    conda install package_name
    conda uninstall package_name

## Pip

http://pip-installer.org

Standard Python installer

Included with Anaconda

    pip search package_name
    pip install package_name
    pip uninstall package_name

```python

```


## Summary
Part of a series that takes the formulas and MATLAB code found in Ernie Chan's book, Algorithmic Trading: Winning Strategies and Their Rationale (2013)

## Version Control

```python
%run ../../code/version_check.py
```

    Python: 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:53:06)
    [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]

    matplotlib:	2.0.0
    numpy:		1.11.3
    pandas:		0.19.2
    quandl:		3.0.1
    sklearn:	0.18.1
    scipy:		0.18.1

## Change Log
    Date Created: 2016-10-26

    Date of Change    Change Notes
    --------------    ----------------------------------------------------------------
    2016-10-26        Initial draft
    2016-10-27        Fixed formatting
    2017-01-20        Layout changes
    2017-04-31        * Added versioning of modules
                      * Added alignment to table holding critical values for test

## 1. General References
  1.1 [Mathjax Basic Tutorial and Quick Reference](http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference)
  1.2 [Github Markdown Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#blockquotes)

## 2. Notebook References
 2.1 [CAGR](http://www.investopedia.com/terms/c/cagr.asp)
 2.2 [Pandas Time Series Offset Aliases](http://pandas.pydata.org/pandas-docs/stable/timeseries.html#offset-aliases)

## 3. Backtesting and its relationship to Hypothesis Testing
---

The book suggests a general form of backtesting follows the hypothesis testing framework as follows:
  1. Compute the test statistic (average daily return for the active period)
  2. Assume the average daily return = 0, and call this the **null hypothesis**
  3. Assume Gaussian (Normal) Distribution of daily returns with 0 (zero) mean
  4. Using the null hypothesis assumed distribution of daily returns, compute the *p*-value.
  5. Lookup *p*-value against critical values to determine significance

If *p*-value is less than 0.01 (less than 1% chance), we reject the null hypothesis, meaning the average daily returns from the backtest are statistically significant.

### 3.1 Sharpe Ratio Test Statistic (srt-stat)

The book cites a reference, (Bernston, 2002) with the citation stated as:

> ... the standard test statistic for a Gaussian distribution is none other than the average divided by the standard deviation and multiplied by the square root of the number of data points ..."

which when written in equation form looks like:

$$\text{srt-stat} = \left(\frac{\mu}{\sigma}\right) \sqrt{n}$$

where:

\begin{align}
\mu  & = \text{the }\textbf{mean of the daily returns} \\
\sigma & = \text{the }\textbf{standard deviation of the daily returns} \\
n & = \text{the number of data points, also known as the }\textbf{number of samples}
\end{align}

The `srt_stat` is then then compared to a table of critical values to determine statistical significance, with the table again being cited from Bernston (2002) in Chan's book (code below):

| p-value   | Critical values |
| ---------:| ---------------:|
| 0.1       | 1.282           |
| 0.05      | 1.645           |
| 0.01      | 2.326           |
| 0.001     | 3.091           |

*Note:*

* The smaller the *p*-value (or the larger the `srt_stat`), the better, with respect to being statistically significant.
* Because the Gaussian distriution is "described" by parameters (mu - $\mu$, and sigma - $\sigma$), the test is classified as a **parametric test**

## 4. Coding the Sharpe Ratio Test Statistic
---

First we setup the Jupyter Notebook to display charts nicely, followed up by sourcing 11 years of adjusted daily close price data from Yahoo! Finance, and then have a quick look at how to find a single days worth of data.

Adjusted closing prices are used as the data has been transformed to adapt for splits, reverse-splits, and dividends. This is important when holding stocks, as these will impact the returns of the strategy employed.

### 4.1 Setup

```python
# Settings for plots
%matplotlib inline

import seaborn as sns

sns.set_style('darkgrid')
```

```python
# Get 11 years worth of daily adjusted prices from Yahoo Finance for Google stock price
#
# Remote Data Access: Yahoo! Finance
#     https://pandas-datareader.readthedocs.io/en/latest/remote_data.html#remote-data-yahoo
import pandas as pd
import pandas_datareader.data as web
from datetime import datetime

start = datetime(2005, 1, 1)
end = datetime(2015, 12, 31)

# Only keep the adjusted closing prices
df = web.DataReader("GOOGL", 'yahoo', start=start, end=end)['Adj Close']
df.describe()
```

    count    2769.000000
    mean      324.799144
    std       152.879584
    min        87.582582
    25%       222.667666
    50%       281.346348
    75%       405.560552
    max       793.960022
    Name: Adj Close, dtype: float64

```python
# How to index into a particular date's adjusted closing price
df.ix['2010-01-07']
```

    297.34735499999999

### 4.2 Compute Returns
We will now create a series of daily returns in percent, and view the top 5 records for the series.

```python
# Create daily returns
#
# Calculate Daily Returns with Pandas DataFrame
#     http://stackoverflow.com/a/33472722/893766
r = df.pct_change(1).dropna()
r.head(5)
```

    Date
    2005-01-04   -0.040501
    2005-01-05   -0.005090
    2005-01-06   -0.025632
    2005-01-07    0.028109
    2005-01-10    0.006242
    Name: Adj Close, dtype: float64

### 4.3 Compute Test Statistic

Here we compute the test statistic and review the outcome. The test is performed by the following line in the code below:

    srt_stat = (mu / sigma) * np.sqrt(n)

```python
# Compute srt-stat
# -> assumes a buy and hold strategy from the first date in the time series
#
# Pandas DataFrame - Find row where values for column is maximal
#     http://stackoverflow.com/a/10213167/893766
import numpy as np

# ensure all numpy exceptions are raised
np.seterr(all='raise')

# critical values to test against for p-value
test_values = pd.DataFrame({
        'crits': [1.282, 1.645, 2.326, 3.091, ],
        'p_values': [0.1, 0.05, 0.01, 0.001, ]})

# test parameters
# r = percent daily returns
mu = r.mean()
sigma = r.std()
n = len(r)

# run the test
srt_stat = (mu / sigma) * np.sqrt(n)

# compare to crits
i = test_values['crits'] <= srt_stat
test_values = test_values[i]

# print if there is a significant result
if (len(test_values) > 0):
    result = test_values.ix[test_values['crits'].idxmax()]
    print("srt-stat\t{:.5}".format(srt_stat))
    print("crit\t\t{:.5}".format(result[0]))
    print("p-value\t\t{:.5}".format(result[1]))
    print()
    print("Significance:\tTest statistic is significant at the {}% level".format(int(result[1] * 100)))
    print("Outcome:\tReject the Null Hypothesis")
else:
    print("Significance:\tTest statistic did not pass any critical values")
    print("Outcome:\tFailed to reject the Null Hypothesis")
```

    srt-stat	2.4804
    crit		2.326
    p-value		0.01

    Significance:	Test statistic is significant at the 1% level
    Outcome:	Reject the Null Hypothesis

## 5. Outcome
---

According to the test the Sharpe Ratio of the average daily returns is statistically significant.

Now what does that look like with $10,000 invested?

```python
# Plot the returns of $10,000
# -> ignores transaction costs
start_value = 10000
actual_returns = (r.cumsum() + 1) * start_value
end_value = actual_returns[-1]

fig = actual_returns.plot(figsize=(10,4), title="Portfolio Vaue ($)")

print("start_value:\t{}\nend_value:\t{}".format(start_value, end_value))
```

## 6. CAGR

Investopedia provide the following for CAGR:

$$\text{CAGR} = \left(\frac{\text{End value}}{\text{Start value}}\right)^\left[\frac{1}{\text{n years}}\right]$$

Where typically the *n years* is adjusted for the total number of days in the investment period, like so:

$$\text{n years} = \frac{\text{Total number of days}}{365}$$

In our case this would give us:

\begin{align}
\text{n years} & = \frac{2769}{365} \\
& = 7.5863 \\
& \approx 7\frac{1}{2} years
\end{align}

However, in our example, because we're using daily data that doesn't trade all year round, we can't just take the number of days and divide ($\div$) by 365 to get an approximation for the number of years.

If we instead use the data to drive our decision we can count the number of days the stock traded each year, and make a decision to adjust the *n years* formula using this information like so:

$$\text{n years} = \frac{\text{Total number of days}}{\text{Average trading days per annum}}$$

### 6.1 Finding the average number of trading days per annum
The code below shows how to find the number of trading days, for each year (assumes every day that we have an adjusted close price, the market was open and the stock was able to be traded):

```python
# Count number of rows in each year. Implicitly we are counting number of days,
# as one row exists for each day.
years = df.groupby(pd.TimeGrouper(freq="AS")).count()
years
```

    Date
    2005-01-01    252
    2006-01-01    251
    2007-01-01    251
    2008-01-01    253
    2009-01-01    252
    2010-01-01    252
    2011-01-01    252
    2012-01-01    250
    2013-01-01    252
    2014-01-01    252
    2015-01-01    252
    Freq: AS-JAN, Name: Adj Close, dtype: int64

As we can see from above, the number of days ranged from 250-253, however if we just take the median of these stats we can use this for our calculation.

### 6.2 Average days traded per year
The code for the median is very simple:

```python
avg_annual_days_traded = years.median()
avg_annual_days_traded
```

    252.0

We can use this to adjust the formula used previously to:

\begin{align}
\text{n years} & = \frac{2769}{252} \\
& = 10.98809 \\
& \approx 11 \text{years}
\end{align}

As can now be seen, there is a vast difference between 7$\frac{1}{2}$ years and 11 years. This increases the amount of time we held our stocks by $\approx$ 46%!

### 6.3 CAGR with correct `avg_annual_days_traded`
We now have enough information to complete the formula, the code:

```python
days_traded = len(r)
n_years = days_traded / avg_annual_days_traded

cagr = (end_value / start_value) ** (1 / n_years)
print("Compound annual growth rate (CAGR) = {:5.4}%".format((cagr - 1) * 100))
```

    Compound annual growth rate (CAGR) = 12.28%

### 6.4 Incorrect `avg_annual_days_traded` and inflated CAGR
And the CAGR with the incorrect `n_years` of $\approx$ 7.5 (instead of the correct $\approx$ 11):

```python
# Note: the n_years calculation is different
days_traded = len(r)
n_years = days_traded / 365

cagr = (end_value / start_value) ** (1 / n_years)
print("Compound annual growth rate (CAGR) = {:5.4}%".format((cagr - 1) * 100))
```

    Compound annual growth rate (CAGR) = 18.26%

The incorrect CAGR of 18.26% is quite impressive for long-term buy-and-hold, unfortunately it is incorrect.

This stresses the importance of correctly measuring the performance of any backtests, which is reliant on understanding the data you're working with.