# Numerical optimization

You will learn to solve non-convex multi-dimensional optimization problems using numerical optimization with multistart and nesting (**scipy.optimize**). You will learn simple function approximation using linear interpolation (**scipy.interp**).

**Links:**

1. **scipy.optimize:** [overview](https://docs.scipy.org/doc/scipy/reference/optimize.html) + [tutorial](https://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html)
2. **scipy.interp:** [overview](https://docs.scipy.org/doc/scipy/reference/interpolate.html) + [tutorial](https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html)

**Useful note:** [Numerical Optimization in MATLAB](http://www.google.com/url?q=http%3A%2F%2Fweb.econ.ku.dk%2Fmunk-nielsen%2Fnotes%2FnoteOptimization.pdf&sa=D&sntz=1&usg=AFQjCNHX4tHx2_YsNaIt5FB5MBU5cfcS8g) (by Anders Munk-Nielsen)

```python
from types import SimpleNamespace
import numpy as np
import scipy as sp
from scipy import linalg
from scipy import optimize
from scipy import interpolate
import sympy as sm

%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D
```

# Introduction

All **optimization problems** are characterized by:

1. Control vector (choices), $\boldsymbol{x} \in \mathbb{R}^k$
2. Objective function (payoff) to minimize, $f:\mathbb{R}^k \rightarrow \mathbb{R}$ (differentiable or not)
3. Constraints, i.e. $\boldsymbol{x}  \in C \subseteq \mathbb{R}^k$ (linear or non-linear interdependence)

Note that $f$ might also take other inputs (parameters or a dataset), but these are fixed, and therefore not variables we optimize over.

**Maximization** is just **minimization** of $-f$.

All **optimizers** (minimizers) follow the structure:

1. Make initial guess
2. Evaluate the function (and perhaps gradients)
3. Check for convergence
4. Update guess and return to step 2

**Convergence:** "Small" change in function value since last iteration (or "zero" gradient).

**Characteristics** of optimizers:

1. Use gradients or not.
2. Allow for specifying bounds.
3. Allow for specifying general constraints.

**Gradients** provide useful information, but can be costly to compute (using analytical formula or numerically).

**Penalty terms** can (sometimes) instead be used to enforce bounds and constraints.

**Optimizers** you should know:

1. **Nelder-Mead:**
 * **Pro:** Robust (to e.g. noise in objective function) and does not require derivatives.
 * **Con:** Slow convergence. No bounds or constraints.
2. **Newton-CG:**
 * **Pro:** Require few iterations. Very precise with analytical hessian for smooth functions.
 * **Con:** Costly computation of hessian. No bounds or constraints.
3. **BFGS:** (like newton, but with smart computation of hessian)
  * **Pro:** Require few function evaluations.
  * **Con:** No bounds or constraints.
4. **L-BFGS-B:** Like BFGS, but allows for bounds.
5. **SLSQP:**
  * **Pro:** Bounds and constraints in multiple dimensions.
  * **Con:** Not as efficient as BFGS.

## Gradient based optimizers

Let us look at the idea behind gradient based optimizers.

**One dimensional intuition:** Consider the second-order Taylor approximation around $x_n$:

$$
f_T(x) = f_T(x_n + \Delta x) \approx f(x_n)+ f^{\prime}(x_n) \Delta x + \frac{1}{2} f^{\prime\prime}(x_n) (\Delta x)^2
$$

Find the minimum wrt. to $\Delta x$ by solving the FOC:

$$
0 = \frac{d}{d\Delta x} f_T(x) = f^{\prime}(x_n) + f^{\prime\prime}(x_n) \Delta x \Leftrightarrow \Delta x = -\frac{f^{\prime}(x_n)}{f^{\prime\prime}(x_n)}
$$

**Algorithm:** `minimize_newton()`

1. Choose tolerance $\epsilon>0$, guess on $\boldsymbol{x}_0$, compute $f(\boldsymbol{x}_0)$, and set $n=1$.
2. Compute $\nabla f(\boldsymbol{x}_{n-1})$ (gradient/jacobian) and $\boldsymbol{H}f(\boldsymbol{x}_{n-1})$ (hessian).
3. Compute new guess

  $$
  \boldsymbol{x}_{n} = \boldsymbol{x}_{n-1} - [\boldsymbol{H}f(\boldsymbol{x}_{n-1})]^{-1} \nabla f(\boldsymbol{x}_{n-1})
  $$

3. If $|f(\boldsymbol{x}_n)-f(\boldsymbol{x}_{n-1})| < \epsilon$ then stop.
5. Set $n = n + 1$ and return to step 2.

```python
def minimize_newton(f,x0,jac,hess,max_iter=500,tol=1e-8):
    """ minimize function with Newtons' algorithm

    Args:

        f (callable): function
        x0 (np.ndarray): initial values
        jac (callable): jacobian
        hess (callable): hessian
        max_iter (int): maximum number of iterations
        tol (float): tolerance

    Returns:

        x (np.ndarray): minimum
        n (int): number of iterations used

    """

    # step 1: initialize
    x = x0
    fx = f(x0)
    n = 1

    # step 2-5: iteration
    while n < max_iter:

        x_prev = x
        fx_prev = fx

        # step 2: evaluate gradient and hessian
        jacx = jac(x_prev)
        hessx = hess(x_prev)

        # step 3: update x
        inv_hessx = linalg.inv(hessx)
        x = x_prev - inv_hessx@jacx

        # step 4: check convergence
        fx = f(x)
        if abs(fx-fx_prev) < tol:
            break

        # step 5: increment n
        n += 1

    return x,n
```

**Algorithm:** `minimize_gradient_descent()`

1. Choose tolerance $\epsilon>0$, potential step sizes, $ \boldsymbol{\alpha} = [\alpha_0,\alpha_1,\dots,\alpha_\#]$, guess on $\boldsymbol{x}_0$, compute $f(\boldsymbol{x}_0)$, and set $n=1$.
2. Compute $\nabla f(\boldsymbol{x}_{n-1})$.
3. Find good step size:

  $$
  \alpha^{\ast} = \arg \min_{\alpha \in \boldsymbol{\alpha}}  f(\boldsymbol{x}_{n-1} - \alpha \nabla f(\boldsymbol{x}_{n-1}))
  $$

4. Compute new guess:

  $$
  \boldsymbol{x}_{n} = \boldsymbol{x}_{n-1} - \alpha^{\ast} \nabla f(\boldsymbol{x}_{n-1})
  $$

5. If $|f(\boldsymbol{x}_n)-f(\boldsymbol{x}_{n-1})| < \epsilon$ then stop.
6. Set $n = n + 1$ and return to step 2.

```python
def minimize_gradient_descent(f,x0,jac,alphas=[0.01,0.05,0.1,0.25,0.5,1],max_iter=500,tol=1e-8):
    """ minimize function with gradient descent

    Args:

        f (callable): function
        x0 (np.ndarray): initial values
        jac (callable): jacobian
        alpha (list): potential step sizes
        max_iter (int): maximum number of iterations
        tol (float): tolerance

    Returns:

        x (np.ndarray): minimum
        n (int): number of iterations used

    """

    # step 1: initialize
    x = x0
    fx = f(x0)
    n = 1

    # step 2-6: iteration
    while n < max_iter:

        x_prev = x
        fx_prev = fx

        # step 2: evaluate gradient
        jacx = jac(x)

        # step 3: find good step size (line search)
        fx_ast = np.inf
        alpha_ast = np.nan
        for alpha in alphas:
            x = x_prev - alpha*jacx
            fx = f(x)
            if fx < fx_ast:
                fx_ast = fx
                alpha_ast = alpha

        # step 4: update guess
        x = x_prev - alpha_ast*jacx

        # step 5: check convergence
        fx = f(x)
        if abs(fx-fx_prev) < tol:
            break

        # d. update i
        n += 1

    return x,n
```

**Many generalizations:**

1. Use both Hessian and line search
2. Stop line search when improvement found
3. Limit attention to a "trust-region"

etc. etc. etc. etc.

# Example: The rosenbrock function

Consider the **rosenbrock function**:

$$
f(\boldsymbol{x}) = f(x_1,x_2) =0.5(1-x_{1})^{2}+(x_{2}-x_{1}^{2})^{2}
$$

with **jacobian** (gradient)

$$
\nabla f(\boldsymbol{x})=\begin{bmatrix}\frac{\partial f}{\partial x_{1}}\\
\frac{\partial f}{\partial x_{2}}
\end{bmatrix}=\begin{bmatrix}-(1-x_{1})-4x_{1}(x_{2}-x_{1}^{2})\\
2(x_{2}-x_{1}^{2})
\end{bmatrix}
$$

and **hessian**:

$$
\boldsymbol{H}f(\boldsymbol{x})=\begin{bmatrix}\frac{\partial f}{\partial x_{1}x_{1}} & \frac{\partial f}{\partial x_{1}x_{2}}\\
\frac{\partial f}{\partial x_{1}x_{2}} & \frac{\partial f}{\partial x_{2}x_{2}}
\end{bmatrix}=\begin{bmatrix}1-4x_{2}+12x_{1}^{2} & -4x_{1}\\
-4x_{1} & 2
\end{bmatrix}
$$

**Note:** Minimum is at $(1,1)$ where $f(1,1)=0$.

**Check jacobian and hessian:**

```python
x1 = sm.symbols('x_1')
x2 = sm.symbols('x_2')
f = 0.5*(1.0-x1)**2 + (x2-x1**2)**2
```

```python
Df = sm.Matrix([sm.diff(f,i) for i in [x1,x2]])
Df
```

$\displaystyle \left[\begin{matrix}- 4 x_{1} \left(- x_{1}^{2} + x_{2}\right) + 1.0 x_{1} - 1.0\\- 2 x_{1}^{2} + 2 x_{2}\end{matrix}\right]$

```python
Hf = sm.Matrix([[sm.diff(f,i,j) for j in [x1,x2]] for i in [x1,x2]])
Hf
```

$\displaystyle \left[\begin{matrix}12 x_{1}^{2} - 4 x_{2} + 1.0 & - 4 x_{1}\\- 4 x_{1} & 2\end{matrix}\right]$

**Implementation:**

```python
def _rosen(x1,x2):
    return 0.5*(1.0-x1)**2+(x2-x1**2)**2
def rosen(x):
    return _rosen(x[0],x[1])
def rosen_jac(x):
    return np.array([-(1.0-x[0])-4*x[0]*(x[1]-x[0]**2),2*(x[1]-x[0]**2)])
def rosen_hess(x):
    return np.array([[1-4*x[1]+12*x[0]**2,-4*x[0]],[-4*x[0],2]])
```

**3D Plot:**

```python
# a. grids
x1_vec = np.linspace(-2,2,500)
x2_vec = np.linspace(-2,2,500)
x1_grid,x2_grid = np.meshgrid(x1_vec,x2_vec,indexing='ij')
rosen_grid = _rosen(x1_grid,x2_grid)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid,x2_grid,rosen_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$u$')

# d. invert xaxis
ax.invert_xaxis()

# e. add colorbar
fig.colorbar(cs);
```

**Contour plot:**

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
levels = [1e-6,5*1e-6,1e-5,5*1e-5,1e-4,5*1e-4,1e-3,5*1e-3,1e-2,5*1e-2,1,2,4,6,8,12,16,20]
cs = ax.contour(x1_grid,x2_grid,rosen_grid,levels=levels,cmap=cm.jet)
fig.colorbar(cs);
```

**Newton:**

```python
x0 = np.array([5,4])
x,n = minimize_newton(rosen,x0,rosen_jac,rosen_hess)
print(n,x,rosen(x))
```

    6 [1. 1.] 0.0

**Gradient descent:**

```python
x0 = np.array([5,4])
x,n = minimize_gradient_descent(rosen,x0,rosen_jac,alphas=[0.01,0.05,0.1,0.25,0.5,1])
print(n,x,rosen(x))
```

    173 [1.00020519 1.00053964] 3.7750814497569406e-08

**Questions:** Any ideas for getting the gradient descent optimizer to converge faster?

## Scipy minimizers

**Preperation I:** Function for collecting infomation while running optimizing:

```python
# complicated -> not necessary to understand it
def collect(x):

    # globals used to keep track across iterations
    global evals # set evals = 0 before calling optimizer
    global x0
    global x1s
    global x2s
    global fs

    # a. initialize list
    if evals == 0:
        x1s = [x0[0]]
        x2s = [x0[1]]
        fs = [rosen(x0)]

    # b. append trial values
    x1s.append(x[0])
    x2s.append(x[1])
    fs.append(rosen(x))

    # c. increment number of evaluations
    evals += 1

```

**Preperation II:** Function plotting the collected information:

```python
# complicated -> not necessary to understand it
def contour():

    global evals
    global x1s
    global x2s
    global fs

    # a. contour plot
    fig = plt.figure(figsize=(10,4))
    ax = fig.add_subplot(1,2,1)
    levels = [1e-6,5*1e-6,1e-5,5*1e-5,1e-4,5*1e-4,1e-3,5*1e-3,1e-2,5*1e-2,1,2,4,6,8,12,16,20]
    cs = ax.contour(x1_grid,x2_grid,rosen_grid,levels=levels,cmap=cm.jet)
    fig.colorbar(cs)
    ax.plot(x1s,x2s,'-o',ms=4,color='black')
    ax.set_xlabel('$x_1$')
    ax.set_ylabel('$x_2$')

    # b. function value
    ax = fig.add_subplot(1,2,2)
    ax.plot(np.arange(evals+1),fs,'-o',ms=4,color='black')
    ax.set_xlabel('iteration')
    ax.set_ylabel('function value')

```

**Nelder-Mead**

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='Nelder-Mead',
                           callback=collect, # call collect() before each iteration
                           options={'disp':True}) # display the results
contour()
```

> **Note:** Does not require a gradient. Slow convergence close to target.
>
> **Iterations:** How many steps the algorithm has taken.
>
> **Function evaluations:** Will be higher than iterations. Used to compute next step.

We can also **print the information on results:**

```python
print(result)
```

     final_simplex: (array([[0.9999932 , 0.99998507],
           [1.00000809, 1.00003102],
           [1.00003176, 1.00007406]]), array([2.48761443e-11, 2.53004854e-10, 6.15284951e-10]))
               fun: 2.4876144250933352e-11
           message: 'Optimization terminated successfully.'
              nfev: 105
               nit: 57
            status: 0
           success: True
                 x: array([0.9999932 , 0.99998507])

We can also acess specific information of the result object:

```python
result.nit
```

    57

**Newton** (with analytical hessian)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,hess=rosen_hess,
                           method='Newton-CG',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Smoother and faster.

**Newton** (with numerical hessian computed by scipy)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='Newton-CG',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Same as above, but gradient evaluations instead of hessian evaluations.

**BFGS** (with analytical gradient)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Non-smooth, but fast. Very low number of function evaluations.

**BFGS** (with numerical gradient computed by scipy)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0, # no jac= specified
                           method='BFGS',
                           callback=collect,
                           options={'disp':True})
contour()
```

> **Note:** Same as above, but more function evaluations.

**L-BFGS-B** (with analytical gradient)

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='L-BFGS-B',
                           bounds=((-3,3),(-3,3)),
                           callback=collect,
                           options={'disp':True})
contour()
```

**SLSQP**

```python
evals = 0 # global used in "collect"
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,jac=rosen_jac,
                           method='SLSQP',
                           bounds=((-2,2),(-2,2)),
                           callback=collect,
                           options={'disp':True})
contour()
```

## Controling the optimizers

> **Note:** See the settings for each optimizer in the [documention](https://docs.scipy.org/doc/scipy/reference/optimize.html).

We can lower the **tolerance**:

```python
evals = 0
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True,'gtol':1e-8}) # note this
contour()
```

We can change the **maximum number of iterations**:

```python
evals = 0
x0 = [-1.5,-1]
result = optimize.minimize(rosen,x0,
                           method='BFGS',
                           callback=collect,
                           options={'disp':True,'maxiter':5}) # note this and warning
contour()
```

**Question:** Can we make the program stop if the maximum number of iterations is too low?

# Sombrero function: Local minima and multistart

Consider the **sombrero** function

$$
f(x_1,x_2) = g\Big(\sqrt{x_1^2 + x_2^2}\Big)
$$

where

$$
g(r) = -\frac{\sin(r)}{r+10^{-4}} + 10^{-4}r^2
$$

The **global minimum** of this function is (0,0). But the function also have (infinitely many) **local minima**. How to avoid these?

```python
def _sombrero(x1,x2):
    r = np.sqrt(x1**2 + x2**2)
    return -np.sin(r)/(r+1e-4) + 1e-4*r**2

sombrero = lambda x: _sombrero(x[0],x[1])
```

## 3D plot

```python
# a. grids
x1_vec = np.linspace(-15,15,500)
x2_vec = np.linspace(-15,15,500)
x1_grid_sombrero,x2_grid_sombrero = np.meshgrid(x1_vec,x2_vec,indexing='ij')
sombrero_grid = _sombrero(x1_grid_sombrero,x2_grid_sombrero)

# b. main
fig = plt.figure()
ax = fig.add_subplot(1,1,1,projection='3d')
cs = ax.plot_surface(x1_grid_sombrero,x2_grid_sombrero,sombrero_grid,cmap=cm.jet)

# c. add labels
ax.set_xlabel('$x_1$')
ax.set_ylabel('$x_2$')
ax.set_zlabel('$u$')

# d. invert xaxis
ax.invert_xaxis()

# e. colorbar
fig.colorbar(cs);
```

## Multi-start - BFGS

**Multi-start:** Draw many random starting values:

```python
np.random.seed(1986)
x0s = -15 + 30*np.random.uniform(size=(5000,2)) # in [-15,15]
xs = np.empty((5000,2))
fs = np.empty(5000)
```

Try to solve with **BFGS** starting from each of these:

```python
fopt = np.inf
xopt = np.nan
for i,x0 in enumerate(x0s):

    # a. optimize
    result = optimize.minimize(sombrero,x0,method='BFGS')
    xs[i,:] = result.x
    f = result.fun

    # b. print first 10 or if better than seen yet
    if i < 10 or f < fopt: # plot 10 first or if improving
        if f < fopt:
            fopt = f
            xopt = xs[i,:]

        print(f'{i:4d}: x0 = ({x0[0]:6.2f},{x0[1]:6.2f})',end='')
        print(f' -> converged at ({xs[i][0]:6.2f},{xs[i][1]:6.2f}) with f = {f:.14f}')

# best solution
print(f'\nbest solution:\n x = ({xopt[0]:6.2f},{xopt[1]:6.2f}) -> f = {fopt:.14f}')
```

       0: x0 = (  2.07,-12.68) -> converged at (  2.26,-13.84) with f = -0.05118272212318
       1: x0 = (  3.25, 11.92) -> converged at (  3.69, 13.53) with f = -0.05118272211067
       2: x0 = (  1.35,  6.08) -> converged at (  1.67,  7.53) with f = -0.12241421149687
       3: x0 = ( -3.42,  4.55) -> converged at ( -4.63,  6.17) with f = -0.12241421149362
       4: x0 = (  5.70,  6.56) -> converged at (  5.06,  5.82) with f = -0.12241421149696
       5: x0 = (  5.71,  9.20) -> converged at (  4.06,  6.56) with f = -0.12241421146807
       6: x0 = ( -6.49,  7.73) -> converged at ( -4.96,  5.91) with f = -0.12241421149702
       7: x0 = ( -5.77, -1.86) -> converged at ( -7.34, -2.37) with f = -0.12241421135584
       8: x0 = (  1.58, -2.01) -> converged at (  0.04, -0.05) with f = -0.99776236017000
       9: x0 = (  2.33, 11.77) -> converged at (  2.72, 13.76) with f = -0.05118272212317
      25: x0 = (  1.41, -3.37) -> converged at (  0.03, -0.06) with f = -0.99776236017097
     150: x0 = (-14.53,  8.53) -> converged at ( -0.06,  0.03) with f = -0.99776236017097
     223: x0 = (  2.87, -0.21) -> converged at ( -0.07,  0.00) with f = -0.99776236017098
     279: x0 = ( -6.90, -8.37) -> converged at ( -0.04, -0.05) with f = -0.99776236017098
     354: x0 = ( -1.24, -2.85) -> converged at ( -0.03, -0.06) with f = -0.99776236017098
    1588: x0 = (-12.93,-11.42) -> converged at ( -0.05, -0.04) with f = -0.99776236017098
    4700: x0 = (  1.06,  0.65) -> converged at (  0.06,  0.04) with f = -0.99776236017098

    best solution:
     x = (  0.06,  0.04) -> f = -0.99776236017098

The solver, wrongly, **converges to many of the local minima**:

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(xs[:,0],xs[:,1]);
```

## Multi-start - Nelder-Mead

Try to solve with **Nelder-Mead** starting from each of these:

```python
fopt = np.inf
xopt = np.nan
for i,x0 in enumerate(x0s):

    # a. optimize
    result = optimize.minimize(sombrero,x0,method='Nelder-Mead')
    xs[i,:] = result.x
    f = result.fun

    # b. print first 10 or if better than seen yet
    if i < 10 or f < fopt: # plot 10 first or if improving
        if f < fopt:
            fopt = f
            xopt = xs[i,:]
        print(f'{i:4d}: x0 = ({x0[0]:6.2f},{x0[1]:6.2f})',end='')
        print(f' -> converged at ({xs[i][0]:6.2f},{xs[i][1]:6.2f}) with f = {f:.12f}')

# best solution
print(f'\nbest solution:\n x = ({xopt[0]:6.2f},{xopt[1]:6.2f}) -> f = {fopt:.12f}')
```

       0: x0 = (  2.07,-12.68) -> converged at (  2.20,-13.85) with f = -0.051182722123
       1: x0 = (  3.25, 11.92) -> converged at (  3.43, 13.60) with f = -0.051182722123
       2: x0 = (  1.35,  6.08) -> converged at (  1.39,  7.59) with f = -0.122414211497
       3: x0 = ( -3.42,  4.55) -> converged at ( -3.71,  6.76) with f = -0.122414211497
       4: x0 = (  5.70,  6.56) -> converged at (  6.69,  3.84) with f = -0.122414211497
       5: x0 = (  5.71,  9.20) -> converged at (  4.98, 13.11) with f = -0.051182722123
       6: x0 = ( -6.49,  7.73) -> converged at ( -7.34,  2.37) with f = -0.122414211497
       7: x0 = ( -5.77, -1.86) -> converged at ( -7.45, -2.01) with f = -0.122414211497
       8: x0 = (  1.58, -2.01) -> converged at ( -0.02,  0.06) with f = -0.997762360171
       9: x0 = (  2.33, 11.77) -> converged at (  2.36, 13.83) with f = -0.051182722123

    best solution:
     x = ( -0.02,  0.06) -> f = -0.997762360171

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(xs[:,0],xs[:,1]);
```

## Is there a better solution than multi-start?

**In short:** No.

**Potential improvement:** Use information from previous run to determine, where to look next. Fundamental trade-off between:

1. **Exploitation.** Focus on areas where previous evaluations returned low function values.
2. **Exploration.** Focus on completely new areas.

**Heuristic:** If the same optimum is obtained for many starting values, this is a good sign for it being the global optimum.

**Further discussion**: [Benchmarking Global Optimizers](https://fguvenendotcom.files.wordpress.com/2019/09/agk2019-september-nber-submit.pdf) ([code](https://github.com/serdarozkan/TikTak#tiktak))

# Constraints

## In general

Consider the **constrained problem**:

$$
\min_{x_1,x_2,x_3,x_4} x_1x_4(x_1+x_2+x_3) + x_3
$$

subject to

$$
\begin{aligned}
x_1x_2x_3x_4 &\geq 25 \\
x_1^2+x_2^2+x_3^2+x_4^2 &= 40 \\
1 \leq x_1,x_2,x_3,x_4 &\leq 5
\end{aligned}
$$

Define **objective** and **constraints**:

```python
def _objective(x1,x2,x3,x4):
    return x1*x4*(x1+x2+x3)+x3

def objective(x):
    return _objective(x[0],x[1],x[2],x[3])

def ineq_constraint(x):
    return x[0]*x[1]*x[2]*x[3]-25.0 # violated if negative

def eq_constraint(x):
    sum_eq = 40.0
    for i in range(4):
        sum_eq = sum_eq - x[i]**2
    return sum_eq # must equal zero
```

```python
# a. setup
bound = (1.0,5.0)
bounds = (bound, bound, bound, bound)
ineq_con = {'type': 'ineq', 'fun': ineq_constraint}
eq_con = {'type': 'eq', 'fun': eq_constraint}

# b. call optimizer
x0 = (40**(1/8),40**(1/8),40**(1/8),40**(1/8)) # fit the equality constraint
result = optimize.minimize(objective,x0,
                             method='SLSQP',
                             bounds=bounds,
                             constraints=[ineq_con,eq_con],
                             options={'disp':True})

print('\nx = ',result.x)
```

    Optimization terminated successfully    (Exit mode 0)
                Current function value: 17.0140172890441
                Iterations: 9
                Function evaluations: 46
                Gradient evaluations: 9

    x =  [1.         4.74299964 3.82114999 1.37940829]

**Alternative:** Extend the **objective function with a penalty term**, where guesses outside the allowed bounds and constraints are projected into the allowed region, but a (large) penalty term is added to discourage this. Solve this problem with an unconstrained solver.

## Economic application

Consider the following **consumption-saving problem**:

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,c_3} \frac{c_{1}^{1-\rho}}{1-\rho} + \beta \frac{c_{2}^{1-\rho}}{1-\rho} + \beta^2\frac{c_{3}^{1-\rho}}{1-\rho} + \beta^2\nu\frac{(a_{3}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_1 &= (1+r)a_0 + y_1\\
a_1 &= m_1-c_1\\
m_2 &= (1+r)a_1 + y_2\\
a_2 &= m_2-c_2\\
m_3 &= (1+r)a_2 + y_3\\
a_3 &= m_3-c_3\\
c_1,c_2,c_3 &\geq 0\\
a_1,a_2,a_3 &\geq 0\\
\end{aligned}
$$

where

* $m_t$ is cash-on-hand in period $t\in\{1,2,\dots,T\}$
* $c_t$ is consumption $t$
* $a_t$ is end-of-period assets and income in period $t$
* ${y_t}$ is income in period $t$
* $\beta > 0$ is the discount factor
* $r > -1$ is the interest rate
* $\rho > 1$ is the CRRA coefficient
* $\nu > 0 $ is the strength of the bequest motive
* $\kappa > 0$ is the degree of luxuriousness in the bequest motive
* $a_t\geq0$ is a no-borrowing constraint.

**Guide to solve such problem:**

1. Setup parameters
2. Formulate objective function
3. Determine how to handle constraints
4. Call optimizer

**Parameters:**

```python
par = SimpleNamespace()
par.a0 = 0.5
par.beta = 0.94
par.r = 0.04
par.rho = 8
par.kappa = 0.5
par.nu = 0.1
par.y = [1,2,4]
par.T = 3
```

**Objetive function:**

```python
def obj(c,par,full_return=False):

    # objective function with penalty term

    # a. allocate
    a = np.zeros(par.T) # end-of-period assets
    m = np.zeros(par.T) # cash-on-hand
    cb = np.zeros(par.T) # bounded

    # b. bound consumption and penalty
    penalty = 0.0
    for t in range(par.T):

        # i. lagged assets
        a_lag = a[t-1] if t > 0 else par.a0

        # ii. cash-on-hand
        m[t] = (1+par.r)*a_lag + par.y[t]

        # ii. bounded consumption
        if c[t] < 0:
            penalty += 10_000*np.abs(c[t]-0.0)
            cb[t] = 0
        elif c[t] > m[t]:
            penalty += 10_000*np.abs(c[t]-m[t])
            cb[t] = m[t]
        else:
            cb[t] = c[t]

        # d. end-of-period assets
        a[t] = m[t] - cb[t]

    # c. utility
    utility = 0.0

    # i. consumption
    for t in range(par.T):
        utility += par.beta**t*(cb[t]**(1-par.rho))/(1-par.rho)

    # ii. bequest
    utility += par.beta**(par.T-1)*par.nu*(a[-1]+par.kappa)**(1-par.rho)/(1-par.rho)

    # d. return negative utility + penalty
    if full_return:
        return utility,m,a
    else:
        return -utility + penalty
```

**Solve:**

```python
def solve(par):

    # a. initial geuss
    x0 = [par.a0/par.T,par.a0/par.T,par.a0/par.T]

    # b. solve
    results = optimize.minimize(obj,x0,args=(par,),method='nelder-mead')
    assert results.success
    print(f'solved in {results.nit} iteratoons [{results.nfev} function evaluations]')

    # c. details
    c = results.x
    utility,m,a = obj(c,par,full_return=True)
    print(f't = 0: a = {par.a0:.4f}')
    for t in range(par.T):
        print(f't = {t+1}: y = {par.y[t]:.4f}, m = {m[t]:.4f}, c = {c[t]:.4f}, a = {a[t]:.4f}')
    print(f'utility = {utility:.8f}')
```

```python
solve(par)
```

    solved in 157 iteratoons [287 function evaluations]
    t = 0: a = 0.5000
    t = 1: y = 1.0000, m = 1.5200, c = 1.5200, a = 0.0000
    t = 2: y = 2.0000, m = 2.0000, c = 2.0000, a = 0.0000
    t = 3: y = 4.0000, m = 4.0000, c = 2.5675, a = 1.4325
    utility = -0.00896688

**What happens if the income path is reversed?**

```python
par.y = list(reversed(par.y))
solve(par)
```

    solved in 138 iteratoons [242 function evaluations]
    t = 0: a = 0.5000
    t = 1: y = 4.0000, m = 4.5200, c = 2.1948, a = 2.3252
    t = 2: y = 2.0000, m = 4.4182, c = 2.1885, a = 2.2297
    t = 3: y = 1.0000, m = 3.3189, c = 2.1823, a = 1.1365
    utility = -0.00207765

**Question I:** Could we easily allowing for borrowing, i.e.

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,c_3} \frac{c_{1}^{1-\rho}}{1-\rho} + \beta \frac{c_{2}^{1-\rho}}{1-\rho} + \beta^2\frac{c_{3}^{1-\rho}}{1-\rho} + \beta^2\nu\frac{(a_{3}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_1 &= (1+r)a_0 + y_1\\
a_1 &= m_1-c_1\\
m_2 &= (1+r)a_1 + y_2\\
a_2 &= m_2-c_2\\
m_3 &= (1+r)a_2 + y_3\\
a_3 &= m_3-c_3\\
c_1,c_2,c_3 &\geq 0\\
a_1,a_2 &\geq -\lambda\\
a_3 &\geq 0
\end{aligned}
$$

**Question II:** Could we easily extend the problem to more periods?

$$
\begin{aligned}
V(a_0) &= \max_{c_1,c_2,\dots c_T} \sum_{t=1}^T \beta^{t-1} \frac{c_{t}^{1-\rho}}{1-\rho} + \beta^{T+1}\nu\frac{(a_{T}+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
&\text{s.t.}&\\
m_t &= (1+r)a_{t-1} + y_t\\
c_t &\geq 0\\
a_t &\geq 0
\end{aligned}
$$

**Follow-up question:** What is the problem for $T \rightarrow \infty$?

# Interpolation

**Intermezzo:** To consider dynamic optimization problems, we need to think about interpolation.

**Inputs:**

1. Sorted vector of known points (grid vector), $G = \{G_i\}_{i=0}^{n-1}$
2. Vector of known values (at these points), $F = \{F_i = f(G_i)\}_{i=0}^{n-1}$
3. A new point, `x`

**Algorithm:** `linear_interpolate()`
1. Determine `i`  such that

$$
G_i \leq x < G_{i+1}
$$

2. Compute interpolated value by

$$
y =  F_{i} + \frac{F_{i+1}-F_{i}}{G_{i+1}-G_{i}}(x-G_{i})
$$

**Extrapolation:**

1. Below where $x < G_1$:

$$
y =  F_{0} + \frac{F_{1}-F_{0}}{G_{1}-G_{0}}(x-G_{0})
$$

2. Above where $x > G_{n-2}$:

$$
y =  F_{n-2} + \frac{F_{n-1}-F_{n-2}}{G_{n-1}-G_{n-2}}(x-G_{n-2})
$$

```python
def linear_interpolate(G,F,x):
    """ linear interpolation (and extrapolation)

    Args:

        G (np.ndarray): known points
        F (np.ndarray): known values
        x (float): point to be interpolated

    Returns:

        y (float): intepolated value

    """

    assert len(G) == len(F)
    n = len(G)

    # a. find index in known points
    if x < G[1]: # exprapolation below
        i = 0
    elif x > G[-2]: # extrapolation above
        i = n-2
    else: # true interpolation

        # search
        i = 0
        while x >= G[i+1] and i < n-1:
            i += 1

        assert x >= G[i]
        assert x < G[i+1]

    # b. interpolate
    diff_G = G[i+1]-G[i]
    diff_F = F[i+1]-F[i]
    slope = diff_F/diff_G
    y = F[i] + slope*(x-G[i])

    return y
```

## Example

Consider the following function and known points:

```python
f = lambda x: (x-3)**3 - 3*x**2 + 5*x

G = np.linspace(-5,10,6)
F = f(G)
```

**Simple test:**

```python
for x in [-2.3,4.1,7.5,9.1]:
    true = f(x)
    y = linear_interpolate(G,F,x)
    print(f'x = {x:4.1f} -> true = {true:6.1f}, interpolated = {y:6.1f}')
```

    x = -2.3 -> true = -176.2, interpolated = -193.5
    x =  4.1 -> true =  -28.6, interpolated =  -27.7
    x =  7.5 -> true =  -40.1, interpolated =  -24.5
    x =  9.1 -> true =   24.1, interpolated =   50.7

**Scipy.interpolate:** Use the *RegularGridInterpolator*

```python
# a. construct interpolation function
interp_func = interpolate.RegularGridInterpolator([G],F,
                                                  bounds_error=False,
                                                  fill_value=None)

# bounds_error=False and fill_value=None allow for extrapolation

# b. interpolate
grid = np.linspace(-7,12,500)
interp_values = interp_func(grid)

# c. evaluate true values
true_values = f(grid)

# d. plot true and interpolated values
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.plot(G,F,'o',label='known points')
ax.plot(grid,true_values,'-',lw=1,label='true function')
ax.plot(grid,interp_values,'-',lw=1,label='interpolated values')
ax.legend(loc='lower right',facecolor='white',frameon=True);
```

**Note:**

1. Linear interpolation works best when the function does not curve too much.
2. Extrapolation is much worse than interpolation.

**Multiple dimensions:** Same principle, ``interpolate.RegularGridInterpolator([G1,G2,G3],F)``.

# Dynamic optimization problems

The following subject is hard. But also extremely useful. *If you master this, you can solve (almost) all economic models you meet on your way in life*.

## Problem formulation

Consider a **household** living in two periods.

In the **second period** it gets utility from **consuming** and **leaving a bequest** (warm glow),

$$
\begin{aligned}
v_{2}(m_{2})&= \max_{c_{2}}\frac{c_{2}^{1-\rho}}{1-\rho}+\nu\frac{(a_2+\kappa)^{1-\rho}}{1-\rho}\\
\text{s.t.} \\
a_2 &= m_2-c_2 \\
a_2 &\geq 0
\end{aligned}
$$

where

* $m_2$ is cash-on-hand
* $c_2$ is consumption
* $a_2$ is end-of-period assets
* $\rho > 1$ is the risk aversion coefficient
* $\nu > 0 $ is the strength of the bequest motive
* $\kappa > 0$ is the degree of luxuriousness in the bequest motive
* $a_2\geq0$ ensures the household *cannot* die in debt

The **value function** $v(m_2)$ measures the household's value of having $m_2$ at the beginning of period 2.

```python
def utility(c,par):
    return c**(1-par.rho)/(1-par.rho)

def bequest(m,c,par):
    return par.nu*(m-c+par.kappa)**(1-par.rho)/(1-par.rho)

def v2(c2,m2,par):
    return utility(c2,par) + bequest(m2,c2,par)
```

In the **first period**, the household gets utility from consuming and takes into account that it will also live in the next-period, where it receives a stochastic income,

$$
\begin{aligned}
v_1(m_1)&=\max_{c_1}\frac{c_{1}^{1-\rho}}{1-\rho}+\beta\mathbb{E}_{1}\left[v_2(m_2)\right]\\&\text{s.t.}&\\
a_1&=m_1-c_1\\
m_2&= (1+r)(m_1-c_1)+y_2 \\
y_{2}&= \begin{cases}
1-\Delta & \text{with prob. }0.5\\
1+\Delta & \text{with prob. }0.5
\end{cases}\\
a_1&\geq0
\end{aligned}
$$

where

* $m_1$ is cash-on-hand in period 1
* $c_1$ is consumption in period 1
* $a_1$ is end-of-period assets in period 1
* $\beta > 0$ is the discount factor
* $\mathbb{E}_1$ is the expectation operator conditional on information in period 1
* $y_2$ is income in period 2
* $\Delta \in (0,1)$ is the level of income risk (mean-preserving)
* $r$ is the interest rate
* $a_1\geq0$ ensures the household *cannot* borrow

```python
def v1(c1,m1,par,v2_interp):

    # a. v2 value, if low income
    m2_low = (1+par.r)*(m1-c1) + 1-par.Delta
    v2_low = v2_interp([m2_low])[0]

    # b. v2 value, if high income
    m2_high = (1+par.r)*(m1-c1) + 1+par.Delta
    v2_high = v2_interp([m2_high])[0]

    # c. expected v2 value
    prob_low = 0.5
    prob_high = 0.05
    expected_v2 = prob_low*v2_low + prob_high*v2_high

    # d. total value
    return utility(c1,par) + par.beta*expected_v2
```

## Solve household problem

Choose **parameters**:

```python
par = SimpleNamespace()
par.rho = 8
par.kappa = 0.5
par.nu = 0.1
par.r = 0.04
par.beta = 0.94
par.Delta = 0.5
```

**Solve second period:**

```python
def solve_period_2(par):

    # a. grids
    m2_vec = np.linspace(1e-4,5,500)
    v2_vec = np.empty(500)
    c2_vec = np.empty(500)

    # b. solve for each m2 in grid
    for i,m2 in enumerate(m2_vec):

        # i. objective
        obj = lambda x: -v2(x[0],m2,par)

        # ii. initial value (consume half)
        x0 = m2/2

        # iii. optimizer
        result = optimize.minimize(obj,[x0],method='L-BFGS-B',bounds=((1e-8,m2),))

        # iv. save
        v2_vec[i] = -result.fun
        c2_vec[i] = result.x

    return m2_vec,v2_vec,c2_vec

# solve
m2_vec,v2_vec,c2_vec = solve_period_2(par)

# illustration
fig = plt.figure(figsize=(10,4))
ax = fig.add_subplot(1,2,1)
ax.plot(m2_vec,c2_vec)
ax.set_xlabel('$m_2$')
ax.set_ylabel('$c_2$')
ax.set_title('consumption function in period 2')

ax = fig.add_subplot(1,2,2)
ax.plot(m2_vec,v2_vec)
ax.set_xlabel('$m_2$')
ax.set_ylabel('$v_2$')
ax.set_title('value function in period 2')
ax.set_ylim([-40,1]);
```

**Note:** We now solve for the consumption function, rather than a specific optimum.

**Question:** Why is there a kink in the consumption function?

**Construct interpolator:**

```python
v2_interp = interpolate.RegularGridInterpolator([m2_vec], v2_vec,
                                                bounds_error=False,fill_value=None)
```

**Solve first period:**

```python
def solve_period_1(par,v2_interp):

    # a. grids
    m1_vec = np.linspace(1e-8,4,100)
    v1_vec = np.empty(100)
    c1_vec = np.empty(100)

    # b. solve for each m1 in grid
    for i,m1 in enumerate(m1_vec):

        # i. objective
        obj = lambda x: -v1(x[0],m1,par,v2_interp)

        # ii. initial guess (consume half)
        x0 = m1/2

        # iii. optimize
        result = optimize.minimize(obj,[x0],method='L-BFGS-B',bounds=((1e-12,m1),))

        # iv. save
        v1_vec[i] = -result.fun
        c1_vec[i] = result.x[0]

    return m1_vec,v1_vec,c1_vec

# solve
m1_vec,v1_vec,c1_vec = solve_period_1(par,v2_interp)

# illustrate
fig = plt.figure(figsize=(10,4))
ax = fig.add_subplot(1,2,1)
ax.plot(m1_vec,c1_vec)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('consumption function in period 1')

ax = fig.add_subplot(1,2,2)
ax.plot(m1_vec,v1_vec)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('value function in period 1')
ax.set_ylim([-40,1]);
```

**Summary:** We can summarize what we have done in a single function doing:

1. Solve period 2 (i.e. find $v_2(m_2)$ og $c_2(m_2)$)
2. Construct interpolator of $v_2(m_2)$
3. Solve period 1 (i.e. find $v_1(m_1)$ og $c_1(m_1)$)

```python
def solve(par):

    # a. solve period 2
    m2_vec,v2_vec,c2_vec = solve_period_2(par)

    # b. construct interpolator
    v2_interp = interpolate.RegularGridInterpolator([m2_vec], v2_vec,
        bounds_error=False,fill_value=None)

    # b. solve period 1
    m1_vec,v1_vec,c1_vec = solve_period_1(par,v2_interp)

    return m1_vec,c1_vec,m2_vec,c2_vec
```

**Plot consumption function for various level of income risk**, i.e varios $\Delta$

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

_Delta = par.Delta
for Delta in [0.05,0.15,0.25]:
    par.Delta = Delta
    m1_vec,c1_vec,m2_vec,c2_vec = solve(par)
    ax.plot(m1_vec,c1_vec,label=f'$\Delta = {Delta}$')

# reset
par.Delta = _Delta
ax.legend(loc='lower right',facecolor='white',frameon=True)
ax.set_xlabel('$m_1$')
ax.set_ylabel('$c_1$')
ax.set_title('value function in period 1')
ax.set_xlim([0,2])
ax.set_ylim([0,1.5]);
```

**Main takeaway:** The household lower its consumption when risk increases (such as in a recession). This is called **precautionary saving**.

## Simulation

**Step 1:** Solve and construct interpolators:

```python
Delta = 0.5
m1_vec,c1_vec,m2_vec,c2_vec = solve(par)

c1_interp = interpolate.RegularGridInterpolator([m1_vec], c1_vec,
                                                bounds_error=False,fill_value=None)

c2_interp = interpolate.RegularGridInterpolator([m2_vec], c2_vec,
                                                bounds_error=False,fill_value=None)
```

**Step 2:** Draw initail distribution of $m_1$ and simulate forward

```python
# a. draw initial m1
simN = 10000
sim_m1 = np.fmax(np.random.normal(1,0.1,size=simN),0) # "randomly" chosen distribution

# b. period 1
sim_c1 = c1_interp(sim_m1)
sim_a1 = sim_m1-sim_c1

# c. transition to period 2 with random draw
sim_m2 = (1+par.r)*sim_a1+np.random.choice([0.5,1.5],p=[0.5,0.5])

# d. period 2
sim_c2 = c2_interp(sim_m2)
```

**Step 3:** Plot distributions

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.hist(sim_c1,bins=100,label='period 1')
ax.hist(sim_c2,bins=100,label='period 2')

ax.legend(loc='lower right',facecolor='white',frameon=True)
ax.set_xlabel('$c_t$')
ax.set_ylabel('freq.')
ax.set_title('consumption');
```

**Conclusion:** You can now solve models with complex heterogeneity and uncertainty, and simulate the implied dynamics. By introducing various policies you can quantify their effect not just for the average, but for the full distribution.

# Summary

**This lecture:**

1. Solving multidimensional optimization problems with and without gradients (and hessians)
2. Using multistart to alleviate problems with local minima (due to non-convexities)
3. Using penalty terms to solve constrained optimization problems
4. Linear interpolation between known points
5. Solving dynamic optimization problems backwards period-by-period

**Dynamic optimization:** Extremely useful technique. Can handle multiple periods, multiple states and choices, more shocks etc. You can solve general equilibrium models where the households solve such problems.

**Need more dynamic optimization?** [Mini-Course on Dynamic Programming](https://github.com/NumEconCopenhagen/ConsumptionSavingNotebooks/tree/master/00.%20DynamicProgramming#mini-course-in-dynamic-programming)

**Next lecture:** Canonical Economic Models.


$\newcommand{\myvector}[2]{\left\{ \begin{array}{c}#1\\#2\end{array} \right\}}$
$\newcommand{\sM}[1]{s_{\mathrm{M#1}}}$
$\newcommand{\xM}[1]{x_{\mathrm{M#1}}}$
$\newcommand{\xpM}[1]{x'_{\mathrm{M#1}}}$

# APPENDIX B:  Bayesian vs. least-squares estimates of population means

This Appendix compares Bayesian to least-squares (LS) estimates of population means, where the LS estimate is simply the sample mean (or arithmetic mean). This Appendix aims to demonstrate that even rudimentary LS concepts like the sample mean can be interpreted somewhat differently when regarding data from a Bayesian perspective.

Below two Bayesian approaches to population mean estimation are described, and these results are then subsequently compared to the LS mean.

Let's start by drawing a random sample from the standard normal distribution, for which the true population mean and standard deviation (SD) are 0 and 1, respectively:

```python
%matplotlib notebook

import numpy as np
from matplotlib import pyplot
import pymc

np.random.seed(4)        #arbitrarily set the random number generator seed
J = 5                    #sample size
y = np.random.randn(J)   #random data sample
print('A random sample from the standard normal distribution:')
print('   (True population mean = 0)')
print('   (True population SD   = 1)')   #standard deviation (SD)
print(y)
```

    A random sample from the standard normal distribution:
       (True population mean = 0)
       (True population SD   = 1)
    [ 0.05056171  0.49995133 -0.99590893  0.69359851 -0.41830152]

## Approach 1: Least-squares

The least squares (LS) solution minimizes the following function:

\begin{equation}
f(x)  = \sum (y_i - x)^2
\end{equation}

where $x$ is the estimate of the population mean, and $y_i$ are the individual sample observations. The value of $x$ which minimizes $f(x)$ is the sample mean:

```python
mean_ls = np.mean( y )  #optimum value for "x" in the equation above
print('Least-squares estimate of the population mean (i.e. sample mean):')
print( mean_ls )
```

    Least-squares estimate of the population mean (i.e. sample mean):
    -0.0340197804923

Note that the sample mean is not zero, even through the true population mean is zero.  For the sample mean to converge precisely to the true population mean, one would generally have to draw an infinitely large sample, or sample the entire population.

## Approach 2: Bayesian (known variance)

The simplest Bayesian approach is to presume that population SD ($\sigma$) is known. Let's implement the full solution in PyMC, then consider the commands afterwards.

```python
#(1) Build model
x          = pymc.Uniform("x", -3, 3)  #prior distribution for the population mean
sigma      = 1                         #known population standard deviation (SD)
tau        = 1.0 / sigma**2            #measurement precision ( 1 / SD^2 )
data_model = pymc.Normal("data_model", x, tau, observed=True, value=y)  #model of the observations

#(2) Conduct inference:
model = pymc.Model([data_model, x])    #full model containing data model and all stochastic variables
mcmc  = pymc.MCMC(model)               #Markov Chain Monte Carlo sampler
mcmc.sample(iter=11000, burn=1000)     #MC sampling
post  = mcmc.trace("x")[:]             #posterior distribution for the population mean
bmean = post.mean()                    #Bayesian estimate of the population mean
print()
print('Bayesian estimate of the population mean (known variance):')
print(bmean)
```

     [-----------------100%-----------------] 11000 of 11000 complete in 0.4 sec
    Bayesian estimate of the population mean (known variance):
    -0.0292631259209

Note that the Bayesian estimate of the population mean is closer to its true value of zero than was the LS estimate. Let's consider each step of the Bayesian estimation procedure outlined in the code above.

In order to conduct Bayesian inference regarding experimental data, a model of those data is required. The first step is to create a prior distribution for each stochastic variable:

```python
x          = pymc.Uniform("x", -3, 3)  #naive prior distribution for the population mean
```

This prior implies that we expect the true population mean to be between -1 and +1, and that all values in this range are equally likely. As described in Appendix A, the prior can affect the final results, and we shall consider effects of priors below.

The next step is to build a model of the data:

```python
data_model = pymc.Normal("data_model", x, tau, observed=True, value=y)  #model of the observations
```

This command implies that we believe that our data sample comes from a normal distribution with an unknown population mean of x and a known precision of tau=1. The "observed=True" and "value=y" keyword arguments specify that the values in y were observed. In other words, based on the values y we want to find the most likely value for x, or more generally the posterior distribution for x given y.

Once we have the model we can conduct inference, and the Markov Chain Monte Carlo (MCMC)-related inference commands above follow from Appendix A so are not discussed further here. Following inference, it can be informative to visualize the prior and the posterior distributions:

```python
prior   = [x.random() for i in range(10000)]

pyplot.figure()
ax = pyplot.axes()
ax.hist(prior, bins=21, alpha=0.5, label='Prior')      #prior distribution for x
ax.hist(post,  bins=21, alpha=0.5, label='Posterior')  #posterior distribution for x
ax.legend();
```

    <IPython.core.display.Javascript object>

The observed data sample y has changed our beliefs regarding x from the depicted prior distribution into the depicted posterior distribution via Bayes' rule.

## Approach 3: Bayesian (unknown variance)

In order to model unknown variance we just need to change the precision variable (tau) into a stochastic variable, then ensure that tau is included in the full pymc model like this:

```python
#(1) Build model
x          = pymc.Uniform("x", -3, 3)       #prior distribution for the population mean
tau        = pymc.Uniform("tau", 0.01, 100) #prior distribution for precision
data_model = pymc.Normal("data_model", x, tau, observed=True, value=y)  #model of the observations

#(2) Conduct inference:
model = pymc.Model([data_model, x, tau])   #full model containing data model and all stochastic variables
mcmc  = pymc.MCMC(model)               #MCMC sampler
mcmc.sample(iter=11000, burn=1000)     #MC sampling
post  = mcmc.trace("x")[:]             #posterior distribution for the population mean
bmean = post.mean()                    #Bayesian estimate of the population mean
print()
print('Bayesian estimate of the population mean (unknown variance):')
print(bmean)
```

     [-----------------100%-----------------] 11000 of 11000 complete in 0.7 sec
    Bayesian estimate of the population mean (unknown variance):
    -0.028969135243

Adding tau as a stochastic variable has not greatly affected the results.

## Systematic comparison between Bayesian and LS approaches

The analyses above represent a single dataset, single sample size, and single prior distributions, and results showed that the Bayesian results were somewhat more accurate. How do Bayesian and LS approaches compare for arbitrary datasets, sample sizes, priors, etc.?  To answer that question comprehenively would require millions of simulations. Here we shall consider only effects of dataset, and we'll check whether Bayesian or LS approaches perform better.

```python
#create a function for estimating population mean using Bayesian inference:
def bayesian_mean(y, iter=11000, burn=1000):
    #build model:
    x       = pymc.Uniform("x", -3, 3)
    tau     = pymc.Uniform("tau", 0.01, 100)
    dmodel  = pymc.Normal("data_model", x, tau, observed=True, value=y)
    #conduct inference:
    model   = pymc.Model([data_model, x])
    mcmc    = pymc.MCMC(model)
    mcmc.sample(iter=iter, burn=burn, progress_bar=False)
    post    = mcmc.trace("x")[:]
    return post.mean()

np.random.seed(0)
nDatasets   = 20   #number of datasets to generate
n           = 0    #number of datasets for which the Bayesian estimate is more accurate
for i in range(nDatasets):
    y       = np.random.randn(J)
    mean_ls = y.mean()
    mean_b  = bayesian_mean(y, iter=110000, burn=10000)
    if (abs(mean_ls) < abs(mean_b)):
        result = 'LS'
    else:
        result = 'Bayesian'
        n     += 1
    print('Dataset %d.  LS: %.5f, Bayesian: %.5f (%s more accurate)' %(i, mean_ls, mean_b, result))
print()
print('Bayesian more accurate in %d/%d datasets.' %(n,nDatasets))
```

    Dataset 0.  LS: 1.45028, Bayesian: 1.44847 (Bayesian more accurate)
    Dataset 1.  LS: 0.15133, Bayesian: 0.15841 (LS more accurate)
    Dataset 2.  LS: -0.72543, Bayesian: -0.72868 (LS more accurate)
    Dataset 3.  LS: -0.22593, Bayesian: -0.22542 (Bayesian more accurate)
    Dataset 4.  LS: -0.11956, Bayesian: -0.11465 (Bayesian more accurate)
    Dataset 5.  LS: 1.47015, Bayesian: 1.46882 (Bayesian more accurate)
    Dataset 6.  LS: -0.80645, Bayesian: -0.80766 (LS more accurate)
    Dataset 7.  LS: -0.40028, Bayesian: -0.39992 (Bayesian more accurate)
    Dataset 8.  LS: 0.22548, Bayesian: 0.22538 (Bayesian more accurate)
    Dataset 9.  LS: -0.56589, Bayesian: -0.56771 (LS more accurate)
    Dataset 10.  LS: 0.00039, Bayesian: 0.00468 (LS more accurate)
    Dataset 11.  LS: 0.21691, Bayesian: 0.21854 (LS more accurate)
    Dataset 12.  LS: -0.14829, Bayesian: -0.14565 (Bayesian more accurate)
    Dataset 13.  LS: -0.18566, Bayesian: -0.18351 (Bayesian more accurate)
    Dataset 14.  LS: -0.41073, Bayesian: -0.41028 (Bayesian more accurate)
    Dataset 15.  LS: 0.22798, Bayesian: 0.22986 (LS more accurate)
    Dataset 16.  LS: 0.25266, Bayesian: 0.24976 (Bayesian more accurate)
    Dataset 17.  LS: -0.35683, Bayesian: -0.35616 (Bayesian more accurate)
    Dataset 18.  LS: 0.20721, Bayesian: 0.20923 (LS more accurate)
    Dataset 19.  LS: -0.47126, Bayesian: -0.46361 (Bayesian more accurate)

    Bayesian more accurate in 12/20 datasets.

We can see that the Bayesian approach is occasionally, but not compellingly more accurate than the LS approach.

## Summary

This Appendix has shown that population means can be estimated not only using the typical LS approach (i.e. the sample mean), but also using at least two different Bayesian approaches. Results show that Bayesian estimates are slightly more accurate than LS estimates for some datasets, but overall that there is not much difference between Bayesian and LS results. However, this is a very simple case of Bayesian inference, with a very simple data model. Subsequent Appendices and the main paper show that Bayesian and LS accuracies diverge as the data model becomes increasingly complex.


```python
%matplotlib inline
import warnings

import matplotlib.pyplot as plt
import numpy as np
from matplotlib import gridspec

warnings.filterwarnings('ignore')
```
<style type="text/css">
.input, .output_prompt {
display:none !important;
}
</style>

# Introduction to Pulsar Timing

[](http://mybinder.org/repo/matteobachetti/timing-lectures)

(These slides are obtained from the iPython notebook that can be found [here](https://github.com/matteobachetti/timing-lectures))

## Contents

* Finding pulsars: the buried clock

* Frequency analysis: the Fourier Transform and the Power Density Spectrum

* Refine the search: Folding search (+ $Z^2_n$, $H$-test, ...)

* Getting pulse arrival times

# Finding pulsars: the buried clock

# Finding pulsars: the buried clock

* Pulsars are stable rotators: very predictable "clocks"

# Finding pulsars: the buried clock

* Pulsars are stable rotators: very predictable "clocks"

* Often signal buried in noise (below: a 0.853-Hz sinusoidal pulse buried in noise ~30 times stronger)

```python
def pulsar(time, period=1):
    return (1 + np.sin(2 * np.pi / period * time)) / 2.

time_bin = 0.009

# --- Parameters ----
period = 0.8532
pulsar_amp = 0.5
pulsar_stdev = 0.05
noise_amp = 15
noise_stdev = 1.5
# --------------------

# refer to the center of the time bin
time = np.arange(0, 100, time_bin) + time_bin / 2

signal = np.random.normal(pulsar_amp * pulsar(time, period), pulsar_stdev)
noise = np.random.normal(noise_amp, noise_stdev, len(time))
total = signal + noise

# PLOTTING -------------------------
plt.plot(time, signal, 'r-', label='signal')
plt.plot(time, noise, 'b-', label='noise')
plt.plot(time, total, 'k-', label='total')
plt.xlim(0, 30)
plt.xlabel('Time')
plt.ylabel('Flux')
a = plt.legend()
# -----------------------------------
```

# Frequency analysis: the Fourier Transform

Through the Fourier transform, we can decompose a function of time into a series of functions of frequency:

\begin{equation}
\mathcal{F}(\omega) = \int^{\infty}_{-\infty} e^{-i\omega t} f(t)
\end{equation}

or, more appropriate to our case, in the discrete form, we can decompose a time series into a frequency series:

\begin{equation}
F_k = \sum^{N-1}_{k=0} e^{-2\pi i k n/N} t_n
\end{equation}

it is, in general, a **complex** function.

The Fourier transform of a sinusoid will give a high (in absolute terms) value of the $F_k$ corresponding to the frequency of the sinusoid. Other periodic functions will produce high contribution at the fundamental frequency plus one or more multiples of the fundamental, called *harmonics*.

## Our example

Let's take the Fourier transform of the signal we simulated above (only taking *positive* frequencies)

```python
ft = np.fft.fft(total)
freqs = np.fft.fftfreq(len(total), time[1] - time[0])
good = freqs >0
freqs = freqs[good]
ft = ft[good]

# PLOTTING ---------------------------
plt.plot(freqs, ft.real, 'r-', label='real')
plt.plot(freqs, ft.imag, 'b-', label='imag')
plt.xlim([-0.1, 10])
a = plt.legend()
_ = plt.xlabel('Frequency (Hz)')
_ = plt.ylabel('FT')
# -------------------------------------
```

Note that the imaginary part and real part of the Fourier transform have different contributions at the pulsar frequency (1/0.85 s ~ 1.2 Hz). This is because they depend strongly on the phase of the signal [Exercise: **why?**].

## Our example - 2

 If we applied a shift of 240 ms (just any value) to the signal:

```python
shift = 0.240
signal_shift = np.roll(total, int(shift / time_bin))
ft_shift = np.fft.fft(signal_shift)
freqs_shift = np.fft.fftfreq(len(total), time[1] - time[0])
good = freqs_shift >0
freqs_shift = freqs_shift[good]
ft_shift = ft_shift[good]

# PLOTTING -------------------------------------
plt.plot(freqs_shift, ft_shift.real, 'r-', label='real')
plt.plot(freqs_shift, ft_shift.imag, 'b-', label='imag')
plt.xlim([-0.1, 10])
a = plt.legend()
_ = plt.xlabel('Frequency (Hz)')
_ = plt.ylabel('FT')
# ----------------------------------------------
```

we would clearly have non-zero values at ~0.85 Hz both in the real and the imaginary parts.

# The Power Density Spectrum

To solve these issues with real and imaginary parts, we can instead take the *squared modulus* of the Fourier transform. This is called **Periodogram**, but most people use the word **Power Density Spectrum** (a periodogram is actually a single realization of the underlying PDS).

\begin{equation}
\mathcal{P}(\omega) = \mathcal{F}(\omega) \cdot \mathcal{F}^*(\omega)
\end{equation}

This function is positive-definite and in our case results in a clear peak at the pulse frequency, *consistent* between original and shifted signal:

```python
pds = np.abs(ft*ft.conj())
pds_shift = np.abs(ft_shift*ft_shift.conj())

fmax = freqs[np.argmax(pds)]
pmax = 1 / fmax

# PLOTTING ---------------------------------
plt.plot(freqs, pds, 'r-', label='PDS of signal')
plt.plot(freqs_shift, pds_shift, 'b-', label='PDS of shifted signal')
a = plt.legend()
a = plt.xlabel('Frequency (Hz)')
a = plt.ylabel('PDS')
plt.xlim([-0.1, 3.5])
_ = plt.gca().annotate('max = {:.2f} s ({:.2f} Hz)'.format(pmax, fmax), xy=(2., max(pds) / 2))
# -------------------------------------------
```

## The Power Density Spectrum -2

The PDS of a generic non-sinusoidal pulse profile will, in general, contain more than one harmonic, with the fundamental not always predominant.

```python
def gaussian_periodic(x, x0, amp, width):
    '''Approximates a Gaussian periodic function by summing the contributions in the phase
    range 0--1 with those in the phase range -1--0 and 1--2'''
    phase = x - np.floor(x)
    lc = np.zeros_like(x)
    for shift in [-1, 0, 1]:
        lc += amp * np.exp(-(phase + shift - x0)**2 / width ** 2)

    return lc

def generate_profile(time, period):
    '''Simulate a given profile with 1-3 Gaussian components'''
    total_phase = time / period
    ngauss = np.random.randint(1, 3)
    lc = np.zeros_like(total_phase)
    for i in range(ngauss):
        ph0 = np.random.uniform(0, 1)
        amp = np.random.uniform(0.1, 1)
        width = np.random.uniform(0.01, 0.2)
        lc += gaussian_periodic(total_phase, ph0, amp, width)

    return lc

# PLOTTING -------------------------
ncols = 2
nrows = 3
fig = plt.figure(figsize=(12, 8))
fig.suptitle('Profiles and their PDSs')

gs = gridspec.GridSpec(nrows, ncols)

for c in range(ncols):
    for r in range(nrows):
# ----------------------------------
        noise = np.random.normal(noise_amp, noise_stdev, len(time))

        lc = generate_profile(time, period)
        lc_noisy = np.random.normal(2 * lc, 0.2) + noise

        lcft = np.fft.fft(lc_noisy)
        lcfreq = np.fft.fftfreq(len(lc_noisy), time[1] - time[0])
        lcpds = np.absolute(lcft) ** 2

        # PLOTTING -------------------------

        gs_2 = gridspec.GridSpecFromSubplotSpec(1, 2, subplot_spec=gs[r, c])
        ax = plt.subplot(gs_2[0])
        good = time < period * 3
        ax.plot(time[good] / period, lc[good])

        ax.set_xlim([0,3])
        ax.set_xlabel('Phase')

        ax = plt.subplot(gs_2[1])
        ax.plot(lcfreq[lcfreq > 0], lcpds[lcfreq > 0] / max(lcpds[lcfreq > 0]))
        ax.set_xlabel('Frequency')
        ax.set_xlim([0, 10])

        # ----------------------------------
```

## Pulsation?

Here are some examples of power density spectra. In some cases, it might look like a pulsation is present in the data. How do we assess this?

```python
# PLOTTING -------------------------
ncols = 2
nrows = 3
fig = plt.figure(figsize=(12, 8))
fig.suptitle('Profiles and their PDSs')

gs = gridspec.GridSpec(nrows, ncols)

for c in range(ncols):
    for r in range(nrows):
# ----------------------------------

        noise = np.random.normal(noise_amp, noise_stdev, len(time))
        lc = np.zeros_like(time)
        lc_noisy = np.random.normal(2 * lc, 0.2) + noise

        lcft = np.fft.fft(lc_noisy)
        lcfreq = np.fft.fftfreq(len(lc_noisy), time[1] - time[0])
        lcpds = np.absolute(lcft) ** 2

        # PLOTTING -------------------------

        gs_2 = gridspec.GridSpecFromSubplotSpec(1, 2, subplot_spec=gs[r, c])
        ax = plt.subplot(gs_2[0])
        good = time < period * 3
        ax.plot(time[good] / period, lc[good])

        ax.set_xlim([0,3])
        ax.set_xlabel('Phase')

        ax = plt.subplot(gs_2[1])
        ax.plot(lcfreq[lcfreq > 0], lcpds[lcfreq > 0] / max(lcpds[lcfreq > 0]))
        ax.set_xlabel('Frequency')
        ax.set_xlim([0, 10])

        # ----------------------------------
```

# Epoch folding

Epoch folding consists of summing equal, one pulse period-long, chunks of data. If the period is just right, the crests will sum up in phase, gaining signal over noise [Exercise: **how much will we gain** by summing up in phase $N$ chunks of data at the right period?].

```python
def epoch_folding(time, signal, period, nperiods=3, nbin=16):
    # The phase of the pulse is always between 0 and 1.
    phase = time / period
    phase -= phase.astype(int)

    # First histogram: divide phase range in nbin bins, and count how many signal bins
    # fall in each histogram bin. The sum is weighted by the value of the signal at
    # each phase
    prof_raw, bins = np.histogram(
        phase, bins=np.linspace(0, 1, nbin + 1),
        weights=signal)
    # "Exposure": how many signal bins have been summed in each histogram bin
    expo, bins = np.histogram(phase, bins=np.linspace(0, 1, nbin + 1))

    # ---- Evaluate errors -------
    prof_sq, bins = np.histogram(
        phase, bins=np.linspace(0, 1, nbin + 1),
        weights=signal ** 2)
    # Variance of histogram bin: "Mean of squares minus square of mean" X N
    hist_var = (prof_sq / expo - (prof_raw / expo) ** 2) * expo
    # Then, take square root -> Stdev, then normalize / N.
    prof_err = np.sqrt(hist_var)
    #-----------------------------
    # Normalize by exposure
    prof = prof_raw / expo
    prof_err = prof_err / expo

    # histogram returns all bin edges, including last one. Here we take the
    # center of each bin.
    phase_bins = (bins[1:] + bins[:-1]) / 2

    # ---- Return the same pattern 'nperiods' times, for visual purposes -----
    final_prof = np.array([])
    final_phase = np.array([])
    final_prof_err = np.array([])
    for n in range(nperiods):
        final_prof = np.append(final_prof, prof)
        final_phase = np.append(final_phase, phase_bins + n)
        final_prof_err = np.append(final_prof_err, prof_err)
    # ---------------------------
    return final_phase, final_prof, final_prof_err

phase, profile, profile_err = epoch_folding(time, total, period)
phase_shift, profile_shift, profile_shift_err = epoch_folding(time, signal_shift, period)

# PLOTTING -------------------------------------------------------------
plt.errorbar(
    phase, profile, yerr=profile_err, drawstyle='steps-mid',
    label='Signal')
plt.errorbar(
    phase_shift, profile_shift, yerr=profile_shift_err,
    drawstyle='steps-mid', label='Shifted signal')

_ = plt.legend()
_ = plt.xlabel('Phase')
_ = plt.ylabel('Counts/bin')
# -------------------------------------------------------------------
```

# Epoch folding search

Now, let's run epoch folding at a number of trial periods around the pulse period. To evaluate how much a given profile "looks pulsar-y", we can use the $\chi^2$ statistics, as follows:

\begin{equation}
\mathcal{S} = \sum_{i=0}^N \frac{(p_i - \bar{p})^2}{\sigma_p^2}
\end{equation}

for each profile obtained for each trial value of the pulse frequency and look for peaks$^1$. [Exercise: do you know what statistics this is? And why does that statistics work for our case? Exercise-2: Note the very large number of trials. Can we optimize the search so that we use less trials without losing sensitivity?]

$^1$ 1.	Leahy, D. A. et al. On searches for pulsed emission with application to four globular cluster X-ray sources - NGC 1851, 6441, 6624, and 6712. _ApJ_ **266**, 160 (1983).

```python
def pulse_profile_stat(profile, profile_err):
    return np.sum(
        (profile - np.mean(profile)) ** 2 / profile_err ** 2)

trial_periods = np.arange(0.7, 1.0, 0.0002)
stats = np.zeros_like(trial_periods)
for i, p in enumerate(trial_periods):
    phase, profile, profile_err = epoch_folding(time, total, p)
    stats[i] = pulse_profile_stat(profile, profile_err)

bestp = trial_periods[np.argmax(stats)]

phase_search, profile_search, profile_search_err = \
    epoch_folding(time, total, bestp)
phase, profile, profile_err = epoch_folding(time, total, period)

# PLOTTING -------------------------------
fig = plt.figure(figsize=(10, 3))
gs = gridspec.GridSpec(1, 2)
ax = plt.subplot(gs[0])
ax.plot(trial_periods, stats)
ax.set_xlim([0.7, 1])
ax.set_xlabel('Period (s)')
ax.set_ylabel('$\chi^2$')
ax.axvline(period, color='r', label="True value")
_ = ax.legend()
ax.annotate('max = {:.5f} s'.format(pmax), xy=(.9, max(stats) / 2))

ax2 = plt.subplot(gs[1])

ax2.errorbar(phase_search, profile_search, yerr=profile_search_err,
             drawstyle='steps-mid', label='Search')
ax2.errorbar(phase, profile, yerr=profile_err, drawstyle='steps-mid',
             label='True period')
ax2.set_xlabel('Phase')
ax2.set_ylabel('Counts/bin')

_ = ax2.legend()
# ------------------------------------------
```

# Times of arrival (TOA)
To calculate the time of arrival of the pulses, we need to:

* Choose what **part of the pulse** is the reference (e.g., the maximum). Once we know that, if $\phi_{max}$ is the phase of the maximum of the pulse, $t_{start}$ the time at the start of the folded light curve, and $p$ is the folding period,

$TOA = t_{start} + \phi_{max} \cdot p$

* Choose a **method** to calculate the TOA:

  + The maximum bin?

  + The phase of a sinusoidal fit?

  + The phase of a more complicated fit?

Hereafter, we are going to use the maximum of the pulse as a reference, and we will calculate the TOA with the three methods above.

## TOA from the maximum bin

**Advantage**

* Very fast and easy to implement

**Disadvantages**

* Very rough (maximum precision, the width of the bin)

* Very uncertain (if statistics is low and/or the pulse is broad, many close-by bins can randomly be the maximum)

```python
phase_bin = 1 / 32.
ph = np.arange(phase_bin / 2, phase_bin / 2 + 1, phase_bin)
shape = np.sin(2 * np.pi * ph) + 2
pr_1 = np.random.poisson(shape * 10000) / 10000
pr_2 = np.random.poisson(shape * 10) / 10

# PLOTTING -----------------------------
plt.plot(ph, shape, label='Theoretical shape', color='k')
plt.plot(
    ph, pr_1, drawstyle='steps-mid', color='r',
    label='Shape - good stat')
plt.plot(
    ph, pr_2, drawstyle='steps-mid', color='b',
    label='Shape - bad stat')
plt.axvline(0.25, ls=':', color='k', lw=2, label='Real maximum')
plt.axvline(
    ph[np.argmax(pr_1)], ls='--', color='r', lw=2,
    label='Maximum - good stat')
plt.axvline(
    ph[np.argmax(pr_2)], ls='--', color='b', lw=2,
    label='Maximum - bad stat')

_ = plt.legend()
# --------------------------------------
```

## TOA from single sinusoidal fit

**Advantage**

* Relatively easy task (fitting with a sinusoid)

* Errors are well determined provided that the pulse is broad

**Disadvantages**

* If profile is not sinusoidal, might not be well determined

Below, the phase of the pulse is always 0.25

```python
def sinusoid(phase, phase0, amplitude, offset):
    return offset + np.cos(2 * np.pi * (phase - phase0))

from scipy.optimize import curve_fit

# PLOTTING ------------------
fig = plt.figure(figsize=(12, 3))
gs = gridspec.GridSpec(1, 4)
ax1 = plt.subplot(gs[0])
ax1.set_title('Theoretical')
ax2 = plt.subplot(gs[1])
ax2.set_title('Sinusoidal, good stat')
ax3 = plt.subplot(gs[2])
ax3.set_title('Sinusoidal, noisy')
ax4 = plt.subplot(gs[3])
ax4.set_title('Complicated profile')
# ---------------------------

# Fit sinusoid to theoretical shape
par, pcov = curve_fit(sinusoid, ph, shape)

# PLOTTING -----------------------------------------------
ax1.plot(ph, sinusoid(ph, *par))
ax1.plot(ph, shape)
par[0] -= np.floor(par[0])
ax1.annotate('phase = {:.2f}'.format(par[0]), xy=(.5, .3))
ax1.set_ylim([0, 4])

# Fit to good-stat line
# ---------------------------------------------------------

par, pcov = curve_fit(sinusoid, ph, pr_1)

# PLOTTING -----------------------------------------------

ax2.plot(ph, sinusoid(ph, *par))
ax2.plot(ph, pr_1)
par[0] -= np.floor(par[0])
ax2.annotate('phase = {:.2f}'.format(par[0]), xy=(.5, .3))
ax2.set_ylim([0, 4])

# Fit to bad-stat line
# ---------------------------------------------------------

par, pcov = curve_fit(sinusoid, ph, pr_2)

# PLOTTING -----------------------------------------------

ax3.plot(ph, sinusoid(ph, *par))
ax3.plot(ph, pr_2, drawstyle='steps-mid')
par[0] -= np.floor(par[0])
ax3.annotate('phase = {:.2f}'.format(par[0]), xy=(.5, .3))
ax3.set_ylim([0, 4])

# Now try with a complicated profile (a double Gaussian)
# ---------------------------------------------------------

pr_3_clean = 0.3 + np.exp(- (ph - 0.25) ** 2 / 0.001) + 0.5 * np.exp(- (ph - 0.75) ** 2 / 0.01)
pr_3 = np.random.poisson(pr_3_clean * 100) / 50
# Let us normalize the template with the same factor (100 / 50) of the randomized one. It will be helpful later
pr_3_clean *= 2

par, pcov = curve_fit(sinusoid, ph, pr_3, maxfev=10000)

# PLOTTING -----------------------------------------------

ax4.plot(ph, sinusoid(ph, *par), label='Fit')
ax4.plot(ph, pr_3, drawstyle='steps-mid', label='Noisy profile')
ax4.plot(ph, pr_3_clean, label='Real profile')

par[0] -= np.floor(par[0])
ax4.annotate('phase = {:.2f}'.format(par[0]), xy=(.5, .3))
ax4.set_ylim([0, 4])
_ = ax4.legend()
# ---------------------------------------------------------
```

## TOA from non-sinusoidal fit: multiple harmonic fitting

**Multiple harmonic fitting**$^1$ (the profile is described by a sum of sinusoids) is just an extension of the single-harmonic fit by adding additional sinusoidal components at multiple frequencies.

**Advantages**

* Still conceptually easy, but more robust and reliable

**Disadvantages**

* The phase is not determined by the fit (in general, it isn't he phase of any of the sinusoids [Exercise: why?]) and needs to be determined from the maximum of the profile. Errors are not straightforward to implement.

$^1$e.g. Riggio, A. et al. Timing of the accreting millisecond pulsar IGR J17511-3057. _A&A_ **526**, 95 (2011).

## TOA from non-sinusoidal fit: Template pulse shapes

* **Cross-correlation** of template pulse shape

* **Fourier-domain fitting (FFTFIT)**$^1$ -> the usual choice. Consists of taking the Fourier transform of the profile $\mathcal{P}$ and of the template $\mathcal{T}$ and minimizing the following objective function (similar to $\chi^2$):
\begin{equation}
F = \sum_k \frac{|\mathcal{P}_k - a\mathcal{T}_k e^{-2\pi i k\phi}|^2}{\sigma^2}
\end{equation}

**Advantages**

* Much more robust and reliable

* Errors well determined whatever the pulse shape

**Disadvantages**

* Relatively trickier to implement

* Needs good template pulse profile

$^1$Taylor, J. H. Pulsar Timing and Relativistic Gravity. _Philosophical Transactions: Physical Sciences and Engineering_ **341**, 117â€“134 (1992).

```python
def fftfit_fun(profile, template, amplitude, phase):
    '''Objective function to be minimized - \'a la Taylor (1992).'''

    prof_ft = np.fft.fft(profile)
    temp_ft = np.fft.fft(template)
    freq = np.fft.fftfreq(len(profile))
    good = freq > 0
    idx = np.arange(0, prof_ft.size, dtype=int)
    sigma = np.std(prof_ft[good])
    return np.sum(np.absolute(prof_ft - temp_ft*amplitude*np.exp(-2*np.pi*1.0j*idx*phase))**2 / sigma)

def obj_fun(pars, data):
    '''Wrap parameters and input data up in order to be used with minimization
    algorithms.'''
    amplitude, phase = pars
    profile, template = data
    return fftfit_fun(profile, template, amplitude, phase)

# Produce 16 realizations of pr_3, at different amplitudes and phases, and reconstruct the phase
from scipy.optimize import fmin, basinhopping

# PLOTTING --------------------------

fig = plt.figure(figsize=(10, 10))
fig.suptitle('FFTfit results')
gs = gridspec.GridSpec(4, 4)

# -----------------------------------
amp0 = 1
phase0 = 0
p0 = [amp0, phase0]
for i in range(16):
    # PLOTTING --------------------------
    col = i % 4
    row = i // 4
    # -----------------------------------

    factor = 10 ** np.random.uniform(1, 3)
    pr_orig = np.random.poisson(pr_3_clean * factor)
    roll_len = np.random.randint(0, len(pr_orig) - 1)
    pr = np.roll(pr_orig, roll_len)

#    # Using generic minimization algorithms is faster, but local minima can be a problem
#    res = fmin(obj_fun, p0, args=([pr, pr_3_clean],), disp=False, full_output=True)
#    amplitude_res, phase_res = res[0]

    # The basinhopping algorithm is very slow but very effective in finding
    # the global minimum of functions with local minima.
    res = basinhopping(obj_fun, p0, minimizer_kwargs={'args':([pr, pr_3_clean],)})
    amplitude_res, phase_res = res.x

    phase_res -= np.floor(phase_res)
    newphase = ph + phase_res
    newphase -= np.floor(newphase)

    # Sort arguments of phase so that they are ordered in plot
    # (avoids ugly lines traversing the plot)
    order = np.argsort(newphase)

    # PLOTTING --------------------------

    ax = plt.subplot(gs[row, col])
    ax.plot(ph, pr, 'k-')

    ax.plot(newphase[order], amplitude_res * pr_3_clean[order], 'r-')
    # -------------------------------------
```

## The Z_n search

$Z_n^2$ is another widely used statistics for high-energy pulsar searches.
It measures how the probability of photons in a given phase is proportional to a given combination of $n$ harmonics. Or in other words, how well the pulse profile is described by a combination of sinusoidal harmonics.
The definition of this statistical indicator is (Buccheri+1983):

$$
Z^2_n = \dfrac{2}{N} \sum_{k=1}^n \left[{\left(\sum_{j=1}^N \cos k \phi_j\right)}^2 + {\left(\sum_{j=1}^N \sin k \phi_j\right)}^2\right] \; ,
$$

The formula can be slightly modified for binned data, by introducing a `weight` quantity giving the number of photons (or another measure of flux) in a given bin (Huppenkothen+2019):

$$
Z^2_n \approx \dfrac{2}{\sum_j{w_j}} \sum_{k=1}^n \left[{\left(\sum_{j=1}^m w_j \cos k \phi_j\right)}^2 + {\left(\sum_{j=1}^m w_j \sin k \phi_j\right)}^2\right]
$$

```python
def z_n(time, p, n=2, weight=1):
    '''Z^2_n statistics, a` la Buccheri+03, A&A, 128, 245, eq. 2.

    Parameters
    ----------
    phase : array of floats
        The phases of the events
    n : int, default 2
        Number of harmonics, including the fundamental

    Other Parameters
    ----------------
    norm : float or array of floats
        A normalization factor that gets multiplied as a weight.

    Returns
    -------
    z2_n : float
        The Z^2_n statistics of the events.
    '''
    phase = time / p

    nbin = len(phase)

    if nbin == 0:
        return 0

    weight = np.asarray(weight)
    if weight.size == 1:
        total_weight = nbin * weight
    else:
        total_weight = np.sum(weight)
    phase = phase * 2 * np.pi
    return 2 / total_weight * \
        np.sum([np.sum(np.cos(k * phase) * weight) ** 2 +
                np.sum(np.sin(k * phase) * weight) ** 2
                for k in range(1, n + 1)])

trial_periods = np.arange(0.7, 1.0, 0.0002)
stats = np.zeros_like(trial_periods)
for i, p in enumerate(trial_periods):
    stats[i] = z_n(time, p, weight=total)

bestp = trial_periods[np.argmax(stats)]

phase_search, profile_search, profile_search_err = \
    epoch_folding(time, total, bestp)
phase, profile, profile_err = epoch_folding(time, total, period)

# PLOTTING -------------------------------
fig = plt.figure(figsize=(10, 3))
gs = gridspec.GridSpec(1, 2)
ax = plt.subplot(gs[0])
ax.plot(trial_periods, stats)
ax.set_xlim([0.7, 1])
ax.set_xlabel('Period (s)')
ax.set_ylabel('$\chi^2$')
ax.axvline(period, color='r', label="True value")
_ = ax.legend()
ax.annotate('max = {:.5f} s'.format(pmax), xy=(.9, max(stats) / 2))

ax2 = plt.subplot(gs[1])

ax2.errorbar(phase_search, profile_search, yerr=profile_search_err,
             drawstyle='steps-mid', label='Search')
ax2.errorbar(phase, profile, yerr=profile_err, drawstyle='steps-mid',
             label='True period')
ax2.set_xlabel('Phase')
ax2.set_ylabel('Counts/bin')

_ = ax2.legend()
# ------------------------------------------
```

## Pulsation searches with HENDRICS

1. To read a fits file into an event list file:

```
$ HENreadevents file.evt.gz
```
a file called something like `file_mission_instr_ev.nc` appears

2. To calculate the light curve (binning the events) with a sample time of 1 s:

```
$ HENlcurve file_mission_instr_ev.nc -b 1
```

3. To calculate the averaged power density spectrum cutting the data by chunks of 128 s:

```
$ HENfspec file_mission_instr_lc.nc -f 128
```

4. To watch the power density spectrum:

```
$ HENplot file_mission_instr_pds.nc
```

5. To run a $Z^2_4$ search, e.g.  between frequencies 0.5 and 0.6:

```
$ HENzsearch file_mission_instr_ev.nc -f 0.5 -F 0.6 -N 4
```

6. To run a $Z^2_2$ search searching in the frequency -- fdot space

```
$ HENzsearch file_mission_instr_ev.nc -f 0.5 -F 0.6 -N 2 --fast
$ HENplot file_mission_instr_Z2n.nc
```

7. Then... follow the instructions...

### BONUS

8. Calculate the TOAs and create a parameter and timing file (can you find how?)

9. Use `pintk` (from `github.com/nanograv/PINT`) to fit the pulse solution

```
$ pintk parfile.par timfile.tim
```

NB: due to a bug to PINT (under investigation), you might need to add the line
```
TZRMJD    55555
```

Substitute 55555 with the value of PEPOCH in the parameter file.

```python

```