```python
%pylab inline
pylab.rcParams['figure.figsize'] = (16.0, 8.0)
```

    Populating the interactive namespace from numpy and matplotlib

# Drawing from multivariate distributions

## Draws from the multivariate normal distribution

Draws from a multivariate normal distribution
$$ N(\mathbf{\mu}, \mathbf{\Sigma}) $$
can be generated by

1) Calculate the Cholesky decomposition $\mathbf{\Sigma} = \mathbf{R}^T\mathbf{R}$
``` python
    from numpy.linalg import cholesky
    R = cholesky(Sigma)
```

2) Generate standard normally distributed values $\mathbf{Z}$

3) Evaluate $$ \mathbf{X} = \mathbf{\mu 1} + \mathbf{RZ}$$

### Exercise 2.1

Draw 1000 samples from the bivariate distribution
$$ N\left(\left( \begin{array}{c}
0.2 \\ -1.0
\end{array}\right),
\left(\begin{array}{cc}
0.01 & -0.014 \\ -0.014 & 0.04
\end{array}\right)
\right)
$$

```python
draws = 1000

mu = array([0.2, -1.0])
Sigma = array([[0.01, -0.014],[-0.014, 0.04]])
R = linalg.cholesky(Sigma)
Z = random.randn(2,draws)

X = dot(mu[:,newaxis], ones((1, draws))) + dot(R, Z)
```

```python
figure()
scatter(X[0,:], X[1,:], 100, edgecolor="none");
```

Draws from the multivariate normal distribution can more easily generated using the built-in **scipy** functions

```python
    from scipy.stats import multivariate_normal
    dist = multivariate_normal(mu, Sigma)
    X = dist.rvs(size)
```
**Note**
Scipy *rvs* functions return arrays of shape (number of draws, size of mean)

### Exercise 2.2

Repeat Exercise 2.1 with the built-in scipy function and compare the results.

```python
from scipy.stats import multivariate_normal
mu = array([0.2, -1.0])
Sigma = array([[0.01, -0.014],[-0.014, 0.04]])
dist = multivariate_normal(mu, Sigma)
X = dist.rvs(1000)

figure()
scatter(X[:,0], X[:,1], 100, edgecolor="none");
```

## Draws using a copula function

In many practical cases knowledge about the input quantities is available in terms of their individual distributions and a correlation coefficient. This is insufficient to assign a unique multivariate distribution. Therefore, a copula function can be defined
$$
C(\mu_1,\ldots,\mu_N) = \mathbb{P} \left[ X_1\leq G_{X_1}^{-1}(\mu_1)\ldots,X_N\leq G_{X_N}^{-1}(\mu_N) \right]
$$

### Example copula functions

* all input quantities are mutually independent
$$ C(\mu_1,\ldots,\mu_N) = \prod_{k=1}^N \mu_k $$

* the input quantities are correlated with $\rho\equiv 1$
$$ C(\mu_1,\ldots,\mu_N) = \min_{k} \mu_k $$

* two input quantities are correlated with $\rho$
$$ C(\mu_1,\mu_2) = F_2(G_{X_1}^{-1}(\mu_1),G_{X_2}^{-1}(\mu_2),\rho) $$

The copula can be used to incorporate the correlation coefficient and the individual distributions $g_{X_i}$ to formally define a multivariate distribution.

#### Example
Input quantities $X_1,X_2$ with correlation coefficient $\rho$ and
\begin{align}
X_1 \sim & N(\mu, \sigma) \\
X_2 \sim & U(a, b)
\end{align}

Use bivariate normal copula function:

1) Draw from bivariate standard normal distribution
$$ z \sim N\left(\mathbf{0}, \left(\begin{array}{cc}
1.0 & \rho \\ \rho & 1.0
\end{array}\right) \right)
$$

2) Evaluate cumulative distribution function of the copula
\begin{align}
\zeta_1 =& G_N(z_1) \\
\zeta_2 =& G_N(z_2)
\end{align}

3) Evaluate inverse cumulative distribution functions
\begin{align}
x_1 =& G_{X_1}^{-1}(\zeta_1) \\
x_2 =& G_{X_2}^{-1}(\zeta_2)
\end{align}

### Exercise 2.3

Consider the input quantities $X_1,X_2$ with
* $X_1$ has best estimate 0.2 with uncertainty of 50%
* $X_2$ has best estimate -1.0 with uncertainty of 20%
* correlation between $X_1$ and $X_2$ is $\rho=-0.7$

Generate 1000 random draws using a bivariate normal copula function.

```python
from scipy.stats import norm
rho = -0.7
draws = 1000

mu = array([0, 0])
Sigma = array([[1, rho],[rho,1]])
dist = multivariate_normal(mu, Sigma)
Z = dist.rvs(1000)

zeta1 = norm.cdf(Z[:,0])
zeta2 = norm.cdf(Z[:,1])

X1 = norm.ppf(zeta1, loc=0.2, scale=0.1)
X2 = norm.ppf(zeta2, loc=-1.0, scale=0.2)

figure()
scatter(X1, X2, 100, edgecolor="none");
```

### Exercise 2.4

Consider the input quantities $X_1, X_2$ with

* $X_1$ has best estimate $x_1=2.4$ with expanded uncertainty $U=0.4 (k=2)$ under normality assumption

* $X_2$ is located in $[-1.5, 1.5]$

* $X_1, X_2$ are correlated with $\rho = 0.4$

Draw 1000 samples from their joint probability distribution using a normal distribution copula function.

```python
from scipy.stats import norm, uniform, multivariate_normal
rho = 0.7
draws = 10000

copula = multivariate_normal(zeros(2), array([[1.0, rho],[rho, 1.0]]))

z = copula.rvs(draws)

xi1 = norm.cdf(z[:,0])
xi2 = norm.cdf(z[:,1])

X1dist = norm(loc=2.4, scale=0.2)
X2dist = uniform(loc=-1.5, scale=3)
x1 = X1dist.ppf(xi1)
x2 = X2dist.ppf(xi2)

figure(1)
scatter(x1,x2,100,edgecolor="none")
```

```python
figure()
subplot(121)
hist(x1, bins=100, edgecolor="none")
subplot(122)
hist(x2, bins=100, edgecolor="none");
```

```python

```

03 Drawing from multivariate distributions.ipynb


# Part 1 - Scalars and Vectors

For the questions below it is not sufficient to simply provide answer to the questions, but you must solve the problems and show your work using python (the NumPy library will help a lot!) Translate the vectors and matrices into their appropriate python  representations and use numpy or functions that you write yourself to demonstrate the result or property.

## 1.1 Create a two-dimensional vector and plot it on a graph

```
import math
import matplotlib.pyplot as plt
import numpy as np
```

```
# 2 dimensional vector
red = [.2, .3]

plt.arrow(0, 0, .2, .3, head_width=.09, head_length=0.01, color= "r")
plt.title("2D Vector")
plt.show()
```

## 1.2 Create a three-dimensional vecor and plot it on a graph

```
from mpl_toolkits.mplot3d import Axes3D
red = [.1, .2, .3]
blue = [.3, .2, .1]
green = [.2, .1, .2]

threeD = np.array([[0, 0, 0, .1, .2, .3],
                  [0, 0, 0, .3, .2, .1],
                    [0, 0, 0, .2, .1, .1]])

X, Y, Z, U, V, W = zip(*threeD)
fig = plt.figure()
ax = fig.add_subplot(111, projection = "3d")
ax.quiver(X, Y, Z, U, V, W, length=1)
ax.set_xlim([0, 1])
ax.set_ylim([0, 1])
ax.set_zlim([0, 1])
plt.show()
```

## 1.3 Scale the vectors you created in 1.1 by $5$, $\pi$, and $-e$ and plot all four vectors (original + 3 scaled vectors) on a graph. What do you notice about these vectors?

```
from math import e, pi
print(-e)
print(pi)
```

    -2.718281828459045
    3.141592653589793

```
red = [.2, .3]

#Scalar Multiplication

blue = np.multiply(5, red)
yellow = np.multiply(pi, red)
green = np.multiply(-e, red)

#Plot Scaled Vectors
plt.arrow(0,0, red[0], red[1],head_width=.09, head_length=0.09, color ='red')
plt.arrow(0,0, green[0], green[1],head_width=.05, head_length=0.05, color ='green')
plt.arrow(0,0, blue[0], blue[1],head_width=.05, head_length=0.05, color ='blue')
plt.arrow(0,0, yellow[0], yellow[1],head_width=.05, head_length=0.05, color ='yellow')
plt.xlim(-2,2)
plt.ylim(-2,2)
plt.title("Scaled Vectors")
plt.show()
```

## 1.4 Graph vectors $\vec{a}$ and $\vec{b}$ and plot them on a graph

\begin{align}
\vec{a} = \begin{bmatrix} 5 \\ 7 \end{bmatrix}
\qquad
\vec{b} = \begin{bmatrix} 3 \\4 \end{bmatrix}
\end{align}

```
a = [5, 7]
b = [3, 4]

plt.arrow(0, 0, 5, 7, head_width = .5, head_length = 0.3, color = "r")
plt.arrow(0, 0, 3, 4, head_width = .5, head_length = 0.3, color = "b")
plt.xlim(0,10)
plt.ylim(0,10)
plt.show()
```

## 1.5 find $\vec{a} - \vec{b}$ and plot the result on the same graph as $\vec{a}$ and $\vec{b}$. Is there a relationship between vectors $\vec{a} \thinspace, \vec{b} \thinspace \text{and} \thinspace \vec{a-b}$

```
c = np.subtract(a,b)

plt.arrow(0, 0, 5, 7, head_width = .5, head_length = 0.3, color = "r")
plt.arrow(0, 0, 3, 4, head_width = .5, head_length = 0.3, color = "b")
plt.arrow(0, 0, 2, 3, head_width = .5, head_length = 0.3, color = "g")

plt.xlim(0,10)
plt.ylim(0,10)
plt.show()

#The length of the vector is shorter
```

## 1.6 Find $c \cdot d$

\begin{align}
\vec{c} = \begin{bmatrix}7 & 22 & 4 & 16\end{bmatrix}
\qquad
\vec{d} = \begin{bmatrix}12 & 6 & 2 & 9\end{bmatrix}
\end{align}

```
c = np.array([7,22,4,16])
d = np.array([12,6,2,9])

np.dot(c, d)
```

    368

##  1.7 Find $e \times f$

\begin{align}
\vec{e} = \begin{bmatrix} 5 \\ 7 \\ 2 \end{bmatrix}
\qquad
\vec{f} = \begin{bmatrix} 3 \\4 \\ 6 \end{bmatrix}
\end{align}

```
e = np.array([5,7,2])
f = np.array([3,4,6])

e = e.T
f = f.T

np.cross(e,f)

```

    array([ 34, -24,  -1])

## 1.8 Find $||g||$ and then find $||h||$. Which is longer?

\begin{align}
\vec{g} = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 8 \end{bmatrix}
\qquad
\vec{h} = \begin{bmatrix} 3 \\3 \\ 3 \\ 3 \end{bmatrix}
\end{align}

```
from numpy import linalg as LA

g = np.array[1,1,1,8]
h = np.array[3,3,3,3]

LA.norm(g)
LA.norm(h)
```

# Part 2 - Matrices

## 2.1 What are the dimensions of the following matrices? Which of the following can be multiplied together? See if you can find all of the different legal combinations.
\begin{align}
A = \begin{bmatrix}
1 & 2 \\
3 & 4 \\
5 & 6
\end{bmatrix}
\qquad
B = \begin{bmatrix}
2 & 4 & 6 \\
\end{bmatrix}
\qquad
C = \begin{bmatrix}
9 & 6 & 3 \\
4 & 7 & 11
\end{bmatrix}
\qquad
D = \begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\qquad
E = \begin{bmatrix}
1 & 3 \\
5 & 7
\end{bmatrix}
\end{align}

```
A = np.mat("1 2; 3 4; 4 5")
print("A ", A.shape)
B = np.array([2,4,6])
print("B ",B.shape)
C = np.mat("9 6 3; 4 7 11")
print("C ", C.shape)
D = np.mat("1 0 0; 0 1 0; 0 0 1")
print("D " ,D.shape)
E = np.mat('1 3; 5 7')
print("E", E.shape)
```

    A  (3, 2)
    B  (3,)
    C  (2, 3)
    D  (3, 3)
    E (2, 2)

## 2.2 Find the following products: CD, AE, and BA. What are the dimensions of the resulting matrices? How does that relate to the dimensions of their factor matrices?

```
CD = np.matmul(C,D)
print("CD ", CD.shape)
AE = np.matmul(A,E)
print("AE ", AE.shape)
BA = np.matmul(B,A)
print("BA ", BA.shape)
```

    CD  (2, 3)
    AE  (3, 2)
    BA  (1, 2)

## 2.3  Find $F^{T}$. How are the numbers along the main diagonal (top left to bottom right) of the original matrix and its transpose related? What are the dimensions of $F$? What are the dimensions of $F^{T}$?

\begin{align}
F =
\begin{bmatrix}
20 & 19 & 18 & 17 \\
16 & 15 & 14 & 13 \\
12 & 11 & 10 & 9 \\
8 & 7 & 6 & 5 \\
4 & 3 & 2 & 1
\end{bmatrix}
\end{align}

```
F = np.mat("20 19 18 17; 16 15 14 13; 12 11 10 9; 8 7 6 5; 4 3 2 1")
print("F ", F.shape)
Ft = F.T
print("Ft ", Ft.shape)

print("The diagonal of the og matrix is the same as the transpose")
```

    F  (5, 4)
    Ft  (4, 5)
    The diagonal of the og matrix is the same as the transpose

# Part 3 - Square Matrices

## 3.1 Find $IG$ (be sure to show your work) 😃

\begin{align}
G=
\begin{bmatrix}
12 & 11 \\
7 & 10
\end{bmatrix}
\end{align}

```
G = np.mat("12 11; 7 10")
I = np.identity(2)
IG = np.matmul(I, G)
print(IG)
```

    [[12. 11.]
     [ 7. 10.]]

## 3.2 Find $|H|$ and then find $|J|$.

\begin{align}
H=
\begin{bmatrix}
12 & 11 \\
7 & 10
\end{bmatrix}
\qquad
J=
\begin{bmatrix}
0 & 1 & 2 \\
7 & 10 & 4 \\
3 & 2 & 0
\end{bmatrix}
\end{align}

```
H = np.mat("12 11; 7 10")
J = np.mat("0 1 2; 7 10 4; 3 2 0")

print(np.linalg.det(H))
print(np.linalg.det(J))

```

    43.000000000000014
    -19.999999999999996

## 3.3 Find $H^{-1}$ and then find $J^{-1}$

```
print(np.linalg.inv(H))
print(np.linalg.inv(J))
```

    [[ 0.23255814 -0.25581395]
     [-0.1627907   0.27906977]]
    [[ 0.4  -0.2   0.8 ]
     [-0.6   0.3  -0.7 ]
     [ 0.8  -0.15  0.35]]

## 3.4 Find $HH^{-1}$ and then find $J^{-1}J$. Is $HH^{-1} == J^{-1}J$? Why or Why not?

```
H2 = np.multiply(H, np.linalg.inv(H))
print(H2)

J2 = np.multiply(np.linalg.inv(J), J)
print(J2)
```

    [[ 2.79069767 -2.81395349]
     [-1.13953488  2.79069767]]
    [[ 0.  -0.2  1.6]
     [-4.2  3.  -2.8]
     [ 2.4 -0.3  0. ]]

No, they are completely different matrices

# Stretch Goals:

A reminder that these challenges are optional. If you finish your work quickly we welcome you to work on them. If there are other activities that you feel like will help your understanding of the above topics more, feel free to work on that. Topics from the Stretch Goals sections will never end up on Sprint Challenges. You don't have to do these in order, you don't have to do all of them.

- Write a function that can calculate the dot product of any two vectors of equal length that are passed to it.
- Write a function that can calculate the norm of any vector
- Prove to yourself again that the vectors in 1.9 are orthogonal by graphing them.
- Research how to plot a 3d graph with animations so that you can make the graph rotate (this will be easier in a local notebook than in google colab)
- Create and plot a matrix on a 2d graph.
- Create and plot a matrix on a 3d graph.
- Plot two vectors that are not collinear on a 2d graph. Calculate the determinant of the 2x2 matrix that these vectors form. How does this determinant relate to the graphical interpretation of the vectors?


```python
 from sympy import *
init_printing()
```

```python
x = Symbol('x')
EI = Symbol('EI')
R = Symbol('R')
omega = Symbol('omega')
ll = Symbol('lambda')
```

```python
gamma_1 = x
```

```python
gamma_2 = 10*x**3/3 - 10*x**4/3 + x**5
```

```python
A11 = integrate(gamma_1*gamma_1, (x, 0, 1))
```

```python
A12 = integrate(gamma_1*gamma_2, (x, 0, 1))
```

```python
A12
```

```python
A22 = integrate(gamma_2*gamma_2, (x, 0, 1))
```

```python
G = integrate(omega**2*x, (x,x*R, R))
```

```python
G
```

```python
def calcB(gamma_1, gamma_2):
    der1_2 = diff(gamma_1,x,2)
    der1_1 = diff(gamma_1,x,1)
    der2_1 = diff(gamma_2,x,1)
    integrando = EI*der1_2*der2_2+R**2*G*der1_1*der2_1
    sol = integrate(integrando,(x, 0, 1))/(2*R)/(omega**2*R**3)
    return(sol*2)
```

```python
B11 = calcB(gamma_1, gamma_1)
```

```python
B12 = calcB(gamma_1, gamma_2)
```

```python
B22 = calcB(gamma_2, gamma_2).simplify()
```

```python
k = Symbol('k')
```

```python
B22 = B22.subs(EI,k*R**4*omega**2)
```

```python
B22
```

```python
LHS =  Matrix([[B11, B12], [B12, B22]])- ll**2*Matrix([[A11, A12], [A12, A22]])
```

```python
LHS
```

```python
determinante = det(LHS).subs(k,0.004).simplify()
```

```python
factor(determinante)
```

```python
soluciones = roots(determinante)
```

```python
sol = list(soluciones.keys());
```

```python
sol
```

```python
phi_1 = Symbol('phi_1')
phi_2 = Symbol('phi_2')
```

```python
Sistema = LHS.subs(k,0.004).subs(ll, sol[2])
Sistema = Sistema.col_insert(2, Matrix([0, 0]))
solve_linear_system(Sistema, phi_1, phi_2)
Sistema
```

```python
Modo1 = gamma_1
```

```python
Sistema = LHS.subs(k,0.004).subs(ll, sol[3])
Sistema = Sistema.col_insert(2, Matrix([0, 0]))
solve_linear_system(Sistema, phi_1, phi_2)
```

```python
Modo2 = -0.76*gamma_1+gamma_2
Modo2 = Modo2/Modo2.subs(x,1.0)
```

```python
Modo2
```

```python
p1 = plot(Modo2,Modo1,(x, 0, 1))
```

```python

```


# What is Regression Dilution?

The motivating example to explain regression dilution is on the attempt to do simple linear regression assuming a linear model between two variables $x^*$ and $y^*$ with residual as the intrinsic scatter $\varepsilon$. However the data collected from these two variables $(x, y)$ include error which we classically assum it's Gaussian noise:

$
\begin{align}
    x_i &= x_i^* + \epsilon_{x,i}\\
    y_i &= y_i^* + \epsilon_{y,i}
\end{align}
$

## Exact Solution without Measurement Error — Ordinary Least Squares

Without error the model has simple solutions to estimate its slope $\hat\theta_1$ where this particular estimator minimize the sum of squared y-residuals $\left(\sum_{i=1}^N{(y^* - \theta_1 x^* - \theta_0)^2}\right)$ (the usual objective of ordinary least squares):

$
\begin{align}
    y_i^* &= \theta_1 x_i^* + \theta_0 + \varepsilon & \forall i \in \{1,2,\ldots,N\}\\
    \hat\theta_1 &= \frac{\text{Cov}(x^*, y^*)}{\text{Var}(x^*)}
\end{align}
$

In fact, the RHS can be expressed by the correlation $r_{xy}$ by its definition:

$
\begin{align}
    \hat\theta_1 &= r_{xy}\frac{\text{SD}(y^*)}{\text{SD}(x^*)}\\
    r_{xy} &= \hat\theta_1\frac{\text{SD}(x^*)}{\text{SD}(y^*)}
\end{align}
$

Here we see indeed for OLS on data without measurement error, solving for slope equivalently solves for correlation and vice versa. These are dual problems.

## With Measurement Error, Ordinary Least Squares is Biased

With measurement error (assume $\varepsilon$ has scatter $\sigma$, $\epsilon_x$ has scatter $\sigma_x$, and $\epsilon_y$ has scatter $\sigma_y$ for all data points for simplicity) and if we apply the same procedure as OLS,

$
\begin{align}
    \hat\theta_1 &= \frac{\text{Cov}(x, y)}{\text{Var}(x)} \\
    &= \frac{\text{Cov}(x^* + \epsilon_x, y + \epsilon_y)}{\text{Var}(x^* + \epsilon_x)}\\
    &= \frac{\text{Cov}(x^*, y^*) + \sigma_{xy}}{\text{Var}(x^*) + \sigma_x^2}\\
    &= \frac{\text{Cov}(x^*, y^*)}{\text{Var}(x^*) + \sigma_x^2}
\end{align}
$

Where $\sigma_{xy}$ is the covariance between $\epsilon_x$ and $\epsilon_y$. Let's take the case where $\epsilon_x$ and $\epsilon_y$ are independent therefore the last line above, we set $\sigma_{xy}=0$.

We can see from the denominator the extra additive term of $\sigma_x$. This defines the regression dilution in OLS where the scatter in x-error causes underestimation of the slope. The same effect applies to correlation as we can see that,

$
\begin{align}
    r_{xy} &\equiv \frac{\text{Cov}(x^* + \epsilon_x, y^* + \epsilon_y)}{\sqrt{\text{Var}(x^* + \epsilon_x)\text{Var}(y^* + \epsilon_y)}} \\
           &= \frac{\text{Cov}(x^*, y^*) + \sigma_{xy}}{\sqrt{\big[\text{Var}(x^*) + \sigma_x^2\big] \big[\text{Var}(y^*) + \sigma_y^2\big]}} \\
           &= \frac{\text{Cov}(x^*, y^*)}{\sqrt{\big[\text{Var}(x^*) + \sigma_x^2\big] \big[\text{Var}(y^*) + \sigma_y^2\big]}} \\
           &= \frac{\hat\theta_1}{\text{Var}(y^*) + \sigma_y^2}
\end{align}
$

Correlation has the additional affect that not only x-error causes underestimation but also y-error.

## Correcting Regression Dilution in Correlation

There are many methods to correct for regresson dilution. One method used here, relies on the reliability metric $r_{xx}$ and $r_{yy}$ defined as

$
\begin{align}
    r_{zz} = \frac{\text{Var}(z) - \text{Var}(\epsilon_z)}{\text{Var}(z)}
\end{align}
$

The correction is then,

$
\begin{align}
    r_{xy}' = \frac{r_{xy}}{\sqrt{r_{xx}r_{yy}}}
\end{align}
$

# Simulating Regression Dilution

```python
import numpy as np
import pandas as pd
from astropy.io import ascii
from scipy import stats
from scipy.optimize import curve_fit, minimize

import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns

np.random.seed(91320)
sns.set(style='ticks', context='talk')
plt.style.use("paper.mplstyle")
```

## Population Definition

### Bivariate Normal

```python
size = 1000000
rho_xy = 0.25
sigma_x = 1
sigma_y = 1
cov_xy = rho_xy * (sigma_x * sigma_y)
cov = np.array([[sigma_x**2,    cov_xy],
                [ cov_xy,   sigma_y**2]])
m = rho_xy * sigma_y / sigma_x
b = 0

population_dist = stats.multivariate_normal(cov=cov)
population_sample = population_dist.rvs(size)
x, y = population_sample[:, 0], population_sample[:, 1]
r_xy = stats.pearsonr(x, y)[0]
m_best =  r_xy * np.std(y) / np.std(x)
b_best =  np.mean(y) - m_best*np.mean(x)

print("Population Correlation:", r_xy)
```

    Population Correlation: 0.24974542732423816

```python
def f(x, m, b):
    return m*x + b

fig = plt.figure(figsize=(10, 10))
plt.hist2d(x, y, bins=100, range=[[-4, 4], [-4, 4]], cmap="gray_r");

xrange = np.linspace(-6, 6, 100)
plt.plot(xrange, f(xrange, m_best, b_best), 'r--', label="Truth")

popt, _ = curve_fit(f, x, y)
plt.plot(xrange, f(xrange, *popt), label="OLS", zorder=1)

fig.suptitle("Population Sample r=0.25")
plt.xlabel("x-value")
plt.ylabel("y-value")
plt.legend()
plt.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.savefig("figures/bivariate.png")
```

## Measurement Sample

Each measurement sample will be measured with error of constant variance.

* The first sample will have equal variance on x and y error.
* The second sample will have x error be 5 times greater than y error.
* The third will have x error be 10 times greater than y error.

```python
data_size = 100
data_idx = np.random.choice(range(size), size=data_size, replace=False)
xdata_true = x[data_idx]
ydata_true = y[data_idx]

sigma_x = np.ones(data_size) * 0.2
sigma_y = np.ones(data_size) * 0.2
xerr = stats.norm(loc=0, scale=sigma_x).rvs(data_size)
yerr = stats.norm(loc=0, scale=sigma_y).rvs(data_size)
```

### Measurement Sample 1

Both x and y has single value errors with error ratio of 1.

```python
xdata = xdata_true + xerr*1
ydata = ydata_true + yerr*1

df1 = pd.DataFrame(
    {
        "x": xdata,
        "xerr": sigma_x,
        "y": ydata,
        "yerr": sigma_y,
    }
).sort_values('x')

plt.errorbar(df1.x, df1.y, yerr=df1.yerr, xerr=df1.xerr, c='k', fmt='ko', lw=0.5, ms=5)
df1.head()
```

### Measurement Sample 2

Both x and y has single value errors with error ratio x of y being 5

```python
xdata = xdata_true + xerr*5
ydata = ydata_true + yerr*1

df2 = pd.DataFrame(
    {
        "x": xdata,
        "xerr": sigma_x*5,
        "y": ydata,
        "yerr": sigma_y,
    }
).sort_values('x')

plt.errorbar(df2.x, df2.y, yerr=df2.yerr, xerr=df2.xerr, c='k', fmt='ko', lw=0.5, ms=5)
df2.head()
```

### Measurement Sample 3

Both x and y has errors of variance uniformly assigned. The error ratio is about 5

```python
xdata = xdata_true + xerr*10
ydata = ydata_true + yerr*1

df3 = pd.DataFrame(
    {
        "x": xdata,
        "xerr": sigma_x*10,
        "y": ydata,
        "yerr": sigma_y,
    }
).sort_values('x')

plt.errorbar(df3.x, df3.y, yerr=df3.yerr, xerr=df3.xerr, c='k', fmt='ko', lw=0.5, ms=5)
df3.head()
```

## Regression Dilution in OLS

```python
def get_slope_parameter(x, y):
    """Return the mean and variance of the OLS slope parameter"""
    corr = stats.pearsonr(x, y)[0]
    slope = corr * (np.std(y) / np.std(x))

    sigma2 = (np.var(y)**2 + np.var(x)**2)
    slope_var = sigma2 / np.sum((x-np.mean(x))**2)

    return slope, slope_var

def get_intercept_parameter(x, y, slope):
    """Return the mean of the OLS intercept parameter"""
    corr = stats.pearsonr(x, y)[0]
    intercept = np.mean(y) - slope*np.mean(x)

    return intercept

slope, slope_var = get_slope_parameter(df1.x, df1.y)
intercept = get_intercept_parameter(df1.x, df1.y, slope)
xrange = np.linspace(-4, 4, 100)
plt.figure()
plt.errorbar(df1.x, df1.y, yerr=df1.yerr, xerr=df1.xerr, c='k', fmt='ko', lw=0.5, ms=5)
plt.plot(xrange, f(xrange, slope, intercept), label="OLS")
plt.plot(xrange, f(xrange, m, b), c='g', label="Truth")

plt.legend()

slope, slope_var = get_slope_parameter(df2.x, df2.y)
intercept = get_intercept_parameter(df2.x, df2.y, slope)
xrange = np.linspace(-4, 4, 100)
plt.figure()
plt.errorbar(df2.x, df2.y, yerr=df2.yerr, xerr=df2.xerr, c='k', fmt='ko', lw=0.5, ms=5)
plt.plot(xrange, f(xrange, slope, intercept), label="OLS")
plt.plot(xrange, f(xrange, m, b), c='g', label="Truth")
plt.legend()

slope, slope_var = get_slope_parameter(df3.x, df3.y)
intercept = get_intercept_parameter(df3.x, df3.y, slope)
xrange = np.linspace(-4, 4, 100)
plt.figure()
plt.errorbar(df3.x, df3.y, yerr=df3.yerr, xerr=df3.xerr, c='k', fmt='ko', lw=0.5, ms=5)
plt.plot(xrange, f(xrange, slope, intercept), label="OLS")
plt.plot(xrange, f(xrange, m, b), c='g', label="Truth")
plt.legend()
```

## Regression Dilution in Simulation

```python
def simulate(x, y, xerr, yerr, repetition=1000):

    corrs = []
    slopes = []
    intercepts = []
    for i in range(repetition):
        xdist = stats.norm(loc=x, scale=xerr)
        ydist = stats.norm(loc=y, scale=yerr)

        xsample = xdist.rvs()
        ysample = ydist.rvs()

        corr = stats.pearsonr(xsample, ysample)[0]
        corrs.append(corr)

        slope = corr * (np.std(ysample) / np.std(xsample))
        slopes.append(slope)

        intercept = np.mean(ysample) - slope*np.mean(xsample)
        intercepts.append(intercept)

    return pd.DataFrame({"corr": corrs, "slope": slopes, "intercept": intercept})

result_df1 = simulate(df1.x, df1.y, df1.xerr, df1.yerr)
result_df2 = simulate(df2.x, df2.y, df2.xerr, df2.yerr)
result_df3 = simulate(df3.x, df3.y, df3.xerr, df3.yerr)
```

```python
# def simulate(x, y, xerr, yerr, sample_size=10000):

#     xsample = []
#     ysample = []
#     for i in range(len(x)):
#         xdist = stats.norm(loc=x[i], scale=xerr[i])
#         ydist = stats.norm(loc=y[i], scale=yerr[i])

#         xsample.extend(xdist.rvs(size=sample_size))
#         ysample.extend(ydist.rvs(size=sample_size))

#     df = pd.DataFrame({
#         "x": xsample,
#         "y": ysample
#     })

#     return df

# s1_df = simulate(df1.x, df1.y, df1.xerr, df1.yerr)
# s2_df = simulate(df2.x, df2.y, df2.xerr, df2.yerr)
# s3_df = simulate(df3.x, df3.y, df3.xerr, df3.yerr)

# small_s1_df = simulate(df1.x, df1.y, df1.xerr, df1.yerr, sample_size=50)
# small_s2_df = simulate(df2.x, df2.y, df2.xerr, df2.yerr, sample_size=50)
# small_s3_df = simulate(df3.x, df3.y, df3.xerr, df3.yerr, sample_size=50)
```

## Regression Dilution in LINMIX

```python
from linmix import linmix

lm1 = linmix.LinMix(df1.x, df1.y, df1.xerr, df1.yerr, K=3)
lm1.run_mcmc(silent=True, miniter=1000, maxiter=1000)

lm2 = linmix.LinMix(df2.x, df2.y, df2.xerr, df2.yerr, K=3)
lm2.run_mcmc(silent=True, miniter=1000, maxiter=1000)

lm3 = linmix.LinMix(df3.x, df3.y, df3.xerr, df3.yerr, K=3)
lm3.run_mcmc(silent=True, miniter=1000, maxiter=1000)
```

```python
def rmse(y, ypred):
    return np.sqrt(np.mean((y - ypred)**2))

def slope_to_corr(x, y, slope):
    """ THIS IS WRONG """
    return slope * (np.var(x) / np.var(y))

def corr_to_slope(corr, x, xerr, y, yerr):
    return corr * np.std(y) / np.std(x)

def get_reliability(x, xerr):
    return (np.var(x) - np.mean(xerr**2)) / np.var(x)

def summary_plot(df, result_df, lm, title=""):
    fig, ax1 = plt.subplots(figsize=(10, 10))
    fig.suptitle(title)
    summary = []

    # Truth
    xrange = np.linspace(-6, 6, 100)
    ax1.plot(xrange, f(xrange, m, b), 'k--', label="Truth", zorder=100)

    # Ordinary Least Squares
    slope, slope_var = get_slope_parameter(df.x, df.y)
    corr = stats.pearsonr(df.x, df.y)[0]
    intercept = get_intercept_parameter(df.x, df.y, slope)

    ax1.errorbar(df.x, df.y, yerr=df.yerr, xerr=df.xerr, fmt='ko', lw=0.5, ms=5)
    ax1.plot(xrange, f(xrange, slope, intercept), 'b', label="OLS")
    ypred = slope*df.x + intercept
#     ax2.plot(df.x, df.y - ypred, marker='.', lw=2)

    summary.append({
        "method": "OLS",
        "rmse": rmse(df.y, ypred),
        "trmse": rmse(y, slope*x+intercept),
        "slope": slope,
        "corr": corr,
        "corr_residual": rho_xy - corr
    })

    rxx = get_reliability(df.x, df.xerr)
    ryy = get_reliability(df.y, df.yerr)
    corr = corr / np.sqrt(rxx*ryy)
    slope = corr_to_slope(corr, df.x, df.xerr, df.y, df.yerr)

    ax1.plot(xrange, f(xrange, slope, intercept), 'b-.', label="OLS (corrected)")
    summary.append({
        "method": "OLS (corrected)",
        "rmse": rmse(df.y, ypred),
        "trmse": rmse(y, slope*x+intercept),
        "slope": slope,
        "corr": corr,
        "corr_residual": rho_xy - corr
    })

    # Bootstrap
    slope = result_df["slope"].mean()
    corr = result_df["corr"].mean()
    intercept = result_df["intercept"].mean()
    ax1.plot(xrange, f(xrange, slope, intercept), 'r', label="Bootstrap")
    ypred = slope*df.x + intercept
#     ax2.plot(df.x, df.y - ypred, marker='.', lw=2)

    summary.append({
        "method": "Bootstrap",
        "rmse": rmse(df.y, ypred),
        "trmse": rmse(y, slope*x+intercept),
        "slope": slope,
        "corr": corr,
        "corr_residual": rho_xy - corr
    })

    rxx = get_reliability(df.x, df.xerr)
    ryy = get_reliability(df.y, df.yerr)
    corr = corr / np.sqrt(rxx*ryy)
    slope = corr_to_slope(corr, df.x, df.xerr, df.y, df.yerr)

    ax1.plot(xrange, f(xrange, slope, intercept), 'r-.', label="Bootstrap (corrected)")
    summary.append({
        "method": "Bootstrap (corrected)",
        "rmse": rmse(df.y, ypred),
        "trmse": rmse(y, slope*x+intercept),
        "slope": slope,
        "corr": corr,
        "corr_residual": rho_xy - corr
    })

    # LINMIX
    slope, intercept = np.mean(lm.chain['beta']), np.mean(lm.chain['alpha'])
    corr = slope_to_corr(df.x, df.y, slope)
    ax1.plot(xrange, f(xrange, slope, intercept), 'g', label="LINMIX")
    ypred = slope*df.x + intercept
#     ax2.plot(df.x, df.y - ypred, marker='.', lw=2)

    summary.append({
        "method": "LINMIX",
        "rmse": rmse(df.y, ypred),
        "trmse": rmse(y, slope*x+intercept),
        "slope": slope,
        "corr": corr,
        "corr_residual": rho_xy - corr
    })

    ax1.legend(loc="upper left")
    ax1.set_xlim(-4, 4)
    ax1.set_ylim(-4, 4)
    ax1.set_ylabel("y-value")
    ax1.set_xlabel("x-value")
#     ax2.set_ylabel("Residual")
#     ax2.xaxis.tick_top()

    summary_df = pd.DataFrame(summary)
    summary_df["rmse rank"] = stats.rankdata(summary_df.rmse, method="ordinal")
    summary_df["trmse rank"] = stats.rankdata(summary_df.trmse, method="ordinal")
    summary_df["corr rank"] = stats.rankdata(np.abs(summary_df["corr_residual"]), method="ordinal")
    summary_df["slope rank"] = stats.rankdata(np.abs(summary_df["slope"] - m), method="ordinal")
    summary_df= summary_df.set_index("method")[["slope", "corr", "rmse", "rmse rank", "trmse", "trmse rank", "corr_residual", "corr rank", "slope rank"]]

    display(summary_df.round(3))
    # plt_table = pd.plotting.table(ax2, summary_df.round(3))
    # plt_table.scale(0.6, 3)
    plt.tight_layout(rect=[0, 0.03, 1, 0.95])
    filename = f'figures/{title.replace(" ", "_").lower()}.png'
    plt.savefig(filename)
    return summary_df

summary_plot(df1, result_df1, lm1, title="Measurement Sample $\sigma_x=\sigma_y$");
summary_plot(df2, result_df2, lm2, title="Measurement Sample $\sigma_x=5\sigma_y$");
summary_plot(df3, result_df3, lm3, title="Measurement Sample $\sigma_x=10\sigma_y$");
```

```python
def chi2_loss(params, x, y, xerr=0, yerr=0):
    slope, intercept, scatter = params

    ypred = slope * x + intercept
    denom = np.sqrt(xerr**2 + yerr**2 + scatter**2)
    chi2 = np.mean((y - ypred)**2 / denom)
    return chi2

def l2_loss(params, x, y):
    slope, intercept, scatter = params

    ypred = slope * x + intercept
    loss = np.mean((y - ypred)**2)
    return loss

xrange = np.random.uniform(0, 1, 1000)
yrange = xrange * 5 + 2 + stats.norm(scale=5).rvs(len(xrange))
res = minimize(chi2_loss, x0=(1, 0, 0), args=(xrange, yrange, 1, 1))

def _plot():
    slope, intercept, scatter = res.x
    print(res.x, res.success, res.message)
    plt.errorbar(xrange, yrange, yerr=1, fmt='ko', lw=0.5, ms=5)
    plt.plot(xrange, slope*xrange + intercept, zorder=100)

_plot()
```

```python
def simulate_all_fittings(x, y, sample_size=100, reps=5, linmix_kwargs=dict(miniter=2500, maxiter=2500)):
    data_size = sample_size
    size = len(x)

    results = {
        "OLS": [],
        "OLS_c": [],
        "Bootstrap": [],
        "Bootstrap_c": [],
        "LINMIX": [],
        "Chi2": []
    }
    slope_results {k: [] for k in results}
    chi2_success = []

    for i in range(reps):
        data_idx = np.random.choice(range(size), size=data_size, replace=False)
        xdata_true = x[data_idx]
        ydata_true = y[data_idx]

        sigma_x = np.ones(data_size)
        sigma_y = np.ones(data_size)
        xerr = stats.norm(loc=0, scale=sigma_x).rvs(data_size)
        yerr = stats.norm(loc=0, scale=sigma_y).rvs(data_size)

        xdata = xdata_true + xerr
        ydata = ydata_true + yerr

        df = pd.DataFrame(
            {
                "x": xdata,
                "xerr": sigma_x,
                "y": ydata,
                "yerr": sigma_y,
            }
        ).sort_values('x')

        # Ordinary Least Squares
        slope, slope_var = get_slope_parameter(df.x, df.y)
        corr = stats.pearsonr(df.x, df.y)[0]
        intercept = get_intercept_parameter(df.x, df.y, slope)
        ypred = slope*df.x + intercept
        results["OLS"].append(corr)
        slope_results["OLS"].append(slope)

        rxx = get_reliability(df.x, df.xerr)
        ryy = get_reliability(df.y, df.yerr)
        corr = corr / np.sqrt(rxx*ryy)
        slope = corr_to_slope(corr, df.x, df.xerr, df.y, df.yerr)
        results["OLS_c"].append(corr)
        slope_results["OLS_c"].append(slope)

        # Chi-2 with y-scatter, x-scatter, and residual
        res = minimize(chi2_loss, x0=(1, 0, 1), args=(df1.x, df.y, df.xerr, df.yerr))
        print(res.success)
        slope, intercept, _ = res.x
        corr = slope_to_corr(df.x, df.y, slope)
        results["Chi2"].append(corr)
        slope_results["Chi2"].append(slope)

        # BOOTSTRAP
        bootstrap_df = simulate(df.x, df.y, df.xerr, df.yerr)
        slope = bootstrap_df["slope"].mean()
        corr = bootstrap_df["corr"].mean()
        intercept = bootstrap_df["intercept"].mean()
        results["Bootstrap"].append(corr)
        slope_results["Bootstrap"].append(slope)

        rxx = get_reliability(df.x, df.xerr)
        ryy = get_reliability(df.y, df.yerr)
        corr = corr / np.sqrt(rxx*ryy)
        slope = corr_to_slope(corr, df.x, df.xerr, df.y, df.yerr)
        results["Bootstrap_c"].append(corr)
        slope_results["Bootstrap_c"].append(slope)

        # LINMIX
        lm = linmix.LinMix(df.x, df.y, df.xerr, df.yerr, K=3)
        lm.run_mcmc(silent=True, **linmix_kwargs)

        slope, intercept = np.mean(lm.chain['beta']), np.mean(lm.chain['alpha'])
        corr = slope_to_corr(df.x, df.y, slope)
        results["LINMIX"].append(corr)
        slope_results["LINMIX"].append(slope)

        print(f"{i}/{reps}", end="\r")

    corr_df = pd.DataFrame(results)
    corr_df.to_csv("results/regression-dilution-model-comparison.csv", index=False)

    slope_df = pd.DataFrame(slope_results)
    slope_df.to_csv("results/regression-dilution-model-comparison_slope.csv", index=False)

# simulate_all_fittings(x, y, reps=250)
_df = pd.read_csv("results/regression-dilution-model-comparison.csv").set_index("Unnamed: 0")
```

```python
bins = np.linspace(-0.4, 1, 30)
nrows, ncols = 3, 2
fig, axs = plt.subplots(nrows, ncols, figsize=(8*ncols, 4*nrows), sharey="row")
plt.subplots_adjust(wspace=0)

for i, col in enumerate(_df.columns):
    corr = _df.T.loc[col]
    ax = axs[i//ncols, i%ncols]
    ax.hist(corr, bins)
    col_title = col.replace("_c", " (Corrected)").replace("LINMIX", "LINMIX (1000 iters)").replace("Chi2", "$\chi^2$")
    ax.set_title(f"{col_title}\n$r={corr.mean():.3f}$ $\sigma_r={corr.std():.3f}$")
    ax.axvline(rho_xy, c='r', linestyle="--", label="True Correlation")

handles, labels = ax.get_legend_handles_labels()
fig.suptitle(f"Bivariate Correlation Fitting ({len(_df)} repetitions)")
fig.legend(handles, labels, loc='upper right')
plt.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.savefig("figures/regression-dilution-model-comparison.png")
```


# 2nd Order Optimization

## Newtons Method

### Part 2

## Newton's method optimization

```python
from sympy import *
from sympy.parsing import sympy_parser as spp

import numpy as np
import matplotlib.pyplot as plt
```

### Hyperparams

```python
plot_from = -7.0
plot_to = 7.0
plot_step = 0.1

target_precision = 0.3

sd_iters = 100000
display_freq = sd_iters//10

m = Matrix(symbols('x1 x2'))
```

```python
def dfdx(x,g):
    return [float(g[i].subs(m[0],x[0]).subs(m[1],x[1])) for i in range(len(g))]
```

```python
def sd(obj,x_start,x_result,alpha=0.0002):
    '''
    Steepest Descent - 1st Order Optimization
    '''
    print('Steepest Descent - Start')

    #Gradient
    g = [diff(obj,i) for i in m]
    #Initialize xs
    xs = [[0.0,0.0]]
    xs[0] = x_start
    #Get gradient at start location (dx/df or grad(f))
    iter_s = 0
    while np.linalg.norm(xs[-1] - x_result) > target_precision:
        if(iter_s % display_freq == 0):
            print ('Steepest Descent - Distance: ',np.linalg.norm(xs[-1] - x_result))
        gs = dfdx(xs[iter_s],g)
        #Compute search direction and magnitude (dx)
        #With dx = -grad but no line searching
        xs.append(xs[iter_s] - np.dot(alpha,gs))
        if(iter_s % display_freq == 0):
            print('Last xs:',xs[-1])
        iter_s += 1
        if iter_s > sd_iters:
            break

    print('Steepest Descent - Result distance: ', np.linalg.norm(xs[-1] - x_result))
    xs = np.array(xs)
    plt.plot(xs[:,0],xs[:,1],'g-o')
```

```python
def nm(obj,x_start,x_result):
    '''
    Newton's method - 2nd order optimization
    '''
    print ('Newton\'s method - Start')

    #Gradient
    g = [diff(obj,i) for i in m]
    #Hessian Matrix
    H = Matrix([[diff(g[j],m[i]) for i in range(len(m))] for j in range(len(g))])
    H_inv = H.inv()

    xn = [[0,0]] #Newton's method result global for comparison
    xn[0] = x_start

    iter_n = 0
    while np.linalg.norm(xn[-1] - x_result) > target_precision:
        print ('Newton\'s method - Distance: ',np.linalg.norm(xn[-1] - x_result))
        gn = Matrix(dfdx(xn[iter_n],g))
        delta_xn = -H_inv * gn
        delta_xn = delta_xn.subs(m[0],xn[iter_n][0]).subs(m[1],xn[iter_n][1])

        #Transform to numpy.ndarray to avoid np.linalg.norm error with 'Float' data type
        aux = []
        for elem in Matrix(xn[iter_n]) + delta_xn:
            aux.append(elem)

        aux = np.array(aux).astype(np.float64)
        xn.append(aux)
        iter_n += 1
    print ('Newton\'s method - Result Distance: ',np.linalg.norm(xn[-1] - x_result))

    xn = np.array(xn)
    plt.plot(xn[:,0],xn[:,1],'k-o')
```

### Test & Plot

```python
def run_and_plot_quadratic():
    #Quadratic function
    x_start = [-4.0,6.0]
    obj = spp.parse_expr('x1**2 - 2* x1 * x2 + 4 * x2 ** 2')
    x_result = np.array([0.0,0.0])

    #Design variables at mesh points
    i1 = np.arange(plot_from,plot_to,plot_step)
    i2 = np.arange(plot_from,plot_to,plot_step)
    x1_mesh,x2_mesh = np.meshgrid(i1,i2)
    f_str = obj.__str__().replace('x1','x1_mesh').replace('x2','x2_mesh')
    f_mesh = eval(f_str)

    #Create contour plot
    plt.figure()
    plt.imshow(f_mesh,cmap='Paired',origin='lower',extent=[plot_from - 20, plot_to + 20, plot_from - 20, plot_to + 2])
    plt.colorbar()

    #Add some text
    plt.title('f(x) = ' + str(obj))
    plt.xlabel('x1')
    plt.ylabel('x2')
    nm(obj,x_start,x_result)
    sd(obj,x_start,x_result,alpha=0.05)
    plt.show()
```

```python
def run_and_plot_rosenbrock():
    #Rosenbrock function
    x_start = [-4.0,-5.0]
    obj = spp.parse_expr('(1-x1)**2 + 100 * (x2 - x1**2)**2')
    x_result = np.array([1,1])

    #Design variables at mesh points
    i1 = np.arange(plot_from,plot_to,plot_step)
    i2 = np.arange(plot_from,plot_to,plot_step)
    x1_mesh,x2_mesh = np.meshgrid(i1,i2)
    f_str = obj.__str__().replace('x1','x1_mesh').replace('x2','x2_mesh')
    f_mesh = eval(f_str)

    #Create contour plot
    plt.figure()
    plt.imshow(f_mesh,cmap='Paired',origin='lower',extent=[plot_from - 20, plot_to + 20, plot_from - 20, plot_to + 2])
    plt.colorbar()

    #Add some text
    plt.title('f(x) = ' + str(obj))
    plt.xlabel('x1')
    plt.ylabel('x2')
    nm(obj,x_start,x_result)
    sd(obj,x_start,x_result,alpha=0.0002)
    plt.show()
```

### Test

```python
run_and_plot_quadratic()
```

```python
run_and_plot_rosenbrock()
```


# Neural Accumulator and ALU
This is a simple implementation of the Neural Accumulator and Neural Arithmetic Logic Unit as a Flux layer.

From the paper:
> Here we propose two models that are able to learn to represent and manipulate numbers in a systematic
way. The first supports the ability to accumulate quantities additively, a desirable inductive bias for
linear extrapolation. This model forms the basis for a second model, which supports multiplicative
extrapolation.

## Neural Accumulator
The NAC consists of a special affine transformation that consists of only -1's, 0's, and 1's. This prevents the layer from rescaling the representations when mapping from input to output.

This is accomplished by combining the saturating nonlinearities $tanh$ and $\sigma$:

$$
\begin{align}
\mathbf{a} & = \mathbf{Wx} \\
\mathbf{W} &= tanh(\mathbf{\hat{W}}) \odot \sigma(\mathbf{\hat{M}}),\\
\end{align}
$$
where $\mathbf{\hat{W}}$ and $\mathbf{\hat{M}}$ are weight matrices.

```julia
# Neural Accumulator
struct NAC
    W
    M
end

NAC(in::Integer, out::Integer) =
    NAC(randn(out, in), randn(out, in))

(nac::NAC)(x) = (tanh.(nac.W) .* σ.(nac.M)) * x
```

## Neural Arithmetic Logic Unit
> The NALU consists of two NAC cells interpolated by a learned sigmoidal gate g, such that if the add/subtract subcell’s output value is applied with a weight of 1 (on), the multiply/divide subcell’s is 0 (off) and vice versa.  The first NAC computes the accumulation vector a, which stores results of the NALU’s addition/subtraction operations; it is computed  identically to the original NAC, (i.e., a = Wx). The second NAC operates in log space and is therefore capable of learning to multiply and divide, storing its results in m.

$$
\begin{align}
\mathbf{y} &= \mathbf{g} \odot \mathbf{a} + (1 - \mathbf{g}) \odot \mathbf{m} \\
\mathbf{m} &= \exp \mathbf{W} (\log (|\mathbf{x}| + \epsilon)), \mathbf{g} = \sigma(\mathbf{Gx})
\end{align}
$$

```julia
# Neural Arithmetic Logic Unit
struct NALU
    nacₐ
    nacₘ
    G
end

NALU(in::Integer, out::Integer) =
    NALU(NAC(in, out), NAC(in, out), randn(out, in))

function (nalu::NALU)(x)
    # gate
    g = σ.(nalu.G*x)

    # addition
    a = nalu.nacₐ(x)

    # multiplication
    m = exp.(nalu.nacₘ(log.(abs.(x) .+ eps())))

    # nalu
    return g .* a + (1 .- g) .* m
end
```


# 823 HW2 -Candice Li
## Number theory and a Google recruitment puzzle

Find the first 10-digit prime in the decimal expansion of 17π.

In order to find the first 10-digit prime in the decimal expansion of 17π, three functions were defined.

### Import libraries going to be used

```python
import math
import decimal
try:
    from sympy.mpmath import mp
except ImportError:
    from mpmath import mp
```

The function *expansion_generater* is used to get n places after the decimal point for irrational number. First the decimal places after the decimal point is defined to be 1000 so that it is long enough to find the target segment. Then, the actual expression being used is genrated based on the input parameters. In this program, only e and π are being considered. Then, the part after the decimal point of the expression with desired length is being extracted and returned.

```python
def expansion_generater(n, symbol, time):
    """Get n places after the decimal point for irrational number.

    n : int
    The length of the wanted number after decimal places.
    """
    mp.dps = 1000
    if symbol == 'pi':
        a = mp.pi
    elif symbol == 'e':
        a = mp.e

    exp = a*time

    before_dec, after_dec = int(exp), exp - int(exp)

    return str(after_dec)[2:n+2]
```

The function *is_prime* is used to check if an integer is a prime number. First, special cases like 0, 1, and 2 are being accounted for. 0, 1 are hard-coded to be not prime and 2 is hard-coded to be prime. Secondly, all the even numbers are coded to be not prime. Then we check if the square root of the number can be diviide by any number smaller than itself. If there is one, then this number is not prime. Otherwise, it is a prime number.

```python
def is_prime(n):
    """Check if an integer is a prime number.

    n : int
    The integer we want to check if it is prime.
    """
    if n == 0 or n == 1:
        return False
    elif n == 2:
        return True
    elif (n > 2 and n % 2 == 0):
        return False
    else:
        for i in range (3, 1 + math.floor(math.sqrt(n)), 2):
            if n % i == 0:
                return False
    return True
```

The function *get_window* is used to get all the integers with desired length in an expression. First, the expression is converted from integer to string. Secondly, a list is created to store the integers. Then, we start from the first index take out every integer segment with desired length.

```python
def get_window(n, expression):
    """Get the integers of length n in an expression.

    n : int
    The length of the wanted intger in the expression.
    """
    expression = str(expression)
    result = []
    for i in range(0, len(expression)):
        if len(expression[i:i+n]) == n:
            result.append(expression[i:i+n])
    return result
```

The function *find_n_digit_prime* is the function that calls all of the functions written above and return the first value that satisfy the requirement. First, we get the segments of desried length using the function *get_window* and *expansion_generator*. *expansion_generator* returns the expression based on input and *get_window* get the segments with certain lenth in the expression. The, we store the segments if it has the same length with the desired length (the length may be shorter when reaching the end of the expression). Fianlly, we return the first segment in this list. This interger will be the final answer from this program.

```python
def find_n_digit_prime(total_len, symbol, time, seg_len):
    segments = get_window(seg_len, expansion_generater(total_len, symbol, time))
    found = []
    for j in range (0, len(segments)):
        if is_prime(int(segments[j])):
            found.append(int(segments[j]))
    return found[0]

```

### Run the code and get the answer for the first 10-digit prime in the decimal expansion of 17π.

```python
answer = find_n_digit_prime(1000, "pi", 17, 10)

print("The first 10-digit prime in the decimal expansion of 17π is "+ str(answer) +".")
```

    The first 10-digit prime in the decimal expansion of 17π is 8649375157.

## Unit tests

### Test for funtion *expansion_generater*

```python
import unittest
```

```python
class TestNotebook(unittest.TestCase):

    def expansion_generater(self):
        self.assertEqual(expansion_generater(10, 'pi', 1), 1415926535)
        self.assertEqual(expansion_generater(11, '2', 2), 43656365692)

unittest.main(argv = [''], verbosity = 2, exit = False)
```

    ----------------------------------------------------------------------
    Ran 0 tests in 0.000s

    OK

    <unittest.main.TestProgram at 0x7fe6ee1b1210>

### Test for funtion *expansion_generater*

```python
class TestNotebook(unittest.TestCase):

    def is_prime(self):
        self.assertEqual(is_prime(150), False)
        self.assertEqual(is_prime(11), True)
        self.assertEqual(is_prime(2), True)

unittest.main(argv = [''], verbosity = 2, exit = False)
```

    ----------------------------------------------------------------------
    Ran 0 tests in 0.000s

    OK

    <unittest.main.TestProgram at 0x7fe6ee24f210>

### Test for funtion *get_window*

```python
class TestNotebook(unittest.TestCase):

    def get_window(self):
        self.assertEqual(get_window(1, 12345), ['1', '2', '3', '4', '5'])
        self.assertEqual(get_window(2, 12345), ['12', '23', '34', '45'])
        self.assertEqual(get_window(3, 12345), ['123', '234', '345'])

unittest.main(argv = [''], verbosity = 2, exit = False)
```

    ----------------------------------------------------------------------
    Ran 0 tests in 0.000s

    OK

    <unittest.main.TestProgram at 0x7fe6ee1d4050>

### Test for funtion *find_first_ten_digit_prime*

```python
class TestNotebook(unittest.TestCase):

    def test_n_ten_digit_prime(self):
        self.assertEqual(find_first_ten_digit_prime(1000, 'e', 1, 10), 7427466391)

unittest.main(argv = [''], verbosity = 2, exit = False)
```

    test_n_ten_digit_prime (__main__.TestNotebook) ... ok

    ----------------------------------------------------------------------
    Ran 1 test in 0.175s

    OK

    <unittest.main.TestProgram at 0x7fe6ee223390>


# Práctica del tema 2
Blanca Cano Camarero

```python
# import necesarios para la práctica
import sympy as sp
import numpy as np

## ficheros propios
#### De vuelve el polinomio de interpolación de newton
import polinomioNewton as pn # polinomioNewton([a-h,a,a+h])

##### Algunas fórmulas de integración
import formulasIntegracion as fs

#### cuadratura gaussina
import nodosGaussiana as ng
```

# Ejercicio 1
1.- Obtenga mediante interpolación en el espacio $\mathbb{P}_2$ una fórmula para aproximar $f''(a)$ del tipo combinación
de $f(a-h)$, $f(a)$ y $f(a+h)$.

```python
a,h,z = sp.symbols('a,h,z')
x = [a-h, a, a+h]

#llamamos a nuestra funcioncilla que nos calcula polinomios de interpolación de newton
## (para ver su código mirar en polinomioNewton.py)
p = pn.polinomioNewton(x)

## Finalemente, derivamos pa obtener nuestra fórmula buscada
formula=sp.diff(p,z,2).subs({z:a}).simplify() # esta será la fórmula
# obtenida esta vez para aproximar la derivada segunda f''(a)
print(formula)
formula
```

    (-2*f(a) + f(a - h) + f(a + h))/h**2

$\displaystyle \frac{- 2 f{\left(a \right)} + f{\left(a - h \right)} + f{\left(a + h \right)}}{h^{2}}$

## Ejercicio 2
2.- Con la fórmula obtenida en el ejercicio 1, halle una tabla de aproximaciones y errores de $f_1''(2.5)$, siendo $f_1(x)=x^x$, para $h=10^{-i},\; i=1,\ldots,5.$

```python
## derivada aproximada
def d2(f):
    return (-2*f(a) + f(a - h) + f(a + h))/h**2

def g(x):
    return x**x

### derivada exacta
g2 = sp.lambdify(z,sp.diff(z**z, z, 2), 'numpy')

#####
def g2_aprox(formula, g,mia, eh):
    """Devuelve la derivada aproximada de g con la fórmula "formula" en el punto mia, con h valiendo eh
    Argumentos:
        formula: aplicación lineal, con la que este caso aproximamos derivadas
        g la función de la que queremos calcular la derivada
        mia: punto en el que geremos calcular a
        eh: variación de h
    """
    return formula(g).evalf(subs={a:mia,h:10**(-eh)})

### hacemos una función pa representar la tablilla

def tabla(formula,faprox, fexac, nodos, vh):
    print(f' h          | a \t  | f(a) aprox \t | error'.expandtabs(25))
    print('-'*65)
    for x in nodos:
        valor_exacto = fexac(x)
        for mih in vh:
            v = g2_aprox(formula,faprox,x,mih)
            print(f'10^(-{mih})    | {x} \t | {v} \t|{abs(v-valor_exacto)}'.expandtabs(25))

tabla(d2,g,g2,[2.5],[*range(1,6)])

```

    h          | a            | f(a) aprox            | error
    -----------------------------------------------------------------
    10^(-1)    | 2.5          | 40.4205682979584      |0.178903441570895
    10^(-2)    | 2.5          | 40.2434502309567      |0.00178537456920225
    10^(-3)    | 2.5          | 40.2416827097676      |0.0000178533801502567
    10^(-4)    | 2.5          | 40.2416650349212      |1.78533753114607E-7
    10^(-5)    | 2.5          | 40.2416648581728      |1.78533099415290E-9

## Ejercicio 3
3.- Sea $f_2(x)=\frac{x^2+40}{x+\sqrt{5x}+7}$. Calcule una tabla que recoja las derivadas de $f_2$ en $x_i=1,2,\ldots,10$, utilizando alguna de las fórmulas de derivación numérica de primer orden obtenidas al inicio de la práctica, con $h=10^{-3}$, y muestre al mismo tiempo el error cometido en cada punto. Repita el ejercicio con la fórmula centrada obtenida para la derivada primera y, finalmente, para la obtenida en el ejercicio 1 (con respecto a la segunda derivada).

```python
def f2(x):
    return (x**2 +40) / (x+(5*x)**(1/2)+7)

### Vamos a comenzar con la segunda derivada
ddf=sp.diff( (z**2 +40) / (z+(5*z)**(1/2)+7), z, 2)
ddf=sp.lambdify(z,ddf, 'numpy')
x = [*range(1,11)]

### tabla para la segunda derivada
print('Las aproximaciones para la segunda derivada son ')
tabla(d2,f2,ddf,x,[3])

## ________Para la fórmula centrada ____
print('\nDerivamos una vez el polinomio de newton, para obtener la primera derivada')
formula=sp.diff(p,z,1).subs({z:a}).simplify() # esta será la fórmula
# obtenida esta vez para aproximar la derivada segunda f''(a)
print(formula)

#copiamso en el return la fórmula anterior
def df_aprox(f):
    return (-f(a - h) + f(a + h))/(2*h)

#calculamos primera derivada exacta
df=sp.diff( (z**2 +40) / (z+(5*z)**(1/2)+7), z, 1)
df=sp.lambdify(z,df, 'numpy')

print('\n\n Las proximaciones para la primera derivada')
tabla(df_aprox,f2,df,x,[3] )

```

    Las aproximaciones para la segunda derivada son
     h          | a            | f(a) aprox            | error
    -----------------------------------------------------------------
    10^(-3)    | 1            | 0.676265261235160     |1.62949784665578E-7
    10^(-3)    | 2            | 0.283220380779308     |1.66032020909590E-8
    10^(-3)    | 3            | 0.168340324507543     |4.57942220255525E-9
    10^(-3)    | 4            | 0.114907314792673     |1.89762004720873E-9
    10^(-3)    | 5            | 0.0846224312804510    |9.93457316411650E-10
    10^(-3)    | 6            | 0.0654364319768901    |6.12947123612706E-10
    10^(-3)    | 7            | 0.0523721747995802    |4.30544377927333E-10
    10^(-3)    | 8            | 0.0430109452392692    |3.36394023747744E-10
    10^(-3)    | 9            | 0.0360420060101230    |2.86374354785401E-10
    10^(-3)    | 10           | 0.0306970069224294    |2.60408326846484E-10

    Derivamos una vez el polinomio de newton, para obtener la primera derivada
    (-f(a - h) + f(a + h))/(2*h)

     Las proximaciones para la primera derivada
     h          | a            | f(a) aprox            | error
    -----------------------------------------------------------------
    10^(-3)    | 1            | -0.633413983453862    |1.41948961474014E-7
    10^(-3)    | 2            | -0.203730021215820    |2.98523985808874E-8
    10^(-3)    | 3            | 0.0135536643822118    |1.22135472690327E-8
    10^(-3)    | 4            | 0.152356375979550     |6.46680242688547E-9
    10^(-3)    | 5            | 0.250865047976407     |3.92670773674553E-9
    10^(-3)    | 6            | 0.325234483749073     |2.59700044802358E-9
    10^(-3)    | 7            | 0.383753087445717     |1.82151538297148E-9
    10^(-3)    | 8            | 0.431201819322740     |1.33390964940361E-9
    10^(-3)    | 9            | 0.470566738047861     |1.00977470829378E-9
    10^(-3)    | 10           | 0.503824069630687     |7.84850628932077E-10

```python

```

```python

# Algunas de las definidas en formulasSimples.py
# declaramos las fórmulas
def rectangulo_izquierdo(f,a,b):
    return f(a)*(b-a)

def simpson(f, a, b):
    return 1/6*(f(a)+4*f((a+b)/2)+f(b))*(b-a)

def trapecio(f,a,b):
    return (f(a)+f(b))/2*(b-a)

def formula_compuesta (f,formula,a,b,n):
    h = (b-a)/n
    return sum([formula(f,a+i*h,a+(i+1)*h) for i in range(n)])

```

# Ejericicios 4 y 5

4.- Divida el intervalo $[1,2]$ en 100 partes iguales y aplique las fórmulas del rectángulo, Simpson y trapecio compuestas para aproximar la integral en dicho intervalo de $f_1$. Compare dichos resultados.

5.- Repita el ejercicio 4 para $f_2$.

```python
def f1(x):
    return x**x
print("___Integral para f1___")
print(f'Integra por rectángulo izquierdo {formula_compuesta(f1,rectangulo_izquierdo,1,2,100)}')
print(f'Integra por trapecio {formula_compuesta(f1,trapecio,1,2,100)}')
print(f'Integra por simpson {formula_compuesta(f1,simpson,1,2,100)}')

def f2(x):
    return (x*x+40)/(x+sqrt(5*x)+7)
print("___Integral para f2___")
print(f'Integra por rectángulo izquierdo {formula_compuesta(f2,rectangulo_izquierdo,1,2,100)}')
print(f'Integra por trapecio {formula_compuesta(f2,trapecio,1,2,100)}')
print(f'Integra por simpson {formula_compuesta(f2,simpson,1,2,100)}')

```

    ___Integral para f1___
    Integra por rectángulo izquierdo 2.0354943390855578
    Integra por trapecio 2.0504943390855574
    Integra por simpson 2.0504462346235286
    ___Integral para f2___
    Integra por rectángulo izquierdo 3.77852320278209
    Integra por trapecio 3.77658469845732
    Integra por simpson 3.77658111777024

Comentario:
Como podemos observar dan un resultado similar, siend trapecio y simpson muy parecidas entre sí.

## Ejercicio 6
6.- Sea $f_3(x)=x^{15} e^x$ en $[0,2]$. Vamos a dividir el intervalo en $10\times 2^n$ subintervalos, es decir, $10,\,20,\,40,\, 80,\ldots $ y a aplicar la fórmula de Simpson compuesta hasta que la diferencia entre dos aproximaciones consecutivas (por ejemplo, podrían
ser con $20$ y $40$ subintervalos) sea menor que $10^{-2}$, dando en tal caso por buena la última aproximación obtenida. Programe
y calcule dicha aproximación. Compare ambas aproximaciones con
el valor exacto.

```python
def f3(x):
    return x**(15) * np.exp(x)

## cálculo de valor exacto por regla de barrow
x = sp.symbols("x")
f3s = x**15 * sp.exp(x)
If3s = sp.integrate(f3s,x)
exacto = If3s.subs(x,2)-If3s.subs(x,0)
print(f'Aplicando la regla de barrow el valor es {exacto.evalf()}')
diferencia = 9
y = fs.formula_compuesta(f3,fs.simpson,0,2,10)
n= 1
while diferencia > 10**(-2):
    ys = fs.formula_compuesta(f3,fs.simpson,0,2,10*2**(n))
    n += 1
    diferencia = abs(ys-y)
    y = ys
print (f'La a integral aproximada (en la iteración {n})  es {y}')
print (f'El error es de {abs(exacto-y).evalf()}')
```

    Aplicando la regla de barrow el valor es 27062.7024138996
    La a integral aproximada (en la iteración 6)  es 27062.702480891196
    El error es de 0.0000213214760707824

## Ejercicio 7

7.- Calcule las fórmulas gaussianas con $2$ y $3$ nodos,en el intervalo $[-1,1]$, siendo la función peso el valor absoluto de la variable. Aplíquelas para aproximar la función $x\; e^x$ en $[-1,1]$ y compare los resultados con el valor exacto (organizando los cálculos de forma adecuada).

```python
#calcula nodos
def nodos_gaussiana (a, b, n,w):
    """ Devuelve los nodos de la fórmula de la gaussiana
    a,b: intervalo [a,b] de definición
    n: número de nodos
    w: función peso
    """
    x = sp.Symbol("x")
    c = list(sp.symbols('c0:'+ str(n)))
    pi = np.prod([ (x - c[i]) for i in range(n)])
    #print('Voy a computar la integral (esto puede tardar)...')
    I = [sp.integrate(pi*w(x)*x**i,(x, a, b)) for i in range(n)]
    #print(f'El valor de la integrales es {I}')
    #print('la solución buscada es: ')
    s = sp.solve(I,c)
    return list(s[0])

```

```python
a = -1
b= 1
n = 3
def w(x):
    return abs(x)#sp.sqrt(1/(1-x**2))
z=sp.Symbol('z')
for i in range(2,n+1):
    q =pn.polinomioNewton(nodos_gaussiana(a,b,i,w))
    print(f'fóruma para {i} nodos')
    print(sp.integrate(q,(z,a,b)))

f_exacta = x*sp.exp(x)
print(f'\nEl valor exacto es {sp.N(sp.integrate(f_exacta,(x,a,b)))}')

## vemos ahora a por el valro aproximado
def f_aprox(x):
    return x*np.exp(x)

def dosNodos(f):
    return f(-np.sqrt(2)/2) + f(np.sqrt(2)/2)
def tresNodos(f):
    return f(0) + f(-np.sqrt(6)/3)/2 + f(np.sqrt(6)/3)/2

print(f'Para la aproximación de dos nodos el valor es {dosNodos(f_aprox)}')
print(f'Para la aproximación de tres nodos el valor es {tresNodos(f_aprox)}')
```

    fóruma para 2 nodos
    f(-sqrt(2)/2) + f(sqrt(2)/2)
    fóruma para 3 nodos
    f(0) + f(-sqrt(6)/3)/2 + f(sqrt(6)/3)/2

    El valor exacto es 0.735758882342885
    Para dos nodos el valor es 1.0854416412726071
    Para tres nodos el valor es 0.7432494342785245

## Ejercicio 8

8.- Programar las técnicas de integración de Romberg y adaptativa, para después aplicarlas a la aproximación de la siguiente integral $$\int_a^b p(x)\, dx$$
siendo  $\;a=\displaystyle\min_{0\leq i\leq 7}{d_i}$, $\;b=\displaystyle\max_{0\leq i\leq 7}{d_i}$ y
$$p(x)=d_0 + d_1 x + d_2 x^2 + d_3 x^3+ d_4 x^4 + d_5 x^5 + d_6 x^6 + d_7 x^7 $$
(siendo $d_0, d_1, \ldots, d_7$ los dígitos).

```python
## El polinomio pedido es:
coef = [2,5,3,7,8,2,9,1]
a = min(coef)
b = max(coef)

def p(x):
    return sum([coef[i]*x*i for i in range(len(coef))])

# fórmula en fichero formulasIntegracion.py
### ADAPTATIVA #######

print(f'El resultado de la adaptativa es {fs.adaptativa(fs.simpson , p, a , b)}')

### ROMBER
def T2n(tn,f,h,a,n):
    """Fórmula recursiva trapecio
    tn: valor de T_n
    f: función a evaluar
    h: diferencia partición h_{n+1}=h_{n}/2, por eso los nodos por calcular son x_i+h_{n+1}     = (a + m*h_n)+h_{n+1} = (a + m*2h_{n+1})+h_{n+1}
    a: intervalo inferior partición
    n: proviene de h{n+1}=(b-a)/2^{n+1}
    """
    return 1/2*(tn)+h*sum(map((lambda e: f(a+(e*2+1)*h )),[*range(n)]))

def Romberg(f,a,b,tol=(10**(-5)),length=20):
    """ Integrar calculada por el método de Romberg,
    devuelve el número ancho de columan exploradas
    f: función sobre la que integrar
    a: intervalo inferior sobre el que integrar
    b: intervalo superior integración
    length: Ancho de la tabla maximo
    tol: diferencia que debe haber entre R(n-1,n-1) y R(n,n)
    """
    #matriz que contentendrá datos integración Romberg
    l = [ [None]*(i+1) for i in range(length)] # ojo que viene dada por l[fila][columan]

    #Calulamos R(N,0), es decir la primera columa de la tabla
    l[0][0]=fs.formula_compuesta(f,fs.trapecio, a, b,1)
    h = (b-a)
    n=1
    for i in range(1,length):
        h=h/2
        n = n*2
        l[i][0] = T2n(l[i-1][0],f,h,a,n)

    ## camos a proceder ahora con los elementos de la forma R(j,k)
    columna = 1
    diferencia = 99999
    #for columna in range(1,length):
    while( columna < length and diferencia > tol):
        for fila in range(columna, length):
            l[fila][columna] = (4**columna*l[fila][columna -1] - l[fila - 1][columna -1] ) / (4**columna -1)

        diferencia = abs(l[columna][columna]-l[columna-1][columna-1])
        columna += 1
    columna -= 1
    return (l[columna][columna],columna)

r = Romberg(p,a,b)
print(f'El resultado de mi fórmula de romber es {r[0]} tras {r[1]} iteraciones')
```

    El resultado de la adaptativa es 5400.0
    El resultado de mi fórmula de romber es 19439.983687473403 tras 19 iteraciones

```python

```


# Pricing a European Up-and-out Call Option

Abstract

This report presents the results of the simulation of a European up-and-out call option over twelve months. The Black-Scholes-Merton model will be used to implement Monte Carlo esti- mations using Python code. The estimations will simulate paths for the underlying share and the counterparty’s firm value using varying sample sizes, and determine estimates of both the default-free value of the option and the Credit Valuation Adjustment (CVA). Following from these it is simple to price the option incorporating counterparty risk, given by the default-free price less the CVA.

#### Keywords:
Exotic Options, European up-and-out call option, Black-Scholes-Merton model and Geometric Brownian Motion.

## Introduction

A European up-and-out call option is type of a Barrier Option, these options are consider path- dependant because barrier option’s payoff is based on the underlying stock’s price path. The payoff of an up-and-out option at maturity T is given by,

$Payoff_T = (S_T - K)^T$  given  $\max_{t\in[0,T]}$  $S_t < L$

Where, where K is the strike of the option, L is the barrier level, and St is the share price at time t.
This is a type of call option whose payoff is reduced to 0 if the share price crosses the barrier level.

The seminal work of Merton [1] pioneered the formula for pricing barrier options. This led pricing formulas under the geometric Brownian motion (GBM) framework for one-asset barrier options by Rich [2] and multi-asset barrier options by Wong and Kwok [3]. Although the return dynamics of underlying shares are not sufficiently well described by the GBM process proposed by Black and Scholes [4], we are going to assume both the stock and counterparty firm values follow GBM with constant drift and volatilities and default only occurs at maturity.
Possible paths for the up-and-out barrier option payoff would look similar to the chart below, where two of the paths have crossed the barrier and resulted in zero payoff, but the other three have not crossed the barrier and behave like a vanilla option.

```python
import numpy as np
import pandas as pd
from scipy import stats
import math
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
from matplotlib.axis import Axis
%matplotlib inline
```

We initialize most variables as given by the question. We assume the counterparty firm's value is at 200, as this is above the counterparty's debt due in one year. Also, we calculate the default-free value of our option to be 6.60, so we select a firm value not too distant such that the CVA value we calculate is not negligible. For example, if we chose counterparty firm value as 1,000, the CVA would be 0.000.

```python
### Initialize problem parameters
T = 1 # option maturity
L = 150 # up-and-out barrier
S0 = 100 # current share price
K = 100 # strike price, at-the-money
risk_free = .08 # risk-free rate
sigma = .3 # volatility
v_0 = 200 # counterparty firm current value (Our assumption)
sigma_firm = .25 # volatility for the counterparty's firm
debt = 175 # counterparty's debt, due in one year
corr = .2 # correlation
recovery_rate = 0.25 # recovery rate
########
corr_matrix = np.array([[1, corr], [corr, 1]])
sample_sizes = range(1000, 50001, 1000)
```

## Simulate paths for the underlying share and for the counterparty’s firm value using sample sizes of 1000, 2000, …, 50000. Do monthly simulations for the lifetime of the option.

Assuming that market has reached current level based on fair trading, and all participants have they same information about underlying asset. We can consider the price of asset to be a stochastic process based on brownian motion with some drift $\mu$ and $\sigma$ diffusion:

\begin{align}
dS_t=S_t(\mu dt + \sigma dW_t)
\end{align}

solving SDE equations and replacing $\mu$ with riskless rate, we can evaluate the share price path as
\begin{align}
S_t=S_0 e^{(r-\frac{1}{2} \sigma^2)t + \sigma W_t} \\
\end{align}

We use a Cholesky decomposition to generate the correlated price paths

```python
def share_path(S_0, risk_free_rate, sigma, Z, dT):
    return S_0*np.exp(np.cumsum((risk_free_rate-sigma**2/2)*dT + sigma*np.sqrt(dT)*Z,1))

def generate_share_and_firm_price(S0, v_0, risk_free, sigma, sigma_firm, corr, T, sample_size = 10, timesteps = 12):
    corr_matrix = np.array([[1, corr], [corr, 1]])
    norm_matrix = stats.norm.rvs(size = np.array([sample_size, 2, timesteps]))
    corr_norm_matrix = np.matmul(np.linalg.cholesky(corr_matrix), norm_matrix)

    share_price_path = pd.DataFrame(share_path(S0, risk_free, sigma, Z=corr_norm_matrix[:,0,], dT=1/timesteps))
    share_price_path = share_price_path.transpose()

    firm_price_path = pd.DataFrame(share_path(v_0, risk_free, sigma_firm, Z=corr_norm_matrix[:,1,], dT=1/timesteps))
    firm_price_path = firm_price_path.transpose()

    return [share_price_path,firm_price_path]
```

To double check that the stock prices and firm values monthly returns are correlated, we check them as follows, generating 20 different price paths with 10,000 timesteps.

```python
#Testing share and firm price correlation
sample_size = 20
test = generate_share_and_firm_price(S0, v_0, risk_free, sigma, sigma_firm, corr, T, sample_size, timesteps = 10000)

share_ret = np.log(test[0])

for i in range(sample_size):
    test[0]['sharelog'] = np.log(test[0][i])
    test[1]['firmlog'] = np.log(test[1][i])
    pearson, p_value = stats.pearsonr(test[0]['sharelog'].diff().dropna(), test[1]['firmlog'].diff().dropna())
    print("Pearson correlation coefficient : {:.3f}, Two-tailed p-value {:.3f}".format(pearson, p_value))
```

    Pearson correlation coefficient : 0.191, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.198, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.230, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.208, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.207, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.208, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.202, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.205, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.206, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.204, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.195, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.202, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.186, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.166, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.181, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.186, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.203, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.214, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.209, Two-tailed p-value 0.000
    Pearson correlation coefficient : 0.198, Two-tailed p-value 0.000

Let's try to simulate the share price with a small number of sample paths and visualize them over the course of 12 months

```python
share_and_firm_price_12_months = \
   generate_share_and_firm_price(S0, v_0, risk_free, sigma, sigma_firm, corr, T, sample_size = 5, timesteps = 12)
share_price_12_months = share_and_firm_price_12_months[0]
share_price_12_months.plot(title='Share price over 12 months', legend=False, figsize=(12,4));
```

We can do the same thing to simulate counterparty firm's value

```python
firm_value_12_months = share_and_firm_price_12_months[1]
firm_value_12_months.plot(title="Counterparty firm's value over 12 months", legend=False, figsize=(12,4));
```

In fact, we can also simulate the dynamic with finer resolution.

```python
share_and_firm_price_252_days = generate_share_and_firm_price(S0, v_0, risk_free, sigma, sigma_firm, corr, T, sample_size = 5, timesteps = 252)
share_price_252_days = share_and_firm_price_252_days[0]
firm_price_252_days = share_and_firm_price_252_days[1]
share_price_252_days.plot(title='Share price over 252 days', legend=False, figsize=(12,4))
firm_price_252_days.plot(title="Counterparty firm's value over 252 days", legend=False, figsize=(12,4));
```

Let's visualize when the stopped process is appied

```python
def stop(s, cond):
    ret = s.copy()
    r = ret[cond]
    if len(r) > 0:
        ret[r.idxmin():] = 0
    return ret
```

```python
share_price_12_months.apply(lambda s: stop(s, s>L), axis=0)\
                     .plot(legend=False, figsize=(12,4))
```

We then define a function to calculate the payoff for the up-and-out call option.

```python
# define payoff for up-and-out call option
def payoff(S_t, K, L):
    stopped_S = S_t.iloc[-1].where((S_t < L).all(), 0)
    return np.maximum(stopped_S - K, 0).to_numpy()

payoff(share_price_12_months, K, L)
```

    array([ 1.81747919,  0.        ,  0.        , 12.92678754,  0.        ])

We now create the share and firm price paths for sample sizes 1000, 2000, ...., 50000, as required for part 1 of the assignment.

```python
share_price_paths = {}
firm_val_paths = {}

sample_sizes = range(1000, 50_001, 1000)

for sample_size in sample_sizes:
    share_val, firm_val = \
        generate_share_and_firm_price(S0, v_0, risk_free, sigma, sigma_firm, corr, T,
                                      sample_size = sample_size, timesteps = 12)

    share_price_paths[sample_size] = share_val
    firm_val_paths[sample_size] = firm_val
```

We now plot the share and firm price paths for sample size of 1000.

```python
share_price_paths[1000].plot(title="Share price value for sample size of 1000", legend=False, figsize=(12,4));
```

```python
firm_val_paths[1000].plot(title="Counterparty firm value for sample size of 1000", legend=False, figsize=(12,4));
```

#  Determine the Monte Carlo estimates of the both the default-free value of the option and the Credit Valuation Adjustment

To calculate Monte Carlo estimate of the default-free option value, we calculate the average payoff of the 1000s of sample price paths, to estimate the price of the option

```python
# Estimate the default-free value of the option:
option_estimate = []
option_std = []

for sample_size, paths in share_price_paths.items():
    payoffs = payoff(paths, K, L)
    option_price = np.exp(-risk_free*T)*payoffs
    option_estimate.append(option_price.mean())
    option_std.append(option_price.std()/np.sqrt(sample_size))

```

Prices of default-free option at various sample sizes:

```python
for i in range(len(option_estimate)):
    print("sample size: {}, Option value: {:.3f}".format((i+1)*1000,option_estimate[i]))
```

    sample size: 1000, Option value: 6.661
    sample size: 2000, Option value: 6.264
    sample size: 3000, Option value: 6.870
    sample size: 4000, Option value: 6.759
    sample size: 5000, Option value: 6.744
    sample size: 6000, Option value: 6.790
    sample size: 7000, Option value: 6.474
    sample size: 8000, Option value: 6.719
    sample size: 9000, Option value: 6.756
    sample size: 10000, Option value: 6.787
    sample size: 11000, Option value: 6.654
    sample size: 12000, Option value: 6.500
    sample size: 13000, Option value: 6.813
    sample size: 14000, Option value: 6.649
    sample size: 15000, Option value: 6.689
    sample size: 16000, Option value: 6.610
    sample size: 17000, Option value: 6.724
    sample size: 18000, Option value: 6.637
    sample size: 19000, Option value: 6.768
    sample size: 20000, Option value: 6.573
    sample size: 21000, Option value: 6.612
    sample size: 22000, Option value: 6.670
    sample size: 23000, Option value: 6.806
    sample size: 24000, Option value: 6.666
    sample size: 25000, Option value: 6.671
    sample size: 26000, Option value: 6.735
    sample size: 27000, Option value: 6.808
    sample size: 28000, Option value: 6.756
    sample size: 29000, Option value: 6.763
    sample size: 30000, Option value: 6.660
    sample size: 31000, Option value: 6.754
    sample size: 32000, Option value: 6.708
    sample size: 33000, Option value: 6.690
    sample size: 34000, Option value: 6.644
    sample size: 35000, Option value: 6.679
    sample size: 36000, Option value: 6.675
    sample size: 37000, Option value: 6.773
    sample size: 38000, Option value: 6.734
    sample size: 39000, Option value: 6.692
    sample size: 40000, Option value: 6.715
    sample size: 41000, Option value: 6.682
    sample size: 42000, Option value: 6.631
    sample size: 43000, Option value: 6.783
    sample size: 44000, Option value: 6.712
    sample size: 45000, Option value: 6.662
    sample size: 46000, Option value: 6.698
    sample size: 47000, Option value: 6.657
    sample size: 48000, Option value: 6.625
    sample size: 49000, Option value: 6.732
    sample size: 50000, Option value: 6.707

```python
# Confidence level as 99.7 gives 3 std. deviations
confidence_sigma = 3

fig, ax = plt.subplots(figsize=(12,4))

ax.plot(option_estimate, '.')
ax.plot(option_estimate + confidence_sigma * np.array(option_std), 'r')
ax.plot(option_estimate - confidence_sigma * np.array(option_std), 'r')
plt.title("Option value estimates")
plt.xlabel("Sample size")
plt.ylabel("Option value")

ticks_x = ticker.FuncFormatter(lambda x, pos: '{0:g}'.format(x * 1000))
ax.xaxis.set_major_formatter(ticks_x)
plt.show()
```

## CVA Adjustment

To calculate Monte Carlo estimate of the credit value adjustment, we calculate the average loss of the 1000s of sample price paths, which happens when we both see a gain in the option with hold and when the counterparty firm value is less than its debt.

According to Merton model, we can make the same assumptions as the Black-Scholes model, except for the no risk of default assumption:

\begin{align}
dV_t=V_t(r dt + \sigma dW_t); V_0 = V
\end{align}

I.e. the firm value, can be modeled in the same way as any asset.

As per notes, we assume that default can only occur at time T, and firm defaults if the firm value is below firm debt amount

```python
def terminal_value(S_0, risk_free_rate, sigma, Z, T): #applies to both firm and stock
    return S_0 * np.exp((risk_free_rate - sigma**2/2) * T + sigma * np.sqrt(T) * Z)
```

```python
cva_estimate = []
cva_std = []

for sample_size, paths in share_price_paths.items():
    payoffs = payoff(paths, K, L)
    term_firm_vals = firm_val_paths[sample_size].iloc[-1].to_numpy()
    amount_lost = np.exp(-risk_free*T)*(1-recovery_rate)*(term_firm_vals < debt)*payoffs
    cva_estimate.append(amount_lost.mean())
    cva_std.append(amount_lost.std()/np.sqrt(sample_size))

```

Credit value adjustment at various sample sizes:

```python
for i in range(len(cva_estimate)):
    print("Sample size: {}, CVA: {:.3f}".format((i+1)*1000,cva_estimate[i]))
```

    Sample size: 1000, CVA: 1.123
    Sample size: 2000, CVA: 0.883
    Sample size: 3000, CVA: 0.972
    Sample size: 4000, CVA: 1.036
    Sample size: 5000, CVA: 0.963
    Sample size: 6000, CVA: 0.930
    Sample size: 7000, CVA: 0.911
    Sample size: 8000, CVA: 1.007
    Sample size: 9000, CVA: 0.943
    Sample size: 10000, CVA: 0.915
    Sample size: 11000, CVA: 0.916
    Sample size: 12000, CVA: 0.912
    Sample size: 13000, CVA: 0.933
    Sample size: 14000, CVA: 0.981
    Sample size: 15000, CVA: 0.914
    Sample size: 16000, CVA: 0.957
    Sample size: 17000, CVA: 0.978
    Sample size: 18000, CVA: 0.936
    Sample size: 19000, CVA: 0.932
    Sample size: 20000, CVA: 0.977
    Sample size: 21000, CVA: 0.974
    Sample size: 22000, CVA: 0.971
    Sample size: 23000, CVA: 0.947
    Sample size: 24000, CVA: 0.946
    Sample size: 25000, CVA: 0.952
    Sample size: 26000, CVA: 0.939
    Sample size: 27000, CVA: 0.953
    Sample size: 28000, CVA: 0.928
    Sample size: 29000, CVA: 0.940
    Sample size: 30000, CVA: 0.907
    Sample size: 31000, CVA: 0.968
    Sample size: 32000, CVA: 0.933
    Sample size: 33000, CVA: 0.957
    Sample size: 34000, CVA: 0.950
    Sample size: 35000, CVA: 0.939
    Sample size: 36000, CVA: 0.958
    Sample size: 37000, CVA: 0.967
    Sample size: 38000, CVA: 0.930
    Sample size: 39000, CVA: 0.926
    Sample size: 40000, CVA: 0.952
    Sample size: 41000, CVA: 0.944
    Sample size: 42000, CVA: 0.925
    Sample size: 43000, CVA: 0.971
    Sample size: 44000, CVA: 0.945
    Sample size: 45000, CVA: 0.960
    Sample size: 46000, CVA: 0.960
    Sample size: 47000, CVA: 0.967
    Sample size: 48000, CVA: 0.943
    Sample size: 49000, CVA: 0.983
    Sample size: 50000, CVA: 0.963

```python
# Confidence level as 99.7 gives 3 std. deviations
fig, ax = plt.subplots(figsize=(12,4))
ax.plot(cva_estimate, '.')
ax.plot(cva_estimate + 3 * np.array(cva_std), 'r')
ax.plot(cva_estimate - 3 * np.array(cva_std), 'r')
plt.title("CVA estimates")
plt.xlabel("Sample size")
plt.ylabel("Option value")

ticks_x = ticker.FuncFormatter(lambda x, pos: '{0:g}'.format(x * 1000))
ax.xaxis.set_major_formatter(ticks_x)
plt.show()
```

# Calculate the Monte Carlo estimates for the price of the option incorporating counterparty risk, given by the default-free price less the CVA.

```python
option_cva_adjusted_prices = []
option_cva_adjusted_std = []

for sample_size, paths in share_price_paths.items():
    payoffs = payoff(paths, K, L)
    option_price = np.exp(-risk_free*T)*payoffs

    term_firm_vals = firm_val_paths[sample_size].iloc[-1].to_numpy()
    amount_lost = np.exp(-risk_free*T)*(1-recovery_rate)*(term_firm_vals < debt)*payoffs

    option_cva_price = option_price - amount_lost

    option_cva_adjusted_prices.append(option_cva_price.mean())
    option_cva_adjusted_std.append(option_cva_price.std()/np.sqrt(sample_size))

```

Credit value adjusted option values at various sample sizes:

```python
for i in range(len(option_cva_adjusted_prices)):
    print("Sample size: {}, CVA-adjusted option value: {:.3f}"
             .format((i+1)*1000, option_cva_adjusted_prices[i]))
```

    Sample size: 1000, CVA-adjusted option value: 5.538
    Sample size: 2000, CVA-adjusted option value: 5.381
    Sample size: 3000, CVA-adjusted option value: 5.897
    Sample size: 4000, CVA-adjusted option value: 5.723
    Sample size: 5000, CVA-adjusted option value: 5.782
    Sample size: 6000, CVA-adjusted option value: 5.860
    Sample size: 7000, CVA-adjusted option value: 5.564
    Sample size: 8000, CVA-adjusted option value: 5.712
    Sample size: 9000, CVA-adjusted option value: 5.813
    Sample size: 10000, CVA-adjusted option value: 5.872
    Sample size: 11000, CVA-adjusted option value: 5.738
    Sample size: 12000, CVA-adjusted option value: 5.589
    Sample size: 13000, CVA-adjusted option value: 5.880
    Sample size: 14000, CVA-adjusted option value: 5.668
    Sample size: 15000, CVA-adjusted option value: 5.775
    Sample size: 16000, CVA-adjusted option value: 5.652
    Sample size: 17000, CVA-adjusted option value: 5.746
    Sample size: 18000, CVA-adjusted option value: 5.701
    Sample size: 19000, CVA-adjusted option value: 5.836
    Sample size: 20000, CVA-adjusted option value: 5.596
    Sample size: 21000, CVA-adjusted option value: 5.639
    Sample size: 22000, CVA-adjusted option value: 5.699
    Sample size: 23000, CVA-adjusted option value: 5.860
    Sample size: 24000, CVA-adjusted option value: 5.720
    Sample size: 25000, CVA-adjusted option value: 5.719
    Sample size: 26000, CVA-adjusted option value: 5.796
    Sample size: 27000, CVA-adjusted option value: 5.855
    Sample size: 28000, CVA-adjusted option value: 5.828
    Sample size: 29000, CVA-adjusted option value: 5.823
    Sample size: 30000, CVA-adjusted option value: 5.753
    Sample size: 31000, CVA-adjusted option value: 5.785
    Sample size: 32000, CVA-adjusted option value: 5.774
    Sample size: 33000, CVA-adjusted option value: 5.733
    Sample size: 34000, CVA-adjusted option value: 5.695
    Sample size: 35000, CVA-adjusted option value: 5.739
    Sample size: 36000, CVA-adjusted option value: 5.717
    Sample size: 37000, CVA-adjusted option value: 5.806
    Sample size: 38000, CVA-adjusted option value: 5.805
    Sample size: 39000, CVA-adjusted option value: 5.766
    Sample size: 40000, CVA-adjusted option value: 5.763
    Sample size: 41000, CVA-adjusted option value: 5.739
    Sample size: 42000, CVA-adjusted option value: 5.706
    Sample size: 43000, CVA-adjusted option value: 5.812
    Sample size: 44000, CVA-adjusted option value: 5.767
    Sample size: 45000, CVA-adjusted option value: 5.701
    Sample size: 46000, CVA-adjusted option value: 5.737
    Sample size: 47000, CVA-adjusted option value: 5.690
    Sample size: 48000, CVA-adjusted option value: 5.682
    Sample size: 49000, CVA-adjusted option value: 5.749
    Sample size: 50000, CVA-adjusted option value: 5.744

```python
# Confidence level as 99.7 gives 3 std. deviations
fig, ax = plt.subplots(figsize=(12,4))

ax.plot(option_cva_adjusted_prices, '.')
ax.plot(option_cva_adjusted_prices + 3 * np.array(option_cva_adjusted_std), 'r')
ax.plot(option_cva_adjusted_prices - 3 * np.array(option_cva_adjusted_std), 'r')
plt.title("CVA-adjusted option prices estimates")
plt.xlabel("Sample size")
plt.ylabel("Option value")

ticks_x = ticker.FuncFormatter(lambda x, pos: '{0:g}'.format(x * 1000))
ax.xaxis.set_major_formatter(ticks_x)
plt.show()
```

##  Analytic calculations for Black-Scholes-Merton for Barrier Options

Let's calculate the price of European call option (up-and-out) using anlytic formulas

```python
def call_option(S, K, T, r, sigma):
    d1 = (np.log(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * np.sqrt(T))
    d2 = d1 - sigma * np.sqrt(T)
    call = (S * stats.norm.cdf(d1, 0.0, 1.0) - K * np.exp(-r * T) * stats.norm.cdf(d2, 0.0, 1.0))
    return call

# call up-and-in
def call_ui_option(S, K, T, r, sigma, H):
    if H > K:
        alambda = (r + 0.5 * sigma ** 2) / sigma ** 2
        x1 = np.log(S/H) / (sigma * np.sqrt(T)) + alambda * sigma * np.sqrt(T)
        y1 = np.log(H/S) / (sigma * np.sqrt(T)) + alambda * sigma * np.sqrt(T)
        y = np.log( H ** 2 / (S * K) ) / (sigma * np.sqrt(T)) + alambda * sigma * np.sqrt(T)

        ui1 = S * stats.norm.cdf(x1, 0.0, 1.0)
        ui2 = -K * np.exp(-r * T) * stats.norm.cdf(x1 - sigma * np.sqrt(T))
        ui3 = -S * ((H/S) ** (2 * alambda)) * ( stats.norm.cdf(-y, 0.0, 1.0) - stats.norm.cdf(-y1, 0.0, 1.0))
        ui4 = K * np.exp(-r * T) * ((H / S ) ** (2 * alambda - 2)) * (
                              stats.norm.cdf(-y + sigma * np.sqrt(T), 0.0, 1.0) -
                              stats.norm.cdf(-y1 + sigma * np.sqrt(T), 0.0, 1.0))
        ui = ui1 + ui2 + ui3 + ui4
    else:
        ui = call_option(S,K,T,r,sigma)

    return ui

def call_uo_option(S, K, T, r, sigma, H):
    if H < K:
        return 0
    else:
        c = call_option(S,K,T,r,sigma)
        ui = call_ui_option(S,K,T,r,sigma,H)
        return c - ui

CALL_UP_AND_OUT=call_uo_option(S0, K, T, risk_free, sigma, L)
print("Price of call European Option with barrier {:.3f}".format(CALL_UP_AND_OUT))
```

    Price of call European Option with barrier 5.313

# Conclusion

We are well aware that Black-Scholes formula was derived under assumptions which might not hold in a real market, as an example, if the share price deviates from the geometric Brownian motion, then our estimates can be wrong by a big margin. However with the assumptions of Black-Scholes-Merton model we were able to price a European Up-and-out Call Option.

On the other hand a small amount of samples in Monte-Carlo simulations may result in wrong values. As an example above, the analytical formula gives **5.31**, while Monter-Carlo gives **~6.61**. But increasing the number of samples _(i.e using 365 days and 100,000 - which requires additional hardware resources)_ shows that value converges to analytical value.

In this paper we used Python code for monthly simulations of the lifetime of the option. Also represented the fact that Monte Carlo estimates are highly dependant on parameters such as: sample size, estimated volatility and etc.

The key factor for CVA is the way how firm value and risk of default are estimated. If firm value is well above debt value and the volatility of the firm is low, the chance of default is really low.

# References
1. Merton, Robert C. "Theory of rational option pricing." The Bell Journal of economics and management science (1973): 141-183.
1. Rich, Don R. "The mathematical foundations of barrier option-pricing theory." Ad- vances in futures and options research 7 (1994).
1. Wong, Hoi Ying, and Yue-Kuen Kwok. "Multi-asset barrier options and occupation time derivatives." Applied Mathematical Finance 10.3 (2003): 245-266.
1. Black, Fisher, and Myron Scholes. "The pricing and Corportate Liabilities." Journal of Political Economy 81 (1973).
1. Yousuf, M. "A fourth-order smoothing scheme for pricing barrier options under stochastic volatility." International Journal of Computer Mathematics 86.6 (2009): 1054-1067.

```python

```

Pricing a European Up-and-out Call Option.ipynb