# ***Introduction to Radar Using Python and MATLAB***
## Andy Harrison - Copyright (C) 2019 Artech House
<br/>

# Infinite Cylinder with Oblique Incidence
***

Referring to Section 7.4.1.3, studies the scattering from infinite length cylinders with oblique incidence, as shown in Figure 7.8.
The two-dimensional bistatic radar cross section for the TMz polarization is given by (Equation 7.28)

$$
    \sigma_{2D} = \frac{2\lambda}{\pi\sin\theta_i}\Bigg| \sum\limits_{n=0}^{\infty} \epsilon_n\frac{J_n(ka\sin\theta_i)}{H_n^{(2)}(ka\sin\theta_i)} \cos(n\, \phi_o)\Bigg|^2 \hspace{0.5in} \text{(m)},
$$

The two-dimensional bistatic radar cross section for the TEz polarization is given by (Equation 7.30)

$$
    \sigma_{2D} = \frac{2\lambda}{\pi\sin\theta_i}\Bigg| \sum\limits_{n=0}^{\infty} \epsilon_n\frac{J_n^\prime(ka\sin\theta_i)}{H_n^{(2)\prime}(ka\sin\theta_i)} \cos(n\, \phi_o)\Bigg|^2 \hspace{0.5in} \text{(m)}.
$$

As in the case of normal incidence, an approximation of the three-dimensional radar cross section from the two-dimensional radar cross section is (Equations 7.31 and 7.32)

\begin{align}
    \sigma_{TM} \approx \frac{4L^2}{\pi}\frac{\sin^2\theta_o}{\sin\theta_i}\, &\Bigg| \sum\limits_{n=0}^{\infty} \epsilon_n\frac{J_n(ka\sin\theta_i)}{H_n^{(2)}(ka\sin\theta_i)} \cos(n\, \phi_o)\Bigg|^2 \nonumber \\ \nonumber \\ &\times \Bigg[ \frac{\sin\big(0.5kL(\cos\theta_i + \cos\theta_o)\big)}{0.5kL(\cos\theta_i + \cos\theta_o)} \Bigg]^2\hspace{0.5in} \text{(m}^2\text{)},
\end{align}

and

\begin{align}
    \sigma_{TE} \approx \frac{4L^2}{\pi}\sin\theta_i\, &\Bigg| \sum\limits_{n=0}^{\infty} \epsilon_n\frac{J_n^\prime(ka\sin\theta_i)}{H_n^{(2)\prime}(ka\sin\theta_i)} \cos(n\, \phi_o)\Bigg|^2 \nonumber \\ \nonumber \\ &\times \Bigg[ \frac{\sin\big(0.5kL(\cos\theta_i + \cos\theta_o)\big)}{0.5kL(\cos\theta_i + \cos\theta_o)} \Bigg]^2\hspace{0.5in} \text{(m}^2\text{)}.
\end{align}
***

Begin by getting the library path

```python
import lib_path
```

Set the operating frequency (Hz), the radius (m), the length (for 3D RCS) (m), the number of modes and the incident angle (from the z-axis) (degrees)

```python
frequency = 10e9

radius = 0.5

length = 20.0

number_of_modes = 30

incident_angle = 20.0
```

Select either 2D or 3D RCS

```python
mode = '2D'
```

Set the observation angles with the `linspace` routine from `scipy`

```python
from numpy import linspace

observation_angle = linspace(-90, 90, 1801)
```

Calculate the radar cross section based on the mode

```python
from Libs.rcs.infinite_cylinder_oblique import radar_cross_section, radar_cross_section_3d

if mode == '2D':

    rcs_te, rcs_tm = radar_cross_section(frequency, radius, incident_angle, observation_angle, number_of_modes)

else:

    rcs_te, rcs_tm = radar_cross_section_3d(frequency, radius, incident_angle, observation_angle, number_of_modes, length)
```

Display the RCS for the infinite cylinder using the `matplotlib` routines

```python
from matplotlib import pyplot as plt

from numpy import log10, degrees

# Set the figure size

plt.rcParams["figure.figsize"] = (15, 10)

# Display the results

plt.plot(observation_angle, 10.0 * log10(rcs_te), '', label='TE$^z$')

plt.plot(observation_angle, 10.0 * log10(rcs_tm), '--', label='TM$^z$')

# Set the plot title and labels

plt.title('RCS vs Bistatic Angle', size=14)

plt.ylabel('RCS (dBsm)', size=12)

plt.xlabel('Observation Angle (deg)', size=12)

plt.ylim(min(10.0 * log10(rcs_te)) - 3, max(10.0 * log10(rcs_te)) + 3)

# Set the tick label size

plt.tick_params(labelsize=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the legend

plt.legend(loc='upper right', prop={'size': 10})
```

```python

```


# Examples on deflections of the beams
보의 처짐에 응용한 사례

ref : Pytel et. al, Mechanics of Materials, 2nd Ed, SI, Ex. 6.9 and Tables 6.2 & 6.3, Cengage Learning, 2011.

```python
import pylab as py
import scipy.optimize as so
import sympy as sym
import sympy.utilities as su

```

```python
sym.init_printing()

```

## Constants for the material & the section
재료와 단면에 관한 상수

$$
\begin{align}
    EI&=20.48\times 10^3 N \cdot m^2 \\
    w_o&=100N/m \\
    L&=1m
\end{align}
$$

## 분포하중 외팔보
Distributed load cantilever

### 질문
Question

The deflection of a uniform distributed load cantilver is as follows.

균일 분포하중 외팔보의 처짐은 다음과 같다.

$$
\begin{align}
\delta(x) &= \frac{w_o x^2}{24EI}\left(6L^2 - 4Lx + x^2 \right) \\
\delta(L) &= \frac{w_o L^4}{8EI}
\end{align}
$$

Using following parameters, find $x$ such that its deflection $\delta(x)$ is the half of the deflection at $x=L$.

다음 매개변수들을 사용하여 처짐이 $x=L$ 에서의 최대 처짐의 절반인 $x$(m)를 구하시오.

### 풀이 예
One possible answer

$$
\begin{align}
\delta(x) &= \frac{1}{2}\delta(L) \\
\frac{w_o x^2}{24EI}\left(6L^2 - 4Lx + x^2 \right) &= \frac{1}{2}\frac{w_o L^4}{8EI} \\
\frac{x^2}{3}\left(6L^2 - 4Lx + x^2 \right) &= \frac{1}{2}L^4
\end{align}
$$

$$
\frac{x^2}{3}\left(6L^2 - 4Lx + x^2 \right) - \frac{1}{2}L^4 = 0
$$

```python
L_m = 1

```

$g(x)=0 $

```python
def g(x_m):
    return (x_m ** 2) * (6 * L_m * L_m - 4 * L_m * x_m + x_m ** 2) / 3. - 0.5 * L_m ** 4

```

이분법

Bisection Method

```python
%%time
x_bis_m = so.bisect(g, a=0, b=L_m, xtol=1e-3)
print(x_bis_m)

```

뉴튼랩슨법

Newotn Raphson Method

$$
\begin{align}
g(x) &= \frac{x^2}{3}\left(6L^2 - 4Lx + x^2 \right) - \frac{1}{2}L^4 \\
&= \left(6L^2\frac{x^2}{3} - 4Lx\frac{x^2}{3} + x^2\frac{x^2}{3} \right) - \frac{1}{2}L^4 \\
&= \frac{1}{3}x^4 - \frac{4L}{3}x^3 + 2L^2x^2 - \frac{1}{2}L^4 \\
\frac{d}{dx}g(x) &= \frac{4}{3}x^3 - 4Lx^2 + 4L^2x
\end{align}
$$

```python
def dg(x_m):
    return 4./3 * x_m ** 3 - 4 * L_m * x_m ** 2 + 4 * L_m ** 2 * x_m

```

```python
%%time
x_nr_m = so.newton(g, .1, fprime=dg, tol=1e-3)
print(x_nr_m)

```

## Simply supported beam under a concentrated load at its center
중점에 집중하중을 받는 단순지지보

### 질문
Question

Find $x$(m) such that its deflection $\delta(x)$ is the half of the maximum deflection at $x=\frac{L}{2}$.

처짐 $\delta(x)$ 가 $x=\frac{L}{2}$ 에서의 최대 처짐의 절반인 $x$(m)를 구하시오.

### 풀이 예
One possible answer

Prepare symbols to use
사용할 기호를 준비

```python
P, x, EI, L = sym.symbols("P x EI L", real=True, nonnegative=True)

```

Deflection $\delta(x)$ within the interval $ 0 \le x \le \frac{L}{2}$
$ 0 \le x \le \frac{L}{2}$ 구간에서의 처짐 $\delta(x)$

```python
delta = (P * x)  * (3 * L * L  - 4 * x * x) / (48 * EI)
delta

```

Maximum deflection at the center $\delta\left( \frac{L}{2} \right)$

가운데 지점에서의 최대 처짐$\delta\left( \frac{L}{2} \right)$

```python
delta_max = delta.subs({x: L/2})
delta_max

```

Equation to solve
풀고자 하는 방정식

```python
eq_p = sym.Eq(delta, delta_max / 2)
eq_p

```

Cancellation
약분

```python
factor = 48 * 2 * EI / P
eq_p = sym.Eq(eq_p.lhs * factor, eq_p.rhs * factor)
eq_p

```

Transposition
이항

```python
h_x = eq_p.lhs - eq_p.rhs
h_x

```

Substituting $L=1$, generating a python function
$L=1$을 대입하면서 파이썬 함수 생성

```python
h = sym.lambdify(x, h_x.subs({L:1}))

```

#### Bisection Method
이분법

```python
%%time
x_bis_m = so.bisect(h, a=0, b=L_m*0.5, xtol=1e-3)
print(x_bis_m)

```

#### Newotn Raphson Method
뉴튼 랩슨법

Differentiate over $x$
$x$에 대해 미분

```python
dh_dx = h_x.diff(x)
dh_dx

```

Substituting $L=1$, generating a python function
$L=1$을 대입하면서 파이썬 함수 생성

```python
dh = sym.lambdify(x, dh_dx.subs({L:1}))

```

```python
%%time
x_nr_m = so.newton(h, .01, fprime=dh, tol=1e-3)
print(x_nr_m)

```

## Final Bell
마지막 종

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, C.D. Cooper, G.F. Forsyth.

# Spreading out

Welcome to the fifth, and last, notebook of Module 4 "_Spreading out: diffusion problems,"_ of our fabulous course **"Practical Numerical Methods with Python."**

In this course module, we have learned about explicit and implicit methods for parabolic equations in 1 and 2 dimensions. So far, all schemes have been first-order in time and second-order in space. _Can we do any better?_ We certainly can: this notebook presents the Crank-Nicolson scheme, which is a second-order method in both time and space! We will continue to use the heat equation to guide the discussion, as we've done throughout this module.

## Crank-Nicolson scheme

The [Crank Nicolson scheme](http://en.wikipedia.org/wiki/Crank–Nicolson_method) is a popular second-order, implicit method used with parabolic PDEs in particular. It was developed by John Crank and [Phyllis Nicolson](http://en.wikipedia.org/wiki/Phyllis_Nicolson). The main idea is to take the average between the solutions at $t^n$ and $t^{n+1}$ in the evaluation of the spatial derivative. Why bother doing that? Because the time derivative will then be discretized with a centered scheme, giving second-order accuracy!

Remember the 1D heat equation from the [first notebook](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_01_Heat_Equation_1D_Explicit.ipynb)? Just to refresh your memory, here it is:

$$
\begin{equation}
    \frac{\partial T}{\partial t} = \alpha \frac{\partial^2 T}{\partial x^2}.
\end{equation}
$$

In this case, the Crank-Nicolson scheme leads to the following discretized equation:

$$
\begin{equation}
    \begin{split}
        & \frac{T^{n+1}_i - T^n_i}{\Delta t} = \\
        & \quad \alpha \cdot \frac{1}{2} \left( \frac{T^{n+1}_{i+1} - 2 T^{n+1}_i + T^{n+1}_{i-1}}{\Delta x^2} + \frac{T^n_{i+1} - 2 T^n_i + T^n_{i-1}}{\Delta x^2} \right) \\
    \end{split}
\end{equation}
$$

Notice how the both time indices $n$ and $n+1$ appear on the right-hand side. You know we'll have to rearrange this equation, right? Now look at the stencil and notice that we are using more information than before in the update.

#### Figure 2. Stencil of the Crank-Nicolson scheme.

Rearranging terms so that everything that we don't know is on the left side and what we do know on the right side, we get

$$
\begin{equation}
    \begin{split}
        & -T^{n+1}_{i-1} + 2 \left( \frac{\Delta x^2}{\alpha \Delta t} + 1 \right) T^{n+1}_i - T^{n+1}_{i+1} \\
        & \qquad = T^{n}_{i-1} + 2 \left( \frac{\Delta x^2}{\alpha \Delta t} - 1 \right) T^{n}_i + T^{n}_{i+1} \\
    \end{split}
\end{equation}
$$

Again, we are left with a linear system of equations. Check out the left side of that equation: it looks a lot like the matrix from [notebook 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_02_Heat_Equation_1D_Implicit.ipynb), doesn't it? Apart from the slight modification in the $T_i^{n+1}$ term, the left side of the equation is pretty much the same. What about the right-hand side? Sure, it looks quite different, but that is not a problem, we know all those terms!

Things don't change much for boundary conditions, either. We've seen all the cases already. Say $T_0^{n+1}$ is a Dirichlet boundary. Then the equation for $i=1$ becomes

$$
\begin{equation}
    \begin{split}
         & 2 \left( \frac{\Delta x^2}{\alpha \Delta t} + 1 \right) T^{n+1}_1 - T^{n+1}_{2} \\
         & \qquad = T^{n}_{0} + 2 \left( \frac{\Delta x^2}{\alpha \Delta t} - 1 \right) T^{n}_1 + T^{n}_{2} + T^{n+1}_{0} \\
    \end{split}
\end{equation}
$$

And if we have a Neumann boundary $\left(\left.\frac{\partial T}{\partial x}\right|_{x=L} = q\right)$ at $T_{n_x-1}^{n+1}$? We know this stuff, right? For $i=n_x-2$ we get

$$
\begin{equation}
    \begin{split}
        & -T^{n+1}_{n_x-3} + \left( 2 \frac{\Delta x^2}{\alpha \Delta t} + 1 \right) T^{n+1}_{n_x-2} \\
        & \qquad = T^{n}_{n_x-3} + 2 \left( \frac{\Delta x^2}{\alpha \Delta t} - 1 \right) T^{n}_{n_x-2} + T^{n}_{n_x-1} + q\Delta x \\
    \end{split}
\end{equation}
$$

The code will look a lot like the implicit method from the [second notebook](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_02_Heat_Equation_1D_Implicit.ipynb). Only some terms of the matrix and right-hand-side vector will be different, which changes some of our custom functions.

### The linear system

Just like in [notebook 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_02_Heat_Equation_1D_Implicit.ipynb), we need to solve a linear system on every time step of the form:

$$
[A][T^{n+1}_\text{int}] = [b]+[b]_{b.c.}
$$

The coefficient matrix is very similar to the previous case, but the right-hand side changes a lot:

$$
\begin{align}
    \left[
        \begin{array}{cccccc}
             2 \left( \frac{1}{\sigma} + 1 \right) & -1 & 0 & \cdots & & 0 \\
             -1 & 2 \left( \frac{1}{\sigma} + 1\right) & -1 & 0 & \cdots & 0 \\
             0 & & \ddots & & & \vdots \\
             \vdots & & & & 2 \left( \frac{1}{\sigma} + 1\right) & \\
             0 & \cdots & & & -1 & \left( 2 \frac{1}{\sigma} + 1\right) \\
        \end{array}
    \right] \cdot
    \left[
        \begin{array}{c}
             T_1^{n+1} \\
             T_2^{n+1} \\
             \vdots \\
             \\
             T_{N-2}^{n+1} \\
        \end{array}
    \right] =
    \left[
        \begin{array}{c}
            T_0^n + 2 \left( \frac{1}{\sigma} - 1 \right) T_1^n + T_2^n \\
            T_1^n + 2 \left( \frac{1}{\sigma} - 1 \right) T_2^n + T_3^n \\
            \vdots \\
            \\
            T_{n_x-3}^n + 2 \left( \frac{1}{\sigma} - 1 \right) T_{n_x-2}^n + T_{n_x-1}^n \\
        \end{array}
    \right] +
    \begin{bmatrix}
        T_0^{n+1} \\
        0\\
        \vdots \\
        0 \\
        q \Delta x \\
    \end{bmatrix}
\end{align}
$$

Let's write a function that will create the coefficient matrix and right-hand-side vectors for the heat conduction problem from [notebook 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_02_Heat_Equation_1D_Implicit.ipynb): with Dirichlet boundary at $x=0$ and zero-flux boundary $(q=0)$ at $x=L$.

```python
import numpy
from scipy import linalg
```

```python
def lhs_operator(N, sigma):
    """
    Computes and returns the implicit operator
    of the system for the 1D diffusion equation.
    We use Crank-Nicolson method, Dirichlet condition
    on the left side of the domain and zero-gradient
    Neumann condition on the right side.

    Parameters
    ----------
    N : integer
        Number of interior points.
    sigma : float
        Value of alpha * dt / dx**2.

    Returns
    -------
    A : numpy.ndarray
        The implicit operator as a 2D array of floats
        of size N by N.
    """
    # Setup the diagonal of the operator.
    D = numpy.diag(2.0 * (1.0 + 1.0 / sigma) * numpy.ones(N))
    # Setup the Neumann condition for the last element.
    D[-1, -1] = 1.0 + 2.0 / sigma
    # Setup the upper diagonal of the operator.
    U = numpy.diag(-1.0 * numpy.ones(N - 1), k=1)
    # Setup the lower diagonal of the operator.
    L = numpy.diag(-1.0 * numpy.ones(N - 1), k=-1)
    # Assemble the operator.
    A = D + U + L
    return A
```

```python
def rhs_vector(T, sigma, qdx):
    """
    Computes and returns the right-hand side of the system
    for the 1D diffusion equation, using a Dirichlet condition
    on the left side and a Neumann condition on the right side.

    Parameters
    ----------
    T : numpy.ndarray
        The temperature distribution as a 1D array of floats.
    sigma : float
        Value of alpha * dt / dx**2.
    qdx : float
        Value of the temperature flux at the right side.

    Returns
    -------
    b : numpy.ndarray
        The right-hand side of the system as a 1D array of floats.
    """
    b = T[:-2] + 2.0 * (1.0 / sigma - 1.0) * T[1:-1] + T[2:]
    # Set Dirichlet condition.
    b[0] += T[0]
    # Set Neumann condition.
    b[-1] += qdx
    return b
```

We will solve the linear system at every time step. Let's define a function to step in time:

```python
def crank_nicolson(T0, nt, dt, dx, alpha, q):
    """
    Computes and returns the temperature along the rod
    after a given number of time steps.

    The function uses Crank-Nicolson method in time,
    central differencing in space, a Dirichlet condition
    on the left side, and a Neumann condition on the
    right side.

    Parameters
    ----------
    T0 : numpy.ndarray
        The initial temperature distribution as a 1D array of floats.
    nt : integer
        Number of time steps to compute.
    dt : float
        Time-step size.
    dx : float
        Distance between two consecutive locations.
    alpha : float
        Thermal diffusivity of the rod.
    q : float
        Value of the temperature gradient on the right side.

    Returns
    -------
    T : numpy.ndarray
        The temperature distribution as a 1D array of floats.
    """
    sigma = alpha * dt / dx**2
    # Create the implicit operator of the system.
    A = lhs_operator(len(T0) - 2, sigma)
    # Integrate in time.
    T = T0.copy()
    for n in range(nt):
        # Generate the right-hand side of the system.
        b = rhs_vector(T, sigma, q * dx)
        # Solve the system with scipy.linalg.solve.
        T[1:-1] = linalg.solve(A, b)
        # Apply the Neumann boundary condition.
        T[-1] = T[-2] + q * dx
    return T
```

And we are good to go! First, let's setup our initial conditions, and the matrix

```python
# Set parameters.
L = 1.0  # length of the rod
nx = 21  # number of points on the rod
dx = L / (nx - 1)  # grid spacing
alpha = 1.22e-3  # thermal diffusivity of the rod
q = 0.0  # temperature gradient at the extremity

# Define the locations on the rod.
x = numpy.linspace(0.0, L, num=nx)

# Set the initial temperature distribution.
T0 = numpy.zeros(nx)
T0[0] = 100.0
```

Check the matrix...

```python
A = lhs_operator(nx - 1, 0.5)
print(A)
```

    [[ 6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [-1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.  0.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6. -1.
       0.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.  6.
      -1.  0.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. -1.
       6. -1.]
     [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
      -1.  5.]]

Looks okay! Now, step in time

```python
# Set the time-step size based on CFL limit.
sigma = 0.5
dt = sigma * dx**2 / alpha  # time-step size
nt = 10  # number of time steps to compute

# Compute the temperature distribution.
T = crank_nicolson(T0, nt, dt, dx, alpha, q)
```

And plot,

```python
from matplotlib import pyplot
%matplotlib inline
```

```python
# Set the font family and size to use for Matplotlib figures.
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
```

```python
# Plot the temperature along the rod.
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('Distance [m]')
pyplot.ylabel('Temperature [C]')
pyplot.grid()
pyplot.plot(x, T, color='C0', linestyle='-', linewidth=2)
pyplot.xlim(0.0, L)
pyplot.ylim(0.0, 100.0);
```

Works nicely. But wait! This method has elements of explicit and implicit discretizations. Is it *conditionally stable* like forward Euler, or *unconditionally stable* like backward Euler? Try out different values of `sigma`. You'll see Crank-Nicolson is an *unconditionally stable scheme* for the diffusion equation!

## Accuracy & convergence

Using some techniques you might have learned in your PDE class, such as separation of variables, you can get a closed expression for the rod problem. It looks like this:

$$
\begin{eqnarray}
T(x,t) = & \nonumber \\
100 - \sum_{n=1}^{\infty} & \frac{400}{(2n-1)\pi}\sin\left(\frac{(2n-1)\pi}{2L}x\right) \exp\left[-\alpha\left(\frac{(2n-1)\pi}{2L}\right)^2t\right]
\end{eqnarray}
$$

Unfortunately, the analytical solution is a bit messy, but at least it gives a good approximation if we evaluate it for large $n$. Let's define a function that will calculate this for us:

```python
def analytical_temperature(x, t, alpha, L, N):
    """
    Computes and returns a truncated approximation
    of the exact temperature distribution along the rod.

    Parameters
    ----------
    x : numpy.ndarray
        Locations at which to calculate the temperature
        as a 1D array of floats.
    t : float
        Time.
    alpha : float
        Thermal diffusivity of the rod.
    L : float
        Length of the rod.
    N : integer
        Number of terms to use in the expansion.

    Returns
    -------
    T : numpy.ndarray
        The truncated analytical temperature distribution
        as a 1D array of floats.
    """
    T = 100.0 * numpy.ones_like(x)
    for n in range(1, N + 1):
        k = (2 * n - 1) * numpy.pi / (2.0 * L)
        T -= (400.0 / (2.0 * L * k) *
              numpy.sin(k * x) * numpy.exp(- alpha * k**2 * t))
    return T
```

And let's see how that expression looks for the time where we left the numerical solution

```python
# Compute the analytical temperature distribution.
T_exact = analytical_temperature(x, nt * dt, alpha, L, 100)

# Plot the numerical and analytical temperatures.
pyplot.figure(figsize=(6.0, 4.0))
pyplot.xlabel('Distance [m]')
pyplot.ylabel('Temperature [C]')
pyplot.grid()
pyplot.plot(x, T, label='numerical',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_exact, label='analytical',
            color='C1', linestyle='--', linewidth=2)
pyplot.legend()
pyplot.xlim(0.0, L)
pyplot.ylim(0.0, 100.0);
```

```python
T1 = analytical_temperature(x, 0.2, alpha, L, 100)
T2 = analytical_temperature(x, 0.2, alpha, L, 200)
numpy.sqrt(numpy.sum((T1 - T2)**2) / numpy.sum(T2**2))
```

    6.927917118260093e-13

That looks like it should. We'll now use this result to study the convergence of the Crank-Nicolson scheme.

### Time convergence

We said this method was second-order accurate in time, remember? That's in theory, but we should test that the numerical solution indeed behaves like the theory says.

Leaving $\Delta x$ constant, we'll run the code for different values of $\Delta t$ and compare the result at the same physical time, say $t=n_t\cdot\Delta t=10$, with the analytical expression above.

The initial condition of the rod problem has a very sharp gradient: it suddenly jumps from $0{\rm C}$ to $100{\rm C}$ at the boundary. To resolve that gradient to the point that it doesn't affect time convergence, we would need a very fine mesh, and computations would be very slow. To avoid this issue, we will start from $t=1$ rather than starting from $t=0$.

First, let's define a function that will compute the $L_2$-norm of the error:

```python
def l2_error(T, T_exact):
    """
    Computes and returns the relative L2-norm
    of the difference between the numerical solution
    and the exact solution.

    Parameters
    ----------
    T : numpy.ndarray
        The numerical solution as an array of floats.
    T_exact : numpy.ndarray
        The exact solution as an array of floats.

    Returns
    -------
    error : float
        The relative L2-norm of the difference.
    """
    error = numpy.sqrt(numpy.sum((T - T_exact)**2) /
                       numpy.sum(T_exact**2))
    return error
```

For fun, let's compare the Crank-Nicolson scheme with the implicit (a.k.a., backward) Euler scheme. We'll borrow some functions from [notebook 2](https://nbviewer.jupyter.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/04_spreadout/04_02_Heat_Equation_1D_Implicit.ipynb) to do this.

```python
def lhs_operator_btcs(N, sigma):
    """
    Computes and returns the implicit operator
    of the system for the 1D diffusion equation.
    We use backward Euler method, Dirichlet condition
    on the left side of the domain and zero-gradient
    Neumann condition on the right side.

    Parameters
    ----------
    N : integer
        Number of interior points.
    sigma : float
        Value of alpha * dt / dx**2.

    Returns
    -------
    A : numpy.ndarray
        The implicit operator as a 2D array of floats
        of size N by N.
    """
    # Setup the diagonal of the operator.
    D = numpy.diag((2.0 + 1.0 / sigma) * numpy.ones(N))
    # Setup the Neumann condition for the last element.
    D[-1, -1] = 1.0 + 1.0 / sigma
    # Setup the upper diagonal of the operator.
    U = numpy.diag(-1.0 * numpy.ones(N - 1), k=1)
    # Setup the lower diagonal of the operator.
    L = numpy.diag(-1.0 * numpy.ones(N - 1), k=-1)
    # Assemble the operator.
    A = D + U + L
    return A
```

```python
def rhs_vector_btcs(T, sigma, qdx):
    """
    Computes and returns the right-hand side of the system
    for the 1D diffusion equation, using a Dirichlet condition
    on the left side and a Neumann condition on the right side.

    Parameters
    ----------
    T : numpy.ndarray
        The temperature distribution as a 1D array of floats.
    sigma : float
        Value of alpha * dt / dx**2.
    qdx : float
        Value of the temperature flux at the right side.

    Returns
    -------
    b : numpy.ndarray
        The right-hand side of the system as a 1D array of floats.
    """
    b = T[1:-1] / sigma
    # Set Dirichlet condition.
    b[0] += T[0]
    # Set Neumann condition.
    b[-1] += qdx
    return b
```

```python
def btcs_implicit(T0, nt, dt, dx, alpha, q):
    """
    Computes and returns the temperature along the rod
    after a given number of time steps.

    The function uses Euler implicit in time,
    central differencing in space, a Dirichlet condition
    on the left side, and a Neumann condition on the
    right side.

    Parameters
    ----------
    T0 : numpy.ndarray
        The initial temperature distribution
        as a 1D array of floats.
    nt : integer
        Number of time steps to compute.
    dt : float
        Time-step size.
    dx : float
        Distance between two consecutive locations.
    alpha : float
        Thermal diffusivity of the rod.
    q : float
        Value of the temperature gradient on the right side.

    Returns
    -------
    T : numpy.ndarray
        The temperature distribution as a 1D array of floats.
    """
    sigma = alpha * dt / dx**2
    # Create the implicit operator of the system.
    A = lhs_operator_btcs(len(T0) - 2, sigma)
    # Integrate in time.
    T = T0.copy()
    for n in range(nt):
        # Generate the right-hand side of the system.
        b = rhs_vector_btcs(T, sigma, q * dx)
        # Solve the system with scipy.linalg.solve.
        T[1:-1] = linalg.solve(A, b)
        # Apply the Neumann boundary condition.
        T[-1] = T[-2] + q * dx
    return T
```

Now, let's do the runs!

```python
# Update parameters.
nx = 1001  # number of points on the rod
dx = L / (nx - 1)  # grid spacing

# Define the locations on the rod.
x = numpy.linspace(0.0, L, num=nx)

# Create a list with the time-step sizes to use.
dt_values = [1.0, 0.5, 0.25, 0.125]

# Create empty lists to hold the errors for both schemes.
errors = []
errors_btcs = []

# Compute the initial temperature distribution at t=1.0.
t0 = 1.0
T0 = analytical_temperature(x, t0, alpha, L, 100)

# Compute the final analytical temperature at t=10.0.
t = 10.0
T_exact = analytical_temperature(x, t, alpha, L, 100)

# Compute the numerical solutions and errors.
for dt in dt_values:
    nt = int((t - t0) / dt)  # number of time steps
    # Compute the solution using Crank-Nicolson scheme.
    T = crank_nicolson(T0, nt, dt, dx, alpha, q)
    # Compute and record the L2-norm of the error.
    errors.append(l2_error(T, T_exact))
    # Compute the solution using implicit BTCS scheme.
    T = btcs_implicit(T0, nt, dt, dx, alpha, q)
    # Compute and record the L2-norm of the error.
    errors_btcs.append(l2_error(T, T_exact))
```

And plot,

```python
# Plot the error versus the time-step size.
pyplot.figure(figsize=(6.0, 6.0))
pyplot.grid()
pyplot.xlabel(r'$\Delta t$')
pyplot.ylabel('Relative $L_2$-norm\nof the error')
pyplot.loglog(dt_values, errors, label='Crank-Nicolson',
              color='black', linestyle='--', linewidth=2, marker='o')
pyplot.loglog(dt_values, errors_btcs, label='BTCS (implicit)',
              color='black', linestyle='--', linewidth=2, marker='s')
pyplot.legend()
pyplot.axis('equal');
```

```python
errors
```

    [0.0005562525604218684,
     0.0001374575644793469,
     3.285170428405964e-05,
     6.771647468538648e-06]

See how the error drops four times when the time step is halved? This method is second order in time!

Clearly, Crank-Nicolson (circles) converges faster than backward Euler (squares)! Not only that, but also the error curve is shifted down: Crank-Nicolson is more accurate.

If you look closely, you'll realize that the error in Crank-Nicolson decays about twice as fast than backward Euler: it's a second versus first order method!

### Spatial convergence

To study spatial convergence, we will run the code for meshes with 21, 41, 81 and 161 points, and compare them at the same non-dimensional time, say $t=20$.

Let's start by defining a function that will do everything for us

```python
# Set parameters.
dt = 0.1  # time-step size
t = 20.0  # final time
nt = int(t / dt)  # number of time steps to compute

# Create a list with the grid-spacing sizes to use.
nx_values = [11, 21, 41, 81, 161]

# Create an empty list to store the errors.
errors = []

# Compute the numerical solutions and errors.
for nx in nx_values:
    dx = L / (nx - 1)  # grid spacing
    x = numpy.linspace(0.0, L, num=nx)  # grid points
    # Set the initial conditions for the grid.
    T0 = numpy.zeros(nx)
    T0[0] = 100.0
    # Compute the solution using Crank-Nicolson scheme.
    T = crank_nicolson(T0, nt, dt, dx, alpha, q)
    # Compute the analytical solution.
    T_exact = analytical_temperature(x, t, alpha, L, 100)
    # Compute and record the L2-norm of the error.
    errors.append(l2_error(T, T_exact))
```

And plot!

```python
# Plot the error versus the grid-spacing size.
pyplot.figure(figsize=(6.0, 6.0))
pyplot.grid()
pyplot.xlabel(r'$\Delta x$')
pyplot.ylabel('Relative $L_2$-norm\nof the error')
dx_values = L / (numpy.array(nx_values) - 1)
pyplot.loglog(dx_values, errors,
              color='black', linestyle='--', linewidth=2, marker='o')
pyplot.axis('equal');
```

That looks good! See how for each quadrant we go right, the error drops two quadrants going down (and even a bit better!).

##### Dig deeper

Let's re-do the spatial convergence, but comparing at a much later time, say $t=1000$.

```python
# Set parameters.
dt = 0.1  # time-step size
t = 1000.0  # final time
nt = int(t / dt)  # number of time steps to compute

# Create a list with the grid-spacing sizes to use.
nx_values = [11, 21, 41, 81, 161]

# Create an empty list to store the errors.
errors = []

# Compute the numerical solutions and errors.
for nx in nx_values:
    dx = L / (nx - 1)  # grid spacing
    x = numpy.linspace(0.0, L, num=nx)  # grid points
    # Set the initial conditions for the grid.
    T0 = numpy.zeros(nx)
    T0[0] = 100.0
    # Compute the solution using Crank-Nicolson scheme.
    T = crank_nicolson(T0, nt, dt, dx, alpha, q)
    # Compute the analytical solution.
    T_exact = analytical_temperature(x, t, alpha, L, 100)
    # Compute and record the L2-norm of the error.
    errors.append(l2_error(T, T_exact))
```

```python
# Plot the error versus the grid-spacing size.
pyplot.figure(figsize=(6.0, 6.0))
pyplot.grid()
pyplot.xlabel(r'$\Delta x$')
pyplot.ylabel('Relative $L_2$-norm\nof the error')
dx_values = L / (numpy.array(nx_values) - 1)
pyplot.loglog(dx_values, errors,
              color='black', linestyle='--', linewidth=2, marker='o')
pyplot.axis('equal');
```

```python
errors
```

    [0.011922719076357474,
     0.006181593859790544,
     0.003142664307189285,
     0.0015838621626866334,
     0.0007950070915380142]

Wait, convergence is not that great now! It's not as good as second order, but not as bad as first order. *What is going on?*

Remember our implementation of the boundary conditions? We used

$$
\begin{equation}
\frac{T^{n}_{N-1} - T^{n}_{N-2}}{\Delta x} = q
\end{equation}
$$

Well, that is a **first-order** approximation!

But, why doesn't this affect our solution at an earlier time? Initially, temperature on the right side of the rod is zero and the gradient is very small in that region; at that point in time, errors there were negligible. Once temperature starts picking up, we start having problems.

**Boundary conditions can affect the convergence and accuracy of your solution!**

---
###### The cell below loads the style of the notebook

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Source+Code+Pro' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "Source Code Pro";
    font-size: 90%;
}

.alert-box {
    padding:10px 10px 10px 36px;
    margin:5px;
}

.success {
    color:#666600;
    background:rgb(240,242,229);
}
</style>


# "Monte Carlo Methods 2"
> "In this blog post we continue to look at Monte Carlo methods and how they can be used. We move from sampling from univariate distributions to multi-variate distributions."
- toc: true
- author: Lewis Cole (2020)
- branch: master
- badges: false
- comments: false
- categories: [Monte-Carlo, Statistics, Probability, Computational-Statistics, Theory, Computation, Copula]
- hide: false
- search_exclude: false
- image: https://github.com/lewiscoleblog/blog/raw/master/images/Monte-Carlo/copula.png

___
This is the second blog post in a series - you can find the previous blog post [here](https://lewiscoleblog.com/monte-carlo-methods)
___

In this blog post we shall continue our exploration of Monte Carlo methods. To briefly recap in the previous blog post we looked at the general principle underlying Monte-Carlo methods, we looked at methods used by pseudo-random number generators to feed our models and we looked at a variety of methods to convert these uniform variates to general univariate distributions.

## Multi-Variate Relations
We start by looking at the problem of sampling from general multi-variate distributions. In many instances simply sampling univariate distributions will be insufficient. For example if we are creating a model that looks of mortgage repayment defaults we do not want to model the default rate by one univariate distribution and the interest rate by another univariate distribution using the techniques described so far. In an ideal world we would know the exact relationship we are trying to model, in physics and the hard sciences the exact mechanism underlying the joint behaviour may be modelled exactly. However in many cases in finance (and the social sciences) this is very hard to elicit (if not impossible) so we want to create "correlated" samples that capture (at least qualitatively) some of the joint behaviour.

### Measures of Correlation
We start by looking at some measures of dependance between variables. This helps us evaluate whether a model is working as we would expect or not. A lot of times joint-dependence is difficult to elicit from the data so often with this sort of model there is a lot of expert judgements, sensitivity testing, back testing and so on in order to calibrate a model. This is out of the scope of what I'm looking to cover in this blog post series but is worth keeping in mind for this section on multi-variate methods.

By far the most prevelant measure of dependance is the Pearson correlation coefficient. This is the first (and in many cases only) measure of depedence we find in textbooks. We can express it algebraically as:
$$ \rho_{(X,Y)} = \frac{\mathbb{E}[(X - \mathbb{E}(X))(Y - \mathbb{E}(Y))]}{\sqrt{\mathbb{E}[(X - \mathbb{E}(X))^2]\mathbb{E}[(Y - \mathbb{E}(Y))^2]}} = \frac{Cov(X,Y)}{SD(X)SD(Y)}$$

That is the ratio of the covariance between two variables divide by the product of their standard deviations. This measure has a number of useful properties - it gives a metric between $[-1,1]$ (via the Cauchy-Schwarz inequality) which varies from "completely anti-dependant" to "completely-dependant". It is also mathematically very tractable, it is very easy to calculate and it often crops up when performing an analytic investigation. However it's use in "the real world" is fairly limited, it is perhaps the most abused of all statistics.

One issue with this measure in practice is that it requires defined first and second moments for the definition to work. We can calculate this statistic on samples from any distribution, however if we are in the realms of fat-tailed distributions (such as a power-law) this sample estimate will be meaningless. In finance and real-world risk applications this is a big concern, it means that many of the early risk management models still being used that do not allow for the possiblity of fat-tails are at best not-useful and at worst highly dangerous for breeding false confidence.

Another issue with the measure is it is highly unintuitive. Given the way it is defined the difference between $\rho=0.05$ and $\rho = 0.1$ is negligible, yet the difference between $\rho = 0.94$ and $\rho = 0.99$ is hugely significant. In this authors experience even very "technical" practitioners fail to remember this and treat a "5% point increase" as having a consistent impact. This issue is compounded further when dealing with subject matter experts who do not necessarily have mathematical/probability training but have an "intuitive" grasp of "correlation"!

The biggest and most limiting factor for the Pearson coefficient however is that it only considers linear relationships between variables - any relation that is not linear will effectively be approximated linearly. In probability distribution terms this is equivalent to a joint-normal distribution. That is to say: **Pearson correlation only exhibits good properties in relation to joint-normal behaviour!**

Another option to use is the Spearmann correlation coefficient. This is strongly related to the Pearson correlation coefficient but with one important difference: instead of using the values themselves we work with the percentiles instead. For example suppose we have a standard normal distribution and we sample the point: $x = 1.64...$ we know that this is the 95th percentile of the standard normal CDF so we would use the value: $y=0.95$ in the definition of Pearson correlation above. This has the benefit that it doesn't matter what distributions we use we will still end up with a reasonable metric of dependance. In the case of joint-normality we will have that the Pearson and Spearmann coefficients are equal - this shows that Spearmann is in some sense a generalization of the Pearson.

However before we get too excited the issues around it being an unintuitive metric still stands. It also has the added complication that if we observe data "in the wild" we don't know what the percentile of our observation is! For example suppose we're looking at relationships between height and weight in a population: if we find a person of height: $1.75m$ what percentile does this correspond to? We will likely have to estimate this given our observations which adds a layer of approximation. In the case of heavy-tail distributions this is particularly problematic since everything may appear to be "normal" for many many observations but suddenly 1 extreme value will totally change our perception of the underlying distribution.

Another issue is that while the Spearmann doesn't rely only on linear relations - it does require at least monotone relations. Anything more complicated that that and the relationship will not be captured. Let's look at a few simple examples to highlight the differences between these metrics:

```python
#hide
import warnings
warnings.filterwarnings('ignore')
```

```python
# Examples of Pearson and Spearman coefficients

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm, rankdata, pearsonr
import seaborn as sns
%matplotlib inline

U = np.random.random(1000)
X = norm.ppf(U)
Y_lin = 4*X + 5
Y_exp = np.exp(X)
Y_con = X**2

# Create function to return estimated percentile
def pctile(x):
    return rankdata(x) / x.shape[0]

# Calculate Pearson coefficients
pea_lin = pearsonr(X, Y_lin)[0]
pea_exp = pearsonr(X, Y_exp)[0]
pea_con = pearsonr(X, Y_con)[0]

# Calculate Spearman coefficients
X_pct = pctile(X)
Y_lin_pct = pctile(Y_lin)
Y_exp_pct = pctile(Y_exp)
Y_con_pct = pctile(Y_con)

spe_lin = pearsonr(X_pct, Y_lin_pct)[0]
spe_exp = pearsonr(X_pct, Y_exp_pct)[0]
spe_con = pearsonr(X_pct, Y_con_pct)[0]

# Create Plots
fig, ax = plt.subplots(1, 3, figsize=(15, 5))
sns.scatterplot(X, Y_lin, ax=ax[0])
ax[0].set_ylabel("Y=4X+5")
ax[0].set_xlabel("X \n Pearson: %f \n Spearman: %f" %(pea_lin, spe_lin))
sns.scatterplot(X, Y_exp, ax=ax[1])
ax[1].set_ylabel("Y=exp(X)")
ax[1].set_xlabel("X \n Pearson: %f \n Spearman: %f" %(pea_exp, spe_exp))
sns.scatterplot(X, Y_con, ax=ax[2])
ax[2].set_ylabel("Y=X^2")
ax[2].set_xlabel("X \n Pearson: %f \n Spearman: %f" %(pea_con, spe_con))
fig.suptitle("Comparison of Pearson and Spearman Correlation", x=0.5, y=0.95, size='xx-large')
plt.show()

```

We can see here that for a linear translation both metrics produce the same result. In the case of a non-linear monotone (e.g. exponential) relation the Spearman correctly identifies full dependancy. However neither metric is capable of capturing the dependency in the last non-monotone example - in fact both metrics suggest independence! From a basic visual inspection we would not describe these two variables as being independent. This shows we need to be careful using correlation metrics such as these. This is even before the introduction of noise!

Another somewhat popular choice of dependancy metric is the Kendall-Tau metric. This is a little different to the preceeding options. Essentially to calculate the Kendall-Tau we look at the joint pairs of samples, if both are concordant we add $1$ to a counter otherwise add $-1$ and move onto the next pair. We then take the average value of this (i.e. divide by the total number of pairs of joint samples). We can equally denote this by the formula:
$$  \tau ={\frac {2}{n(n-1)}}\sum _{i<j}\operatorname{sgn}(x_{i}-x_{j})\operatorname{sgn}(y_{i}-y_{j})$$

Where $\operatorname{sgn}(.)$ is the sign operator. $x_i$ and $y_i$ represent the rank or percentile of each sample within the set of all $x$ or $y$ samples. The benefit of the Kendall-tau is it is slightly more intuitive than the Spearman however it still suffers from some of the same issues (e.g. it will fail the $X^2$ example above).

We can show that Pearson, Spearman and Kendall are both particular cases of the generalized correlation coefficient:
$$\Gamma ={\frac  {\sum _{{i,j=1}}^{n}a_{{ij}}b_{{ij}}}{{\sqrt  {\sum _{{i,j=1}}^{n}a_{{ij}}^{2}\sum _{{i,j=1}}^{n}b_{{ij}}^{2}}}}}$$

Where $a_{ij}$ is the x-score and $b_{ij}$ the y-score for pairs of samples $(x_i, y_i)$ and $(x_j, y_j)$.

For example for the Kendall Tau we set: $a_{ij} = \operatorname{sgn}(x_i - x_j)$ and $b_{ij} = \operatorname{sgn}(y_i - y_j)$. For Spearman we set: $a_{ij} = (x_i - x_j)$ and $b_{ij} =(y_i - y_j)$. Where again we are working with ranks (or percentiles) rather than sampled values themselves.

Can we improve on these metrics from a practical standpoint? The answer is: yes! Unfortunately it is difficult to do and there is as much art as there is science to implementing it. We instead consider the mutual-information:
$$I(X, Y) = \int \int p_{X,Y}(x,y) \operatorname{log}\left( \frac{p_{X,Y}(x,y)}{p_X(x)p_Y(y)} \right) dx dy $$

Or similar for discrete distributions. Here $p_{X,Y}(.)$ represents the joint pdf of $(X,Y)$ and $p_X$, $p_Y$ represent the marginal distributions of $X$ and $Y$ respectively. This has the interpretation that it represents the information gained in knowing about the joint distribution compared to assuming independance of the marginal distributions. This is essentially matches our intuition of what dependance "is". However it is a bit of a pain to work with since we usually have to make distributional assumptions.

We can use generalizations of mutual information also. For example total correlation:
$$C(X_{1},X_{2},\ldots ,X_{n})=\left[\sum _{{i=1}}^{n}H(X_{i})\right]-H(X_{1},X_{2},\ldots ,X_{n}) $$

Which is the sum of information for each marginal distribution less the joint information. This compares to mutual information that can be expressed:
$$ \operatorname{I} (X;Y)=\mathrm {H} (Y)-\mathrm {H} (Y|X) $$

Another possible generalization to use is that of dual-total correlation which we can express as:
$$D(X_{1},\ldots ,X_{n})=H\left(X_{1},\ldots ,X_{n}\right)-\sum _{i=1}^{n}H\left(X_{i}\mid X_{1},\ldots ,X_{i-1},X_{i+1},\ldots ,X_{n}\right) $$

In the above we use the functional $H(.)$ to represent information:
$$H=-\sum _{i}p_{i}\log _{2}(p_{i}) $$

Or similar for continuous variables.

Despite the complications in working with these definitions these functions do pass the "complicated" examples such as $Y=X^2$ above and are a much better match for our intuition around dependence. I have long pushed for these metrics to be used more widely.

### Other Measurements of Dependence
We now appreciate that there are many different ways in which random variables can relate to each other (linear, monotone, independent, and so on.) So far the metrics and measures presented are "overall" measures of dependence, they give us one number for how the variables relate to each other. However in many cases we know that relations are not defined by just one number - typically we may find stronger relations in the tails (in extreme cases) than in "everyday" events. In the extreme we may have independence most of the time and yet near complete depednence when things "go bad". For an example of this consider we are looking at property damage and one factor we consider is wind-speed. At every day levels of wind there is unlikely to be much of a relationship between wind-speed and property damage - any damage that occurs is likely to be due to some other cause (vandalism, derelict/collapsing buildings, and so on). However as the wind-speed increases at a certain point there will be some property damage caused by the wind. At more extreme levels (say at the hurricane level) there will be almost complete dependence between wind level and property damage level. It is not difficult to think of other examples. In fact you'll likely find that most relations display this character to some degree often as a result of a structural shift in the system (e.g. in the wind example through bits of debris flying through the air!) Of course we would like to model these structural shifts exactly but this is often complicated if not impossible - through Monte-Carlo methods we are able to generate samples with behaviour that is at least qualitatively "correct".

We call this quality "tail dependence". This can affect upside and downside tails or just one tail depending on the phenomena.

Naively we would think that an option would be to use one of the correlation metrics above on censored data (that is filtering out all samples below/above a certain threshold). Unfortunately this does not work, the correlation metric is not additive - meaning you can't discretize the space into smaller units and combine them to give the overall correlation measure. This is another reason correlation metrics don't fit well with our intuition. However the information theoretic quantities can satisfy this condition but as before are a little more difficult to work with. We have some specialist metrics that are easier to use for this purpose.

The simplest metric to look at to study tail-dependence is the "joint exceedance probability". As the name suggests we look at the probability of time that the joint distribution exists beyond a certain level (or below a certain level). Suppose we have joint observations $(X_i, Y_i)_{i=1}^N$ and we have: $\mathbb{P}(X > \tilde{x}) = p$ and $\mathbb{P}(Y > \tilde{y}) = p$ then the joint exceedance probability of $(X,Y)$ at percentile $p$ is:
$$JEP_{(X,Y)}(p) = \frac{\sum_i\mathbb{1}_{(X_i > \tilde{x})}\mathbb{1}_{(Y_i > \tilde{y}})}{N} $$

Where: $\mathbb{1}_{(.)}$ is the indicator variable/Heaviside function. By construction we have that for $X$ and $Y$ fully rank-dependent then $JEP_{(X,Y)}(p) = p$ if $X$ and $Y$ fully rank-independent then $JEP_{(X,Y)}(p) = p^2$. It often helps to standardise the metric to vary between $[0,1]$ through the transform (we wouldn't use this metric if we are in the case of negative-dependence so can limit ourselves to indpendence as the lower-bound):
$$JEP_{(X,Y)}(p) = \frac{\sum_i\mathbb{1}_{(X_i > \tilde{x})}\mathbb{1}_{(Y_i > \tilde{y}})}{pN} - p$$

As with other rank-methods this metric is only useful with monotone relations, it cannot cope with non-monotone relations such as the $Y=X^2$ example.

We can also view tail dependence through the prism of probability theory. We can define the upper tail dependence ($\lambda _{u}$) between variables $X_1$ and $X_2$ with CDFs $F_1(.)$ and $F_2(.)$ respectively as:
$$\lambda _{u}=\lim _{q\rightarrow 1}\mathbb{P} (X_{2}>F_{2}^{-1}(q)\mid X_{1}>F_{1}^{-1}(q))$$

The lower tail dependence ($\lambda _{l}$) can be defined similarly through:
$$\lambda _{l}=\lim _{q\rightarrow 0}\mathbb{P} (X_{2}\leq F_{2}^{-1}(q)\mid X_{1} \leq F_{1}^{-1}(q))$$

The JEP metric above is in some sense a point estimate of these quantities, if we had infinite data and could make $p$ arbitrarily small in $JEP_{(X,Y)}(p)$ we would tend to $\lambda _{u}$. We will revisit the tail-dependence metric once we have built up some of the theoretical background.

## Multi-Variate Generation
We now move onto the issue of generating multi-variate samples for use within our Monte-Carlo models. It is not always to encode exact mechanisms for how variables relate (e.g. if we randomly sample from a distribution to give a country's GDP how do we write down a function to turn this into samples of the country's imports/exports? It is unlikely we'll be able to do this with any accuracy, and if we could create functions like this we would better spend our time speculating on the markets than developing Monte-Carlo models!) So we have to rely on broad-brush methods that are at least qualitatively correct.

### Multi-Variate Normal
We start with perhaps the simplest and most common joint-distribution (rightly or wrongly) the multi-variate normal. As the name suggests this is a multi-variate distribution with normal distributions as marginals. The joint behaviour is specified by a covariance matrix ($\mathbf{\Sigma}$) representing pairwise covariances between the marginal distributions. We can then specify the pdf as:
$$f_{\mathbf {X} }(x_{1},\ldots ,x_{k})={\frac {\exp \left(-{\frac {1}{2}}({\mathbf {x} }-{\boldsymbol {\mu }})^{\mathrm {T} }{\boldsymbol {\Sigma }}^{-1}({\mathbf {x} }-{\boldsymbol {\mu }})\right)}{\sqrt {(2\pi )^{k}|{\boldsymbol {\Sigma }}|}}}$$

Which is essentially just the pdf of a univariate normal distribution just with vectors replacing the single argument. To avoid degeneracy we require that $\mathbf{\Sigma}$ is a positive definite matrix. That is for any vector $\mathbf{x} \in \mathbb{R}^N_{/0}$ we have: $\mathbf{x}^T \mathbf{\Sigma} \mathbf{x} > 0$.

How can we sample from this joint distribution? One of the most common was is through the use of a matrix $\mathbf{A}$ such that: $\mathbf{A} \mathbf{A}^T = \mathbf{\Sigma}$. If we have a vector of independent standard normal variates: $\mathbf{Z} = (Z_1, Z_2, ... , Z_N)$ then the vector: $\mathbf{X} = \mathbf{\mu} + \mathbf{A}\mathbf{Z}$ follows a multivariate normal distribution with means $\mathbf{\mu}$ and covariances $\mathbf{\Sigma}$. The Cholesky-decomposition is typically used to find a matrix $\mathbf{A}$ of the correct form, as a result I have heard it called the "Cholesky-method". We will not cover the Cholesky decomposition here since it is a little out of scope, you can read more about it [here](https://en.wikipedia.org/wiki/Cholesky_decomposition) if you would like.

Let's look at an example of generating a 4d multivariate normal using this method:

```python
# Simulating from a multivariate normal
# Using Cholesky Decomposition

import numpy as np
from scipy.stats import norm
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
%matplotlib inline

# Fix number of variables and number simulations
N = 4
sims = 1000

# Fix means of normal variates
mu = np.ones(N)*0.5

# Initialize covariance matrix
cov = np.zeros((N, N))

# Create a covariance matrix
# Randomly sampled
for i in range(N):
    for j in range(N):
        if i==j:
            cov[i, j] = 1
        else:
            cov[i, j] = 0.5
            cov[j, i] = cov[i, j]

# Calculate cholesky decomposition
A = np.linalg.cholesky(cov)

# Sample independent normal variates
Z = norm.rvs(0,1, size=(sims, N))

# Convert to correlated normal variables
X = Z @ A + mu

# Convert X to dataframe for plotting
dfx = pd.DataFrame(X, columns=["X1", "X2", "X3", "X4"])

# Create variable Plots
def hide_current_axis(*args, **kwds):
    plt.gca().set_visible(False)

pp = sns.pairplot(dfx, diag_kind="kde")
pp.map_upper(hide_current_axis)
plt.show()
```

Here we can see we have geerated correlated normal variates.

We see here that this is a quick and efficient way of generating correlated normal variates. However it is not without its problems. Most notably is specifying a covariance/correlation matrix quickly becomes a pain as we increase the number of variates. For example if we had $100$ variables we would need to specify: $4950$ coefficients (all the lower off diagonals). If we have to worry about the matrix being positive definite this quickly becomes a pain and we can even begin to run into memory headaches when matrix multiplying with huge matrices.

An alternative highly pragmatric approach is to use a "driver" approach. Here we keep a number of standard normal variables that "drive" our target variables - each target variable has an associated weight to each of the driver variables and a "residual" component for its idiosyncratic stochasticity. In large scale models this can drastically reduce the number of variables to calibrate. Of course we lose some of the explanatory power in the model, but in most cases we can still capture the behaviour we would like. This procedure in some sense "naturally" meets the positive definite criteria and we do not need to think about it. Let's look at this in action:

```python
# Simulating from a multivariate normal
# Using a driver method

import numpy as np
from scipy.stats import norm
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

# Fix number of variables and number simulations
N = 4
sims = 1000

# Target means and standard deviations
mu = np.ones(N)*0.5
sig = np.ones(N)*0.5

# Fix number of driver variables
M = 2

# Specify driver weight matrix
w_mat = np.zeros((M, N))

for i in range(M):
    for j in range(N):
        w_mat[i, j] = 0.25

# Residual weights
w_res = 1 - w_mat.sum(axis=0)

# Simulate driver variables
drv =  norm.rvs(0,1, size=(sims, M))

# Simulate residual variables
res = norm.rvs(0,1, size=(sims, N))

# Calculate correlated variables
X = drv @ w_mat + w_res * res

# Standardise variables
X = (X - X.mean(axis=0)) / (X.std(axis=0))

# Apply transforms
X = X*sig + mu

# Convert X to dataframe for plotting
dfx = pd.DataFrame(X, columns=["X1", "X2", "X3", "X4"])

# Create variable Plots
def hide_current_axis(*args, **kwds):
    plt.gca().set_visible(False)

pp = sns.pairplot(dfx, diag_kind="kde")
pp.map_upper(hide_current_axis)
plt.show()
```

We can see that this "driver" method can significanlty reduce the number of parameters needed for our models. They can also be used to help "interpret" the model - for example we could interpret one of our "driver" variables as being a country's GDP we could then use this to study what happens to our model when GDP falls (e.g. is below the 25th percentile). The nature of the drivers is such that we do not need to know how to model them exactly, this is useful for modelling things such as "consumer sentiment" where a sophisticated model does not necessarily exist. This is part of the "art" side of Monte-Carlo modelling rather than a hard science.

The tail dependence parameters from the multi-variate normal satisfy:
$$\lambda_u = \lambda_l = 0$$

That is there is no tail-dependence. This can be a problem for our modelling.

### Other Multivariate Distributions
With the multivariate normal under our belts we may now want to move onto other joint distributions. Unfortunately things are not that simple, using the methods above we are essentially limited to multivariate distributions that can be easily built off of the multivariate normal distribution. One such popular example is the multivariate student-t distribution. We can sample from this using the transform:
$$ \mathbf{X} = \frac{\mathbf{Z}}{\sqrt{\frac{\chi_{\nu}}{\nu}}}$$

Where: $\mathbf{Z}$ is a multivariate normal and $\chi_{\nu}$ is a univariate chi-square with $\nu$ degrees of freedom. The resulting multivariate student-t has covariance matrix $\mathbf{\Sigma}$ and $\nu$ degrees of freedom. Here the degrees of freedom is a parameter that controls the degree of tail-dependence with lower values representing more tail dependence. In the limit $\nu \to \infty$ we have $\mathbf{X} \to \mathbf{Z}$.

We can add a couple of extra lines to our Gausssian examples above to convert them to a multi-variate student t distribution:

```python
# Simulating from a multivariate student t
# Using Cholesky Decomposition and a chi2 transform

import numpy as np
from scipy.stats import norm, chi2
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
%matplotlib inline

# Fix number of variables and number simulations
N = 4
sims = 1000

# Fix means of normal variates
mu = np.ones(N)*0.5

# Initialize covariance matrix
cov = np.zeros((N, N))

# Set degrees of freedom nu
nu = 5

# Create a covariance matrix
# Randomly sampled
for i in range(N):
    for j in range(N):
        if i==j:
            cov[i, j] = 1
        else:
            cov[i, j] = 0.5
            cov[j, i] = cov[i, j]

# Calculate cholesky decomposition
A = np.linalg.cholesky(cov)

# Sample independent normal variates
Z = norm.rvs(0,1, size=(sims, N))

# Convert to correlated normal variables
X = Z @ A + mu

# Standardize the normal variates
Z = (X - X.mean(axis=0)) / X.std(axis=0)

# Sample from Chi2 distribution with nu degrees of freedom
chis = chi2.rvs(nu, size=sims)

# Convert standard normals to student-t variables
T = (Z.T / np.sqrt(chis / nu)).T

# Convert T to dataframe for plotting
dft = pd.DataFrame(T, columns=["T1", "T2", "T3", "T4"])

# Create variable Plots
def hide_current_axis(*args, **kwds):
    plt.gca().set_visible(False)

pp = sns.pairplot(dft, diag_kind="kde")
pp.map_upper(hide_current_axis)
plt.show()
```

The tail dependence parameter fro the student-t copula for 2 variates with correlation parameter $\rho$ and $\nu$ degrees of freedom is:

$$\lambda_u = \lambda_l = 2 t_{\nu+1} \left(- \frac{\sqrt{\nu+1}\sqrt{1-\rho}}{\sqrt{1+\rho}} \right) > 0$$

We can see that even if we specify $\rho=0$ there will be some level of tail-dependence between the variates. In fact it is impossible to enforce independence between variables using the student-t copula. This can be an issue in some instances but there are extensions we can make to overcome - for example see my [insurance aggregation model](https://lewiscoleblog.com/insurance-aggregation-model).

Another interesting property of the student-t copula is that it also has "opposing" tail dependency. Namely:

$$\lim _{q\rightarrow 0}\mathbb{P} (X_{2}\leq F_{2}^{-1}(q)\mid X_{1} > F_{1}^{-1}(1-q)) = \lim _{q\rightarrow 0}\mathbb{P} (X_{2}>F_{2}^{-1}(1-q)\mid X_{1} \leq F_{1}^{-1}(q)) = 2 t_{\nu+1} \left(- \frac{\sqrt{\nu+1}\sqrt{1+\rho}}{\sqrt{1-\rho}} \right) > 0$$

This gives rise to the familiar "X" shape of a joint student-t scatter plot.

### The Need for More Sophistication
So far we are limited to modelling joint behaviour with either normal or student-t marginal distributions. This is very limiting, in practice we will want to sample from joint distributions with arbitrary marginal distributions. Typically we will find it easier to fit marginal distributions to data and we will want our models to produce reasonable marginal results.  We therefore need a better method of modelling joint behaviour.

## Copula Methods
One way to do this is through a copula method. A copula is nothing more than a multi-variate distribution with uniform $[0,1]$ marginals. This is particularly useful for modelling purposes as it allows us to fully delineate the univariate distributions from the joint behaviour. For example we can select a copula with the joint behaviour we would like and then use (for example) a generalized inverse transform on the marginal distributions to get the results we desire. We can do this simply by adapting the multi-variate Gaussian above, we can use the CDF function on the generated normal variates to create joint-uniform variates (i.e. a copula) and then inverse transform these to give Gamma marginals. This will result in Gamma marginals that display a rank-normal correlation structure:

```python
# Simulating from Gamma variates with rank-normal dependency
# Using a copula type method

import numpy as np
from scipy.stats import norm, gamma
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
%matplotlib inline

# Fix number of variables and number simulations
N = 4
sims = 1000

# Fix means of normal variates
mu = np.ones(N)*0.5

# Initialize covariance matrix
cov = np.zeros((N, N))

# Select Gamma parameters a and b
a = 5
b = 3

# Create a covariance matrix
# Randomly sampled
for i in range(N):
    for j in range(N):
        if i==j:
            cov[i, j] = 1
        else:
            cov[i, j] = 0.5
            cov[j, i] = cov[i, j]

# Calculate cholesky decomposition
A = np.linalg.cholesky(cov)

# Sample independent normal variates
Z = norm.rvs(0,1, size=(sims, N))

# Convert to correlated normal variables
X = Z @ A + mu

# Standardize the normal variates
Z = (X - X.mean(axis=0)) / X.std(axis=0)

# Apply the normal CDF to create copula C
C = norm.cdf(Z)

# Use inverse transform to create Gamma variates
G = gamma.ppf(C, a=a, scale=1/b)

# Convert T to dataframe for plotting
dfg = pd.DataFrame(G, columns=["G1", "G2", "G3", "G4"])

# Create variable Plots
def hide_current_axis(*args, **kwds):
    plt.gca().set_visible(False)

pp = sns.pairplot(dfg, diag_kind="kde")
pp.map_upper(hide_current_axis)
plt.show()
```

This is really quite neat! It makes our modelling that much easier as we do not need to consider the joint-distribution in its entirety, we can break it down into smaller pieces that are easier to handle. The example above is quite a simple one however, we would like to have more freedom than this as we are still bound by sampling from a joint-distribution.

Let's look at some notation. Suppose we have variates: $(X_1, ... ,X_N)$ where each follows a cdf $F_i(x) = \mathbb{P}(X_i \leq x)$. We then have: $(U_1, ... , U_N) = (F_1(X_1), ..., F_N(X_N)$ with uniform random variates by definition. The copula defined by variables $(X_1, ..., X_N)$ is: $C(U_1, ... , U_N) = \mathbb{P}(X_1 \leq F_1(U_1), ... , X_N \leq F_N(U_N))$. From this definition $C: [0,1]^N \to [0,1]$ is an N-dimensional copula if it is a cumulative distribution function (CDF) on the unit cube with uniform marginals.

Going the other way any function $C: [0,1]^N \to [0,1]$ defines a copula if:
- $C(u_{1},\dots ,u_{i-1},0,u_{i+1},\dots ,u_{N})=0$, the copula is zero if any one of the arguments is zero,
- $C(1,\dots ,1,u,1,\dots ,1)=u$, the copula is equal to u if one argument is u and all others 1,
- C is d-non-decreasing, i.e., for each hyperrectangle $B=\prod _{i=1}^{N}[x_{i},y_{i}]\subseteq [0,1]^{N}$ the C-volume of B is non-negative:
$$\int _{B}\mathrm {d} C(u)=\sum _{\mathbf {z} \in \times _{i=1}^{N}\{x_{i},y_{i}\}}(-1)^{N(\mathbf {z} )}C(\mathbf {z} )\geq 0$$
where the $N(\mathbf {z} )=\#\{k:z_{k}=x_{k}\}$.

We now move onto the fundamental theorem of copulae: Sklar's theorem. The statement of the theorem exists in two parts:
1. Let $H(X_1, ..., X_N)$ represent a joint distribution function with marginal distributions $X_i \sim F_i$. Then there exists a copula $C(.)$ such that: $H(X_1, ..., X_N) = C(F_1(X_1), ... , F_N(X_N))$. Moreover if $F_i$ are continuous then the copula $C(.)$ is unique
2. Given a copula function $C(.)$ and univariate distribution functions $F_i$ then $C(F_1(X_1), ... , F_N(X_N))$ defines a joint distribution with marginal distributions $F_i$

Sklar's theorem shows there is a bijection relation between the space of joint distributions and the space of copulae (for continuous variables). This shows that this is indeed a powerful method to use in our modelling.

There are a couple of things to note here: the copula is itself a rank-order method of applying dependency. It is invariant under monotonic transforms (due to the use of the generalized inverse method). We can also derive the Fréchet–Hoeffding bounds:

$$ \operatorname{max}\left\{ 1 - N + \sum_{i=1}^{N} u_i, 0 \right\} \leq C(u_1, ..., u_N) \leq \operatorname{min} \{u_1, ... , u_N \}$$

We can see this quite easily by noting that for the lower-bound:
\begin{align}
C(u_1, ... , u_N) &= \mathbb{P} \left( \bigcap_{i=1}^N \{ U_i \leq u_i \} \right) \\
                  &= 1 - \mathbb{P} \left( \bigcup_{i=1}^N \{ U_i \geq u_i \} \right) \\
                  &\geq 1 - \sum_{i=1}^N \mathbb{P}( U_i \geq u_i ) \\
                  &= 1 - N + \sum_{i=1}^{N} u_i
\end{align}

For the upper bound we have:

$$ \bigcap_{i=1}^N \{ U_i \leq u_i \} \subseteq \{ U_i \leq u_i \} \qquad  \implies \qquad \mathbb{P} \left( \bigcap_{i=1}^N \{ U_i \leq u_i \} \right)  \leq \mathbb{P} \left(  U_i \leq u_i  \right) = u_i $$

For all possible indices $i$. We have that the upper-bound function $\operatorname{min} \{u_1, ... , u_N \}$ itself defines a copula corresponding to complete (rank) dependence. It is often called the "co-monotonic copula". The lower bound exists as a copula only in the case $N=2$ whereby it represents complete negative (rank) dependence.

We can also calculate the Spearman ($\rho$) and Kendall-Tau ($\tau$) dependency coefficients using the copula construction:

$$ \rho = 12 \int_0^1 \int_0^1 C(u,v) du dv - 3 $$

And:

$$ \tau = 4 \int_0^1 \int_0^1 C(u,v) dC(u,v) - 1$$

We can view the Gaussian example above in a copula frame via:

$$C^{Gauss}(u_1, ... , u_N) = \Phi_{\mathbf{\mu,\Sigma}}(\Phi^{-1}(u_1), ..., \Phi^{-1}(u_N))$$

Where: $\Phi_{\mathbf{\mu,\Sigma}}$ is the CDF of a multivariate Gaussian with means $\mathbf{\mu}$ and correlation matrix $\mathbf{\Sigma}$ The function: $\Phi^{-1}(.)$ is the inverse CDF of the univariate standard normal distribution. We call this the Gaussian copula or the rank-normal copula.

We can define the student-t copula in a similar way. I have developed a model based around a hierarchical driver structure with a generalization of a student-t copula for the purposes of modelling insurance aggregation. You can read my blog-post dedicated to this specifically [here!](https://lewiscoleblog.com/insurance-aggregation-model)

But what are some other options? The copula methods are very flexible so lets look at some other examples. Starting off with the Archimedean family of copulae. A copula is Archimedian if it admits a representation:

$$ C_{Arch}(u_{1},\dots ,u_{N};\theta )=\psi ^{[-1]}\left(\psi (u_{1};\theta )+\cdots +\psi (u_{N};\theta );\theta \right) $$

Here the function $\psi$ is called the generator function. We also have parameter $\theta$ taking values in some arbitrary parameter space - the role that $\theta$ takes depends on the form of the generator. $\psi ^{-1}$ is the pseudo-inverse of $\psi$:

\begin{equation}
  \psi ^{[-1]}(t;\theta ) =
    \begin{cases}
      \psi ^{-1}(t;\theta )&{\mbox{if }}0\leq t\leq \psi (0;\theta ) \\
      0&{\mbox{if }}\psi (0;\theta )\leq t\leq \infty
    \end{cases}
\end{equation}

We note that this functional form has some very useful properties. For example we note that we can express the upper and lower tail dependence parameters (defined above). If we place further assumptions on the generator function being a Laplace transform of strictly positive random variables we can write down neat forms of the upper and lower tail depdence metrics:

$$ \lambda _{u} = 2 - 2 \lim_{s \downarrow 0} \frac{\psi'^{[-1]} (2s)}{\psi'^{[-1]}  (s)} $$

and:
$$ \lambda _{l} = 2 \lim_{s \to \infty} \frac{\psi'^{[-1]} (2s)}{\psi'^{[-1]}  (s)} $$

Similarly we can conveniently calculate the Kendall-tau measure of dependence via:

$$ \tau = 1 + 4 \int_0^1 \frac{\psi(t)}{\psi'(t)} dt $$

This is very convenient for modelling purposes since we can easily control the joint properties in a predictable way. Unforunately the calculation of Spearman correlation coefficients is not as simple and in many cases a closed form analytic solution does not exist.

We can summarise some of the common Archimedean copulae in the table below:

|Copula Name     | $\psi(t)$ | $\psi^{[-1]}(t)$ | $\theta$-defined-range | Lower-Tail-$\lambda_l(\theta)$ | Upper-Tail-$\lambda_u(\theta)$ |$\tau$ |
|:--------|:------:|:-------------:|:----------------:|:-----------:|:-----------:|:-------:|
| Frank   | $-\log \left({\frac {\exp(-\theta t)-1}{\exp(-\theta )-1}}\right)$ | $-{\frac {1}{\theta }}\,\log(1+\exp(-t)(\exp(-\theta )-1))$ | $\mathbb {R} \backslash \{0\}$ | 0               | 0               | $1 + 4(D_1(\theta)-1)/\theta$ |
| Clayton | $\frac {1}{\theta }(t^{-\theta }-1)$ | $\left(1+\theta t\right)^{-1/\theta }$ | $[-1,\infty) \backslash \{0\}$                 | $2^{-1/\theta}$ | 0 | $\frac{\theta}{\theta + 2}$ |
| Gumbel  | $\left(-\log(t)\right)^{\theta }$     | $\exp\left(-t^{1/\theta }\right)$ | $[1, \infty)$ | 0 | $2-2^{1/\theta}$ | $1 - \theta^{-1}$  |

Where: $D_k(\alpha) = \frac{k}{\alpha^k} \int_0^{\alpha} \frac{t^k}{exp(t)-1} dt$ is the Debye function.

We can see that the Frank copula applies dependence without any tail-dependence, whereas the Clayton and Gumbel are options for lower or upper-tail dependence modelling. This is in contrast to the student-t we observed before which applies symmetrically to both lower and upper tails.

To finish we will look at the bi-variate Frank, Clayton and Gumbel copulae. We will use a variety of methods to do this. For the Frank copula we will look at the "conditional" copula (that is the probability of one variate conditional on a specific value of the other): we will sample the overall percentile for the copula ($z$) and one of the variates ($u_1$) and then back "out" the remaining variate ($u_2$). The pair $\{ u_1, u_2\}$ is then distributed as the copula:

$$ u_2 = -\frac{1}{\theta} \log \left[1 + z\frac{1-e^{-\theta}}{z(e^{-\theta u}-1) - e^{-\theta u}} \right]$$

We can do the same for the Clayton:

$$ u_2 = \left(1 + u^{-\frac{1}{\theta}}  \left( z^{-\frac{1}{1+\theta}}-1 \right)  \right)^{-\theta}$$

Unforunately the conditinal copula of the Gumbel is not invertible and so we are unable to follow the same approach. Instead we follow the approach shown by Embrechts where we sample a uniform variate $v$ and then find $0<s<1$ such that: $sln(s) = \theta(s-v)$. We then sample another uniform variate $u$ and the pair: $\{ exp(u^{1/\theta}ln(s)), exp((1-u)^{1/\theta}ln(s))\}$ is a sample from the Gumbel copula with parameter $\theta$.

We can implement this as:

```python
# Sampling from the Frank, Clayton and Gumbel archimedean Copulae
# A very basic bi-variate implementation

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.optimize import fsolve
%matplotlib inline

# Set number of simulations and theta parameter
sims = 1000
theta = 2

# Simulate u1 - first component of copula
U1 = np.random.random(sims)

# Simulate z - the joint probaibility
Z = np.random.random(sims)

# Define conversion functions for each copula
def frank(theta, z, u):
    return -1/theta*np.log(1+ z*(1-np.exp(-theta))/(z*(np.exp(-theta*u)-1) - (np.exp(-theta*u))))

def clayton(theta, z, u):
    return np.power((1+ np.power(u, -1/theta)*(np.power(z, -1/(1+theta))-1)), -theta)

# Define function to find S such that sln(s) = theta(s-u)
def gumfunc(s):
    return s*np.log(s) - theta*(s-U1)

# Use fsolve to find roots
S = fsolve(gumfunc, U1)

U1_gumbel = np.exp(np.log(S)*np.power(Z, 1/theta))
U2_gumbel = np.exp(np.log(S)*np.power(1-Z, 1/theta))

U1_frank = U1
U2_frank = frank(theta, Z, U1)

U1_clayton = U1
U2_clayton = clayton(theta, Z, U1)

# Create Plots
fig, ax = plt.subplots(1, 3, figsize=(15, 5))
sns.scatterplot(U1_frank, U2_frank, ax=ax[0])
ax[0].set_title("Frank")
ax[0].set_xlim([0,1])
ax[0].set_ylim([0,1])
sns.scatterplot(U1_clayton, U2_clayton, ax=ax[1])
ax[1].set_title("Clayton")
ax[1].set_xlim([0,1])
ax[1].set_ylim([0,1])
sns.scatterplot(U1_gumbel, U2_gumbel, ax=ax[2])
ax[2].set_title("Gumbel")
ax[2].set_xlim([0,1])
ax[2].set_ylim([0,1])
fig.suptitle(r"Comparison of Archemedean Copulae with $\theta =$%i" %theta, x=0.5, y=1, size='xx-large')
plt.show()
```

As expected we can see little dependence in the tails of the Frank Copula, some dependence in the lower tail (small percentiles) of the Clayton and some dependence in the upper tail (larger percentiles) of the Gumbel copulae.

It was a bit of work to sample from these copulae. Thankfully packages exist to make our lives easier - for example: [copulas](https://pypi.org/project/copulas/). But is important to understand at least the basics behind how some of these packages work before we jump in and use them.

## Conclusion
We have now extended our abilities from being able to sample from univariate distributions in our previous blog post to how to sample from multi-variate distributions. We saw how we could use a Cholesky decomposition to sample from a mulit-variate normal and further how we could implement a driver approach in order to reduce the number of model parameters. We then saw how we could extend this to sample from a multi-variate student-t distribution. We then looked at copula methods in order to separate the marginal and joint behaviour of a system. We ended by looking at the powerful class of Archimedean copulae andthe Frank, Clayton and Gumbel copulae specifically.


# Quantum Teleportation with qoqo & the use of conditional measurements

This notebook is designed to demonstrate the use of conditional measurements, by following through an example of quantum state teleportation.

In quantum teleportation there are two end users: The first user, Alice, wishes to send a particular quantum state to the second user, Bob. The protocol requires a total of three qubits, and the transmission of two classical bits.

The sender Alice controls qubits 0 and 1, and the reciever Bob controls qubit 2.

```python
from qoqo_pyquest import PyQuestBackend
from qoqo import Circuit
from qoqo import operations as ops
import numpy as np
from math import sqrt, pi

```

## State preparation

The first step is to prepare the quantum state which Alice will send to Bob. As an example, the most general single qubit quantum state is given by:
\begin{equation}
|\psi \rangle = cos(\frac{\theta}{2}) |0 \rangle + e^{i \phi} sin(\frac{\theta}{2}) |1 \rangle.
\end{equation}
This state can be prepared by a sequence of two single qubit rotations. In the code block below we first define a function that takes the angles $\theta$ and $\phi$ as input and prepares qubit 0 of a quantum register in the state $| \psi \rangle$.

Next we use an instance of the function with the angles $\theta=\frac{\pi}{2}$ and $\phi=0$ to create a circuit which prepares the state:
\begin{equation}
|\psi \rangle = \frac{1}{\sqrt{2}} \big{(} |0 \rangle + |1 \rangle \big{)} = | + \rangle.
\end{equation}

```python
def prep_psi(Theta: float, Phi: float) -> Circuit:
    circuit = Circuit()
    circuit += ops.RotateY(qubit=0, theta=Theta)
    circuit += ops.RotateZ(qubit=0, theta=Phi)
    return circuit

init_circuit = prep_psi(pi/2, 0.0)

```

## Preparing an entangled resource state

Quantum teleportation requires that the end users initially share an entangled resource state,
\begin{equation}
|\Phi_{+} \rangle = \frac{1}{\sqrt(2)} \big{(} |00 \rangle + |11 \rangle \big{)} .
\end{equation}

The following circuit prepares the state $|\Phi_{+} \rangle$ between qubit 1, held by Alice, and qubit 2, held by Bob.

```python
entangling_circ = Circuit()
entangling_circ += ops.Hadamard(qubit=1)
entangling_circ += ops.CNOT(control=1, target=2)
```

## Encoding the state to be sent in the entangled resource state

The next step of the procedure is to encode the state of qubit 0, $\psi$, into the entangled resource state. This is accomplished by way of the circuit defined below, which is similar to that used to prepare the entangled resource.

```python
encoding_circ = Circuit()
encoding_circ += ops.CNOT(control=0, target=1)
encoding_circ += ops.Hadamard(qubit=0)
```

## State transfer part 1: Measurement

At this stage in the process both of Alice's qubits, 0 and 1, are measured. The measurement consumes the entangled resource and leaves the state of qubit 2,Bob's qubit, in a state that depends on the two measurement outcomes.

Let us call the classical bit which results from measuring qubit 0 'M1' and the bit resulting from measuring qubit 1 'M2'. The circuit below defines the classical register named 'M1M2', performs the measurement of qubits 0 and 1, and stores the results in the register 'M1M2'.

```python
meas_circ = Circuit()
meas_circ += ops.DefinitionBit(name='M1M2', length=2, is_output=True) #for classical bits corresponding to measurement outcomes
meas_circ += ops.MeasureQubit(qubit=0,readout='M1M2',readout_index=0)
meas_circ += ops.MeasureQubit(qubit=1,readout='M1M2',readout_index=1)
```

## Defining the circuit for a conditional operation

Conditional operations in qoqo have three inputs: the name of a classical register containing boolean values, the index of the register containing the value to be used to condition the operation, and the operation or sequence of operations to be performed if the boolean condition value is True.

To prepare the third input, it is necessary to create circuit snippets corresponding to the operations to be completed if the condition is True.

In the case of quantum teleportation, we need two conditional operations. The first is a Pauli Z acting on Bob's qubit, conditioned on the measurement result M1. The second is a Pauli X acting on Bob's qubit, conditioned on the measurement result M2.

Hence we prepare circuit snippets correspponding to a Pauli Z and a Pauli X operation.

```python
conditional_Z = Circuit()
conditional_Z += ops.PauliZ(qubit=2)

conditional_X = Circuit()
conditional_X += ops.PauliX(qubit=2)

```

## State transfer part 2: conditional operations

The final stage of the teleportation protocol is to perform corrections to the state of Bob's qubit 2, according to the measurement outcomes 'M1' and 'M2'.

The below circuit makes use of the circuit snippets defined above to perform the conditional corrections to the state of qubit 2. 2.

```python
conditional_circ = Circuit()
conditional_circ += ops.PragmaConditional(condition_register='M1M2',condition_index=1, circuit=conditional_X)
conditional_circ += ops.PragmaConditional(condition_register='M1M2',condition_index=0, circuit=conditional_Z)
```

## Putting it all together

Combining each of the circuits we have defined yeilds the full teleportation protocol. We can verify that the protocol is successful by reading out the final state vector and comparing it to the state which was to be sent, $|\psi \rangle$.

```python
verification = Circuit()
# Create register for state vector readout
verification += ops.DefinitionComplex(name='psi', length=8, is_output=True)
verification += ops.PragmaGetStateVector(readout='psi', circuit=Circuit())

# Combine parts for full protocol
teleportation_circuit = init_circuit + entangling_circ + encoding_circ + meas_circ + conditional_circ + verification

# Run simulation and collect outputs
backend = PyQuestBackend(number_qubits=3)
(result_bit_registers, result_float_registers, result_complex_registers)=backend.run_circuit(teleportation_circuit)

# View measurement outcomes and post-protocol state of qubits
print(result_bit_registers['M1M2'])
print(result_complex_registers['psi'])
```

    [[True, False]]
    [array([0.        +0.j, 0.70710678+0.j, 0.        +0.j, 0.        +0.j,
           0.        +0.j, 0.70710678+0.j, 0.        +0.j, 0.        +0.j])]

```python

```


```python
import numpy as np  #Importa libreria numerica
import sympy as sym #simbolica
import matplotlib.pyplot as plt #importa matplotlib solo pyplot
import matplotlib.image as mpimg
from sympy.plotting import plot  #para plotear 2 variables
from sympy.plotting import plot3d  # para 3
from sympy.plotting import plot3d_parametric_surface
from IPython.display import Image
import ipympl  #Para importar gestor de imagenes
sym.init_printing() #activa a jupyter para mostrar simbolicamente el output

%matplotlib inline
```

```python
Image(filename='LAB2.png',width=300)
# Diseño tiene que cumplir
# Amplificador Operacional LM741 o LM324
# Alimentación Vcc = 10V, Vss = − 10V
# Ganancia en banda media A = Vo/V1 y A = Vo/V2 debe ser igual a 30 veces.
# Zi del amplificador no puede alterar o cargar la fuente de señal, es decir, Ri << Zi1 y Zi2. (al menos 10 veces)
# Usar Resistencias <= 1MΩ
# Las fuentes V1 y V2 (fig 2) deben considerarse en las condiciones 1.A y 1.B
# 1.A Ri = 50Ω
# 1.B Ri = 100KΩ

#Se usa LM741 que segun la tabla 6.7-1 del libro Dorf Svoboda
#Ipol = 80nA ; Ios = 20nA
#Vos = 1mV ; Ad=200 V/mV = 200k V/V
#SR = 0.5 V/uS = 0.5 Meg V/S = 500k V/S
#CMRR = 31.6 V/mV = 31.6k V/V

```

```python
#Forma generica
V1, V2 = sym.symbols('V_1, V_2')
Vo = sym.Function('V_o')(V1,V2)
eq_Vo = sym.Eq(Vo,(sym.diff(Vo.subs(V2,0),V1))*V1+sym.diff(Vo.subs(V1,0),V2)*V2)
sym.pprint(eq_Vo)
```

                     d                   d
    Vₒ(V₁, V₂) = V₁⋅───(Vₒ(V₁, 0)) + V₂⋅───(Vₒ(0, V₂))
                    dV₁                 dV₂

```python
#CASO A) Ri=50
Ri, R, Rf = sym.symbols('Ri, R, Rf')
#Si hacemos el diseño como un OPAMP ideal
eq_Vo1 = sym.Eq(Vo.subs(V2,0),-V1*(Rf/R))
sym.pprint(eq_Vo1)
eq_Vo2 = sym.Eq(Vo.subs(V1,0),-V2*(Rf/R))
sym.pprint(eq_Vo2)
eq_Vo = sym.Eq(Vo, eq_Vo1.rhs + eq_Vo2.rhs)
sym.pprint(sym.simplify(eq_Vo))

#Si por diseño A=30 y Ri = 50
eq_Ri_A = sym.Eq(Ri, 50)
sym.pprint(eq_Ri_A)

eq_R_A = sym.Eq(R , Ri*10 + Ri)
sym.pprint(eq_R_A)

eq_Rf_A = sym.Eq(Rf , 30 * (eq_R_A.rhs).subs(Ri,eq_Ri_A.rhs) )
sym.pprint(eq_Rf_A)
```

                -Rf⋅V₁
    Vₒ(V₁, 0) = ───────
                   R
                -Rf⋅V₂
    Vₒ(0, V₂) = ───────
                   R
                 -Rf⋅(V₁ + V₂)
    Vₒ(V₁, V₂) = ──────────────
                       R
    Ri = 50
    R = 11⋅Ri
    Rf = 16500

```python
#CASO B) Ri=100k
eq_Vo = sym.Eq(Vo, eq_Vo1.rhs + eq_Vo2.rhs)
sym.pprint(sym.simplify(eq_Vo))

#Si por diseño A=30 y Ri = 100k

eq_Ri_B = sym.Eq(Ri , 100e3)
sym.pprint(eq_Ri_B)

eq_R_B = sym.Eq(R , Ri*10 + Ri)
sym.pprint(eq_R_B)

eq_Rf_B = sym.Eq(Rf , 30 * (eq_R_B.rhs).subs(Ri,eq_Ri_B.rhs) )
sym.pprint(eq_Rf_B)
```

                 -Rf⋅(V₁ + V₂)
    Vₒ(V₁, V₂) = ──────────────
                       R
    Ri = 100000.0
    R = 11⋅Ri
    Rf = 33000000.0

```python
#En el ultimo caso Rf es de orden de los Megas
sym.pprint("{:.2e}".format(eq_Rf_B.rhs))
```

    3.30e+7

```python
#Entonces se utiliza una red T
# Rf = Vo/If cuando Vi=0
Image(filename='redT.png',width=300)
```

```python
Ra, Rb, Rc = sym.symbols('Ra, Rb, Rc')
If, Irc, Vx, VoRT = sym.symbols('I_f, I_Rc, V_x, V_oRT')
eq_Vx =sym.Eq(Vx, If * Ra)
sym.pprint(eq_Vx)

eq_Irc =sym.Eq(Irc,eq_Vx.rhs/Rc)
sym.pprint(eq_Irc)

eq_VoRT=sym.Eq(VoRT, Vx + (If+Irc)*Rb)
sym.pprint(eq_VoRT)
#Remplazando
eq_VoRT=sym.Eq(VoRT, eq_Vx.rhs + (If+eq_Irc.rhs)*Rb)
sym.pprint(sym.apart(eq_VoRT,If))

eq_Rf_T=sym.Eq(Rf,eq_VoRT.rhs/If)
sym.pprint(sym.simplify(eq_Rf_T))

eq_Rc=sym.Eq(Rc , sym.solve(eq_Rf_T ,Rc)[0] )

sym.pprint(eq_Rc)

eq_Rc_val=sym.Eq(Rc,eq_Rc.rhs.subs({Ra:100e3,Rb:100e3,Rf:eq_Rf_B.rhs}))
sym.pprint(eq_Rc_val)

```

    Vₓ = I_f⋅Ra
           I_f⋅Ra
    I_Rc = ──────
             Rc
    V_oRT = Rb⋅(I_Rc + I_f) + Vₓ
            I_f⋅(Ra⋅Rb + Ra⋅Rc + Rb⋅Rc)
    V_oRT = ───────────────────────────
                         Rc
         Ra⋅Rb
    Rf = ───── + Ra + Rb
           Rc
           -Ra⋅Rb
    Rc = ────────────
         Ra + Rb - Rf
    Rc = 304.878048780488

```python
#BLACKMAN
#Primero analizamos el resultado con Ad=200k V/V
#entonces se utiliza la formula de BLACKMAN Avf= Ad/(1-T)
#Para el caso A)
Image(filename='black1.png',width=300)
```

```python
#Para el caso A)
Avf, Av, T = sym.symbols('A_vf, A_v, T')
eq_Avf=sym.Eq(Avf,Av/(1-T))
sym.pprint(eq_Avf)

V1, V2, Vp = sym.symbols('V_1,V_2,V_p')
Vo=sym.Function('V_o')(V1,V2,Vp)

eq_Av=sym.Eq(Av,sym.diff(Vo.subs({V2:0,Vp:0})))
sym.pprint(eq_Av)

eq_Av=sym.Eq(Av,sym.diff(Vo.subs({V1:0,Vp:0})))
sym.pprint(eq_Av)

eq_T=sym.Eq(T,sym.diff(Vo.subs({V1:0,V2:0})))
sym.pprint(eq_T)

```

             Aᵥ
    A_vf = ─────
           1 - T
          d
    Aᵥ = ───(Vₒ(V₁, 0, 0))
         dV₁
          d
    Aᵥ = ───(Vₒ(0, V₂, 0))
         dV₂
         d
    T = ───(Vₒ(0, 0, Vₚ))
        dVₚ

```python
#Av= Vo/V- * V-/V1
#solo para el informe
Vo_o_Vneg, Vneg_o_V1 = sym.symbols('(V_{o}/V_{-}), (V_{-}/V_{1})')
eq_Av=sym.Eq(Av,Vo_o_Vneg*Vneg_o_V1)
display(eq_Av)

Ad=sym.Symbol('A_d')
eq_Av_A=sym.Eq(Av,-Ad*((R**-1+Rf**-1)**-1)/(R+(R**-1+Rf**-1)**-1))
sym.pprint(eq_Av_A)
sym.pprint(sym.simplify(eq_Av_A))

sym.pprint( eq_Av_A.subs( {R:eq_R_A.rhs , Ri:eq_Ri_A.rhs , Rf:eq_Rf_A.rhs , Ad:200e3}))
```

```python
eq_T_A=sym.Eq(T,-Ad*(R/2)/(R/2+Rf))
sym.pprint(sym.simplify(eq_T_A))

eq_T_A_val=eq_T_A.subs( {R:eq_R_A.rhs , Ri:eq_Ri_A.rhs , Rf:eq_Rf_A.rhs , Ad:200e3})
sym.pprint(eq_T_A_val)
```

        -A_d⋅R
    T = ────────
        R + 2⋅Rf
    T = -3278.68852459016

```python
eq_Avf_A=sym.Eq(Avf,eq_Av_A.rhs/(1-eq_T_A.rhs))
sym.pprint(sym.simplify(eq_Avf_A))
sym.pprint( eq_Avf_A.subs( {R:eq_R_A.rhs , Ri:eq_Ri_A.rhs , Rf:eq_Rf_A.rhs , Ad:200e3}))
```

               -A_d⋅Rf
    A_vf = ────────────────
           A_d⋅R + R + 2⋅Rf
    A_vf = -29.9908527898991

```python
#BLACKMAN
#Primero analizamos el resultado con Ad=200k V/V
#entonces se utiliza la formula de BLACKMAN Avf= Ad/(1-T)
#Para el caso B)
Image(filename='black2.png',width=300)
```

```python
#Para el caso B)
Avf, Av, T = sym.symbols('A_vf, A_v, T')
eq_Avf=sym.Eq(Avf,Av/(1-T))
sym.pprint(eq_Avf)
```

             Aᵥ
    A_vf = ─────
           1 - T

```python
#Av= Vo/V- * V-/V1
#solo para el informe
Vo_o_Vneg, Vneg_o_V1 = sym.symbols('(V_{o}/V_{-}), (V_{-}/V_{1})')
eq_Av=sym.Eq(Av,Vo_o_Vneg*Vneg_o_V1)
display(eq_Av)

eq_Av_B=sym.Eq(Av,-Ad*(R**-1+(Ra+(Rc**-1+Rb**-1)**-1)**-1)**-1/(R+(R**-1+(Ra+(Rc**-1+Rb**-1)**-1)**-1)**-1))
sym.pprint(sym.simplify(eq_Av_B))
sym.pprint(eq_Av_B.subs({R:eq_R_B.rhs , Ri:eq_Ri_B.rhs , Ra:100e3, Rb:100e3, Rc:eq_Rc_val.rhs , Ad:200e3}))
```

```python
#T= Vo/Vp = Vo/V- V-/Ia Ia/Ib Ib/Vp
#Solo para informe
Vo_o_Vneg, Vneg_o_Ia, Ia_o_Ib, Ib_o_Vp = sym.symbols('(V_{o}/V_{-}), (V_{-}/I_{a}), (I_{a}/I_{b}). (I_{b}/V_{p})')
eq_T=sym.Eq(T,Vo_o_Vneg*Vneg_o_Ia*Ia_o_Ib*Ib_o_Vp)
display(eq_T)

eq_T_B=sym.Eq(T,-Ad*(R/2)*(Rc/((R/2)+Ra+Rc))*(1/(Rb+((Rc**-1+((R/2)+Ra)**-1)**-1))))
sym.pprint(sym.simplify(eq_T_B))
eq_T_B_val=eq_T_B.subs({R:eq_R_B.rhs , Ri:eq_Ri_B.rhs , Ra:100e3, Rb:100e3, Rc:eq_Rc_val.rhs , Ad:200e3})
sym.pprint(eq_T_B_val)
```

```python
eq_Avf_B=sym.Eq(Avf,eq_Av_B.rhs/(1-eq_T_B.rhs))
sym.pprint(sym.simplify(eq_Avf_B))
sym.pprint(eq_Avf_B.subs({R:eq_R_B.rhs , Ri:eq_Ri_B.rhs , Ra:100e3, Rb:100e3, Rc:eq_Rc_val.rhs , Ad:200e3}))
```

                      -A_d⋅(Ra⋅Rb + Ra⋅Rc + Rb⋅Rc)
    A_vf = ────────────────────────────────────────────────────
           A_d⋅R⋅Rc + R⋅Rb + R⋅Rc + 2⋅Ra⋅Rb + 2⋅Ra⋅Rc + 2⋅Rb⋅Rc
    A_vf = -29.941763270439

```python
#ERROR POR Vos y Ipol-, como en (+) no hay resistencia, Vo(Ipol+)= 0 => esto genera que el error sea mayor
Ipol, Vos = sym.symbols("Ipol_-, V_os")
DVo=sym.Function('\Delta V_o')(Ipol,Vos)
eq_Dvo=sym.Eq(DVo,sym.diff(DVo.subs(Vos,0))*Ipol+sym.diff(DVo.subs(Ipol,0))*Vos)
display(eq_Dvo)

#Si se aplica blackman
DVo=sym.Function('\Delta V_o')(Ipol,Vos,Vp)
eq_Dvo=sym.Eq(DVo,(sym.diff(DVo.subs({Vos:0,Vp:0}))/(1-T))*Ipol+(sym.diff(DVo.subs({Ipol:0,Vp:0}))/(1-T)*Vos))
display(eq_Dvo)
Image(filename='black3.png',width=300)
```

```python
#CASO A)
DVo=sym.Function('\Delta V_o')(Ipol)
eq_DVo_Ipol=sym.Eq(DVo,Ipol*(((R/2)**-1+Rf**-1)**-1)*-Ad/(1-T))
display(eq_DVo_Ipol)
eq_DVo_Ipol_val=(eq_DVo_Ipol.subs({T:eq_T_A_val.rhs, Ad:200e3,Ipol:80e-9,Rf:eq_Rf_A.rhs,R:eq_R_A.rhs , Ri:eq_Ri_A.rhs}))
sym.pprint(eq_DVo_Ipol_val)

DVo=sym.Function('\Delta V_o')(Vos)
eq_DVo_Vos=sym.Eq(DVo,Vos*(Ad/(1-T)))
eq_DVo_Vos_val=(eq_DVo_Vos.subs({T:eq_T_A_val.rhs, Ad:200e3,Vos:1e-3,Rf:eq_Rf_A.rhs,R:eq_R_A.rhs , Ri:eq_Ri_A.rhs}))
sym.pprint(eq_DVo_Vos_val)

DVo=sym.Function('\Delta V_o')(Ipol,Vos)
eq_DVo_Val=sym.Eq(DVo,abs(eq_DVo_Ipol_val.rhs)+abs(eq_DVo_Vos_val.rhs))
sym.pprint(eq_DVo_Val)

```

```python
#Error por CMRR no infinito y Ad no infinito
#Como la entrada + esta a tierra entonces el error por CMRR es despreciable
DVo=sym.Symbol('\Delta V_o')
FS, CMRR = sym.symbols('FS, CMRR')
eq_DVo_fin = sym.Eq(DVo,FS/abs(T)+FS/CMRR)
sym.pprint(eq_DVo_fin)
eq_DVo_fin = sym.Eq(DVo,FS/abs(T))
sym.pprint(eq_DVo_fin)
eq_DVo_fin_Val=eq_DVo_fin.subs({CMRR:31.6e3,FS:10,T:eq_T_A_val.rhs})
sym.pprint(eq_DVo_fin_Val)
```

                 FS    FS
    \Delta Vₒ = ─── + ────
                │T│   CMRR
                 FS
    \Delta Vₒ = ───
                │T│
    \Delta Vₒ = 0.00305

```python
#Sumando todos los errores
eq_DVo_T=sym.Eq(DVo,eq_DVo_Val.rhs + eq_DVo_fin_Val.rhs)
sym.pprint(eq_DVo_T)
```

    \Delta Vₒ = 0.0653509981955503

```python
#ERROR AC: Ancho de banda PLENA POTENCIA
t,Vpap,W,WHP,fHP=sym.symbols('t, V_pap, omega, omega_HP, f_HP')
SR = sym.Function('SR')()
Vo = sym.Function('V_o')(t)
eq_SR=sym.Eq(SR,sym.diff(Vo))
sym.pprint(eq_SR)

eq_Vo_t=sym.Eq(Vo,Vpap*sym.sin(W*t))
sym.pprint(eq_Vo_t)
eq_SR=sym.Eq(SR,sym.diff(eq_Vo_t.rhs,t).subs(t,0))
sym.pprint(eq_SR)
eq_WHP=sym.Eq(WHP,sym.solve(eq_SR,W)[0])
sym.pprint(eq_WHP)
eq_WHP_Val=eq_WHP.subs({SR:500e3, Vpap:10})
sym.pprint(eq_WHP_Val)
eq_fHP_Val=sym.Eq(fHP,eq_WHP_Val.rhs/(2*np.pi))
sym.pprint(eq_fHP_Val)
```

           d
    SR() = ──(Vₒ(t))
           dt
    Vₒ(t) = Vₚₐₚ⋅sin(ω⋅t)
    SR() = Vₚₐₚ⋅ω
           SR()
    ω_HP = ────
           Vₚₐₚ
    ω_HP = 50000.0
    f_HP = 7957.74715459477

```python
#ERROR AC amcho de banda de peq señal
GBW=1e6
Wh=GBW/Avf
WH1=2*np.pi*GBW/30
display(WH1)
Image(filename='WH.png',width=300)

```

```python
# Errores AC ERROR VECTORIAL NORMALIZADO
#Función de transferencia de la ganancia
W, Wh, Avfi = sym.symbols('omega, omega_h, A_vfi')

Avf=Avfi/(1+sym.I*W/Wh)
display(Avf)
avf=1/(1+sym.I*W/Wh)
display(avf)
avf_mod=1/sym.sqrt(1+(W/Wh)**2)
display(avf_mod)
avf_arg=-sym.atan(W/Wh)
display(avf_arg)
Ev=abs(1-avf)
display(Ev)
Ev_mod=1-1/sym.sqrt(1+(W/Wh)**2)
display(Ev_mod)
Ev_arg=(sym.pi/2) - sym.atan(W/Wh)
display(Ev_arg)
```

```python

```