$\newcommand{\xv}{\mathbf{x}}
 \newcommand{\wv}{\mathbf{w}}
 \newcommand{\vv}{\mathbf{v}}
 \newcommand{\yv}{\mathbf{y}}
 \newcommand{\zv}{\mathbf{z}}
 \newcommand{\tv}{\mathbf{t}}
 \newcommand{\Chi}{\mathcal{X}}
 \newcommand{\R}{\rm I\!R}
 \newcommand{\sign}{\text{sign}}
 \newcommand{\Tm}{\mathbf{T}}
 \newcommand{\Xm}{\mathbf{X}}
 \newcommand{\Zm}{\mathbf{Z}}
 \newcommand{\Wm}{\mathbf{W}}
 \newcommand{\Ym}{\mathbf{Y}}
 \newcommand{\I}{\mathbf{I}}
 \newcommand{\muv}{\boldsymbol\mu}
 \newcommand{\Sigmav}{\boldsymbol\Sigma}
$

# Support Vector Machines (SVM)

What is a good classifier? There can be multiple hyperplanes that divides the data in two classes. SVMs, largin margin classifiers, choose the hyperplane that has a large margine.

The idea of support vector machines, or the large margin linear classifiers, first suggested in 1963 by Vapnik, et al. In 1992, [Boser, et al.](https://dl.acm.org/citation.cfm?id=130401), extended this to nonlinear classification and [Cortes, et al.](http://image.diku.dk/imagecanon/material/cortes_vapnik95.pdf), further examines non-separable data in 1995. After this, lots of large margin algorithms have been developed.

Let us take a look at the classifier choice with a largin margin and talk about other issues.

## What is the best classifier?

## Large Margin Classifier

Margin is the distance or room between the hyperplane and the closest data samples from each class.
The margin gives the classifer flexibility to changes of data or variant future data. Thus, having the largest margin can be a good choice.

# Geometric Margin

The margin, $d$, can be calculated by the linear projection with a unit vector in the direction of $\wv$, $\hat{\wv}$.

Let us say that there is a point $\zv$ on a line,
$$
\wv^\top \zv + b = 0.
$$

Also, we assume that $\zv$ is orthogoal to a data point $\xv$:
$$
\zv = \xv + d \hat{\wv}.
$$

Plugging in this to the hyperplane equation $\yv = \wv^\top \xv + b$,

$$
\begin{align*}
\wv^\top \zv + b &= \wv^\top (\xv + d \hat{\wv}) + b \\
0 &= \wv^\top \xv + d \wv^\top \hat{\wv} + b \\
0 &= y + d \wv^\top \frac{\wv}{\Vert \wv \Vert} = y + d \frac{\Vert \wv \Vert^2}{\Vert \wv \Vert}\\
d &= -\frac{y}{\Vert \wv \Vert}
\end{align*}
$$

Now, let us check the sign of the label. We assume that the data point $\xv$ here was from negative samples.
That is, the $\yv$ is negative.
Looking back our perceptron, we use our label to unify the objective function:

$$
t_i y_i = \tv_i (\wv^\top \xv_i + b) \gt 0.
$$

If we rescaling the weights and bias, we can get the following canonical hyperplane with the minimum distance to be 1:

$$
\min_{n=1, \dots, N} t_n (\wv^\top \xv + b) = 1.
$$

Under this assumption, we get the margin that equals to

$$d = \frac{1}{\Vert \wv \Vert}.$$

Remember that we want *maximize* this margine $\frac{1}{\Vert \wv \Vert}$, so we can invert this to *minimize*  $\Vert \wv \Vert$.

## The Linear SVM

The objective function can be written as:

$$
\begin{equation*}
\begin{aligned}
& \underset{\wv, b}{\text{minimize}}
& & \frac{1}{2} \Vert \wv \Vert^2 \\
& \text{subject to}
& & t_i (\wv^\top \xv_i + b) \ge 1, \; i = 1, \ldots, N.
\end{aligned}
\end{equation*}
$$

The Lagrangian is

$$
L = \frac{\Vert \wv \Vert^2 }{2} - \sum_i^N \lambda_i \big( t_i (\wv^\top \xv_i + b) - 1 \big).
$$

Let us find the optimal solution with gradient!
With positive Lagrangian multipliers $\lambda = [\lambda_1, \ldots, \lambda_N]$ and the constraint,
we can find the two additional gradient against $\lambda$ and $\wv$ and set to to zero to obtain the optimal conditions. That is,

$$
\begin{cases}
\frac{\partial L}{\partial \wv} = 0, \\
\frac{\partial L}{\partial b} = 0, \\
t_i (\wv^\top \xv_i + b) - 1 \ge 0, \\
\lambda \ge 0, \\
\lambda_i \big(t_i (\wv^\top \xv_i + b) - 1\big) = 0.
\end{cases}
$$

This is called **Karush–Kuhn–Tucker (KKT)** conditions (See http://www.svms.org/kkt/ and http://mat.gsia.cmu.edu/classes/QUANT/NOTES/chap4/node6.html).

Going through few steps for the derivaiton,
$$
\begin{align*}
\frac{\partial L}{\partial \wv} &= \wv - \sum_i^N \lambda_i t_i \xv_i = 0\\
\wv &= \sum_i^N \lambda_i t_i \xv_i,
\end{align*}
$$
and
$$
\begin{align*}
\frac{\partial L}{\partial b} &= - \sum_i^N \lambda_i t_i = 0\\
    \sum_i^N \lambda_i t_i &= 0.
\end{align*}
$$

Rewriting and summarizing the KKT conditions,

$$
\begin{align*}
&\wv = \sum_i^N \lambda_i t_i \xv_i, \\
&\sum_i^N \lambda_i t_i = 0, \\
&t_i (\wv^\top \xv_i + b) - 1 \ge 0, \\
\\
&\lambda \ge 0, \\
\\
&\lambda_i \big(t_i (\wv^\top \xv_i + b) - 1\big) = 0.
&\end{align*}
$$

# Dual Problem

For duality convention, let us change the notation $\lambda$ to $\alpha$:

$$
\begin{equation}
    \wv = \sum_i^N \alpha_i t_i \xv_i. \label{eq_w} \tag{1}
\end{equation}
$$

Since there is no solution for $b$, use the KKT condition and find it:

$$
\begin{align}
\lambda_i \big(t_i (\wv^\top \xv_i + b) - 1\big) &= 0 \\
t_i (\wv^\top \xv_i + b) &= 1 \\
\wv^\top \xv_i + b = t_i \\
\\
b = t_i - \wv^\top \xv_i. \label{eq_b} \tag{2}
\end{align}
$$

With the KKT conditions,
$$
\alpha_i > 0 \\
\text{and}\\
t_i (\wv^\top \xv + b) = 1,
$$
the $\xv_i$ are the data samples that lie on the re-weighted margin *1*.

The corresponding $\xv_i$ are called **support vectors**, that support the hyperplane construction.

Now, using the KKT conditions, replace $\wv$ and $b$ in the objective function $L$ with $\eqref{eq_w}$ and $\eqref{eq_b}$,

$$
\begin{align*}
L &= \frac{\Vert \wv \Vert^2 }{2} - \sum_i^N \alpha_i \big( t_i (\wv^\top \xv_i + b) - 1 \big) \\
  &= \frac{1}{2} \Big( \sum_i^N \alpha_i t_i \xv_i \Big)^\top \Big( \sum_i^N \alpha_i t_i \xv_i \Big)
     - \sum_i^N \alpha_i t_i \Big( \sum_j^N \alpha_j t_j \xv_j \Big)^\top \xv_i
     - b \sum_i^N \alpha_i t_i
     + \sum_i^N \alpha_i \\
  &= \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \xv_i^\top \xv_j
     - \sum_i^N \sum_j^N  \alpha_i \alpha_j t_i t_j \xv_i^\top  \xv_j
     - b \sum_i^N \alpha_i t_i
     + \sum_i^N \alpha_i \\
  &= - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \xv_i^\top \xv_j + \sum_i^N \alpha_i.
\end{align*}
$$

Let us rewrite the optimization problem in duality:

$$
\begin{equation*}
\begin{aligned}
& \underset{\alpha}{\text{maximize}}
& & \sum_i^N \alpha_i - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \xv_i^\top \xv_j \\
& \text{subject to}
& & \sum_i^N \alpha_i t_i = 0, \\
& & &\alpha_i \ge 0.
\end{aligned}
\end{equation*}
$$

# Soft Margin SVM

So far, we constrained the data is clearly separable:

$$
t_i (\wv^\top \xv_i + b) \ge 1.
$$

To relax this constraints, we can define additional slack variables $\xi_i \ge 0$:

$$
t_i (\wv^\top \xv_i + b) \ge 1 - \xi_i.
$$

With the slack variables, the objective function is

$$
\begin{equation*}
\begin{aligned}
& \underset{\wv, b, \xi}{\text{minimize}}
& & \frac{1}{2} \Vert \wv \Vert^2 + C \sum_i \xi_i \\
& \text{subject to}
& & t_i (\wv^\top \xv_i + b) \ge 1 - \xi_i,\\
& & & \xi_i \ge 0 \quad \; i = 1, \ldots, N.
\end{aligned}
\end{equation*}
$$

From the Lagrangin,

$$
L = \frac{\Vert \wv \Vert^2 }{2} - \sum_i^N \alpha_i \big( t_i (\wv^\top \xv_i + b) - 1 + \xi_i \big) - \sum_i^N \beta_i \xi_i + C \sum_i^N \xi_i,
$$

we get the same saddle point equations as in $\eqref{eq_w}$ and $\eqref{eq_b}$ along with the following:

$$
\frac{\partial L}{\partial \xi_i} = C - \alpha_i - \beta_i = 0.
$$

This leads us to dual problem of

$$
\begin{equation*}
\begin{aligned}
& \underset{\alpha}{\text{maximize}}
& & \sum_i^N \alpha_i - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \xv_i^\top \xv_j \\
& \text{subject to}
& & \sum_i^N \alpha_i t_i = 0, \\
& & &\alpha_i \ge 0, \\
& & &\beta_i \ge 0, \\
& & &C - \alpha_i - \beta_i = 0.
\end{aligned}
\end{equation*}
$$

Using the last saddle point equaiton, we can replace $\beta_i$ inequality with the
$$
0 \le \alpha_i \le C.
$$

$$
\begin{equation*}
\begin{aligned}
& \underset{\alpha}{\text{maximize}}
& & \sum_i^N \alpha_i - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \xv_i^\top \xv_j \\
& \text{subject to}
& & \sum_i^N \alpha_i t_i = 0, \\
& & &0 \le \alpha_i \le C.
\end{aligned}
\end{equation*}
$$

# Kernels

For the data that cannot be classified with a linear classifier, we used a non-linear function mapping from $\xv$ to $\phi(\xv)$. We can use this nonlinear mapping for the SVM as well:

$$
\begin{equation*}
\begin{aligned}
& \underset{\wv, b, \xi}{\text{minimize}}
& & \frac{1}{2} \Vert \wv \Vert^2 + C \sum_i \xi_i \\
& \text{subject to}
& & t_i (\wv^\top \phi(\xv_i) + b) \ge 1 - \xi_i,\\
& & & \xi_i \ge 0 \quad \; i = 1, \ldots, N.
\end{aligned}
\end{equation*}
$$

In duality, we can write the optimization problem as

$$
\begin{equation*} \
\begin{aligned}
& \underset{\alpha}{\text{maximize}}
& & \sum_i^N \alpha_i - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j  \phi(\xv_i)^\top \phi(\xv_j) \\
& \text{subject to}
& & \sum_i^N \alpha_i t_i = 0, \\
& & &0 \le \alpha_i \le C.
\end{aligned}
\end{equation*}
$$

In nonlinear feature space $\Phi$, the inner product $\phi^\top \phi$ compute the euclidean distance between betweeen two feature vectors. We define this distance, or similarity measure as a kernel $k$:

$$
k(\xv, \vv) = \phi(\xv)^\top \phi(\vv).
$$

With this kernel notation, the dual problem can be written as

$$
\begin{equation*}
\begin{aligned}
& \underset{\alpha}{\text{maximize}}
& & \sum_i^N \alpha_i - \frac{1}{2} \sum_i^N \sum_j^N \alpha_i \alpha_j t_i t_j k(\xv_i, \xv_j) \\
& \text{subject to}
& & \sum_i^N \alpha_i t_i = 0, \\
& & &0 \le \alpha_i \le C.
\end{aligned}
\end{equation*}
$$

Here are summary of well-known stard kernels.
- Linear Kernel
$$ k(\xv, \yv) = \xv^\top \yv $$
- Polynomial Kernel
$$ k(\xv, \yv) = (\xv^\top \yv + 1)^d $$
- Gaussian Kernel
$$ k(\xv, \yv) = \exp(-\gamma \Vert \xv - \yv \Vert^2) $$

# Then, How do we solve this?

So far, we set up the optimization problems. How do we need to solve or program?
Any of optimization tools can be used to solve this, but there are some fast optimization tools for the SVM including the sequential minimal optimization (SMO) and libsvm.

Scikit-learn uses liblinear for the primal and libsvm for the dual problem.

Now, let us implement svm and try to classify the circles data. You can use any libsvm wrapper like scikit-learn, but for exercise, let us use gradient descent for learning.

The derivative of the objective function w.r.t. $\alpha$ is

$$
\frac{\partial L}{\partial \alpha_i} = 1 - t_i \sum_j^N \alpha_j t_j k(\xv_i, \xv_j) - \sum_j^N t_i.
$$

From

$$
\wv = \sum_i^N \alpha_i t_i \xv_i,
$$

$$
\wv^\top \xv_i = \sum_j^N \alpha_j t_j \xv_j^\top \xv_i = \sum_j^N \alpha_j t_j k(\xv_j, \xv_i).
$$
Thus,
$$
b = t_i - \wv^\top \xv_i =
t_i - \sum_j^N \alpha_j t_j k(\xv_j, \xv_i),
$$

we can make a prediction using the $\wv$, $b$, and the kernel evaluations.

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from sklearn.datasets import make_circles

X, T = make_circles(n_samples=800, noise=0.07, factor=0.4)
T[T==0] = -1

plt.figure(figsize=(8, 8))
plt.scatter(X[:, 0], X[:, 1], marker='o', c=T)
plt.title("Circles")
```

```python
# linear kernel
def linear_kernel(x, y):
    return x @ y

def polynomial_kernel(x, y, d=3):
    return (x @ y +1) ** d

def gaussian_kernel(x, y, r=1.):
    d = (x - y)
    return np.exp(-r * (d @ d))
```

```python
from functools import partial

kernelf = partial(linear_kernel)

###### build a kernel matrix (optional)
def build_Kmat(X, Xn=X, kernelf=linear_kernel):
    N = X.shape[0]
    Nn = Xn.shape[0]

    K = # TODO: finish this line to create a kernel matrix
    return K
```

```python
K = build_Kmat(X, X, kernelf)
T.shape, K[0, :].shape
```

    ((800,), (800,))

```python
def train(X, T, K, n_epochs=1, learning_rate=0.01, C=10):

    N = X.shape[0]

    #alpha = np.random.rand(N)
    alpha = np.zeros(N)
    T_sum = np.sum(T)
    for n in range(n_epochs):
        for i in range(N):
            grad = # TODO: fill this gradient
            alpha += learning_rate * grad

            alpha[alpha <= 0] = 0
            alpha[alpha >= C] = C
    b = # TODO: Finish this b
    return alpha, b

```

```python
# predict
#
# Parameters
# ===========
# Xtest    test input
# alpha    SVM learned parameters
# X        Support Vectors
# T        Target for SVs
# K        kernel matrix if exists
def predict(Xtest, alpha, b, **params):
    # support vectors
    i_sv = np.where(np.logical_and(alpha<C, alpha>0))[0]

    K = params.pop('K', None)
    T = params.pop('T', None)
    if T is None:
        raise ValueError("predict: None support vector targets!")

    if K is None:
        kernelf = params.pop('kernelf', linear_kernel)
        X = params.pop('X', None)
        if X is None:
            raise ValueError("predict: None support vectors!")
        print("rebuilding kernel matrix")
        K = build_Kmat(X, Xtest, kernelf)

    y = # TP
    return y

```

```python
# train
alpha, b = train(X, T, K)

# prediction
y = predict(X, alpha, b, K=K, T=T)

# plot
plt.plot(T)
plt.plot(np.sign(y))
plt.show()

print("Accuracy: ", 100 * np.sum(T == np.sign(y)) / N, "%")
```

```python

x = np.linspace(-1.5, 1.5, 100)
y = np.linspace(-1.5, 1.5, 100)

xs, ys = np.meshgrid(x, y)

Xt = np.vstack((xs.flat, ys.flat)).T

Y = predict(Xt, alpha, b, X=X, T=T)
classes = np.sign(Y)

zs = classes.reshape(xs.shape)

plt.figure(figsize=(6,6))
plt.contourf(xs, ys, zs.reshape(xs.shape), alpha=0.3)
plt.title("Decision Boundary")

plt.scatter(X[:, 0], X[:, 1], marker='o', c=T+3)
```

```python
# nonlinear now
gausskf = partial(gaussian_kernel, r=0.1)
K = build_Kmat(X, X, gausskf)
```

```python
alpha, b = train(X, T, K)
```

```python
# predict
y = predict(X, alpha, b, K=Kg, T=T)
print("Accuracy: ", 100 * np.sum(T == np.sign(y)) / N, "%")
```

    Accuracy:  100.0 %

```python

x = np.linspace(-1.5, 1.5, 100)
y = np.linspace(-1.5, 1.5, 100)

xs, ys = np.meshgrid(x, y)

Xt = np.vstack((xs.flat, ys.flat)).T

Y = predict(Xt, alpha, b, X=X, T=T, kernelf=gausskf)
classes = np.sign(Y)

zs = classes.reshape(xs.shape)

plt.figure(figsize=(6,6))
plt.contourf(xs, ys, zs.reshape(xs.shape), alpha=0.3)
plt.title("Decision Boundary")

plt.scatter(X[:, 0], X[:, 1], marker='o', c=T+3)
```


Earthquake physics

 Dynamic Rupture in 1D

##### Authors:
* Mitchell Williams
* Kenneth Duru

---

## Description

### Background

This notebook simulates  dynamic earthquake ruptures and elastic wave propagation in one space dimensions (1D). Spontaneously propagating shear ruptures on a frictional interface in an elastic solid is a useful idealization of natural earthquakes. The conditions relating slip-rate and fault strength are often expressed as nonlinear friction laws.

During an earthquake, two sides of the fault, initially held in contact by a high level of frictional resistance, slip suddenly when that resistance catastrophically decreases, generating strong ground shaking which is carried by seismic waves to remote areas, far away from fault zones

This notebook describes, and implements, a high order accurate finite difference method for enforcing nonlinear friction laws, in a consistent and provably stable manner, suitable for efficient explicit time integration and accurate propagation of seismic waves in heterogeneous media with free surface boundary conditions.

--------------------------------------------------------------------
### The elastic wave equation
--------------------------------------------------------------------
Consider two elastic solids separated by a fault at the interface $x = 0$. In each elastic solid, wave motion is described by the elastic wave equation, in velocity-stress formulation:

\begin{align}
{\rho} \frac{ \partial v}{\partial t} = \frac{ \partial \sigma}{\partial x},   {\quad}   \frac{1}{\mu} \frac{ \partial \sigma}{\partial t} = \frac{\partial v}{\partial x},
\end{align}

 with particle velocity ${v}$, stress field ${\sigma }$, shear modulus $\mu$, and density ${\rho}$.

-----------------------------------------------------------------

Denoting the fields in the positive and the negative parts of the fault with the superscripts $\pm$, we introduce the jumps in particle velocities by $\lbrack{v}\rbrack = {v}^{+}-{v}^{-}$, and  the total traction by $\sigma = \sigma_{0} + \Delta{\sigma}$, where $\sigma_{0}$ is the initial background shear traction and $\Delta{\sigma}$ is the traction changes on the fault. The conditions on the fault connecting the two elastic solids are

\begin{align}
\text{force balance}&: \quad \sigma^{+} = \sigma^{-} = \sigma, \\
\text{friction laws}&: \quad \sigma = \sigma_n\frac{f(V, \theta)}{V}\lbrack{v}\rbrack, \quad V = \left|\lbrack{v}\rbrack\right|.
\end{align}

 Here $\sigma_n > 0$ is the compressive normal stress and $ f(V, \theta) \ge 0 $ is the nonlinear fricition coefficient relating to the fault's shear strength, $\tau = |\sigma| > 0$, and will be described  in more detail below.

-------------------------------------------------------------

### Friction laws

It is necessary to consider the role of friction in all of this because it modulates the stresses and slip speed as the separate sides of the fault slip past each other. The first thing to consider is that as the fault slips it experiences shear strength ${\tau}$ which is exerted by the normal stresses on the fault ${\sigma_n}$. The way these two variables are related is through the friction coefficient ${f}$: ${\quad} {\tau} = f \cdot {\sigma_n}$.

 For the purposes of this notebook  we consider two widely used models of friction, *slip-weakening* and *rate-and-state*.

#### Slip Weakening Friction
Slip weakening friction describes a friction behavior in which friction coefficient weakens linearly with slip.  When the fault is at rest the friction coefficient is denoted by static friction coefficient ${f_s}$, when the fault is in motion the friction coefficient is denoted by the dynamic friction coefficient ${f_d}$. An earthquake begins when the load $\tau$ at a finite patch on the fault overcomes the peak frictional strength, $\tau \ge \tau_p$, ${\quad \tau_p = f_s \cdot \sigma_n}$.  As the earthquake continues the fault surfaces slide relative to each other  and the frictional coefficient evolves from linearly from static friction coefficient $f_s$  to the dynamic friction coefficient ${f_d}$. The fault is fully weakened when the slip reaches the critical slip ${D_c}$. This can be seen in the figure below which plots the friction coefficient as a function of slip.

#### Rate and State Friction

Rate and state friction is a constitutive law that was empirically found and is considered more realistic than slip weakening. It takes the form:

\begin{align}
 &\tau = \sigma_n \cdot f(V, \theta), \quad f(V, \theta) = [ f_0 + a \cdot ln(\frac {V}{V_0}) + b \cdot ln( \frac{V_0 \theta}{D_c})], \\
  &\frac {d \theta}{dt} = G(V, \theta).
 \end{align}

In this equation ${V}$ is the current slip velocity and ${V_0}$ is a reference slip velocity while ${f_0}$ is the steady state friction coefficient at the reference velocity: ${f_{V= V_0}}$. There is the state variable ${\theta}$, and the state evolution law $G(V, \theta)$. We consider specifically the aging law: ${ G(V, \theta) = 1 - \frac {V\theta }{D_c}.\quad}$ At steady state the state variable is proportional to the critical distance divided by velocity ${ \theta_{ss} = \frac {D_c}{V}}$

Here ${a}$ is the *direct effect* and is used to model how the system responds to velocity changes. ${b}$ is the *evolution effect* and describes the magnitude of the steady state friction.  ${D_c}$ is the critical distance and corresponds to the slip length over which the evolution of ${a \to b}$ occurs. The relationship of ${a}$ and ${b}$ to the slip weakening friction variables is as follows: ${\quad b = \frac {df_s}{d(ln(t))} \quad, \quad (a-b) = \frac {df_d}{d(ln(V))} \quad}$ where ${t}$ is the amount of time of contact without an earthquake. It is interesting to notice that there is a relationship between slip velocity ${V}$ and dynamic friction ${f_d}$ that depends on the difference ${(a-b)}$. If ${(a-b)>0}$ it shows that there is a velocity strengthening relationship ${\frac {df_d}{d(ln(V))} > 0}$. Any rupture that enters this region will arrest and is called a *stable* region. If ${(a-b)<0}$ it shows that there is a velocity weakening relationship ${\frac {df_d}{d(ln(V))} < 0}$, this is where earthquakes can nucleate and grow and is called *unstable*. On the border between *stable* and *unstable* regions is a zone of *conditional stability* where ruptures can sustain propagation while not growing or terminating. This notebook only exists in the unstable zone.

The equations are discretized using the SBP-SAT finite difference scheme. Time integration is performed using the classical fourth order accurate Runge-Kutta method. For more elaborate discussions, we refer the reader to the references and the  notebooks on the SBP-SAT method. A summary of the numerical implementations used in the this notebook is presentated in the flow-chart below.

### Putting it together

In the parameter window you can modulate the material properties, run time, domain size, and CFL criterion. In addition, you can choose rate and state friction or slip weakening friction, notice the differences that occur with this choice. In the Calculations and plotting cell the resulting on fault tractions and slip velocities are solved for in the time_integrator.elastic_RK4 function, which also calls on the other associated functions. The bottom cell contains 5 output plots of the rupture behavior.

#### References
Byerlee, J. “Friction of Rocks.” Rock Friction and Earthquake Prediction, vol. 116, 1978, pp. 615–626., doi:10.1007/978-3-0348-7182-2_4.

Duru, Kenneth, and Eric M. Dunham. “Dynamic Earthquake Rupture Simulations on Nonplanar Faults Embedded in 3D
Geometrically Complex,
${\quad}$ Heterogeneous Elastic Solids.” Journal of Computational Physics, vol. 305, 2016, pp. 185–207., doi: 10.1016/j.jcp.2015.10.0215

Gustafsson, Bertil. High Order Difference Methods for Time Dependent PDE. Springer, 2008.

Kozdon, J. E., E. M. Dunham, and J. Nordström (2012), Interaction of waves with frictional interfaces using summation-by-parts difference operators:
${\quad}$  Weak enforcement of nonlinear boundary conditions, Journal of Scientific Computing, 50(2), 341-367, doi:10.1007/s10915-011-9485-3

Leeman, John. “Modeling Rate and State Friction with Python.” SciPy 2016. Austin, USA.

Scholz, Christopher H. “Earthquakes and Friction Laws.” Nature, vol. 391, no. 6662, 1998, pp. 37–42., doi:10.1038/34097.

### Exercises

Explore the following questions by interacting with the variables in the parameter window.

* **1)** Explore the differences between slip weakening and rate and state frictions. How does this alter the slip rate and Traction drop profiles?

* **2)** How does altering the CFL change the model time? Does lowering it increase or decrease the amount of numerical artifacts?

* ** 3)** Does changing the density alter the slip rate?

* **4) **What do you notice when altering the order of accuracy in terms of computation time and accuracy of the model? Is the trade-off worth it for this model?

* **5)** What is the relationship between fault slip and traction in both types of friction?

```python
# Import necessary routines

import numpy as np
import matplotlib.pyplot as plt
import time_integrator
import rate
import utils
import timeit

plt.switch_backend("nbagg")           # plots within this notebook
```

```python
# Parameter cell
# All user defined parameters can be entered in this window.

tend = 2.0            # final time
cfl = 0.5             # CFL number

L = 15.0              # length of the domain (km)
nx = 601              # grid points in x
cs = 3.464            # shear wave velocity (km/s) (can be an array)
rho = 2.6702          # density [g/cm^3]

iplot = 5             # snapshot frequency
order = 6             # order of accuracy

fric_law = 'SW'       #'SW = slip weakening', 'RS' = rate and state

```

```python
# Initialization cell
# Do not alter
# spacing and material parameters

ny = nx                              # grid points in y
dx = L/(nx-1)                        # grid increment in x
dy = dx                              # grid increment in y

rho_l = rho                          # density [g/cm^3]
mu_l = rho_l*cs**2                   # shear modulus [GPa]
Zs_l = rho_l*cs                      # shear impedance

rho_r = rho                          # density [g/cm^3]
mu_r = rho_r*cs**2                   # shear modulus [GPa]
Zs_r = rho_r*cs                      # shear impedance

#Initialize the domains L R
y_l = np.zeros((nx, 1))
y_r = np.zeros((nx, 1))

# Initial particle velocity perturbation and discretize the domain L R
for j in range(0, nx):
    y_l[j, :] = -L + j*dx            # discrete domain
    y_r[j, :] =  0 + j*dx            # discrete domain

# Time stepping parameters
t=0   # initial time
dt = (cfl/cs)*dx                     # Time step
nt = int(round(tend/dt))             # number of time steps

# Initialize: particle velocity (v); and shear stress (s); and returns L
v_l = np.zeros((nx,1))
s_l = np.zeros((nx,1))
U_l = np.zeros((nx,1))
V_l = np.zeros((nx,1))
U_t_l = np.zeros((nx,1))
V_t_l = np.zeros((nx,1))
U_x_l = np.zeros((nx,1))
V_x_l = np.zeros((nx,1))

# Initialize: particle velocity (v); and shear stress (s); and returns R
v_r = np.zeros((nx,1))
s_r = np.zeros((nx,1))
U_r = np.zeros((nx,1))
V_r = np.zeros((nx,1))
U_t_r = np.zeros((nx,1))
V_t_r = np.zeros((nx,1))
U_x_r = np.zeros((nx,1))
V_x_r = np.zeros((nx,1))

# Initialize slip parameters
slip = np.zeros((1, 1))
psi = np.zeros((1, 1))

# Initial particle velocity perturbation and discretize the domain
for j in range(0, nx):
    y_l[j, :] = -L + j*dx            # discrete domain
    y_r[j, :] =  0 + j*dx

# Boundary condition reflection coefficients  L
r0_l = 1                             # r=0:absorbing, r=1:free-surface, r=-1: clamped
r1_l = 1                             # r=0:absorbing, r=1:free-surface, r=-1: clamped
# penalty parameters L
tau_11_l = 1
tau_12_l = 1
tau_21_l = 1
tau_22_l = 1

# Boundary condition reflection coefficients  R
r0_r = 1                            # r=0:absorbing, r=1:free-surface, r=-1: clamped
r1_r = 1                            # r=0:absorbing, r=1:free-surface, r=-1: clamped
# penalty parameters R
tau_11_r = 1
tau_12_r = 1
tau_21_r = 1
tau_22_r = 1

if fric_law not in ('SW', 'RS'):
    # Choose friction law: fric_law
    # We use linear (LN: T = alpha*v, alpha >=0)
    # Slip-weakening (SW)
    # Rate-and-state friction law (RS)

     print('friction law not implemented. choose fric_law = SW or fric_law = RS')
     exit(-1)

if fric_law  in ('SW'):
     alpha = 1e1000000                      # initial friction coefficient
     slip[:,:] = 0.0                        # initial slip (in m)
     Tau_0 = 81.24+1.0*0.36                 # initial load (81.24 in MPa), slight increase will unlock the fault
     alp_s = 0.677                          # stastic friction
     alp_d = 0.525                          # dynamic friction
     D_c = 0.4                              # critical slip
     sigma_n = 120.0                        # normal stress

     # These are not needed for the slip weakening case
     psi[:,:] = 0.0                         # initial condition for the state variable in friction law
     L0 = 1.0                               # state evolution distance
     f0 = 1.0                               # referance friction coeff
     a = 1.0                                # direct effect
     b = 1.0                                # evolution parameter
     V0 = 1.0                               # reference slip rate

if fric_law  in ('RS'):
     alpha = 1e1000000                      # initial friction coefficient
     slip[:,:] = 0.0                        # initial slip (in m)
     Tau_0 = 81.24+0.1*0.36                 # initial load (81.24 in MPa), slight increase will unlock the fault
     psi[:,:] = 0.4367                      # initial condition for the state variable in friction law
     L0 = 0.02                              # state evolution distance
     f0 = 0.6                               # referance friction coeff
     a = 0.008                              # direct effect
     b = 0.012                              # evolution parameter
     V0 = 1.0e-6                            # reference slip rate
     sigma_n = 120.0                        # background normal stress

     # These are not needed for the rate and state case
     alp_s = 1.0                             # stastic friction
     alp_d = 1.0                             # dynamic friction
     D_c = 1.0                               # critical slip

friction_parameters = [fric_law, alpha, Tau_0, L0, f0, a, b, V0, sigma_n, alp_s, alp_d, D_c]
#                        0         1      2     3   4  5  6   7    8       9      10    11

# Initial variables
Vd = [0]                                     # initial velocity
Sd = [Tau_0]                                 # background stress
T = [t]                                      # time
Slip = [0]

# setting slip, slip rate, traction, and time vectors to empty sets
traction_vector = []
slip_vector = []
time_vector = []
sliprate_vector = []

```

```python
# Calculations and plottings for on fault slip velocity and stress

# Initialize animated plot for velocity and stress
fig1 = plt.figure(figsize=(10,10))
ax1 = fig1.add_subplot(5,1,1)
line1 = ax1.plot(y_l, v_l, 'r', y_r, v_r, 'g')
plt.title('Off fault velocity')
plt.xlabel('x[km]')
plt.ylabel('velocity [m/s]')

ax2 = fig1.add_subplot(5,1,2)
line2 = ax2.plot(y_l, s_l, 'r', y_r, s_r, 'g')
plt.title('Off fault stress')
plt.xlabel('x[km]')
plt.ylabel('stress [MPa]')

ax3 = fig1.add_subplot(5,1,3)
line3 = ax3.plot(0,0,'g')
plt.title('Slip')
plt.xlabel('Time [s]')
plt.ylabel('Slip [m]')

ax4 = fig1.add_subplot(5,1,4)
line4 = ax4.plot(0,0,'g')
plt.title('Slip rate')
plt.xlabel('Time [s]')
plt.ylabel('Slip rate[m/s]]')

ax5 = fig1.add_subplot(5,1,5)
line5 = ax5.plot(0,0,'g')
plt.title('Traction')
plt.xlabel('Time [s]')
plt.ylabel('stress [MPa]')

plt.tight_layout()
plt.ion()
plt.show()

# Loop through time and evolve the wave-fields using ADER time-stepping scheme of N+1 order of accuracy
start = timeit.default_timer()

n = 0                             # counter
for t in utils.drange (0.0, tend+dt,dt):
    n = n+1

    # compute numerical solution
    time_integrator.elastic_RK4(v_l, s_l, v_l, s_l, rho_l, mu_l, nx, dx, order, y_l, t, dt, r0_l, r1_l,  tau_11_l,\
                    tau_21_l, tau_12_l, tau_22_l,v_r, s_r, v_r, s_r, rho_r, mu_r, nx, dx, order,\
                                y_r, t, dt, r0_r, r1_r,  tau_11_r, tau_21_r, tau_12_r, tau_22_r,\
                               slip, psi, slip, psi,friction_parameters)

    # appending slip, slip rate, traction, and time vector through each time step.
    traction_vector.append(Tau_0+s_l[-1,0])
    slip_vector.append(slip[0,0])
    time_vector.append(n*dt)
    sliprate_vector.append(np.abs(v_r[0,0]-v_l[-1,0]))

    # Updating plots
    if n % iplot == 0:
        for l in line1:
            l.remove()
            del l
        for l in line2:
            l.remove()
            del l
        for l in line3:
            l.remove()
            del l

        # Display lines
        line1 = ax1.plot(y_r, v_r, 'r',y_l, v_l, 'g')
        ax1.legend(iter(line1),('Right', 'Left'))
        line2 = ax2.plot(y_r, s_r+Tau_0, 'r',y_l, s_l+Tau_0, 'g')
        ax2.legend(iter(line2),('Right', 'Left'))
        line3 = ax3.plot(time_vector,slip_vector,'g')
        ax3.legend(iter(line3),('slip'))
        line4 = ax4.plot(time_vector,sliprate_vector,'g')
        ax4.legend(iter(line4),('slip rate'))
        line5 = ax5.plot(time_vector,traction_vector,'g')
        ax5.legend(iter(line5),('traction'))
        plt.gcf().canvas.draw()

plt.ioff()
plt.show()
# Simulation end time
stop = timeit.default_timer()
print('total simulation time = ', stop - start)                   # print the time required for simulation
print('spatial order  of accuracy = ', order)                     # print the polynomial degree used
print('number of grid points = ', nx)                             # print the degree of freedom
```


# Cross matching of 3D data sets

The idea is to take two different data sets, and perform a cross-match of objects between the two.

Perfect cross-match
--------

In the case of a perfect cross-match, we have $a \in A$ matches with $b \in B$ if $a = b$.

Cross-match with uncertainty
--------

Say now A and B have some uncertainty in the position of their objects, and their cross-match is subject to a definition of neighbourhood, or minimum "distance" between them (keep in mind, we are in the 3 dimensional space!).
Depending on how we define the neighbourhood (that is which metric to compare two objects a priori close but not identical), the objects matching can be very different.

Let's illustrate this with an example.

Question: Does A match with B? Does C match with D?
--------

**Using the distance to centers: **
Let's suppose our metric is the (euclidean) distance between the centers of objects, and two points are said identical if their distance is smaller than $\epsilon$. We have $||AB|| < \epsilon$ and $||CD|| > \epsilon$, therefore we would say that A matches with B, but C does not match with D.

**Using the angular separation: **
Let's suppose our metric is the angular separation between the centers of objects (at same radius), and two points are said identical if their angular separation is smaller than $\beta_c$ . Both points have the same angular separation $\beta < \beta_c$, therefore we would say that A matches with B, and C matches with D.

We see in this simple example that depending on the metric chosen, the results are different.

Number of matched pairs
--------

Let A and B be two finite sets of points $\{ (r, \theta, \phi) \}$ with $r \in U(0, \infty), \, \theta \in U(0, \pi), \, \phi \in U(0, 2\pi)$, and $U(x, y)$ is the uniform distribution over [x, y].
Let denote $x$ the cross-match between points according to the distance to centers, and $o$ the cross-match between points according to the angular separation. We have

$$\begin{align}
\text{card}(AxB)(r) &\propto 1/r \\
\text{card}(AoB)(r) &\propto cste
\end{align}$$

Of course this scaling depends on how we sample the parameters and fill the space. Instead for example, we could have sampled $\cos(\theta)$ from $U(-1, 1)$, and the results would have changed. But all of that is to illustrate the fact that the scaling of the cardinal for the cross-match depends upon the metric chosen.

What has been implemented in spark3D so far?
--------

So far, two sets of methods have been implemented:
* `PixelCrossMatch`, and in particular `CrossMatchHealpixIndex` which performs the cross-match between two sets A and B according to the healpix index of the objects at a given radial position (space is partitioned in redshift shells). (angular position)
* `CenterCrossMatch`, and in particular `CrossMatchPosition` which performs the cross-match between two sets A and B according to the distance between object centers. (distance to centers)

In this notebook, we show the use of both.

```scala
// Package to read data from FITS file
%AddDeps com.github.astrolabsoftware spark-fits_2.11 0.6.0

// Smile provides visualisation tools
%AddDeps com.github.haifengl smile-plot 1.5.1
%AddDeps com.github.haifengl smile-math 1.5.1
%AddDeps com.github.haifengl smile-core 1.5.1
%AddDeps com.github.haifengl smile-scala_2.11 1.5.1

// Contains extensions to the Swing GUI toolkit
%AddDeps org.swinglabs swingx 1.6.1

// Add the spark3d JAR. To generate it, run `sbt ++2.11.8` package at the root of the package
%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.1.1.jar

// Add healpix JAR
%AddJar file:/Users/julien/Documents/workspace/myrepos/spark3D/lib/jhealpix.jar
```

    Marking com.github.astrolabsoftware:spark-fits_2.11:0.6.0 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/com/github/astrolabsoftware/spark-fits_2.11/0.6.0/spark-fits_2.11-0.6.0.jar
    Marking com.github.haifengl:smile-plot:1.5.1 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/com/github/haifengl/smile-plot/1.5.1/smile-plot-1.5.1.jar
    Marking com.github.haifengl:smile-math:1.5.1 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/com/github/haifengl/smile-math/1.5.1/smile-math-1.5.1.jar
    Marking com.github.haifengl:smile-core:1.5.1 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/com/github/haifengl/smile-core/1.5.1/smile-core-1.5.1.jar
    Marking com.github.haifengl:smile-scala_2.11:1.5.1 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/com/github/haifengl/smile-scala_2.11/1.5.1/smile-scala_2.11-1.5.1.jar
    Marking org.swinglabs:swingx:1.6.1 for download
    Preparing to fetch from:
    -> file:/var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/
    -> https://repo1.maven.org/maven2
    -> New file at /var/folders/my/lfvl285927q2hzk545f39sy40000gn/T/toree_add_deps7614799531666607290/https/repo1.maven.org/maven2/org/swinglabs/swingx/1.6.1/swingx-1.6.1.jar
    Starting download from file:/Users/julien/Documents/workspace/myrepos/spark3D/target/scala-2.11/spark3d_2.11-0.1.0.jar
    Finished download of spark3d_2.11-0.1.0.jar
    Using cached version of jhealpix.jar

# From raw data RDD to Point3D RDD

Load data from the test files provided in the spark3d repo.
Our raw data contains points with 3D coordinates (spherical: r, theta, phi) sampled from uniform distributions. Let's transform it into a Point3D RDD

```scala
import com.astrolabsoftware.spark3d.spatial3DRDD._
import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder().appName("Xmatch").getOrCreate()

val fnA = "../../src/test/resources/astro_obs.fits"
val fnB = "../../src/test/resources/astro_obs2.fits"
val hdu = 1
val columns = "Z_COSMO,RA,DEC"
val spherical = true

// Load the data
val pointRDDA = new Point3DRDDFromFITS(spark, fnA, hdu, columns, spherical)
val pointRDDB = new Point3DRDDFromFITS(spark, fnB, hdu, columns, spherical)
```

# Repartitioning of the space

By default, the pointRDD is partitioned randomly (i.e. Spark made partition regardless to the content of the file).
Let's repartition our data based on the distance to the center (Onion Space). In addition, to ease the cross-match between the two data sets, let's partition A and B the same way.

```scala
import com.astrolabsoftware.spark3d.utils.GridType
import com.astrolabsoftware.spark3d.spatialPartitioning.SpatialPartitioner

// As we are in local mode, and the file is very small, the RDD pointRDD has only 1 partition.
// For the sake of this example, let's increase the number of partition to 100.
val pointRDD_partA = pointRDDA.spatialPartitioning(GridType.LINEARONIONGRID, 100).cache()
// Get the partitioner of A
val partitionerA = pointRDD_partA.partitioner.get.asInstanceOf[SpatialPartitioner]
// Repartition B as A
val pointRDD_partB = pointRDDB.spatialPartitioning(partitionerA).cache()
```

# Cross matching using Healpix index

Cross match 2 RDD based on the healpix index of geometry center.
You have to choice to return:
* (1) Elements of (A, B) matching (returnType="AB")
* (2) Elements of A matching B (returnType="A")
* (3) Elements of B matching A (returnType="B")
* (4) Healpix pixel indices matching (returnType="healpix")

Which one you should choose? That depends on what you need:
(1) gives you all elements but is slow.
(2) & (3) give you all elements only in one side but is faster.
(4) gives you only healpix center but is even faster.

```scala
import com.astrolabsoftware.spark3d.spatialOperator.PixelCrossMatch

// Shell resolution
val nside = 512
// Keeping only elements from A with counterpart in B
val xMatchA = PixelCrossMatch.CrossMatchHealpixIndex(pointRDD_partA, pointRDD_partB, nside, "A")
// Keeping only elements from B with counterpart in A
val xMatchB = PixelCrossMatch.CrossMatchHealpixIndex(pointRDD_partA, pointRDD_partB, nside, "B")
// Keeping all elements with counterparts in both A and B
val xMatchAB = PixelCrossMatch.CrossMatchHealpixIndex(pointRDD_partA, pointRDD_partB, nside, "AB")

println("Keeping only elements from A with counterpart in B: ", xMatchA.count(), " points")
println("Keeping only elements from B with counterpart in A: ", xMatchB.count(), " points")
println("Keeping all elements with counterparts in both A and B: ", xMatchAB.count(), " points")
```

    (Keeping only elements from A with counterpart in B: ,1337, points)
    (Keeping only elements from B with counterpart in A: ,1278, points)
    (Keeping all elements with counterparts in both A and B: ,1382, points)

# Cross matching using distance between object centers

Cross match 2 RDD based on based on the object centers.
You have to choice to return:
* (1) Elements of (A, B) matching (returnType="AB")
* (2) Elements of A matching B (returnType="A")
* (3) Elements of B matching A (returnType="B")

Which one you should choose? That depends on what you need:
(1) gives you all elements but is slow.
(2) & (3) give you all elements only in one side but is faster.

```scala
import com.astrolabsoftware.spark3d.spatialOperator.CenterCrossMatch

// Distance threshold for the match
val epsilon = 0.004
// Keeping only elements from A with counterpart in B
val CxMatchA = CenterCrossMatch.CrossMatchCenter(pointRDD_partA, pointRDD_partB, epsilon, "A")
// Keeping only elements from B with counterpart in A
val CxMatchB = CenterCrossMatch.CrossMatchCenter(pointRDD_partA, pointRDD_partB, epsilon, "B")
// Keeping all elements with counterparts in both A and B
val CxMatchAB = CenterCrossMatch.CrossMatchCenter(pointRDD_partA, pointRDD_partB, epsilon, "AB")

println("Keeping only elements from A with counterpart in B: ", CxMatchA.count(), " points")
println("Keeping only elements from B with counterpart in A: ", CxMatchB.count(), " points")
println("Keeping all elements with counterparts in both A and B: ", CxMatchAB.count(), " points")
```

    (Keeping only elements from A with counterpart in B: ,695, points)
    (Keeping only elements from B with counterpart in A: ,497, points)
    (Keeping all elements with counterparts in both A and B: ,6121, points)

# Display the results

```scala
import smile.plot._
import java.awt.Color
import java.awt.{GridLayout, Dimension}

import javax.swing.JFrame
import javax.swing.JPanel

import com.astrolabsoftware.spark3d.utils.Utils.sphericalToCartesian
import org.apache.spark.rdd.RDD
import com.astrolabsoftware.spark3d.geometryObjects._

/** Define palette of colors */
def colors : Array[java.awt.Color] = {
    Array(
        Color.BLACK, Color.RED, Color.GREEN, Color.BLUE,
        Color.ORANGE, Color.YELLOW, Color.DARK_GRAY, Color.PINK,
        Color.MAGENTA, Color.CYAN)
}

/** Define markers */
def markers : Array[Char] = {
    val strings = Array(
        "o", "s", "x", "+", "@", "q",
        "-", "|", "O", "S", "#", "Q", "."
    )
    strings.map(x => x.toCharArray).flatten

}

/**
  * format the data for smile.
  * The data for ScatterPlot must be Array[Array[Double]] (=Array[point3d])
  * We add one more dimension which is the partition.
  *
  * @param rdd : (RDD[Point3D])
  *   RDD whose elements are Point3D instances.
  * @return (Array[Array[Array[Double]]]) data as partitions -> points -> point -> coordinate
  *
  */
def format_data_for_smile(rdd: RDD[Point3D]) : Array[Array[Array[Double]]] = {
    rdd.map(
        x=> sphericalToCartesian(x).center.getCoordinate.toArray)
    .glom.collect().toArray
}

/**
  * Plot 3D data sets.
  *
  * @param display : (String)
  *   Either show or save. If save, extension will be given in the outname.
  * @param rddArr : (Array(RDD[Point3D]))
  *   Array containing RDD for data sets X whose elements are instances of Point3D.
  * part : (Int)
  *   Partition index to plot.
  * @param outname : (String)
  *   If save mode, name (incl. extenstion) for the out file.
  * @param title : (String)
  *   Title of the window.
  *
  */
def MyScatterPlotCross(
    display: String, rddArr: Array[RDD[Point3D]],
    part: Int, outname: String, title: String) : Unit = {

    // Re-arange the data for plotting
    val dataInit = format_data_for_smile(rddArr(0))

    // Plot the results
    val window = ScatterPlot.plot(dataInit(part), markers(0), colors(0))

    for (pos <- 1 to rddArr.size - 1) {
      val dataOther = format_data_for_smile(rddArr(pos))
      window.points(dataOther(part), markers(pos), colors(pos))
    }

    display match {
      case "show" => {
        val partFrame = new JFrame(title)
        partFrame.setLocationRelativeTo(null)
        partFrame.getContentPane().add(window)
        partFrame.setVisible(true)
        partFrame.setSize(new Dimension(500, 500))
      }
      case "save" => {
        val partHeadless = new Headless(window);
        partHeadless.pack();
        partHeadless.setVisible(true);
        partHeadless.setSize(new Dimension(500, 500))
        window.save(new java.io.File(outname))
      }
      case _ => throw new AssertionError("""
        I do not understand the kind of display you want.
        Choose between "show" and "save".
        """)
    }
}

// Set to "show" or "save"
val display = "show"
val partition = 5

// Display the result for healpix
MyScatterPlotCross(display, Array(pointRDD_partA, pointRDD_partB,
                    xMatchA.asInstanceOf[RDD[Point3D]]), partition,
                    "crossmatchAxB.png", "Healpix Cross match (A, B, AxB)")
MyScatterPlotCross(display, Array(xMatchA.asInstanceOf[RDD[Point3D]],
                     xMatchB.asInstanceOf[RDD[Point3D]]), partition,
                     "crossmatchAxBOnly.png", "Healpix Cross match (AxB, BxA)")

// Display the result for object centers
MyScatterPlotCross(display, Array(pointRDD_partA, pointRDD_partB,
                    CxMatchA.asInstanceOf[RDD[Point3D]]), partition,
                    "CcrossmatchAxB.png", "Center Cross match (A, B, AxB)")
MyScatterPlotCross(display, Array(CxMatchA.asInstanceOf[RDD[Point3D]],
                     CxMatchB.asInstanceOf[RDD[Point3D]]), partition,
                     "CcrossmatchAxBOnly.png", "Center Cross match (AxB, BxA)")
```

Here is a plot for Partition #5

Cross match based on angular separation (A, B, AxB)    |Cross match based on angular separation (AxB, BxA)
:-------------------------:|:-------------------------:
|

Cross match based on center distance (A, B, BxA)    |Cross match based on center distance (AxB, BxA)
:-------------------------:|:-------------------------:
|

```scala
val display = "show"

/** Define line styles */
def lines : Array[Line.Style] = {
    Array(Line.Style.SOLID, Line.Style.SOLID,
          Line.Style.SOLID, Line.Style.SOLID,
          Line.Style.SOLID, Line.Style.SOLID)
}

/**
  * Plot the number of match per partition.
  *
  * @param display : (String)
  *   Either show or save. If save, extension will be given in the outname.
  * @param rddArr : (Array(RDD[Point3D]))
  *   Array containing RDD from the Xmatch.
  * part : (Int)
  *   Partition index to plot.
  * @param outname : (String)
  *   If save mode, name (incl. extenstion) for the out file.
  * @param title : (String)
  *   Title of the window.
  *
  */
def plotNumberOfMatch(
    display: String, rddArr: Array[RDD[Point3D]],
    part: Int, outname: String, title: String) : Unit = {

    // Re-arange the data for plotting
    val numberOfMatchInit = rddArr(0).mapPartitions(
      iter => Array(iter.size.toDouble).iterator, true).collect()

    // Plot the results
    val window = LinePlot.plot(numberOfMatchInit, lines(0), colors(0))

    for (pos <- 1 to rddArr.size - 1) {
      val numberOfMatchOther = rddArr(pos).mapPartitions(
        iter => Array(iter.size.toDouble).iterator, true).collect()
      window.line(numberOfMatchOther, lines(pos), colors(pos))
    }

    window.setAxisLabels(
        "partition index (sorted by increasing r)",
        "Number of pairs")

    display match {
      case "show" => {
        val partFrame = new JFrame(title)
        partFrame.setLocationRelativeTo(null)
        partFrame.getContentPane().add(window)
        partFrame.setVisible(true)
        partFrame.setSize(new Dimension(500, 500))
      }
      case "save" => {
        val partHeadless = new Headless(window);
        partHeadless.pack();
        partHeadless.setVisible(true);
        partHeadless.setSize(new Dimension(500, 500))
        window.save(new java.io.File(outname))
      }
      case _ => throw new AssertionError("""
        I do not understand the kind of display you want.
        Choose between "show" and "save".
        """)
    }
}

plotNumberOfMatch(display,
                  Array(xMatchA.asInstanceOf[RDD[Point3D]], CxMatchA.asInstanceOf[RDD[Point3D]]),
                  partition, "line.png", "Number of match per partition (black=angular, red=center distance)")
```

Number of match per partition (black=angular, red=center distance)

```scala

```


```python

```

```python
from sympy import *
from pylab import *
init_printing(use_unicode=True)
```

```python
a1 = Symbol("a1")
a2 = Symbol("a2")
a3 = Symbol("a3")
b1 = Symbol("b1")
b2 = Symbol("b2")
A = Matrix([[a1,0,0],[0,a2,0],[0,0,a3]])
B = Matrix([[b1,0,0],[0,0,b2],[0,b2,0]])
#A = Operator("A")
#B = Operator("B")

```

```python
A
```

```python
B
```

```python
A*B - B*A
```

```python
A.eigenvects()
```

```python
B.eigenvects()
```

```python
btwo = Bra(np.exp(a1))
btwo
```

```python
ktwo = Ket("2")
ktwo
```

```python
InnerProduct(btwo,ktwo)
```

```python
btwo*A*ktwo
```

```python
btwo*B*ktwo
```

```python
a = Symbol("a")
A = Matrix([[0,a],[a,0]])
A.eigenvects()
```

```python
A.norm()
```

```python
E1 = Symbol("E1")
E2 = Symbol("E2")
H = Matrix([[E1,0],[0,E2]])
H.eigenvects()
```

```python
am = matrix([[1],[1]])
```

```python
E0 = Symbol("E0")
E1 = Symbol("E1")
A = Symbol("A")
H = Matrix([[E0,0,A],[0,E1,0],[A,0,E0]])
H.eigenvects()
```

```python
def spinx(s):

    n = int(2.0*s+1)
    sx = matrix(zeros((n,n)))
    for a in range(0,n):
        for b in range(0,n):
            if (a==b+1):
                sx[a,b] = sx[a,b] + 0.5*sqrt((s+1)*(a+b+1)-(a+1)*(b+1))
            elif (a==b-1):
                sx[a,b] = sx[a,b] + 0.5*sqrt((s+1)*(a+b+1)-(a+1)*(b+1))

    return sx

def spiny(s):
    n = int(2.0*s+1)
    sy = matrix(zeros((n,n),dtype='complex'))

    for a in range(0,n):
        for b in range(0,n):
            if (a==b+1):
                sy[a,b] = sy[a,b] + 0.5j*sqrt((s+1)*(a+b+1)-(a+1)*(b+1))
            elif (a==b-1):
                sy[a,b] = sy[a,b] - 0.5j*sqrt((s+1)*(a+b+1)-(a+1)*(b+1))

    return sy

def spinz(s):

    n = int(2.0*s+1)
    sz = matrix(zeros((n,n)))

    for a in range(0,n):
        for b in range(0,n):

            if (a==b):
                sz[a,b] = (s+1-b-1)

    return sz
```

Sx = Sx1+Sx2+Sx3
Sy = ...
Sz = ...

```python
s = 0.5
I = eye(2)
Sx = spinx(s)
Sy = spiny(s)
Sz = spinz(s)
Sx = kron(Sx,kron(I,I)) + kron(I,kron(Sx,I)) + kron(I,kron(I,Sx))
Sy = kron(Sy,kron(I,I)) + kron(I,kron(Sy,I)) + kron(I,kron(I,Sy))
Sz = kron(Sz,kron(I,I)) + kron(I,kron(Sz,I)) + kron(I,kron(I,Sz))
S = Sx**2 + Sy**2 + Sz**2
Ssym = Matrix(S)
Ssym.eigenvals()
```

```python
print(eigh(S))
```

    (array([0.90945995, 0.90945995, 0.92998827, 0.92998827, 1.81247626,
           1.81247626, 1.81767956, 1.81767956, 2.27954832, 2.27954832,
           3.1811134 , 3.1811134 , 3.52165545, 3.52165545, 4.00233218,
           4.00233218, 5.29574662, 5.29574662]), matrix([[-2.84935689e-02+0.j,  1.44440448e-02+0.j, -1.95806287e-17+0.j,
             -1.55860475e-16+0.j, -1.53235477e-05+0.j, -2.29912893e-02+0.j,
             -4.30480258e-16+0.j,  1.82021408e-16+0.j, -5.31456195e-03+0.j,
             -2.61639022e-01+0.j, -4.17645920e-01+0.j, -5.19639391e-03+0.j,
              7.27492515e-01+0.j, -1.18784550e-01+0.j, -1.96978566e-17+0.j,
             -2.05332620e-20+0.j,  4.60605492e-01+0.j,  3.83752466e-16+0.j],
            [ 2.68755940e-01+0.j,  5.30171152e-01+0.j, -8.24115253e-15+0.j,
              8.38123015e-15+0.j,  6.49836820e-01+0.j, -4.33112096e-04+0.j,
              1.28368843e-13+0.j,  3.62709402e-14+0.j,  3.50216631e-01+0.j,
             -7.11380117e-03+0.j, -1.45545278e-03+0.j,  1.16978028e-01+0.j,
              4.29526955e-02+0.j,  2.63062533e-01+0.j,  2.33067269e-16+0.j,
              3.21388126e-16+0.j,  3.33066907e-16+0.j, -1.30252106e-01+0.j],
            [ 5.00075678e-02+0.j, -2.53499853e-02+0.j, -4.92503134e-03+0.j,
              4.74501091e-02+0.j, -8.84871661e-05+0.j, -1.32765211e-01+0.j,
              2.23073554e-01+0.j,  6.92011875e-02+0.j,  5.51225716e-03+0.j,
              2.71371674e-01+0.j,  4.66219358e-01+0.j,  5.80074968e-03+0.j,
              9.30382477e-02+0.j, -1.51912303e-02+0.j,  4.27148304e-01+0.j,
              5.10605467e-01+0.j,  4.23409984e-01+0.j, -2.49800181e-16+0.j],
            [-2.41579035e-01+0.j, -4.76559643e-01+0.j,  3.44002418e-01+0.j,
              3.57053486e-02+0.j,  2.17910750e-01+0.j, -1.45236125e-04+0.j,
             -1.74602510e-01+0.j,  5.62840089e-01+0.j,  2.98812494e-01+0.j,
             -6.06965085e-03+0.j,  7.61672859e-04+0.j, -6.12173687e-02+0.j,
              3.37823423e-02+0.j,  2.06898972e-01+0.j,  1.39570698e-01+0.j,
             -1.16758223e-01+0.j,  1.66533454e-16+0.j, -1.73271566e-01+0.j],
            [ 1.59097102e-01+0.j, -8.06499773e-02+0.j,  6.34826409e-02+0.j,
             -6.11622147e-01+0.j,  2.67097837e-04+0.j,  4.00750777e-01+0.j,
             -2.99259090e-01+0.j, -9.28352285e-02+0.j, -9.43190159e-03+0.j,
             -4.64338083e-01+0.j,  2.83229049e-01+0.j,  3.52396524e-03+0.j,
             -4.47687004e-02+0.j,  7.30980706e-03+0.j,  9.88082999e-02+0.j,
              1.18113680e-01+0.j,  9.79532481e-02+0.j, -1.38777878e-17+0.j],
            [ 8.06499773e-02+0.j,  1.59097102e-01+0.j, -6.11622147e-01+0.j,
             -6.34826409e-02+0.j, -4.00750777e-01+0.j,  2.67097837e-04+0.j,
             -9.28352285e-02+0.j,  2.99259090e-01+0.j,  4.64338083e-01+0.j,
             -9.43190159e-03+0.j,  3.52396524e-03+0.j, -2.83229049e-01+0.j,
              7.30980706e-03+0.j,  4.47687004e-02+0.j,  1.18113680e-01+0.j,
             -9.88082999e-02+0.j,  7.17047823e-17+0.j, -9.79532481e-02+0.j],
            [ 5.00075678e-02+0.j, -2.53499853e-02+0.j,  4.92503134e-03+0.j,
             -4.74501091e-02+0.j, -8.84871660e-05+0.j, -1.32765211e-01+0.j,
             -2.23073554e-01+0.j, -6.92011875e-02+0.j,  5.51225716e-03+0.j,
              2.71371674e-01+0.j,  4.66219358e-01+0.j,  5.80074968e-03+0.j,
              9.30382477e-02+0.j, -1.51912303e-02+0.j, -4.27148304e-01+0.j,
             -5.10605467e-01+0.j,  4.23409984e-01+0.j,  8.60422844e-16+0.j],
            [-2.41579035e-01+0.j, -4.76559643e-01+0.j, -3.44002418e-01+0.j,
             -3.57053486e-02+0.j,  2.17910750e-01+0.j, -1.45236125e-04+0.j,
              1.74602510e-01+0.j, -5.62840089e-01+0.j,  2.98812494e-01+0.j,
             -6.06965085e-03+0.j,  7.61672859e-04+0.j, -6.12173687e-02+0.j,
              3.37823423e-02+0.j,  2.06898972e-01+0.j, -1.39570698e-01+0.j,
              1.16758223e-01+0.j,  1.59932480e-16+0.j, -1.73271566e-01+0.j],
            [ 6.18591115e-02+0.j, -3.13578052e-02+0.j, -9.59050080e-17+0.j,
              2.52147256e-16+0.j,  2.36353807e-04+0.j,  3.54622759e-01+0.j,
              6.75044327e-15+0.j, -2.78160125e-15+0.j,  4.60509878e-03+0.j,
              2.26711731e-01+0.j, -4.57252385e-01+0.j, -5.68918167e-03+0.j,
             -5.18314396e-01+0.j,  8.46300695e-02+0.j,  2.70025729e-16+0.j,
              2.13305642e-16+0.j,  5.77138798e-01+0.j,  4.02455846e-16+0.j],
            [ 3.13578052e-02+0.j,  6.18591115e-02+0.j, -9.39817382e-16+0.j,
              8.96043271e-16+0.j, -3.54622759e-01+0.j,  2.36353807e-04+0.j,
             -6.99321496e-14+0.j, -1.98393920e-14+0.j, -2.26711731e-01+0.j,
              4.60509878e-03+0.j, -5.68918167e-03+0.j,  4.57252385e-01+0.j,
              8.46300695e-02+0.j,  5.18314396e-01+0.j, -6.35680782e-16+0.j,
              3.85180507e-16+0.j,  4.64375452e-16+0.j, -5.77138798e-01+0.j],
            [-4.76559643e-01+0.j,  2.41579035e-01+0.j, -3.57053486e-02+0.j,
              3.44002418e-01+0.j, -1.45236125e-04+0.j, -2.17910750e-01+0.j,
             -5.62840089e-01+0.j, -1.74602510e-01+0.j, -6.06965085e-03+0.j,
             -2.98812494e-01+0.j,  6.12173687e-02+0.j,  7.61672859e-04+0.j,
             -2.06898972e-01+0.j,  3.37823423e-02+0.j,  1.16758223e-01+0.j,
              1.39570698e-01+0.j,  1.73271566e-01+0.j, -2.08166817e-17+0.j],
            [ 2.53499853e-02+0.j,  5.00075678e-02+0.j,  4.74501091e-02+0.j,
              4.92503134e-03+0.j,  1.32765211e-01+0.j, -8.84871660e-05+0.j,
              6.92011875e-02+0.j, -2.23073554e-01+0.j, -2.71371674e-01+0.j,
              5.51225716e-03+0.j,  5.80074968e-03+0.j, -4.66219358e-01+0.j,
             -1.51912303e-02+0.j, -9.30382477e-02+0.j,  5.10605467e-01+0.j,
             -4.27148304e-01+0.j,  2.94925276e-16+0.j, -4.23409984e-01+0.j],
            [ 1.59097102e-01+0.j, -8.06499773e-02+0.j, -6.34826409e-02+0.j,
              6.11622147e-01+0.j,  2.67097837e-04+0.j,  4.00750777e-01+0.j,
              2.99259090e-01+0.j,  9.28352285e-02+0.j, -9.43190159e-03+0.j,
             -4.64338083e-01+0.j,  2.83229049e-01+0.j,  3.52396524e-03+0.j,
             -4.47687004e-02+0.j,  7.30980706e-03+0.j, -9.88082999e-02+0.j,
             -1.18113680e-01+0.j,  9.79532481e-02+0.j,  1.90819582e-16+0.j],
            [ 8.06499773e-02+0.j,  1.59097102e-01+0.j,  6.11622147e-01+0.j,
              6.34826409e-02+0.j, -4.00750777e-01+0.j,  2.67097837e-04+0.j,
              9.28352285e-02+0.j, -2.99259090e-01+0.j,  4.64338083e-01+0.j,
             -9.43190159e-03+0.j,  3.52396524e-03+0.j, -2.83229049e-01+0.j,
              7.30980706e-03+0.j,  4.47687004e-02+0.j, -1.18113680e-01+0.j,
              9.88082999e-02+0.j,  7.11101758e-17+0.j, -9.79532481e-02+0.j],
            [-4.76559643e-01+0.j,  2.41579035e-01+0.j,  3.57053486e-02+0.j,
             -3.44002418e-01+0.j, -1.45236125e-04+0.j, -2.17910750e-01+0.j,
              5.62840089e-01+0.j,  1.74602510e-01+0.j, -6.06965085e-03+0.j,
             -2.98812494e-01+0.j,  6.12173687e-02+0.j,  7.61672859e-04+0.j,
             -2.06898972e-01+0.j,  3.37823423e-02+0.j, -1.16758223e-01+0.j,
             -1.39570698e-01+0.j,  1.73271566e-01+0.j,  2.84494650e-16+0.j],
            [ 2.53499853e-02+0.j,  5.00075678e-02+0.j, -4.74501091e-02+0.j,
             -4.92503134e-03+0.j,  1.32765211e-01+0.j, -8.84871660e-05+0.j,
             -6.92011875e-02+0.j,  2.23073554e-01+0.j, -2.71371674e-01+0.j,
              5.51225716e-03+0.j,  5.80074968e-03+0.j, -4.66219358e-01+0.j,
             -1.51912303e-02+0.j, -9.30382477e-02+0.j, -5.10605467e-01+0.j,
              4.27148304e-01+0.j,  2.94325374e-16+0.j, -4.23409984e-01+0.j],
            [ 5.30171152e-01+0.j, -2.68755940e-01+0.j,  6.12140547e-16+0.j,
              2.80157841e-15+0.j, -4.33112096e-04+0.j, -6.49836820e-01+0.j,
             -1.25038868e-14+0.j,  4.71844785e-15+0.j, -7.11380117e-03+0.j,
             -3.50216631e-01+0.j, -1.16978028e-01+0.j, -1.45545278e-03+0.j,
             -2.63062533e-01+0.j,  4.29526955e-02+0.j,  1.38777878e-17+0.j,
              2.77555756e-17+0.j,  1.30252106e-01+0.j,  1.00613962e-16+0.j],
            [-1.44440448e-02+0.j, -2.84935689e-02+0.j,  6.03199447e-16+0.j,
             -3.78043352e-16+0.j,  2.29912893e-02+0.j, -1.53235477e-05+0.j,
              4.59292239e-15+0.j,  1.09646352e-15+0.j,  2.61639022e-01+0.j,
             -5.31456195e-03+0.j, -5.19639391e-03+0.j,  4.17645920e-01+0.j,
             -1.18784550e-01+0.j, -7.27492515e-01+0.j, -6.51029224e-16+0.j,
             -7.45584286e-17+0.j,  3.07932528e-16+0.j, -4.60605492e-01+0.j]]))

```python
a = Symbol("a")
sqrt2 = Symbol("sqrt(2)")
H = Matrix([[1,1],[1,-1]])
Heig = H.eigenvects()
Heig
```

```python
npH = matrix([[1,1,],[1,-1]])
npHeig = eigh(npH)
test = np.linalg.norm(npHeig[1])

#matrix(Heig[0][2])
```

```python
np.linalg.norm(matrix(Heig[0][2]))
```

```python

```


# Tutorial 3b: Probability distributions and their stories

```python
import numpy as np
import scipy.stats as st

import bokeh.io
import bokeh.layouts
import bokeh.plotting

bokeh.io.output_notebook()
```

    Loading BokehJS ...

Because this tutorial consists mainly of a list of probability distributions and their stories and properties, it is useful to have a little index of them up front.

* [Discrete distributions](#Discrete-distributions)
    - [Bernoulli distribution](#Bernoulli-distribution)
    - [Geometric distribution](#Geometric-distribution)
    - [Negative Binomial distribution](#Negative-Binomial-distribution)
    - [Binomial distribution](#Binomial-distribution)
    - [Hypergeometric distribution](#Hypergeometric-distribution)
* [Continuous distributions](#Continuous-distributions)
    - [Uniform distribution](#Uniform-distribution)
    - [Gaussian (a.k.a. Normal) distribution](#Gaussian--(a.k.a.-Normal)-distribution)
    - [Log-Normal distribution](#Log-Normal-distribution)
    - [Von Mises distribution](#Von-Mises-distribution)
    - [Chi-square distribution](#Chi-square-distribution)
    - [Student-t/Cauchy distribution](#Student-t/Cauchy-distribution)
    - [Exponential distribution](#Exponential-distribution)
    - [Gamma distribution](#Gamma-distribution)
    - [Weibull distribution](#Weibull-distribution)

We have talked about the three levels of building a model in the
biological sciences.
* **Cartoon model**: A cartoon or verbal phenomenological
  description of the system of interest.
* **Mathemtical model**: A mathematization of a cartoon model. This gives quantitative predictions on how a biological system should behave.
* **Statistical model**: A description of what we expect from an
  experiment, given our model mathematical.  In the Bayesian context, this is
  specification of the likelihood and prior.

We have talked briefly about specifying priors.  We endeavor to be uninformative when we do not know much about model parameters or about the hypothesis in question. We will be a bit sloppy in our prior formulation knowing that if we are sufficiently uninformative and we have enough data, the prior has little impact on the posterior.

To specify a likelihood, we need to be careful, as this is very much
at the heart of our model.  Specifying the likelihood amounts to
choosing a probability distribution that describes how the data will
look under your model.  In many cases, you need to derive the
likelihood (or even numerically compute it when it cannot be written
in closed form).  In many practical cases, though, the choice of
likelihood is among standard probability distributions.  These
distributions all have **stories** associated with them.  Importantly, *if your data
and model match the story of a distribution, you know that this is the
distribution to choose for your likelihood.*

## Review of probability distributions

Before we begin talking about distributions, let's remind ourselves
what probability distributions are.  We cut some corners in our
definitions here, but these definitions are functional for most of our
data analysis purposes.

A **probability mass function** (PMF), $P(x)$, describes the
probability of a discrete variable obtaining value $x$.  The variable
$x$ takes on discrete values, so the **normalization condition**
is

$\begin{align}
\sum_x P(x) = 1.
\end{align}$

A **probability density function** (PDF), which we shall
also call $P(x)$, is defined such that the probability that a
continuous variable $x$ is $a \le x \le b$ is

$\begin{align}
\int_a^b \mathrm{d}x\,P(x).
\end{align}$

A **cumulative distribution function** (CDF), denoted $F(x)$ is defined such that $F(x)$ is the probability that a variable $X$ is less than or equal to $x$.

### Moments

Probability distributions **moments**.  The way they are defined can
vary, but we will define the $n$th moment computed from a PMF as

$\begin{align}
\left\langle x^n \right\rangle = \sum_i x_i^n P(x_i),
\end{align}$

and computed from a PDF as

$\begin{align}
\left\langle x^n \right\rangle = \int \mathrm{d}x\, x^n P(x).
\end{align}$

These moments are often used to compute summary statistics.

$\begin{align}
\text{mean} &= \langle x \rangle \\[1em]
\text{variance} &= \left\langle x^2 \right\rangle - \langle x \rangle^2
= \left\langle(x - \langle x \rangle)^2 \right\rangle \\[1em]
\text{skew} &= \frac{\left\langle(x - \langle x \rangle)^3 \right\rangle}
{\left\langle(x - \langle x \rangle)^2 \right\rangle^{\frac{3}{2}}} \\[1em]
\text{Pearson kurtosis} &= \frac{\left\langle(x - \langle x \rangle)^4 \right\rangle}
{\left\langle(x - \langle x \rangle)^2 \right\rangle^2} \\[1em]
\text{Fisher kurtosis} &= \frac{\left\langle(x - \langle x \rangle)^4 \right\rangle}
{\left\langle(x - \langle x \rangle)^2 \right\rangle^2} - 3.
\end{align}$

We will present PMFs, PDFs, and CDFs for distributions below.  We only show
the univariate forms; the multivariate versions are easily derived or
looked up.

### Sampling

Given that we know a probability distribution, we can take
*samples* out of it.  This means that we can randomly draw
numbers and the probability that we draw a certain number $x$ is
proportional to the PMF or PDF, $P(x)$.  As we will soon discover in the course, sampling
out of a distribution is often easier than computing the distribution
over a range of values because many of those values are zero.

## Distributions and their stories

In what follows, I will present some commonly used probability distributions and their associated stories. For each distribution, I will give the name of the distribution, its story, its parameters, a biological example that follows this distribution, expressions for the PDF or PMF and CDF, as well as plots of these. Along the way, we'll define other terms, such as **Bernoulli trial** and **Poisson process**. I will use the `scipy.stats` module (imported as `st`) to compute the PDF/PMF and CDF. The API for this module will be apparent as I use it.

Before we begin, I'll write some functions to generate the distributions for plotting. For this tutorial, I'll use Bokeh.

```python
def plot_pmf(p, x, dist, params, **kwargs):
    """
    Generate plot of PMF at specified values of x.
    """
    y = dist.pmf(x, *params, **kwargs)
    p.circle(x, y, size=7, color='dodgerblue')
    p.segment(x0=x, x1=x, y0=0, y1=y, line_width=3, color='dodgerblue')
    return p
```

```python

def plot_pdf(p, x, dist, params, **kwargs):
    """
    Generate plot of PDF at specified values of x.
    """
    y = dist.pdf(x, *params, **kwargs)
    p.line(x, y, line_width=3, color='dodgerblue')
    return p
```

```python
def plot_discrete_cdf(p, x, dist, params, show_segments=False, **kwargs):
    """
    Plot CDF of discrete distribution.
    """
    y = dist.cdf(x, *params, **kwargs)
    if show_segments:
        p.circle(x[:-1], y[:-1], size=7, color='dodgerblue')
        p.segment(x0=x[:-1], x1=x[1:], y0=y[:-1], y1=y[:-1], line_width=3,
                  color='dodgerblue')
    else:
        p.circle(x, y, size=7, color='dodgerblue')
    return p
```

```python
def plot_continuous_cdf(p, x, dist, params, **kwargs):
    """
    Plot CDF of continuous distribution.
    """
    y = dist.cdf(x, *params, **kwargs)
    p.line(x, y, line_width=3, color='dodgerblue')
    return p
```

```python
def plot_dists(x, dist, params, param_names, dist_name, **kwargs):
    """
    Plot PDF/PMF and CDF next to each other.
    """
    # Tools for plots
    tools='pan,wheel_zoom,reset'

    # Title for plots
    t1 = dist_name + ', ' \
            + ''.join([param_names[i] + ' = ' + str(params[i]) + ', '
                                      for i in range(len(params) - 1)])
    t1 += param_names[-1] + ' = ' + str(params[-1])

    # Last half of y-axis label
    ylabel = ''.join(pname + ', ' for pname in param_names[:-1])
    ylabel += param_names[-1] + ')'

    # Set up plots
    p1 = bokeh.plotting.figure(width=325, height=250, tools=tools, title=t1)
    p2 = bokeh.plotting.figure(width=325, height=250, tools=tools,
                               y_range=[-0.05, 1.05])

    # Make plots
    if hasattr(dist, 'pmf'):
        p1 = plot_pmf(p1, x, dist, params, **kwargs)
        p1.xaxis.axis_label = 'k'
        p1.yaxis.axis_label = 'P(k; ' + ylabel
        p2 = plot_discrete_cdf(p2, x, dist, params, **kwargs)
        p2.xaxis.axis_label = 'k'
        p2.yaxis.axis_label = 'F(k; ' + ylabel
    else:
        p1 = plot_pdf(p1, x, dist, params, **kwargs)
        p1.xaxis.axis_label = 'x'
        p1.yaxis.axis_label = 'P(x; ' + ylabel
        p2.yaxis.axis_label = 'F(x; ' + ylabel
        p2 = plot_continuous_cdf(p2, x, dist, params, **kwargs)
        p2.xaxis.axis_label = 'x'
        p2.yaxis.axis_label = 'F(k; ' + ylabel

    # Link the x-axes
    p1.x_range = p2.x_range

    return bokeh.layouts.row([p1, p2])
```

With these in place, we now describe some useful named distributions. For each, I will give the mathematical form of the PDF or PMF, but will omit the the CDF because they are often expressed as special functions, but as regularized incomplete beta functions or error functions. Remember that in general, for a discrete distribution

$\begin{align}
F(k) = \sum_{k'=k_\mathrm{min}}^k P(k'),
\end{align}$

and for a continuous distribution,

$\begin{align}
F(x) = \int_{-\infty}^x \mathrm{d}x'\,P(x').
\end{align}$

I do plot the CDF for each distribution.

## Discrete distributions

### Bernoulli distribution

**Story.** A single trial with either a success (`k = True`) or
failure (`k=False`) is performed. Such a trial is called a *Bernoulli trial*.  The Bernoulli distribution defines the probability of getting each outcome.

**Parameter.** The Bernoulli distribution is parametrized by a
single value, $p$, the probability that the trial is successful.
These trials are called *Bernoulli trials*.

**Example.** Check to see if a given bacterium is competent, given that it has probability $p$ of being competent.

**Probability mass function.**

$\begin{align}
P(k;p) = \left\{ \begin{array}{ccc}
1-p & & k = 0 \\[0.5em]
p & & k = 1 \\[0.5em]
0 & & \text{otherwise.}
\end{array}
\right.
\end{align}$

```python
k = np.array([0, 1])
p = plot_dists(k, st.bernoulli, (0.4,), ('p',), 'Bernoulli')
bokeh.io.show(p)
```

### Geometric distribution

**Story.** We perform a series of Bernoulli trials until we
get a success.  We have $k$ failures before the success.

**Parameter.** The Geometric distribution is parametrized by a
single value, $p$, the probability that the Bernoulli trial is
successful.

**Example.** Consider actin polymerization.  At each time
step, an actin monomer may add to the filament ("failure"), or an
actin monomer may fall off (""success"") with (usually very low)
probability $p$.  The length of actin filaments are Geometrically
distributed.

**Probability mass function.**

\begin{align}
P(k;p) = (1-p)^k p.
\end{align}

**Notes.** The Geometric distribution is only defined for non-negative
integer $k$.

```python
k = np.arange(11)
p = plot_dists(k, st.geom, (0.4,), ('p',), 'Geometric', **{'loc': -1})
bokeh.io.show(p)
```

### Negative Binomial distribution

**Story.** We perform a series of Bernoulli trials until we
get $r$ successes.  The number of failures, $k$, before we get $r$
successes is Negative Binomially distributed.

**Parameters.** There are two parameters: the probability $p$
of success for each Bernoulli trial, and the desired number of
successes, $r$.

**Example.** Bursty gene expression can give mRNA count
distributions that are Negative Binomially distributed.  Here,
"success" is that a burst in gene expression stops.  So, the
parameter $p$ is related to the length of a burst in expression (lower
$p$ means a longer burst).  The parameter $r$ is related to the
frequency of the bursts.  If multiple bursts are possible within the
lifetime of mRNA, then $r > 1$.  Then, the number of "failures" is
the number of mRNA transcripts that are made in the characteristic
lifetime of mRNA.

**Probability mass function.**

$\begin{align}
P(k;r,p) = \begin{pmatrix}
k+r-1 \\
r-1
\end{pmatrix}
p^r (1-p)^k.
\end{align}$

Here, we use a combinatorial notation;

$\begin{align}
  \begin{pmatrix}
k+r-1 \\
r-1
\end{pmatrix} = \frac{(k+r-1)!}{(r-1)!\,k!}.
\end{align}$

**Notes.** If $r = 1$, this distribution reduces to the Geometric
distribution.

```python
k = np.arange(21)
p = plot_dists(k, st.nbinom, (3, 0.4), ('r', 'p',), 'Negative Binomial')
bokeh.io.show(p)
```

### Binomial distribution

**Story.** We perform $n$ Bernoulli trials with probability $p$ of success.  The number of
successes, $k$, is binomially distributed.

**Parameters.** There are two parameters: the probability $p$
of success for each Bernoulli trial, and the number of trials, $n$.

**Example.** Distribution of plasmids between daughter cells
in cell division.  Each of the $n$ plasmids as a chance $p$ of being
in daughter cell 1 ("success").  The number of plasmids, $k$, in
daughter cell 1 is binomially distributed.

**Probability mass function.**

$\begin{align}
P(k;n,p) = \begin{pmatrix}
n \\
k
\end{pmatrix}
p^k (1-p)^{n-k}.
\end{align}$

```python
k = np.arange(21)
p = plot_dists(k, st.binom, (20, 0.4), ('n', 'p',), 'Binomial')
bokeh.io.show(p)
```

### Poisson distribution

**Story.**  Rare events occur with a rate $\lambda$ per unit
time.  There is no "memory" of previous events; i.e., that rate is
independent of time. A process that generates such events is called a *Poisson process*. The occurrence of a rare event in this context is referred to as an *arrival*. The number $k$ of arrivals in unit time is Poisson distributed.

**Parameter.** The single parameter is the rate $\lambda$ of
the rare events occurring.

**Example.** The number of mutations in a strand of DNA per
unit length (since mutations are rare) are Poisson distributed.

**Probability mass function.**
$\begin{align}
P(k;\lambda) = \frac{\lambda^k}{k!}\,\mathrm{e}^{-\lambda}.
\end{align}$

**Notes.**
The Poisson distribution is a limit of the binomial distribution in
which the number of trials goes to infinity, but the expected number
of successes, $np$, stays fixed.  Thus,

$\begin{align}
P_\mathrm{Poisson}(k;\lambda) \approx P_\mathrm{Binomial}(k;n, p),
\end{align}$

with $\lambda = np$.  Considering the biological example of mutations,
this is binomially distributed: There are $n$ bases, each with a
probability $p$ of mutation, so the number of mutations, $k$ is
binomially distributed.  Since $p$ is small, it is approximately
Poisson distributed.

```python
k = np.arange(11)
p = plot_dists(k, st.poisson, (1,), ('λ',), 'Poisson')
bokeh.io.show(p)
```

### Hypergeometric distribution

**Story.** Consider an urn with $w$ white balls and $b$ black
balls.  Draw $n$ balls from this urn without replacement.  The number
white balls drawn, $k$, is Hypergeometrically distributed.

**Parameters.** There are three parameters: the number of
draws $n$, the number of white balls $w$, and the number of black
balls $b$.

**Example.** There are $N$ finches on an island, and $n_t$ of
them are tagged.  You capture $n$ finches.  The number of tagged
finches $k$ is Hypergeometrically distributed, $P(k;n_t, N-n_t, n)$,
as defined below.

**Probability mass function.**

$\begin{align}
P(k;w, b, n) = \frac{\begin{pmatrix}w\\k\end{pmatrix}\begin{pmatrix}b\\n-k\end{pmatrix}}
{\begin{pmatrix}w+b\\n\end{pmatrix}}.
\end{align}$

Alternatively, if we define $N = w + b$, we could write

$\begin{align}
P(k;N, w, n) = \frac{\begin{pmatrix}w\\k\end{pmatrix}\begin{pmatrix}N-w\\n-k\end{pmatrix}}
{\begin{pmatrix}N\\n\end{pmatrix}}.
\end{align}$

**Notes.** This distribution is analogous to the Binomial
distribution, except that the Binomial distribution describes draws
from an urn *with* replacement.  In the analogy, $p = w/(w+b)$.

```python
k = np.arange(11)
p = plot_dists(k, st.hypergeom, (25, 5, 10), ('N', 'w', 'n'), 'Hypergeometric')
bokeh.io.show(p)
```

## Continuous distributions

### Uniform distribution

**Story.** Any outcome in a given range has equal probability.

**Parameters.** The Uniform distribution is not defined on an
infinite or semi-infinite domain, so bounds, $x_\mathrm{min}$ and
$x_\mathrm{max}$ are necessary parameters.

**Example.** Anything in which all possibilities are equally
likely.

**Probability density function.**

\begin{align}
P(x;x_\mathrm{min}, x_\mathrm{max}) = \left\{ \begin{array}{ccc}
\frac{1}{x_\mathrm{max} - x_\mathrm{min}} & & x_\mathrm{min} \le x \le x_\mathrm{max} \\[0.5em]
0 & & \text{otherwise.}
\end{array}
\right.
\end{align}

```python
x = np.array([0, 1])
p = plot_dists(x, st.uniform, (0, 1), ('xmin', 'xmax'), 'Uniform')
bokeh.io.show(p)
```

### Gaussian  (a.k.a. Normal) distribution

**Story.** Any quantity that emerges from a large number of
subprocesses tends to be Gaussian distributed provided none of the
subprocesses is very broadly distributed.

**Parameters.** The Gaussian distribution has two parameters,
the mean $\mu$, which determines the location of its peak, and the
standard deviation $\sigma$, which is strictly positive (the
$\sigma\to 0$ limit defines a Dirac delta function) and determines the
width of the peak.

**Example.** We measure the length of many *C. elegans*
eggs.  The lengths are Gaussian distributed.

**Probability density function.**

\begin{align}
P(x;\mu, \sigma) = \frac{1}{\sqrt{2\pi \sigma^2}}\,\mathrm{e}^{-(x-\mu)^2/2\sigma^2}.
\end{align}

**Notes.** This is a limiting distribution in the sense of the
central limit theorem, but also in that many distributions have a
Gaussian distribution as a limit.  This is seen by formally taking
limits of, e.g., the Gamma, Student-t, Binomial distributions, which
allows direct comparison of parameters.

```python
x = np.linspace(0, 20, 200)
p = plot_dists(x, st.norm, (10, 2), ('µ', 'σ'), 'Gaussian')
bokeh.io.show(p)
```

### Log-Normal distribution

**Story.** If $\ln x$ is Gaussian distributed, $x$ is
Log-Normally distributed.

**Parameters.** As for a Gaussian, there are two parameters,
the mean logarithm, $\ln \mu$, and the variance $\sigma^2$.

**Example.** A measure of fold change in gene expression can
be Log-Normally distributed.

**Probability density function.**

\begin{align}
P(x;\mu, \sigma) = \frac{1}{x\sqrt{2\pi \sigma^2}}\,\mathrm{e}^{-(\ln x - \ln \mu)^2/2\sigma^2}.
\end{align}

```python
x = np.linspace(0, 10, 200)
p = plot_dists(x, st.lognorm, (0.5, 1), ('σ', 'µ'), 'Log-normal')
bokeh.io.show(p)
```

### Von Mises distribution

**Story.** Gaussian, except on a periodic domain.

**Parameters.** As for a Gaussian, with $\mu$ being the
location of the peak, and $\beta$ being analogous to the variance.

**Example.** Repeated measurements on a periodic domain, e.g.,
the location of an ingression along the azimuthal angle of a
developing embryo.

**Probability density function.**

\begin{align}
P(\theta;\mu, \beta) = \frac{1}{2\pi I_0(\beta)}\,\mathrm{e}^{\beta \cos(\theta - \mu)},
\end{align}

where $I_0(\beta)$ is a modified Bessel function of the first kind.

```python
x = np.linspace(0, 2*np.pi, 200)
p = plot_dists(x, st.vonmises, (0.5, 3.14), ('β', 'µ'), 'Von Mises')
bokeh.io.show(p)
```

### Chi-square distribution

**Story.** If $X_1$,
$X_2$, $\ldots$, $X_n$ are Gaussian distributed,
$X_1^2 + X_2^2 + \cdots + X_n^2$ is $\chi^2$-distributed.

**Parameters.** There is only one parameter, the degrees of
freedom $n$.

**Probability density function.**
\begin{align}
  P(x;n) \equiv \chi^2_n(x;n) = \frac{1}{2^{n/2}\Gamma\left(\frac{n}{2}\right)}\,
x^{\frac{n}{2}-1}\,\mathrm{e}^{-x/2}.
\end{align}

```python
x = np.linspace(0, 30, 200)
p = plot_dists(x, st.chi2, (10,), ('n'), 'Chi-square')
bokeh.io.show(p)
```

### Student-t/Cauchy distribution

**Story.**  We get this distribution whenever we marginalize
an unknown $\sigma$ out of a Gaussian distribution with a Jeffreys
prior for $\sigma$.

**Parameters.** The Student-t distribution is peaked, and its
peak is located at $m$.  The peak's width is dictated by parameter
$s$.  Finally, we define the "degrees of freedom" as $n$.

**Example.** The story says it all!

**Probability density function.**

$ \begin{align}
P(x;m, s, n) = \frac{\Gamma\left(\frac{n}{2}\right)}{\Gamma\left(\frac{n+1}{2}\right)}\,
\frac{\sqrt{\pi n s^2}}{\left(1 + \frac{(x-m)^2}{ns^2}\right)^{\frac{n+1}{2}}}.
\end{align}$

**Notes.** For $n\to \infty$, we get a Gaussian distribution.
When $n = 1$, we get the **Cauchy distribution**,

$\begin{align}
P(x;m, s) = \left[\pi s \left(\frac{x-m}{s}\right)^2\right]^{-1}.
\end{align}$

```python
x = np.linspace(0, 20, 200)
p = plot_dists(x, st.t, (5, 10, 2), ('n', 'm', 's'), 'Student-t')
bokeh.io.show(p)
```

### Exponential distribution

**Story.** This is the waiting time for an arrival from a
Poisson process.  I.e., the inter-arrival time of a Poisson process is
Exponentially distributed.

**Parameter.** The single parameter is the average arrival
rate, $r$.

**Example.** The time between conformational switches in a
protein is Exponentially distributed (under simple mass action
kinetics).

**Probability density function.**
\begin{align}
P(x;r) = r \mathrm{e}^{-rx}.
\end{align}

Alternatively, we could parametrize it as

\begin{align}
P(x;\lambda) = \frac{1}{\lambda}\, \mathrm{e}^{-x/\lambda}.
\end{align}

**Notes.** The Exponential distribution is the continuous
analog of the Geometric distribution.  The "rate" in the Exponential
distribution is analogous to the probability of success of the
Bernoulli trial. Note also that because they are uncorrelated, the
amount of time between any two arrivals is independent of all other
inter-arrival times.

```python
x = np.linspace(0, 10, 200)
p = plot_dists(x, st.expon, (0,), ('λ',), 'Exponential')
p.children[0].title.text = 'Exponential, λ = 1'
bokeh.io.show(p)
```

### Gamma distribution

**Story.** The amount of time we have to wait for $a$ arrivals
of a Poisson process.  More concretely, if we have events, $X_1$,
$X_2$, $\ldots$, $X_a$ that are exponentially distributed,
$X_1 + X_2 + \cdots + X_a$ is gamma distributed.

**Parameters.** The number of arrivals, $a$, and the rate of
arrivals, $r$.

**Example.** Any multistep process where each step happens at
the same rate.  This is common in molecular rearrangements, and we
will use it in class to describe the nature of processes triggering
microtubule catastrophe.

**Probability density function.**

$\begin{align}
P(x;a, r) = \frac{1}{\Gamma(a)}\,\frac{(rx)^a}{x}\,\mathrm{e}^{-rx},
\end{align}$

where $\Gamma(a)$ is the gamma function.

**Notes.** The Gamma distribution is the continuous
analog of the Negative Binomial distribution.

```python
x = np.linspace(0, 10, 200)
p = plot_dists(x, st.gamma, (3, 0), ('a', 'r'), 'Gamma')
p.children[0].title.text = 'Gamma, a = 3, r = 1'
bokeh.io.show(p)
```

### Weibull distribution

**Story.** Distribution of $x = y^\beta$ if $y$ is
exponentially distributed.  For $\beta > 1$, the longer we have
waited, the more likely the event is to come, and vice versa for
$\beta < 1$.

**Parameters.** There are two parameters, both strictly
positive: the shape parameter $\beta$, which dictates the shape of the
curve, and the scale parameter $\lambda$, which dictates the rate of
arrivals of the event.

**Example.** This is a model for aging.  The longer an
organism lives, the more likely it is to die.

**Probability density function.**

\begin{align}
P(x;\lambda, \beta) = \frac{\beta}{\lambda}\left(\frac{x}{\lambda}\right)^{\beta - 1}\,
\mathrm{e}^{-(x/\lambda)^\beta}.
\end{align}

```python
x = np.linspace(0, 3, 200)
p = plot_dists(x, st.frechet_r, (3, 0), ('β', 'λ'), 'Weibull')
p.children[0].title.text = 'Weibull, β = 3, λ = 1'
bokeh.io.show(p)
```

    C:\Users\reddy\AppData\Local\Programs\Python\Python36\lib\site-packages\ipykernel_launcher.py:5: DeprecationWarning: `frechet_r` is deprecated!
    The distribution `frechet_r` is a synonym for `weibull_min`; this historical
    usage is deprecated because of possible confusion with the (quite different)
    Frechet distribution.  To preserve the existing behavior of the program, use
    `scipy.stats.weibull_min`.  For the Frechet distribution (i.e. the Type II
    extreme value distribution), use `scipy.stats.invweibull`.
      """


# Integración de funcións de varias variables en 2 e 3 dimensións

O cálculo de integrais indefinidas (primitivas) de funcións dunha única variable pódese entender como desandar o camiño feito no cálculo dunha derivada. O mesmo se pode entender de maneira informal ao cálcular integrais indefinidas de funcións de varias variables. De feito, grazas á regra de Barrow aplicada a cada unha das integrais, é doado calcular o valor das integrais iteradas en varias variables. Este mesmo proceso é o que seguiremos usando o módulo **Sympy**. Ademais, destes cálculos, tamén revisaremos a interpretación xeométrica das integrais de varias variables, mediante o concepto de área dunha superficie e o volume dun sólido.

Nesta práctica volveremos a usar o módulo **Sympy** para calcular integrais e para representar as funcións empregaremos os módulos **Matplotlib** e **Numpy**.

### Obxectivos:

- Cálculo de primitivas e integrais definidas de funcións dunha variable
- Cálculo de integrais de función de varias variables: integrais iteradas
- Comprobación do Teorema de Fubini
- Cálculo de áreas de superficies e volumes de sólidos

Nesta práctica usaremos tanto o módulo **Sympy**, como tamén **Numpy** e **Matplotlib**. Así que debemos importalos para o resto do guión de prácticas:

```python
import sympy as sp
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

## Cálculo de primitivas e integrais definidas de funcións dunha variable

O cálculo de integrais é moi sinxelo usando **Sympy**: basta usar o comando ``sp.integrate``. Vexamos isto cun exemplo no que se pretende calcular unha primitiva da función $f(x)=x^2+3$:

```python
x = sp.symbols('x', real=True) # define as variables simbólicas x
F = sp.integrate(x**2+3,x)
display(F)
```

$\displaystyle \frac{x^{3}}{3} + 3 x$

Debemos notar aquí que **Sympy** só calcula unha das infinitas primitivas que posúe a función $f(x)$. Para obtelas todas, debemos lembrarnos de engadir a suma da contante arbitraria $C$. Se pola contra precisamos calcular unha integral definida, por exemplo, $\int_{0}^{1} f(x)dx$, empregaríamos o mesmo comando ``sp.integrate``, pero incluíndo o dominio de integración:

```python
I = sp.integrate(x**2+3,(x,0,1))
display(I)
```

$\displaystyle \frac{10}{3}$

## Cálculo de integrais de funcións de varias variables

Como xa se comentou anteriormente, podemos realizar o cálculo de integrais indefinidas usando varias variables. Este cálculo se pode entender como o proceso de *desandar* o camiño feito na derivación. Por exemplo, se consideramos $f(x,y)=xy^2$ entón a súa integral con respecto ás variables $x$ e $y$ resultan:

```python
x, y = sp.symbols('x y', real=True) # define as variables simbólicas x, y
f = sp.Lambda((x,y), x*y**2)
# Integral con respecto a x
Fx = sp.integrate(f(x,y),x)
display(Fx)
display(sp.diff(Fx,x))
# Integral con respecto a y
Fy = sp.integrate(f(x,y),y)
display(Fy)
display(sp.diff(Fy,y))
```

$\displaystyle \frac{x^{2} y^{2}}{2}$

$\displaystyle x y^{2}$

$\displaystyle \frac{x y^{3}}{3}$

$\displaystyle x y^{2}$

Como se pode comprobar, ao calcular as derivadas parciais destas integrais indefinidas recupérase a expresión orixinal do integrando.

Do mesmo xeito, **Sympy** tamén é capaz de calcular integrais iteradas de funcións de varias variables. Por exemplo, para calcular
$$
\int_{2}^{3}\int_{1}^{x-1} (x^2+y^2)dydx,
$$
empregaríase:

```python
f = sp.Lambda((x,y), x ** 2 + y ** 2)
res = sp.integrate(f(x,y), (y, 1, x-1), (x, 2, 3))
display(res)
```

$\displaystyle \frac{9}{2}$

A partir desta integral, podemos facer unha comprobación inmediata do teorema de Fubini tendo en conta que o recinto de integración pódese describir como
$$
\mathcal{R}=\{(x,y)\in\mathbb{R}^2: 1\le y\le x-1,\ 2\le x\le 3\}=
\{(x,y)\in\mathbb{R}^2: y+1\le x\le 3,\ 1\le y\le 2\},
$$
e polo tanto o Teorema de Fubini garante que
$$
\int_{2}^{3}\int_{1}^{x-1} (x^2+y^2)dydx = \int_{1}^{2}\int_{y+1}^{3} (x^2+y^2)dxdy.
$$

```python
res = sp.integrate(f(x,y), (x, y+1, 3), (y, 1, 2))
display(res)
```

$\displaystyle \frac{9}{2}$

Do mesmo xeito, se poderían calcular integrais iteradas de tres ou máis variables, e o teorema de Fubini garante que a orde das variables na que se faga a integración non vai alterar o resultado.

### **Exercicio 10.1**
Calcula unha función de dúas variables $f(x,y)$ tal que satisfai:
$$
\frac{\partial f}{\partial x}(x,y)=x^3y^2\quad\text{e}\quad f(0,y)=y-1.
$$

```python
# O TEU CÓDIGO AQUÍ
```

### **Exercicio 10.2**
Calcula a seguinte integral dunha función de tres variables:
$$
\int_{2}^{3}\int_{1}^{x-1}\int_{0}^{y^2-1} (x^2+zy^2)dzdydx.
$$

```python
# O TEU CÓDIGO AQUÍ
```

## Cálculo de áreas de superficies e volumes de sólidos

Para ilustrar o cálculo de áreas de superficies e volumes de sólidos imos a calcular o área dun triángulo e o volume limitado por un paraboloide eliptico sobre un triángulo, empregando integrais de varias variables. En primeiro lugar, para o cálculo do área do triángulo rectángulo de vértices $(0,0)$, $(0,1)$ e $(1,1)$, poderíamos usar unha integral dunha función dunha variable, xa que o triángulo é o recinto encerrado pola gráfica de $f(x)=x$ e as rectas $x=0$ e $x=1$, ou ben integrando a constante $1$ no recinto bidimensional:
$$
\mathcal{R}=\{(x,y)\in\mathbb{R}^2: 0\le y\le x,\ 0\le x\le 1\}
$$

```python
x, y = sp.symbols('x y', real=True)
# cálculo da integral dunha única variable
A = sp.integrate(x, (x,0,1))
display(A)
# cálculo da integral de dúas variables no recinto R
B = sp.integrate(1, (y,0,x), (x,0,1))
display(B)
```

$\displaystyle \frac{1}{2}$

$\displaystyle \frac{1}{2}$

Para o caso do cálculo do volume limitado polo paraboloide elíptico $z=x^2+y^2$ sobre o recinto triangular $\mathcal{R}$ descrito anteriormente, podemos seguir dúas estratexias:

```python
x, y, z = sp.symbols('x y z', real=True)
# cálculo da integral de dúas variables
A = sp.integrate(x**2+y**2, (y,0,x), (x,0,1))
display(A)
# cálculo da integral de tres variables
B = sp.integrate(1, (z,0,x**2+y**2), (y,0,x), (x,0,1))
display(B)
```

$\displaystyle \frac{1}{3}$

$\displaystyle \frac{1}{3}$

### **Exercicio 10.3**
A fórmula de Heron permite calcular o área dun triángulo de vértices arbitrarios, que se pode implementar do seguinte xeito:

```python
def heron(p1,p2,p3):
    a=sp.sqrt((p1[0]-p2[0])**2+(p1[1]-p2[1])**2)
    b=sp.sqrt((p1[0]-p3[0])**2+(p1[1]-p3[1])**2)
    c=sp.sqrt((p3[0]-p2[0])**2+(p3[1]-p2[1])**2)
    s=(a+b+c)/sp.S('2')
    return sp.sqrt(s*(s-a)*(s-b)*(s-c))
```

Por exemplo, se queremos calcular o área do triángulo de vértices $(0,0)$, $(1,1)$ e $(1,2)$, resultaría

```python
A = heron((0.,0.),(1.,1.),(1.,2.))
display(A)
```

$\displaystyle 0.5$

Emprega tres integrais diferentes (usando o teorema de Fubini) para calcular o área do triángulo anterior.

```python
# O TEU CÓDIGO AQUÍ
```