# Exercise T: Optimized code and gradients computation using Theano

So far you have learn how to perform numerical methods in Python mainly using numpy. One of the main criticism of Python is its relative slow performance compared with other languages. Despite the huge leap that using numpy respect pure Python represents most of the operation in numpy still use single core architectures.

However, if Python suffers in performance where it really shines is in flexibility. Cython (http://cython.org/) or PyCUDA (https://documen.tician.de/pycuda/) are two good examples of libraries that use the easy to learn Python syntaxis combining it with the speed or lower levels languages.

Today we will learn to use the symbolic package Theano (http://deeplearning.net/software/theano/)). Theano originally started as a research project in neuro networks by the University of Monreal but also its symbolic code facilitate the use of Cython and PyCUDA.

## Introducing Theano

Theano is a Python package that take over many of the optimization tasks in order to create a feasible and fast code. Theano relies on the creation of symbolical graphs that represent the mathematical expressions to compute. Most of the extended programming paradigms (e.g. procedural languages and object-oriented programming) are executed sequentially without any interaction with the subsequent instructions. In contrast, symbolic programming allows to have the primary data structure and logic of a function defined from the beginning to the end what, in turn, enables the manipulation of this description.

The Python code is used to create an acyclic network graph where the parameters are represented by the nodes while the connections determine the mathematical operator that relates them. The creation of the graph is done in the class \texttt{theanograph}. Each method correspond to a piece of the graph starting from the input data all the way to the geological model or the forward gravity (see figure \ref{fig:overall}, theano area).

The symbolic graphs is later analyzed to perform the optimizers, the symbolic differentiation and the compilation to a faster language than Python (C or CUDA). This process is computational demanding and therefore it must to be avoided as much as possible.

Among the most outstanding optimizers be can find: (i) the canonicalization of the operations to simplify unnecessary computations, (ii) specialization of operation what improves consecutive element-wise operation, (iii) in-place operations to avoid duplications of memory or (iv) Open MP parallelization for CPU computations. These optimizations and more can speed up the code an order of magnitude.

## Minimal example

### Installation
First of all we will need to install Theano. In macOS and linux with a ```conda install theano``` in the console should be suficient. For Windows users they would need the extra step of installing the GCC compiler for using C code.

To use the GPU would be necessary to install CUDA drivers but we will skip this part for this class.

Once you have installed the library try running the following cell to make sure that is working properly.

```python
install theano
```

```python
import theano.tensor as tt
import theano

a = tt.arange(100)

(a**2).eval()
```

### Adding two scalars

```python
# Declare the input variables
x = tt.dscalar('x') # In this case d is from double and scalar that is only
y = tt.dscalar('y') # a number
z = 2*(x + y**3)       # We define z as a deterministic operation of the input
z.name = 'z'        # We name the parameter.

# We compile the function. Depending on theano configuration this would be
# in Python, C or Cuda
f = theano.function([x, y], z)
```

If you have the right dependencies
```pip install pydot-ng```
and
http://www.graphviz.org/
installed you can visualize the function in
a graphical way:

```python
import theano.d3viz as d3v
from IPython.display import IFrame
# create the graph
d3v.d3viz(f, 'theano_graphs/small.html')

# visualize the graph. Dependencies are not necessary
IFrame('theano_graphs/small.html', width=700, height=500)
```

Now we can call the function as any other:

```python
f(3,5)
```

    array(256.0)

### Exercise: Assignment 1 in Theano

So here we have the solution to the first assignment using numpy. Theano tensor has most of the numpy methods what makes relatively easy to translate the code to the symbolic syntax of theano. Try to create a theano function analogous to s.

NOTE: theano do not include tt.factorial operation but an equivalent one. Stack overflow is your friend.

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Defining variables
h0_value=100
t=10*3600
step=0.1
radius=100.05

# ------------------------
# Drawdown function
def s(t,r,Q=1E-5,T=1E-5,S=1E-4):
    u=r**2*S/(4*T*t)
    W=-0.577215-np.log(u)+u-(u**2/(2*np.math.factorial(2)))
    +(u**3/(3*np.math.factorial(3)))-(u**4/(4*np.math.factorial(4)))
    s=(Q/(4*np.pi*T))*W
    return s
# ------------------------

# Generating the 2D meshgrid
r=np.array(np.arange(-radius,radius+step,step)) #r is the radius vector
(X, Y) = np.meshgrid(r, r)
distance = np.sqrt(X**2+Y**2)

# Calling the drawdown function
h = s(t, distance)

# Plotting in 3D
from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure(figsize=(12,12))

ax2 = fig.add_subplot(212, projection='3d')
sf = ax2.plot_surface(X, Y, -h, cmap='Reds_r', rstride=100, cstride=100)
ax2.set_xlabel("x")
ax2.set_ylabel("y")
ax2.set_zlabel("z")
ax2.set_title("Drawdown cone")
fig.colorbar(sf)

```

```python
# Here your code (theano version).

t_th = tt.scalar('time')
r_th = tt.matrix('radio')

# ...
```

```python
IFrame('theano_graphs/drawdown.html', width=700, height=500)
```

```python
h_th = function(t, distance)
```

```python
fig = plt.figure(figsize=(12,12))

ax2 = fig.add_subplot(212, projection='3d')
sf = ax2.plot_surface(X, Y, -h_th, cmap='viridis_r', rstride=100, cstride=100)
ax2.set_xlabel("x")
ax2.set_ylabel("y")
ax2.set_zlabel("z")
ax2.set_title("Drawdown cone")
fig.colorbar(sf)
```

## Performance Comparison

Now that you have a numpy function and a theano function you can compare the performance for different resolutions (controlled by the step size) of one code and the other.

```python
# Example of profiling
theano_gpu = []

for step in np.logspace(-1, 1, num=10):
    # Generating the 2D meshgrid
    r=np.array(np.arange(-radius,radius+step,step)) #r is the radius vector
    (X, Y) = np.meshgrid(r, r)
    distance = np.sqrt(X**2+Y**2)

    time = %timeit -r 2 -o function(t, distance)
    theano_gpu.append(time.best)

# You can save the results in a file using
np.save('gpu_results', theano_gpu)

```

    11.1 ms ± 15.9 µs per loop (mean ± std. dev. of 2 runs, 100 loops each)
    4.05 ms ± 33.9 µs per loop (mean ± std. dev. of 2 runs, 100 loops each)
    1.54 ms ± 5.09 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    729 µs ± 1.93 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    483 µs ± 1.62 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    403 µs ± 3.4 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    91.8 µs ± 42.1 ns per loop (mean ± std. dev. of 2 runs, 10000 loops each)
    365 µs ± 4.87 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    364 µs ± 446 ns per loop (mean ± std. dev. of 2 runs, 1000 loops each)
    359 µs ± 4.86 µs per loop (mean ± std. dev. of 2 runs, 1000 loops each)

```python
# Here you code
```

Next is to compare it with multi threading code. To do so you need to do something before you run the notebook. Theano documentation is your friend

```python
# Plotting the results
# Here your code
```

The final step would be to add also the gpu performance. For using the gpu you need a nvidia card (actually I think in the last update of Theano they made the compatibility with the rest of cards but I did not try). In any case, setting up the system for using GPUs it is a bit tricky so you do not need to try (spoilert alert: it is the fastest)).

## Automatic differentiation

However, although theano code optimization is useful, the real deal here is the automatic differentiation. There are extensive literature (for a clear explanation see \cite{baydin2015automatic}) explaining all the capabilities and intuitions of the method since it is the core algorithm to train neural networks. Here, we will highlight the main difference with numerical approaches and how can be exploited to improve the modelling process.

Many of the most advanced algorithms in computer science rely on an inverse framework i.e. the result of a forward computation f(\textbf{x}) influence the value of one or many of the \textbf{x} latent variables, being the most emblematic example of this is an optimization of a cost function. All these problems can be described somehow as an exploration of a multidimensional manifold $f: \mathbb{R}^N \rightarrow \mathbb{R}$ and hence the gradient of the function $\nabla f = \left(\frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \frac{\partial f}{\partial x_n}\right)$ becomes key for an efficient analysis. In case that the output is also multidimensional---i.e. $f: \mathbb{R}^N \rightarrow \mathbb{R}^M$ the entire manifold gradient can be expressed by the Jacobian matrix

\begin{equation}
Jf =
\begin{bmatrix}
\frac{\partial f_1}{\partial x_1} & ... & \frac{\partial f_1}{\partial x_n}\\
\vdots							& \ddots & \vdots \\
\frac{\partial f_n}{\partial x_1} & ... & \frac{\partial f_m}{\partial x_n}
\end{bmatrix}
\end{equation}
of dimension $N \cdot M$, where $N$ is the number of variables and $M$ the number of functions that depend on those variables. Once, we have posed the problem now the question is how we compute the Jacobian matrix in a consistent and effective manner.

The most straightforward methodology consist in approximate the derivate by numerical differentiation applying finite differences approximations
\begin{equation}
\frac{\partial f_i}{\partial x_i} = \lim_{\it{h}\to 0} \frac{f(x_i+h)-f(x_i)}{h}
\end{equation}

where h is some discrete increment. The main advantage of numerical differentiation is that only computes $f$---evaluated for different values of \textbf{x}---what makes it very easy to implement in any available code. By contrast, the drawback is that for every element of the Jacobian we are introducing an approximation error that eventually can lead to mathematical instabilities but above all, the main limitation is the need of $M \cdot N$ evaluations of the function $f$ what in high dimensional problems quickly becomes prohibitive computationally expensive.

The alternative is to create the symbolic differentiation of $f$, this is decomposing $f$ in its primal operators and applying the chain rule to the correspondent transformation following the rules of differentiation to obtain $f'$. However, symbolic differentiation is not enough since the application of the chain rule leads to exponentially large expressions of $f'$ in what is known as expression swell \citep{cohen2003computer}. Luckily these large symbolic expressions are highly redundant on their terms allowing to exploit this redundancy by storing the repeated intermediate steps in memory and simply invoke them when necessary instead of computing the whole graph every time. This division of the program in sub-routines to store the intermediate results which are invoked several times is called dynamic programming \citep{bellman2013dynamic}. The simplified symbolic differentiation graph is ultimately what is called automatic differentiation. But that is not all, in the multivariate/multi-objective case the benefits of using AD increases linearly as the difference between the number of parameters $N$ and the number of objective functions $M$ get larger. Applying the same principle of redundancy explained above---this time between intermediate steps shared across multiple variables or multiple objective function---it is possible to reduce the number of evaluations necessary to compute the Jacobian either to $N$ in forward propagation or to $M$  in back propagation plus a small overhead on the evaluations (for a more detail description of the two modes of AD see for instance \cite{cohen2003computer}).

```python
# Declare the input variables
x = tt.dscalar('x') # In this case d is from double and scalar that is only
y = tt.dscalar('y') # a number
z = 2*(x + y**3)    # We define z as a deterministic operation of the input
z.name = 'z'        # We name the parameter.
g = theano.grad(z, [x, y])
# We compile the function. Depending on theano configuration this would be
# in Python, C or Cuda
f_grad = theano.function([x, y], g)
```

```python
d3v.d3viz(f_grad, 'theano_graphs/gradient.html')
IFrame('theano_graphs/gradient.html', width=700, height=500)
```

### Exercise: Compute the gradient

Try to compute the gradient of a point of the drawdown next to the well respect the constant parameters of the function (Q, T, S).

```python
IFrame('theano_graphs/gradient_drawdown.html', width=700, height=500)
```

What means to have the gradient of a parameter respect the others?


Computational Seismology

Finite Differences - Taylor Operators and Numerical Differentiation

---

This notebook is part of the supplementary material
to [Computational Seismology: A Practical Introduction](https://global.oup.com/academic/product/computational-seismology-9780198717416?cc=de&lang=en&#),
Oxford University Press, 2016.

##### Authors:
* Lion Krischer ([@krischer](https://github.com/krischer))
* Heiner Igel ([@heinerigel](https://github.com/heinerigel))
---

This exercise covers the following aspects:

* Use the Taylor series to calculate finite-difference operators
* Learn how to define high-order finite-difference operators
* Demonstrate the accuracy improvement by testing the operators on (arbitrary) functions

---

## Basic Equations

The Taylor expansion of $f(x + dx)$ around $x$ is defined as

$$
f(x+dx)=\sum_{n=0}^\infty \frac{f^{(n)}(x)}{n!}dx^{n}
$$

Finite-difference operators can be calculated by seeking weights (here: $a$, $b$, $c$) with which function values have to be multiplied to obtain an interpolation or a derivative. Example:

$$
\begin{align}
a ~ f(x + dx) & \ = \ a ~ \left[ ~  f(x) +  f^{'} (x) dx +  \frac{1}{2!}  f^{''} (x) dx^2   + \dotsc  ~ \right] \\
b ~ f(x) & \  = \ b ~ \left[ ~  f(x)  ~ \right] \\
c ~ f(x - dx) & \ = \ c ~ \left[ ~  f(x) -  f^{'} (x) dx +  \frac{1}{2!}  f^{''} (x) dx^2   - \dotsc  ~ \right]
\end{align}
$$

This can  be expressed in matrix form by comparing coefficients, here seeking a 2nd derivative

$$
\begin{align}
&a  ~~+~~            ~~~~b           &+~~  c & = & 0 \\
&a  ~~\phantom{+}~~ \phantom{b}  &-~~  c & = & 0 \\
&a  ~~\phantom{+}~~ \phantom{b}  &+~~  c & = & \frac{2!}{\mathrm{d}x^2}
\end{align}
$$

which leads to

$$
\begin{pmatrix}
  1 & 1 & 1 \\
  1 & 0 & -1 \\
  1 & 0 & 1
 \end{pmatrix}
\begin{pmatrix}
  a\\
 b \\
 c
 \end{pmatrix}
 =
 \begin{pmatrix}
  0\\
 0 \\
 \frac{2!}{dx^2}
 \end{pmatrix}
$$

and using matix inversion we obtain

$$
\begin{pmatrix}
a \\
b\\
c
\end{pmatrix}
=
\begin{pmatrix}
\frac{1}{2 \mathrm{d}x^2} \\
 - \frac{2}{2 \mathrm{d}x^2} \\
  \frac{1}{2 \mathrm{d}x^2}
\end{pmatrix}
$$

This is the the well known 3-point operator for the 2nd derivative. This can easily be generalized to higher point operators and higher order derivatives. Below you will find a routine that initializes the system matrix and solves for the Taylor operator.

---

## Calculating the Coefficients

The subroutine `central_difference_coefficients()` initalizes the system matrix and solves for the difference weights assuming $dx=1$. It calculates the centered differences using an arbitrary number of coefficients, also for higher derivatives.

It returns a central finite difference stencil of order `points` for the `n`th derivative.

```python
%matplotlib inline

import math
import numpy as np

def central_difference_coefficients(points, n):
    """
    Calculate the central finite difference stencil for an arbitrary number
    of points and an arbitrary order derivative.

    :param points: The number of points for the stencil. Must be
        an odd number.
    :param n: The derivative order. Must be a positive number.
    """
    if not points % 2:
        raise ValueError("Only uneven number of points allowed.")
    if points <= n:
        raise ValueError("The %ith derivative requires at least %i points" %
                         (n, n + 2))
    a = points
    m = np.zeros((a, a))
    for i in range(a):
        for j in range(a):
            dx = j - a // 2
            m[i][j] = dx ** i

    s = np.zeros(a)
    s[n] = math.factorial(n)
    return np.linalg.solve(m, s)
```

---

## (Almost) Christmas

The coefficients quickly decay and yield diminishing returns for higher order stencils. Higher order derivatives naturally require longer stencils.

```python
import matplotlib.pyplot as plt
plt.style.use('ggplot')

plt.figure(figsize=(10, 8))

for _i, points in enumerate(range(3, 20, 2)):
    coeffs = central_difference_coefficients(points, 1)
    numbers = np.linspace(- (points // 2), points // 2, points)
    plt.plot(numbers, coeffs - _i * 1, color="#55A868", lw=2.5)
    plt.plot(numbers, coeffs - _i * 1, color="#F1DB40", lw=0, marker="*", markersize=15)
    plt.hlines(-(_i * 1), -100, 100, color="0.5", zorder=0)

plt.scatter([0], [1.5], marker="*", s=1000, color="#C95241")

plt.yticks([])
plt.vlines(0, *plt.ylim(), color="0.5", zorder=0)
plt.xlim(-(points // 2 + 1), points // 2 + 1)
plt.ylim(-(_i + 2), 3)
plt.title("FD Tree for the first derivative")
plt.show()
```

```python
import matplotlib.pyplot as plt
plt.style.use('ggplot')

plt.figure(figsize=(10, 8))

for _i, points in enumerate(range(3, 20, 2)):
    coeffs = central_difference_coefficients(points, 2)
    numbers = np.linspace(- (points // 2), points // 2, points)
    plt.plot(numbers, coeffs - _i * 3, color="#55A868", lw=2.5)
    plt.plot(numbers, coeffs - _i * 3, color="#F1DB40", lw=0, marker="*", markersize=15)
    plt.hlines(-(_i * 3), -100, 100, color="0.5", zorder=0)

plt.scatter([0], [3.5], marker="*", s=1000, color="#C95241")

plt.yticks([])
plt.vlines(0, *plt.ylim(), color="0.5", zorder=0)
plt.xlim(-(points // 2 + 1), points // 2 + 1)
plt.ylim(-(3 * _i + 2 * 3), 6)
plt.title("FD Tree for the second derivative")
plt.show()
```

```python
import matplotlib.pyplot as plt
plt.style.use('ggplot')

plt.figure(figsize=(10, 8))

for _i, points in enumerate(range(5, 20, 2)):
    coeffs = central_difference_coefficients(points, 3)
    numbers = np.linspace(- (points // 2), points // 2, points)
    plt.plot(numbers, coeffs - _i * 3, color="#55A868", lw=2.5)
    plt.plot(numbers, coeffs - _i * 3, color="#F1DB40", lw=0, marker="*", markersize=15)
    plt.hlines(-(_i * 3), -100, 100, color="0.5", zorder=0)

plt.scatter([0], [5.5], marker="*", s=1000, color="#C95241")

plt.yticks([])
plt.vlines(0, *plt.ylim(), color="0.5", zorder=0)
plt.xlim(-(points // 2 + 1), points // 2 + 1)
plt.ylim(-(3 * _i + 2 * 3), 7)
plt.title("FD Tree for the third derivative")
plt.show()
```

---

## Accuracy of centered finite differences

Define a simple function, e.g. sine, cosine, gaussian, some polynomial, ..., differentiate it using the stencils returned from the above function and compare the result to the analytically calculated derivative.

```python
sine_curve = np.sin(np.linspace(0, 10 * np.pi, 10000))
dx = 10 * np.pi / 10000

plt.figure(figsize=(12, 6))
plt.subplot(211)
plt.plot(sine_curve)

stencil = central_difference_coefficients(5, 1)

# Dummy application.
cosine_curve = np.zeros(len(sine_curve))
for _i in range(2, len(sine_curve) - 2):
    cosine_curve[_i] = stencil[0] * sine_curve[_i - 2] + stencil[1] * sine_curve[_i - 1] + \
        stencil[2] * sine_curve[_i] + stencil[3] * sine_curve[_i + 1] + \
        stencil[4] * sine_curve[_i + 2]
# Scale by dt.
cosine_curve /= dx

plt.subplot(212)
plt.plot(cosine_curve[10:-10], label="Numerical derivative", lw=2)
plt.plot(np.cos(np.linspace(0, 10 * np.pi, 10000))[10:-10], ls="--",
         label="Analytic derivative")
plt.ylim(-1.1, 1.1)
plt.xlim(0, 9980)
plt.legend()
plt.show()
```

### Exercise 2

This exercise aims to illustrate the accuracy gains by using longer finite difference stencils. To this end we define a helper function that takes an array of x values, an array of the function sampled at these values to differentiate, the analytical derivative, the length of the stencil, and the order of the derivative.

```python
def diff_helper(x, data, analytical_diff, stencil_length, diff_order):
    plt.close()
    dx = x[1] - x[0]

    plt.figure(figsize=(12, 5))
    plt.subplot(311)
    # Be super conservative about edge effect.
    buf = stencil_length // 2 + 1
    x = x[buf:-buf]
    plt.plot(x, data[buf:-buf], label="Function")
    plt.legend()

    stencil = central_difference_coefficients(stencil_length, diff_order)

    # Its effectively a convolution. So use a pre-built convolution function.
    diff = np.convolve(data, stencil[::-1])
    diff /= dx ** diff_order
    diff = diff[stencil_length // 2: -(stencil_length // 2)]

    plt.subplot(312)
    plt.plot(x, diff[buf:-buf], label="Numerical derivative", lw=2)
    plt.plot(x, analytical_diff[buf:-buf], ls="--", label="Analytic derivative")
    plt.legend()

    plt.subplot(313)
    plt.plot(x, (diff - analytical_diff)[buf:-buf], label="Numerical - Analytical Derivative")
    plt.legend()

    plt.tight_layout()
    plt.show()
```

This shows a simple usage example. Play around a bit to get a feel how the accuracy improves with higher order operators. Make sure the function is not too oversampled, otherwise the effect is naturally not very strong. Polynomials of degree $N-1$ are exactly differentiated (first derivatives) by central differences of order $\ge N$. The coefficients are derived from Taylor polynomials after all.

```python
x = np.linspace(-2, 2, 150)
p = np.poly1d([1, 0, -2, 1, 0])
p_diff = np.poly1d([4, 0, -4, 1])
diff_helper(x, p(x), p_diff(x), 3, 1)
```

## Bonus: Theoretical Accuray

```python
def H(N, omega):
    coeffs = central_difference_coefficients(N, 1)
    HH = np.zeros_like(omega)
    for _i, c in enumerate(coeffs):
        HH += c * np.sin((-(N // 2) + _i) * omega)
    return HH

omega = np.linspace(0.0, np.pi, 100)

f = omega / (2 * np.pi)

plt.plot(f * 200, omega, color='0.2', ls="--")
for _i in range(3, 17, 2):
    plt.plot(f * 200, H(_i, omega), label="%i points" % _i)
plt.legend(loc="upper left")
plt.xlabel("% Nyquist")
plt.ylabel("$|H(\omega)|$")
plt.title("Amplitude Response of First Derivative Stencils")
plt.show()
```

```python
# Some tests against analytically calculated stencils. Not important for the exercises.

# f'(x) 5-point-stencil.
np.testing.assert_allclose(
    central_difference_coefficients(5, 1),
    np.array([-1.0 / 12.0, + 2.0 / 3.0, 0, -2.0 / 3.0, 1.0 / 12.0])[::-1],
    rtol=1E-8, atol=1E-10)

# f''(x) 5-point-stencil.
np.testing.assert_allclose(
    central_difference_coefficients(5, 2),
    np.array([-1.0 / 12.0, + 16.0 / 12.0, -30.0 / 12.0, 16.0 / 12.0, -1.0 / 12.0])[::-1],
    rtol=1E-8, atol=1E-10)

# f'''(x) 5-point-stencil.
np.testing.assert_allclose(
    central_difference_coefficients(5, 3),
    np.array([1.0 / 2.0, -1.0, 0, 1.0, -1.0 / 2.0])[::-1],
    rtol=1E-8, atol=1E-10)

# f''''(x) 5-point-stencil.
np.testing.assert_allclose(
    central_difference_coefficients(5, 4),
    np.array([1.0, -4.0, 6.0, -4.0, 1.0])[::-1],
    rtol=1E-8, atol=1E-10)
```

notebooks/Computational Seismology/The Finite-Difference Method/fd_taylor_operators_advanced.ipynb


In this notebook, you will implement the forward longitudinal vehicle model. The model accepts throttle inputs and steps through the longitudinal dynamic equations. Once implemented, you will be given a set of inputs that drives over a small road slope to test your model.

The input to the model is a throttle percentage $x_\theta \in [0,1]$ which provides torque to the engine and subsequently accelerates the vehicle for forward motion.

The dynamic equations consist of many stages to convert throttle inputs to wheel speed (engine -> torque converter -> transmission -> wheel). These stages are bundled together in a single inertia term $J_e$ which is used in the following combined engine dynamic equations.

\begin{align}
    J_e \dot{\omega}_e &= T_e - (GR)(r_{eff} F_{load}) \\ m\ddot{x} &= F_x - F_{load}
\end{align}

Where $T_e$ is the engine torque, $GR$ is the gear ratio, $r_{eff}$ is the effective radius, $m$ is the vehicle mass, $x$ is the vehicle position, $F_x$ is the tire force, and $F_{load}$ is the total load force.

The engine torque is computed from the throttle input and the engine angular velocity $\omega_e$ using a simplified quadratic model.

\begin{align}
    T_e = x_{\theta}(a_0 + a_1 \omega_e + a_2 \omega_e^2)
\end{align}

The load forces consist of aerodynamic drag $F_{aero}$, rolling friction $R_x$, and gravitational force $F_g$ from an incline at angle $\alpha$. The aerodynamic drag is a quadratic model and the friction is a linear model.

\begin{align}
    F_{load} &= F_{aero} + R_x + F_g \\
    F_{aero} &= \frac{1}{2} C_a \rho A \dot{x}^2 = c_a \dot{x}^2\\
    R_x &= N(\hat{c}_{r,0} + \hat{c}_{r,1}|\dot{x}| + \hat{c}_{r,2}\dot{x}^2) \approx c_{r,1} \dot{x}\\
    F_g &= mg\sin{\alpha}
\end{align}

Note that the absolute value is ignored for friction since the model is used for only forward motion ($\dot{x} \ge 0$).

The tire force is computed using the engine speed and wheel slip equations.

\begin{align}
    \omega_w &= (GR)\omega_e \\
    s &= \frac{\omega_w r_e - \dot{x}}{\dot{x}}\\
    F_x &= \left\{\begin{array}{lr}
        cs, &  |s| < 1\\
        F_{max}, & \text{otherwise}
        \end{array}\right\}
\end{align}

Where $\omega_w$ is the wheel angular velocity and $s$ is the slip ratio.

We setup the longitudinal model inside a Python class below. The vehicle begins with an initial velocity of 5 m/s and engine speed of 100 rad/s. All the relevant parameters are defined and like the bicycle model, a sampling time of 10ms is used for numerical integration.

```python
import sys
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

class Vehicle():
    def __init__(self):

        # ==================================
        #  Parameters
        # ==================================

        #Throttle to engine torque
        self.a_0 = 400
        self.a_1 = 0.1
        self.a_2 = -0.0002

        # Gear ratio, effective radius, mass + inertia
        self.GR = 0.35
        self.r_e = 0.3
        self.J_e = 10
        self.m = 2000
        self.g = 9.81

        # Aerodynamic and friction coefficients
        self.c_a = 1.36
        self.c_r1 = 0.01

        # Tire force
        self.c = 10000
        self.F_max = 10000

        # State variables
        self.x = 0
        self.v = 5
        self.a = 0
        self.w_e = 100
        self.w_e_dot = 0

        self.sample_time = 0.01

    def reset(self):
        # reset state variables
        self.x = 0
        self.v = 5
        self.a = 0
        self.w_e = 100
        self.w_e_dot = 0
```

Implement the combined engine dynamic equations along with the force equations in the cell below. The function $\textit{step}$ takes the throttle $x_\theta$ and incline angle $\alpha$ as inputs and performs numerical integration over one timestep to update the state variables. Hint: Integrate to find the current position, velocity, and engine speed first, then propagate those values into the set of equations.

```python
class Vehicle(Vehicle):
    def step(self, throttle, alpha):
        # ==================================
        #  Implement vehicle model here
        # ==================================

        self.v += self.sample_time * self.a
        self.x += self.sample_time * self.v
        self.w_e += self.sample_time * self.w_e_dot

        F_g = self.m * self.g * np.sin(alpha)
        R_x = self.c_r1 * self.v
        F_aero = self.c_a * (self.v ** 2)
        F_load = F_aero + R_x + F_g

        w_w = self.GR * self.w_e
        s = (w_w * self.r_e - self.v) / self.v
        F_x = 0
        if abs(s) < 1:
            F_x = self.c * s
        else:
            F_x = self.F_max
        accel = (F_x - F_load) / self.m

        T_e = throttle * (self.a_0 + self.a_1 * self.w_e + self.a_2 * (self.w_e ** 2))

        w_e_dot = (T_e - self.GR * self.r_e * F_load) / self.J_e

        self.a = accel
        self.w_e_dot = w_e_dot
```

Using the model, you can send constant throttle inputs to the vehicle in the cell below. You will observe that the velocity converges to a fixed value based on the throttle input due to the aerodynamic drag and tire force limit. A similar velocity profile can be seen by setting a negative incline angle $\alpha$. In this case, gravity accelerates the vehicle to a terminal velocity where it is balanced by the drag force.

```python
sample_time = 0.01
time_end = 100
model = Vehicle()

t_data = np.arange(0,time_end,sample_time)
v_data = np.zeros_like(t_data)

# throttle percentage between 0 and 1
throttle = 0.2

# incline angle (in radians)
alpha = 0

for i in range(t_data.shape[0]):
    v_data[i] = model.v
    model.step(throttle, alpha)

plt.plot(t_data, v_data)
plt.show()
```

We will now drive the vehicle over a slope as shown in the diagram below.

To climb the slope, a trapezoidal throttle input is provided for the next 20 seconds as shown in the figure below.

The vehicle begins at 20% throttle and gradually increases to 50% throttle. This is maintained for 10 seconds as the vehicle climbs the steeper slope. Afterwards, the vehicle reduces the throttle to 0.

In the cell below, implement the ramp angle profile $\alpha (x)$ and throttle profile $x_\theta (t)$ and step them through the vehicle dynamics. The vehicle position $x(t)$ is saved in the array $\textit{x_data}$. This will be used to grade your solution.

```python
time_end = 20
t_data = np.arange(0,time_end,sample_time)
x_data = np.zeros_like(t_data)
throttle_data = np.zeros_like(t_data)
alpha_data = np.zeros_like(t_data)
v_data = np.zeros_like(t_data)

# reset the states
model.reset()

throttle_data[:500] = np.arange(0.2, 0.5, 0.3/500)
throttle_data[500:1500] = 0.5
throttle_data[1500:] = np.arange(0.5, 0.0, -0.5/500)

alpha_data[:673] = np.arctan2(1, 20)
alpha_data[673:1509] = np.arctan2(1, 10)
alpha_data[1509:] = 0

# ==================================
#  Learner solution begins here
# ==================================

for i in range(t_data.shape[0]):
    x_data[i] = model.x
    v_data[i] = model.v

    model.step(throttle_data[i], alpha_data[i])

# ==================================
#  Learner solution ends here
# ==================================

# Plot x vs t for visualization
# plt.plot(t_data, x_data)
plt.plot(t_data, v_data)
plt.show()
```

If you have implemented the vehicle model and inputs correctly, you should see that the vehicle crosses the ramp at ~15s where the throttle input begins to decrease.

The cell below will save the time and vehicle inputs as text file named $\textit{xdata.txt}$. To locate the file, change the end of your web directory to $\textit{/notebooks/Course_1_Module_4/xdata.txt}$

Once you are there, you can download the file and submit to the Coursera grader to complete this assessment.

```python
data = np.vstack([t_data, x_data]).T
np.savetxt('xdata.txt', data, delimiter=', ')
```

Congratulations! You have now completed the assessment! Feel free to test the vehicle model with different inputs in the cell below, and see what trajectories they form. In the next module, you will see the longitudinal model being used for speed control. See you there!

```python
sample_time = 0.01
time_end = 30
model.reset()

t_data = np.arange(0,time_end,sample_time)
x_data = np.zeros_like(t_data)

# ==================================
#  Test various inputs here
# ==================================
for i in range(t_data.shape[0]):

    model.step(0,0)

plt.axis('equal')
plt.plot(x_data, y_data)
plt.show()
```


# The Exponential Distribution and the Poisson Process
## Introduction

One simplifying assumption that is often made is to assume that certain $r.v.\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\Exp}{\mathrm{E}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\Var}[2][\,\!]{\mathrm{Var}_{#1}\left[#2\right]}
\newcommand{\Avar}[2][\,\!]{\mathrm{Avar}_{#1}\left[#2\right]}
\newcommand{\Cov}[2][\,\!]{\mathrm{Cov}_{#1}\left(#2\right)}
\newcommand{\Corr}[2][\,\!]{\mathrm{Corr}_{#1}\left(#2\right)}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N} \left( #1 \right)}
\newcommand{\ow}{\text{otherwise}}$ are exponentially distributed.

- easy to work with
- good approximation to the actual distribution

## The Exponential Distribution
### Definition

A continuous $r.v.$ $X$ is said to have an exponential distribution with parameter $\lambda, \lambda>0$, if its probability density function is given by

$\bspace f\P{x} = \begin{cases}
\lambda e^{-\lambda x}, &\text{if } x \geq 0\\
0 , &\text{if }x<0
\end{cases}$

or *equivalently*, if its cumulative density function is given by

$\bspace F\P{x} = \d{\int_{-\infty}^{x} f\P{s}\;\dd s} = \begin{cases}
1-e^{-\lambda x}, &\text{if } x \geq 0\\
0 , &\text{if }x<0
\end{cases}$

Its mean, $\Exp\SB{X}$ is given by

$\bspace\begin{align}
\Exp\SB{X} &= \int_{-\infty}^{\infty} xf\P{x}\;\dd x\\
&= \int_{0}^{\infty} x\cdot \lambda e^{-\lambda x}\;\dd x\\
&= \left.\P{\ffrac{1}{\lambda} - x}e^{-\lambda x}\right|_{0}^\infty = \ffrac{1}{\lambda}
\end{align}$

Its moment generating function $\phi\P{t}$ is given by

$\bspace\begin{align}
\phi\P{t} &= \Exp\SB{e^{tX}}\\
&= \int_{0}^{\infty} e^{tx}\cdot \lambda e^{-\lambda x}\;\dd x\\
&= \left. \ffrac{\lambda}{t-\lambda}e^{\P{t-\lambda} x} \right|_{0}^\infty = \ffrac{-\lambda}{t-\lambda}, \bspace t<\lambda
\end{align}$

Other useful moments and variance, from $\text{mgf}$, we have $\Exp\SB{X^2} = 2/\lambda^2$ and $\Var{X} = 1/\lambda^2$

**e.g.1** Exponential Random Variables and Expected Discounted Returns

Here's the short version. $R\P{x}$ is the random rate at which you are receiving rewards at time $x$. For the discounted rate $\alpha$ we have the total discounted reward

$\bspace R = \d{\int_0^\infty e^{-\alpha x} R\P{x}\;\dd x} \Rightarrow \Exp\SB{R} = \int_0^\infty e^{-\alpha x}\Exp\SB{R\P{x}}\;\dd x$

If we let $T$ be an exponential random variable with parameter $\alpha$, which is independent of all other $r.v.$. Prove $\Exp\SB{R} = \Exp\SB{\d \int_0^T R\P{x}\;\dd x}$

> First define, for each $x \geq 0$, a $r.v.$ $I\P{x}$ by
>
>$\bspace I\P{x} = \begin{cases}
1,&\text{if } x\leq T\\
0,&\text{if } x > T
\end{cases} \Rightarrow \d \int_0^T R\P{x}\;\dd x = \int_{0}^\infty R\P{x}I\P{x} \;\dd x$
>
>Thus,
>
>$\bspace\begin{align}
\Exp\SB{\d \int_0^T R\P{x}\;\dd x} &= \Exp\SB{\d \int_0^\infty R\P{x}I\P{x}\;\dd x}\\
&= \d \int_0^\infty \Exp\SB{R\P{x}}\Exp\SB{I\P{x}}\;\dd x\\
&= \int_0^\infty \Exp\SB{R\P{x}}P\CB{T\geq x}\;\dd x\\
&= \int_0^\infty \Exp\SB{R\P{x}}\cdot e^{-\alpha x}\;\dd x = \Exp\SB{R}
\end{align}$

***

### Properties of the Exponential Distribution

A $r.v.$ $X$ is said to be *without memory*, or ***memoryless***, if

$\bspace P\CB{X>s+t \mid X>t} = P\CB{X>s} \iff \ffrac{P\CB{X > s+t,X>t}}{P\CB{X>t}} = P\CB{X>s},\bspace \forall s,t \geq 0$

Consider the expondentially distributed $X$ which is certainly satisfy this condition since

$\bspace e^{-\lambda t} \cdot e^{-\lambda s} = e^{-\lambda\P{s+t}}$

To illustrate this see the following examples.

**e.g.2**

Suppose that the amount of time one spends in a bank is exponentially distributed with mean ten minutes, that is $\lambda = 0.1$. What is the probability that a customer will spend more than fifteen minutes in the bank given that she is still in the bank after ten minutes?

>Let $X$ represents the amount of time that the customer spends in the bank.
>
>$P\CB{X > 15 \mid X>10} = P\CB{X>5} = e^{-5\lambda} \approx 0.604$

***

**e.g.4**

The dollar amount of damage of an accident is exponentially distributed with mean $1000$. However the insurance company only pays that amount exceeding $400$. Find the mean and the variance of the amount the insurance company pays per accident.

>Let $X$ denote the dollar amount of damage resulting from an accident, then payment from insurance company is $\P{X-400}^+$, or define a indicator $r.v.$ $I = \begin{cases}
1,&\text{if } X>400\\
0,&\text{if } X \leq 400
\end{cases}$

>Then letting $Y = \P{X-400}^+$, we have $\Exp\SB{Y\mid I} = 1000\cdot I$, $\Var{Y\mid I} = 10^6\cdot I$. Then
>
>$\bspace \Exp\SB{Y} = \Exp\SB{\Exp\SB{Y\mid I}} = 10^3\Exp\SB{I} = 10^3\cdot P\CB{X>400} = 10^3\cdot \exp\CB{-\ffrac{1}{1000}\cdot 400} \approx 670.32$
>
>$\bspace \Var{Y} = \Exp\SB{\Var{Y\mid I}} + \Var{\Exp\SB{Y \mid I}} = 10^6 \cdot e^{-0.4} + 10^6\cdot e^{-0.4}\P{1-e^{-0.4}}$

***

It turns out that exponential distribution is the only memoryless one. To see this, suppose $X$ is memoryless and let $\bar F\P{X} = P\CB{X>x} = 1 - F\P{X}$. Then by the definition we have $\bar F\P{s+t} = \bar F\P{s} \bar F\P{t}$, meaning that $\bar F$ satisfies the functional equation $g\P{s+t} = g\P{s}g\P{t}$. However, it turns out that the only *right continuous* solution of this functional equation is $g\P{x} = e^{-\lambda x}$.

> Given the equation we have $g\P{m/n} = g^m\P{1/n}$. Especially, we have $g\P{1} = g^n\P{1/n}$. Hence
>
>$\bspace g\P{m/n} = g^{m/n}\P{1}$
>
>which implies, since $g$ is *right continuous*, that $g\P{x} = \P{g\P{1}}^x$. Since $g\P{1} = g^2\P{0.5}\geq0$, we obtain $g\P{x} = e^{-\lambda x}$ where $\lambda = -\log \P{g\P{1}}$.

So we must have $\bar F\P{x} = e^{-\lambda x}$, and consequently, $F\P{x} = 1-e^{-\lambda x}$, which shows that $X$ is exponentially distributed.

Then we define the ***failure rate function***: $r\P{t} = \ffrac{f\P{t}}{1-F\P{t}}$. And to interpret this, suppose that an item with lifetime $X$, has survived for $t$ hours, then what's the probability that it doesn't survive for an additional time $\dd t$?

$\bspace \begin{align}
P\CB{X \in\P{t,t+\dd t}\mid X>t} &= \ffrac{P\CB{X \in\P{t,t+\dd t},X>t}}{P\CB{X>t}}\\
&= \ffrac{P\CB{X \in\P{t,t+\dd t}}}{P\CB{X>t}}\approx \ffrac{f\P{t}\dd t}{1-F\P{t}} = r\P{t}\dd t
\end{align}$

$\bspace r\P{t}$ *represents the conditional probability density that a* $t$*-year-old item will fail.*

And by the memoryless property, for the exponential distribution, we have

$\bspace r\P{t} = \ffrac{f\P{t}}{1-F\P{t}} = \ffrac{\lambda e^{-\lambda t}}{e^{-\lambda t}} = \lambda$

Here $\lambda$ is often referred to as the ***rate*** of the distribution (also as the *reciprocal倒数* of the mean $1/\lambda$). Besides, this **rate function** uniquely determine the cdf $F$:

$\bspace r\P{t} = \ffrac{\ffrac{\dd}{\dd t}F\P{t}}{1-F\P{t}}\Rightarrow \log\P{1-F\P{t}}= -\d{\int_0^t r\P t\;\dd t + k} \Rightarrow 1-F\P t = e^k \exp\CB{-\d \int_0^t r\P t\;\dd t}$

And $k$ can be determined by letting $t=0$, so that the final solution is

$\bspace F\P t = 1-\exp\CB{-\d\int_0^t r\P t\;\dd t}$

More than that, by letting $r\P t = \lambda$, we have $F\P t = 1 - e^{-\lambda t}$ showing that this $r.v.$ is exponential.

**e.g.6** Hyperexponential $r.v.$

Let $X_1,\dots,X_n$ be *independent* exponential random variables with respective rates $\lambda_1,\dots,\lambda_n$, where $\lambda_i \neq \lambda_j$ when $i\neq j$. Then let $T$ be independent of all of them and suppose that

$\bspace \d\sum_{j=1}^n P_j = 1$, where $P_j = P\CB{T = j}$

Then $X_T$ is said to be a ***hyperexponential*** $r.v.$. Then what's its distribution? Surely we condition on $T$ and this yields

$\bspace \begin{align}
1-F\P t &= P\CB{X>t}\\
&= \sum_{i=1}^{n} P\CB{X>t \mid T = i}\cdot P\CB{T = i}\\
&= \sum_{i=1}^{n} P_i \cdot e^{-\lambda_i t}
\end{align}$

Then we differentiate the equation on $t$ leading to the pdf and failure function

$\bspace f\P t = \d\sum_{i=1}^n \lambda_i P_i e^{-\lambda_i t},\bspace r\P t = \ffrac{\d{\sum_{i=1}^n \lambda_i P_i e^{-\lambda_i t}}}{\d{\sum_{i=1}^n P_i e^{-\lambda_i t}}}$

Also note that

$\bspace\begin{align}
P\CB{T = j \mid X>t}&= \ffrac{P\CB{X>t \mid T=j}\cdot P\CB{T=j}}{P\CB{X>t}}\\
&= \ffrac{P_j e^{-\lambda_j t}}{{\sum_{i=1}^n P_i e^{-\lambda_i t}}}\\
\Longrightarrow r\P{t} &= \d{\sum_{j=1}^n\lambda_j} P\CB{T=j\mid X>t}
\end{align}$

And one last conclusion, that if $\lambda_1<\lambda_i, \forall\; i>1$, then

$\bspace\begin{align}
P\CB{T=1\mid X>t}&= \ffrac{P_1 e^{-\lambda_1 t}}{P_1 e^{-\lambda_1 t} + \sum_{i=1}^n P_i e^{-\lambda_i t}}\\
&= \ffrac{P_1}{P_1 + \sum_{i=1}^n P_i e^{-\P{\lambda_i -\lambda_1} t}} \to 1,\bspace t\to \infty
\end{align}$

So that actually $\d{\lim_{t\to\infty} r\P t = \min_i \lambda_i}$

***

### Further Properties of the Exponential Distribution

Let $X_1,\dots,X_n$ be *independent* exponential random variables with common mean $1/\lambda$. Following the discussion in Chap_02, we know that the distribution of their sum has a *gamma* distribution with parameters $n$ and $\lambda$. Here's another proof by using *mathematical induction*.

The conclusion holds true at $n=1$, so now assume that $X_1 + X_2+\cdots+X_{n-1}$ has density given by

$\bspace\d{f_{X_1+ X_2+\cdots+X_{n-1}}\P t = \lambda e^{-\lambda t}\ffrac{ \P{\lambda t}^{n-2}}{\P{n-2}!}}$

Hence, the $n$ sum is, by the independency,

$\bspace\begin{align}
f_{X_1+ X_2+\cdots+X_{n-1} \:+ X_n}\P t &= \int_0^\infty f_{X_n}\P{t-s} f_{X_1+ X_2+\cdots+X_{n-1}}\P s \;\dd s\\
&= \int_0^t \lambda e^{-\lambda\P{t-s}} \lambda e^{-\lambda s} \ffrac{\P{\lambda s}^{n-2}}{\P{n-2}!} \;\dd s = \lambda e^{-\lambda t} \ffrac{\P{\lambda t}^{n-1}}{\P{n-1}!}
\end{align}$

***

Another useful thing, that is to determine the probability that one exponenetial $r.v.$ is smaller than another. Suppose that $X_1$ and $X_2$ are independent exponential $r.v.$ with respective rates $\lambda_1$ and $\lambda_2$, then what's $P\CB{X_1<X_2}$? Still, we condition on $X_1$ to find the result.

$\bspace\begin{align}
P\CB{X_1<X_2}&= \int_0^\infty P\CB{X_1<X_2\mid X_1 = x}\cdot P\CB{X_1 = x}\;\dd x\\
&= \int_0^\infty P\CB{x<X_2}\cdot \lambda_1 e^{-\lambda_{\void_1} x}\;\dd x\\
&= \int_0^\infty e^{-\lambda_{\void_2} x}\cdot \lambda_1 e^{-\lambda_{\void_1} x}\;\dd x\\
&= \ffrac{\lambda_1}{\lambda_1 + \lambda_2}
\end{align}$

More than this, we have the minimum of them is a exponential with a rate equal to the sum of all individual rates.

$\bspace\begin{align}
P\CB{\min_i\P{X_i}>x} &= P\CB{X_1>x,X_2>x,\dots,X_n>x}\\
&= \prod_{i=1}^n P\CB{X_i>x} = \prod_{i=1}^n e^{-\lambda_{\void_i}x} = \exp\CB{-\P{\sum_i \lambda_i} x}
\end{align}$

**e.g.7** Analyzing Greedy Algorithms for the Assignment Problem

Attractive! Skipped, though
***

Combine the two preceding results, we have

$\bspace \d{P\CB{X_i = \min_j X_j} = P\CB{X_i <\min_{j\neq i}X_j} = \ffrac{\lambda_i}{\lambda_i + \sum_{j\neq i} \lambda_j} = \ffrac{\lambda_i}{\sum_j \lambda_j}}$

$Proposition$

If $X_1,\dots,X_n$ are independent exponential $r.v.$ with respective rates $\lambda_1,\dots,\lambda_n$, then $\d{\min_i X_i}$ is exponential with rate $\d{\sum_{i=1}^{n}\lambda_i}$. Further, $\d{\min_i X_i}$ and the rank order of the variables $X_1,\dots,X_n$ are independent.

>The only thing we need to prove here is the last conclusion, the independency. And that can be easily seen using the property that it's **memoryless**
>
>$\bspace \d{P\CB{X_{i_1}<\cdots X_{i_n}\mid \min_i X_i >t} = P\CB{X_{i_1}-t<\cdots X_{i_n}-t\mid \min_i X_i >t} = P\CB{X_{i_1}<\cdots X_{i_n}}}$

**e.g.9** and **e.g.10** are also very interesting, and comprehensive. The strategies basically, are taking conditional probability, using recursions, or breaking one $r.v.$ to the sum of $k$ $r.v.$s.

***

### Convolutions of Exponential $r.v.$ (skipped)

The sum of $n$ independent exponential $r.v.$ with different respective rates $\lambda_i$, is said to be a ***hypoexponential*** $r.v.$. To calculate its pdf, let's start with case $n=2$. Now,

$\bspace\begin{align}
f_{X_1+X_2}\P t &= \int_0^t f_{X_1} \P s f_{X_2} \P{t-s}\;\dd s\\
&= \int_0^t \lambda_1 e^{-\lambda_{\void_1} s} \lambda_2 e^{-\lambda_{\void_2} \P{t-s}}\;\dd s\\
&= \lambda_1\lambda_2e^{-\lambda_{\void_2} t} \int_0^t e^{-\P{\lambda_{\void_1}-\lambda_{\void_2}}s}\;\dd s\\
&= \ffrac{\lambda_1\lambda_2e^{-\lambda_{\void_2} t}}{\lambda_1-\lambda_2} \P{1-e^{-\P{\lambda_{\void_1}-\lambda_{\void_2}}t}} = \ffrac{\lambda_1}{\lambda_1-\lambda_2}\lambda_2 e^{-\lambda_{\void_2}t} + \ffrac{\lambda_2}{\lambda_2-\lambda_1}\lambda_1 e^{-\lambda_{\void_1}t}
\end{align}$

And similarly, we have when $n=3$,

$\bspace\d{f_{X_1+X_2+X_3}\P t = \sum_{i=1}^3 \P{\lambda_i e^{-\lambda_{\void_i} t} \prod_{j\neq i} \ffrac{\lambda_j}{\lambda_j-\lambda_i} }}$

which suggests the general result, letting $S = X_1 + \cdots+X_n$,

$\bspace \d{f_{S}\P t = \sum_{i=1}^n \P{\lambda_i e^{-\lambda_{\void_i}t} \prod_{j\neq i} \ffrac{\lambda_j}{\lambda_j - \lambda_i}}}$

This can be proved by induction on $n$, still very complex through. Skipped. Alright, now we integrate both sides of the expression for $f_{S}$ from $t$ to $\infty$. The ***tail distribution function*** of $S$ is given by

$\d{\bspace P\CB{S>t} = \sum_{i=1}^n C_{i,n} e^{-\lambda_{\void_i}t},\bspace C_{i,n} = \prod_{j\neq i} \ffrac{\lambda_j}{\lambda_j - \lambda_i}}$

And the following **failure rate function** of $S$, is given by

$\bspace r_S\P t = \ffrac{\sum_{i=1}^n C_{i,n}\lambda_i e^{-\lambda_{\void_i}t}}{\sum_{i=1}^n C_{i,n} e^{-\lambda_{\void_i}t}}$

In addition, if we let $\lambda_j = \d{\min_j \lambda_j}$, then it follows that $\d{\lim_{t\to\infty}r_S\P t = \lambda_j}$

$Remark$

>Although
>
>$\bspace \d{1 = \int_0^\infty f_S\P t\;\dd t = \sum_{i=1}^n C_{i,n} = \sum_{i=1}^n \prod_{i\neq j} \ffrac{\lambda_j}{\lambda_j - \lambda_i}}$
>
>$C_{i,n}$ are NOT probabilities. Some of them may be negative. Also, this is far from same with **hyperexponential** density.

***

**e.g.11** Coxian $r.v.$

Whatever, skipped
***

## The Poisson Process
### Counting Processes

A stochastic process $\CB{N\P t , t\geq 0}$ is said to be a ***counting process*** if $N\P t$ represents the total number of "events" that occur by time $t$. And we can see this process must satisfy

1. $N\P{t}\geq 0$
2. $N\P t$ is integer valued
3. If $s<t$, then $N\P s \leq N\P t$
4. For $s<t$, $N\P t - N\P s$ equals the number of events that occur in $(s,t]$

A counting process is said to possess ***independent increments*** if the numbers of events that occur in disjoint time intervals are independent, like $N\P{10}$ and $\P{N\P{15} - N\P{10}}$. And a counting process is said to possess ***stationary increments*** if the distribution of the number of events that occur in any interval of time depends only on the *length* of the time interval, say for all $s$ the number of events in the interval $\P{s,s+t}$ has the same distribution.

### Definition of the Poisson Process

$Def.1$ $o\P h$

The function $f\P \cdot$ is said to be $o\P h$ if $\d{\lim_{h\to\infty} \ffrac{f\P h}{h} = 0}$

And with this we could now write

$\bspace\begin{align}
P\P{t < X <t+h} \approx f\P t \cdot h \Longrightarrow f\P t \cdot h + o\P h\\
P\P{t < X <t+h\mid X>t} \approx r\P t \cdot h \Longrightarrow r\P t \cdot h + o\P h
\end{align}$

$Def.2$

The counting process $\CB{N\P t, t\geq 0}$ is said to be a ***Poisson process*** with rate $\lambda >0$ if the following *axioms* hold:

1. $N\P 0 = 0$
2. $\CB{N\P t, t\geq 0}$ has independent increments
3. $P\CB{N\P{t+h} - N\P t = 1} = \lambda \cdot h + o\P h$
4. $P\CB{N\P{t+h} - N\P t \geq 2} = o\P h$

$Theorem.1$

If $\CB{N\P t, t\geq 0}$ is a **Poisson process** with rate $\lambda>0$, then for all $s>0$, $t>0$, $N\P{s+t} - N\P{s}$ is a Poisson $r.v.$ with mean $\lambda t$. That is to say, that the number of events in *any* interval of length $t$ is a Poisson $r.v.$ with mean $\lambda t$.

$Proof$

>We begin by deriving $\Exp\SB{e^{-uN\P t}}$, the ***Laplace transform*** of $N\P t$. First fix $u>0$ and define $g\P t = \Exp\SB{e^{-uN\P t}}$. Then a differential equation
>
>$\bspace\begin{align}
g\P{t+h} &= \Exp\SB{e^{-uN\P{t+h}}}\\
&= \Exp\SB{e^{-u\P{N\P{t+h} +N\P t - N\P t }}}\\
&= \Exp\SB{e^{-uN\P{t}} \cdot e^{-u\P{N\P{t+h} - N\P t}}}\\
&= \Exp\SB{e^{-uN\P{t}}} \cdot \Exp\SB{e^{-u\P{N\P{t+h} - N\P t}}} = g\P t \cdot \Exp\SB{e^{-u\P{N\P{t+h} - N\P t}}}
\end{align}$
>
>Then from the last two axioms, we have
>
>$\bspace\begin{align}P\CB{N\P{t+h} - N\P t = 0} &= 1 -P\CB{N\P{t+h} - N\P t = 1} -P\CB{N\P{t+h} - N\P t \geq 2}\\
&= 1 - \lambda\cdot h + o\P h\\
\Longrightarrow \Exp\SB{e^{-u\P{N\P{t+h} - N\P t}}} &= 1-\lambda \cdot h + o\P h + e^{-u}\P{\lambda \cdot h + o\P h} + o\P h\\
&= 1-\lambda \cdot h + \lambda e^{-u} \cdot h + o\P h
\end{align}$

>Therefore, $g\P{t+h} = g\P t \big(1+\lambda\P{e^{-u}-1}h + o\P h\big)$. Then solve this by first rewrite
>
>$\bspace \ffrac{g\P{t+h} - g\P t}{h} = g\P t \lambda \P{e^{-u}-1} + \ffrac{o\P h}{h}$
>
>Letting $h\to0$ yields the differential equation
>
>$\bspace g'\P t = g\P t \lambda \P{e^{-u} - 1 } \Longrightarrow \log\big(g\P t\big) = \lambda \P{e^{-u} - 1}t + C$
>
>Noting that $g\P 0 = \Exp\SB{e^{-uN\P 0}} = \Exp\SB{e^0} = 1$, it follows that $C=0$, and so the **Laplace transform** of $N\P t$ is
>
>$\bspace\Exp\SB{e^{-uN\P t}} = g\P t = e^{\lambda \P{e^{-u} \;-1}t}$
>
>Comparing this to the **Laplace transform** of a typical Poisson $r.v.$ $X$, with mean $\lambda t$,
>
>$\bspace\Exp\SB{e^{-uX}} = \d{\sum_i e^{-u i} \cdot\ffrac{e^{-\lambda t} \P{\lambda t}^i}{i!} = e^{-\lambda t}\sum_i \ffrac{\P{\lambda t e^{-u}}^i}{i!} } = e^{-\lambda t} \cdot e^{\lambda t e^{-u}} = e^{\lambda t \P{e^{-u}\; -1}}$
>
>Like $\text{mgf}$, **Laplace transform** uniquely determines the distribution, we can thus conclude that $N\P t$ is **Poisson** $r.v.$ with mean $\lambda t$.
>
>Then, to show that $N_s\P{t} =N\P{t+s} - N\P s$ is also Poisson with mean $\lambda t$, fix $s$ and then $N_s\P t$ is the number of events in the first $t$ time units when we start our count at time $s$. Then we can verify that $\CB{N_s\P t, t\geq 0}$ satisfies all the axioms for being a Poisson process with rate $\lambda$. Consequently, using the preceding result, we can conclude that $N_s\P t$ is a Poisson $r.v.$ with mean $\lambda t$.

$Remark$

>Since the distribution of $N\P{t+s} - N\P t$ is the same for all $s$, it follows that the **Poisson process** has **stationary increments**.
>
>Also, this conclusion is a consequnce of the *Poisson approximation to the **binomial** distribution*. To see this we can subdivide the interval $\SB{0,k}$ into $k$ equal parts. Using $Axiom.4$, as $k$ increases to $\infty$, the probability of having two or more events in any of the $k$ subintervals goes to $0$. Hence, $N\P t$ will equal to the number of subintervals in which an event occurs.
>
>Then by $Axiom.2$, about the independent increments, and the preceding **stationary increments** conclusion, we know that this number will have a binomial distribution with parameters $k$ and $p = \lambda t/k +o\P{t/k}$.
>
>Hence, by the Poisson approximation to the binomial, we see that when $k$ goes to $\infty$, $N\P t$ will have a **Poisson distribution** with mean:
>
>$\bspace\d{\lim_{t\to\infty} k\cdot\SB{\ffrac{\lambda t}{k} + o\P{\ffrac{t}{k}}} =\lambda t + \lim_{t\to\infty} \ffrac{t\cdot o\P{t/k}}{t/k} = \lambda t }$
>
>The last step can be seen by: $k\to \infty\Rightarrow t/k \to 0 \Rightarrow \ffrac{o\P{t/k}}{t/k}\to 0$
***

### Interarrival and Waiting Time Distributions

Consider a Poisson process, and let us denote the time when the first event happened by $T_1$. Further, for $n>1$, let $T_n$ denote the *elapsed time* between the $\P{n-1}$st and the $n$th event. Then the sequence $\CB{T_n, n=1,2,\dots}$ is called the ***sequence of interarrival times***. What's the distribution of $T_n$?

First note that $P\CB{T_1>t} = P\CB{N\P t = 0} = e^{-\lambda t}$. Hence, $T_1$ has an **exponential** distribution with mean $1/\lambda$. Now

$\bspace
P\CB{T_2 > t} = \Exp\big[P\CB{T_2>t\mid T_1}\big] = \d\int_0^\infty P\CB{T_2>t\mid T_1 = s} \cdot P\CB{T_1 = s} \;\dd s$

However, $P\CB{T_2>t\mid T_1 = s} = P\CB{0\text{ events in }(s,s+t]\mid T_1 = s}$ and because of the independent and stationary increments, we have

$\bspace P\CB{T_2>t\mid T_1 = s} = P\CB{0\text{ events in }(s,s+t]} = e^{-\lambda t}\Rightarrow P\CB{T_2 > t} = \d\int_0^\infty e^{-\lambda t} \cdot \lambda e^{-\lambda s} \;\dd s = e^{-\lambda t}$

Repeating the same argument yields the following

$Proposition.1$

$T_n$, $n=1,2,\dots$ are $i.i.d.$ **exponential** $r.v.$s with mean $1/\lambda$.

$Remark$

>Define $S_n$ as the ***waiting time*** until the $n$th events. Of course, $S_n = \sum\limits_{i=1}^n T_i$, for $n\geq 1$. Then as the sum of $i.i.d.$ **exponential** $r.v.$s with common rate $\lambda$, we have its density:
>
>$\bspace f_{S_n}\P t = \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^{n-1}}{ \P{n-1}! },\bspace t\geq 0$
>
>Another way to derive this is to first notice that $N\P t \geq n \iff S_n \leq t$, hence
>
>$\bspace\begin{align}
F_{S_n}\P t &= P\CB{S_n \leq t} = P\CB{N\P t \geq n} = \sum_{j=n}^\infty e^{-\lambda t}\ffrac{\P{\lambda t}^j}{j!}\\
f_{S_n}\P t &= -\sum_{j=n}^\infty \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^j}{j!} + \sum_{j=n}^\infty \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^\P{j-1}}{\P{j-1}!} \\
&= -\sum_{j=n}^\infty \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^j}{j!} + \sum_{j=n+1}^\infty \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^\P{j-1}}{\P{j-1}!} + \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^\P{n-1}}{\P{n-1}!} = \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^\P{n-1}}{\P{n-1}!}
\end{align}$

**e.g.13**

Suppose that people immigrate into a territory at a Poisson rate $\lambda = 1$ per day.

$\P{\text{a}}$ What is the expected time until the tenth immigrant arrives?

> $\Exp\SB{S_{10}} = 10\cdot \ffrac{1}{\lambda} = 10$ days

$\P{\text{b}}$ What is the probability that the elapsed time between the tenth and the eleventh arrival exceeds two days?

>$P\CB{T_{11}>2} = e^{-\lambda \cdot 2} = e^{-2} \approx 0.133$

***

And this newly defined $T_n$ leads to a new definition of a counting process. Given a sequence $\CB{T_n,n\geq1}$, the $n$th event of this process occurs at time $S_n \equiv T_1+T_2+\cdots+T_n$. So we define

$\bspace N\P t \equiv \max\CB{n:S_n\leq t}$

this resultant counting process $\CB{N\P t, t\geq 0}$ will be Poisson process with rate $\lambda$.

$Remark$

>This new definition leads to a new method to find the pdf of $S_n$.
>
>$\begin{align}
P\CB{t<S_n<t+h} &= P\CB{N\P t = n-1, \text{another one event in }(t,t+h]} + o\P h\\
&= P\CB{N\P t = n-1}\cdot P\CB{\text{another one event in }(t,t+h]} + o\P h\\
&= e^{-\lambda t}\ffrac{\P{\lambda t}^{n-1}}{\P{n-1}!} \cdot\P{\lambda h+o\P h} + o\P h\\
&= \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^{n-1}}{\P{n-1}!}\cdot h + o\P h\\
\Rightarrow f_{S_n}\P t &= \lambda e^{-\lambda t}\ffrac{\P{\lambda t}^{n-1}}{\P{n-1}!}
\end{align}$

***

### Further Properties of Poisson Processes

Consider a Poisson process $\CB{N\P t,t\geq 0}$ having rate $\lambda$, suppose we can classify the event as type **I** or a type **II** event, with probability $p$ and $1-p$ respectively. Let $N_1\P t$ and $N_2\P t$ denote respectively the number of type I and type II events occuring in $\SB{0,t}$. Then we have

$Proposition.2$

$N\P t = N_1\P t + N_2\P t$ and actually, $\CB{N_1\P t,t\geq 0}$ and $\CB{N_2\P t,t\geq 0}$ are both Poisson processes having rate $\lambda p$ and $\lambda\P{1-p}$, respectively. Also, the two processes are independent.

$Proof$

>First we prove that $\CB{N_i\P t,t\geq 0}$ are Poisson process, by checking the definition
>
>- $N_i\P{0} = 0$ follows from the fact that $N\P0 = 0$
>- Stationary and independent increment also holds. Whatever, just believe this.
>- $\begin{align}
P\CB{N_1\P h = 1} &= P\CB{N_1\P h = 1\mid N\P h = 1} \cdot P\CB{N\P h = 1} \\
&\bspace+ P\CB{N_1\P h = 1\mid N\P h \geq 2} \cdot P\CB{N\P h \geq 2} \\
&= p\P{\lambda h + o\P h} + o\P{h}\\
&= \lambda p h + o\P h
\end{align}$, and same for $P\CB{N_2\P h = 1}$
>- $P\CB{N_1\P h \geq 2}\leq P\CB{N\P h \geq 2} = o\P{h}$, similar for $P\CB{N_2\P h \geq 2}$
>
>Since the probability of a type I event in $(t,t+h]$ is independent of those in intervals that do not overlap $\P{t,t+h}$, therefore, it's independent of knowledge of when type II events occur.

**e.g.14**

Easy one.

**e.g.15**

Interesting, skipped though.

***

**e.g.16** About dividing into $r$ groups, $r>2$

Consider a system in which individuals at any time are classified as being in one of $r$ possible states, and assume that an individual changes states in accordance with a Markov chain having transition probabilities $P_{ij}$, $i , j = 1,\dots, r$. The individuals move independently. Suppose that the numbers of people initially in states $1,\dots, r$ are independent Poisson $r.v.$ with respective means $\lambda_1,\dots, \lambda_r$. We are interested in determining the *joint distribution* of the numbers of individuals in states $1,\dots, r$ at some time $n$.

>For fixed $i$, let $N_j\P i$, $j=1,2,\dots,r$ denote the number of those individuals initially in state $i$, that are in state $j$ at time $n$. And now each of the Poisson distributed number of people in state $i$ will, independently, be in state $j$ at time $n$ with probability $P_{ij}^n$.
>
>Hence, $N_j\P i$, $j=1,2,\dots,r$ will be independent Poisson random variables with respective means $\lambda_iP_{ij}^n$, $j=1,2,\dots,r$. Because *the sum of independent Poisson $r.v.$s is also a Poisson $r.v.$*. it follows that the number of individuals in state $j$ at time $n$, namely, $\sum\limits_{i=1}^r N_j\P i$, will be independent Poisson $r.v.$s with respective means $\sum\limits_i \lambda_i P_{ij}^n$, for $j=1,2,\dots,r$.
***

**e.g.17** Coupon collecting problem

Interesting, easy to understand, skipped though.
***

We now trying to find the probability that $n$ events occur in one Poisson process before $m$ events have occurred in a second and independent Poisson process. Let $\CB{N_1\P t,t\geq 0}$ and $\CB{N_2\P t,t\geq 0}$ be two independent Poisson processes having respective rates $\lambda_1$ and $\lambda_2$. Let $S_n^1$ denote the time of the $n$th event of the first process and $S_m^2$ the second one.

What's $P\CB{S_n^1 < S_m^2}$?

The answer is easy to obtain when $n=m=1$, which is $P\CB{S_1^1 < S_1^2} = \ffrac{\lambda_1}{\lambda_1 + \lambda_2}$. Follow the intuition, we then consider $P\CB{S_2^1 < S_1^2}$ which gives us (**memoryless property** will remove the effect of conditioning on $S_1^1 < S_1^2$)

$\bspace\d{P\CB{S_2^1 < S_1^2} = \P{P\CB{S_1^1 < S_1^2}}^2 = \P{\ffrac{\lambda_1}{\lambda_1 + \lambda_2}}^2}$

In fact, this reasoning shows that given *all* that has previously occurred, which is independent from the future, each event that occurs is going to be an event of $N_1\P t$ process with probability $\ffrac{\lambda_1}{\lambda_1 + \lambda_2}$ or an event of $N_2\P t$ process $w.p.$ $\ffrac{\lambda_2}{\lambda_1 + \lambda_2}$. Thus

$\bspace\d{P\CB{S_n^1 < S_m^2} = \sum_{k=n}^{n+m-1} \binom{n+m-1}{k} \P{\ffrac{\lambda_1}{\lambda_1 + \lambda_2}}^k\P{\ffrac{\lambda_2}{\lambda_1 + \lambda_2}}^{n+m-1-k}}$

### Conditional Distribution of the Arrival Times
Given the fact that *exactly one* event has occured by time $t$, what's the distribution of the time at which the event occurred? Owing to the statinary and independent increments, we guess it's uniformly distributed over $\SB{0,t}$. We now check that, and for $s\leq k$

$\bspace\begin{align}
P\CB{T_1< s\mid N\P t = 1} &= \ffrac{P\CB{T_1< s, N\P t = 1}}{P\CB{ N\P t = 1}}\\
&= \ffrac{P\CB{\text{one event in }[0,s),\text{zero event in }\SB{s,t}}}{P\CB{ N\P t = 1}}\\
&= \ffrac{P\CB{\text{one event in }[0,s)}\cdot P\CB{\text{zero event in }\SB{s,t}}}{P\CB{ N\P t = 1}}\\
&= \ffrac{e^{\lambda s}\ffrac{\lambda s}{1!} e^{\lambda \P{t-s}}\ffrac{\P{\lambda \P{t-s}}^0}{0!}}{e^{\lambda t}\ffrac{\lambda t}{1!}} = \ffrac{s}{t}
\end{align}$

$Remark$

>About the ***order statistics***: Let $Y_1,\dots,Y_n$ be $n$ $r.v.$. We say $Y_{\P{1}},\dots,Y_{\P n}$ are the **order statistics** corresponding to $Y_1,\dots,Y_n$ if $Y_{\P k}$ is the $k$th smallest value among $Y_1,\dots,Y_n$. Now if suppose further, that $Y_i$ are $i.i.d.$ continuous $r.v.$ with pdf $f$, we now have the the joint density of the **order statistics**.
>
>$\d{f\P{y_1,\dots,y_n} = n! \prod_{i=1}^n f\P{y_i},\bspace y_1<y_2<\cdots<y_n}$
>
>Especially, when all $Y_i$ are uniformly distributed over $\P{0,t}$, we have
>
>$\d{f\P{y_1,\dots,y_n} = \ffrac{n!}{t^n},\bspace 0< y_1<y_2<\cdots<y_n<t}$

With this we have the following Theorem

$Theorem.2$

Given that $N\P t = n$, the **arrival times** $S_1,S_2,\dots,S_n$ have the same distribution as the **order statistics** corresponding to $n$ independent $r.v.$ uniformly distributed on the interval $\P{0,t}$.

$Proof$

>Note that event $\P{s_1,s_2,\dots,s_n, N\P t = n}$ where $s_n$ marks the arrival time, is equivalent to $\P{T_1 = s_1,T_2 = s_2,\dots,T_n = s_n,T_{n+1}>t-s_n}$ where $T_n$ marks the interarrival time. *Since $T_n$ are $i.i.d.$ exponential $r.v.$ with rate $\lambda$*, we have
>
>$\begin{align}
f\P{s_1,s_2,\dots,s_n\mid N\P t = n} &= \ffrac{f\P{s_1,s_2,\dots,s_n,n}}{P\CB{N\P t = n}}\\
&= \ffrac{f\P{s_1,s_2,\dots,s_n,n}}{P\CB{N\P t = n}}\\
&= \ffrac{\lambda e^{-\lambda s_{\void_1}}\cdot\lambda e^{-\lambda \P{s_{\void_2} - s_{\void_1}}} \cdots \lambda e^{-\lambda \P{s_{\void_n} - s_{\void_{n-1}}\,}} \cdot e^{-\lambda \P{t-s_{\void_n}}}}{ e^{-\lambda t } \ffrac{ \P{\lambda t}^n}{n!}}\\
&= \ffrac{n!}{t^n},\bspace 0<s_1<s_2<\cdots<s_n < t
\end{align}$

$Proposition.3$

Assume Possion process $\CB{N\P t,t\geq 0}$ with rate $\lambda$ and $k$ possible sub types. Further, suppose that if an event occurs at time $y$ then it will be classified as a type $i$ event independently with probability $P_i\P y$, $i=1,\dots,k$ where $\sum_{i=1}^k P_i\P y = 1$.

If $N_i\P t$, $i=1,\dots,k$ represents the number of type $i$ events occuring by time $t$, then $N_i\P t$ are independent Poisson $r.v.$ with means

$\bspace\d{ \Exp\SB{N_i\P t} = \lambda \int_0^t P_i\P s \;\dd s}$

Whatever, skipped. **e.g.21** and **e.g.22** are very interesting, though.
***
$Proposition.1$

Given that $S_n = t$, he set $S_1, S_2, \dots, S_{n-1}$ has the distribution of a set of $n-1$ independent uniform $\P{0,t}$ $r.v.$.

$Proof$

> Other than using the same reasoning of how we prove the $Theorem 5.2$, here's a loose proof
>
>$\begin{align}
&\P{S_1,S_2\dots,S_{n-1} \mid S_n = t}\\
\sim\; & \P{S_1,S_2\dots,S_{n-1} \mid S_n = t, N\P{t^-} = n-1} \\
\sim\; & \P{S_1,S_2\dots,S_{n-1} \mid N\P{t^-} = n-1}
\end{align}$
>
>where $t^-$ is infinitesimally smaller than $t$ and $\sim$ means "has the same distribution as".

### Estimating Software Reliability

skipped

## Generalizations of the Poisson Process
### Nonhomogeneous Poisson Process

As is also known as ***nonstationary Poisson Process***, we obtain this by allowing the arrival rate at time $t$ to be a function of $t$, meaning $\lambda \P t$

$Def.3$

The counting process $\CB{N\P t, t \geq 0}$ is said to be a ***Nonhomogeneous Poisson Process*** with *intensity function* $\lambda \P t$, $t\geq 0$, if

1. $N\P 0 = 0$
2. $\CB{N\P t, t \geq 0}$ has independent increments
3. $P\CB{N\P{t+h} - N\P t \geq 2} = o\P h$
4. $P\CB{N\P{t+h} - N\P t = 1} = \lambda\P t \cdot h + o\P h$

We also define the ***mean value function*** of a **nonhomogeneous poisson process** as

$\bspace m\P t = \d\int_0^t \lambda \P y \;\dd y$

$Theorem.3$

If $\CB{N\P t, t \geq 0}$ is a **nonstationary Poisson process** with **intensity function** $\lambda \P t$, $t\geq 0$, then $N\P{t+s} - N\P{s}$ is a Poisson $r.v.$ with *mean* $m\P{t+s} - m\P s = \d\int_{s}^{t+s} \lambda \P y\;\dd y$.

$Proof$

>Mimick the proof of $Theorem5.1$, the stationary one, we have, by letting $g\P t=\Exp\SB{e^{-uN\P{t}}}$
>
>$\bspace g\P{t+h}=g\P{t} \Exp\SB{e^{-uN_t\P{h}}}$

>where $N_t\P{h} = N\P{t+h} - N\P{t}$. Since $P\CB{N\P{t+h} - N\P t = 0} = 1- \lambda\P t \cdot h + o\P h$, the preceding is
>
>$\bspace\begin{align}
g\P{t+h} &= g\P{t} \big(e^0\cdot \P{1- \lambda\P t \cdot h + o\P h} + e^{-u}\cdot \P{\lambda\P t \cdot h + o\P h} + e^{-2u}\cdot o\P h\big)\\
&= g\P t\P{1-\lambda \P t\cdot h + e^{-u} \lambda \P t\cdot h + o\P h}\\
\Rightarrow g\P{t+h} - g\P h &= g\P{t}\lambda \P t\P{e^{-u} -1}h + o\P h
\end{align}$
>
>Dividing by $h$ and letting $h\to0$ yields the differential equation $g'\P t = g\P t \lambda \P t \P{e^{-u} - 1}$. Then we write $\ffrac{g'\P t}{g\P t} = \lambda\P t\P{e^{-u} -1}$. Solve this we have
>
>$\bspace g\P t = \exp\CB{m\P t \P{e^{-u}-1}}$
>
>However, $\exp\CB{m\P t \P{e^{-u}-1}}$ is also the **Laplace transform** of a Poisson $r.v.$ with mean $m\P t$. We assert that $N\P t $ is also Poisson with mean $m\P t$. Then $N\P{t+s} - N\P{s} = N_s\P t$ is a nonstationary Poisson process with **intensity function** $\lambda_s \P t = \lambda \P{s+t}$, $t>0$. Consequently, $N_s\P t$ is Poisson with mean
>
>$\bspace\d{\int_0^t \lambda_s\P y \;\dd y = \int_0^t \lambda\P{s+y} \;\dd y = \int_s^{s+t} \lambda\P x \;\dd y}$

One special case is the time sampling of an ordinary Poisson process. Consider an ordinary Poisson process with rate $\lambda$, $\CB{N\P t, t\geq0}$ and suppose that an event occuring at time $t$ is independently of what has occurred prior to $t$. We count the event with probability $p\P t$ and let $N_c\P t$ denote the counted events by time $t$, thus, the counting process $\CB{N_c\P t, t\geq 0}$ is a nonhomogeneous Poisson process with **intensity function** $\lambda \P t = \lambda \cdot p\P t$. We verify this by noting that this process satisfies the four nonhomogeneous Poisson process axioms.

1. $N_c\P 0 = 0$
2. The number of *counted events* in $\P{s,s+t}$ depends solely on the number of *events* of the Poisson process in $\P{s,s+t}$, which is independent of what has occurred prior to time $s$. Consequently, the number of *counted events* in $\P{s,s+t}$ is independent of the process of *counted events* prior to $s$.
3. Let $N_c\P{t,t+h} = N_c\P{t+h} - N_c\P{t}$, with a similar definition of $N\P{t,t+h}$.$\\[0.6em]$
$\bspace P\CB{N_c\P{t,t+h} \geq 2} \leq P\CB{N\P{t,t+h} \geq 2} = o\P h\\[0.6em]$

4. Compute $P\CB{N_c\P{t,t+h} =1}$ by conditioning on $N\P{t,t+h}$.$\\[0.6em]$
$\bspace\begin{align}
P\CB{N_c\P{t,t+h} =1} &= P\CB{N_c\P{t,t+h} =1\mid N\P{t,t+h} = 1}\cdot P\CB{N\P{t,t+h} =1}\\
&\bspace + P\CB{N_c\P{t,t+h} =1\mid N\P{t,t+h} \geq 2}\cdot P\CB{N\P{t,t+h} \geq 2} \\
&= P\CB{N_c\P{t,t+h} =1\mid N\P{t,t+h} = 1} \cdot \lambda h + o\P h\\
&= p\P t \cdot \lambda h+ o\P h
\end{align}$

**e.g.24**

Arrivals in a hot dog stores from $8$ A.M. consitute a nonhomogeneous Poisson process with **intensity function** $\lambda\P t$ given by

$\bspace\lambda\P t = \begin{cases}
0,   & 0 \leq t 8\\
5+5\P{t-8},& 8\leq t \leq 11\\
20,&   11\leq t \leq 13\\
20-2\P{t-13},& 13\leq t \leq 17\\
0, & 17 < t \leq 24
\lambda\P{t-24},& t>24
\end{cases}$

>The number of arrivals between $8:30$ A.M. and $9:30$ A.M. will be a Poisson with mean $m\P{9.5} - m\P{8.5}$, so that the probability of this number being $0$ is
>
>$\bspace \d{ \exp\CB{-\int_{8.5}^{9.5} 5+5\P{t-8} \;\dd t} = e^{-10}}$
>
>And the mean number of arrivals is $ \d{\int_{8.5}^{9.5} 5+5\P{t-8} \;\dd t = 10}$

***

We can further suppose that an event at time $s$ is a type $1$ event $w.p.$ $P_1\P s$ or $2$ for $P_2\P{s} = 1 - P_1\P s$. Let $N_i\P t$, $t\geq0$ denote the number of type $i$ events by time $t$, then we have the similar result like stationary Poisson proecss that are

- $\CB{N_1\P t, t\geq0}$ and $\CB{N_2\P t, t\geq0}$ are independent nonhomogeneous Poisson process with respective **intensity function** $\lambda_i\P t = \lambda\P t P_i\P t = \lambda P_i\P t$, $i=1,2$.
- $N_1\P t$ and $N_2\P t$ are independent Poisson $r.v.$ with $\Exp\SB{N_i\P t} = \lambda \d{\int_0^t P_i\P s \;\dd s}$, $i=1,2$

If we let $S_n$ denote the time of the $n$th event of the nonhomogeneous Poisson process, then we can obtain its density as follows:

$\bspace\begin{align}
P\CB{t<S_n < t+h} &= P\CB{N\P t = n-1,\text{ one event in }\P{t,t+h}} + o\P h\\
&= e^{-m\P t} \ffrac{\P{m\P t}^{n-1}}{\P{n-1}!} \cdot P\CB{N\P{t+h} - N\P t = 1}+ o\P h\\
&= \lambda \P t e^{-m\P t} \ffrac{\P{m\P t}^{n-1}}{\P{n-1}!}+ o\P h
\end{align}$

which implies that $f_{S_n}\P t = \lambda \P t e^{-m\P t} \ffrac{\P{m\P t}^{n-1}}{\P{n-1}!}$, where $m\P t = \d\int_0^t \lambda \P s\;\dd s$

### Compound Poisson Process
A stochastic process $\CB{X\P t, t\geq 0}$ is said to be a ***compound Poisson process*** if it can be represented as $X\P t = \sum_{i=1}^{N\P t} Y_i$, $t\geq0$ where

- $\CB{N\P t,t\geq0}$ is a Poisson process
- $\CB{Y_i,i\geq1}$ is a family off $i.i.d.$ that is also independent of $\CB{N\P t,t\geq0}$

And the result in Chap_03 tells us that $\Exp\SB{X\P t} = \lambda t \Exp\SB{Y_1}$, $\Var{X\P t} = \lambda t\Exp\SB{Y_1^2}$.

A lot of contents are skipped and then it comes to another result, that if $\CB{X\P t, t\geq0}$ and $\CB{Y\P t, t\geq0}$ are independent compound Poisson processes with respective Poisson parameters and distributions $\lambda_1$, $F_1$ and $\lambda_2$, $F_2$, then $\CB{X\P t + Y\P t, t\geq0}$ is also a compound Poisson process, with Poisson parameter $\lambda_1 + \lambda_2$ and distribution function $F$ given by

$\bspace F\P x = \ffrac{\lambda_1}{\lambda_1 + \lambda_2}F_1\P x +\ffrac{\lambda_2}{\lambda_1 + \lambda_2}F_2\P x$

### Conditional or Mixed Poisson Processes

skipped

## Random Intensity Functions and Hawkes Processes
skipped

Probability and Statistics/Applied Random Process/Chap_05_The Exponential Distribution and the Poisson Process.ipynb


```python
cd ../src
```

    /home/sankha/Documents/programs/ML/densityNN/src

Most of the section here is obtained from this website: http://abdulfatir.com/Normalizing-Flows-Part-1/

Given a mapping $f: R^d \Rightarrow R^d$ such that $\mathbf{y} = f(\mathbf{z})$, the PDF of $\mathbf{y} = q_1(\mathbf{y})$ may be obtained from the pdf of $\mathbf{z} = q_0(\mathbf{z})$ as:

\begin{align}
q_1(\mathbf{y}) &= q_0(\mathbf{z}) \Big| \det \frac { \partial f ^ {-1} } {\partial \mathbf{y}}  \Big|  \\
                &= q_0(\mathbf{z}) \Big| \det \frac { \partial f  } {\partial \mathbf{y}}  \Big| ^ {-1} \\
\end{align}

I should work out the equations at another time.

# Planar Flow

The planar flow results from the equation:

$$f(\mathbf{z}) = \mathbf{z} + \mathbf{u}h( \mathbf{w}^T\mathbf{z} + b ) $$

where $\mathbf{u}, \mathbf{w} \in R^n, b \in R$, and $h$ is a nonlinear function such as `tanh`.

The Jacobian is given by:

$$ \frac { \partial f(\mathbf{z}) } { \partial \mathbf{z} } = \mathbf{I} + \mathbf{u}h'( \mathbf{w}^T\mathbf{z} + b )\mathbf{w}^T  $$

Using the matrix determinant Lemma,

$$ \det \frac { \partial f(\mathbf{z}) } { \partial \mathbf{z} } = 1 + \mathbf{u}h'( \mathbf{w}^T\mathbf{z} + b )\mathbf{w}^T\mathbf{u}  $$

We shall solve the following specific equation:

\begin{align}
q_0(\mathbf{z}) &= N(\mathbf{z} | \mathbf{0}, \mathbf{I} )   \\
\mathbf{w}     &= [5, 0]^T                                    \\
\mathbf{u}     &= [1, 0]^T                                    \\
b              &= 0                                          \\
h(\mathbf{x})  &= \tanh( \mathbf{x} )
\end{align}

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
pos = np.linspace(-5, 5, 40)
x, y = [], []
for i, p in enumerate(pos):
    t = np.linspace(-5, 5, 100)
    x.append(t)
    y.append(np.ones( t.shape )*p)
    y.append(t)
    x.append(np.ones( t.shape )*p)

xs = np.array(x)
ys = np.array(y)

plt.figure()
_, ax=plt.subplots(1,1)
for x,y in zip(xs, ys):
    plt.plot(x, y, color='black')
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

```python
z = np.vstack((xs.flatten(), ys.flatten()))

w = np.array([5, 0]).reshape((-1, 1))
u = np.array([1, 0]).reshape((-1, 1))
b = 0

f = z + u @ np.tanh( w.T @ z + b )
fx = f[0].reshape( xs.shape )
fy = f[1].reshape( ys.shape )

plt.figure()
_, ax=plt.subplots(1,1)
for x,y in zip(fx, fy):
    plt.plot(x, y, color='black')
plt.show()

```

Now we will be sampling directly form the Gaussian distribution.

```python
N = 10000000
z, t = np.random.random((N, 2)), np.random.random((N,))
z = (z - 0.5) * 8
pz = np.exp(-(z**2).sum(axis=1)/2)
mask = pz >= t
x, y = z[mask].T

# Plot the figure
_, ax=plt.subplots(1,1)
_ = plt.hist2d(zX, zY, bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

Now take these points and direectly shift them ...

```python
z = np.vstack((x.flatten(), y.flatten()))
f_comp = z + u @ np.tanh( w.T @ z + b )

_, ax=plt.subplots(1,1)
_ = plt.hist2d(f_comp[0], f_comp[1], bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.xlim([-4, 4])
plt.ylim([-3, 3])
plt.show()
plt.close()
```

Now, we should be able to obtain the same distribution of points if we sample from the derived distribution. First, we shall compute:

$$ \det \frac { \partial f(\mathbf{z}) } { \partial \mathbf{z} } = 1 + h'( \mathbf{w}^T\mathbf{z} + b )\mathbf{w}^T\mathbf{u}  $$

And then generate the density using the function

\begin{align}
q_1(\mathbf{y}) &= q_0(\mathbf{z}) \Big| \det \frac { \partial f ^ {-1} } {\partial \mathbf{y}}  \Big|  \\
                &= q_0(\mathbf{z}) \Big| \det \frac { \partial f  } {\partial \mathbf{y}}  \Big| ^ {-1} \\
\end{align}

```python
N = 1000000
z, t = np.random.random((N, 2)), np.random.random((N,))
z = (z - 0.5) * 14

```

```python
w.shape, z.shape
```

    ((2, 1), (1000000, 2))

```python

def dtanh(x):
    return 1-np.tanh(x)**2

pz = np.exp(-(z**2).sum(axis=1)/2)
detDf = 1 + dtanh(z @ w + b) @ w.T @ u
detDf = np.abs((1/detDf).flatten())
detDf = pz * detDf
mask = detDf >= t
x, y = z[mask].T

_, ax=plt.subplots(1,1)
_ = plt.hist2d(x, y, bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.xlim([-4, 4])
plt.ylim([-3, 3])
plt.show()
plt.close()
```

# Radial flow

Radial flows use the functional form:

$$ f( \mathbf{z} ) = \mathbf{z} + \beta h(\alpha, r)( \mathbf{z} - \mathbf{z}_0 )  $$

Here,

$\alpha \in R^+, \beta \in R, h(\alpha, r) = (\alpha + r)^{-1}$, and $r = || \mathbf{z} - \mathbf{z}_0 ||$

$$  \det \frac {\partial f(\mathbf{z})} {\partial \mathbf{z}} = (1 + \beta h(\alpha,r)+ \beta h'(\alpha,r) )(1 + \beta h(\alpha,r))^{d-1} $$

We shall be specifically be looking at the following conditions:

\begin{align}
q_0(\mathbf{z}) &= N(\mathbf{z}; \mathbf{0}, \mathbf{I})           \\
\mathbf{z}_0    &= [1, 0]^T                                        \\
\alpha          &= 2                                               \\
\beta           &= 5
\end{align}

```python
pos = np.linspace(-5, 5, 40)
x, y = [], []
for i, p in enumerate(pos):
    t = np.linspace(-5, 5, 100)
    x.append(t)
    y.append(np.ones( t.shape )*p)
    y.append(t)
    x.append(np.ones( t.shape )*p)

xs = np.array(x)
ys = np.array(y)

plt.figure()
_, ax=plt.subplots(1,1)
for x,y in zip(xs, ys):
    plt.plot(x, y, color='black')
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

```python
z = np.vstack((xs.flatten(), ys.flatten()))

alpha, beta = 2, 5
z0 = np.array([1, 0]).reshape((-1, 1))
r = np.sqrt(((z - z0)**2).sum(axis=0))
h = 1/(alpha + r)

f = z + beta * h * (z - z0)
fx = f[0].reshape( xs.shape )
fy = f[1].reshape( ys.shape )

plt.figure()
_, ax=plt.subplots(1,1)
for x,y in zip(fx, fy):
    plt.plot(x, y, color='black')
plt.show()
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

```python
z.shape, z0.shape
```

    ((2, 8000), (2, 1))

```python
N = 10000000
z, t = np.random.random((N, 2)), np.random.random((N,))
z = (z - 0.5) * 8
pz = np.exp(-(z**2).sum(axis=1)/2)
mask = pz >= t
x, y = z[mask].T

# Plot the figure
_, ax=plt.subplots(1,1)
_ = plt.hist2d(zX, zY, bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

```python
z = np.vstack((x.flatten(), y.flatten()))

alpha, beta = 2, 5
z0 = np.array([1, 0]).reshape((-1, 1))
r = np.sqrt(((z - z0)**2).sum(axis=0))
h = 1/(alpha + r)

f_comp = z + beta * h * (z - z0)

_, ax=plt.subplots(1,1)
_ = plt.hist2d(f_comp[0], f_comp[1], bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.show()
plt.close()
```

We shall try to figure out what the distribution is now, with the other equation ...

$$  \det \frac {\partial f(\mathbf{z})} {\partial \mathbf{z}} = (1 + \beta h(\alpha,r)+ \beta h'(\alpha,r) r)(1 + \beta h(\alpha,r))^{d-1} $$

```python
N = 50000000
z, t = np.random.random((N, 2)), np.random.random((N,))
z = (z - 0.5) * 14
```

```python
def dtanh(x):
    return 1-np.tanh(x)**2

d = z.shape[1]

alpha, beta = 2, 5
z0 = np.array([1, 0]).reshape((-1, 1))
r = np.sqrt(((z.T - z0)**2).sum(axis=0))
h = 1/(alpha + r)
hBar = -1/(alpha + r)**2

pz = np.exp(-(z**2).sum(axis=1)/2)

detDf = (1 + beta * h + beta * hBar * r) * (1 + beta * h)**(d-1)
detDf = np.abs((1/detDf).flatten())
detDf = pz * detDf
mask = detDf >= t
x, y = z[mask].T

_, ax=plt.subplots(1,1)
_ = plt.hist2d(x, y, bins=100, cmap=plt.cm.inferno)
ax.set_aspect('equal', 'box')
plt.xlim([-4, 4])
plt.ylim([-3, 3])
plt.show()
plt.close()
```

```python

```


# The Explicit Forward Time Centered Space (FTCS) Difference Equation for the Heat Equation

#### John S Butler john.s.butler@tudublin.ie   [Course Notes](https://johnsbutler.netlify.com/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf)    [Github](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)
## Overview
This notebook will implement the explicit Forward Time Centered Space (FTCS) Difference method for the Heat Equation.

## The Heat Equation
The Heat Equation is the first order in time ($t$) and second order in space ($x$) Partial Differential Equation [1-3]:
$$  \frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2},$$
The equation describes heat transfer on a domain
$$ \Omega = \{ t \geq 0\leq x \leq 1\}. $$
with an initial condition at time $t=0$ for all $x$ and boundary condition on the left ($x=0$) and right side ($x=1$).

## Forward Time Centered Space (FTCS) Difference method
This notebook will illustrate the Forward Time Centered Space (FTCS) Difference method for the Heat Equation with the __initial conditions__
$$ u(x,0)=2x, \ \ 0 \leq x \leq \frac{1}{2}, $$
$$ u(x,0)=2(1-x), \ \ \frac{1}{2}  \leq x \leq 1, $$
and __boundary condition__
$$ u(0,t)=0,  u(1,t)=0. $$

```python
# LIBRARY
# vector manipulation
import numpy as np
# math functions
import math

# THIS IS FOR PLOTTING
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import warnings
warnings.filterwarnings("ignore")
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>''')

```

<form action="javascript:code_toggle()"><input type="submit" value="Click here to toggle on/off the raw code."></form>

## Discete Grid
The region $\Omega$ is discretised into a uniform mesh $\Omega_h$. In the space $x$ direction into $N$ steps giving a stepsize of
$$ h=\frac{1-0}{N},$$
resulting in
$$x[i]=0+ih, \ \ \  i=0,1,...,N,$$
and into $N_t$ steps in the time $t$ direction giving a stepsize of
$$ k=\frac{1-0}{N_t}$$
resulting in
$$t[j]=0+jk, \ \ \ j=0,...,15.$$
The Figure below shows the discrete grid points for $N=10$ and $Nt=100$,  the known boundary conditions (green), initial conditions (blue) and the unknown values (red) of the Heat Equation.

```python
N=10
Nt=1000
h=1/N
k=1/Nt
r=k/(h*h)
time_steps=15
time=np.arange(0,(time_steps+.5)*k,k)
x=np.arange(0,1.0001,h)
X, Y = np.meshgrid(x, time)
fig = plt.figure()
plt.plot(X,Y,'ro');
plt.plot(x,0*x,'bo',label='Initial Condition');
plt.plot(np.ones(time_steps+1),time,'go',label='Boundary Condition');
plt.plot(x,0*x,'bo');
plt.plot(0*time,time,'go');
plt.xlim((-0.02,1.02))
plt.xlabel('x')
plt.ylabel('time (ms)')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.title(r'Discrete Grid $\Omega_h,$ h= %s, k=%s'%(h,k),fontsize=24,y=1.08)
plt.show();
```

## Discrete Initial and Boundary Conditions

The discrete initial conditions are
$$ w[i,0]=2x[i], \ \ 0 \leq x[i] \leq \frac{1}{2} $$
$$ w[i,0]=2(1-x[i]), \ \ \frac{1}{2}  \leq x[i] \leq 1 $$
and the discrete boundary conditions are
$$ w[0,j]=0,  w[10,j]=0, $$
where $w[i,j]$ is the numerical approximation of $U(x[i],t[j])$.

The Figure below plots values of $w[i,0]$ for the inital (blue) and boundary (red) conditions for $t[0]=0.$

```python
w=np.zeros((N+1,time_steps+1))
b=np.zeros(N-1)
# Initial Condition
for i in range (1,N):
    w[i,0]=2*x[i]
    if x[i]>0.5:
        w[i,0]=2*(1-x[i])

# Boundary Condition
for k in range (0,time_steps):
    w[0,k]=0
    w[N,k]=0

fig = plt.figure(figsize=(8,4))
plt.plot(x,w[:,0],'o:',label='Initial Condition')
plt.plot(x[[0,N]],w[[0,N],0],'go',label='Boundary Condition t[0]=0')
#plt.plot(x[N],w[N,0],'go')
plt.xlim([-0.1,1.1])
plt.ylim([-0.1,1.1])
plt.title('Intitial and Boundary Condition',fontsize=24)
plt.xlabel('x')
plt.ylabel('w')
plt.legend(loc='best')
plt.show()
```

## The Explicit Forward Time Centered Space (FTCS) Difference Equation
The explicit Forwards Time Centered Space (FTCS) difference equation of the Heat Equation
is derived by discretising
$$  \frac{\partial u_{ij}}{\partial t} = \frac{\partial^2 u_{ij}}{\partial x^2},$$
around $(x_i,t_{j})$ giving the difference equation
$$
\frac{w_{ij+1}-w_{ij}}{k}=\frac{w_{i+1j}-2w_{ij}+w_{i-1j}}{h^2},
$$
Rearranging the equation we get
$$
w_{ij+1}=rw_{i-1j}+(1-2r)w_{ij}+rw_{i+1j},
$$
for $i=1,...9$ where $r=\frac{k}{h^2}$.

This gives the formula for the unknown term $w_{ij+1}$ at the $(ij+1)$ mesh points
in terms of $x[i]$ along the jth time row.

Hence we can calculate the unknown pivotal values of $w$ along the first row of $j=1$ in terms of the known boundary conditions.

This can be written in matrix form
$$ \mathbf{w}_{j+1}=A\mathbf{w}_{j} +\mathbf{b}_{j} $$
for which $A$ is a $9\times9$ matrix:
$$
\left(\begin{array}{c}
w_{1j+1}\\
w_{2j+1}\\
w_{3j+1}\\
w_{4j+1}\\
w_{5j+1}\\
w_{6j+1}\\
w_{7j+1}\\
w_{8j+1}\\
w_{9j+1}\\
\end{array}\right).
=\left(\begin{array}{cccc cccc}
1-2r&r& 0&0&0 &0&0&0\\
r&1-2r&r&0&0&0 &0&0&0\\
0&r&1-2r &r&0&0& 0&0&0\\
0&0&r&1-2r &r&0&0& 0&0\\
0&0&0&r&1-2r &r&0&0& 0\\
0&0&0&0&r&1-2r &r&0&0\\
0&0&0&0&0&r&1-2r &r&0\\
0&0&0&0&0&0&r&1-2r&r\\
0&0&0&0&0&0&0&r&1-2r\\
\end{array}\right)
\left(\begin{array}{c}
w_{1j}\\
w_{2j}\\
w_{3j}\\
w_{4j}\\
w_{5j}\\
w_{6j}\\
w_{7j}\\
w_{8j}\\
w_{9j}\\
\end{array}\right)+
\left(\begin{array}{c}
rw_{0j}\\
0\\
0\\
0\\
0\\
0\\
0\\
0\\
rw_{10j}\\
\end{array}\right).
$$
It is assumed that the boundary values $w_{0j}$ and $w_{10j}$ are known for $j=1,2,...$, and $w_{i0}$ for $i=0,...,10$ is the initial condition.

The Figure below shows the values of the $9\times 9$  matrix in colour plot form for $r=\frac{k}{h^2}$.

```python
A=np.zeros((N-1,N-1))
for i in range (0,N-1):
    A[i,i]=1-2*r # DIAGONAL

for i in range (0,N-2):
    A[i+1,i]=r # UPPER DIAGONAL
    A[i,i+1]=r # LOWER DIAGONAL

fig = plt.figure(figsize=(6,4));
#plt.matshow(A);
plt.imshow(A,interpolation='none');
plt.xticks(np.arange(N-1), np.arange(1,N-0.9,1));
plt.yticks(np.arange(N-1), np.arange(1,N-0.9,1));
clb=plt.colorbar();
clb.set_label('Matrix elements values');
clb.set_clim((-1,1));
plt.title('Matrix r=%s'%(np.round(r,3)),fontsize=24)
fig.tight_layout()
plt.show();
```

## Results
To numerically approximate the solution at $t[1]$ the matrix equation becomes
$$ \mathbf{w}_{1}=A\mathbf{w}_{0} +\mathbf{b}_{0} $$
where all the right hand side is known.
To approximate solution at time $t[2]$ we use the matrix equation
$$ \mathbf{w}_{2}=A\mathbf{w}_{1} +\mathbf{b}_{1}. $$
Each set of numerical solutions $w[i,j]$ for all $i$ at the previous time step is used to approximate the solution $w[i,j+1]$.

The Figure below shows the numerical approximation $w[i,j]$ of the Heat Equation using the FTCS method at $x[i]$ for $i=0,...,10$ and time steps $t[j]$ for $j=1,...,15$. The left plot shows the numerical approximation $w[i,j]$ as a function of $x[i]$ with each color representing the different time steps $t[j]$. The right plot shows the numerical approximation $w[i,j]$ as colour plot as a function of $x[i]$, on the $x[i]$ axis and time $t[j]$ on the $y$ axis.

For $r>\frac{1}{2}$ the method is unstable resulting a solution that oscillates unnaturally between positive and negative values for each time step.

```python
fig = plt.figure(figsize=(12,6))

plt.subplot(121)
for j in range (1,time_steps+1):
    b[0]=r*w[0,j-1]
    b[N-2]=r*w[N,j-1]
    w[1:(N),j]=np.dot(A,w[1:(N),j-1])
    plt.plot(x,w[:,j],'o:',label='t[%s]=%s'%(j,time[j]))
plt.xlabel('x')
plt.ylabel('w')
#plt.legend(loc='bottom', bbox_to_anchor=(0.5, -0.1))
plt.legend(bbox_to_anchor=(-.4, 1), loc=2, borderaxespad=0.)

plt.subplot(122)
plt.imshow(w.transpose())
plt.xticks(np.arange(len(x)), x)
plt.yticks(np.arange(len(time)), time)
plt.xlabel('x')
plt.ylabel('time')
clb=plt.colorbar()
clb.set_label('Temperature (w)')
plt.suptitle('Numerical Solution of the  Heat Equation r=%s'%(np.round(r,3)),fontsize=24,y=1.08)
fig.tight_layout()
plt.show()
```

## Local Trunction Error
The local truncation error of the classical explicit difference approach to
\begin{equation}
\frac{\partial U}{\partial t} - \frac{\partial^2 U}{\partial x^2}=0,
\end{equation}
with

\begin{equation}
F_{ij}(w)=\frac{w_{ij+1}-w_{ij}}{k}-\frac{w_{i+1j}-2w_{ij}+w_{i-1j}}{h^2}=0,
\end{equation}
is
\begin{equation}
T_{ij}=F_{ij}(U)=\frac{U_{ij+1}-U_{ij}}{k}-\frac{U_{i+1j}-2U_{ij}+U_{i-1j}}{h^2},
\end{equation}
By Taylors expansions we have
\begin{eqnarray*}
U_{i+1j}&=&U((i+1)h,jk)=U(x_i+h,t_j)\\
&=&U_{ij}+h\left(\frac{\partial U}{\partial x} \right)_{ij}+\frac{h^2}{2}\left(\frac{\partial^2 U}{\partial x^2} \right)_{ij}+\frac{h^3}{6}\left(\frac{\partial^3 U}{\partial x^3} \right)_{ij} +...\\
U_{i-1j}&=&U((i-1)h,jk)=U(x_i-h,t_j)\\
&=&U_{ij}-h\left(\frac{\partial U}{\partial x} \right)_{ij}+\frac{h^2}{2}\left(\frac{\partial^2 U}{\partial x^2} \right)_{ij}-\frac{h^3}{6}\left(\frac{\partial^3 U}{\partial x^3} \right)_{ij} +...\\
U_{ij+1}&=&U(ih,(j+1)k)=U(x_i,t_j+k)\\
&=&U_{ij}+k\left(\frac{\partial U}{\partial t} \right)_{ij}+\frac{k^2}{2}\left(\frac{\partial^2 U}{\partial t^2} \right)_{ij}+\frac{k^3}{6}\left(\frac{\partial^3 U}{\partial t^3} \right)_{ij} +...
\end{eqnarray*}

substitution into the expression for $T_{ij}$ then gives

\begin{eqnarray*}
T_{ij}&=&\left(\frac{\partial U}{\partial t} - \frac{\partial^2 U}{\partial x^2} \right)_{ij}+\frac{k}{2}\left(\frac{\partial^2 U}{\partial t^2} \right)_{ij}
-\frac{h^2}{12}\left(\frac{\partial^4 U}{\partial x^4} \right)_{ij}\\
& &	+\frac{k^2}{6}\left(\frac{\partial^3 U}{\partial t^3} \right)_{ij}
-\frac{h^4}{360}\left(\frac{\partial^6 U}{\partial x^6} \right)_{ij}+ ...
\end{eqnarray*}
But $U$ is the solution to the differential equation so
\begin{equation} \left(\frac{\partial U}{\partial t} - \frac{\partial^2 U}{\partial x^2} \right)_{ij}=0,\end{equation}

the principal part of the local truncation error is

\begin{equation}
\frac{k}{2}\left(\frac{\partial^2 U}{\partial t^2} \right)_{ij}-\frac{h^2}{12}\left(\frac{\partial^4 U}{\partial x^4} \right)_{ij}.
\end{equation}

Hence the truncation error is
\begin{equation}
T_{ij}=O(k)+O(h^2).
\end{equation}

## Stability Analysis

To investigating the stability of the fully explicit FTCS difference method of the Heat Equation, we will use the von Neumann method.
The FTCS difference equation is:
$$\frac{1}{k}(w_{pq+1}-w_{pq})=\frac{1}{h_x^2}(w_{p-1q}-2w_{pq}+w_{p+1q}),$$
approximating
$$\frac{\partial U}{\partial t}=\frac{\partial^2 U}{\partial x^2}$$

at $(ph,qk)$. Substituting $w_{pq}=e^{i\beta x}\xi^{q}$ into the difference equation gives:
$$e^{i\beta ph}\xi^{q+1}-e^{i\beta ph}\xi^{q}=r\{e^{i\beta (p-1)h}\xi^{q}-2e^{i\beta ph}\xi^{q}+e^{i\beta (p+1)h}\xi^{q} \}
$$

where $r=\frac{k}{h_x^2}$. Divide across by $e^{i\beta (p)h}\xi^{q}$ leads to

$$ \xi-1=r(e^{i\beta (-1)h} -2+e^{i\beta h}), $$

$$\xi= 1+r (2\cos(\beta h)-2),$$

$$\xi=1-4r(\sin^2(\beta\frac{h}{2})).$$
Hence
$$\left| 1-4r(\sin^2(\beta\frac{h}{2}) )\right|\leq 1$$
for this to hold
$$ 4r(\sin^2(\beta\frac{h}{2}) )\leq 2 $$
which means
$$ r\leq \frac{1}{2}. $$
therefore the equation is conditionally stable as $0 < \xi \leq 1$ for $r<\frac{1}{2}$ and all $\beta$ .

## References
[1] G D Smith Numerical Solution of Partial Differential Equations: Finite Difference Method Oxford 1992

[2] Butler, J. (2019). John S Butler Numerical Methods for Differential Equations. [online] Maths.dit.ie. Available at: http://www.maths.dit.ie/~johnbutler/Teaching_NumericalMethods.html [Accessed 14 Mar. 2019].

[3] Wikipedia contributors. (2019, February 22). Heat equation. In Wikipedia, The Free Encyclopedia. Available at: https://en.wikipedia.org/w/index.php?title=Heat_equation&oldid=884580138 [Accessed 14 Mar. 2019].

```python

```

Chapter 08 - Heat Equations/.ipynb_checkpoints/01_Heat Equation- FTCS-checkpoint.ipynb


<div>

</div>

#**Artificial Intelligence - MSc**

##CS6462 - PROBABILISTIC AND EXPLAINABLE AI

##SEM2 2021/2

###CS6462_Lab_1.3

###Instructor: Enrique Naredo
Copyright (C) 2022 - All rights reserved, do not copy or distribute without permission of the author.

# Probabilistic Modelling

* A model is a mathematical description of a phenomenon.
* A probabilistic model is a probabilistic description of a phenomenon  that models our **uncertainty** about it.
* Mathematically, a probabilistic model is a (set of) probabilistic distribution(s)

You can express it
 *  either writing down its analytical formula
 *  or plotting its probability mass function

### Generative model
* A probabilistic model is a generative model.
* It means that you can use it to simulate a phenomenon (by *sampling* from the model)

What is a sample?
* A sample is an instance of the simulation

### Queries
Given a probabilistic model, we want to  query the model
* Will it rain tomorrow?
* Given that today temperature is 15°C, will it rain tomorrow?

We can answer this type of questions
* using probability calculus (marginalisation,  Bayes'rule, expectations)

However, we will see for the case of continuous variables that
* answering these questions requires solving difficult integrals

Instead, we will see that by
* *sampling the model*

we can get an approximate answer to our questions without the need of computing those integrals.

### What you will learn in this Notebook
1. to build a probabilistic model for a coin, dice, ...
2. to sample from these models
3. to answer queries either analytically or via sampling

You will also learn
4. what a Bernoulli and Categoirical distribution are
5. what a cumulative distribution is
6. How to compute expectations (mean, variance)
7. indpendence/dependence and conditional indpendence/dependence

The following subsections summarize the coin model we discussed during the lecture and introduce the Dice model.

## Model of a coin

Variables:
* bias of the coin $\theta$
* outcome of the coin toss $x$

Meaning of $\theta$: $\theta=0.5$ (fair coin), $\theta=1$ (two heads),  $\theta=0$ (two tails).

We denote with
* $x$ the outcome of the coin toss
* $dom(x) = \{1,0\}$

We use 1 for Heads and 0 for Tail.

### Probabilistic model
$x$ is a binary variable and, its distribution can in general be written as:

$$
\left\{\begin{array}{rcl}
p(x=1)&=&\theta\\
p(x=0)&=&1-\theta
\end{array}\right.
$$

or, in a line,

$$
p(x=\textsf{x})=\theta^{\textsf{x}}(1-\theta)^{1-\textsf{x}} ~~~~ \text{ for } ~~~~ \textsf{x}\in\{0,1\}
$$

This is called **Bernoulli distribution**

### Examples
For instance, for a fair coin ($\theta=0.5$),

$$
\left\{\begin{array}{ll}
p(x=1)&=&0.5\\
p(x=0)&=&0.5
\end{array}\right.
$$

Here an example of a tricked coin

$$
\left\{\begin{array}{ll}
p(x=1)&=&\frac{2}{3}\\
p(x=0)&=&\frac{1}{3}
\end{array}\right.
$$

## Dice
* x denotes the result of the dice thrown
* dom(x)=$\{1,2,3,4,5,6\}$
The probability mass functions
$$
\begin{array}{l}
p(x=1) = \theta_1\\
p(x=2) = \theta_2\\
p(x=3) = \theta_3\\
p(x=4) = \theta_4\\
p(x=5) = \theta_5\\
p(x=6) = \theta_6\\
\end{array}
$$
where $\theta_i\geq0$ is the probability that the outcome is the i-th face and, therefore, $\sum_{i=1}^6\theta_i=1$.

Example Fair Dice:

$$
\begin{array}{l}
p(x=1) = \frac{1}{6}\\
p(x=2) = \frac{1}{6}\\
p(x=3) = \frac{1}{6}\\
p(x=4) = \frac{1}{6}\\
p(x=5) = \frac{1}{6}\\
p(x=6) = \frac{1}{6}\\
\end{array}
$$

### Categorical distribution
We can encode the six possible outcomes as follows

$$
\begin{array}{rcl}
1 &\rightarrow& [1,0,0,0,0,0]\\
2 &\rightarrow& [0,1,0,0,0,0]\\
3 &\rightarrow& [0,0,1,0,0,0]\\
4 &\rightarrow& [0,0,0,1,0,0]\\
5 &\rightarrow& [0,0,0,0,1,0]\\
6 &\rightarrow& [0,0,0,0,0,1]\\
\end{array}
$$

then if we call the generic binary vector as $y$ and its $i$-th component as $y_i$, we can write

$$
p(x=y)=\theta_1^{y_1}\theta_2^{y_2}\theta_3^{y_3}\theta_4^{y_4}\theta_5^{y_5}\theta_6^{y_6}
$$

example $p(x=[0,1,0,0,0,0])=\theta_2$.

The above distribution is called **Categorical distribution** (we have seen it for the case of six categories (dice), but we can extend it to any generic number of categories).

## Expectation
Give a function $f(x)$ of the variable $x$, its expectation with respect the probability distribution
of $x$ is equal

$$
E[f(x)]=\sum_{\mathscr{x}\in dom(x)} f(\mathscr{x})p(x=\mathscr{x})
$$

If
* $f(x)=x$ (identity function) then $E[x]$ is the mean of $x$;
* $f(x)=(x-E[x])^2$ then $E[(x-E[x])^2]=Var(x)$ is called the variance of $x$;
* $f(x)=(x==a)$  then $E[(x==a)]$ is the probability of $x=a$;
* $f(x)=(x>=a)$  then $E[(x>=a)]$ is the probability of $x\geq a$;

Example Fair dice:
$$
\begin{aligned}
E[x]&=1\frac{1}{6}+2\frac{1}{6}+3\frac{1}{6}+4\frac{1}{6}+5\frac{1}{6}+6\frac{1}{6}=3.5\\
E[x^2]&=1\frac{1}{6}+4\frac{1}{6}+9\frac{1}{6}+16\frac{1}{6}+25\frac{1}{6}+36\frac{1}{6}=13.5\\
Var(x)=E[(x-3.5)^2]&=(1-3.5)^2\frac{1}{6}+(2-3.5)^2\frac{1}{6}+(3-3.5)^2\frac{1}{6}+(4-3.5)^2\frac{
1}{6}+(5-3.5)^2\frac{1}{6}+(6-3.5)^2\frac{1}{6}=\frac{35}{12}\\
E[x==1]&=1\frac{1}{6}+0\frac{1}{6}+0\frac{1}{6}+0\frac{1}{6}+0\frac{1}{6}+0\frac{1}{6}=\frac{1}{6}\\
E[x>=2]&=0\frac{1}{6}+1\frac{1}{6}+1\frac{1}{6}+1\frac{1}{6}+1\frac{1}{6}+1\frac{1}{6}=\frac{5}{6}\\
\end{aligned}
$$

## Statistics

How do we do statistical analysis?

###  How can we test if a coin is fair or not?
Assume we observe the following outcome:

$$
Observations=[1,1,1,1,1]
$$

is the coin fair?

###  Bag of coins model
For the moment, let us assume we have only three types of coins

1. $\theta_1=0.25$
2. $\theta_2=0.5$
3. $\theta_3=0.75$

and the number of coin of type 1,2 and 3 in the bag is the same.

We can build a model that describes the way the observations could be generated.

    A coin is extracted at random from a bag containing the coins of type 1, 2 and 3.
    The coin is then tossed five times.

###  Posterior

We want to compute the posterior probabilities

1. $p(\theta_1=0.25 ~|~ [1,1,1,1,1])$
2. $p(\theta_2=0.5~|~[1,1,1,1,1])$
3. $p(\theta_3=0.75~|~[1,1,1,1,1])$

and use these probabilities to answer the following queries:

*Is the coin fair?
*Is the coin biased towards Head?

```python
# SYMPY libray
from sympy.stats import Bernoulli, sample

θ = 0.5
C = Bernoulli("Coin", θ)
C
```

$\displaystyle Coin$

```python
# sympy.stats library
# https://docs.sympy.org/latest/modules/stats.html
from sympy.stats import density

def pmf(D):
    return (list(density(D).dict.keys()),list(density(D).dict.values()))
```

```python
print(pmf(C))
```

    ([0, 1], [0.500000000000000, 0.500000000000000])

```python
# matplotlib
# https://matplotlib.org/
import matplotlib.pyplot as plt
%matplotlib inline

plt.plot(pmf(C)[0], pmf(C)[1], 'o', ms=8)
```

```python
plt.vlines(pmf(C)[0], 0, pmf(C)[1], linestyles='-', lw=2 )
```

```python
print(pmf(C))
plt.plot(pmf(C)[0], pmf(C)[1], 'o', ms=8)
plt.vlines(pmf(C)[0], 0, pmf(C)[1], linestyles='-', lw=2 )
```