# Continuous-Time Markov Chains
## Introduction
***Pure birth process*** and ***birth and death model***.
## Continuous-Time Markov Chains

$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\bbspace}{\;\;\;\;\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\given}[1]{\left. #1 \right|}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N}\left( #1 \right)}
\newcommand{\Exp}[1]{\mathrm{E}\left[ #1 \right]}
\newcommand{\Var}[1]{\mathrm{Var}\left[ #1 \right]}
\newcommand{\Avar}[1]{\mathrm{Avar}\left[ #1 \right]}
\newcommand{\Cov}[1]{\mathrm{Cov}\left( #1 \right)}
\newcommand{\Corr}[1]{\mathrm{Corr}\left( #1 \right)}
\newcommand{\ExpH}{\mathrm{E}}
\newcommand{\VarH}{\mathrm{Var}}
\newcommand{\AVarH}{\mathrm{Avar}}
\newcommand{\CovH}{\mathrm{Cov}}
\newcommand{\CorrH}{\mathrm{Corr}}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\FSD}{\text{FSD}}
\newcommand{\SSD}{\text{SSD}}\CB{X\P t,t\geq 0}$ is a ***continuous-time Markov chain*** if $\forall\; s,t \geq 0$ and *nonnegative* integers $i,j,x\P u$, $0\leqq u < s$, we have

$\bspace P\CB{X\P{t+s} = j \mid X\P s = i,X\P u = x\P u,0\leq u<s} = P\CB{X\P{t+s} = j\mid X\P s = i}$

meaning that the conditional distribution of the future $X\P{t+s}$ given the present $X\P s$ and the past $X\P u$ for $0\leq u < s$, depends *only on the present* and is *independent of the past*. If $P\CB{X\P{t+s} = j\mid X\P s = i}$ is independent of $s$, the we say this **continuous-time Markov chain** have ***stationary*** or ***homogeneous transition probabilities***.

***

Another way to define this. Consider this as a stochastic process having the properties that each time it enters state $i$

1. the amount of time it spends in that state before making a transition into a different state is **exponentially distributed** with mean, say $1/v_i$
2. and when the process leaves state $i$, it next enters state $j$ with some probability, say, $P_{ij}$. Of course, the $P_{ij}$ must satisfy, $\forall\; i$

$\bspace\begin{align}
P_{ii} = 0&\\
\sum_j P_{ij} = 1&
\end{align}$

In addition, the amount of time the process spends in state $i$, $T_i$, should be independent of the time the next state visited. With this we have $P\CB{T_i > s+t\mid T_i > s} = P\CB{T_i > t}$, $\forall\; s,t\geq 0$. Hence, the random variable $T_i$ is memoryless and must thus be **exponenetially distributed**.

## Birth and Death Processes
A system, with states are represented by the number of people in the system at that time. Suppose that whenever there're $n$ people in the system,

1. new arrivals enter the system at an exponential rate $\lambda_n$
2. people leave the system at an exponential rate $\mu_n$

then we have the state transition rates and probabilities

$\bspace\begin{align}
v_0 &= \lambda_0,\\
v_i &= \lambda_i + \mu_i,\bspace i>0 \\
P_{01} &= 1,\\
P_{i,i+1} &= \ffrac{\lambda_i}{\lambda_i + \mu_i},\bspace i>0\\
P_{i,i-1} &= \ffrac{\mu_i}{\lambda_i + \mu_i},\bspace i>0
\end{align}$

Note that $v_i$ represents the rate of time that either a birth or a death occurs, which is exponentially distributed with rate $\lambda_i + \mu_i$

**e.g.2** Poisson Process

A birth and death process for which $\forall\; n \geq 0$, we have

$\bspace\begin{align}
\mu_n &= 0\\
\lambda_n &= \lambda
\end{align}\newcommand{\wp}{\text{with probability }}$

**e.g.3** Yule process

Consider a population whose members can only give birth to new members but cannot die, with each member acting independently and the time for each member giving birth exponentially distributed with rate $\lambda$. Let $X\P t$ be the population size at time $t$, then $\CB{X\P t,t\geq0}$ is a pure birth process with $\lambda_n = n\lambda$, $n\geq 0$.

**e.g.4** A Linear Growth Model with Immigration

$\bspace\begin{align}
\mu_n &= n\mu, &n\geq 1\\
\lambda_n &= n\lambda + \theta,& n\geq 0
\end{align}$

This is the ***linear growth process with immigration***. Suppose that $X\P 0 = i$ and let $M\P t = \Exp{X\P t}$. Now solve $M\P t$.

>$M\P{t+h} = \Exp{X\P{t+h}} = \Exp{\Exp{X\P{t+h}\mid X\P t}}$
>
>So given the population at time $t$, the population at time $t+h$ could be the following with certain probabilities.
>
>$\bspace X\P{t+h} = \begin{cases}
X\P t + 1,&\wp \P{\theta + X\P t \lambda}\cdot h + o\P h\\
X\P t - 1,&\wp \P{X\P t \mu} \cdot h + o\P h\\
X\P t,& \wp 1 - \P{\theta + X\P t \lambda + X\P t +\mu}\cdot h + o\P h
\end{cases}$
>
>Therefore, $\Exp{X\P{t+h}\mid X\P t} = X\P t + \P{\theta + X\P t \lambda - X\P t \mu}\cdot h + o\P h$ and taking the expectation yields $M\P{t+h} = M\P t + \P{\lambda - \mu}M\P t \cdot h + \theta \cdot h + o\P h$
>
>Letting $h\to0$ we have $M'\P t = \P{\lambda - \mu}M\P t + \theta$. Solve the equation, we have
>
>$\bspace M\P t = \ffrac{\theta}{\lambda - \mu} \P{e^{\P{\lambda - \mu}t} - 1} +ie^{\P{\lambda - \mu}t}$
>
>Finally by the fact that $M\P 0 = i$ we reach the final solution that
>
>$\bspace M\P t = \theta t + i$
***

**e.g.5** The Queueing System, $M/M/1$

Consider a single-server station. Customers arrives in accordance with a Poisson process having rate $\lambda$. Upon arrival, each customer goes directly into service if the server is free; if not, then the customer joins the queue. The successive service times are assumed to be independent exponential random variables having mean $1/\mu$ (or rate $\mu$).

The preceding is known as the $M/M/1$ queueing system. The first $M$ refers to the fact that the interarrival process is ***Markovian***, since it's a Poisson process and the second to the fact that the service distribution is **exponential**, hence, **Markovian**. The $1$ refers to the fact that there's only one server.

And we can write this system as a birth and death process with

$\bspace\begin{align}
\mu_n &= \mu,&n\geq 1\\
\lambda_n &= \lambda,& n\geq 0
\end{align}$
***

**e.g.6** A Multiserver Exponential Queueing System

Now there're $s$ servers and thus the birth and death process is with parameters

$\bspace\begin{align}
\mu_n &= \begin{cases}
n\mu,& 1\leq n \leq s\\
s\mu,& n > s\\
\end{cases}\\
\lambda_n &= \lambda, \bbspace n\geq 0
\end{align}$

And this is known as $M/M/s$ queueing model.
***

Now we implement a deeper discussion on the mean and variance of the process. Consider a general birth and death process with birth rates $\CB{\lambda_n}$ and death rates $\CB{\mu_n}$ where $\mu_0 = 0$. And let $T_i$ denote the time, starting from state $i$, it takes for the process to enter state $i+1$, $i\geq0$. Recursively, we can compute $\Exp{T_i}$, $i\geq 0$. Firstly

$\bspace \Exp{T_0 } = \ffrac{1}{\lambda_0}$

Then for $i>0$, we condition whether the first transition takes the process into state $i-1$ or $i+1$. Here's the indicator variable and the corresponding conditional expectation

$\bspace I_i = \begin{cases}
1,&\text{from state } i \text{ to state } i+1\\
0,&\text{from state } i \text{ to state } i-1\\
\end{cases} \bspace\Longrightarrow\bspace \left\{\begin{align}
\Exp{T_i \mid I_i = 1} &= \ffrac{1}{\lambda_i + \mu_i}\\
\Exp{T_i \mid I_i = 0} &= \ffrac{1}{\lambda_i + \mu_i} + \Exp{T_{i-1}} + \Exp{T_i}\\
\end{align}\right.\bspace\Longrightarrow$

$\bspace\begin{align}\Exp{T_i} &= \ffrac{1}{\lambda_i + \mu_i} \cdot \ffrac{\lambda_i}{\lambda_i + \mu_i} + \P{\ffrac{1}{\lambda_i + \mu_i} + \Exp{T_{i-1}} + \Exp{T_i}} \cdot \ffrac{\mu_i}{\lambda_i + \mu_i} \\
&= \ffrac{1}{\lambda_i + \mu_i} + \ffrac{\mu_i}{\lambda_i + \mu_i} \cdot\P{\Exp{T_{i-1}} + \Exp{T_i}}\\
&= \ffrac{1}{\lambda_i} + \ffrac{\mu_i}{\lambda_i}\cdot\Exp{T_{i-1}},\bspace i\geq 1
\end{align}$

Suppose now that we wanted to determine the expected time to go from state $i$ to state $j$ where $i<j$. Using the preceding, this quantity will equal to $\Exp{T_i} + \cdots + \Exp{T_{j-1}}$

**e.g.7**

Now the birth and death process with $\lambda_i \equiv \lambda$ and $\mu_i \equiv \mu$, we have

$\bspace\Exp{T_i} = \ffrac{1}{\lambda} + \ffrac{\mu}{\lambda} \Exp{T_{i-1}}\Rightarrow \Exp{T_i} = \ffrac{1}{\lambda} \P{1+\ffrac{\mu}{\lambda} + \P{\ffrac{\mu}{\lambda}}^2 + \cdots + \P{\ffrac{\mu}{\lambda}}^i} = \ffrac{ 1- \P{\mu/\lambda}^{i+1}}{\lambda - \mu},\bspace i\geq 0$

and the expected time to reach state $j$ from state $k$, for $k<j$, is

$\bspace \Exp{\text{time to go from state } i \text{ to }j} = \d{\sum_{i=k}^{j-1} \Exp{T_{i-1}}} = \ffrac{j-k}{\lambda- \mu}-\ffrac{\P{\mu/\lambda}^{k+1}}{\lambda - \mu} \ffrac{\P{1-\P{\mu/\lambda}^{j-k}}}{1-\mu/\lambda} $

Further if we assume that $\lambda = \mu$, $\Exp{T_i} = \ffrac{i+1}{\lambda}$ and the preceding degenerate to

$\bspace \Exp{\text{time to go from state } i \text{ to }j} = \ffrac{j\P{j+1} - k\P{k+1}}{2\lambda}$
***

Similarly, we have the variance of the time to go from state $0$ to $i+1$ using the similar reasoning. We first rewrite the conditional expectation of $T_i$ on $I_i$.

$\bspace \Exp{T_i \mid I_i}=\ffrac{1}{\lambda_i + \mu_i} + \P{1-I_i} \P{\Exp{T_{i-1}}+\Exp{T_i}} \Longrightarrow$

$\bspace \Var{\Exp{T_i \mid I_i}} = \P{\Exp{T_{i-1}}+\Exp{T_i}}^2 \cdot\Var{I_i} = \P{\Exp{T_{i-1}}+\Exp{T_i}}^2\ffrac{\mu_i\lambda_i}{\P{\mu_i + \lambda_i}^2}$

And if we let $X_i$ denote the time until the transition from $i$ occurs, then

$\bspace\begin{align}
\Var{T_i\mid I_i =1}&= \Var{X_i \mid I_i = 1} \\
&= \Var{X_i}  = \P{\ffrac{1}{\lambda_i + \mu_i}}^2 \\
\Var{T_i\mid I_i =0}&=  \Var{X_i + \text{time to get back to }i + \text{time to reenter }i+1}\\
&= \Var{X_i} + \Var{T_{i-1}} + \Var{T_i} \\[0.7em]
\Longrightarrow \Var{T_i\mid I_i} &= \Var{X_i} + \P{1-I_i} \P{\Var{T_{i-1}} + \Var{T_i}}\\
\Longrightarrow \Exp{\Var{T_i\mid I_i}} &= \P{\ffrac{1}{\lambda_i + \mu_i}}^2 + \ffrac{\mu_i}{\lambda_i + \mu_i} \P{\Var{T_{i-1}} + \Var{T_i}}\\
\Longrightarrow \Var{T_i} &= \Var{\Exp{T_i \mid I_i}} + \Exp{\Var{T_i\mid I_i}}\\
&= \P{\ffrac{1}{\lambda_i + \mu_i}}^2 + \ffrac{\mu_i}{\lambda_i + \mu_i}\P{\Var{T_{i-1}} + \Var{T_i}} + \ffrac{\mu_i\lambda_i}{\P{\mu_i + \lambda_i}^2}\P{\Exp{T_{i-1}}+\Exp{T_i}}^2
\end{align}$

or equivalently

$\bspace \Var{T_i} = \ffrac{1}{\lambda_i\P{\lambda_i + \mu_i}}+ \ffrac{\mu_i}{\lambda_i} \Var{T_{i-1}} + \ffrac{\mu_i}{\lambda_i +\mu_i} \P{\Exp{T_{i-1}} + \Exp{T_i}}^2$

Additionally, about the time to go from state $k$ to $j$, we have

$\bspace \Var{\text{time to go from $k$ to $j$}} = \d{\sum_{i=k}^{j-1} \Var{T_i}}$

## The Transition Probability Function $P_{ij}\P t$

First let $P_{ij}\P t = P\CB{X\P{t+s} = j \mid X\P s = i}$. Note that in this very expression, 时齐性与马氏性 both are contained. We can have a first glance on pure birth process. Let $X_k$ denote the time the process spends in state $k$ before making a transition into state $k+1$, $k\geq 1$. Then $\sum_{k=1}^{j-1} X_k$ is the time it takes until the process enters state $j$ given it currently in state $i$. And we have two equivalent events

$\bspace X\P t < j \iff X_i + X_{i+1} + \cdots + X_{j-1} > t$

Therefore, for $i<j$, if $\forall i \neq j, \lambda_i \neq \lambda_j $

$\bspace \begin{align}
P\CB{X\P t < j \mid X\P 0 = i} &= P\CB{\sum_{k=i}^{j-1} X_k > t}\\
&= \sum_{k=i}^{j-1}\P{ e^{-\lambda_k \cdot t} \prod_{r\neq k,r=i}^{j-1} \ffrac{\lambda_r}{\lambda_r - \lambda_k}}\\
P\CB{X\P t = j \mid X\P 0 = i} &= P\CB{X\P t < j+1 \mid X\P 0 = i} - P\CB{X\P t < j \mid X\P 0 = i} \\
&= \sum_{k=i}^{j}\P{ e^{-\lambda_k \cdot t} \prod_{r\neq k,r=i}^{j} \ffrac{\lambda_r}{\lambda_r - \lambda_k}} - \sum_{k=i}^{j-1} \P{e^{-\lambda_k \cdot t} \prod_{r\neq k,r=i}^{j-1} \ffrac{\lambda_r}{\lambda_r - \lambda_k}}\\
&= P_{ij}\P t \\
P_{ii}\P t &= P\CB{X_i > t} = e^{-\lambda_i \cdot t}
\end{align}$

***

For the general cases, considering the corresponding ***Chapman–Kolmogorov equations*** we have

$Lemma.3$

$\bspace\begin{align}
P_{ij}\P{t+s} &= P\CB{X\P{t+s} = j \mid X\P0 = i}\\
&= \sum_{k=0}^\infty P\CB{X\P{t+s} = j,X\P t = k \mid X\P0 = i}\\
&= \sum_{k=0}^\infty P\CB{X\P{t+s} = j \mid X\P0 = i,X\P t = k}\cdot P\CB{X\P t = k \mid X\P0 = i}\\
&= \sum_{k=0}^\infty P\CB{X\P{t+s} = j \mid X\P t = k}\cdot P\CB{X\P t = k \mid X\P0 = i}\\
&= \sum_{k=0}^\infty P_{kj}\P s \cdot P_{ik}\P t = \sum_{k=0}^\infty P_{ik}\P t P_{kj}\P s
\end{align}$

We then derive

$\bspace\begin{align}
P_{ij}'\P t &= \lim_{h \to 0} \ffrac{P_{ij}\P{t+h} - P_{ij}\P t}{h}= \lim_{h \to 0} \ffrac{ \P{\sum\limits_{k\neq i} P_{ik}\P h \cdot P_{kj} \P t} - \P{1-P_{ii}\P h} P_{ij}\P t}{h} \\
&= \lim_{h \to 0}\P{ \sum_{k\neq i} \ffrac{P_{ik}\P h}{h}P_{kj} \P t - \P{\ffrac{1-P_{ii}\P h}{h}}P_{ij}\P t }
\end{align}$

To find the limit, we assume that we can interchange the limit and the summation in the preceding, and obtain

$\bspace \d{P_{ij}'\P t = \sum_{k\neq i}\P{ \lim_{h \to 0} \ffrac{P_{ik}\P h}{h}}P_{kj} \P t - \lim_{h \to 0}\P{\ffrac{1-P_{ii}\P h}{h}}P_{ij}\P t}$

Now to find $\lim_{h \to 0} \ffrac{P_{ik}\P h}{h}$ and $\lim_{h \to 0}\ffrac{1-P_{ii}\P h}{h}$, we need to introduce $q_{ij}$, which is defined as (***different from the textbook!!! Frecking!!!***)

$\bspace q_{ij} = \begin{cases}
v_i P_{ij}, & j\neq i\\
-\sum\limits_{i\neq j} q_{ij} = -v_i,& j = i
\end{cases}, \bspace \d{\forall \: i, \sum_{j}q_{ij} = 0}$

Since $v_i$ is the rate at which the process makes a transition when in state $i$ and $P_{ij}$ (from $\mathbf P$) is the probability that this transition is into state $j$, it follows that $q_{ij}$ is the rate, when in state $i$, at which the process makes a transition into state $j$, called the ***instantaneous transition rates***.

After this we have

$Lemma.2$

>$\bspace\d{\lim_{h \to 0}}\ffrac{1-P_{ii}\P h}{h} = v_i,\bspace \d{\lim_{h \to 0}} \ffrac{P_{ij}\P h}{h} = q_{ij}, i\neq j$

$Proof$

>Skipped...

Then after so many efforts, we finally make it to

$Theorem.1$ Kolmogorov’s Backward Equations

>For all states $i$, $j$, and times $t\geq 0$,
>
>$\bspace \d{P_{ij}'\P t = \sum_{k\neq i} q_{ik}P_{kj}\P t - v_i P_{ij}\P t}$

$Remark$

>We also write $q_{ii} = -v_i = -q_i$ where $q_i = \abs{q_{ii}}>0$, thus, $P_{ii} = 0$ and $P_{ij} = \ffrac{q_{ij}}{q_{i}} = \ffrac{q_{ij}}{\sum_{j\neq i} q_{ij}}$

**e.g.9** Pure Birth Process

The backward equations for the pure birth process

>$\bspace\begin{align}
P_{ij}'\P{t} &= \sum_{k\neq i,i+1} 0\cdot P_{kj}\P t + q_{i,i+1} P_{i+1,j} \P t - v_i P_{ij}\P t\\
&= v_i \cdot P_{i,i+1}P_{i+1,j}\P t - \P{\lambda_i + 0} P_{ij}\P t\\
&= \lambda_i \ffrac{\lambda_i}{\lambda_i + 0}P_{i+1,j}\P t - \lambda_i P_{ij}\P t\\
&= \lambda_i P_{i+1,j}\P t - \lambda_i P_{ij}\P t
\end{align}$
***

**e.g.10** Birth and Death Process

The backward equations for the birth and death process

>$\bspace\begin{align}
P_{0j}'\P{t} &= v_0 P_{01}P_{1j}\P t - v_0 P_{0j}\P t = \lambda_0 P_{1j}\P t - \lambda_0 P_{0j} \P t = \lambda_0\P{P_{1j}\P t - P_{0j}\P t}\\
P_{1j}'\P t &= v_i\P{P_{i,i+1}P_{i+1,j}\P t + P_{i,i-1}P_{i-1,j}\P t} - v_i P_{ij}\P t\\
&= \P{\lambda_i + \mu_i}\P{\ffrac{\lambda_i}{\lambda_i + \mu_i} P_{i+1,j}\P t + \ffrac{\mu_i}{\lambda_i + \mu_i} P_{i-1,j}\P t} - \P{\lambda_i + \mu_i}P_{ij}\P t\\
&= \lambda_i P_{i+1,j}\P t + \mu_i P_{i-1,j}\P t - \P{\lambda_i + \mu_i}P_{ij}\P t,\bspace i > 0
\end{align}$
***

Another set of equations, similarly, are

$\bspace\begin{align}
P_{ij}'\P t &= \lim_{h \to 0} \ffrac{P_{ij}\P{t+h} - P_{ij}\P t}{h}= \lim_{h \to 0} \ffrac{ \P{\sum\limits_{k\neq j} P_{ik}\P t \cdot P_{kj} \P h} - \P{1-P_{jj}\P h} P_{ij}\P t}{h} \\
&= \lim_{h \to 0}\P{ \sum_{k\neq j} P_{ik} \P t \ffrac{P_{kj}\P h}{h} - \P{\ffrac{1-P_{jj}\P h}{h}}P_{ij}\P t }\\
\end{align}$

Still we assume that we can interchange limit with summation, we obtain, from $Lemma.2$

$Theorem.2$ Kolmogorov’s Forward Equations

>Under suitable regularity conditions, say, models with finite states, or all birth and death processes,
>
>$\bspace \d{P_{ij}'\P t = \sum_{k\neq j} P_{ik}\P t q_{kj}- v_j P_{ij}\P t}$

For the pure birth process, the forward equation is written as

$\bspace \begin{align}
P_{ij}'\P t &= \sum_{k\neq j,j-1} P_{ik}\P t q_{kj} + P_{i,j-1}\P t q_{j-1,j} - v_j P_{ij}\P t\\
&= 0 + P_{i,j-1}\P t \cdot \lambda_{j-1} \ffrac{\lambda_{j-1}}{\lambda_{j-1} + 0} - \lambda_j P_{ij}\P t \\
&= \lambda_{j-1} P_{i,j-1}\P t - \lambda_j P_{ij}\P t\\
&=\begin{cases}
-\lambda_i P_{ii}\P t,& j=i\\
\lambda_{j-1} P_{i,j-1}\P t - \lambda_j P_{ij}\P t, & j\geq i+1
\end{cases}
\end{align}$

Then we solve the ODE and obtain

$Proposition.4$ Pure Birth Process

>$\bspace \begin{align}
P_{ii}\P t &= e^{-\lambda_i t}, && i \geq 0\\
P_{ij}\P t &= \lambda_{j-1} e^{-\lambda_j t}\int_0^t e^{-\lambda_j s}P_{i,j-1} \P s \;\dd s, && j \geq i+1
\end{align}$

$Proof$

>Just keep in mind that $P_{ii}\P 0 = 1$ and $P_{ij}\P 0 = 0$ and it can't be more obvious

**e.g.12** Birth and Death Process

The forward equation now is

>$\bspace\begin{align}
P_{0j}'\P{t} &= \sum_{k\neq 0} P_{ik}\P t q_{k0} - v_0 P_{i0}\P t\\
&= \sum_{k\neq 0,1} P_{ik}\P t q_{k0} + P_{i1}\P t q_{10} - \lambda_0 P_{i0}\P t\\
&= 0 + P_{i1}\P t v_1 P_{10} - \lambda_0 P_{i0}\P t\\
&= P_{i1}\P t \P{\lambda_1 + \mu_1}\ffrac{\mu_1}{\lambda_1 + \mu_1}- \lambda_0 P_{i0}\P t\\
&= \mu_1 P_{i1}\P t- \lambda_0 P_{i0}\P t\\
P_{1j}'\P t &= P_{i,j-1}\P t q_{j-1,j} + P_{i,j+1}\P t q_{j+1,j} - v_j P_{ij}\P t\\
&= P_{i,j-1}\P t \P{\lambda_{j-1} + \mu_{j-1}}\ffrac{\lambda_{j-1}}{\lambda_{j-1} + \mu_{j-1}} + P_{i,j+1}\P t \P{\lambda_{j+1} + \mu_{j+1}} \ffrac{\mu_{j+1}}{\lambda_{j+1} + \mu_{j+1}} - \P{\lambda_j + \mu_j}P_{ij}\P t\\
&= \lambda_{j-1}P_{i,j-1}\P t + \mu_{j+1}P_{i,j+1}\P t  \P{\lambda_j + \mu_j}P_{ij}\P t
\end{align}$

## Limiting Probabilities
In analogy with discrete-time Markov chains, the probability that a continuous-time Markov chain will be in state $j$ at time $t$ often converges to a limiting value that is independent of the initial state.

$\bspace P_j \equiv \d{\lim_{t\to\infty} P_{ij}\P t}$

where we are assuming that the limit *exists* and is *independent of the initial state* $i$.

To find $P_{j}$, consider the **forward equations**.

$\bspace P_{ij}'\P t = \d{\sum_{k \neq j} P_{ik}\P t q_{kj} - v_j P_{ij}\P t \Longrightarrow \lim_{t\to\infty} P_{ij}'\P t = \sum_{k \neq j} P_k \cdot q_{kj} - v_j \cdot P_j}$

for all states $i$ and $\sum_j P_j = 1$ will do the problem. One last thing is to note that $\d{\lim_{t\to\infty} P_{ij}'\P t = 0}$, since if not, then with $t\to \infty$, $P_{ij}\P t$ will move to either $+\infty$ or $-\infty$. So in short, we have

$\bspace \begin{cases}
\sum\limits_{k \neq j} P_k \cdot q_{kj} - v_j \cdot P_j = 0, & \forall\: j\\
\sum\limits_j P_j = 1
\end{cases}$

$Remark$

>**backword equations** won't do, since
>
>$\bspace P_{ij}'\P t = \d{\sum_{k \neq i} q_{ik} P_{kj}\P t - v_i P_{ij}\P t \Longrightarrow \lim_{t\to\infty} P_{ij}'\P t = \sum_{k \neq i} q_{ik} P_j - v_i \cdot P_j =P_j\sum_k q_{ik} = P_j \cdot 0= 0}$
>
>And the two sufficient conditions for the existence $P_j$ are
>
>- ***irreducible***: all states of the Markov chain ***communicate***, meaning that there's positive probability of starting in state $i$ and ever being in state $j$, for all $i$, $j$. Or in mathematical form: *Let $S$ be the state space*. $\forall \: i,j\in S$, $\exists\: t > 0$, $s.t.$ $P_{ij}\P t > 0$. And if not, the two states are in different classes.
>- ***positive recurrrent***: *starting in any state, the mean time to return to that state is finite*
>
>Unfortunately to decide whether a continuous-time Markov chain is **positive recurrent** is not a easy job. Write $\mathbf Q = \P{q_{ij}}$ and still we have $-q_{ii} = v_i = q_i$. If $\sup q_i < \infty$ and $\inf q_i > 0$, whether state $i$ is **positive recurrent** can be determined exactly the same way as if we're dealing with a discrete-time Markov Chain. Later we'll show that **time reversible** $\Rightarrow$ **positive recurrent**: $\d{\lim_{t\to\infty} P_{ii}\P t > 0}$
>
>And more about **recurrent**. In discrete-time Markov Chain, beinig **recurrent** means $f_{ii} = \sum_{n=1}^\infty f_{ii}^\P{n} = 1 \iff \sum_{n=1}^\infty P_{ii}^\P{n} = \infty$. Now in continuous-time Markov Chain, we change the summation to integral, $\d{\int_{0}^\infty P_{ii}\P t\;\dd t = \infty}$.
>
>**Irreducibility** is an easier job now. First we update some notation. $\mathbf Q = \P{q_{ij}}$, $\mathbf P\P t = \P{P_{ij}\P{t}}$, and the **jump chain** or the **embedded chain** (ref [Time Reversibility](#Time-Reversibility)), $\mathbf P = \P{P_{ij}}$.
>
>And from $\mathbf Q$ we can easily write $\mathbf P$ like
>
>$\bspace \mathbf Q = \begin{pmatrix}
-v_1 & q_{12} & q_{13}\\
q_{21} & -v_2 & q_{23}\\
q_{31} & q_{32} & -v_3
\end{pmatrix} \iff \mathbf P = \begin{Vmatrix}
0 & \ffrac{q_{12}}{v_1} & \ffrac{q_{13}}{v_1}\\
\ffrac{q_{21}}{v_2} & 0 & \ffrac{q_{23}}{v_2}\\
\ffrac{q_{31}}{v_3} & \ffrac{q_{32}}{v_3} & 0
\end{Vmatrix}$
>
>Since it's too hard to derive $\mathbf P\P t = \P{P_{ij}\P{t}}$, although you can do so that you can directly see whether such $t$ exists so that $P_{ij}\P t > 0$, we suggest handling $\mathbf P$ as if it were a transition probabilities matrix from a discrete-time Markov Chain.
>
>And since through working on $\mathbf P$ we can decide whether the Markov Chain is **irreducible**, and $\mathbf P$ is equivalent to $\mathbf Q$, could we obtain the conclusion by only working on $\mathbf Q$. **YES YOU CAN!**
>
>*Our object is to find whether there's an $n$ such that $q_{ij}^\P{n} > 0$, where $q_{ij}^\P{n}$ have the similar definitions like $\d{P_{ij}^{n} = \sum_{k=0}^\infty P_{ik}^{m} P_{kj}^{n-m}}$*.

$Remark$

>Up to this point, back to solving the equation to obtain the **limiting probabilities**, remember we **CANNOT** first obtain $\mathbf P$ and solve $\begin{cases}
\pi = \pi \mathbf P\\
\pi \cdot \mathbf 1 = 1
\end{cases}$, whose solution is the **limiting distribution** of the **jump chain**...
>
>So, anyway, if we also define $\pi_i = \d{\lim_{t \to \infty} P_{ii}\P t}$, the equation could be simplied as

>$\bspace\begin{cases}
\sum\limits_{k \neq j} P_k \cdot q_{kj} - v_j \cdot P_j = \pi \mathbf Q = 0, & \forall\: j\\
\sum\limits_j \pi_j =\pi\cdot \mathbf 1 =  1
\end{cases}$
>
>And more **importantly**, **this equation set has a solution** $\iff$ **the limiting probabilities or the stationary probabilities exist**.
>
>And if it exists, we call this Markov Chain ***ergodic***.
>
>And we do this because it's hard to find the limiting probabilities in **Continuous Markov Chain** while at that time, **limiting probabilities** $\Leftrightarrow$ **stationary probabilities**. Thus we let the **stationary probabilities** be a row vector $\pi$ with $\pi = \pi P\P{t}$.
>
>And this requires that $\d{\lim_{t\to 0} \dfrac{P\P t - I}{t} = P'\P 0 \equiv Q}$ or, $\d{\lim_{t\to 0} \dfrac{P_{ij}\P t - \delta_{ij}}{t} = q_{ij}}$.
>
>Here $Q$, is a matrix with zero row sum, negative diagram entries and all other positive probabilities. Also we see that $P\P 0 = I$, $i.e.$, $P_{ij}\P0 = \delta_{ij}$

$Remark$

>$\sum\limits_{k \neq j} P_k \cdot q_{kj} - v_j \cdot P_j =0$, this equation (also known as the ***balance equations***) has one more intepretation:
>
>- $v_j \cdot P_j$ is the *rate at which the process **leaves** state $j$*: When the process is in state $j$, it leaves at rate $v_j$; and $P_j$ is the *proportion of time it is in state $j$*
>- $\sum\limits_{k \neq j} P_k \cdot q_{kj}$ is the *rate at which the process **enters** state $j$*: for all states named $k$ other than $j$, the *process enters $j$ at a rate $q_{kj}$*; and $P_k$ is the *proportion of time it is in state $k$*

Let us now determine the limiting probabilities for a birth and death process. For each state, from $0$ to $n\geq1$, we have a table

$$\begin{array}{cc}\hline
\text{State} & \text{Rate at which leave} = \text{rate at which enter}\\ \hline
0 & \lambda_0 P_0 = \mu_1 P_1 \\
1 & \P{\lambda_1+\mu_1}P_1 = \mu_2P_2 + \lambda_0 P_0 \\
2 & \P{\lambda_2+\mu_2}P_2 = \mu_3P_3 + \lambda_1 P_1 \\
n,n\geq 1 & \P{\lambda_n+\mu_n}P_n = \mu_{n-1}P_{n-1} + \lambda_{n+1} P_{n+1}\\ \hline
\end{array}
$$

Adding these equations together, we have $\forall\: n\geq 0$, $\lambda_n P_n = \mu_{n+1}P_{n+1}$. Solving these inductively, we have, in terms of $P_0$

$\bspace\begin{align}
P_1 &= \ffrac{\lambda_0}{\mu_1}P_0 \\
P_2 &= \ffrac{\lambda_1}{\mu_2}P_1 = \ffrac{\lambda_1\lambda_0}{\mu_2\mu_1}P_0 \\
P_3 &= \ffrac{\lambda_2}{\mu_3}P_2 = \ffrac{\lambda_2\lambda_1\lambda_0}{\mu_3\mu_2\mu_1}P_0 \\
&\vdots\\
P_n &= \ffrac{\lambda_{n-1}}{\mu_n}P_{n-1} = \ffrac{\lambda_{n-1}\lambda_{n-2}\cdots\lambda_2\lambda_1\lambda_0}{\mu_n\mu_{n-1}\cdots\mu_2\mu_1}P_0
\end{align}$

And by using the fact that $\sum_{n=0}^\infty P_n = 1 $, we can solve for $P_0$ that

$\bspace P_0 = \ffrac{1}{1 + \sum\limits_{n=1}^{\infty}\ffrac{\lambda_{n-1}\lambda_{n-2}\cdots\lambda_2\lambda_1\lambda_0}{\mu_n\mu_{n-1}\cdots\mu_2\mu_1}} \Rightarrow P_n = \ffrac{\lambda_0 \lambda_1 \cdots \lambda_{n-1}}{\mu_1 \mu_2\cdots \mu_n\P{1 + \sum\limits_{n=1}^{\infty}\ffrac{\lambda_{n-1}\lambda_{n-2}\cdots\lambda_2\lambda_1\lambda_0}{\mu_n\mu_{n-1}\cdots\mu_2\mu_1}}},\bspace n\geq1$

The foregoing equations also show us to the *necessary* condition that the limiting probabilities exist:

$\bspace \d{\sum\limits_{n=1}^{\infty}\ffrac{\lambda_{n-1}\lambda_{n-2}\cdots\lambda_2\lambda_1\lambda_0}{\mu_n\mu_{n-1}\cdots\mu_2\mu_1} < \infty}$

(and this condition is also *sufficient*.)

And in **e.g.6**, the multiserver exponential queueing system, we have the condition reduced to

$\bspace \d{\sum_{n=s+1}^\infty \ffrac{\lambda^n}{\P{s\mu}^n}<\infty \iff \ffrac{\lambda}{s\mu}<1}$

In **e.g.4**, the linear growth model with immigration, the condition is reduced to

$\bspace \d{\sum_{n=1}^\infty \ffrac{\theta\P{\theta+\lambda}\cdots\P{\theta+\P{n-1}\lambda}}{n!\mu^n}<\infty \iff \lim_{n\to\infty} \ffrac{\ffrac{\theta\P{\theta+\lambda}\cdots\P{\theta+n\lambda}}{\P{n+1}!\mu^{n+1}}}{\ffrac{\theta\P{\theta+\lambda}\cdots\P{\theta+\P{n-1}\lambda}}{n!\mu^n}} = \lim_{n\to\infty}\ffrac{\theta+n\lambda}{\P{n+1}\mu} = \ffrac{\lambda}{\mu}<1}$

$Remark$

>The previous method to find the limit of a summationis called the ***ratio test***.

**e.g.13** Machine Repair Model

There're $M$ machines and $1$ serviceman. Suppose that the amount of time each machine runs(survives) before breaking down is *exponentially distributed with mean* $\ffrac{1}{\lambda}$, and suppose that the amount of time that it takes for the serviceman to fix a machine is exponentially distributed with mean $\ffrac{1}{\mu}$. Show time!

>Usually, we'll focus on those that are **NOT** in use. We say the system is in state $n$ whenever $n$ machines are not in use, then the preceding is a *birth and death process* having parameters
>
>$\bspace\begin{align}
\mu_n &= \mu,\bbspace\bbspace\:\!\; n\geq 1\\
\lambda_n &= \begin{cases}
\P{M-n}\lambda, & n\leq M\\
0, & n>M
\end{cases}
\end{align}$
>
>where a *failing* machine is regarded as an *arrival* and a *fixed* machine as a *departure*. With this we have the $P_0$
>
>$\bspace\begin{align}
P_0 &= \ffrac{1}{1+\sum\limits_{n=1}^M\P{M\lambda \P{M-1}\lambda \cdots \P{M-n+1}\lambda \cdot \mu^{-n}}} = \ffrac{1}{1+\sum\limits_{n=1}^M \P{\ffrac{M!}{\P{M-n}!}\P{\ffrac{\lambda}{\mu}}^n}}\\
P_n &= \ffrac{\ffrac{M!}{\P{M-n}!}\P{\ffrac{\lambda}{\mu}}^n}{1+\sum\limits_{n=1}^M \P{\ffrac{M!}{\P{M-n}!}\P{\ffrac{\lambda}{\mu}}^n}},\bspace n=0,1,\dots,M
\end{align}$
>
>We then find the average number of machines not in use

>$\bspace \d{\sum_{n=0}^MnP_n = \ffrac{\sum\limits_{n=0}^M\P{n\cdot\ffrac{M!}{\P{M-n}!}\P{\ffrac{\lambda}{\mu}}^n}}{1+\sum\limits_{n=1}^M \P{\ffrac{M!}{\P{M-n}!}\P{\ffrac{\lambda}{\mu}}^n}}}$
>
>And the long-run proportion of time that a given machine is working, which is equivalent to the limiting probability of the machine working
>
>$\bspace\begin{align}
P\CB{\text{machine } i\text{ is working}} &= \sum_{n=0}^M P\CB{\text{machine } i\text{ is working} \mid \text{state }n}\cdot P_n\\
&= \sum_{n=0}^M P\CB{\text{machine } i\text{ is not one of the }n\text{ broken machines}}\cdot P_n\\
&= \sum_{n=0}^M \ffrac{M-n}{M}P_n = 1 - \sum_{n=0}^M\ffrac{nP_n}{M}
\end{align}$

## Time Reversibility

For an **ergodic** **continuous-time** Markov chain, with limiting probablities $P_i$, ignoring the amount of time spent in each state, then this sequence constitutes a **discrete-time** Markov chain with transition probabilities $P_{ij}$, called ***jump chain*** or ***embedded chain***. This is also **ergodic** and we denote its limiting probabilities as $\pi_i$. That is, $\pi_i$ is the unique solution of

$\bspace \left\{\begin{align}
\pi_i &= \sum\nolimits_j \pi_j P_{ji}, & \forall\: i\\
\sum\nolimits_i \pi_i &= 1
\end{align}\right. \iff  $$\left\{\begin{align}
\pi &= \pi \mathbf P\\
\pi \cdot \mathbf 1 &= 1
\end{align}\right.$

Thus, intuitively, $P_i = \ffrac{\pi_i/v_i}{\sum\limits_j\pi_j/v_j}$. Alternatively (or to check the intuition<span style="font-size:50px;">😆</span>), since for the limiting probabilities, the equality $\forall\: i$, $v_i P_i = \sum\limits_{j\neq i} P_j \, q_{ji}$ must hold, or equivalently since $P_{ii} = 0$, $v_i P_i = \sum\limits_{j} P_j v_j P_{ji}$, after plug it in,

$\bspace \pi_i = \d{\sum_j \pi_j P_{ji}},\bspace \forall\: i$

***

Now consider the reversed process. Suppose now that the **continuous-time** Markov chain has been in operation for a long time and we trace the process going back from time $T$. First note that

$\bspace\begin{align}
P&\CB{\text{process is in state }i\text{throughout }\SB{t-s,t}\mid X\P t = i}\\
&= \ffrac{\text{process is in state }i\text{throughout }\SB{t-s,t}}{P\CB{X\P t = i}}\\
&= \ffrac{P\CB{X\P{t-s} = i}e^{-v_i\cdot s}}{P\CB{X\P t = i}}\\
&\using{\text{large }t}e^{-v_i\cdot s}
\end{align}$

In other words, *going backward in time, the amount of time the process spends in state $i$ is also exponentially distributed with rate $v_i$*. And our conclusion is that

$\bspace$*The continuous-time Markov chain will be **time reversible**, in the sense that the process reversed in time has the same probabilistic structure as the original process, if the **embedded chain** is **time reversible***. That is:

$\bspace\forall\: i,j\bspace \pi_i P_{ij} = \pi_j P_{ji}$

And further using the fact that $P_i = \ffrac{\pi_i/v_i}{\sum\limits_j\pi_j/v_j}$, the preceding condition is equivalent to

$\bspace\forall\: i,j\bspace P_i \cdot q_{ij} = P_j \cdot q_{ji}$

which has the interpretation: *the rate at which the process goes directly from state $i$ to state $j$ is equal to the rate at which it goes directly from $j$ to $i$*.

$Remark$

>**Time reversible** $\Rightarrow$ **POSITIVE recurrent** (From Instructor GAO Wujun)

$Proposition.5$

>An ergodic birth and death process is time reversible.

Recall that a process is time reversible $iff$

$\bspace\forall\: i,j\bspace P_i \cdot q_{ij} = P_j \cdot q_{ji}$

Thus, if, as lucky as we are, able to find a set of numbers $P_i$ such that the foregoing is satisfied, then the Markov chain is **time reversible** and the $P_i$s are the **long-run probabilities**. That is,

$Proposition.7$

>If for some set $\CB{P_i}$, $\d{\sum_i P_i = 1}$, $P_i \geq 0$ and $\forall\: i\neq j$, $P_i \cdot q_{ij} = P_j \cdot q_{ji}$, then the continuous-time Markov chain is **time reversible** and $P_i$ represents the **limiting probability** of being in state $i$.

$Proof$

>For fixed $i$ we obtain upon summing $P_i \cdot q_{ij} = P_j \cdot q_{ji}$ over all $j \neq i$
>
>$\bspace \d{\sum_{j\neq i}P_i \cdot q_{ij} = \sum_{j\neq i}P_j \cdot q_{ji}}$
>
>since $\sum_{j\neq i} q_{ij} = v_i$, the preceding is just equivalent to
>
>$\bspace \d{v_i P_i = \sum_{j\neq i}P_j \cdot q_{ji}}$
>
>That's just the **balance equations**! Thus, $P_i$s are the **limiting probabilities**.

Consider a **continuous-time** Markov chain with state space $S$, the ***truncated*** to the set $A \subset S$ one is obtained by further define $q_{ij} = 0$, $\forall\: i\in A$, $j\notin A$. That is:

$\bspace$*Transitions out of the class $A$ are no longer allowed, whereas ones in $A$ continue at the same rates as before*. A useful result is that: *if the chain is **time reversible**, then so is the **truncated** one.

$Proposition.8$

>A time reversible chain with limiting probabilities $P_j$, $j\in S$ that is truncated to the set $A\in S$ and remains *irreducible*, is also time reversible and has limiting probabilities $P_j^A$ given by
>
>$\bspace P_j^A = \ffrac{P_j}{\sum\limits_{i\in A}P_i},\bspace j\in A$

$Proof$

>By $Proposition\,6.7$, we need to show that, with $P_j^A = \ffrac{P_j}{\sum\limits_{i\in A}P_i}$,
>
>$\bspace P_i^A q_{ij} = P_j^A q_{ji},\bspace $for $i\in A$, $j\in A$
>
>Or equivalently, $P_i^A q_{ij} = P_j^A q_{ji}$ for $i\in A$, $j\in A$. But this follows since the original chain is, by assumption, time reversible.

$Proposition.9$

>If $\CB{X_i\P t, t\geq 0}$ are, for $i=1,\dots,n$, *independent* **time reversible** **continuous-time** Markov chains, then the vector process $\CB{\P{X_i\P t,\dots,X_n\P t},t\geq 0}$ is also a **time reversible continuous-time** Markov chain.

## The Reversed Chain
Skipped
## Uniformization
Skipped
## Computing the Transition Probabilities
Now the backward equations is rewritten as (here we use a slightly different notation from the textbook...)

$\d{\bspace P_{ij}'\P t = \sum_{k\neq i}q_{ik}P_{kj}\P t - v_i P_{ij}\P t = \sum_{k} q_{ik}P_{kj}\P t } \Rightarrow \mathbf P'\P t= \mathbf Q \mathbf P\P t$

and similarly the forward ond, $\mathbf P'\P t= \mathbf P\P t \mathbf Q$. We then solve the equation and obtain

$\bbspace\mathbf P\P t = \mathbf P\P 0 \cdot e^{\mathbf Q \cdot t}$

Since $\mathbf P\P 0 =\mathbf I$, this yields that $\mathbf P\P t = e^{\mathbf Q \cdot t} = \sum\limits_{n=0}^\infty \mathbf Q^n\ffrac{t^n}{n!}$

And that's not for practical uses, so we need to introduce two approximation method:

- $\d{e^{\mathbf R t} = \lim_{n\to\infty} \P{\mathbf I + \mathbf R\ffrac{t}{n}}^n }$
- for large $n$, $\d{e^{-\mathbf R t} = \lim_{n\to\infty} \P{\mathbf I - \mathbf R\ffrac{t}{n}}^n}\Rightarrow \mathbf P\P t = \P{\mathbf I - \mathbf R\ffrac{t}{n}}^{-n}$

***

Probability and Statistics/Applied Random Process/Chap_06_Continuous-Time Markov Chains.ipynb


```python
#mapping some polygons -
import pandas as pd
import geopandas as gpd
from shapely.geometry.polygon import Polygon
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.pyplot as plt
from matplotlib import cm
import folium
import numpy as np
%matplotlib inline
```

```python
#USE ONCE
# Lat = np.linspace(36.01,40.01,200)
# Lon = np.linspace(-84.01, -75.01,200)
# vapoly = {'Lat':Lat,'Lon':Lon}
# vapolydf = pd.DataFrame(vapoly)
# vapolydf.head()
# vapolydf.to_csv(r'vapoly.csv')
```

```python
#dataframes for csv files - Buffers
dfbuffer = pd.read_csv('bufferlatlongV2.csv', delimiter = ',').astype(float)
dfhurdat = pd.read_csv('hurdatcleanva.csv', delimiter = ',')
dfcoast = pd.read_csv('coastlatlongV2.csv', delimiter = ',').astype(float)
dfbasin = pd.read_csv('jamesbasinlatlongV2.csv', delimiter = ',').astype(float)
dfva = pd.read_csv('vapoly.csv',delimiter = ',').astype(float)

```

```python
dfhurdat.head()
```

      Storm Number
      Storm Name
      Storm Status
      Lat
      Lon
      Time
      Maxspeed

      0
      AL041851
      UNNAMED
      TS
      36.8
      -75.1
      1851-08-25 18:00:00
      40.0

      1
      AL031854
      UNNAMED
      TS
      36.8
      -75.9
      1854-09-10 06:00:00
      40.0

      2
      AL031856
      UNNAMED
      TS
      37.0
      -76.0
      1856-08-20 00:00:00
      50.0

      3
      AL031856
      UNNAMED
      TS
      38.0
      -75.3
      1856-08-20 06:00:00
      50.0

      4
      AL021857
      UNNAMED
      TS
      36.3
      -75.8
      1857-09-14 06:00:00
      50.0

```python
dfbuffer.head()
```

      Lat
      Lon

      0
      37.3
      -75.4

      1
      37.3
      -75.3

      2
      37.3
      -75.3

      3
      37.0
      -75.5

      4
      36.4
      -75.3

```python
# # import Point, Polygon
# from sympy import Point, Polygon, Line
# #isolate the storm and get a list of points.
# #dfhurdat.head()
# points = dfhurdat[dfhurdat['Storm Number'] == "AL031856"]
# datachop = points[['Lat','Lon']].copy().values #conversion to an array
# #create a line object
# # using intersection()
# isIntersection = poly1.intersection(Line(p1, Point(3, 2)))

```

```python
import scipy as sp
import scipy.interpolate
```

```python
import numpy as np
lat = dfhurdat[['Lat']].copy().values
latmin = min(lat)
latmax = max(lat)
lon = dfhurdat[['Lon']].copy().values
lonmin = min(lon)
lonmax = max(lon)
length = 100000
# londf = pd.DataFrame(lat)
# latmin = min(latdf['Lat'])
# latmax = max(latdf['Lat'])
# lonmin = min(londf['Lat'])
# lonmax = max(londf['Lat'])
upsample_lat = np.linspace(latmin,latmax,length)
#print(upsample_lat)
upsample_lon = np.linspace(lonmin,lonmax,length)
#print(upsample_lon)
df1 = pd.DataFrame(upsample_lat)
df2 = pd.DataFrame(upsample_lon)
# Place the DataFrames side by side
dfupsample = pd.concat([df1, df2], axis=1)
dfupsample.columns = ['Lat', 'Lon']
dfupsample
len(dfupsample)
```

    100000

```python
#create the polygons
#polygon to reach beyond state a little
#listPoint = [[13.415449261665342, 52.502674590782519],[13.416039347648621, 52.50250152147968],[13.415787220001221, 52.501845158120446],[13.416162729263306, 52.502201097675766],[13.415406346321104, 52.502334982450677],[13.415111303329468,52.50204435400651]]
#polygon = {'type': 'Polygon', 'coordinates': [listPoint]}

buffer_geom =Polygon(zip(dfbuffer['Lon'],dfbuffer['Lat']))
basin_geom = Polygon(zip(dfbasin['Lon'],dfbasin['Lat']))
crs = {'init' : 'epsg:4326'}
bufferpoly = gpd.GeoDataFrame(index = [0], crs = crs, geometry = [buffer_geom])
bufferpoly.to_file(filename = 'buffer.geojson', driver = 'GeoJSON')
basinpoly = gpd.GeoDataFrame(index = [0], crs = crs, geometry = [basin_geom])
basinpoly.to_file(filename = 'basin.geojson', driver = 'GeoJSON')
```

    /Users/williampc/opt/anaconda3/envs/geop/lib/python3.9/site-packages/pyproj/crs/crs.py:53: FutureWarning: '+init=<authority>:' syntax is deprecated. '<authority>:' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6
      return _prepare_from_string(" ".join(pjargs))

```python
### for use with original data from nhc ###
# import pandas as pd
# from datetime import datetime
# def lat_lon_to_float (v):
#     """Convert strings from NHC to float locations"""
#     if (v[-1] == 'S') or (v[-1] == 'W'):
#         multiplier = -1
#     else:
#         multiplier = 1
#     return float(v[:-1])*multiplier
```

```python
### for use with original data from nhc ###
# hurdata = []
# with open ('hurdat2.txt', 'r') as f:
#     for line in f.readlines():
#         if line.startswith('AL'):
#             storm_id = line.split(',')
#             storm_number = storm_id[0].strip()
#             storm_name = storm_id[1].strip()
#         else:
#             location_line = line.split(',')
#             dt = datetime.strptime(location_line[0] + location_line[1],"%Y%m%d %H%M")
#             storm_status = location_line[3].strip()
#             storm_lat = lat_lon_to_float(location_line[4].strip())
#             storm_lon = lat_lon_to_float(location_line[5].strip())
#             max_speed = float(location_line[6].strip())
#             hurdata.append([storm_number,storm_name,storm_status,storm_lat,storm_lon,dt,max_speed])
#df = pd.DataFrame(hurdata, columns = ['Storm Number','Storm Name', 'Storm Status', 'Lat', 'Lon','Time', 'Max Speed'])

```

```python
dfhurdat.head()

```

      Storm Number
      Storm Name
      Storm Status
      Lat
      Lon
      Time
      Maxspeed

      0
      AL041851
      UNNAMED
      TS
      36.8
      -75.1
      1851-08-25 18:00:00
      40.0

      1
      AL031854
      UNNAMED
      TS
      36.8
      -75.9
      1854-09-10 06:00:00
      40.0

      2
      AL031856
      UNNAMED
      TS
      37.0
      -76.0
      1856-08-20 00:00:00
      50.0

      3
      AL031856
      UNNAMED
      TS
      38.0
      -75.3
      1856-08-20 06:00:00
      50.0

      4
      AL021857
      UNNAMED
      TS
      36.3
      -75.8
      1857-09-14 06:00:00
      50.0

```python
datachop = dfhurdat[['Lat','Lon']].copy().values
dfhd = pd.DataFrame(datachop).astype(float)
dfhd.columns = ['Lat', 'Lon']
dfhd.values.tolist()
print(dfhd)
```

          Lat   Lon
    0    36.8 -75.1
    1    36.8 -75.9
    2    37.0 -76.0
    3    38.0 -75.3
    4    36.3 -75.8
    ..    ...   ...
    285  37.8 -82.0
    286  38.8 -82.0
    287  39.5 -80.5
    288  36.5 -77.7
    289  37.0 -76.7

    [290 rows x 2 columns]

```python
# # #changing to a GeoDataFrame to create geometry series
# hurdatgdf = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.Lon,df.Lat))
# # hurdatgdf.head()
# # len(hurdatgdf)
# T = pd.merge(dfbuffer, dfhurdat, how='inner', on=['Lat', 'Lon'])
# print(T)
# len(T)
# T
#dfinbuffer = pd.merge(dfhurdat,dfupsample,on=['Lat','Lon'])
#dfinbuffer = pd.merge(left=dfhurdat, right=dfupsample, left_on='Lat', right_on='Lon')
#dfinbuffer = pd.DatFrame.merge(left = dfhurdat, right = dfupsample, left_on = ['Lat','Lon'], right_on = ['Lat','Lon'], how = 'right')

# isIntersection = bufferpoly.intersection(Line(dfhd))
#dfinbuffer = pd.merge(dfupsample.astype(float), dfva.astype(float), on=["Lat","Lon"])
#pd.set_option("display.max_rows", None, "display.max_columns", None)
#print(dfinbuffer)
# len(dfinbuffer)
# print(dfinbuffer)
```

```python
from sympy import Point, Polygon, Line
from sympy import Point, Polygon
import pandas as pd
import geopandas as gpd
from shapely.geometry import *
```

```python
# total map with all storms and buffers
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature
```

```python
plot_crs = ccrs.LambertConformal(central_longitude =-100., central_latitude = 45)
data_crs = ccrs.PlateCarree()
```

```python
import matplotlib.patches as mpatches
import matplotlib.pyplot as plt
import shapely.geometry as sgeom
import cartopy.crs as ccrs
import cartopy.io.shapereader as shpreader

def basincoords():
    """
    Return a list of latitudes and a list of longitudes (lons, lats)
    for James River Basin

    """
    basinlon = dfbasin['Lon']
    basinlat = dfbasin['Lat']

    return basinlon, basinlat

def buffercoords():
    """
    Return a list of latitudes and a list of longitudes (lons, lats)
    for James River Basin

    """
    bufferlon = dfbuffer['Lon']
    bufferlat = dfbuffer['Lat']

    return bufferlon, bufferlat
bufferlon, bufferlat = buffercoords()
basinlon, basinlat = basincoords()

#ax.set_title('James River Basin and Buffer - Virginia, USA')

# turn the lons and lats into a shapely LineString
buffer = sgeom.LineString(zip(bufferlon, bufferlat))
basin = sgeom.LineString(zip(basinlon, basinlat))

fig = plt.figure(figsize = (7,7))
ax = plt.subplot(1,1,1,projection = plot_crs)

ax.set_extent([-85,-70,32,40],data_crs)
ax.coastlines('50m', edgecolor = 'k', linewidth = 0.75)
ax.add_feature(cfeature.STATES, linewidth = 0.5)
ax.add_feature(cfeature.RIVERS, linewidth = 0.85)
ax.add_feature(cfeature.OCEAN)
ax.add_geometries([buffer], ccrs.PlateCarree(),facecolor='#C8A2C8', alpha=0.5)
ax.add_geometries([basin], ccrs.PlateCarree(),facecolor='rgb', edgecolor='k')
ax.plot(dfupsample['Lon'], dfupsample['Lat'],transform = data_crs)

#for storm_number in vahurc['Storm Number'].unique():
    #data = df[jameshurc['Storm Number'] == storm_number]
    #print(vahurc)
ax.plot(dfupsample['Lon'], dfupsample['Lat'],dfbuffer['Lon'],dfbuffer['Lat'],dfbasin['Lat'],dfbasin['Lon'], transform = data_crs)
```

```python

```

```python

```

```python

```


## Background

### Nitrogen fertilizer recommendations
The concept behind the **Economic Optimum Nitrogen Rate** approach (also referred to as the **Maximum Return to Nitrogen** approach) is to make the most favorable nitrogen fertilizer recommendation considering three variables:

* Grain price (\$ per kg)
* Fertilizer cost (\$ per kg)
* Grain yield response to nitrogen fertilizer (modeled from input data)

#### Why *nitrogen* and not other nutrients?
Many crops, (e.g., corn, wheat, potatoes) depend on nitrogen fertilizer to achieve profitable yields. Although the same is true for other nutrients (e.g., phosphorus, potassium, sulfur, etc.), many would probably agree that nitrogen deserves special attention because it is usually required in high quantities, it can cause environmental harm, and it tends to be more difficult to manage because of its elusive behavior in the soil. A diagram describing the fundamental pathways among the different pools in the nitrogen cycle illustrates its complexity (**Figure 1**). When nitrogen is introduced to agricultural soils, either naturally or synthetically, it is immediately vulnerable to be lost. Some losses may be temporary, while others are permanent. When nitrogen is lost, it can contribute to environmental pollution and cause other damages related to health and quality of living. Nitrogen pollution can be present in both water (including groundwater, lakes, rivers, etc.) and in the air (as particulate matter or as nitrous oxides, which are strong greenhouse gases).

**Figure 1**: The nitrogen cycle

**With so many potential loss pathways, it is no wonder why farmers care so much about the nitrogen they apply to their crops. The** `EONR` **tool is an attempt to support them, through economics-driven research.**

### EONR? MRTN? ..what?
As its name suggests, the **Maximum Return to Nitrogen** (MRTN) is the maximum achievable profit expected after accounting for the cost of nitrogen fertilizer. The economic optimum nitrogen rate (EONR) is the nitrogen rate where the MRTN is reached. In statistical terms, the EONR is the point where the monetary return from yield is equal to the cost of the increase in nitrogen fertilizer input. The approach uses a best-fit statistical model that utilizes input data from replicated field trials with several nitrogen rate treatments. **Figure 2** is a typical nitrogen rate response curve for corn.

**Figure 2:** Plot generated from the `EONR` package. The blue points are observed experimental data (nitrogen rate vs yield return to nitrogen). Yield is expressed as monetary return, and is simply the grain price multiplied by grain yield. The blue line is the best-fit quadratic-plateau model representing gross return to nitrogen, the red line is the cost of nitrogen fertilizer, and the green line is the difference between the two and represents the net return to nitrogen.

A model is fit to the experimental data to describe the return to nitrogen for *each* particular dataset (e.g., quadratic, quadratic-plateau, etc.). Differences among replications from these trials can contribute to uncertainty in estimating the EONR, and this is illustrated by the rather wide confidence intervals on either side of the 162 kg ha$^{-1}$ estimated EONR value (90% confidence bounds are 130 kg ha$^{-1}$ and 208$^{-1}$). Although there is perhaps a lot of uncertainty in the EONR estimate, it is a good baseline for making nitrogen fertilizer recommendations for a given location or soil.

To generate an EONR/MRTN plot as in **Figure 2**, it is necessary to conduct a nitrogen response field experiment (**Figure 3**). Experiments can be "small-plot", where it is more feasible to include more nitrogen rates and replications, or experiments can be "strip trials", where experiments are generally easier to establish.

**Figure 3:** Aerial photo of a nitrogen rate response experiment for corn (photo captured in July when the crop is about shoulder-high).

Nitrogen deficiencies are visible in the small-plot experiment in **Figure 3** (plot boundaries were rendered over the photo to visualize the individual plots more easily). The experiment in this photo had nine (9) nitrogen rates applied at three (3) different times during the season, and was replicated four (4) times.

### The quadratic-plateau model
The quadratic-plateau model is often identified as the most appropriate model for nitrogen response in corn. It is a piecewise function that can be described as:

\begin{equation}
y_i =\begin{cases}\beta_0 + \beta_1x_i + \beta_2x_i^2 + \epsilon_i & if\ x_i < \frac{-\beta_1}{2\beta_2}
\\{\beta_0} - \frac{{\beta_1^2}}{4{\beta_2}} + \epsilon_i & if\ x_i \geq \frac{-\beta_1}{2\beta_2}\end{cases}
\label{qp}
\end{equation}

where $y_{i}$ can represent grain yield (kg ha$^{-1}$) or monetary return (\\$ ha$^{-1}$) from grain yield, $x_{i}$ represents quantity of nitrogen fertilizer applied, and $\beta_{0}$, $\beta_{1}$, and $\beta_{2}$ are the coefficients estimated from the experimental data assuming identically and independently distributed errors, $\epsilon_{i}$.

The point on the x-axis where the net return curve (green) reaches the maximum return is the **EONR/MRTN**. The profile-likelihood 90\% CIs are illustrated as a transparent grey box surrounding the EONR/MRTN point.

### The quadratic model
The quadratic model may be desireable if grain yield reaches a maximum, then begins to decline with increasing rates of nitrogen fertilizer (i.e., it exhibits a sort of toxicity effect). Compared to the quadratic-plateau model, the estimated ONR generally tends to be greater when using the quadratic model. It can be described as:

\begin{equation}
y_i =\beta_0 + \beta_1x_i + \beta_2x_i^2 + \epsilon_i
\label{q}
\end{equation}

where $y_{i}$ can represent grain yield (kg ha$^{-1}$) or monetary return (\\$ ha$^{-1}$) from grain yield, $x_{i}$ represents quantity of nitrogen fertilizer applied, and $\beta_{0}$, $\beta_{1}$, and $\beta_{2}$ are the coefficients estimated from the experimental data assuming identically and independently distributed errors, $\epsilon_{i}$.

Quadratic models tend to be most popular in the literature, at least for describing how to calculate confidence intervals (CIs) for the EONR ([Hernandez & Mulla, 2008](https://dl.sciencesocieties.org/publications/aj/abstracts/100/5/1221); [Sela et al., 2017](https://dl.sciencesocieties.org/publications/jeq/abstracts/46/2/311)). However, even in cases where a quadratic model happens to fit the observed data best, [Cerrato & Blackmer (1990)](https://www.agronomy.org/publications/aj/abstracts/82/1/AJ0820010138) imply that the idea of using the quadratic model for maize is absurd because it predicts rapid decreases in yields when fertilizer is applied at higher than optimal rates, a trend that is not generally supported by evidence for maize. Furthermore, the quadratic model produces a systematic bias of overestimated maximum grain yield and optimum nitrogen fertilizer rate [(Bullock & Bullock, 1994)](https://www.agronomy.org/publications/aj/abstracts/86/1/AJ0860010191).

### Confidence intervals
One of the major novelties of the `EONR` package is that it calculates profile-likelihood CIs (as well as Wald and bootstrap CIs) from data fit by the model.

From a scientific perspective, it is widely recognized that large uncertainties exist around the estimated EONR computed from yield data and that it is essential to report CIs. Still, few examples exist in the agronomic literature where CIs are actually estimated ([Bachmaier & Gandorfer, 2009](https://www.researchgate.net/publication/225680100_A_conceptual_framework_for_judging_the_precision_agriculture_hypothesis_with_regard_to_site-specific_nitrogen_application); [Hernandez & Mulla, 2008](https://dl.sciencesocieties.org/publications/aj/abstracts/100/5/1221); [Jaynes, 2011](https://link.springer.com/article/10.1007/s11119-010-9168-3); [Sela et al., 2017](https://dl.sciencesocieties.org/publications/jeq/abstracts/46/2/311); [Qin et al., 2018](https://dl.sciencesocieties.org/publications/aj/articles/110/6/2596). Of these examples, only Jaynes (2011) calculated CIs for the quadratic-plateau response function (which has generally been recognized as the most appropriate model for describing yield response to nitrogen in corn). [Hernandez & Mulla, 2008](https://dl.sciencesocieties.org/publications/aj/abstracts/100/5/1221) describe three general methods that can be used for estimating CIs about the EONR:

* the profile-likelihood based CI
* a bootstrap-derived CI
* the Wald CI

#### Profile-likelihood
The profile-likelihood is the most accurate of any of the approaches (requires reparameterization). The profile-likelihood confidence intervals are computed according to the following general steps:

1. Fit a model (e.g., quadratic-plateau, quadratic, etc.) to the observed data and calculate the sum of squared errors (refered to as the sum of squared errors of the "full" model, $SSE(\hat{\theta_{2}})$).
2. Calculate the $SSE$ of the model subject to the constraint that $\theta_{2}$ (the parameter representing the optimum nitrogen rate) has a fixed value (refered to as the sum of squared errors of the "reduced" model, $SSE(\tilde{\theta_{2}})$).
3. The likelihood ratio statistic succinctly expresses how much more likely the fit of the full model is than reduced model. Calculate the likelihood ratio statistic to quantify the difference between the full and reduced model:

\begin{equation}
\tau(\theta_{2}) = \frac{{(SSE(\tilde{\theta_{2}})-SSE(\hat{\theta_{2}}))}}{SSE(\hat{\theta_{2}})/(n-p)}
\label{tau_lr}
\end{equation}

where $n$ is the number of observations and $p$ is the total number of parameters in the model (i.e., $p$ = 3 for the quadratic-plateau model).

4. Invert the likelihood ratio statistic to obtain a confidence interval about the full model for $\theta_{2}$. That is, for a given $\alpha$ level of significance, the profile-likelihood CI for $\theta_{2}$ is the set of all $\tilde{\theta_{2}}$ for which:

\begin{equation}
\tau(\theta_{2}) \leq Q(t_{d}, f)
\label{tau_inv}
\end{equation}

where $d$ is the degrees of freedom and $Q({t_{d}, f})$ is the ${f}$th quantile of the ${t}$-value distribution ($t_{d}$).

Because $\tilde{\theta_{2}}$ was intentionally set away from $\hat{\theta_{2}}$ (step 2), any increase in $SSE(\tilde{\theta_{2}})$ compared to $SSE(\hat{\theta_{2}})$ is derived from $\theta_{2}$ instead of any of the other model parameters. Because $SSE(\tilde{\theta_{2}})$ should never be less than $SSE(\hat{\theta_{2}})$, $\tau(\theta_{2})$ is a positive value for both the lower and upper confidence interval.

The algorithm for computing the profile-likelihood confidence intervals iteratively checks all $\tau(\theta_{2})$ that are less than or equal to the test statistic for the given $\alpha$ value. The `EONR` package uses the Nelder-Mead optimization algorithm ([Nelder & Mead, 1965](https://academic.oup.com/comjnl/article-abstract/7/4/308/354237?redirectedFrom=fulltext)) to efficiently find the confidence interval.

#### Bootstrapping
Bootstrapping involves sampling residuals of the original data with replacement (requires reparameterization similar to the profile-likelihood approach). This is a worthy alternative to the profile-likelihood approach, but it is not always perfect.

#### Wald-type (+/- 1 standard error)
The Wald-type approach is the simplest, but it has poor performance with small sample sizes and nonlinear models.
\begin{equation}
\hat{\theta} = Q(t_{d}, 1-\alpha/2)\:{SE}(\hat{\theta})
\label{wald}
\end{equation}

Where $\hat{\theta}$ is the parameter estimate, ${d}$ is the degrees of freedom, ${d=n-k,n}$ is the number of observations, $Q({t_{d}, f})$ is the ${f}$th quantile of the ${t}$-value distribution ($t_{d}$) with ${k}$ treatments, and ${SE}(\hat{\theta})$ is the standard error of the parameter estimate ([Cook & Weisberg, 1990](https://www.tandfonline.com/doi/abs/10.1080/01621459.1990.10476233); [Hernandez & Mulla, 2008](https://dl.sciencesocieties.org/publications/aj/abstracts/100/5/1221)).

### The social cost of nitrogen
The `EONR` package also allows the user to define a __social cost of nitrogen__, which is then used in the optimum nitrogen rate calculations based on residual soil nitrogen (nitrogen fertilizer not taken up by the crop at the end of the season).

The traditional approach for calculating the EONR considers only the cost of the nitrogen fertilizer product, and does not consider other unintended costs of nitrogen application. The social cost of nitrogen, defined as the present value of monetary damages caused by an incremental increase in nitrogen, has been suggested as a method to place a value on pollution and other damages (e.g., health, quality of living, etc.) caused by nitrogen fertilizer application ([Keeler et al., 2016](http://stacks.iop.org/1748-9326/9/i=7/a=074002?key=crossref.75a91c07d59a4043a07280d01299d0d8)). Because of the complexity of the nitrogen cycle and the spatial and temporal variability associated with it, the social cost of nitrogen is extremely difficult to quantify and is fraught with uncertainty. Additionally, the basis for what might be considered a healthy environment or an acceptable quality of living is highly subjective and may change abruptly depending on many factors. The social cost of nitrogen is a straightforward concept, however, and it can be a useful method to assess the value of economic gain versus damages from agricultural production.

When total nitrogen uptake is measured in field experiments, there is an opportunity to calculate the quantity of nitrogen that we know was not utilized by the crop (residual nitrogen). `EONR` uses crop nitrogen uptake (and optionally, nitrogen in the soil at the beginning of the season) to model end-of-season residual nitrogen as a function of crop-available nitrogen at the beginning of the season (including nitrogen applied as fertilizer). This residual nitrogen value can be multiplied by a social cost (per unit residual nitrogen) to determine the monetary damages that a given experimental treatment might be contributing to pollution or other damages caused by nitrogen fertilizer application. `EONR` then adjusts the optimum nitrogen rate after also considering these social costs derived from experimental data. This is meaningful because it provides a basis for analyzing the costs of pollution and other damages caused by nitrogen fertilizer at the field scale. Although analysis at the regional scale is worthwhile, results oftentimes to not translate to the field scale.

Depending on the economic scenario defined by the user `EONR` will calculate one of the following:

1. **Agronomic Optimum Nitrogen Rate** *(AONR)*: both the cost of nitrogen fertilizer and social cost of nitrogen are ignored
2. **Economic Optimum Nitrogen Rate** *(EONR)*: cost of nitrogen fertilizer is considered but social cost is ignored
3. **Socially Optimum Nitrogen Rate** *(SONR)*: both cost of nitrogen fertilizer and social cost of nitrogen are considered

The `EONR` package is able to calculate the **AONR**, **EONR**, or **SONR** for your dataset by simply adjusting the economic scenario (i.e., by adjusting `price_grain`, `cost_n_fert`, `cost_n_social`, and even `costs_fixed`).


# Online Drift Detection on the Wine Quality Dataset

In the context of deployed models, data (model queries) usually arrive sequentially and we wish to detect it as soon as possible after its occurence. One approach is to perform a test for drift every $W$ time-steps, using the $W$ samples that have arrived since the last test. Such a strategy could be implemented using any of the offline detectors implemented in `alibi-detect`, but being both sensitive to slight drift and responsive to severe drift is difficult. If the window size $W$ is too small then slight drift will be undetectable. If it is too large then the delay between test-points hampers responsiveness to severe drift.

An alternative strategy is to perform a test each time data arrives. However the usual offline methods are not applicable because the process for computing p-values is too expensive and doesn't account for correlated test outcomes when using overlapping windows of test data.

Online detectors instead work by computing the test-statistic once using the first $W$ data points and then updating the test-statistic sequentially at low cost. When no drift has occured the test-statistic fluctuates around its expected value and once drift occurs the test-statistic starts to drift upwards. When it exceeds some preconfigured threshold value, drift is detected.

Unlike offline detectors which require the specification of a threshold p-value (a false positive rate), the online detectors in `alibi-detect` require the specification of an expected run-time (ERT) (an inverted FPR). This is the number of time-steps that we insist our detectors, on average, should run for in the absense of drift before making a false detection. Usually we would like the ERT to be large, however this results in insensitive detectors which are slow to respond when drift does occur. There is a tradeoff between the expected run time and the expected detection delay.

To target the desired ERT, thresholds are configured during an initial configuration phase via simulation. This configuration process is only suitable when the amount reference data (most likely the training data of the model of interest) is relatively large (ideally around an order of magnitude larger than the desired ERT). Configuration can be expensive (less so with a GPU) but allows the detector to operate at low-cost during deployment.

This notebook demonstrates online drift detection using two different two-sample distance metrics for the test-statistic, the maximum mean discrepency (MMD) and least-squared density difference (LSDD), both of which can be updated sequentially at low cost.

### Backend

The online detectors are implemented in both the *PyTorch* and *TensorFlow* frameworks with support for CPU and GPU. Various preprocessing steps are also supported out-of-the box in Alibi Detect for both frameworks and an example will be given in this notebook. Alibi Detect does however not install PyTorch for you. Check the [PyTorch docs](https://pytorch.org/) how to do this.

### Dataset

The [Wine Quality Data Set](https://archive.ics.uci.edu/ml/datasets/wine+quality) consists of 4898 and 1599 samples of white and red wine respectively. Each sample has an associated quality (as determined by experts) and 11 numeric features indicating its acidity, density, pH etc. We consider the regression problem of tring to predict the quality of white wine samples given these features. We will then consider whether the model remains suitable for predicting the quality of red wine samples or whether the associated change in the underlying distribution should be considered as drift.

## Online detection with MMD and Pytorch

The Maximum Mean Discepency (MMD) is a distance-based measure between 2 distributions *p* and *q* based on the mean embeddings $\mu_{p}$ and $\mu_{q}$ in a reproducing kernel Hilbert space $F$:

\begin{align}
MMD(F, p, q) & = || \mu_{p} - \mu_{q} ||^2_{F} \\
\end{align}

Given reference samples $\{X_i\}_{i=1}^{N}$ and test samples $\{Y_i\}_{i=t}^{t+W}$ we may compute an unbiased estimate $\widehat{MMD}^2(F, \{X_i\}_{i=1}^N, \{Y_i\}_{i=t}^{t+W})$ of the squared MMD between the two underlying distributions. Depending on the size of the reference and test windows, $N$ and $W$ respectively, this can be relatively expensive. However, once computed it is possible to update the statistic to estimate to the squared MMD between the distributions underlying $\{X_i\}_{i=1}^{N}$ and $\{Y_i\}_{i=t+1}^{t+1+W}$ at a very low cost, making it suitable for online drift detection.

By default we use a [radial basis function kernel](https://en.wikipedia.org/wiki/Radial_basis_function_kernel), but users are free to pass their own kernel of preference to the detector.

```python
import matplotlib.pyplot as plt
import numpy as np
import torch
import tensorflow as tf
import pandas as pd
import scipy
from sklearn.decomposition import PCA

np.random.seed(0)
torch.manual_seed(0)
tf.random.set_seed(0)
```

### Load data

First we load in the data:

```python
red = pd.read_csv(
    "https://storage.googleapis.com/seldon-datasets/wine_quality/winequality-red.csv", sep=';'
)
white = pd.read_csv(
    "https://storage.googleapis.com/seldon-datasets/wine_quality/winequality-white.csv", sep=';'
)
white.describe()
```

      fixed acidity
      volatile acidity
      citric acid
      residual sugar
      chlorides
      free sulfur dioxide
      total sulfur dioxide
      density
      pH
      sulphates
      alcohol
      quality

      count
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000
      4898.000000

      mean
      6.854788
      0.278241
      0.334192
      6.391415
      0.045772
      35.308085
      138.360657
      0.994027
      3.188267
      0.489847
      10.514267
      5.877909

      std
      0.843868
      0.100795
      0.121020
      5.072058
      0.021848
      17.007137
      42.498065
      0.002991
      0.151001
      0.114126
      1.230621
      0.885639

      min
      3.800000
      0.080000
      0.000000
      0.600000
      0.009000
      2.000000
      9.000000
      0.987110
      2.720000
      0.220000
      8.000000
      3.000000

      25%
      6.300000
      0.210000
      0.270000
      1.700000
      0.036000
      23.000000
      108.000000
      0.991723
      3.090000
      0.410000
      9.500000
      5.000000

      50%
      6.800000
      0.260000
      0.320000
      5.200000
      0.043000
      34.000000
      134.000000
      0.993740
      3.180000
      0.470000
      10.400000
      6.000000

      75%
      7.300000
      0.320000
      0.390000
      9.900000
      0.050000
      46.000000
      167.000000
      0.996100
      3.280000
      0.550000
      11.400000
      6.000000

      max
      14.200000
      1.100000
      1.660000
      65.800000
      0.346000
      289.000000
      440.000000
      1.038980
      3.820000
      1.080000
      14.200000
      9.000000

We can see that the data for both red and white wine samples take the same format.

```python
red.describe()
```

      fixed acidity
      volatile acidity
      citric acid
      residual sugar
      chlorides
      free sulfur dioxide
      total sulfur dioxide
      density
      pH
      sulphates
      alcohol
      quality

      count
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000

      mean
      8.319637
      0.527821
      0.270976
      2.538806
      0.087467
      15.874922
      46.467792
      0.996747
      3.311113
      0.658149
      10.422983
      5.636023

      std
      1.741096
      0.179060
      0.194801
      1.409928
      0.047065
      10.460157
      32.895324
      0.001887
      0.154386
      0.169507
      1.065668
      0.807569

      min
      4.600000
      0.120000
      0.000000
      0.900000
      0.012000
      1.000000
      6.000000
      0.990070
      2.740000
      0.330000
      8.400000
      3.000000

      25%
      7.100000
      0.390000
      0.090000
      1.900000
      0.070000
      7.000000
      22.000000
      0.995600
      3.210000
      0.550000
      9.500000
      5.000000

      50%
      7.900000
      0.520000
      0.260000
      2.200000
      0.079000
      14.000000
      38.000000
      0.996750
      3.310000
      0.620000
      10.200000
      6.000000

      75%
      9.200000
      0.640000
      0.420000
      2.600000
      0.090000
      21.000000
      62.000000
      0.997835
      3.400000
      0.730000
      11.100000
      6.000000

      max
      15.900000
      1.580000
      1.000000
      15.500000
      0.611000
      72.000000
      289.000000
      1.003690
      4.010000
      2.000000
      14.900000
      8.000000

We shuffle and normalise the data such that each feature takes a value in \[0,1\], as does the quality we seek to predict. We assue that our model was trained on white wine samples, which therefore forms the reference distribution, and that red wine samples can be considered to be drawn from a drifted distribution.

```python
white, red = np.asarray(white, np.float32), np.asarray(red, np.float32)
n_white, n_red = white.shape[0], red.shape[0]

col_maxes = white.max(axis=0)
white, red = white / col_maxes, red / col_maxes
white, red = white[np.random.permutation(n_white)], red[np.random.permutation(n_red)]
X = white[:, :-1]
X_corr = red[:, :-1]
```

Although it may not be necessary on this relatively low-dimensional data for which individual features are semantically meaningful, we demonstrate how [principle component analysis (PCA)](https://en.wikipedia.org/wiki/Principal_component_analysis) can be performed as a preprocessing stage to project raw data onto a lower dimensional representation which more concisely captures the factors of variation in the data. As not to bias the detector it is necessary to fit the projection using a split of the data which isn't then passed as reference data. We additionally split off some white wine samples to act as undrifted data during deployment.

```python
X_train = X[:(n_white//2)]
X_ref = X[(n_white//2):(3*n_white//4)]
X_h0 = X[(3*n_white//4):]
```

Now we define a PCA object to be used as a preprocessing function to project the 11-D data onto a 2-D representation. We learn the first 2 principal components on the training split of the reference data.

```python
pca = PCA(2)
pca.fit(X_train)
```

    PCA(n_components=2)

Hopefully the learned preprocessing step has learned a projection such that in the lower dimensional space the two samples are distinguishable.

```python
enc_h0 = pca.transform(X_h0)
enc_h1 = pca.transform(X_corr)

plt.scatter(enc_h0[:,0], enc_h0[:,1], alpha=0.2, color='green', label='white wine')
plt.scatter(enc_h1[:,0], enc_h1[:,1], alpha=0.2, color='red', label='red wine')
plt.legend(loc='upper right')
plt.show()
```

Now we can define our online drift detector. We specify an expected run-time (in the absence of drift) of 50 time-steps, and a window size of 10 time-steps. Upon initialising the detector thresholds will be computed using 2500 boostrap samples. These values of `ert`, `window_size` and `n_bootstraps` are lower than a typical use-case in order to demonstrate the average behaviour of the detector over a large number of runs in a reasonable time.

```python
from alibi_detect.cd import MMDDriftOnline

ert = 50
window_size = 10

cd = MMDDriftOnline(
    X_ref, ert, window_size, backend='pytorch', preprocess_fn=pca.transform, n_bootstraps=2500
)
```

    No GPU detected, fall back on CPU.

      3%|▎         | 74/2500 [00:00<00:03, 736.80it/s]

    Generating permutations of kernel matrix..

    100%|██████████| 2500/2500 [00:04<00:00, 599.06it/s]
    Computing thresholds: 100%|██████████| 10/10 [00:02<00:00,  3.59it/s]

We now define a function which will simulate a single run and return the run-time. Note how the detector acts on single instances at a time, the run-time is considered as the time elapsed after the test-window has been filled, and that the detector is stateful and must be reset between detections.

```python
def time_run(cd, X, window_size):
    n = X.shape[0]
    perm = np.random.permutation(n)
    t = 0
    cd.reset()
    while True:
        pred = cd.predict(X[perm[t%n]])
        if pred['data']['is_drift'] == 1:
            return t
        else:
            t += 1
```

Now we look at the distribution of run-times when operating on the held-out data from the reference distribution of white wine samples. We report the average run-time, however note that the targeted run-time distribution, a Geometric distribution with mean `ert`, is very high variance so the empirical average may not be that close to `ert` over a relatively small number of runs. We can see that the detector accurately targets the desired Geometric distribution however by inspecting the linearity of a [Q-Q plot](https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot).

```python
n_runs = 250
times_h0 = [time_run(cd, X_h0, window_size) for _ in range(n_runs)]
print(f"Average run-time under no-drift: {np.mean(times_h0)}")
_ = scipy.stats.probplot(np.array(times_h0), dist=scipy.stats.geom, sparams=1/ert, plot=plt)
```

If we run the detector in an identical manner but on data from the drifted distribution of red wine samples the average run-time is much lower.

```python
n_runs = 250
times_h1 = [time_run(cd, X_corr, window_size) for _ in range(n_runs)]
print(f"Average run-time under drift: {np.mean(times_h1)}")
```

    Average run-time under drift: 6.004

## Online detection with LSDD and TensorFlow

Here we address the same problem but using the least squares density difference (LSDD) as the two-sample distance in a manner similar to [Bu et al. (2017)](https://ieeexplore.ieee.org/abstract/document/7890493). The LSDD between two distributions $p$ and $q$ on $\mathcal{X}$ is defined as $$LSDD(p,q) = \int_{\mathcal{X}} (p(x)-q(x))^2 \,dx$$ and also has an empirical estimate $\widehat{LSDD}(\{X_i\}_{i=1}^N, \{Y_i\}_{i=t}^{t+W})$ that can be updated at low cost as the test window is updated to $\{Y_i\}_{i=t+1}^{t+1+W}$.

We additionally show that TensorFlow can also be used as the backend and that sometimes it is not necessary to perform preprocessing, making definition of the drift detector simpler. Moreover, in the absence of a learned preprocessing stage we may use all of the reference data available.

```python
X_ref = np.concatenate([X_train, X_ref], axis=0)
```

And now we define the LSDD-based online drift detector, again with an `ert` of 50 and `window_size` of 10.

```python
from alibi_detect.cd import LSDDDriftOnline

cd = LSDDDriftOnline(
    X_ref, ert, window_size, backend='tensorflow', n_bootstraps=2500,
)
```

    Computing thresholds: 100%|██████████| 9/9 [00:09<00:00,  1.00s/it]

We run this new detector on the held out reference data and again see that in the absence of drift the distribution of run-times follows a Geometric distribution with mean `ert`.

```python
n_runs = 250
times_h0 = [time_run(cd, X_h0, window_size) for _ in range(n_runs)]
print(f"Average run-time under no-drift: {np.mean(times_h0)}")
_ = scipy.stats.probplot(np.array(times_h0), dist=scipy.stats.geom, sparams=1/ert, plot=plt)
```

And when drift has occured the detector is very fast to respond.

```python
n_runs = 250
times_h1 = [time_run(cd, X_corr, window_size) for _ in range(n_runs)]
print(f"Average run-time under drift: {np.mean(times_h1)}")
```

    Average run-time under drift: 4.328


# Start-to-Finish Example: Unit Testing `GiRaFFE_NRPy`: Interpolating Metric Face-Values

## Author: Patrick Nelson

## This module Validates the `FCVAL` routine for `GiRaFFE`.

**Notebook Status:** Validated

**Validation Notes:** This module will validate the routines in [Tutorial-GiRaFFE_NRPy-Metric_Face_Values](Tutorial-GiRaFFE_NRPy-Metric_Face_Values.ipynb).

### NRPy+ Source Code for this module:
* [GiRaFFE_NRPy/GiRaFFE_NRPy_Metric_Face_Values.py](../../edit/in_progress/GiRaFFE_NRPy/GiRaFFE_NRPy_Metric_Face_Values.py) [\[**tutorial**\]](Tutorial-GiRaFFE_NRPy-FCVAL.ipynb) Generates the C code to interpolate gridfunctions to cell faces.

## Introduction:

This notebook validates the code that will interpolate the metric gridfunctions on cell faces. These values, along with the reconstruction of primitive variables on the faces, are necessary for the Riemann solvers to compute the fluxes through the cell faces.

It is, in general, good coding practice to unit test functions individually to verify that they produce the expected and intended output. We will generate test data with arbitrarily-chosen analytic functions and calculate gridfunctions at the cell centers on a small numeric grid. We will then compute the values on the cell faces in two ways: first, with our interpolator, then second, we will shift the grid and compute them analytically. Then, we will rerun the function at a finer resolution. Finally, we will compare the results of the two runs to show third-order convergence.

When this notebook is run, the difference between the approximate and exact metric gridfunctions will be output to text files that can be found in the same directory as this notebook. These will be read in in [Step 3](#convergence), and used there to confirm convergence order of the algorithm.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#setup): Set up core functions and parameters for unit testing the FCVAL algorithm
    1. [Step 1.a](#expressions) Write expressions for the metric gridfunctions
    1. [Step 1.b](#ccodekernels) Generate C functions to calculate the gridfunctions
    1. [Step 1.c](#free_parameters) Set free parameters in the code
1. [Step 2](#mainc): `FCVAL_unit_test.c`: The Main C Code
    1. [Step 2.a](#compile_run): Compile and run the code
1. [Step 3](#convergence): Code validation: Verify that relative error in numerical solution converges to zero at the expected order
1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Set up core functions and parameters for unit testing the FCVAL algorithm \[Back to [top](#toc)\]
$$\label{setup}$$

We'll start by appending the relevant paths to `sys.path` so that we can access sympy modules in other places. Then, we'll import NRPy+ core functionality and set up a directory in which to carry out our test.

```python
import os, sys           # Standard Python modules for multiplatform OS-level functions
# First, we'll add the parent directory to the list of directories Python will check for modules.
nrpy_dir_path = os.path.join("..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)
nrpy_dir_path = os.path.join("..","..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

from outputC import outCfunction, lhrh # NRPy+: Core C code output module
import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends
import finite_difference as fin  # NRPy+: Finite difference C code generation module
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import grid as gri               # NRPy+: Functions having to do with numerical grids
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import reference_metric as rfm   # NRPy+: Reference metric support
import cmdline_helper as cmd     # NRPy+: Multi-platform Python command-line interface

out_dir = "Validation/"
cmd.mkdir(out_dir)
subdir = "FCVAL"
cmd.mkdir(os.path.join(out_dir,subdir))

thismodule = "Start_to_Finish_UnitTest-GiRaFFE_NRPy-Metric_Face_Values"

# Set the finite-differencing order to 2
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER", 2)

```

## Step 1.a: Write expressions for the metric gridfunctions \[Back to [top](#toc)\]
$$\label{expressions}$$

Now, we'll choose some functions with arbitrary forms to generate test data. We'll need to set ten gridfunctions, so expressions are being pulled from several previously written unit tests.

\begin{align}
\gamma_{xx} &= ax^3 + by^3 + cz^3 + dy^2 + ez^2 + f \\
\gamma_{yy} &= gx^3 + hy^3 + lz^3 + mx^2 + nz^2 + p \\
\gamma_{zz} &= px^3 + qy^3 + rz^3 + sx^2 + ty^2 + u. \\
\gamma_{xy} &= a \exp\left(-\left((x-b)^2+(y-c)^2+(z-d)^2\right)\right) \\
\gamma_{xz} &= f \exp\left(-\left((x-g)^2+(y-h)^2+(z-l)^2\right)\right) \\
\gamma_{yz} &= m \exp\left(-\left((x-n)^2+(y-o)^2+(z-p)^2\right)\right), \\
\beta^x &= \frac{2}{\pi} \arctan(ax + by + cz) \\
\beta^y &= \frac{2}{\pi} \arctan(bx + cy + az) \\
\beta^z &= \frac{2}{\pi} \arctan(cx + ay + bz) \\
\alpha &= 1 - \frac{1}{2+x^2+y^2+z^2} \\
\end{align}

```python
a,b,c,d,e,f,g,h,l,m,n,o,p,q,r,s,t,u = par.Cparameters("REAL",thismodule,["a","b","c","d","e","f","g","h","l","m","n","o","p","q","r","s","t","u"],1e300)
M_PI  = par.Cparameters("#define",thismodule,["M_PI"], "")

gammaDD = ixp.register_gridfunctions_for_single_rank2("AUXEVOL","gammaDD","sym01",DIM=3)
betaU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","betaU",DIM=3)
alpha = gri.register_gridfunctions("AUXEVOL","alpha")

par.set_parval_from_str("reference_metric::CoordSystem","Cartesian")
rfm.reference_metric()
x = rfm.xxCart[0]
y = rfm.xxCart[1]
z = rfm.xxCart[2]

gammaDD[0][0] = a*x**3 + b*y**3 + c*z**3 + d*y**2 + e*z**2 + f
gammaDD[1][1] = g*x**3 + h*y**3 + l*z**3 + m*x**2 + n*z**2 + o
gammaDD[2][2] = p*x**3 + q*y**3 + r*z**3 + s*x**2 + t*y**2 + u
gammaDD[0][1] = a * sp.exp(-((x-b)**2 + (y-c)**2 + (z-d)**2))
gammaDD[0][2] = f * sp.exp(-((x-g)**2 + (y-h)**2 + (z-l)**2))
gammaDD[1][2] = m * sp.exp(-((x-n)**2 + (y-o)**2 + (z-p)**2))

betaU[0] = (sp.sympify(2)/M_PI) * sp.atan(a*x + b*y + c*z)
betaU[1] = (sp.sympify(2)/M_PI) * sp.atan(b*x + c*y + a*z)
betaU[2] = (sp.sympify(2)/M_PI) * sp.atan(c*x + a*y + b*z)

alpha = sp.sympify(1) - sp.sympify(1) / (sp.sympify(2) + x**2 + y**2 + z**2)
```

## Step 1.b: Generate C functions to calculate the gridfunctions \[Back to [top](#toc)\]
$$\label{ccodekernels}$$

Here, we will use the NRPy+ function `outCfunction()` to generate C code that will calculate our metric gridfunctions over an entire grid. We will also call the function to generate the function we are testing.

```python
metric_gfs_to_print = [\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD00"),rhs=gammaDD[0][0]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD01"),rhs=gammaDD[0][1]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD02"),rhs=gammaDD[0][2]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD11"),rhs=gammaDD[1][1]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD12"),rhs=gammaDD[1][2]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","gammaDD22"),rhs=gammaDD[2][2]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","betaU0"),rhs=betaU[0]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","betaU1"),rhs=betaU[1]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","betaU2"),rhs=betaU[2]),\
                       lhrh(lhs=gri.gfaccess("aux_gfs","alpha"),rhs=alpha),\
                      ]

desc = "Calculate the metric gridfunctions"
name = "calculate_metric_gfs"
outCfunction(
    outfile  = os.path.join(out_dir,name+".h"), desc=desc, name=name,
    params   ="const paramstruct *restrict params,REAL *restrict xx[3],REAL *restrict auxevol_gfs",
    body     = fin.FD_outputC("returnstring",metric_gfs_to_print,params="outCverbose=False").replace("IDX4","IDX4S"),
    loopopts="AllPoints,Read_xxs")

import GiRaFFE_NRPy.GiRaFFE_NRPy_Metric_Face_Values as FCVAL
FCVAL.GiRaFFE_NRPy_FCVAL(os.path.join(out_dir,subdir))

```

    Output C function calculate_metric_gfs() to file Validation/calculate_metric_gfs.h

## Step 1.c: Set free parameters in the code \[Back to [top](#toc)\]
$$\label{free_parameters}$$

We also need to create the files that interact with NRPy's C parameter interface.

```python
# Step 3.d.i: Generate declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
# par.generate_Cparameters_Ccodes(os.path.join(out_dir))

# Step 3.d.ii: Set free_parameters.h
with open(os.path.join(out_dir,"free_parameters.h"),"w") as file:
    file.write("""
// Override parameter defaults with values based on command line arguments and NGHOSTS.
params.Nxx0 = atoi(argv[1]);
params.Nxx1 = atoi(argv[2]);
params.Nxx2 = atoi(argv[3]);
params.Nxx_plus_2NGHOSTS0 = params.Nxx0 + 2*NGHOSTS;
params.Nxx_plus_2NGHOSTS1 = params.Nxx1 + 2*NGHOSTS;
params.Nxx_plus_2NGHOSTS2 = params.Nxx2 + 2*NGHOSTS;
// Step 0d: Set up space and time coordinates
// Step 0d.i: Declare \Delta x^i=dxx{0,1,2} and invdxx{0,1,2}, as well as xxmin[3] and xxmax[3]:
const REAL xxmin[3] = {-1.0,-1.0,-1.0};
const REAL xxmax[3] = { 1.0, 1.0, 1.0};

params.dxx0 = (xxmax[0] - xxmin[0]) / ((REAL)params.Nxx0);
params.dxx1 = (xxmax[1] - xxmin[1]) / ((REAL)params.Nxx1);
params.dxx2 = (xxmax[2] - xxmin[2]) / ((REAL)params.Nxx2);
printf("dxx0,dxx1,dxx2 = %.5e,%.5e,%.5e\\n",params.dxx0,params.dxx1,params.dxx2);
params.invdx0 = 1.0 / params.dxx0;
params.invdx1 = 1.0 / params.dxx1;
params.invdx2 = 1.0 / params.dxx2;
\n""")

# Generates declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
par.generate_Cparameters_Ccodes(os.path.join(out_dir))
```

# Step 2: `FCVAL_unit_test.c`: The Main C Code \[Back to [top](#toc)\]
$$\label{mainc}$$

```python
%%writefile $out_dir/FCVAL_unit_test.c
// These are common packages that we are likely to need.
#include "stdio.h"
#include "stdlib.h"
#include "math.h"
#include "string.h" // Needed for strncmp, etc.
#include "stdint.h" // Needed for Windows GCC 6.x compatibility
#include <time.h>   // Needed to set a random seed.

#define REAL double
#include "declare_Cparameters_struct.h"

const int NGHOSTS = 3;

REAL a,b,c,d,e,f,g,h,l,m,n,o,p,q,r,s,t,u;

// Standard NRPy+ memory access:
#define IDX4S(g,i,j,k) \
( (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * ( (k) + Nxx_plus_2NGHOSTS2 * (g) ) ) )

const int kronecker_delta[4][3] = { { 0,0,0 },
                                    { 1,0,0 },
                                    { 0,1,0 },
                                    { 0,0,1 } };

// Give gridfunctions their names:
#define GAMMADD00GF 0
#define GAMMADD01GF 1
#define GAMMADD02GF 2
#define GAMMADD11GF 3
#define GAMMADD12GF 4
#define GAMMADD22GF 5
#define BETAU0GF 6
#define BETAU1GF 7
#define BETAU2GF 8
#define ALPHAGF 9
#define GAMMA_FACEDD00GF 10
#define GAMMA_FACEDD01GF 11
#define GAMMA_FACEDD02GF 12
#define GAMMA_FACEDD11GF 13
#define GAMMA_FACEDD12GF 14
#define GAMMA_FACEDD22GF 15
#define BETA_FACEU0GF 16
#define BETA_FACEU1GF 17
#define BETA_FACEU2GF 18
#define ALPHA_FACEGF 19
#define NUM_AUXEVOL_GFS 20

#include "calculate_metric_gfs.h"
#include "FCVAL/interpolate_metric_gfs_to_cell_faces.h"

int main(int argc, const char *argv[]) {
    paramstruct params;
#include "set_Cparameters_default.h"

    // Step 0c: Set free parameters, overwriting Cparameters defaults
    //          by hand or with command-line input, as desired.
#include "free_parameters.h"
#include "set_Cparameters-nopointer.h"

    // Step 0e: Set up cell-centered Cartesian coordinate grids
    REAL *xx[3];
    xx[0] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS0);
    xx[1] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS1);
    xx[2] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS2);
    for(int j=0;j<Nxx_plus_2NGHOSTS0;j++) xx[0][j] = xxmin[0] + (j-NGHOSTS)*dxx0;
    for(int j=0;j<Nxx_plus_2NGHOSTS1;j++) xx[1][j] = xxmin[1] + (j-NGHOSTS)*dxx1;
    for(int j=0;j<Nxx_plus_2NGHOSTS2;j++) xx[2][j] = xxmin[2] + (j-NGHOSTS)*dxx2;

    //for(int i=0;i<Nxx_plus_2NGHOSTS0;i++) printf("xx[0][%d] = %.15e\n",i,xx[0][i]);

    // This is the array to which we'll write the NRPy+ variables.
    REAL *auxevol_gfs  = (REAL *)malloc(sizeof(REAL) * NUM_AUXEVOL_GFS * Nxx_plus_2NGHOSTS2 * Nxx_plus_2NGHOSTS1 * Nxx_plus_2NGHOSTS0);
    // And another for exact data:
    REAL *auxevol_exact_gfs  = (REAL *)malloc(sizeof(REAL) * NUM_AUXEVOL_GFS * Nxx_plus_2NGHOSTS2 * Nxx_plus_2NGHOSTS1 * Nxx_plus_2NGHOSTS0);

    // Generate some random coefficients. Leave the random seed on its default for consistency between trials.
    a = (double)(rand()%20)/5.0;
    f = (double)(rand()%20)/5.0;
    m = (double)(rand()%20)/5.0;
    b = (double)(rand()%10-5)/100.0;
    c = (double)(rand()%10-5)/100.0;
    d = (double)(rand()%10-5)/100.0;
    g = (double)(rand()%10-5)/100.0;
    h = (double)(rand()%10-5)/100.0;
    l = (double)(rand()%10-5)/100.0;
    n = (double)(rand()%10-5)/100.0;
    o = (double)(rand()%10-5)/100.0;
    p = (double)(rand()%10-5)/100.0;

    // First, calculate the test data on our grid:
    calculate_metric_gfs(&params,xx,auxevol_gfs);
    // Now, run our function:
    int flux_dirn = 0;
    interpolate_metric_gfs_to_cell_faces(&params,auxevol_gfs,flux_dirn+1);
    // For comparison, shift the grid half a gridpoint, then calculate the test data again.
    REAL dxx[3] = {dxx0,dxx1,dxx2};
    REAL Nxxp2NG[3] = {Nxx_plus_2NGHOSTS0,Nxx_plus_2NGHOSTS1,Nxx_plus_2NGHOSTS2};
    for(int i=0;i<Nxxp2NG[flux_dirn];i++) {
        xx[flux_dirn][i] -= dxx[flux_dirn]/2.0;
    }
    calculate_metric_gfs(&params,xx,auxevol_exact_gfs);

    char filename[100];
    sprintf(filename,"out%d-numer.txt",Nxx0);
    FILE *out2D = fopen(filename, "w");
    for(int i2=0;i2<Nxx_plus_2NGHOSTS2;i2++) for(int i1=0;i1<Nxx_plus_2NGHOSTS1;i1++) for(int i0=0;i0<Nxx_plus_2NGHOSTS0;i0++) {
        // We print the difference between approximate and exact numbers.
        fprintf(out2D,"%.16e\t %e %e %e\n",
                auxevol_gfs[IDX4S(BETA_FACEU0GF,i0,i1,i2)]-auxevol_exact_gfs[IDX4S(BETAU0GF,i0,i1,i2)],
                xx[0][i0],xx[1][i1],xx[2][i2]
                );
    }
    fclose(out2D);
}
```

    Overwriting Validation//FCVAL_unit_test.c

## Step 2.a: Compile and run the code \[Back to [top](#toc)\]
$$\label{compile_run}$$

Now that we have our file, we can compile it and run the executable.

```python
import time

print("Now compiling, should take ~2 seconds...\n")
start = time.time()
cmd.C_compile(os.path.join(out_dir,"FCVAL_unit_test.c"), os.path.join(out_dir,"FCVAL_unit_test"))
end = time.time()
print("Finished in "+str(end-start)+" seconds.\n\n")

print("Now running...\n")
start = time.time()
cmd.Execute(os.path.join("Validation","FCVAL_unit_test"), "10 10 10","out.txt")
# To do a convergence test, we'll also need a second grid with twice the resolution.
cmd.Execute(os.path.join("Validation","FCVAL_unit_test"), "20 20 20","out.txt")
end = time.time()
print("Finished in "+str(end-start)+" seconds.\n\n")

```

    Now compiling, should take ~2 seconds...

    Compiling executable...
    (EXEC): Executing `gcc -Ofast -fopenmp -march=native -funroll-loops Validation/FCVAL_unit_test.c -o Validation/FCVAL_unit_test.exe -lm`...
    (BENCH): Finished executing in 1.0330591201782227 seconds.
    Finished compilation.
    Finished in 1.18806791305542 seconds.

    Now running...

    (EXEC): Executing `cmd /c Validation\FCVAL_unit_test 10 10 10`...
    (BENCH): Finished executing in 0.23601365089416504 seconds.
    (EXEC): Executing `cmd /c Validation\FCVAL_unit_test 20 20 20`...
    (BENCH): Finished executing in 0.2540144920349121 seconds.
    Finished in 0.7620437145233154 seconds.

# Step 3: Code validation: Verify that relative error in numerical solution converges to zero at the expected order \[Back to [top](#toc)\]
$$\label{convergence}$$

Here, we import the data at two resolutions and wrote to text files. This data consists of the absolute error of a metric gridfunction at each in the grid. We'll plot a portion of this data along the axis at the lower resolution along with that same data at the higher resolution scaled to demonstrate that this error converges to 0 at the expected rate. Since our algorithm uses a third-order polynomial, we expect fourth-order convergence here.

```python
import numpy as np
import matplotlib.pyplot as plt

plt.rcParams.update({'font.size': 12})

Data1 = np.loadtxt("out10-numer.txt")
Data2 = np.loadtxt("out20-numer.txt")

def IDX4(i,j,k,Nxx_plus_2NGHOSTS0,Nxx_plus_2NGHOSTS1,Nxx_plus_2NGHOSTS2):
    return (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * ( (k) + Nxx_plus_2NGHOSTS2 * (0) ) )

x1 = np.zeros(10)
a1 = np.zeros(10)
for i in range(10):
    x1[i] = Data1[IDX4(i+3,8,8,16,16,16),1]
    a1[i] = Data1[IDX4(i+3,8,8,16,16,16),0]
x2 = np.zeros(20)
a2 = np.zeros(20)
for i in range(20):
    x2[i] = Data2[IDX4(i+3,13,13,26,26,26),1]
    a2[i] = Data2[IDX4(i+3,13,13,26,26,26),0]

plt.figure()
a = plt.plot(x1,a1,'.',label="dx")
b = plt.plot(x2,a2*(2**4),label="dx/2, times (20/10)^4")
plt.legend()
plt.xlabel("x")
plt.ylabel("alpha")
plt.show()

convergence_satisfactory = np.log2(a1/a2[0:-1:2])>3
if not convergence_satisfactory.all():
    sys.exit(1)
```

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-Metric_Face_Values.pdf](Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-Metric_Face_Values.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-Metric_Face_Values")
```

    Notebook output to PDF is only supported on Linux systems, with pdflatex installed.