NOTE: Videos and graphs do not display immediately on all devices.  If this is the case for you, click the "Cell" menu option, and select "Run All".  If you still do not see the videos/graphs, try a different browser.  If that also does not work, please let your instructor know by email.

# Partial Derivatives

In this notebook, we extend the concept of the derivative of a function of one variable to partial derivatives for functions of several variables.

Recall that the derivative of a function of one variable, $y=f(x)$, represents the rate of change of $y$ with respect to $x$.

For functions of several variables, we can measure rates of change in the direction of each of the independent variables.  These are called partial derivatives.  Many of the ideas about derivatives carry over.

### 1. Partial Derivatives for Functions of Two Variables

Consider the function $z=f(x,y)$.  Suppose we keep $y=b$ fixed.  In that case, we essentially obtain a function of a single variable, $x$, namely $g(x) = f(x,b)$.  By definition, the derivative of $g(x)$ at $x=a$ is
$$g'(a) = \lim_{h \to 0} \frac{ g(a+h) - g(a) }{ h }.$$
Introducing the notation $f_x$ to denote the partial derivative of $f$ with respect to $x$, we thus have
$$f_x(a,b) = \lim_{h \to 0} \frac{ f(a+h,b) - f(a,b) }{ h }.$$

Similarly, using $f_y$ to denote the partial derivative of $f$ with respect to $y$, we have
$$f_y(a,b) = \lim_{h \to 0} \frac{ f(a,b+h) - f(a,b) }{ h }.$$

Generalizing (by letting $a$ and $b$ vary), we have
\begin{align}
f_x(x,y) & = \lim_{h \to 0} \frac{ f(x+h,y) - f(x,y) }{ h }, \\
f_y(x,y) & = \lim_{h \to 0} \frac{ f(x,y+h) - f(x,y) }{ h }.
\end{align}

These definitions suggest the following practical approach to determining partial derivatives of $z=f(x,y)$:
- To determine $f_x$, treat $y$ as a constant, and differentiate with respect to $x$.
- To determine $f_y$, treat $x$ as a constant, and differentiate with respect to $y$.

Example 1:

Consider $f(x,y) = 3 - x^3 - y^2$.  Determine $f_x(1,1)$ and $f_y(1,1)$.

> Treating $y$ as a constant, and differentiating with respect to $x$ gives
$$f_x(x,y) = 0 - 3x^2 - 0 = -3x^2,$$
and so $f_x(1,1) = -3$.

> Treating $x$ as a constant, and differentiating with respect to $y$ gives
$$f_y(x,y) = 0 - 0 - 2y = -2y,$$
and so $f_y(1,1) = -2$.

Those readers wishing to see the above example (and all following examples) worked out in a video are referred to Section 6 below.

### 2. Partial Derivatives: Additional Notation

Alternative notation is similar to Leibniz notation, but using the symbol $\partial$ instead of $d$.  If $z = f(x,y)$, we can write
\begin{align}
f_x(x,y) & = f_x = \frac{\partial f}{\partial x} = \frac{\partial}{\partial x} f(x,y) = \frac{\partial z}{\partial x}, \\
f_y(x,y) & = f_y = \frac{\partial f}{\partial y} = \frac{\partial}{\partial y} f(x,y) = \frac{\partial z}{\partial y}.
\end{align}

Example 2:

Consider $f(x,y) = \sin(xy^2)$.  Determine $f_x(x,y)$ and $f_y(x,y)$.

> Treating $y$ as a constant, and differentiating with respect to $x$ (keeping the chain rule in mind) gives
\begin{align}
f_x(x,y)
& = \cos(xy^2) \cdot \frac{\partial}{\partial x} (xy^2) \\
& = \cos(xy^2) \cdot y^2 \\
& = y^2 \cos(xy^2).
\end{align}

> Treating $x$ as a constant, and differentiating with respect to $y$ (keeping the chain rule in mind) gives
\begin{align}
f_y(x,y)
& = \cos(xy^2) \cdot \frac{\partial}{\partial y} (xy^2) \\
& = \cos(xy^2) \cdot x \cdot 2y \\
& = 2 xy \cos(xy^2).
\end{align}

Example 3:

Consider $f(x,y) = e^{xy} (x^2 + y^2)$.  Determine $f_x(x,y)$ and $f_y(x,y)$.

> Treating $y$ as a constant, and differentiating with respect to $x$ (keeping the product rule and the chain rule in mind) gives
\begin{align}
f_x(x,y)
& = \frac{\partial}{\partial x} e^{xy} \cdot (x^2 + y^2) + e^{xy} \cdot \frac{\partial}{\partial x}(x^2 + y^2) \\
& = e^{xy} \cdot \frac{\partial}{\partial x}(xy) \cdot (x^2 + y^2) + e^{xy} \cdot (2x + 0) \\
& = e^{xy} \cdot y \cdot (x^2 + y^2) + 2 x e^{xy} \\
& = y(x^2+y^2)e^{xy} + 2xe^{xy}.
\end{align}

> Similarly (try it yourself!), we have
$$f_y(x,y) = x(x^2+y^2)e^{xy} + 2ye^{xy}.$$

### 3. Checking Your Work: Partial Differentiation with Technology

On WolframAlpha, you can simply use the `differentiate` command, but you need to indicate with respect to which variable you want to differentiate, for example `differentiate sin(xy^2) with respect to x`.

### 4. Partial Derivatives of Functions of More Than Two Variables

The concept of the partial derivative extends naturally to functions of more than two variables.  Formally, for a function of $n$ variables, $f(x_1, x_2, \ldots, x_n)$, we have
$$f_{x_i} = \frac{\partial f}{\partial x_i} =
\lim_{h \to 0} \frac{ f(x_1, \ldots, x_{i-1}, x_i + h, x_{i+1}, \ldots, x_n) - f(x_1, \ldots, x_{i-1}, x_i, x_{i+1}, \ldots, x_n) }{h}.$$

In practice, to determine $f_{x_i}$, treat $x_1, \ldots, x_{i-1}, x_{i+1}, \ldots, x_n$ as constants, and differentiate with respect to $x_i$.

Example 4:

Consider $f(x,y,z) = \ln(xyz^2)$.  Determine $f_x$ and $f_z$.

>Treating $y$ and $z$ as constants, and differentiating with respect to $x$ gives
$$f_x(x,y,z) = \frac{1}{xyz^2} \frac{\partial}{\partial x}(xyz^2) = \frac{yz^2}{xyz^2}.$$

>Treating $x$ and $y$ as constants, and differentiating with respect to $z$ gives
$$f_z(x,y,z) = \frac{1}{xyz^2} \frac{\partial}{\partial z}(xyz^2) = \frac{2xyz}{xyz^2}.$$

### 5. Higher-Order Partial Derivatives

In general, for $z=f(x,y)$, the partial derivatives $f_x$ and $f_y$ also are functions of $x$ and $y$.  We can differentiate each of these functions with respect to $x$ and $y$ to obtain $(f_x)_x$, $(f_x)_y$, $(f_y)_x$, and $(f_y)_y$.  We introduce the following notation:
\begin{align}
(f_x)_x & = f_{xx} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial x} \right) = \frac{\partial^2 f}{\partial x^2} = \frac{\partial^2 z}{\partial x^2}, \\
(f_x)_y & = f_{xy} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial x} \right) = \frac{\partial^2 f}{\partial y \ \partial x} = \frac{\partial^2 z}{\partial y \ \partial x}, \\
(f_y)_x & = f_{yx} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial y} \right) = \frac{\partial^2 f}{\partial x \ \partial y} = \frac{\partial^2 z}{\partial x \ \partial y}, \\
(f_y)_y & = f_{yy} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial y} \right) = \frac{\partial^2 f}{\partial y^2} = \frac{\partial^2 z}{\partial y^2}, \\
\end{align}

> Note the order of the subscript $xy$ and the order of $\partial y \ \partial x$ in the denominator: $\displaystyle f_{xy} = \frac{\partial^2 f}{\partial y \ \partial x}$ indicates differentiation with respect to $x$ first, followed by differentiation with respect to $y$.

Luckily, for well-behaved functions (where $f$, $f_x$, $f_y$, $f_{xy}$, and $f_{yx}$ all are continuous in the neighbourhood of the point $(a,b)$), the mixed derivatives are equal, that is,
$$f_{xy}(a,b) = f_{yx}(a,b),$$
and the order of differentiation does not matter.

The notation extends naturally to functions of more than two variables, as well as to higher-order partial derivatives.

Example 5

Consider $f(x,y,z) = \cos(4x+3y+2z)$.  Determine $f_{xyz}$ and $f_{yzz}$.

> For $f_{xyz}$, we could use the following steps (but other steps will lead to the same result):
First,
\begin{align}
f_x
& = -\sin(4x+3y+2z) \cdot \frac{\partial}{\partial x} (4x+3y+2z) \\
& = -\sin(4x+3y+2z) \cdot (4+0+0) \\
& = -4\sin(4x+3y+2z).
\end{align}
Second,
\begin{align}
f_{xy}
& = (f_x)_y \\
& = -4\cos(4x+3y+2z) \cdot \frac{\partial}{\partial y} (4x+3y+2z)\\
& = -4\cos(4x+3y+2z) \cdot (0+3+0) \\
& = -12\cos(4x+3y+2z).
\end{align}
Third,
\begin{align}
f_{xyz} & = (f_{xy})_z \\
& = 12\sin(4x+3y+2z) \cdot \frac{\partial}{\partial z} (4x+3y+2z)\\
& = 12\sin(4x+3y+2z) \cdot (0+0+2) \\
& = 24 \sin(4x+3y+2z).
\end{align}

> For $f_{yzz}$, we could use the following steps (but other steps will lead to the same result):
First,
\begin{align}
f_y
& = -\sin(4x+3y+2z) \cdot \frac{\partial}{\partial y} (4x+3y+2z) \\
& = -\sin(4x+3y+2z) \cdot (0+3+0) \\
& = -3\sin(4x+3y+2z).
\end{align}
Second,
\begin{align}
f_{yz}
& = (f_y)_z \\
& = -3\cos(4x+3y+2z) \cdot \frac{\partial}{\partial z} (4x+3y+2z)\\
& = -3\cos(4x+3y+2z) \cdot (0+0+2) \\
& = -6\cos(4x+3y+2z).
\end{align}
Third,
\begin{align}
f_{yzz} & = (f_{yz})_z \\
& = 6\sin(4x+3y+2z) \cdot \frac{\partial}{\partial z} (4x+3y+2z)\\
& = 12\sin(4x+3y+2z).
\end{align}

### 6. All Examples Collected in One Video (Optional)

In the following video, we work through all of the examples treated in this notebook.

```python
from IPython.display import YouTubeVideo
YouTubeVideo('uMjJlIxiPeU')
```

### 7. Summary

- For $f(x_1, x_2, \ldots, x_n)$, the partial derivative $\displaystyle f_{x_i} = \frac{\partial f}{\partial x_i}$ is obtained by treating $x_1, \ldots, x_{i-1}, x_{i+1}, \ldots, x_n$ as constants, and differentiating with respect to $x_i$.

- Notation for partial derivatives of $z=f(x,y)$:
\begin{align}
f_x(x,y) & = f_x = \frac{\partial f}{\partial x} = \frac{\partial}{\partial x} f(x,y) = \frac{\partial z}{\partial x}, \\
f_y(x,y) & = f_y = \frac{\partial f}{\partial y} = \frac{\partial}{\partial y} f(x,y) = \frac{\partial z}{\partial y}.
\end{align}

- Notation for higher-order partial derivatives:

$\displaystyle f_{xy} = \frac{\partial^2 f}{\partial y \ \partial x}$ indicates differentiation with respect to $x$ first, followed by differentiation with respect to $y$.

- In general (for continuous functions with continuous derivatives), the order of differentiation for higher-order partial derivatives does not matter.  For example, $f_{xxy} = f_{xyx} = f_{yxx}$.

### 8. Further Study

Please refer to Section 9.2 in the textbook for additional treatment of this topic.

### 9. Don't Forget

Don't forget to return to eClass to complete the pre-class quiz.


```python
import math
import numpy as np
import sympy as sym
```

$$
    \Psi_{k,l}(t) = - \frac{\partial^{l - 1}}{\partial t ^{l - 1}} \left( \prod_{j = 0, j \neq k}^a (\lambda_j + t)^{-r_j} \right)
    =
    \begin{cases}
        \frac{(-1)^{l} (l-1)!}{t^l (t + \lambda_2)^l}
        \times \sum_{j=1}^{l} {l \choose j} t^{l - j} \lambda_2 ^ {j - 1}, & k=1 \\
        - \frac{1}{t (t + \lambda_1)^{r_1}}, & k=2
    \end{cases}
$$

```python
def general_psi_function(arg, k, l, exp_rates, freq, a):
    t = sym.symbols("t")
    product = np.math.prod([(exp_rates[j] + t) ** (- freq[j]) for j in range(a + 1) if j != k])
    psi_val = - sym.diff(product, t, l - 1)
    psi_val =  psi_val.subs({t: arg})
    return psi_val

def specific_psi_function(arg, k, l, exp_rates, freq, a):
    psi_val = 0
    if k == 1:
        for term in range(1, l + 1):
            psi_val += math.comb(l, term) * arg ** (l - term) * exp_rates[2] ** (term - 1)
        psi_val *= (-1) ** l * math.factorial(l - 1) / (arg ** l * (arg + exp_rates[2]) ** l)
    if k == 2:
        psi_val = -1 / (arg * (arg + exp_rates[1]) ** freq[1])
    return psi_val
```

```python
lambda_1 = sym.Symbol("lambda_1")
lambda_2 = sym.Symbol("lambda_2")
r_2 = sym.Symbol("r_2")
t = sym.Symbol("t")

exp_rates = [sym.S(0), lambda_1, lambda_2]
freq = [1, r_2, 1]
a = len(exp_rates) - 1
```

```python
sym.factor(general_psi_function(arg=9, k=1, l=5, exp_rates=exp_rates, freq=freq, a=a))
```

$\displaystyle - \frac{8 \left(\lambda_{2}^{4} + 45 \lambda_{2}^{3} + 810 \lambda_{2}^{2} + 7290 \lambda_{2} + 32805\right)}{19683 \left(\lambda_{2} + 9\right)^{5}}$

```python
sym.factor(specific_psi_function(arg=9, k=1, l=5, exp_rates=exp_rates, freq=freq, a=a))
```

$\displaystyle - \frac{8 \left(\lambda_{2}^{4} + 45 \lambda_{2}^{3} + 810 \lambda_{2}^{2} + 7290 \lambda_{2} + 32805\right)}{19683 \left(\lambda_{2} + 9\right)^{5}}$

```python

```

# $\Large{\textbf{Need to show that}}$:
$$
    \Psi_{k,l}(t) = - \frac{\partial^{l - 1}}{\partial t ^{l - 1}} \left( \prod_{j = 0, j \neq k}^a (\lambda_j + t)^{-r_j} \right)
    =
    \begin{cases}
        \frac{(-1)^{l} (l-1)!}{t^l (t + \lambda_2)^l}
        \times \sum_{j=1}^{l} {l \choose j} t^{l - j} \lambda_2 ^ {j - 1}, & k=1 \\
        - \frac{1}{t (t + \lambda_1)^{r_1}}, & k=2
    \end{cases}
$$

$$
\vec{\lambda} = (0, \lambda_1, \lambda_2) \\
\vec{r} = (1, r_1, 1)
$$

Given our specific problem, we know that $k = 1,2$ and:
\begin{align}
    & k = 1 \quad \Rightarrow \quad l \in [1, n] \\
    & k = 2 \quad \Rightarrow \quad l = 1
\end{align}

Therefore by applying this to $\Psi_{k,l}(t)$ we can get the values of $\Psi(t)$ for $k=1$ and $k=2$:

# $\Large{\textbf{Proof for } k = 2}$:

 \begin{align}
    k=2, l=1 \qquad \Rightarrow \Psi_{2,1}(t) &= - \frac{\partial^0}{\partial t ^ 0} \left( \prod_{j = 0, j \neq k}^2 (\lambda_j + t)^{-r_j} \right) \\
    & = - (\lambda_0 + t)^{-r_0} (\lambda_1 + t)^{-r_1} \\
    & = - \frac{1}{t(t + \lambda_1)^{r_1}} \\
    & \hspace{4cm} \square
\end{align}

# $\Large{\textbf{Proof for } k = 1}$:

$$
    - \frac{\partial^{l - 1}}{\partial t ^{l - 1}} \left( \prod_{j = 0, j \neq k}^a (\lambda_j + t)^{-r_j} \right)
    = \frac{(-1)^{l} (l-1)!}{t^l (t + \lambda_2)^l} \times \sum_{j=1}^{l} {l \choose j} t^{l - j} \lambda_2 ^ {j - 1}
$$

\begin{align}
    LHS &= - \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \prod_{j = 0, j \neq k}^2 (\lambda_j + t)^{-r_j} \right) \\
    & = - \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( (\lambda_0 + t)^{-r_0} (\lambda_2 + t)^{-r_2} \right) \\
    & = - \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \frac{1}{t (t + \lambda_2)} \right) \\
    \\
    \\
    RHS &=  \frac{(-1)^{l} (l-1)!}{t^l (t + \lambda_2)^l} \times \sum_{j=1}^{l} {l \choose j} t^{l - j} \lambda_2 ^ {j - 1} \\
    & =  \frac{(-1)^{l} (l-1)!}{t^l \lambda_2 (t + \lambda_2)^l} \times \sum_{j=1}^{l} {l \choose j} \lambda_2^j t^{l - j} \\
    & =  \frac{(-1)^{l} (l-1)!}{t^l \lambda_2 (t + \lambda_2)^l} \times \left( \left[ \sum_{j=0}^{l} {l \choose j} \lambda_2^j t^{l - j}  \right] -t^l \right) \\
    * & =  \frac{(-1)^{l} (l-1)!}{t^l \lambda_2 (t + \lambda_2)^l} \times \left((t + \lambda_2)^l -t^l \right) \\
    & = \frac{(-1)^{l} (l-1)!}{\lambda_2} \left[\frac{1}{t^l} - \frac{1}{(t + \lambda_2)^l}\right] \\
\end{align}

### *Binomial Theorem:

$$
    (x + y)^n = \sum_{j=0}^n {n \choose j} x^{n-j} y^j
$$

## Need to show that:
$$
-\frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \frac{1}{t (t + \lambda_2)} \right) =
\frac{(-1)^{l} (l-1)!}{\lambda_2} \left[\frac{1}{t^l} - \frac{1}{(t + \lambda_2)^l}\right] \\
$$

## Proof by induction:

- Base Case: $l=1$:
    $$
        LHS = -\frac{\partial^{1-1}}{\partial t ^ {1-1}} \left( \frac{1}{t (t + \lambda_2)} \right) = - \frac{1}{t (t + \lambda_2)} \\
        RHS = \frac{(-1)^{1} (1-1)!}{\lambda_2} \left[\frac{1}{t^1} - \frac{1}{(t + \lambda_2)^1}\right] =
        - \frac{1}{\lambda_2} \left[\frac{t + \lambda_2 - t}{t (t + \lambda_2)}\right] = - \frac{\lambda_2}{\lambda_2 t (t + \lambda_2)} = - \frac{1}{t (t + \lambda_2)}
    $$

- Assume true for $l=x$:
    $$
        -\frac{\partial^{x-1}}{\partial t ^ {x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) =
        \frac{(-1)^{x} (x-1)!}{\lambda_2} \left[\frac{1}{t^x} - \frac{1}{(t + \lambda_2)^x}\right] \\
    $$

- Prove true for $l=x+1$:

    (Show that: $ -\frac{\partial^{x}}{\partial t^{x}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{(-1)^{x+1} (x)!}{\lambda_2} \left( \frac{1}{t^{x+1}} - \frac{1}{(t + \lambda_2)^{x+1}}\right)$)

    \begin{align}
        LHS &= -\frac{\partial^{x}}{\partial t^{x}} \left( \frac{1}{t (t + \lambda_2)} \right) \\
        &= \frac{\partial}{\partial t} \left[ -\frac{\partial^{x-1}}{\partial t^{x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) \right] \\
        &= \frac{\partial}{\partial t} \left[ \frac{(-1)^{x} (x-1)!}{\lambda_2} \left( \frac{1}{t^x} - \frac{1}{(t + \lambda_2)^x}\right) \right] \\
        &= \frac{(-1)^{x} (x-1)!}{\lambda_2} \frac{\partial}{\partial t} \left( \frac{1}{t^x} - \frac{1}{(t + \lambda_2)^x}\right) \\
        &= \frac{(-1)^{x} (x-1)!}{\lambda_2} \left( \frac{-x}{t^{x+1}} - \frac{-x}{(t + \lambda_2)^{x+1}}\right) \\
        &= \frac{(-1)^{x} (x-1)!}{\lambda_2} (-x) \left( \frac{1}{t^{x+1}} - \frac{1}{(t + \lambda_2)^{x+1}}\right) \\
        &= \frac{(-1)^{x+1} (x)!}{\lambda_2} \left( \frac{1}{t^{x+1}} - \frac{1}{(t + \lambda_2)^{x+1}}\right) \\
        &= RHS \\
        & \hspace{8cm}  \square
    \end{align}

```python

```

```python

```

- # Proof $k=1$ case by induction:
    $$
    - \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{(-1)^{l} (l-1)!}{t^l (t + \lambda_2)^l}
        \times \sum_{j=1}^{l} {l \choose j} t^{l - j} \lambda_2 ^ {j - 1}
    $$

    - Base Case: $l = 1$
        $$
            LHS = - \frac{\partial^{1-1}}{\partial t ^ {1-1}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{-1}{t(t + \lambda_2)} \\
            RHS = \frac{(-1)^1 (1-1)!}{t(t+\lambda_2)} \times {1 \choose 1} t^0 \lambda_2^0 = \frac{-1}{t(t + \lambda_2)}
        $$
    - Assume true for $l = x$:
        $$
            - \frac{\partial^{x-1}}{\partial t ^ {x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{(-1)^{x} (x-1)!}{t^x (t + \lambda_2)^x} \times \sum_{j=1}^{x} {x \choose j} t^{x - j} \lambda_2 ^ {j - 1}
        $$
    - Prove true for $l = x + 1$:
        $$
            - \frac{\partial^{x}}{\partial t ^ {x}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{\partial}{\partial t} \left[ \frac{\partial^{x-1}}{\partial t ^ {x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) \right]
            = \frac{(-1)^{x+1} x!}{t^{x+1} (t + \lambda_2)^{x+1}} \times \sum_{j=1}^{x+1} \, {{x + 1} \choose j} \, t^{x+1-j} \, \lambda_2^{j-1}
        $$

- $l = x+1$:
$$
    - \frac{\partial^{x}}{\partial t ^ {x}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{\partial}{\partial t} \left[ \frac{\partial^{x-1}}{\partial t ^ {x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) \right] \\
    = \frac{\partial}{\partial t} \left[ \frac{(-1)^{x} (x-1)!}{t^x (t + \lambda_2)^x} \times \sum_{j=1}^{x} {x \choose j} t^{x - j} \lambda_2 ^ {j - 1} \right] \\
    = \frac{\partial}{\partial t} \left[ u \times v \right] \\
    = \frac{\partial u}{\partial t} v + u \frac{\partial v}{\partial t}
$$

## Derivative of u

\begin{align}
    \frac{\partial u}{\partial t} &= \frac{\partial}{\partial t} \left[ \frac{(-1)^{x} (x-1)!}{t^x (t + \lambda_2)^x} \right] \\
    &= (-1)^x (x-1)! (-x) t^{-x-1} (t + \lambda_2)^{-x} + (-1)^x (x-1)! t^{-x} (-x) (t + \lambda_2)^{-x+1} \\
    &= (-1)^{x+1} x! \frac{2t + \lambda_2}{t^{x+1} (t + \lambda_2)^{x+1}}
\end{align}

## Derivative of v

\begin{align}
    \frac{\partial v}{\partial t} &= \frac{\partial}{\partial t} \left[ \sum_{j=1}^{x} \, {x \choose j} \, t^{x - j} \, \lambda_2 ^ {j - 1} \right] \\
    &= \frac{\partial}{\partial t} \left[ \sum_{j=1}^{x} \, \frac{x!}{(x-j)! j!} \, t^{x - j} \, \lambda_2 ^ {j - 1} \right] \\
    &= \sum_{j=1}^{x} \, \frac{x! (x - j)}{(x-j)! \, j!} \, t^{x - j - 1} \, \lambda_2 ^ {j - 1} \\
\end{align}

## Proof (cont'd)

\begin{align}
    =& \frac{\partial u}{\partial t} v + u \frac{\partial v}{\partial t} \\
    =& \frac{(-1)^{x+1} x!}{t^{x+1} (t + \lambda_2)^{x+1}} (2t + \lambda_2) \sum_{j=1}^{x} \frac{x!}{(x-j)! j!} t^{x-j} \lambda_2^{j - 1} \\
    & \qquad \qquad + \frac{(-1)^{x} (x-1)!}{t^x(t + \lambda_2)^{x}} \sum_{j=1}^x \frac{x!}{(x-j)! j!} t^{x-j-1} \lambda_2^{j-1} \\
    =& \frac{(-1)^{x+1} \, x!}{t^{x+1} (t + \lambda_2)^{x+1}} \left[ (2t+\lambda_2) \, \sum_{j=1}^x \, \frac{x!}{(x-j)! \, j!} \, t^{x-j} \, \lambda_2^{j-1}
    - \frac{t(t+\lambda_2)}{x} \, \sum_{j=1}^{x} \, \frac{x!(x-j)}{(x-j)! j!} \, t^{x-j-1} \, \lambda_2^{j-1}    \right]
\end{align}

#### ONLY LEFT TO SHOW THAT

$$
(2t+\lambda_2) \, \sum_{j=1}^x \, \frac{x!}{(x-j)! \, j!} \, t^{x-j} \, \lambda_2^{j-1} - \frac{t(t+\lambda_2)}{x} \, \sum_{j=1}^{x} \, \frac{x!(x-j)}{(x-j)! j!} \, t^{x-j-1} \, \lambda_2^{j-1} = \sum_{j=1}^{x+1} \, {{x + 1} \choose j} \, t^{x+1-j} \, \lambda_2^{j-1}
$$

```python
def LHS_demo(x, t, lambda_2):
    sum_1 = sum((math.factorial(x) * t ** (x - j) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))

    sum_2 = sum((math.factorial(x) * (x - j) * t ** (x - j - 1) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))

    sum_1 *= (2 * t + lambda_2)
    sum_2 *= (-t * (t + lambda_2)) / x

    return sum_1 + sum_2

def RHS_demo(x, t, lambda_2):
    sum_1 = sum((math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j)) for j in range(1, x+2))
    return sum_1
```

```python
sym.expand(LHS_demo(x=10, t=t, lambda_2=lambda_2))
```

$\displaystyle \lambda_{2}^{10} + 11 \lambda_{2}^{9} t + 55 \lambda_{2}^{8} t^{2} + 165 \lambda_{2}^{7} t^{3} + 330 \lambda_{2}^{6} t^{4} + 462 \lambda_{2}^{5} t^{5} + 462 \lambda_{2}^{4} t^{6} + 330 \lambda_{2}^{3} t^{7} + 165 \lambda_{2}^{2} t^{8} + 55 \lambda_{2} t^{9} + 11 t^{10}$

```python
sym.expand(RHS_demo(x=10, t=t, lambda_2=lambda_2))
```

$\displaystyle \lambda_{2}^{10} + 11 \lambda_{2}^{9} t + 55 \lambda_{2}^{8} t^{2} + 165 \lambda_{2}^{7} t^{3} + 330 \lambda_{2}^{6} t^{4} + 462 \lambda_{2}^{5} t^{5} + 462 \lambda_{2}^{4} t^{6} + 330 \lambda_{2}^{3} t^{7} + 165 \lambda_{2}^{2} t^{8} + 55 \lambda_{2} t^{9} + 11 t^{10}$

```python
for x in range(1,50):
    are_equal = sym.expand(LHS_demo(x=x, t=t, lambda_2=lambda_2)) == sym.expand(RHS_demo(x=x, t=t, lambda_2=lambda_2))
    print(are_equal, x)
```

    True 1
    True 2
    True 3
    True 4
    True 5
    True 6
    True 7
    True 8
    True 9
    True 10
    True 11
    True 12
    True 13
    True 14
    True 15
    True 16
    True 17
    True 18
    True 19
    True 20
    True 21
    True 22
    True 23
    True 24
    True 25
    True 26
    True 27
    True 28
    True 29
    True 30
    True 31
    True 32
    True 33
    True 34
    True 35
    True 36
    True 37
    True 38
    True 39
    True 40
    True 41
    True 42
    True 43
    True 44
    True 45
    True 46
    True 47
    True 48
    True 49

```python

```

```python

```

## Attempt 1:

$$
    \sum_{j=1}^{x} \frac{x!}{(x-j)! j!} t^{x-j} \lambda_2^{j-1} = \left[ \sum_{j=1}^{x+1} \frac{x! j}{(x-j+1)! j!} t^{x-j+1} \lambda_2^{j-2} \right] - \frac{t^x}{\lambda_2}
$$

```python
def RHS_new_demo(x, t, lambda_2):
    sum_1 = sum((math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j)) for j in range(1, x+2))
    return sum_1
```

```python
def LHS_new_demo(x, t, lambda_2):
    sum_1 = sum((math.factorial(x) * t ** (x - j) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))
    return sum_1

def RHS_new_demo(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x + 2):
        to_add = j*(math.factorial(x) * t ** (x + 1 - j) * lambda_2 ** (j - 2)) / (math.factorial(x + 1 - j) * math.factorial(j))
        sum_1 += to_add
    sum_1 -= t ** x / lambda_2
    return sum_1
```

```python
sym.simplify(LHS_new_demo(5, t, lambda_2))
```

$\displaystyle \lambda_{2}^{4} + 5 \lambda_{2}^{3} t + 10 \lambda_{2}^{2} t^{2} + 10 \lambda_{2} t^{3} + 5 t^{4}$

```python
sym.expand(RHS_new_demo(5, t, lambda_2))
```

$\displaystyle \lambda_{2}^{4} + 5 \lambda_{2}^{3} t + 10 \lambda_{2}^{2} t^{2} + 10 \lambda_{2} t^{3} + 5 t^{4}$

```python

```

```python

```

## Attempt 2:

$$

$$

```python
def LHS_demo_3(x, t, lambda_2):
    sum_1 = sum((math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j)) for j in range(1, x+2))
    return sum_1

def RHS_demo_3(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x + 1):
        to_add = (math.factorial(x) * t ** (x - j) * lambda_2 ** (j - 1) * (2 * t + lambda_2)) / (math.factorial(x - j) * math.factorial(j))
        to_add -= (math.factorial(x) * (x - j) * t ** (x - j - 1) * lambda_2 ** (j - 1) * (t ** 2 + t * lambda_2) / x) / (math.factorial(x - j) * math.factorial(j))
        sum_1 += to_add
    return sum_1

def RHS_demo_4(x, t, lambda_2):
    sum_1 = sum((math.factorial(x - 1) * t ** (x - j) * lambda_2 ** (j - 1)) * (x * t + j * t + lambda_2 * j) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))
    return sum_1
```

```python
sym.simplify(LHS_demo_3(7, t, lambda_2))
```

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python
sym.simplify(RHS_demo_4(7, t, lambda_2))
```

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python

```

```python

```

## Attempt 3:

$$
    - \frac{t^{x+1}}{\lambda_2} + \sum_{j=1}^{x+1} \, \frac{x!}{(x-j+1)! j!} \, t^{x+1-j} \, \lambda_2^{j-1} \left( \frac{xjt + tj^2 -tj +\lambda_2 j^2 - \lambda_2 j}{x \lambda_2} \right)= \sum_{j=1}^{x+1} \, {{x + 1} \choose j} \, t^{x+1-j} \, \lambda_2^{j-1}
$$

```python
def LHS_demo_5(x, t, lambda_2):
    sum_1 = sum((math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j)) for j in range(1, x+2))
    return sum_1

def RHS_demo_5(x, t, lambda_2):
    sum_1 = sum((math.factorial(x) * t ** (x + 1 - j) * lambda_2 ** (j - 1) * (x*j*t + t*j**2 - t*j + lambda_2 * j**2 - lambda_2 * j)) / (x * lambda_2 * math.factorial(x + 1 - j) * math.factorial(j)) for j in range(2, x+2))
    return sum_1 #- t ** (x+1) / lambda_2

# def RHS_demo_5(x, t, lambda_2):
#     sum_1 = sum((math.factorial(x) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))
#     sum_2 = sum((math.factorial(x) * j * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))
#     return sum_1 + sum_2 * (t + lambda_2) / (t * x)
```

```python
sym.expand(LHS_demo_5(7, t, lambda_2))
```

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python
sym.expand(RHS_demo_5(7, t, lambda_2))
```

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python

```

```python

```

```python

```

## Attempt 4

$$

\frac{xjt + tj^2 - tj + \lambda_2 j^2 - \lambda_2 j}{x \lambda_2 (x + 1)} - \frac{(x-j+1)! j!}{(x+1)!} \frac{t^j}{\lambda_2^j} = 1

$$

```python
from sympy.printing.latex import LatexPrinter, print_latex
```

```python
def LHS_demo_5(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+2):
        term = (math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j))
        print(term)
        sum_1 += term
    return sum_1

def RHS_demo_5(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+2):
        extra_term = (x*j*t + t*j**2 - t*j + lambda_2 * j**2 - lambda_2 * j) / (x * lambda_2)
        extra_term -= (math.factorial(x - j + 1) * math.factorial(j) * t ** (x + 1)) / (math.factorial(x+1) * t ** (x + 1 - j) * lambda_2 ** j)
        extra_term /= (x + 1)
        term = (math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1) * extra_term) / (math.factorial(x + 1 - j) * math.factorial(j))
        print(term)
        sum_1 += term
    return sum_1
```

```python
sym.expand(LHS_demo_5(7, t, lambda_2))
```

    8*t**7
    28*lambda_2*t**6
    56*lambda_2**2*t**5
    70*lambda_2**3*t**4
    56*lambda_2**4*t**3
    28*lambda_2**5*t**2
    8*lambda_2**6*t
    lambda_2**7

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python
sym.expand(RHS_demo_5(7, t, lambda_2))
```

    7*t**8/(8*lambda_2)
    28*lambda_2*t**6*((2*lambda_2 + 16*t)/(56*lambda_2) - t**2/(224*lambda_2**2))
    56*lambda_2**2*t**5*((6*lambda_2 + 27*t)/(56*lambda_2) - t**3/(448*lambda_2**3))
    70*lambda_2**3*t**4*((12*lambda_2 + 40*t)/(56*lambda_2) - t**4/(560*lambda_2**4))
    56*lambda_2**4*t**3*((20*lambda_2 + 55*t)/(56*lambda_2) - t**5/(448*lambda_2**5))
    28*lambda_2**5*t**2*((30*lambda_2 + 72*t)/(56*lambda_2) - t**6/(224*lambda_2**6))
    8*lambda_2**6*t*((42*lambda_2 + 91*t)/(56*lambda_2) - t**7/(64*lambda_2**7))
    lambda_2**7*((56*lambda_2 + 112*t)/(56*lambda_2) - t**8/(8*lambda_2**8))

$\displaystyle \lambda_{2}^{7} + 8 \lambda_{2}^{6} t + 28 \lambda_{2}^{5} t^{2} + 56 \lambda_{2}^{4} t^{3} + 70 \lambda_{2}^{3} t^{4} + 56 \lambda_{2}^{2} t^{5} + 28 \lambda_{2} t^{6} + 8 t^{7}$

```python

```

```python

```

```python

```

# Going back:

## Assuming true for $l=x$ prove true for $l=x+1$
\begin{align}
    & - \frac{\partial^{x}}{\partial t ^ {x}} \left( \frac{1}{t (t + \lambda_2)} \right) \\
    & = \frac{\partial}{\partial t} \left[ \frac{\partial^{x-1}}{\partial t ^ {x-1}} \left( \frac{1}{t (t + \lambda_2)} \right) \right] \\
    & = \frac{\partial}{\partial t} \left[ \frac{(-1)^{x} (x-1)!}{t^x (t + \lambda_2)^x} \times \sum_{j=1}^{x} {x \choose j} t^{x - j} \lambda_2 ^ {j - 1} \right] \\
    & = \frac{\partial}{\partial t} \left[ \sum_{j=1}^{x} \frac{(-1)^{x} (x-1)!}{t^x (t + \lambda_2)^x} \times {x \choose j} t^{x - j} \lambda_2 ^ {j - 1} \right] \\
    & =  \sum_{j=1}^{x} \left[ \frac{\partial}{\partial t} \frac{(-1)^{x} (x-1)!}{(t + \lambda_2)^x} \times {x \choose j} t^{- j} \lambda_2 ^ {j - 1} \right] \\
    & =  \sum_{j=1}^{x} \left[ (-1)^{x} (x-1)! {x \choose j} \lambda_2 ^ {j - 1} \frac{\partial}{\partial t} \left( \frac{t^{- j}}{(t + \lambda_2)^x} \right) \right] \\
    & = \sum_{j=1}^{x} \left[ (-1)^{x} (x-1)! {x \choose j} \lambda_2 ^ {j - 1} \left( \frac{(-1)(jt + j \lambda_2 + xt)}{t^{j+1}(t + \lambda_2)^{x+1}} \right) \right] \\
    & = \frac{(-1)^{x+1} (x-1)!}{(t + \lambda_2)^{x+1}} \sum_{j=1}^{x} \left[ {x \choose j} \lambda_2 ^ {j - 1} t^{-j-1} (jt + j \lambda_2 + xt) \right] \\
\end{align}

```python
def LHS_demo(x, t, lambda_2):
    sum_1 = sum((math.factorial(x) * j * t ** (- j - 1) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))
    sum_2 = sum((math.factorial(x) * t ** (- j) * lambda_2 ** (j - 1)) / (math.factorial(x - j) * math.factorial(j)) for j in range(1, x+1))

    sum_1 *= (-1) ** (x+1) * math.factorial(x - 1) / (t + lambda_2) ** x
    sum_2 *= (-1) ** (x+1) * math.factorial(x) / (t + lambda_2) ** (x + 1)

    return sum_1 + sum_2

def LHS_demo(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+2):
        term = math.factorial(x) * t ** (- j) * lambda_2 ** (j - 2) * ((j - 1) * (t + lambda_2) + x*t) / (math.factorial(x + 1 - j) * math.factorial(j - 1))
        # if j > 1:
        print(term)
        sum_1 += term
    sum_1 -= x / lambda_2
    sum_1 *= (-1) ** (x+1) * math.factorial(x - 1) / (t + lambda_2) ** (x + 1)

    return sum_1

def RHS_demo(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+2):
        term = (math.factorial(x + 1) * t ** (x + 1 - j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j))
        print(term)
        sum_1 += term
    sum_1 *= ((-1)**(x+1) * math.factorial(x)) / (t ** (x+1) * (t + lambda_2) ** (x + 1))
    return sum_1
```

```python
sym.factor(RHS_demo(2, t, lambda_2))
```

    3*t**2
    3*lambda_2*t
    lambda_2**2

$\displaystyle - \frac{2 \left(\lambda_{2}^{2} + 3 \lambda_{2} t + 3 t^{2}\right)}{t^{3} \left(\lambda_{2} + t\right)^{3}}$

```python
sym.factor(LHS_demo(2, t, lambda_2))
```

    2/lambda_2
    2*(lambda_2 + 3*t)/t**2
    lambda_2*(2*lambda_2 + 4*t)/t**3

$\displaystyle - \frac{2 \left(\lambda_{2}^{2} + 3 \lambda_{2} t + 3 t^{2}\right)}{t^{3} \left(\lambda_{2} + t\right)^{3}}$

```python

```

```python

```

```python

```

```python

```

```python

```

$$
    \frac{(-1)^{x+1} (x-1)!}{(t + \lambda_2)^{x+1}} \sum_{j=1}^{x} \left[ {x \choose j} \lambda_2 ^ {j - 1} t^{-j-1} (jt + j \lambda_2 + xt) \right] = \frac{(-1)^{x+1} x!}{(t + \lambda_2)^{x+1}} \times \sum_{j=1}^{x+1} \, {{x + 1} \choose j} \, t^{-j} \, \lambda_2^{j-1} \\
    \frac{(-1)^{x+1} x!}{(t + \lambda_2)^{x+1}} \times \sum_{j=1}^{x+1} \, {{x + 1} \choose j} \, t^{-j} \, \lambda_2^{j-1} - \frac{(-1)^{x+1} (x-1)!}{(t + \lambda_2)^{x+1}} \sum_{j=1}^{x} \left[ {x \choose j} t^{-j-1} \lambda_2 ^ {j - 1} (jt + j \lambda_2 + xt) \right] = 0 \\

\vdots \\
\vdots \\

\sum_{j=1}^{x} \left[ \frac{(x-1)! \, t^{-j} \, \lambda_2^{j-1} \, (j-1)}{(x+1-j)! \, (j-1)!} - \frac{(x-1)! \, t^{-j-1} \lambda_2^j}{(x-j)! (j-1)!} \right] + \frac{\lambda_2^x}{t^{x+1}} = 0 \\

\vdots \\
\vdots \\

(t + \lambda_2) \sum_{j=2}^{x} j \, \frac{(x-1)! \, t^{-j-1} \, \lambda_2^{j-1}}{(x+1-j)! \, (j-1)!} - (t + \lambda_2 + \lambda_2 x) \sum_{j=2}^{x} \, \frac{(x-1)! \, t^{-j-1} \, \lambda_2^{j-1}}{(x+1-j)! \, (j-1)!} = \frac{\lambda_2}{t^2} - \frac{\lambda_2^x}{t^{x+1}}
$$

```python
def LHS_demo(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+2):
        term = (math.factorial(x + 1) * t ** (- j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j))
        sum_1 += term
    sum_1 *= ((-1)**(x+1) * math.factorial(x)) / ((t + lambda_2) ** (x + 1))
    return sum_1

def RHS_demo(x, t, lambda_2):
    sum_1 = 0
    for j in range(1, x+1):
        term = (math.factorial(x) * t ** (- j - 1) * lambda_2 ** (j - 1) * (j*t + j*lambda_2 + x*t)) / (math.factorial(x - j) * math.factorial(j))
        sum_1 += term
    sum_1 *= ((-1)**(x+1) * math.factorial(x - 1)) / ((t + lambda_2) ** (x + 1))
    return sum_1
```

```python
def diff_demo_2(x, t, lambda_2):
    summ = 0
    for j in range(2, x+1):
        summ += (math.factorial(x - 1) * t ** (-j) * lambda_2 ** (j - 1)) / (math.factorial(x + 1 - j) * math.factorial(j - 2))
        summ -= math.factorial(x-1) * (t ** (- j - 1) * lambda_2 ** j) / (math.factorial(x-j) * math.factorial(j - 1))

    summ -= lambda_2 / (t ** 2)
    summ += lambda_2 ** x / (t ** (x + 1))

    return summ

def diff_demo(x, t, lambda_2):
    return LHS_demo(x, t, lambda_2) - RHS_demo(x, t, lambda_2)

def diff_demo(x, t, lambda_2):
    summ = 0
    for j in range(1, x+1):
        term = math.factorial(x - 1) * t ** (-j - 1) * lambda_2 ** (j - 1)

        # term *= (x*(x+1)*t - (x+1-j)*(j*(t+lambda_2) + t*x))
        term *= (j*t*(j-1) - j*lambda_2*(x+1-j))

        term /= (math.factorial(x + 1 - j) * math.factorial(j))
        summ += term

    summ += lambda_2 ** x / (t ** (x + 1))
    return summ

def diff_demo(x, t, lambda_2):
    summ = 0
    for j in range(1, x+1):
        term_1 = math.factorial(x - 1) * t ** (-j) * lambda_2 ** (j - 1)
        term_1 *= (j-1)
        term_1 /= (math.factorial(x + 1 - j) * math.factorial(j - 1))

        term_2 = math.factorial(x - 1) * t ** (-j - 1) * lambda_2 ** j
        term_2 *= 1
        term_2 /= (math.factorial(x - j) * math.factorial(j - 1))

        summ += term_1 - term_2

    summ += lambda_2 ** x / (t ** (x + 1))
    return summ
```

```python
sym.expand(diff_demo(x=5, t=t, lambda_2=lambda_2))
```

$\displaystyle 0$

```python
sym.expand(diff_demo_2(x=10, t=t, lambda_2=lambda_2))
```

$\displaystyle 0$

```python

```

```python

```

## Attempt Using Taylor Series

$$
    \frac{-1}{t (t + \lambda_2)} = \sum_{n=l-1}^{\infty} \left[ \frac{-1}{t_0 (t_0 + \lambda_2)} \right]^{(n)} \frac{(t - t_0)^n}{n!}
$$

$$
\begin{align}
    \Psi_{1,l}(t) & = \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \frac{-1}{t (t + \lambda_2)} \right) \\
    & = \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \sum_{n=0}^{\infty} \left[ \frac{-1}{t_0 (t_0 + \lambda_2)} \right]^{(n)} \frac{(t - t_0)^n}{n!} \right) \\
    & = \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \sum_{n=l-1}^{\infty} \left[ \frac{-1}{t_0 (t_0 + \lambda_2)} \right]^{(n)} \frac{(t - t_0)^n}{n!} \right) \\
\end{align}
$$

```python

```

```python

```

```python

```

```python
lambda_1 = sym.Symbol("lambda_1")
lambda_2 = sym.Symbol("lambda_2")
r_2 = sym.Symbol("r_2")
t = sym.Symbol("t")

exp_rates = [sym.S(0), lambda_1, lambda_2]
freq = [1, r_2, 1]
a = len(exp_rates) - 1
```

```python
t = 3
lambda_2 = 5
```

# To prove:
$$
    - \frac{\partial^{l-1}}{\partial t ^ {l-1}} \left( \frac{1}{t (t + \lambda_2)} \right) = \frac{(-1)^{l} (l-1)!}{(t + \lambda_2)^{l}} \times \sum_{j=1}^{l} \, {l \choose j} \, t^{-j} \, \lambda_2^{j-1}
$$

# Attempt by computing the $n^{th}$ derivative

### Leibniz formula for $n^{th}$ derivative:
$$
    (u(x)v(x))^{(n)} = \sum_{i=1}^{n} {n \choose i} u^{i} v^{n-i}
$$

```python
def RHS_demo(arg, k, l, exp_rates, freq, a):
    psi_val = 0
    for term in range(1, l + 1):
        psi_val += math.comb(l, term) * arg ** (- term) * exp_rates[2] ** (term - 1)
    psi_val *= (-1) ** l * math.factorial(l - 1) / ((arg + exp_rates[2]) ** l)
    return psi_val

def LHS_demo(arg, k, l, exp_rates, freq, a):
    psi_val = 0
    for term in range(1, l + 1):
        psi_val += (arg + exp_rates[2]) ** (term - 1) / (arg ** (term))
    psi_val *= (-1) ** l * math.factorial(l - 1) / ((arg + exp_rates[2]) ** l)
    return psi_val

# def LHS_demo(arg, k, l, exp_rates, freq, a):
#     psi_val = 0
#     for term in range(1, l + 1):
#         for r in range(term):
#             psi_val += math.comb(term-1, r) * arg ** (-r-1) * exp_rates[2] ** r
#     psi_val *= (-1) ** l * math.factorial(l - 1) / ((arg + exp_rates[2]) ** l)
#     return psi_val
```

```python
sym.factor(RHS_demo(arg=t, k=1, l=5, exp_rates=[sym.S(0), lambda_1, lambda_2], freq=[1, r_2, 1], a=2))
```

$\displaystyle - \frac{24 \left(\lambda_{2}^{4} + 5 \lambda_{2}^{3} t + 10 \lambda_{2}^{2} t^{2} + 10 \lambda_{2} t^{3} + 5 t^{4}\right)}{t^{5} \left(\lambda_{2} + t\right)^{5}}$

```python
sym.factor(LHS_demo(arg=t, k=1, l=5, exp_rates=[sym.S(0), lambda_1, lambda_2], freq=[1, r_2, 1], a=2))
```

$\displaystyle - \frac{24 \left(\lambda_{2}^{4} + 5 \lambda_{2}^{3} t + 10 \lambda_{2}^{2} t^{2} + 10 \lambda_{2} t^{3} + 5 t^{4}\right)}{t^{5} \left(\lambda_{2} + t\right)^{5}}$

```python

```

```python

```

## More specific

```python
def RHS_demo(t, l, lambda_2):
    summ = 0
    for i in range(1, l + 1):
        summ += math.comb(l, i) * t ** (-i) * lambda_2 ** (i-1)
    return summ

def LHS_demo(t, l, lambda_2):
    summ = 0
    for i in range(1, l + 1):
        summ += (t + lambda_2) ** (i - 1) / (t ** i)
    return summ

def LHS_demo(t, l, lambda_2):
    return ((t + lambda_2) ** l - t ** l) / (t**l * lambda_2)
```

```python
sym.factor(RHS_demo(t, 4, lambda_2))
```

$\displaystyle \frac{\left(\lambda_{2} + 2 t\right) \left(\lambda_{2}^{2} + 2 \lambda_{2} t + 2 t^{2}\right)}{t^{4}}$

```python
sym.factor(LHS_demo(t, 4, lambda_2))
```

$\displaystyle \frac{\left(\lambda_{2} + 2 t\right) \left(\lambda_{2}^{2} + 2 \lambda_{2} t + 2 t^{2}\right)}{t^{4}}$

```python

```

nbs/src/Markov/markov-proportion-formula/proof of hypoexponential closed-form.ipynb


```python
%matplotlib inline

import sys
sys.path.insert(0, '../')

from AircraftIden import FreqIdenSIMO, TransferFunctionFit
import numpy as np

import matplotlib.pyplot as plt
import math
import pickle
from os.path import basename,splitext

plt.rc('figure', figsize=(10.0, 5.0))

def lat_dyn_freq(fn,show_freq_iden_plots=True):
    name = splitext(basename(fn))[0]
    arr = np.load(fn)
    print("Start analyse case {}".format(name))
    time_seq = arr[:, 0]
    print("Time up to {}:{}".format(time_seq[0],time_seq[-1]))
    ele_seq = arr[:, 1]
    q_seq = arr[:, 4]
    vvi_seq = arr[:, 6]
    theta_seq = arr[:, 2] / 180 * math.pi
    airspeed_seq = arr[:, 3]
    aoa_seq = arr[:, 5] / 180 * math.pi

    vx_seq = arr[:, 8]
    vy_seq = arr[:, 9]
    vz_seq = arr[:, 10]

    ax_source = arr[:, 11].copy()
    ax_seq = arr[:, 11]  # + np.sin(theta_seq) * 9.8
    # ay_seq = arr[:, 12]
    az_seq = arr[:, 13]
    print("U0 {} W0 {} th0 {}".format(vx_seq[0],vz_seq[0],theta_seq[0]))
    # X = [u,w,q,th]
    # Y = [w,q,th,ax,az]
    # Note ax ay contain gravity acc
    simo_iden = FreqIdenSIMO(time_seq, 1, 20, q_seq,vx_seq, vz_seq)
    freqres = simo_iden.get_freqres()
    output_path = "../data/{}_freqres.pkl".format(name)
    with open(output_path, 'wb') as output:
        pickle.dump(simo_iden, output, pickle.HIGHEST_PROTOCOL)
        print("Saved Freq Res to {}".format(output_path))

    if show_freq_iden_plots:
        fig = plt.figure("source data")
        fig.set_size_inches(18, 10)
        plt.subplot(311)
        # plt.plot(time_seq, aoa_seq * 180 / math.pi, label='aoa')
        plt.plot(time_seq, theta_seq * 180 / math.pi, label='theta')
        # plt.plot(time_seq, az_seq - az_seq[0], label='az - {:2.1f}'.format(az_seq[0]))
        plt.legend()

        plt.subplot(312)
        plt.plot(time_seq, vz_seq, label='vz')
        # plt.plot(time_seq, vx_seq - vx_seq[0], label='vx - {:3.1f}'.format(vx_seq[0]))
        plt.legend()

        plt.subplot(313)
        # plt.plot(time_seq, q_seq, label='q')
        # plt.plot(time_seq, theta_seq*57, label='theta')
        plt.plot(time_seq, ax_seq, label='ax')
        plt.plot(time_seq, - np.sin(theta_seq) * 9.8, label='ax_by_theta')
        plt.plot(time_seq, ax_source, label='axsource')
        plt.plot(time_seq, aoa_seq * 50, label='aoa')
        plt.grid(which='both')
        # plt.plot(time_seq, ele_seq, label="ele")

        plt.legend()
        plt.figure("Q->U(vx)")
        simo_iden.plt_bode_plot(0)
        plt.figure("Q->W(vz)")
        simo_iden.plt_bode_plot(1)
        plt.show()

    return simo_iden

fn = "../data/sweep_data_2017_12_10_19_05.npy"
freqres = lat_dyn_freq(fn,show_freq_iden_plots=True)
%store freqres
```

```python
import sys
sys.path.insert(0, '../')
from AircraftIden import FreqIdenSIMO, TransferFunctionFit
import math
import matplotlib.pyplot as plt

%store -r
# X = [u,w,th]
# Y = [u,w]
import sympy as sp
from AircraftIden.StateSpaceIden import StateSpaceIdenSIMO, StateSpaceParamModel

M = sp.Matrix([[1, 0, 0],
               [0, 1 , 0],
               [0, 0, 1]])

# Tailsittel
#    0---Vx, W
#--- | ----
# |  |  |
#    |
#  Vz ,-U
# 10m/s

# Vz trim is -6.05,Vz is negative U
# Vx trim is 4.55, Vx is W
g = 9.78

Xu, Xw, Xq = sp.symbols('Xu Xw Xq')
Zu, Zw, Zq = sp.symbols('Zu Zw Zq')
Mu, Mw, Mq = sp.symbols('Mu Mw Mq')

def callback(xk,state):
    print(xk)
    print(state)

def process_ssm(freqres,trims):
    th0 = trims["theta"]
    W0 = trims["W0"]
    U0 = trims["U0"]
    F = sp.Matrix([[Xu, Xw, -g * math.cos(th0)],
               [Zu, Zw, -g * math.sin(th0)],
               [0, 0, 0]])
    G = sp.Matrix([[Xq-W0],[Zq+U0],[1]])
    # direct using u w q for y
    #U equal to negative u
    H0 = sp.Matrix([
        [1, 0, 0],
        [0, 1, 0]])
    H1 = sp.Matrix([
        [0, 0, 0],
        [0, 0, 0],
    ])
    syms = [Xu, Xw,Zu, Zw,Zq,Xq]
    LatdynSSPM = StateSpaceParamModel(M, F, G, H0, H1, syms)

    plt.rc('figure', figsize=(10.0, 5.0))
    freqres = freqres.get_freqres(indexs = [0,1])
    ssm_iden = StateSpaceIdenSIMO(freqres, accept_J=20,
                              enable_debug_plot=False,
                              y_names=['U',"w"],reg=0, cpu_use = 32, max_sample_times=20)
    J, ssm = ssm_iden.estimate(LatdynSSPM, syms, constant_defines={},rand_init_max = 10)
    ssm.check_stable()
    ssm_iden.draw_freq_res()
    ssm_iden.print_res()
```

```python
process_ssm(freqres,{
    "theta":0,
    "U0":64.24,
    "W0":-1.14
    })
```

```python
def process_status_freqres(freqres, i):
    freq, H, gamma2, gxx, gxy, gyy = freqres.get_freq_iden(i)
    a,b,c,d,tau,s = sp.symbols("a b c d tau s")

    num = d
    den = s*(b *s + c)
    tfpm = TransferFunctionParamModel(num,den,tau)
    fitter = TransferFunctionFit(freq, H, gamma2, tfpm,nw=20,iter_times=1000,reg = 0)
    tf = fitter.estimate(10,accept_J=20)
    return fitter

plt.rc('figure', figsize=(20.0, 10.0))
def plot_fitter(fitter, status_name):
    plt.figure(status_name)
    fitter.plot(status_name+":")
    trans_str = fitter.latex()
    plt.text(20, 0,trans_str, fontsize=20,color='red')
    plt.show()

def poly_latex(poly,cha = "s"):
    ret_str = ""
    ords = len(poly) - 1
    for i in range(ords+1):
        ordn = ords - i
        if ordn == 0:
            ret_str = ret_str + "{:4.2f}".format(poly[i])
        else:
            if (poly[i] == 1):
                ret_str = ret_str + "s^{:d}+".format(ordn)
            else:
                ret_str = ret_str + "{:4.2f} s^{:d}+".format(poly[i],ordn)
    return ret_str

def transfer_func_latex(num,den,tau):
    return r"$\frac{" + poly_latex(num) + "}{" + poly_latex(den) +"}" + "e^{-" + "{:4.3f}".format(tau) + "t}$"
```


# QuTiP Example: Clifford Group Elements

[Christopher Granade](http://www.cgranade.com/)

University of Sydney
$\newcommand{\ket}[1]{\left|#1\right\rangle}$
$\newcommand{\bra}[1]{\left\langle#1\right|}$
$\newcommand{\cnot}{{\scriptstyle \rm CNOT}}$
$\newcommand{\Tr}{\operatorname{Tr}}$

## Introduction

In this notebook, we will demonstrate the ``qubit_clifford_group`` generator, which yields each element of the Clifford group on a single qubit.

## Preamble

### Features

We enable a few features such that this notebook runs in both Python 2 and 3.

```python
from __future__ import division, print_function
```

### Imports

```python
import numpy as np
import qutip as qt

from qutip.ipynbtools import version_table
from IPython.display import display
```

### Plotting Support

```python
%matplotlib inline
```

### Settings

```python
qt.settings.colorblind_safe = True
```

## Clifford Group ##

The Clifford group $\mathcal{C}_n$ on $n$ qubits is the group of automorphisms of the Pauli group $\mathcal{P}_n$, $\mathcal{C}_n = \{U | \forall P \in \mathcal{P}_n: U P U^\dagger \in \mathcal{P}_n\}$, and is generated by the CNOT, Hadamard and phase gates. The Clifford group is very useful in a number of different contexts in quantum information such as error correction and randomized benchmarking. In particular, the Clifford group is often of interest because the Gottesman-Knill theorem gives that a circuit made up only of Clifford gates, Pauli preparations and Pauli measurements can be efficiently simulated with classical resources. This simulation proceeds by representing Clifford group elements by their action on a generating set for the Pauli group--- this representation requires only polynomially many classical bits with the number of qubits.

At times, however, we will need representations of Clifford group elements as unitary operators on the full Hilbert space. This is useful, for instance, in reasoning about how *imperfect* Clifford group elements behave. QuTiP provides a generator, ``qubit_clifford_group``, that iterates over the elements of the Clifford group on a single qubit.

```python
cliffords = list(qt.qubit_clifford_group())
```

This generator works by using the decomposition exploited by [Ross and Selinger](https://hackage.haskell.org/package/newsynth) in their circuit synthesis package. In particular, they use that the single-qubit Clifford group can be characterized as
\begin{equation}
    \mathcal{C}_1 = \left\{
        \omega^i E^j X^k S^\ell |
        i \in \operatorname{range}(8),\
        j \in \operatorname{range}(3),\
        k \in \operatorname{range}(2),\
        \ell \in \operatorname{range}(4)
    \right\},
\end{equation}
where $\omega = \mathrm{e}^{2\pi \mathrm{i} / 8}$, $S$ is the $\pi / 2$ phase gate, $E = \omega^3 H S^3$, $H$ is the Hadamard gate and $X$ is the familiar Pauli matrix. The function ``qubit_clifford_group`` yields this characterization, save for the iteration over powers of ``\omega``.

```python
display("S", cliffords[1])
display("X", cliffords[4])
display("E", cliffords[8])
```

    'S'

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = False\begin{equation*}\left(\begin{array}{*{11}c}1.0 & 0.0\\0.0 & 1.0j\\\end{array}\right)\end{equation*}

    'X'

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 1.0\\1.0 & 0.0\\\end{array}\right)\end{equation*}

    'E'

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = False\begin{equation*}\left(\begin{array}{*{11}c}(-0.500+0.500j) & (0.500+0.500j)\\(-0.500+0.500j) & (-0.500-0.500j)\\\end{array}\right)\end{equation*}

These three gates can be understood in terms of their actions on the Pauli group. To demonstrate, we plot the superoperator representations for each of $S$, $X$ and $E$ in the Pauli basis as Hinton diagrams.

```python
qt.visualization.hinton(qt.to_super(cliffords[1]));
qt.visualization.hinton(qt.to_super(cliffords[4]));
qt.visualization.hinton(qt.to_super(cliffords[8]));
```

## Epilouge

```python
version_table()
```

Software	Version

QuTiP	4.2.0

Numpy	1.13.1

SciPy	0.19.1

matplotlib	2.0.2

Cython	0.25.2

Number of CPUs	2

BLAS Info	INTEL MKL

IPython	6.1.0

Python	3.6.1 |Anaconda custom (x86_64)| (default, May 11 2017, 13:04:09)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]

OS	posix [darwin]

Wed Jul 19 22:10:30 2017 MDT

```python

```


# Eigenvalue Approximation by Ellipse Equation

Given an eigenvalue pair $\Gamma_{l} = \{a_{2l}, a_{2l+2}\}$, we approximate the coalescing of the pair at EPs as

* Diffusive regime:

\begin{equation}
\Gamma_{l} = \gamma_{l} \pm \frac{\Delta_{l}}{|q_{l}|}\sqrt{|q_{l}|^2 - 4k^2 Pe^2}, \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; 2kPe < |q_{l}|
\end{equation}

* Dispersive regime:

\begin{equation}
\Gamma_{l} = \pm \frac{\Delta_{l}}{|q_{l}|}\sqrt{|q_{l}|^2 - 4k^2 Pe^2}, \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;  |q_{l}| < 2kPe < 2|q_{l}|
\end{equation}

where:
* $q_{l}$ is the Exceptional Point (EP) or the eigenvalue pair.
* $\Delta_{l} = 8l + 2$ is the difference between the eigenvalues that define the pair $\Gamma_{l}$, evaluated at $q=0$.
* $\gamma_{n} = 16l^2 + 8l + 2$ centers the ellipse between the eigenvalue pair at the line $q=0$.

For example, we have for the first three eigenvalue pairs: $q_{0}=1.468i$, $q_{1}=16.471i$, $q_{2}=47.806i$ (in manuscript, these are $q_{l}^{EP}$), $\Delta_{0}=2$, $\Delta_{1}=10$, $\Delta_{2}=18$, and $\gamma_{0}=2$ and $\gamma_{1}=26$ and $\gamma_{2}=82$.

### NOTE:
The approximation above describes a purely real eigenvalue pair before the Exceptional Point, or a purely imaginary pair of (complex conjugated) Eigenvalues for values of q past the Exceptional Point. Thus, we need to add extra terms for values beyond the EPs.
### Solution:
We need to add a correction associated with the asymptotic limit $|q|\rightarrow\infty$. For the first two pairs of (coalesced) eigenvalues, the asymptotic relations are

\begin{align}
\Gamma_{0} \sim & \sqrt{2|q|}-\frac{1}{4} \pm i\left(\sqrt{2|q|}-2|q| \right) , \\
 \Gamma_{2} \sim & 5\sqrt{2|q|}-\frac{13}{4} \pm i\left(5\sqrt{2|q|} - 2|q| \right) ,
\end{align}

where $|q|=2kPe$ is the absolute value of Mathieu's canonical parameter. The correction only involves the real component above.

* In general (from Zeiner et al 2012), a good leading order approximation of eigenvalue pairs is

\begin{equation}
    \Gamma_{l} \approx \left[4l +1 \right]\left[1+i \right]\sqrt{2|q|} - 2q - 2l^2 -l -1/4, \;\;\;\text{for}\;\; q{>>}q_{l}
\end{equation}

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import gridspec
```

```python
from mathieu_functions import mathieu_functions as mfs
from mathieu_functions import A_coefficients
```

```python
# =================================
# Important parameters to define
# =================================

# Location of Exceptional Points
ql = [1.466466* (1j), 16.466466* (1j), 47.797797 * (1j) ] # Exceptional Points q_{l}
gamma = [2, 26, 82]  # \Gamma_{l}
Delta = [2, 10, 18]  # \Delta_{l}

N = 70  # length of k-array
alpha = 2  # length of channel periodic in x. I have used alpha=10 before, but for the gaussian initial condition a value of 2 is better.
eps = 0.03  # ta / td << 1 for weakly diffusive processes.

K_test = np.linspace(0, N/alpha, 1000)
Q = (1j) * K_test / (8*eps)
qf = Q[-1].imag  # Largest value of Mathieu's parameter. SHOULD NOT exceed 1000, for now...

M = 25  # matrix size.
colors = ['#0000cc', '#990000', 'darkgreen',
          'indigo', '#009999', 'orange',
          '#ff66ff', '#b266ff', '#33ff99']
```

```python
print('last value of q-array: ')
qf
```

```python
A_vals = A_coefficients(Q, M, 'even', 'one')  # even function with period one-pi
```

## First Pair of Eigenvalues and Asymptotic Correction

```python
l=0  # EP index
ls = np.where(Q <= ql[l])[0][-1]
l1 = np.where(Q > ql[l])[0][0]
l2 = np.where(Q > 2 * ql[l])[0][0]
Qs = Q[:ls]
Ql = Q[l1:]
Ql1 = Q[l1:l2]
Ql2 = Q[l2:]

# Region I (purely real)
a0cs = gamma[l] - (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
a2cs = gamma[l] + (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
# Region II (purely imag)
a0cl1 = - (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
a2cl1 = + (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
# Region III (purely imag)
a0cl2 = - 2 * np.sqrt(Ql2**2) + (4*l + 1) * np.sqrt(2*Ql2)
a2cl2 = + 2 * np.sqrt(Ql2**2) - (4*l + 1) * np.sqrt(2*Ql2)

# Real part not in quadratic approximation (Regions II and III)
a02 = (4*l + 1)* np.sqrt(2 * Ql.imag) - 2*(l**2) - l - 1 / 4
```

```python
fig, ax = plt.subplots(figsize=(20, 8))
gs = gridspec.GridSpec(1,2)
gs.update(hspace=0.05)
ax1 = plt.subplot(gs[0])
plt.plot(Q.imag, A_vals['a0'].real, color=colors[0], lw=4, label='n=0')
# plt.plot(Q.imag, a0c.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Qs.imag, a0cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a2'].real, color=colors[1], lw=4, label='n=1')
# plt.plot(Q.imag, a2c.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Qs.imag, a2cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql.imag, a02, '#606060', lw=4)
plt.xlim(0, 10)
plt.ylim(0, 4.5)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
plt.legend(fontsize=20, frameon=False)
ax2 = plt.subplot(gs[1])
plt.plot(Q.imag, A_vals['a0'].imag, color=colors[0], lw=4, ls='--', label='n=0')
# plt.plot(Q.imag, 1.15*a0c.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql1.imag, a0cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a0cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a2'].imag, color=colors[1], lw=4, ls='--', label='n=1')
# plt.plot(Q.imag, 1.15*a2c.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql1.imag, a2cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a2cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.xlim(0, 100)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
# plt.ylim(-30, 30)
plt.show()
```

## Second Pair of Eigenvalues and Asymptotic Correction

```python
l=1  # EP index
ls = np.where(Q < ql[l])[0][-1]
l1 = np.where(Q > ql[l])[0][0]
l2 = np.where(Q > 2 * ql[l])[0][0]
Qs = Q[:ls]
Ql = Q[l1:]
Ql1 = Q[l1:l2]
Ql2 = Q[l2:]

# Region I (purely real)
a4cs = gamma[l] - (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
a6cs = gamma[l] + (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
# Region II (purely imag)
a4cl1 = - 1.95 * (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
a6cl1 = + 1.95 * (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
# Region II (purely imag)
a4cl2 = - 1.9 * np.sqrt(Ql2**2) +  (4*l + 1) * np.sqrt(2*Ql2)
a6cl2 = + 1.9 * np.sqrt(Ql2**2) - (4*l + 1) * np.sqrt(2*Ql2)

# Real part not in quadratic approximation (Regions II and III)
a46 = (4*l + 1)* np.sqrt(2 * Ql.imag) - 2*(l**2) - l - 1 / 4

```

```python
1.95 * 2, 5*2
```

```python
fig, ax = plt.subplots(figsize=(20, 8))
gs = gridspec.GridSpec(1,2)
gs.update(hspace=0.05)
ax1 = plt.subplot(gs[0])
plt.plot(Q.imag, A_vals['a4'].real, color=colors[2], lw=4, label='n=2')
plt.plot(Qs.imag, a4cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a6'].real, color=colors[3], lw=4, label='n=3')
plt.plot(Qs.imag, a6cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql.imag, a46 , '#606060', lw=4)
plt.xlim(0, Q[-1].imag)
# plt.ylim(10, 40)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
plt.legend(fontsize=20, frameon=False)
ax2 = plt.subplot(gs[1])
plt.plot(Q.imag, A_vals['a4'].imag, color=colors[2], lw=4, ls='--', label='n=2')
plt.plot(Ql1.imag, a4cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a4cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a6'].imag, color=colors[3], lw=4, ls='--', label='n=3')
plt.plot(Ql1.imag, a6cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a6cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.xlim(0, Q[-1].imag)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
# plt.ylim(-60, 60)
plt.show()
```

## Third Pair of Eigenvalues and Asymptotic Correction

```python
l=2  # EP index
ls = np.where(Q < ql[l])[0][-1]
l1 = np.where(Q > ql[l])[0][0]
l2 = np.where(Q > 2 * ql[l])[0][0]
Qs = Q[:ls]
Ql = Q[l1:]
Ql1 = Q[l1:l2]
Ql2 = Q[l2:]

a8cs = gamma[l] - (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
a10cs = gamma[l] + (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Qs**2)
# Region II (purely imag)
a8cl1 = - 2.5 * (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
a10cl1 = + 2.5 * (Delta[l]/ql[l].imag)*np.sqrt(-(ql[l])**2 + Ql1**2)
# region III (purely imag)
a8cl2 = - 1.75 * np.sqrt(Ql2**2) + (4*l + 1) * np.sqrt(2*Ql2)
a10cl2 = + 1.75 * np.sqrt(Ql2**2) - (4*l + 1) * np.sqrt(2*Ql2)

# Real part not in quadratic approximation (Regions II and III)
a810 = (4*l + 1)* np.sqrt(2 * Ql.imag) - 2*(l**2) - l - 1 / 4

```

```python
fig, ax = plt.subplots(figsize=(20, 8))
gs = gridspec.GridSpec(1,2)
gs.update(hspace=0.05)
ax1 = plt.subplot(gs[0])
plt.plot(Q.imag, A_vals['a8'].real, color=colors[4], lw=4, label='n=4')
plt.plot(Qs.imag, a8cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a10'].real, color=colors[5], lw=4, label='n=5')
plt.plot(Qs.imag, a10cs.real, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql.imag, a810 , '#606060', lw=4)
plt.xlim(0, qf)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
plt.legend(fontsize=20, frameon=False)
ax2 = plt.subplot(gs[1])
plt.plot(Q.imag, A_vals['a8'].imag, color=colors[4], lw=4, ls='--', label='n=4')
# plt.plot(Q.imag, 2 * a8c.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql1.imag, a8cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a8cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Q.imag, A_vals['a10'].imag, color=colors[5], lw=4, ls='--', label='n=5')
# plt.plot(Q.imag, 2 * a10c.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql1.imag, a10cl1.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.plot(Ql2.imag, a10cl2.imag, '#202020', lw=2, ls='', marker='*', markersize=7)
plt.xlim(0, qf)
plt.yticks(size=15)
plt.xlabel('q', fontsize=25)
plt.xticks(size=15)
# plt.ylim(-30, 30)
plt.show()
```

```python

```


## ML - II (Assignment 2 and assignment 3)

## Non Linear classification based on SVM and Sparse Random Projections

#### All the analysis has been done in breast cancer dataset

### Importing libraries

```python
# All SVM and kernel dependencies
import numpy as np
import pandas as pd
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn import svm
from sklearn.metrics import classification_report
from sklearn import metrics
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import KFold, cross_val_score, validation_curve
from sklearn.metrics import fbeta_score
from sklearn.metrics import make_scorer
from sklearn.model_selection import GridSearchCV
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
import warnings
warnings.filterwarnings("ignore")
from IPython.display import display, Math, Latex
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC
from sklearn.metrics import f1_score, precision_score, confusion_matrix, recall_score, accuracy_score,classification_report,roc_auc_score
from sklearn.metrics import fbeta_score
from sklearn.metrics import make_scorer
from sklearn.random_projection import johnson_lindenstrauss_min_dim
from sklearn.random_projection import SparseRandomProjection
from sklearn.svm import LinearSVC
```

### The data souce is collected from datasets package and the data is about breast cancer

```python
#Load dataset
cancer = datasets.load_breast_cancer()

# Convert to df
df = pd.DataFrame(cancer.data, columns=cancer.feature_names)
df.head(5)
```

      mean radius
      mean texture
      mean perimeter
      mean area
      mean smoothness
      mean compactness
      mean concavity
      mean concave points
      mean symmetry
      mean fractal dimension
      ...
      worst radius
      worst texture
      worst perimeter
      worst area
      worst smoothness
      worst compactness
      worst concavity
      worst concave points
      worst symmetry
      worst fractal dimension

      0
      17.99
      10.38
      122.80
      1001.0
      0.11840
      0.27760
      0.3001
      0.14710
      0.2419
      0.07871
      ...
      25.38
      17.33
      184.60
      2019.0
      0.1622
      0.6656
      0.7119
      0.2654
      0.4601
      0.11890

      1
      20.57
      17.77
      132.90
      1326.0
      0.08474
      0.07864
      0.0869
      0.07017
      0.1812
      0.05667
      ...
      24.99
      23.41
      158.80
      1956.0
      0.1238
      0.1866
      0.2416
      0.1860
      0.2750
      0.08902

      2
      19.69
      21.25
      130.00
      1203.0
      0.10960
      0.15990
      0.1974
      0.12790
      0.2069
      0.05999
      ...
      23.57
      25.53
      152.50
      1709.0
      0.1444
      0.4245
      0.4504
      0.2430
      0.3613
      0.08758

      3
      11.42
      20.38
      77.58
      386.1
      0.14250
      0.28390
      0.2414
      0.10520
      0.2597
      0.09744
      ...
      14.91
      26.50
      98.87
      567.7
      0.2098
      0.8663
      0.6869
      0.2575
      0.6638
      0.17300

      4
      20.29
      14.34
      135.10
      1297.0
      0.10030
      0.13280
      0.1980
      0.10430
      0.1809
      0.05883
      ...
      22.54
      16.67
      152.20
      1575.0
      0.1374
      0.2050
      0.4000
      0.1625
      0.2364
      0.07678

5 rows × 30 columns

Let's check the data by EDA and if an cleaning is required or not

```python
print("The missing values in data columns :")
print(df.isna().sum())
```

    The missing values in data columns :
    mean radius                0
    mean texture               0
    mean perimeter             0
    mean area                  0
    mean smoothness            0
    mean compactness           0
    mean concavity             0
    mean concave points        0
    mean symmetry              0
    mean fractal dimension     0
    radius error               0
    texture error              0
    perimeter error            0
    area error                 0
    smoothness error           0
    compactness error          0
    concavity error            0
    concave points error       0
    symmetry error             0
    fractal dimension error    0
    worst radius               0
    worst texture              0
    worst perimeter            0
    worst area                 0
    worst smoothness           0
    worst compactness          0
    worst concavity            0
    worst concave points       0
    worst symmetry             0
    worst fractal dimension    0
    dtype: int64

##### Let's prepare for higher dimensionality mapping i.e Kernelization

```python
# Split to X and y
X = cancer.data
y = cancer.target

# Split into train and test set
X_train, X_test, y_train, y_test = train_test_split(
                                    X,y, test_size=0.3, random_state=330)
```

#### Our aim is find that function that linearly maps the points in higher dimensional space perfectly

 **$$ K(x,y) = <φ(x),φ(y)> $$**

*The K value which follows Mercer's rule with the function φ(x)*

### Normalizing values

Fitting the values with standard scaler for bringing all the axis in one scale

```python
# Scaling the values
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
```

### Training The model

#### The concept used  here is SVM with three types of kernels:

##### 1.  Gaussian Kernels
          Gaussian kernels are used to map to infinite dimensional space

\begin{equation}
k(x,x') = e^{\frac{-||x - x'||}{2 * \sigma ^2}}
\end{equation}

##### 2.  Polynomial Kernels
          Polynomial kernels can be used to find projections of data in higher dimensional space to find linearity

\begin{equation}
k(x,x') = (1 + X^T*X')^d
\end{equation}

##### 3.  Linear Kernels
          Linear kernels helps in training the data by the simple notion of finding weight

\begin{equation}
k(x,x') = (1 + X^T*X')
\end{equation}

#### Sparse Random Projections
 It is based on Johnson-Lindenstrauss lemma, which states that if points in a vector space are of sufficiently high dimension, then they may be projected into a suitable lower-dimensional space in a way which approximately preserves the distances between the points.

$$k = O({\xi^-2*log(\frac{1}{\xi}))$$

#### This class contains all the necessary dependencies to run the SVM kernel and sparse projections

```python
##***** The class is used for classifying in SVC kernel with support vector machines*****
class classification:

 def __init__(self, X, y):

    self.X_train, self.X_test, self.y_train, self.y_test = train_test_split(X, y, test_size=0.2, random_state=42)
    self.labels = np.unique(y).tolist()

 def confusion_matrix(self):
    #"For plotting matrix plot in kernel"
    cm = confusion_matrix(self.y_test, self.pred, self.labels)
    print(" The confusion matrix is :", '\n' , cm)
    fig = plt.figure()
    ax = fig.add_subplot(111)
    cax = ax.matshow(cm)
    plt.title('Confusion matrix of the classifier')
    fig.colorbar(cax)
    ax.set_xticklabels([''] + self.labels)
    ax.set_yticklabels([''] + self.labels)
    plt.xlabel('$Predicted$')
    plt.ylabel('$True$')
    plt.show()

 def calc_metrics_class(self):
    #"For getting matrix of all values"
    precision = precision_score(self.pred, self.y_test)
    recall = recall_score(self.pred,self.y_test)
    f1 = f1_score(self.pred,self.y_test)
    accuracy = accuracy_score(self.pred,self.y_test)
    roc = roc_auc_score(self.y_test, self.pred)
    print("The precision for the model is :", precision, '\n', "The recall for the model is : ", recall, '\n'
          , "The f1 score of the model is :", f1, '\n',
          "The accuracy of the model is : ", accuracy, '\n', "The ROC curve of the model is : ", roc )
    # print classification report
    print(classification_report(self.y_test,self.pred))

 def f2_scorer(self,y_true, y_pred):
    #"For giving it to model as best criterion param"
    y_true, y_pred, = np.array(y_true), np.array(y_pred)
    return fbeta_score(y_true, y_pred, beta=2, pos_label=0, average='binary')

 def validation_plot_rbf(self,kernel):
    #"For validating and plotting"
    l = len(self.gamma)
    kfold = 5
    self.f2 = make_scorer(self.f2_scorer,greater_is_better=True)

    train_scores, valid_scores = validation_curve(kernel, X_train, y_train,'gamma',
                                                  self.gamma, cv=KFold(n_splits=5,random_state=7),
                                                  scoring = self.f2,verbose=True)
    train_scores_poly_recall = pd.DataFrame(data=train_scores,
                                        index=np.arange(0, l),
                                        columns=np.arange(0,kfold))
    valid_scores_poly_recall = pd.DataFrame(data=valid_scores,
                                        index=np.arange(0, l),
                                        columns=np.arange(0,kfold))

    plt.plot(self.gamma, train_scores_poly_recall.mean(axis=1), label='training')
    plt.plot(self.gamma, valid_scores_poly_recall.mean(axis=1), label='test')
    plt.xlabel('gamma')
    plt.ylabel('F2 - Score')
    plt.title('SVM with RBF Kernel')
    plt.legend()

 def validation_plot_polynomial(self,kernel):
    #"For validating and plotting"
    l = len(self.degrees)
    kfold = 5
    self.f2 = make_scorer(self.f2_scorer,greater_is_better=True)

    train_scores, valid_scores = validation_curve(kernel, X_train, y_train,'degree',
                                                  self.degrees, cv=KFold(n_splits=5,random_state=7),
                                                  scoring = self.f2,verbose=True)
    train_scores_poly_recall = pd.DataFrame(data=train_scores,
                                        index=np.arange(0, l),
                                        columns=np.arange(0,kfold))
    valid_scores_poly_recall = pd.DataFrame(data=valid_scores,
                                        index=np.arange(0, l),
                                        columns=np.arange(0,kfold))

    plt.plot(self.degrees, train_scores_poly_recall.mean(axis=1), label='training')
    plt.plot(self.degrees, valid_scores_poly_recall.mean(axis=1), label='test')
    plt.xlabel('degree')
    plt.ylabel('F2 - Score')
    plt.title('SVM with Polynomial Kernel')
    plt.legend()

 def rbf_kernel_svc(self):
 #"RBF Kernel"
      print('Performing modelling for RBF Kernel using SVM')
      self.f2 = make_scorer(self.f2_scorer,greater_is_better=True)
      self.gamma = [1,0.1,0.01,0.001]
      param_grid = {
             'C': [0.1,1, 10,100],
             'gamma': self.gamma,
             'kernel': ['rbf'],
             'max_iter':[100]}
      rbf = SVC()
      grid_model = GridSearchCV(SVC(), param_grid, refit = True, verbose = 3,cv=5, n_jobs=-1,scoring=self.f2)
      grid_model.fit(self.X_train,self.y_train)
      print("Best parameters =", grid_model.best_params_)
      model_clf = rbf.set_params(**grid_model.best_params_)
      model_clf.fit(self.X_train, self.y_train)
      self.pred = model_clf.predict(self.X_test)
      self.calc_metrics_class()
      self.confusion_matrix()
      self.validation_plot_rbf(rbf)

 def polynomial_kernel_svc(self):
    #"Polynomial Kernel"
      print('Performing modelling for Polynomial Kernel using SVM')
      self.f2 = make_scorer(self.f2_scorer,greater_is_better=True)
      self.degrees = [2,4,6,10,22]
      param_grid = {
             'C': [0.1,1, 10,100],
             'degree': self.degrees,
             'kernel': ['poly'],
             'max_iter':[200]}
      polynomial = SVC()
      grid_model = GridSearchCV(SVC(), param_grid, refit = True, verbose = 3,cv=5, n_jobs=-1,scoring=self.f2)
      grid_model.fit(self.X_train,self.y_train)
      print("Best parameters =", grid_model.best_params_)
      model_clf = polynomial.set_params(**grid_model.best_params_)
      model_clf.fit(self.X_train, self.y_train)
      self.pred = model_clf.predict(self.X_test)
      self.calc_metrics_class()
      self.confusion_matrix()
      self.validation_plot_polynomial(polynomial)

 def linear_svc(self,feature_len):
        # Linear kernel
        model = LinearSVC()
        model.fit(self.X_train, self.y_train)
        baseline = metrics.accuracy_score(model.predict(self.X_test), self.y_test)
        accuracies = []
        dims = np.int32(np.linspace(2, feature_len, 20))

       # Loop over the projection sizes, k
        for dim in dims:
            # Create random projection
            sp = SparseRandomProjection(n_components = dim)
            X = sp.fit_transform(self.X_train)

            # Train classifier of your choice on the sparse random projection
            model = LinearSVC()
            model.fit(X, self.y_train)

            # Evaluate model and update accuracies
            test = sp.transform(self.X_test)
            accuracies.append(metrics.accuracy_score(model.predict(test), self.y_test))

        #Create figure
        plt.figure()
        plt.xlabel(r'Number of dimensions k')
        plt.ylabel(r'Accuracy')
        plt.xlim([1, feature_len])
        plt.ylim([0, 1])
        # Plot baseline and random projection accuracies
        plt.plot(dims, [baseline] * len(accuracies), color = "r",label = 'baseline')
        plt.plot(dims, accuracies,color="b",label = 'accuracy with dimensions')
        plt.legend()
        plt.show()
```

```python
p1 = classification(X,y)
p1.rbf_kernel_svc()
p1.polynomial_kernel_svc()
```

##### Now let's check with sparse dimensions whats the accuracy score

```python
p1.linear_svc(len(df.columns))
```

- We see a heavy deviation of projections with baseline and as we approach towards higher dimensions we reach stability
- Overall there has been improvement in accuracy when compared to baseline model thus verifying Johnson-Lindenstraus Lemma

```python
print("Professor Nagler is the Best")
```

    Professor Nagler is the Best


# A short document on the image analysis  and statistics for multiphoton FRET

Using two-photon techniques for performing FRET-based stoichiometry requires including additional terms as compared to the one photon case. Namely, we cannot ignore the excitation of donor due to the acceptor excitation as well as the emission of acceptor into the donor emission channel. This has been detailed wonderfully in Meredith's thesis and I will not repeat it here. Instead, I will outline the image processing techniques that are used in the data analysis developed by Dan and Amar. The basic algorithm is as follows:

* Use Donor only and FRET only cells
    - measure lifetimes in SMART center. Get an average value of the FRET efficiency, E

* Donor only cells
    - pick out individual cells from each image, average $I_d$, $I_n$, $I_f$ over each cell
    - calculate $\beta$ and $\theta$
    - repeat for each image
    - calculate average $\beta$ and $\theta$
    - 74 cells used for beta
    - 66 cells used for theta

    Shown below is an example of how individual cells are thresholded (on the right) against the background and then each cell is individually selected (on the left, shown by the color coding) and calculations are performed on the signal from the entire cell

* Acceptor only cells
    - pick out individual cells from each image, average $I_a$, $I_n$, $I_f$ over each cell
    - calculate $\alpha$ and $\eta$
    - repeat for each image
    - calculate average $\alpha$ and $\eta$
    - 28 cells used for $\alpha$
    - 29 cells used for $\eta$

* Use FRET only cells
    - pick out individual cells, average $I_a$, $I_d$, $I_f$ over each cell
    - use the average values of E, $\beta$, $\theta$, $\alpha$ and $\eta$ as calculated above
    - calculate using \begin{equation} \gamma = \frac{E}{-1 + \frac{\eta \left(- \alpha + \theta\right) \left(- I_{d} \beta + I_{f}\right)}{\alpha \left(- \beta + \eta\right) \left(I_{a} \theta - I_{f}\right)}} \end{equation}
    - repeat for each image
    - calculate average $\gamma$
    - 77 cells used for calculating $\gamma$

    In the graph below, the histogram of $\gamma$ is shown.

* Use FRET only cells
    - pick out individual cells, average $I_a$, $I_d$, $I_f$ over each cell
    - use the average values of E, $\beta$, $\theta$, $\alpha$, $\eta$ and $\gamma$ as calculated above
    - calculate using \begin{equation} \xi = \frac{E \gamma \left(I_{d} \eta - I_{f}\right)}{\left(- E + 1\right) \left(- \beta + \eta\right) \left(- \frac{I_{a} \alpha \theta}{- \alpha + \theta} - \frac{I_{d} \beta \eta}{- \beta + \eta} + \frac{I_{f} \left(- \alpha \beta + \eta \theta\right)}{\left(- \alpha + \theta\right) \left(- \beta + \eta\right)}\right)} \end{equation}
    - repeat for each image
    - calculate average $\xi$
    - also 77 cells, because the same data set is used as $\gamma$

    In the graph below, the histogram of $\xi$ is shown.

* Use FRET only cells (trivial case sanity check)
    - Individual cells have already been picked out above for $\gamma$ and $\xi$ calculation. Use the averaged $I_a$, $I_d$, $I_f$ over each cell as above.
    - use all the above quantities to calculate fraction of donor $f_d$, and acceptor $f_a$
    - Use \begin{equation}
    f_{a} = \frac{\gamma}{E} \left(-1 + \frac{\eta \left(- \alpha + \theta\right) \left(- I_{d} \beta + I_{f}\right)}{\alpha \left(- \beta + \eta\right) \left(I_{a} \theta - I_{f}\right)}\right)
    \end{equation}
    - Use \begin{equation} f_{d} = \frac{1}{E} \left(1 - \frac{I_{d} \eta - I_{f}}{\left(- \beta + \eta\right) \left(\frac{I_{d} \eta - I_{f}}{- \beta + \eta} + \frac{\xi}{\gamma} \left(- \frac{I_{a} \alpha \theta}{- \alpha + \theta} - \frac{I_{d} \beta \eta}{- \beta + \eta} + \frac{I_{f} \left(- \alpha \beta + \eta \theta\right)}{\left(- \alpha + \theta\right) \left(- \beta + \eta\right)}\right)\right)}\right) \end{equation}
    - confirm that they average to one (trivial case because $\gamma$ and $\xi$ are calculated assuming $f_a$ and $f_d$ are one for FRET case samples

    Shown below are the histograms of $f_a$ and $f_d$ for the FRET only case. The histograms should be centered about 1. Also included are the same numbers plotted $f_a$ vs $f_d$. Ideally, this distribution should be a Gaussian distribution around the point (1,1) in this image.

* Use unlinked donor and acceptor cells (sanity check)
    - pick out individual cells, average $I_a$, $I_d$, $I_f$ over each cell
    - use all the above quantities to calculate fraction of donor $f_d$, and acceptor $f_a$
    - confirm that they average to zero
    - Use same formulas as above for $f_a$ and $f_d$
    - 45 cells used for calculating $f_a$ and $f_d$

    Shown below are the histograms of $f_a$ and $f_d$ for the donor and acceptor unlinked only case. The histograms should be centered about 0. Also included are the same numbers plotted $f_a$ vs $f_d$. Ideally, this distribution should be a Gaussian distribution around the point (0,0) in this image.

Here lies the worrying part. The distribution of $f_a$ and $f_d$ should not display significant covariance with respect to each other, since they should be random quantities. Is there an underlying relation that causes those numbers to naturally vary together because many of the numbers used for their calculations are the same? Would a completely random set of $I_a$, $I_d$ and $I_f$ still generate this covariance?

data/github.com/azazeus/two-photon-FRET-image-analysis/60914765501c2f2a1aee85a0f04fd5fc80c72034/Image data analysis statistics writeup.ipynb