# Task Two: Qubits, Bloch Sphere and Basis States

# Installing Qiskit

Installing Qiskit:

Step 1: Go to Anaconda Powershell
Step 2: Type "pip install qiskit"
Alternatively, pip install qiskit[visualization]

-----OR-----

Uncomment the next line of code to dirtectly install Qiskit from the Jupyter Notebook Code Shell

```python
#Uncomment the below line

#!pip install qiskit
#!pip install qiskit[visualization]
```

```python
import numpy as np
```

Only after running above cell or installing Qiskit from the Anaconda Prompt,Run the below cells-->

```python
import qiskit
qiskit.__qiskit_version__
```

    {'qiskit-terra': '0.16.0',
     'qiskit-aer': '0.7.0',
     'qiskit-ignis': '0.5.0',
     'qiskit-ibmq-provider': '0.11.0',
     'qiskit-aqua': '0.8.0',
     'qiskit': '0.23.0'}

\begin{align}
|\psi \rangle = \begin{pmatrix}
\alpha \\ \beta
\end{pmatrix}, \quad \text{where } \sqrt{\langle \psi | \psi \rangle} = 1.
\end{align}

Think of Qubit as an Electron:

\begin{align}
\text{spin-up}: \ |0\rangle &= \begin{pmatrix} 1\\0 \end{pmatrix} \\
\text{spin-down}: \ |1\rangle & = \begin{pmatrix} 0\\1 \end{pmatrix}
\end{align}

```python
#Import everything from qiskit
from qiskit import *
```

 Another representation is via Bloch Sphere

```python
from qiskit.visualization import plot_bloch_vector

plot_bloch_vector([0,0,1],title = 'spinup')
```

## Spin + / - :

\begin{align}
\text{spin +}: \ |+\rangle &= \begin{pmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{pmatrix} = \frac{1}{\sqrt{2}} \left(|0\rangle + |1\rangle\right) \\
\text{spin -}: \ |-\rangle & = \begin{pmatrix} 1/\sqrt{2} \\ -1/\sqrt{2} \end{pmatrix} = \frac{1}{\sqrt{2}} \left(|0\rangle - |1\rangle\right)
\end{align}

```python
plot_bloch_vector([1,0,0])
```

## Basis States

\begin{align}
 |0\rangle &= \begin{pmatrix} 1\\0 \end{pmatrix} \\
 |1\rangle & = \begin{pmatrix} 0\\1 \end{pmatrix}
\end{align}
Preapring other states from Basis States:
\begin{align}
|00 \rangle &= |0\rangle \otimes |0\rangle = \begin{pmatrix} 1\\0 \end{pmatrix} \otimes \begin{pmatrix} 1\\0 \end{pmatrix} = \begin{pmatrix} 1\\0\\0\\0 \end{pmatrix} \\
|01 \rangle &= |0\rangle \otimes |1\rangle = \begin{pmatrix} 1\\0 \end{pmatrix} \otimes \begin{pmatrix} 0\\1 \end{pmatrix} = \begin{pmatrix} 0\\1\\0\\0 \end{pmatrix} \\
|10 \rangle &= |1\rangle \otimes |0\rangle = \begin{pmatrix} 0\\1 \end{pmatrix} \otimes \begin{pmatrix} 1\\0 \end{pmatrix} = \begin{pmatrix} 0\\0\\1\\0 \end{pmatrix} \\
|11 \rangle &= |1\rangle \otimes |1\rangle = \begin{pmatrix} 0\\1 \end{pmatrix} \otimes \begin{pmatrix} 0\\1 \end{pmatrix} = \begin{pmatrix} 0\\0\\0\\1 \end{pmatrix}
\end{align}

```python
ket_zero = np.array([[1],[0]])
ket_one = np.array([[0],[1]])
```

```python
np.kron(ket_one,ket_zero)
```

    array([[0],
           [0],
           [1],
           [0]])

**So we have prepared the basis state of a Two Qubit System**

# That's all for Task 2
## Thank You!

Where can you find me?

LinkedIn : https://www.linkedin.com/in/arya--shah/

Twitter : https://twitter.com/aryashah2k

Github : https://github.com/aryashah2k

If you Like My Work, Follow me/ Connect with me on these platforms.
Show some Love ❤ Sponsor me on Github!

```python

```

Guided Project - Programming a Quantum Computer with Qiskit - IBM SDK/Task 2/Task 2 - Qubits, Bloch Sphere and Basis States.ipynb


```python
from datascience import *

import sympy
import matplotlib.pyplot as plt
import matplotlib as mpl
import matplotlib.patches as patches
plt.style.use('seaborn-muted')
%matplotlib inline

from IPython.display import display
import numpy as np
import pandas as pd
solve = lambda x,y: sympy.solve(x-y)[0] if len(sympy.solve(x-y))==1 else "Not Single Solution"

import warnings
warnings.filterwarnings('ignore')
```

# Demand Curves

In this chapter, we will explore one of the most foundational yet important concepts in economics: demand curves. The demand curve shows the graphical relationship between the price of a good or service and the quantity demanded for it over a given period of time.
In other words, it shows the quantity of goods or services consumers are willing to buy at each market price.
The quantity of goods or services demanded or supplied can be modeled as a function of price, as in:

$$\text{Quantity} = f(\text{Price})$$

Notably, the curve is downwards sloping because of the law of demand, which states that *as the price of a good or service increases, the quantity demanded for it decreases, assuming all other factors are held constant*.

This should make intuitive sense: as prices increase, fewer people are willing to pay the higher price for the same good. On the other hand, as prices decrease, more people are willing to pay the lower price for the same good. Hence, the quantity demanded of a good or service has an inverse relationship with the price.

In this course, we will assume that the relationship is somewhat linear and can be described as

$$\text{Quantity}_{d}=a \cdot \text{Price}_{d} + b$$

We can interpret the equation above as follows: *as the price of a unit increases by 1, there is an $a$ unit increase (or decrease, if $a$ is negative) in the quantity demanded.* For example, $\text{Quantity}_{d}=-2 \cdot \text{Price}_{d} + 3$ would suggest that a price increase by 1 would decrease overall quantity demanded in the market by 2.

Price can also be measured as function of quantity to denote demand. In this case, we use an inverse demand function, as it is the inverse function of the demand function above. Since price is a function of quantity,

$$\text{Price} = f(\text{Quantity})$$

As we are solving for the inverse of the previous demand function, the inverse demand function for the example above is

$$\text{Price}_{d}=-\frac{1}{2} \cdot \text{Quantity}_{d} - \dfrac{3}{2}$$

## Movement and the Demand Curve

### Shifts in the Demand Curve
The demand curve can shift in or out based on exogenous events that occur outside of the market.
Some factors other than a change in price of the good/service could be changes in:

*  buyer's income
*  consumer preferences
*  expectation of future price/supply/demand
*  changes in the price of related goods

If any of these changes occur and causes the demand for the selected good/service to decrease, then the curve shifts to the left as less of the good or service will be demanded at every price. Similarly, if any of these changes causes the demand for the selected good/service to increase, the curve would shift to the right. This signifies that more of the good or service will be demanded at every price. For example, consumers' incomes decreased during the 2008 recession, thus decreasing overall buying power and shifting the demand curve leftwards; a left shift in the demand curve suggests that consumers would purchase fewer quantities of goods at every price.

```python
xs = np.linspace(-10, 20, 100)
ys = -3 * xs + 8
ys2 = -3 * xs + 3

arx = 5
ary = -8
ardx = -1
ardy = 0

plt.figure(figsize=[7,7])
plt.plot(xs, ys, label="Original Demand Curve")
plt.plot(xs, ys2, label="Shifted Demand Curve")
plt.arrow(arx, ary, ardx, ardy, head_width=0.8, head_length=0.5, length_includes_head=True)
plt.xticks([])
plt.yticks([])
plt.xlabel("Quantity", size=16)
plt.ylabel("Price", size=16)
plt.xlim([-0.5, 10.5])
plt.ylim([-25, 10])
plt.legend();
```

### Movements along the Demand Curve

Above, we looked at when exogenous events affect the demand curve. Another concept is movements along the demand curve, which would be considered endogenous events. In movements along the demand curve, changes in price affect the quantity demanded of the good or service. This assumes ceteris paribus, which means holding all other factors constant. This phenomenom is called the Movement of the Demand Curve. With a change in price, the quantity demanded for the good or service can shift the quantity demanded either upward or downward on the demand curve. For example, consider the shift on the graph below from quantity $q_1$ at price $p_1$ to quantity $q_2$ at price $p_2$.

```python
xs = np.linspace(-10, 20, 100)
ys = -3 * xs + 8

q_1, p_1 = 1, 5
q_2, p_2 = 8, -16

plt.figure(figsize=[7,7])
plt.plot(xs, ys)
plt.scatter([q_1, q_2], [p_1, p_2], s=200, color="g", zorder=15)
plt.arrow(q_1, p_1, q_2 - q_1, p_2 - p_1, color="tab:orange", width=.1, head_length=1.5, head_width=0.5, length_includes_head=True, zorder=-1)

# (q_1, p_1)
plt.vlines(q_1, -1000, p_1, linestyle="dashed")
plt.hlines(p_1, -1000, q_1, linestyle="dashed")

# (q_2, p_2)
plt.vlines(q_2, -1000, p_2, linestyle="dashed")
plt.hlines(p_2, -1000, q_2, linestyle="dashed")

plt.xticks([q_1, q_2], [r"$q_1$", r"$q_2$"], size=14)
plt.yticks([p_1, p_2], [r"$p_1$", r"$p_2$"], size=14)
plt.xlabel("Quantity", size=16)
plt.ylabel("Price", size=16)
plt.xlim([-0.5, 10.5])
plt.ylim([-25, 10]);
```

## Income and Substitution Effect

Changes in the price of a good also leads to 2 possible effects:

- **Income Effect**: Examines how the change in price of the good affects income, which then affects the quantity demanded of a good or service.
- **Substitution Effect**: Examines how the change in price of the good affects its demand relative to other goods and services

If the price of a good increases, it would require the consumer to spend more of their income on the good. This dissuades consumers from purchasing the good, decreases the quantity demanded. If the price of a good decreases, consumers would spend less money to receive the same good. This increases the quantity demanded for the good, because more consumers would want to purchase the good, given the decrease in the amount of purchasing power required to obtain the good.

If the price of a good increases, then consumers might look at similar goods that function in the same way or yield an equivalent amount of utility as the original good. Consumers are effectively shifting or substituting away from the relatively more expensive good to a cheaper alternative, thereby decreasing the quantity demanded for the good. The converse is also true: if the price of a good decreases, then consumers that currently purchase other similar goods might start consuming this good instead, because it would cost less money for them to obtain.

For example, if the price of gas increases, the higher price for gas might encourage consumers to look at purchasing more efficient cars, such as electric or hybrid cars. This is the subsitution effect. If the consumer stays with their original car, then they would have less disposable income after purchasing the now-more-expensive gas, so they might purchase less gas. This is the income effect.


```julia
using Pkg; Pkg.status()
```

    Project MixedModelsTutorials v0.1.0
        Status `~/.juliapro/JuliaPro_v1.1.1.1/dev/MixedModelsTutorials/Project.toml`
      [a93c6f00] + DataFrames v0.18.4
      [31c24e10] + Distributions v0.20.0
      [f6369f11] + ForwardDiff v0.10.3
      [7073ff75] + IJulia v1.18.1
      [91a5bcdd] + Plots v0.25.3
      [ce6b1742] + RDatasets v0.6.2
      [3eaba693] + StatsModels v0.6.1
      [f3b207a7] + StatsPlots v0.11.0
      [bd369af6] + Tables v0.2.8
      [44d3d7a6] + Weave v0.9.1
      [e88e6eb3] + Zygote v0.3.2
      [b77e0a4c] + InteractiveUtils
      [37e2e46d] + LinearAlgebra
      [44cfe95a] + Pkg

# Nonlinear regression using automatic differentiation

Nonlinear regression or nonlinear least squares fits the parameters of a _model function_ to a vector of _observed responses_ and corresponding values of _covariates_.  Typically the data are organized in a table such as a `DataFrame`.

A simple example from enzyme kinetics relates the rate or velocity, `v`, of an enzymatic reaction to the concentration, `c`, of the substrate according to the _Michaelis-Menten_ model
\begin{equation}
v = \frac{V_m\,c}{K+c}
\end{equation}
The parameters of the model are $V_m$, the maximum velocity, and $K$, the Michaelis parameter.  Data from such an experiment is available in the `RDatasets` package as the `Puromycin` data set.  This contains data from two experiments.  We will concentrate on the data for the treated cells.

```julia
using BenchmarkTools, DataFrames, ForwardDiff, LinearAlgebra
using RDatasets, StatsPlots, Tables, Zygote
```

```julia
gr();
```

```julia
Pur = groupby(RDatasets.dataset("datasets", "Puromycin"), :State)
```

GroupedDataFrame with 2 groups based on key: State

First Group (12 rows): State = "treated"

	Conc	Rate	State

	Float64	Int32	Categorical…

1	0.02	76	treated

2	0.02	47	treated

3	0.06	97	treated

4	0.06	107	treated

5	0.11	123	treated

6	0.11	139	treated

7	0.22	159	treated

8	0.22	152	treated

9	0.56	191	treated

10	0.56	201	treated

11	1.1	207	treated

12	1.1	200	treated

⋮

Last Group (11 rows): State = "untreated"

	Conc	Rate	State

	Float64	Int32	Categorical…

1	0.02	67	untreated

2	0.02	51	untreated

3	0.06	84	untreated

4	0.06	86	untreated

5	0.11	98	untreated

6	0.11	115	untreated

7	0.22	131	untreated

8	0.22	124	untreated

9	0.56	144	untreated

10	0.56	158	untreated

11	1.1	160	untreated

```julia
@df Pur[1] scatter(:Conc, :Rate)  # Data plot for the treated cells
```

As can be seen from the plot, the maximum velocity is around 200 and the Michaels parameter, `K`, which is the concentration at which the velocity is half the maximum, is about 0.05.

## Evaluating the mean vector and the Jacobian

In the statistical model the response vector, $\mathbf{y}$, is the realization of a random variable, $\mathcal{Y}$, that has a "spherical" Gaussian distribution with mean, $\mathbf{\mu}$, determined from the mean function, $\mathbf{\mu}(\mathbf{\theta},\mathbf{C})$. Here $\mathbf{\theta}$ is the parameter vector and $\mathbf{C}$ is the table of covariate values.  The "spherical" multivariate Gaussian (or "normal") distribution means that the covariance matrix is a multiple of the identity, in which case contours of constant probability density are spheres centered at $\mathbf{\mu}$.
\begin{equation}
\mathcal{Y}\sim\mathcal{N}\left(\mathbf{\mu}(\mathbf{\theta},\mathbf{C}),\sigma^2\mathbf{I}\right)
\end{equation}

The maximum likelihood estimates of $\mathbf{\theta}$ are the values that minimize the sum of squared residuals
\begin{equation}
\widehat{\mathbf{\theta}}=\arg\min_{\mathbf{\theta}} \|\mathbf{y}-\mathbf{\mu}(\mathbf{\theta},\mathbf{C})\|^2
\end{equation}

A simple iterative scheme for this _nonlinear least squares_ problem is the _Gauss-Newton_ method of successive linear approximation.  Writing $\mathbf{\mu}^{(i)}$ for the value of the mean vector at the $i$ iterate, $\mathbf{\theta}^{(i)}$, the increment is the linear least squares solution of the residual, $\mathbf{r}^{(i)}=\mathbf{y}-\mathbf{\mu}^{(i)}$, on the Jacobian, $\mathbf{J}^{(i)}=\partial\mathbf{\mu}(\mathbf{\theta},\mathbf{C})/\partial\mathbf{\theta}|_{\mathbf{\theta}^{(i)}}$
\begin{equation}
\mathbf{\mathbf{\delta}^{(i)}}=\arg\min_{\mathbf{\delta}}\|\mathbf{r}^{(i)}-\mathbf{J}^{(i)}\mathbf{\delta}\|^2
\end{equation}

### Vector evaluation

In the Michaelis-Menten example the mean function could be written for vector evaluation using dot-broadcast fusion

```julia
μ(Vm, K, conc) = @. Vm * conc / (K + conc)
```

    μ (generic function with 1 method)

producing the residual

```julia
r = Pur[1].Rate - μ(200., 0.05, Pur[1].Conc)
```

    12-element Array{Float64,1}:
      18.85714285714286
     -10.142857142857139
     -12.090909090909093
      -2.0909090909090935
     -14.5
       1.5
      -3.962962962962962
     -10.962962962962962
       7.393442622950829
      17.39344262295083
      15.695652173913032
       8.695652173913032

The `ForwardDiff` package can provide the Jacobian but it requires a unary function on which to operate.  This means that the parameters must be passed as an `AbstractVector` and the covariates must be in the closure of the function passed to, say, `ForwardDiff.jacobian!`.  Some applications require reuse of the model function for different sets of covariates, such as the treated and untreated cells here, and I think that would entail using a reference to the data table when defining the model function.

```julia
const dataref = Ref(first(Pur))
function μ₁(pars)
    Vm, K = pars
    conc = dataref[].Conc
    @. Vm * conc / (K + conc)
end
const θ = [200., 0.05]
const results = DiffResults.JacobianResult(dataref[].Conc, θ)
const cfg = ForwardDiff.JacobianConfig(μ₁, θ);
```

```julia
ForwardDiff.jacobian!(results, μ₁, θ, cfg)
adjoint(results.value)
```

    1×12 Adjoint{Float64,Array{Float64,1}}:
     57.1429  57.1429  109.091  109.091  …  183.607  183.607  191.304  191.304

```julia
results.derivs[1]
```

    12×2 Array{Float64,2}:
     0.285714  -816.327
     0.285714  -816.327
     0.545455  -991.736
     0.545455  -991.736
     0.6875    -859.375
     0.6875    -859.375
     0.814815  -603.567
     0.814815  -603.567
     0.918033  -300.994
     0.918033  -300.994
     0.956522  -166.352
     0.956522  -166.352

```julia
map!(-, results.value, dataref[].Rate, results.value) # replace μ by residual
```

    12-element Array{Float64,1}:
      18.85714285714286
     -10.142857142857139
     -12.090909090909093
      -2.0909090909090935
     -14.5
       1.5
      -3.962962962962962
     -10.962962962962962
       7.393442622950829
      17.39344262295083
      15.695652173913032
       8.695652173913032

```julia
sum(abs2, results.value)                 # current sum of squared residuals
```

    1636.5857075901783

```julia
δ = qr(results.derivs[1])\results.value
```

    2-element Array{Float64,1}:
     11.157219609869895
      0.011627130326972698

```julia
θ .+= δ
```

    2-element Array{Float64,1}:
     211.15721960986988
       0.0616271303269727

```julia
ForwardDiff.jacobian!(results, μ₁, θ, cfg)
sum(abs2, map!(-, results.value, dataref[].Rate, results.value))
```

    1205.6196313926841

To change to a new data table, reassign `dataref[]` and create a new `JacobianResult`.

```julia
dataref[] = last(Pur)
const res2 = DiffResults.JacobianResult(dataref[].Conc, θ)
ForwardDiff.jacobian!(res2, μ₁, θ, cfg)
adjoint(res2.value)
```

    1×11 Adjoint{Float64,Array{Float64,1}}:
     51.737  51.737  104.166  104.166  …  164.951  190.223  190.223  199.955

```julia
res2.derivs[1]
```

    11×2 Array{Float64,2}:
     0.245017  -633.821
     0.245017  -633.821
     0.493311  -856.439
     0.493311  -856.439
     0.640924  -788.545
     0.640924  -788.545
     0.781175  -585.706
     0.781175  -585.706
     0.900862  -306.009
     0.900862  -306.009
     0.946948  -172.133

Of course, these computations can be made cleaner by defining a data structure and operating on the struct but it seems to be tricky to get the closure of the model function right, if the model function is to be an element of the struct.  Because of lexical scoping the closure is defined at the time the function is defined.

## Scalar evaluation

As all the cool kids are switching to the `Zygote` package for automatic differentiation, I read up on it.  As far as I can see, Zygote is primarily used to evaluate gradients of scalar functions, which in this application would mean iterating over the rows of the data table.  To do so it is worthwhile converting the `DataFrame`, which is column-oriented, to a "row table".

The sum of squared residuals can be accumulated during the loop.  Also a Cholesky factor can be used instead of a QR factorization of the Jacobian to evaluate the increment.  It is convenient to zero out the Cholesky factor and use `LinearAlgebra.lowrankupdate!` for the update.  If the residual value is appended to the gradient in the update vector, `v`, then the Cholesky factor can be solved for the increment in one step.

```julia
const rt = rowtable(first(Pur))
const ch = cholesky(zeros(3, 3) + I)
const v = zeros(3);
```

```julia
function updatech!(ch::Cholesky{T, Matrix{T}}, θ::AbstractVector{T}, rowtbl, v::Vector{T}) where {T}
    rss = zero(T)             # residual sum of squares
    fill!(ch.factors, false)
    Vm, K = θ
    for r in rowtbl
        copyto!(v, gradient((Vm, K) -> μ(Vm, K, r.Conc), Vm, K))
        resid = r.Rate - μ(Vm, K, r.Conc)
        rss += abs2(resid)
        v[end] = resid
        lowrankupdate!(ch, v)
    end
    rss
end
```

    updatech! (generic function with 1 method)

```julia
updatech!(ch, θ, rt, v)  # returns sum of squared residuals at current θ
```

    1205.6196313926841

```julia
ch
```

    Cholesky{Float64,Array{Float64,2}}
    U factor:
    3×3 UpperTriangular{Float64,Array{Float64,2}}:
     2.46279  -1607.44   -0.225002
      ⋅        1364.85    3.02266
      ⋅            ⋅    -34.5895

```julia
ldiv!(LowerTriangular(view(ch.factors, 1:2, 1:2)), copyto!(δ, view(ch.factors, 1:2, 3)))
```

    2-element Array{Float64,1}:
     -0.09136051999062601
      0.002214648452543934

As I understand it, `Zygote.forwarddiff` could be a better choice of AD algorithm but I don't quite understand how to use it.  I thought it could work like this but it doesn't

```julia
function forwardupdate!(ch::Cholesky{T, Matrix{T}}, θ::AbstractVector{T}, rowtbl, v::Vector{T}) where {T}
    rss = zero(T)             # residual sum of squares
    fill!(ch.factors, false)  # zero out the Cholesky factor
    for r in rowtbl
        copyto!(v, Zygote.forwarddiff(θ -> μ(θ[1], θ[2], r.Conc), θ))
        resid = r.Rate - μ(θ[1], θ[2], r.Conc)
        rss += abs2(resid)
        v[end] = resid
        lowrankupdate!(ch, v)
    end
    rss
end
```

    forwardupdate! (generic function with 1 method)

```julia
forwardupdate!(ch, θ, rt, v)
```

    1205.6196313926841

```julia
ch
```

    Cholesky{Float64,Array{Float64,2}}
    U factor:
    3×3 UpperTriangular{Float64,Array{Float64,2}}:
     520.036   466.663  -0.225002
        ⋅     1086.95   14.9687
        ⋅         ⋅     31.329

Of course the `updatech!` function is sufficiently fast that it is not a problem to use the reverse mode AD.  I am just concerned that I may be using a sledgehammer to

```julia
@benchmark updatech!($ch, $θ, $rt, $v)
```

    BenchmarkTools.Trial:
      memory estimate:  4.69 KiB
      allocs estimate:  36
      --------------
      minimum time:     4.114 μs (0.00% GC)
      median time:      4.193 μs (0.00% GC)
      mean time:        4.924 μs (11.80% GC)
      maximum time:     472.251 μs (98.10% GC)
      --------------
      samples:          10000
      evals/sample:     7


DEL (DELETE THIS CELL WHEN DISTRIBUTING)
This notebook is for developing intro materials for PDP team 11's inquiry activity.  Cells marked with **DEL** at the top are only for design/notes and will be deleted for the version given to learners.

**TODO**
1. Review and adopt useful practices in CMSE NBs you have access to

# Outline

1. Overview of activity
2. A basic finite difference
3. A basic ODE meta-model (compartmental model)
4. Mapping out an algorithm
5. Putting this together to model the evolution of a system

#  Overview of activity

#  A basic finite difference

#  A basic ODE meta-model (compartmental model)

DEL

**Compartmental model resources**
* [CMSE NB](http://nbviewer.jupyter.org/url/amjacobs.net/assets/NBs/Day-10_In-Class_CompartmentalModeling-STUDENT.ipynb)
* Wikipedia: [Multi-component model](https://en.wikipedia.org/wiki/Multi-compartment_model), [Compartmental model (epidemiology)](https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology)
* Turku PET Centre: [Compartmental models](http://www.turkupetcentre.net/petanalysis/model_compartmental.html), [ODE solutions of compartmental models](http://www.turkupetcentre.net/petanalysis/model_ode.html)
* PMOD: [Intro to compartmental models](http://www.pmod.com/files/download/v31/doc/pkin/4614.htm)

In this section, we introduce the notion of a *compartmental model*.  While this modeling framework is typically seen in the medical field (e.g. epidemiology, physiology, related medical imaging), it can be used as a general framework for building basic models of many systems of interest in a variety of scientific disciplines.

TODO continue narrative, introduce ODEs to make sure they don't get caught-up/intimidated by seeing an ODE.

for now, outline and minimal info, you can add color/explanation to it later.

Generic eqn:
The evolution of each compartment $C_i$ in an $N$-compartment model is expressed as

$
\begin{equation}
    \frac{dC_i(t)}{dt} = f_i(u_i(t), C_1(t), ... , C_N(t)),
\end{equation}
$
where $u_i(t)$ is an input function.  There may be one for each compartment ($u_i(t)$), for only some, or only one may be specified for the whole model ($u(t)$).

While it's useful to have this general definition, we'll get a better feel for compartmental models with some examples.

First, let's consider a 1-compartment model composed of an input funtion and a single compartment $C_1$:

$
\begin{equation}
    \frac{dC_1(t)}{dt} = K_1 u(t) - k_2 C_1(t).
\end{equation}
$

Loosely, this can be read as

$
\begin{equation}
    (\text{change in "amount" of }C_1\text{ over time})  =  \\
        (\text{rate of flow from } u \text{ to } C_1) (\text{"amount" in }u(t)) -
        (\text{rate of flow from } C_1 \text{ out of the system}) (\text{"amount" of }C_1).
\end{equation}
$

A diagram of this model can be given as:

Though some real systems are modeled with a 1-compartment model, we can learn a bit more about using compartmental models by adding another compartment that can be interacted with.  You may be able to determine how we'd express such a system by analogy with the 1-compartment model.

We now have two compartments, so we'll want a $C_2(t)$. The quantity being tracked by a particular model can now flow in from the input function into $C_1$, which can in turn exchange with $C_2$, so we'll need some new coefficients for the rates describing such flows.  This could look like

$
\begin{equation}
    \frac{dC_1(t)}{dt} = K_1 u(t) - (k_2 + k_3) C_1(t) + k_4 C_2(t),\\
    \frac{dC_2(t)}{dt} = k_3 C_1(t) - k_4 C_2(t) - k_5 C_2(t).
\end{equation}
$

We can represent this with a diagram as

You may realize this isn't the only way we could try a 2-compartment model.  Any given compartment may have (1) input from an input function (like $u(t)$ into $C_1$ proportional to $K_1$), (2) flow out into another compartment (like from $C_1$ to $C_2$ proportional to $k_3$), (3) flow in from another compartment (like from $C_2$ to $C_1$ proportional to $k_4$), or (4) output that leaves the system (like that marked with constants $k_2$ and $k_5$). For a given compartmental model, each compartment will only have whichever of these makes sense for the model.  Thus, with this basic framework you can model different systems, even if both models are 2-compartment.

So far, we've been pretty generic.  We wanted to first get introduced to the basic concepts before getting our hands dirty.  But now we're ready for a real 2-compartment model that models the spread of diseases like the common cold that do not confer immunity after recovery.  This means people are either susceptible (S) to infection or are infectious (I) themselves.  Highly original scientists calls this the SI model of disease spread.  Let's look at the equations and diagram, then discuss it:

$
\begin{equation}
    \frac{dS(t)}{dt} = \mu N - \frac{\beta S I}{N} - \nu S  \\
    \frac{dI(t)}{dt} = \frac{\beta S I}{N} - \nu I
\end{equation}
$

$S(t)$ is the fraction of the population that's currently susceptible to infection.  The $\frac{dS(t)}{dt}$ is saying that
* (term 1) this population is increased by new births, with $\mu$ being the rate at which people are born for a population of the size $N = S + I$,
* (term 2) this population is decreased according to the rate ($\beta$) at which susceptible individuals ($S$) in contact with infectious individuals ($I$) become infected, leaving $S$ and joining $I$, and finally
* (term 3) this population is decreased acocrding to the mortality rate $\nu$.

Consider the $\frac{dI(t)}{dt}$ and see if you can express in words what each term represents, similar to what we've just done for $\frac{dS(t)}{dt}$.

At this point, you should be equipped with enough understanding of compartmental models to be able to carry out an inquiry into what ODE models like this can tell us and what constraints there may be when solving them computationally.

#  Mapping out an algorithm

#  Putting this together to model a system's evolution


# Laboratório 12: Reator Biológico

### Referente ao capítulo 19

Solo contaminado possui muitas vezes bactérias que são capazes de eliminar o contaminante. Então um método para limpar essas áreas é aumentar o nível dessas bactérias. A injeção de nutrientes necessárias para o metabolismo e crescimento da colônia provaram-se técnicas de sucesso no aumento da população de bactérias. Todavia essa modelagem é bem complexa, dada aos diversos fatores que se relacionam,
Por esse motivo, uma configuração mais controlada é utilizada nesse laboratório, como encontramos no reatores biológicos.

Consideraremos uma reator com mistura ideal onde bactéria $x(t)$ e contaminante  $z(t)$ se distribuem uniformemente (unidade g/L), e a bactéria degrada o contaminante. Seja $u(t)$ o nutriente que controlamos espacialmente uniforme. Assim

$$
x'(t) = Gu(t)x(t) - Dx(t)^2, x(0) = x_0 > 0
$$

onde $G$ é a taxa máxima de crescimento e $D$ é a taxa de morte natural. A taxa de degradação do contaminante é proporcional à concentração de ambos,

$$
z'(t) = -Kx(t)z(t), z(0) = z_0 > 0,
$$

onde $K$ é a taxa de degradação da bactéria. O solo tem limitações físicas e, portanto, $0 \le u(t) \le M$. O objetivo é minimizar o uso do nutriente e o nível final de contaminante:

$$
\ln(z(T)) + \int_0^T Au(t) dt,
$$

Seja $A = 1$. Temos que

$$
z(t) = z_0\exp\left(-\int_0^t Kx(s)ds\right),
$$

de forma que

$$
- \int_0^T Kx(s)ds = \ln\left(\frac{z(T)}{z_0}\right) = \ln(z(T)) - \ln(z_0)
$$

Assim podemos tirar $z(T)$ do nosso funcional objetivo. Além disso, $z_0$ não tem efeito na minimização e pode ser removido. Assim, nosso problema pode ser transformado em (invertendo o sinal transforma o problema em maximização):

$$
\max_u \int_0^T Kx(t) - u(t) dt,
$$
$$
\text{sujeito a   }x'(t) = Gu(t)x(t) - Dx^2(t), x(0) = x_0, 0 \le u(t) \le M
$$

Agora $K$ também tem papel de peso e, por isso, podíamos colocar $A = 1$.

### Importando bibliotecas

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
import sympy as sp

import sys
sys.path.insert(0, '../pyscripts/')

from optimal_control_class import OptimalControl
```

## Condições necessárias

Mais uma vez vamos utilizar a biblioteca simbólica para nos ajudar nos cálculos. Note que esse sistema é tão simples que de fato não seria muito necessário, mas vamos usar assim mesmo.

### Hamiltoniano

```python
x, u, l, t, D, G, K = sp.symbols('x u lambda t D G K')
H = K*x - u + l*(G*u*x - D*x**2)
H
```

$\displaystyle K x + \lambda \left(- D x^{2} + G u x\right) - u$

### Condições do Estado

```python
print(sp.diff(H,l))
```

    -D*x**2 + G*u*x

### Equaçãoes Adjuntas

```python
dl = (-1)*sp.diff(H,x)
print(dl)
```

    -K - lambda*(-2*D*x + G*u)

### Transversalidade

$\lambda(T) = 0$

### Otimalidade

```python
psi = sp.diff(H,u)
psi
```

$\displaystyle G \lambda x - 1$

## Demonstração que o problema é Bang-Bang

$$
(x\lambda)' = x'\lambda + x\lambda ' = -Dx^2\lambda + Gux\lambda -Kx + 2Dx^2\lambda - Gux\lambda = Dx(x\lambda - K/D)
$$

Como $x = 0$ é um equilíbrio e $x(0) > 0$, vale que $x(t) > 0$ para todo $t$. Suponha que $(x\lambda)(t) < K/D$ para algum $t$. Assim $(x\lambda)'(t) < 0$ e a função decresce nesse ponto, isto é, ela permanecerá abaixo desse limiar. O mesmo vale para a desigualdade oposta ou igualdade. Portanto, para todo $t$, vale que essa expressão é $=, > \text{ ou } <$. Como $(x\lambda)(T) = 0 < K/D \implies (x\lambda)(t) < K/D$, para todo $t$. Portanto essa função é monotonicamente decrescente, então $G\lambda x = 1$ pode ser mantido em apenas 1 ponto.

Conscluímos que $u^*$ é Bang-Bang com ponto de troca quando $G\lambda x - 1$ troca de sinal.

## Aplicando a classe ao exemplo

Vamos começar as experimentações. Podemos fazer uma função específica para plotar nesse exemplo.

```python
def plotting(t,x,z,u, fig = None, ax = None, label = ''):

    if ax is None:
        fig, ax = plt.subplots(3, sharex=True, figsize = (12,8))
        fig.suptitle('Comportamento das bactérias e do contaminante', fontsize = 18)

        ax[0].set_ylabel('Bactérias')
        ax[1].set_ylabel('Contaminante')
        ax[2].set_ylabel('Nutriente')

    ax[0].plot(t,x, label = label)
    ax[1].plot(t,z, label = label)
    ax[2].plot(t,u, label = label)

    return fig, ax
```

```python
parameters = {'D': None, 'K': None, 'G': None, 'M': None}

diff_state = lambda t, x, u, par: -par['D']*x**2 + par['G']*u*x

diff_lambda = lambda t, x, u, lambda_, par: -par['K'] - lambda_*(-2*par['D']*x + par['G']*u)

update_u = lambda t, x, lambda_, par: 0.0 if par['G']*x*lambda_ < 1 else par['M']
```

```python
problem = OptimalControl(diff_state, diff_lambda, update_u,
                         n_controls = 1, n_states = 1)
```

```python
x0 = 0.5
z0 = 0.1
T = 2
parameters['K'] = 2
parameters['G'] = 1
parameters['D'] = 1
parameters['M'] = 1
```

```python
t,x,u,lambda_ = problem.solve(x0, T, parameters)

h = T/(len(x)-1)
z = z0*np.exp(-parameters['K']*h*np.cumsum(x))

ax = plotting(t, x, z, u)
```

A combinação dos valores de $K$ e $G$ não nos favorece a colocar nutriente no reator e, portanto, deixá-lo sem perturbação é melhor em quase todo o percurso. Isso diminui a população de bactérias rapidamente. Agora faremos uma pequena variação no parâmetro $K$. Observe que por muito mais tempo a injeção de nutrientes é mais eficiente, o que aumenta a população de bactérias o suficiente para reduzir muitoo o contaminante.

```python
parameters['K'] = 2.1
t,x,u,lambda_ = problem.solve(x0, T, parameters)

h = T/(len(x)-1)
z = z0*np.exp(-parameters['K']*h*np.cumsum(x))

ax = plotting(t, x, z, u)
```

```python
parameters['K'] = 2
G_values = [1, 1.2, 1.4]
u_values = []
x_values = []
z_values = []
for G in G_values:
    parameters['G'] = G
    _,x,u,_ = problem.solve(x0, T, parameters)
    u_values.append(u)
    x_values.append(x)
    z = z0*np.exp(-parameters['K']*h*np.cumsum(x))
    z_values.append(z)
```

Percebemos que quanto maior o valor de $G$, maior a injeção de nutrientes. Isso faz sentido, pois o valor de $u(t)$ fica mais tempo nesse valor máximo. Outro ponto interessante é que o ganho por aumentar em 0.2 diminui no decréscimo de contaminante.

```python
fig, ax = plotting(t, x_values[0], z_values[0], u_values[0], label = 'G = 1.0')
plotting(t, x_values[1], z_values[1], u_values[1], ax = ax, label = 'G = 1.2')
plotting(t, x_values[2], z_values[2], u_values[2], ax = ax, label = 'G = 1.4')

handles, labels = ax[-1].get_legend_handles_labels()
fig.legend(handles, labels)

plt.show()
```

```python
parameters['G'] = 1.2
x0_values = [0.3, 0.5, 0.7]
u_values = []
x_values = []
z_values = []
for x0 in x0_values:
    _,x,u,_ = problem.solve(x0, T, parameters)
    u_values.append(u)
    x_values.append(x)
    z = z0*np.exp(-parameters['K']*h*np.cumsum(x))
    z_values.append(z)
```

Agora vamos ver o que a condição inicial de bactérias causa no sistema. Esperamos que quanto maior o índice de bactérias, menor a necessidade de aumentar a injeção de nutrientes para reduzir a quantidade do contaminante. Mas na verdade o oposto acontece. Isso ocorre, pois a população de bactérias cresce e degrada o contaminante exponencialmente. Assim, com mais bactérias, a degradação se dá mais rápido e ainda vale a pena aumentar o número de bactérias. No caso em que $x_0 = 0.3$, a quantidade é tão pequena que deixar o sistema decair sozinho é a melhor estratégia.

```python
fig, ax = plotting(t, x_values[0], z_values[0], u_values[0], label = r'$x_0$ = 0.3')
plotting(t, x_values[1], z_values[1], u_values[1], ax = ax, label = r'$x_0$ = 0.5')
plotting(t, x_values[2], z_values[2], u_values[2], ax = ax, label = r'$x_0$ = 0.7')

handles, labels = ax[-1].get_legend_handles_labels()
fig.legend(handles, labels)

plt.show()
```

Por fim, vamos verificar o que $M$ faz no sistema.

```python
x0 = 0.5
M_values = [0.5, 1.0, 2.0]
u_values = []
x_values = []
z_values = []
for M in M_values:
    parameters['M'] = M
    _,x,u,_ = problem.solve(x0, T, parameters)
    u_values.append(u)
    x_values.append(x)
    z = z0*np.exp(-parameters['K']*h*np.cumsum(x))
    z_values.append(z)
```

Por mais contraditório que pareça, quanto maior o valor de $M$, por mais tempo a injeção de nutrientes ocorre. Isso novamente se deve ao caráter exponencial das populações.

```python
fig, ax = plotting(t, x_values[0], z_values[0], u_values[0], label = r'M = 0.5')
plotting(t, x_values[1], z_values[1], u_values[1], ax = ax, label = r'$M$ = 1.0')
plotting(t, x_values[2], z_values[2], u_values[2], ax = ax, label = r'$M$ = 2.0')

handles, labels = ax[-1].get_legend_handles_labels()
fig.legend(handles, labels)

plt.show()
```

### Este é o final do notebook


# 1. 1-D example with 3 parameters

The following example shows how to construct the kernel, automatically, from a symbolic expression defining the linear differential operator in **1D**.

We consider the following operator, for an unknwon *u*

$$
\mathcal{L}^{\mu, \alpha, \beta} u := \mu u + \alpha \partial_x u + \beta \partial_{xx} u
$$

```python
# imports
from mlhiphy.calculus import dx, dy, dz
from mlhiphy.calculus import Constant
from mlhiphy.calculus import Unknown
from mlhiphy.kernels import compute_kernel, generic_kernel

from sympy import expand
from sympy import symbols
from sympy import exp
```

```python
x, x_i, x_j = symbols('x x_i x_j')

u = Unknown('u')

alpha = Constant('alpha')
beta  = Constant('beta')
mu    = Constant('mu')
theta = Constant('theta')

expr = mu * u + alpha * dx(u) + beta * dx(dx(u))
```

```python
kuu = generic_kernel(expr, u, (x_i, x_j))
```

```python
from IPython.display import Math
from sympy import latex

Math(latex(expand(kuu)))
```

$$\alpha^{2} \frac{\partial^{2}}{\partial x_{i}\partial x_{j}}  u{\left (x_{i},x_{j} \right )} + \alpha \beta \frac{\partial^{3}}{\partial x_{i}^{2}\partial x_{j}}  u{\left (x_{i},x_{j} \right )} + \alpha \beta \frac{\partial^{3}}{\partial x_{i}\partial x_{j}^{2}}  u{\left (x_{i},x_{j} \right )} + \alpha \mu \frac{\partial}{\partial x_{i}} u{\left (x_{i},x_{j} \right )} + \alpha \mu \frac{\partial}{\partial x_{j}} u{\left (x_{i},x_{j} \right )} + \beta^{2} \frac{\partial^{4}}{\partial x_{i}^{2}\partial x_{j}^{2}}  u{\left (x_{i},x_{j} \right )} + \beta \mu \frac{\partial^{2}}{\partial x_{i}^{2}}  u{\left (x_{i},x_{j} \right )} + \beta \mu \frac{\partial^{2}}{\partial x_{j}^{2}}  u{\left (x_{i},x_{j} \right )} + \mu^{2} u{\left (x_{i},x_{j} \right )}$$

```python
# RBF kernel
kuu = theta * exp(-0.5*((x_i - x_j)**2))
```

```python
kuf = compute_kernel(expr, kuu, x_i)
kfu = compute_kernel(expr, kuu, x_j)
kff = compute_kernel(expr, kuu, (x_i, x_j))
```

```python
Math(latex(expand(kuf)))
```

$$- 1.0 \alpha \theta x_{i} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \alpha \theta x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta \theta x_{i}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 2.0 \beta \theta x_{i} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta \theta x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 1.0 \beta \theta e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + \mu u{\left (x_{i} \right )}$$

```python
Math(latex(expand(kfu)))
```

$$1.0 \alpha \theta x_{i} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 1.0 \alpha \theta x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta \theta x_{i}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 2.0 \beta \theta x_{i} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta \theta x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 1.0 \beta \theta e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + \mu u{\left (x_{j} \right )}$$

```python
Math(latex(expand(kff)))
```

$$- 1.0 \alpha^{2} \theta x_{i}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 2.0 \alpha^{2} \theta x_{i} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 1.0 \alpha^{2} \theta x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \alpha^{2} \theta e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta^{2} \theta x_{i}^{4} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 4.0 \beta^{2} \theta x_{i}^{3} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 6.0 \beta^{2} \theta x_{i}^{2} x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 6.0 \beta^{2} \theta x_{i}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 4.0 \beta^{2} \theta x_{i} x_{j}^{3} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 12.0 \beta^{2} \theta x_{i} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 1.0 \beta^{2} \theta x_{j}^{4} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 6.0 \beta^{2} \theta x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 3.0 \beta^{2} \theta e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 2.0 \beta \mu \theta x_{i}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 4.0 \beta \mu \theta x_{i} x_{j} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + 2.0 \beta \mu \theta x_{j}^{2} e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} - 2.0 \beta \mu \theta e^{- 0.5 x_{i}^{2}} e^{- 0.5 x_{j}^{2}} e^{1.0 x_{i} x_{j}} + \mu^{2} u{\left (x_{i},x_{j} \right )}$$

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:600px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 135%;
        font-size: 120%;
        width:600px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#054BCD;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #054BCD;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


#  Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty

Monteiro et al. 2020, accepted at NeurIPS 2020.

```python
%matplotlib inline

import matplotlib.pyplot as plt
import numpy as np
import torch
import tqdm.notebook as tqdm

plt.rcParams.update({
    'font.size': 8,
    'text.usetex': True,
    'text.latex.preamble': r'\usepackage{amsfonts}'
})
```

```python
MARGIN = 8
DIM = 32

rng = np.random.RandomState(7)

def build_input(batch_size, shape):
    h, w = shape
    img = np.ones((batch_size, h, w), dtype=np.float32) + 0.5*rng.randn(h, w)
    line_pos = rng.randint(0, h - MARGIN - 1, size=batch_size)
    for i, lp in enumerate(line_pos):
        img[i, lp:lp+MARGIN] = 0.9*rng.randn(MARGIN, w)
        img[i, :lp] = -1.0 + 0.5*rng.randn(lp, w)
    return img.astype(np.float32), line_pos

def build_output(inp, line_pos):
    batch_size = inp.shape[0]
    offsets = line_pos
    output = np.zeros_like(inp)
    line_pos = rng.randint(0, MARGIN, size=batch_size) + offsets
    for i, lp in enumerate(line_pos):
        output[i, lp:] = 1.0
    return output.astype(np.float32)

def produce_data(batch_size, dim):
    while True:
        xs, line_pos = build_input(batch_size, (dim, dim))
        ys = build_output(xs, line_pos)
        yield np.expand_dims(xs, 1), np.expand_dims(ys, 1)

def plot_im(xs, cmap, title=None):
    fig, ax = plt.subplots(1, 1, figsize=(6, 6))
    xs = xs.squeeze().T[:16]
    ax.imshow(xs, cmap, interpolation="none")
    ax.set_xticks([])
    ax.set_yticks([])
    ax.set_title(title, fontsize=40, pad=20)

def sample_bernoulli_im(im, n_samples=1):
    im = np.concatenate([1 - im, im], axis=1)[0].transpose(1, 2, 0)
    n_ims = np.tile(im[None, ...], [n_samples, 1, 1, 1])
    sample = (n_ims.cumsum(-1) >= rng.uniform(size=n_ims.shape[:-1])[..., None]).argmax(-1)
    return sample
```

```python
inp, lp = build_input(1, (DIM, DIM))
out = np.concatenate([build_output(inp, lp) for _ in range(100)], axis=0)
```

## Some data is messy

For the same image $x$ we have multiple ambigous labels $y_{1,2,\dots}$.

```python
plot_im(inp[0], cmap="coolwarm", title=r"$$x$$")
```

```python
plot_im(inp[0], cmap="coolwarm", title=r"$$x$$")
plot_im(out[0], cmap="gray", title=r"$$y_1$$")
```

```python
plot_im(inp[0], cmap="coolwarm", title=r"$$x$$")
plot_im(out[4], cmap="gray", title=r"$$y_2$$")
```

```python
plot_im(out.mean(0), cmap="gray", title=r"$$\mathbb{E}[y \mid x]$$")
```

## Probabilistic perspective

#### Typical modelling approach:

 * Assume an underlying data distribution $p(y \mid x)$,
 * construct a parametric model: $p_\phi(y \mid x) = \int p(y \mid \phi) p(\phi \mid x)d\phi$,
 * optimise $\phi$ to minimise a divergence between $p$ and $p_\phi$.

#### Simplifications:

 * Model only an "output" variable $\eta$ (e.g. the logits before softmax);
     - the model becomes $p_\phi(y \mid x) = \int p(y \mid \eta) p_\phi(\eta \mid x)d\eta$
 * set $p_\phi(\eta \mid x)$ to be a delta function, i.e. $\eta = f_\phi(x)$ is deterministic
     - the integral reduces to the likelihood: $p_\phi(y \mid x) = p(y \mid f_\phi(x))$
 * set $p(y \mid f_\phi(x))$ to be a factorised categorical distribution
 * optimise KL-divergence between $p$ and $p_\phi$ using MC approximation:

 Using these assumptions, we can derive an easy to implement model:

 $$
 \begin{align}
 &\text{argmin}_\phi D_\text{KL}\left(p(y \mid x) \mid\mid p(y \mid f_\phi(x))\right) \\
 &= \text{argmin}_\phi \int p(y \mid x) \log{\frac{p(y \mid x)}{p(y \mid f_\phi(x)))}} \\
 &= \text{...steps left as an exercise to the reader...} \\
 &\approx \text{argmin}_\phi -\frac{1}{N}\sum_{n,i,c}y^n_{i,c}\log{\text{softmax}(f_\phi(x^n))_{i,c}},\quad (x^n, y^n) \sim p(x, y) \quad \text{for} \, n=1, 2, \dots, N
 \end{align}
 $$

 **For a good solution $\phi^*$ we will have $p(y_i \mid f_{\phi^*}(x)_i) = p(y_i \mid x)$**

```python
class MLP(torch.nn.Module):
    def __init__(self, inp_size, n_hidden):
        super(MLP, self).__init__()
        self.layers = [torch.nn.Linear(inp_size, n_hidden[0])]
        for cur, nxt in zip(n_hidden, n_hidden[1:]):
            self.layers.append(torch.nn.Linear(cur, nxt))
        self.layers.append(torch.nn.Linear(n_hidden[-1], inp_size))
        self.model = torch.nn.Sequential(*self.layers)

    def forward(self, xs, as_probs=False):
        orig_shape = xs.shape
        out = xs.view(xs.shape[0], -1)
        for layer in self.layers[:-1]:
            out = torch.nn.functional.relu(layer(out))
        out = self.layers[-1](out)
        if as_probs:
            out = torch.sigmoid(out)
        return out.view(*orig_shape)

    def log_likelihood(self, xs, ys):
        params = self.forward(xs)
        logprobs = torch.distributions.Bernoulli(logits=params).log_prob(ys).sum((1, 2, 3))
        return logprobs
```

```python
def run_step(xs_t, ys_t, opt, net):
    opt.zero_grad()
    loss = -torch.mean(net.log_likelihood(xs_t, ys_t))
    loss.backward()
    opt.step()
    return loss.detach().numpy()
```

```python
producer = produce_data(10, DIM)
net = MLP(inp_size=DIM*DIM, n_hidden=[64, 64, 64])
opt = torch.optim.Adam(net.parameters(), lr=3e-4)

N_ITERS = 1000
pbar = tqdm.tqdm(range(N_ITERS))

for i in pbar:
    xs, ys = next(producer)
    xs_t = torch.as_tensor(xs)
    ys_t = torch.as_tensor(ys)
    loss = run_step(xs_t, ys_t, opt, net)
    pbar.set_description_str(f"loss: {loss:4.2f} ")

ys_pred = net(torch.as_tensor(inp[:, None, ...]), as_probs=True).detach().numpy()
plot_im(ys_pred[0], cmap="gray", title=r"$$p(y \mid f_{\phi^*}(x))$$")
```

* What we see is a distribution, not a sample from it.
* We can obtain a useful sample by applying argmax/thresholding operation (mode extraction)

```python
plot_im(ys_pred[0] > 0.5, cmap="gray", title=r"$$y = \mathrm{argmax}(p(y \mid f_{\phi^*}(x)))$$")
```

* Alternatively we can also sample from $p(y \mid f_{\phi^*}(x))$ at the cost of label inconsistencies.

```python
fd_sample = sample_bernoulli_im(ys_pred)
plot_im(fd_sample, cmap="gray", title=r"$$y \sim p(y \mid f_{\phi^*}(x))$$")
```

## Modelling dependence in $y$

Before:

Now:

#### Complications:

 Discard two simplifying assumptions of the older model:
 1. Logits are independent
 2. $p(\eta \mid x)$ is a delta function

Both can be tackled by letting $p(\eta \mid x) = \mathcal{N}\left(\eta \mid \mu(x), \Sigma(x)\right)$

However, $\Sigma(x)$ will become intractable if one has to model all $\frac{1}{2}(H\times W\times C)^2$ correlations!

Introduce low-rank parameterisation for $\Sigma = PP^T + D$ where $P\in \mathbb{R}^{(H\times W\times C)\times R}$ and $R \ll H\times W\times C$.

$R$ is a hyperparameter controlling the rank of the covariance factor matrix $P$ and $D$ is a diagonal matrix.

This modifies the objective slightly:

* the model becomes $p_\phi(y \mid x) = \int p(y \mid \eta) \mathcal{N}\left(\eta \mid \mu(x),\Sigma(x)\right)d\eta \approx \frac{1}{M}\sum_i p(y \mid \eta_i),\quad \eta_i \sim \mathcal{N}\left(\mu(x),\Sigma(x)\right)$
$$
\begin{align}
&\text{argmin}_\phi -\frac{1}{N}\sum_{n,i,c}y^n_{i,c}\log{\text{softmax}(\underset{\eta^n}{\underbrace{f_\phi(x^n)}})_{i,c}},\quad (x^n, y^n) \sim p(x, y)\\
&\rightarrow \text{argmin}_\phi -\frac{1}{N}\sum_n \log{\left(\frac{1}{M}\sum_j\exp{\left(\sum_{i,c}y^n_{i,c}\log{\text{softmax}\left(\eta^n_j\right)_{i,c}}\right)}\right)},\quad \eta^n\sim \mathcal{N}\left(\mu(x^n), \Sigma(x^n)\right);\quad (x^n, y^n) \sim p(x, y)
\end{align}
$$

Lastly, we need to learn $\mu(x)$ and $\Sigma(x)$. Apply reparametrisation trick:

$$
\eta \sim \mathcal{N}\left(\mu(x), \Sigma(x)\right) \iff \mu(x) + \sqrt{\Sigma(x)}\epsilon,\quad \epsilon \sim \mathcal{N}\left(0, I\right)
$$

```python
class StochasticMLP(torch.nn.Module):
    def __init__(self, inp_size, n_hidden, cov_factor=1):
        super(StochasticMLP, self).__init__()
        self.cov_factor = cov_factor
        self.layers = [torch.nn.Linear(inp_size, n_hidden[0])]
        for cur, nxt in zip(n_hidden, n_hidden[1:]):
            self.layers.append(torch.nn.Linear(cur, nxt))
        self.mu = torch.nn.Linear(n_hidden[-1], inp_size)
        self.P = torch.nn.Linear(n_hidden[-1], inp_size*cov_factor)
        self.D = torch.nn.Linear(n_hidden[-1], inp_size)
        self.layers.extend([self.mu, self.P, self.D])
        self.model = torch.nn.Sequential(*self.layers)

    def forward(self, xs, as_probs=False, n_samples=1, use_mode=False):
        out = xs.view(xs.shape[0], -1)
        for layer in self.layers[:-3]:
            out = torch.nn.functional.relu(layer(out))
        if not use_mode:
            cov_factor = self.P(out).view(xs.shape[0], -1, self.cov_factor)
            cov_diag = torch.exp(self.D(out)) + 1e-3
            out = torch.distributions.LowRankMultivariateNormal(
                self.mu(out), cov_factor, cov_diag
            ).rsample((n_samples,)).permute(1, 0, 2)
        else:
            out = self.mu(out)
        if as_probs:
            out = torch.sigmoid(out)
        return out.view(xs.shape[0], n_samples, *xs.shape[1:])

    def log_likelihood(self, xs, ys, n_samples=8):
        params = self.forward(xs, n_samples=n_samples).view(xs.shape[0], n_samples, -1)
        logprobs = torch.distributions.Bernoulli(logits=params).log_prob(ys.view(xs.shape[0], 1, -1))
        logprobs = torch.logsumexp(logprobs.sum(-1), dim=1) - np.log(n_samples)
        return logprobs
```

```python
def compute_cov(self, xs):
    out = xs.view(xs.shape[0], -1)
    for layer in self.layers[:-3]:
        out = torch.nn.functional.relu(layer(out))
    cov_factor = self.P(out).view(xs.shape[0], -1, self.cov_factor)
    cov_diag = torch.exp(self.D(out)) + 1e-3
    return cov_factor, cov_diag

StochasticMLP.compute_cov = compute_cov
```

```python
net = StochasticMLP(inp_size=DIM*DIM, n_hidden=[64, 64, 64], cov_factor=2)
opt = torch.optim.Adam(net.parameters(), lr=5e-4)

N_ITERS = 20000
pbar = tqdm.tqdm(range(N_ITERS))

for i in pbar:
    xs, ys = next(producer)
    xs_t = torch.as_tensor(xs)
    ys_t = torch.as_tensor(ys)
    loss = run_step(xs_t, ys_t, opt, net)
    pbar.set_description_str(f"loss: {loss:4.2f} ")
```

    HBox(children=(FloatProgress(value=0.0, max=20000.0), HTML(value='')))

Sampling from $\mathcal{N}\left(\eta \mid \mu(x),\Sigma(x)\right)$ results different probabilities for the output. Applying argmax on each gives samples $y_{1,2,\dots}$:

```python
ys_pred = net(torch.as_tensor(inp[:, None, ...]), as_probs=True, n_samples=200).detach().numpy()[0]
plot_im(ys_pred[0] > 0.5, cmap="gray", title=r"$$y_1 = \mathrm{argmax}(p(y \mid \eta_1))$$")
plot_im(ys_pred[1] > 0.5, cmap="gray", title=r"$$y_2 = \mathrm{argmax}(p(y \mid \eta_2))$$")
```

Verfiy that the average of all samples resembles the average of all ground truth labels.

```python
plot_im(ys_pred.mean(0), cmap="gray", title="$$\mathbb{E}[\mathrm{softmax}(\eta)\mid x]$$")
```

```python
mode_pred = net(torch.as_tensor(inp[:, None, ...]), as_probs=True, use_mode=True).detach().numpy()
plot_im(mode_pred > 0.5, cmap="gray", title=r"$$y = \mathrm{argmax}(p(y \mid \mu(x)))$$")
```

## Extra: explore the covariance

```python
cf, cd = net.compute_cov(torch.as_tensor(inp[:, None, ...]))
```

```python
plot_im(cf.view(1, 32, 32, 2).detach().numpy().sum(-1), cmap="gray")
```

```python
cf = cf.detach().numpy()[0]
cd = cd.detach().numpy()[0]
cov_mat = cf @ cf.T + np.diag(cd)
```

```python
plt.imshow(cov_mat, cmap="gray")
plt.gca().set_xticks([])
plt.gca().set_yticks([])
```


Table of Contents

VarianceComponentModels.jl

Package Features

Heritability Analysis

Data files

Read in binary SNP data

EUR_subset

Empirical kinship matrix

Simulating phenotypes

Phenotypes

Pre-processing data for heritability analysis

Heritability of single trait

Multivariate trait analysis

Exercise

Testing SNP association using maximum likelihoods of variance component models

Fit the null model

Fit the alternative model

Likelihood ratio test

Exercise

# Heritability analysis and testing SNP association using maximum likelihoods of variance component models

**ASHG OpenMendel Workshop**

**Juhyun Kim, juhkim111@ucla.edu**

**Department of Biostatistics, UCLA**

**Oct 2020**

Machine information:

```julia
versioninfo()
```

    Julia Version 1.5.0
    Commit 96786e22cc (2020-08-01 23:44 UTC)
    Platform Info:
      OS: Linux (x86_64-pc-linux-gnu)
      CPU: Intel(R) Core(TM) i9-9920X CPU @ 3.50GHz
      WORD_SIZE: 64
      LIBM: libopenlibm
      LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

## VarianceComponentModels.jl

### Package Features
* [Heritability analysis for quantitative traits](https://openmendel.github.io/VarianceComponentModels.jl/latest/man/heritability/#Heritability-Analysis-1) in genetics
* SNP association testing
* Maximum likelihood estimation (MLE) and restricted maximum likelihood estimation (REML) of mean parameters $B$ and variance component parameters $Σ$
* Allow constraints in the mean parameters $B$
* Choice of optimization algorithms: [Fisher scoring](https://books.google.com/books?id=QYqeYTftPNwC&lpg=PP1&pg=PA142#v=onepage&q&f=false) and [minorization-maximization algorithm](http://hua-zhou.github.io/media/pdf/ZhouHuZhouLange19VCMM.pdf)

[`VarianceComponentModels.jl`](https://github.com/OpenMendel/VarianceComponentModels.jl/) is a package that resides in [OpenMendel](https://github.com/OpenMendel) ecosystem. It implements computation routines for fitting and testing variance component model of form

$$Y \sim \text{Normal}(XB, \sigma_1^2 V_1 + \cdots + \sigma_m^2 V_m).$$

In this model, **data** are represented by

* $Y$: continuously varying quantitative trait(s)
* $X$: covariates (e.g. sex, age, principal components)
* $V_i$: structuring matrix corresponding to $i$-th variance component $(i=1,\ldots, m)$

and **parameters** are

* $B$: mean fixed effects coefficient
* $\sigma_i^2$: $i$-th variance component $(i=1,\ldots, m)$.

## Heritability Analysis

Variance component estimation can be used to estimate heritability of a quantitative trait.
Here we decompose the overall phenotypic variance into genetic and environmental components:

$$\Omega = \sigma_a^2 (2\Phi) + \sigma_e^2 I $$

where

- $\sigma_a^2$: aggregate additive genetic effects of an unspecified number of loci at unknown locations in the genome
- $\sigma_e^2$: unique, unshared environmental effects
- $2\Phi$: genetic relationship matrix; twice the kinship coefficient
- $I$: identity matrix, which is ones down the diagonal and zeros everywhere else.

Then the (narrow-sense) heritability is defined as

$$h^2 = \frac{\sigma_a^2}{\sigma_a^2 + \sigma_e^2}.$$

### Data files

For this analysis, we use a sample data set [`EUR_subset`](https://openmendel.github.io/SnpArrays.jl/latest/#Example-data-1) from `SnpArrays.jl`. This data set is available in the `data` folder of the package. Note that we are using a small dataset for the purpose of this tutorial, but you can use a dataset much larger than the example data set.

```julia
using SnpArrays
```

```julia
datapath = normpath(SnpArrays.datadir())
```

    "/home/juhkim111/.julia/packages/SnpArrays/YOSme/data"

`EUR_subset.bed`, `EUR_subset.bim`, and `EUR_subset.fam` are the Plink binary files.

```julia
using Glob
readdir(glob"EUR_subset.*", datapath)
```

    3-element Array{String,1}:
     "/home/juhkim111/.julia/packages/SnpArrays/YOSme/data/EUR_subset.bed"
     "/home/juhkim111/.julia/packages/SnpArrays/YOSme/data/EUR_subset.bim"
     "/home/juhkim111/.julia/packages/SnpArrays/YOSme/data/EUR_subset.fam"

### Read in binary SNP data

We use the [`SnpArrays.jl`](https://openmendel.github.io/SnpArrays.jl/latest) package to read in binary SNP data and compute the empirical kinship matrix.

```julia
# read in genotype data from Plink binary file
const EUR_subset = SnpArray(SnpArrays.datadir("EUR_subset.bed"))
```

    WARNING: redefinition of constant EUR_subset. This may fail, cause incorrect answers, or produce other errors.

    379×54051 SnpArray:
     0x03  0x03  0x03  0x02  0x02  0x03  …  0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x02  0x03  0x02  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x03  0x03  0x03     0x02  0x02  0x02  0x03  0x03  0x02
     0x03  0x03  0x03  0x00  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x00  0x03  0x03     0x02  0x02  0x02  0x03  0x03  0x03
     0x02  0x03  0x03  0x03  0x03  0x03  …  0x03  0x03  0x03  0x03  0x03  0x02
     0x02  0x03  0x03  0x02  0x02  0x03     0x03  0x03  0x02  0x02  0x03  0x03
     0x02  0x03  0x03  0x03  0x02  0x02     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x00  0x02  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x02  0x03  0x03  0x02  0x03  0x02     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x02  0x03  0x03  …  0x03  0x03  0x02  0x02  0x03  0x03
     0x03  0x03  0x03  0x02  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x02
     0x03  0x02  0x03  0x02  0x02  0x03     0x03  0x03  0x03  0x03  0x03  0x03
        ⋮                             ⋮  ⋱     ⋮                             ⋮
     0x03  0x03  0x03  0x00  0x02  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x02  0x02  0x03     0x02  0x02  0x02  0x03  0x02  0x03
     0x03  0x03  0x03  0x02  0x02  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x02  0x03  0x03  0x02  0x03  0x03  …  0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x00  0x00  0x03     0x02  0x02  0x02  0x03  0x03  0x03
     0x02  0x03  0x03  0x03  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x02  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x02  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x02  0x03  0x03  0x03  0x03  0x03  …  0x03  0x03  0x02  0x02  0x03  0x03
     0x03  0x03  0x03  0x00  0x03  0x03     0x03  0x03  0x03  0x03  0x03  0x03
     0x02  0x03  0x03  0x02  0x00  0x02     0x03  0x03  0x03  0x03  0x03  0x03
     0x03  0x03  0x03  0x02  0x02  0x03     0x03  0x03  0x03  0x03  0x03  0x03

### `EUR_subset`

`EUR_subset` contains **379** individuals and **54,051** SNPs. There is no missing genotype in `EUR_subset`.

Minor allele frequencies (MAF) for each SNP:

```julia
maf_EUR = maf(EUR_subset)
```

    54051-element Array{Float64,1}:
     0.09762532981530347
     0.01319261213720313
     0.04485488126649073
     0.48944591029023743
     0.32189973614775724
     0.09102902374670185
     0.3733509234828496
     0.05277044854881263
     0.0554089709762533
     0.11345646437994727
     0.20448548812664913
     0.16226912928759896
     0.27176781002638517
     ⋮
     0.341688654353562
     0.13192612137203164
     0.24802110817941958
     0.21240105540897103
     0.12532981530343013
     0.13192612137203164
     0.07387862796833777
     0.07783641160949872
     0.13588390501319259
     0.0554089709762533
     0.01319261213720313
     0.02638522427440637

Histogram of minor allele frequency:

The above plot can be generated by using the commands commented out below:

```julia
# using Plots
# hist_maf = histogram(maf_EUR, xlab = "Minor Allele Frequency (MAF)",
#                    ylab = "Number of SNPs", label="") # generate histogram
# png(hist_maf, "hist_MAF.png") # save as png file
```

Note that about 29% of the SNPs have MAF < 0.05.

```julia
count(!iszero, maf_EUR .< 0.05) / length(maf_EUR)
```

    0.2914839688442397

### Empirical kinship matrix

For a measure of relatedness, we compute empirical kinship matrix based on all SNPs by the genetic relation matrix (GRM). If there are missing genotypes, they are imputed on the fly by drawing according to the minor allele frequencies.

Kinship coefficients summarize genetic similarity between pairs of individuals. To estimate kinship coefficient $\Phi_{ij}$ between individuals $i$ and $j$ using GRM:

$$\widehat{\Phi}_{GRMij} = \frac{1}{2S} \sum_{k=1}^S \frac{(G_{ik}-2p_k)(G_{jk}-2p_k)}{2p_k(1-p_k)},$$

where

* $S$: number of SNPs in this set
* $p_k$: minor allele frequency of SNP $k$
* $G_{ik} \in \{0,1,2\}$: number of copies of minor alleles at the $k$-th SNP of the $i$-th individual.

This matrix is known as the classical kinship matrix.

Now we construct the empirical kinship matrix using the above formula. Note that `grm` command excludes SNPs with minor allele frequency below 0.01. This can be changed by the keyword argument `minmaf`. For more information, see [SnpArrays.jl documentation](https://openmendel.github.io/SnpArrays.jl/latest/#Genetic-relationship-matrix-(GRM)).

```julia
# GRM using SNPs with maf > 0.01 (default)
Φgrm = grm(EUR_subset; method = :GRM) # classical genetic relationship matrix
```

    379×379 Array{Float64,2}:
      0.526913     -0.010026     -0.0012793    …   0.00536883    0.00713397
     -0.010026      0.500049      0.00147092      -0.00178778   -0.00344277
     -0.0012793     0.00147092    0.521904        -0.0109387    -0.00262695
     -0.00239381    0.00550462    0.00755985      -0.00265867   -0.000141742
     -0.00391296    0.00422806    0.0222034       -0.0107694    -0.00248895
     -0.000555581   0.000696874   0.0125771    …  -0.0100831    -0.00575495
     -0.0095376     0.00231344   -0.00259641      -0.00282701    0.000732385
     -0.00823869    0.00556861    0.0060825       -0.00911662   -0.00638629
      0.00117402   -0.00444907   -0.0029182       -0.00244795    0.00634087
     -0.0111617     0.00436269    0.000537307     -0.00483523   -0.00621726
     -0.00252813   -0.000626719   0.00753937   …  -0.00180836    0.00714953
      0.0112036    -0.0024306     0.00446458      -0.00983116   -0.00296109
     -0.000451414   0.00707358   -0.00620136      -0.00473171   -0.00720874
      ⋮                                        ⋱
      0.00299953   -0.0113682    -0.00331268       0.00565771    0.00808339
     -0.000521854  -0.00646386   -0.00512474       0.00167572    0.00482433
      0.000970612  -0.00430486   -0.0129335        0.00490171    0.0147362
     -0.000163565  -0.00179531   -0.0108611    …   0.000459623   0.0036255
     -0.00668885    0.00617908   -0.00415613       0.0031146    -0.00221311
      0.00297165   -0.00743712   -0.0137732        0.0058214     0.00386105
     -0.0024849     0.00320647   -0.0127255       -0.0058388     0.00697399
     -0.00166949   -0.0086805     0.00108433       0.0173396     0.0188838
     -0.00412641    0.00783768   -0.00977001   …   0.00661336    0.00828826
      0.00678741   -0.00598927   -0.00121965      -0.00176491   -0.00611699
      0.00536883   -0.00178778   -0.0109387        0.492959      0.0131705
      0.00713397   -0.00344277   -0.00262695       0.0131705     0.512193

You can compute the empirical kinship matrix using other methods: the method of moment method, `grm(A, model=:MoM)`, or the robust method, `grm(A, model=:Robust)`. Uncomment the following commands and try them for yourself!

```julia
# Φgrm = grm(EUR_subset; method = :MoM) # method of moment method
# Φgrm = grm(EUR_subset; method = :Robust) # robust method
```

### Simulating phenotypes

We simulate a phenotype vector from

$$\mathbf{y} \sim \text{Normal}(\mathbf{1}, 0.1 \widehat{\Phi}_{GRM} + 0.9 \mathbf{I})$$

where $\widehat{\Phi}_{GRM}$ is the estimated empirical kinship matrix `Φgrm` and $\mathbf{I}$ is the identity matrix.

The simulated phenotype is available in `pheno.txt`.

### Phenotypes

Read in the phenotype data and plot a histogram.

```julia
using DelimitedFiles
pheno = readdlm("pheno.txt")
```

    379×1 Array{Float64,2}:
      1.846582104608307
      0.12019614558345848
      0.5172368025545149
      0.11933401051509973
      1.8407354203053767
      3.1553094044176166
      1.518422163488851
      0.737544574135081
      1.4904102203720164
      0.4942945743765428
      0.456648703052165
      0.9830094325553045
      1.1241872723791884
      ⋮
      3.2200310377598873
      2.581586233004705
      0.6636455378448316
      1.807461571000498
      1.067894878511225
     -0.06137646537271868
      0.016827921246634125
      0.5801978239606949
      0.08351113077031458
      0.04408173384902825
      0.7937573165253659
      0.9279379783077415

Histogram of phenotype values:

The above plot can be generated and saved using the commands commented out below:

```julia
# using Plots
# hist_pheno = histogram(pheno, xlab="Phenotype", ylab="Frequency", label="") # generate histogram
# png(hist_pheno, "hist_pheno.png") # save the plot as png file
```

### Pre-processing data for heritability analysis

First load the VarianceComponentModels package.

```julia
using VarianceComponentModels
```

Our covariate matrix includes a vector of ones, age, and sex. Both age and sex vectors are randomly generated.

```julia
# load package for random number generation
using Random
# set seed
Random.seed!(123)
# no. of observations
nobs = size(pheno, 1)
# radomly generate age and sex vectors
age = 9*randn(nobs) .+ 30
sex = bitrand(nobs) .+ 1
# covariate matrix
X = [ones(nobs) sex age]
```

    379×3 Array{Float64,2}:
     1.0  2.0  40.7124
     1.0  1.0  48.4336
     1.0  1.0  40.2839
     1.0  1.0  34.1347
     1.0  2.0  26.4299
     1.0  1.0  24.0176
     1.0  2.0  38.8287
     1.0  1.0  29.3207
     1.0  1.0  32.4643
     1.0  2.0  28.2519
     1.0  1.0  26.9457
     1.0  1.0  22.4051
     1.0  1.0  21.9996
     ⋮
     1.0  1.0  38.2156
     1.0  2.0  20.3147
     1.0  2.0  19.8511
     1.0  2.0  20.2451
     1.0  2.0  22.1222
     1.0  2.0  25.6851
     1.0  1.0  22.2179
     1.0  2.0  21.5676
     1.0  1.0  31.895
     1.0  1.0  29.1916
     1.0  2.0  42.7991
     1.0  2.0  32.5419

Now form an instance of `VarianceComponentVariate` by gathering our data.  We plug in the phenotype vector `pheno`, covariate matrix `X` and two matrices corresponding to each variance component $(2\Phi, I)$: `(2Φgrm, Matrix(1.0I, nobs, nobs))`.

```julia
using LinearAlgebra
# form instance of VarianceComponentVariate
EURdata = VarianceComponentVariate(pheno, X, (2Φgrm, Matrix(1.0I, nobs, nobs)))
```

    VarianceComponentVariate{Float64,2,Array{Float64,2},Array{Float64,2},Array{Float64,2}}([1.846582104608307; 0.12019614558345848; … ; 0.7937573165253659; 0.9279379783077415], [1.0 2.0 40.712410928876494; 1.0 1.0 48.43361737010316; … ; 1.0 2.0 42.79905841925565; 1.0 2.0 32.541938843137075], ([1.0538262213132035 -0.02005203927901296 … 0.010737654900514978 0.014267936018676418; -0.02005203927901296 1.0000975487266064 … -0.00357555339105328 -0.006885530334701236; … ; 0.010737654900514978 -0.00357555339105328 … 0.9859187797469418 0.026340992959466743; 0.014267936018676418 -0.006885530334701236 … 0.026340992959466743 1.0243852451056221], [1.0 0.0 … 0.0 0.0; 0.0 1.0 … 0.0 0.0; … ; 0.0 0.0 … 1.0 0.0; 0.0 0.0 … 0.0 1.0]))

### Heritability estimation of single trait

Before fitting the variance component model, we pre-compute the eigen-decomposition of $2\Phi_{\text{GRM}}$, the rotated responses, and the constant part in log-likelihood, and store them as a TwoVarCompVariateRotate instance, which is re-used in various variance component estimation procedures.

We use the Fisher scoring algorithm to fit variance component model for our trait.

```julia
# pre-compute eigen-decomposition
@time EURdata_rotated = TwoVarCompVariateRotate(EURdata)
fieldnames(typeof(EURdata_rotated))

# form data set for trait
trait_data = TwoVarCompVariateRotate(EURdata_rotated.Yrot,
    EURdata_rotated.Xrot, EURdata_rotated.eigval, EURdata_rotated.eigvec,
    EURdata_rotated.logdetV2)

# initialize model parameters
trait_model = VarianceComponentModel(trait_data)

# estimate variance components
_, _, _, Σcov, = mle_fs!(trait_model, trait_data; solver=:Ipopt, verbose=false)
σ2a = trait_model.Σ[1][1] # additive genetic variance
σ2e = trait_model.Σ[2][1] # environmental variance
@show σ2a, σ2e
```

      0.032257 seconds (18 allocations: 3.443 MiB)
    (σ2a, σ2e) = (0.16153996925054054, 0.7622642974157461)

    (0.16153996925054054, 0.7622642974157461)

Additive genetic variance:

```julia
σ2a
```

    0.16153996925054054

Environmental/non-genetic variance:

```julia
σ2e
```

    0.7622642974157461

Now we get heritability estimate and the standard error.

```julia
h, hse = heritability(trait_model.Σ, Σcov)
[h[1], hse[1]] # heritability estimate and standard error
```

    2-element Array{Float64,1}:
     0.1748638484140006
     0.5116600593153141

We can also run MM algorithm.

```julia
trait_model = VarianceComponentModel(trait_data)
@time _, _, _, Σcov, = mle_mm!(trait_model, trait_data; verbose=false)
σ2a = trait_model.Σ[1][1]
σ2e = trait_model.Σ[2][1]
@show σ2a, σ2e
```

      0.730872 seconds (111.83 k allocations: 9.029 MiB)
    (σ2a, σ2e) = (0.17312796952899193, 0.7507276711209097)

    (0.17312796952899193, 0.7507276711209097)

Obtain heritability estimate and its standard error.

```julia
h, hse = heritability(trait_model.Σ, Σcov)
[h[1], hse[1]] # heritability estimate and standard error
```

    2-element Array{Float64,1}:
     0.18739721003078164
     0.5052219467970395

### Multivariate trait analysis

For the joint analysis of multiple traits, see [`VarianceComponentModels.jl` documentation](https://openmendel.github.io/VarianceComponentModels.jl/latest/man/heritability/) and  [`OpenMendel` tutorial](https://github.com/OpenMendel/Tutorials/blob/master/Heritability/HERITABILITY-VCexample.ipynb).

## Testing SNP association using maximum likelihoods of variance component models
credit: [Tutorial by Sarah Ji, Janet Sinsheimer and Hua Zhou](https://github.com/OpenMendel/Tutorials/blob/master/Heritability/HERITABILITY-VCexample.ipynb)

Suppose we want to see a group of SNPs has an effect on a given phenotype after accounting for relatedness among individuals. Here we fit variance component model with SNPs as fixed effect.

$$\hspace{5em}  \mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{G}_s \gamma + \mathbf{g} + \mathbf{\epsilon} \hspace{5em} (1)$$

\begin{equation}
\begin{array}{ll}
\mathbf{g} \sim N(\mathbf{0}, \sigma_g^2\mathbf{\Phi}) \\
\mathbf{\epsilon} \sim N(\mathbf{0}, \sigma_e^2\mathbf{I})
\end{array}
\end{equation}

$$\hspace{5em}  \mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{G}_s \gamma + \mathbf{g} + \mathbf{\epsilon}, \hspace{1em} \mathbf{g} \sim N(\mathbf{0}, \sigma_g^2\mathbf{\Phi}), \hspace{1em} \mathbf{\epsilon} \sim N(\mathbf{0}, \sigma_e^2\mathbf{I})$$

where

* $\mathbf{y}$: phenotype
* Fixed effects:
    * $\mathbf{X}$: matrix of covariates including intercept
    * $\beta$: vector of covariate effects, including intercept
    * $\mathbf{G}_s$: genotype matrix of SNPs of interest
    * $\gamma$: association parameter of interest, measuring the effect of genotype on phenotype
* Random effects:
    * $\mathbf{g}$: random vector of polygenic effects with $\mathbf{g} \sim N(\mathbf{0}, \sigma_g^2 \mathbf{\Phi})$
        * $\sigma_g^2$: additive genetic variance
        * $\mathbf{\Phi}$: matrix of pairwise measures of genetic relatedness
    * $\epsilon$: random vector with $\epsilon \sim N(\mathbf{0}, \sigma_e^2\mathbf{I})$
        * $\sigma_e^2$: non-genetic variance due to non-genetic effects assumed to be acting independently on individuals

To test whether our SNPs are associated with phenotype, we fit two models. First consider the model without SNPs as fixed effects (aka null model):

$$\hspace{5em}  \mathbf{y} = \mathbf{X}\mathbf{\beta} + \mathbf{g} + \mathbf{\epsilon} \hspace{5em} (2)$$

and the model with SNPs as fixed effects (1). Then we can compare the log likelihood to see if there is improvement in the model fit with inclusion of our SNPs of interest.

### Fit the null model

```julia
using VarianceComponentModels
```

```julia
# covariate matrix without SNP information
X = [ones(nobs) sex age]
# construct VarianceComponentVariate instance
nulldata = VarianceComponentVariate(pheno, X, (2Φgrm, Matrix(1.0I, nobs, nobs)))
# initialize model
nullmodel = VarianceComponentModel(nulldata)
# fit the model using Fisher scoring algorithm
@time nulllogl, nullmodel, = fit_mle!(nullmodel, nulldata; algo=:FS)
```

    This is Ipopt version 3.13.2, running with linear solver mumps.
    NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

    Number of nonzeros in equality constraint Jacobian...:        0
    Number of nonzeros in inequality constraint Jacobian.:        0
    Number of nonzeros in Lagrangian Hessian.............:        3

    Total number of variables............................:        2
                         variables with only lower bounds:        0
                    variables with lower and upper bounds:        0
                         variables with only upper bounds:        0
    Total number of equality constraints.................:        0
    Total number of inequality constraints...............:        0
            inequality constraints with only lower bounds:        0
       inequality constraints with lower and upper bounds:        0
            inequality constraints with only upper bounds:        0

    iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
       0  5.6697935e+02 0.00e+00 1.00e+02   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
       5  5.2279951e+02 0.00e+00 1.91e-02 -11.0 1.33e-02    -  1.00e+00 1.00e+00f  1 MaxS
      10  5.2279950e+02 0.00e+00 5.29e-08 -11.0 4.14e-07    -  1.00e+00 1.00e+00f  1 MaxSA

    Number of Iterations....: 11

                                       (scaled)                 (unscaled)
    Objective...............:   5.1285503705850340e+02    5.2279950327884421e+02
    Dual infeasibility......:   6.5524759453520796e-09    6.6795310993228422e-09
    Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
    Complementarity.........:   0.0000000000000000e+00    0.0000000000000000e+00
    Overall NLP error.......:   6.5524759453520796e-09    6.6795310993228422e-09

    Number of objective function evaluations             = 12
    Number of objective gradient evaluations             = 12
    Number of equality constraint evaluations            = 0
    Number of inequality constraint evaluations          = 0
    Number of equality constraint Jacobian evaluations   = 0
    Number of inequality constraint Jacobian evaluations = 0
    Number of Lagrangian Hessian evaluations             = 11
    Total CPU secs in IPOPT (w/o function evaluations)   =      0.053
    Total CPU secs in NLP function evaluations           =      0.323

    EXIT: Optimal Solution Found.
      0.105506 seconds (12.62 k allocations: 4.373 MiB)

    (-522.7995032788442, VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([1.0982528445105832; -0.049790965461479236; -0.0011173387859553952], ([0.16153996925054054], [0.7622642974157461]), Array{Float64}(undef,0,3), Char[], Float64[], -Inf, Inf), ([0.28637160399826644], [0.29037628696484363]), [0.08200869557653992 -0.08091120209956988; -0.0809112020995699 0.08431838803148921], [0.22756326068408744; 0.09903256896133299; 0.005218403921070817], [0.05178503761317394 -0.01572358847762115 -0.0008603839232000639; -0.015723588477620322 0.009807449715081174 2.7208545465831223e-5; -0.000860383923200106 2.7208545465857677e-5 2.7231739483447272e-5])

The null model log-likelihood (no SNP effects)

```julia
nulllogl
```

    -522.7995032788442

The null model mean effects (a grand mean)

```julia
nullmodel.B
```

    3×1 Array{Float64,2}:
      1.0982528445105832
     -0.049790965461479236
     -0.0011173387859553952

The null model additive genetic variance

```julia
nullmodel.Σ[1]
```

    1×1 Array{Float64,2}:
     0.16153996925054054

The null model environmental variance

```julia
nullmodel.Σ[2]
```

    1×1 Array{Float64,2}:
     0.7622642974157461

### Fit the alternative model

For the alternative model, we include genotypes of SNPs of interest in the covariate matrix.

```julia
# extract genotypes of SNPs of interest
snp_mat = convert(Matrix{Float64}, EUR_subset[:, 10:20])
# covariate matrix with SNP information
Xalt = [ones(nobs) sex age snp_mat]
# construct VarianceComponentVariate instance
altdata = VarianceComponentVariate(pheno, Xalt, (2Φgrm, Matrix(1.0I, nobs, nobs)))
# initialize model
altmodel = VarianceComponentModel(altdata)
# fit the model using Fisher scoring algorithm
@time altlogl, altmodel, = fit_mle!(altmodel, altdata; algo=:FS)
```

    This is Ipopt version 3.13.2, running with linear solver mumps.
    NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

    Number of nonzeros in equality constraint Jacobian...:        0
    Number of nonzeros in inequality constraint Jacobian.:        0
    Number of nonzeros in Lagrangian Hessian.............:        3

    Total number of variables............................:        2
                         variables with only lower bounds:        0
                    variables with lower and upper bounds:        0
                         variables with only upper bounds:        0
    Total number of equality constraints.................:        0
    Total number of inequality constraints...............:        0
            inequality constraints with only lower bounds:        0
       inequality constraints with lower and upper bounds:        0
            inequality constraints with only upper bounds:        0

    iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
       0  5.6322585e+02 0.00e+00 1.00e+02   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
       5  5.1493071e+02 0.00e+00 1.47e-03 -11.0 1.44e-03    -  1.00e+00 1.00e+00f  1 MaxS
      10  5.1493071e+02 0.00e+00 3.75e-07 -11.0 4.97e-07    -  1.00e+00 1.00e+00f  1 MaxSA

    Number of Iterations....: 13

                                       (scaled)                 (unscaled)
    Objective...............:   4.8516209021328206e+02    5.1493071005158538e+02
    Dual infeasibility......:   3.1521046336854756e-09    3.3455117576623036e-09
    Constraint violation....:   0.0000000000000000e+00    0.0000000000000000e+00
    Complementarity.........:   0.0000000000000000e+00    0.0000000000000000e+00
    Overall NLP error.......:   3.1521046336854756e-09    3.3455117576623036e-09

    Number of objective function evaluations             = 14
    Number of objective gradient evaluations             = 14
    Number of equality constraint evaluations            = 0
    Number of inequality constraint evaluations          = 0
    Number of equality constraint Jacobian evaluations   = 0
    Number of inequality constraint Jacobian evaluations = 0
    Number of Lagrangian Hessian evaluations             = 13
    Total CPU secs in IPOPT (w/o function evaluations)   =      0.067
    Total CPU secs in NLP function evaluations           =      0.442

    EXIT: Optimal Solution Found.
      0.114052 seconds (14.55 k allocations: 5.188 MiB)

    (-514.9307100515854, VarianceComponentModel{Float64,2,Array{Float64,2},Array{Float64,2}}([-0.5650772534154904; -0.04175952202257888; … ; 0.24813567380735943; -0.22240223729988207], ([0.36160628542343937], [0.5266911593044307]), Array{Float64}(undef,0,14), Char[], Float64[], -Inf, Inf), ([0.282704223783742], [0.2795753961755862]), [0.07992167814516807 -0.07694976655753723; -0.07694976655753723 0.07816240214673596], [1.3865711843978776; 0.09836535838981278; … ; 0.2505911842018573; 0.25472360006392125], [1.9225796494025333 -0.01681528153698582 … -0.13499832245206594 -0.12824841110772908; -0.016815281537015045 0.009675743731156311 … 0.0019277350041501063 -0.0018897539417217569; … ; -0.13499832245239285 0.0019277350041509665 … 0.06279594159968915 -0.022119153796151553; -0.12824841110814747 -0.001889753941720035 … -0.022119153796144195 0.06488411242952451])

The alternative model log-likelihood

```julia
altlogl
```

    -514.9307100515854

The alternative model mean effects

```julia
altmodel.B
```

    14×1 Array{Float64,2}:
     -0.5650772534154904
     -0.04175952202257888
     -0.0020631936965069325
      0.13798788088282812
     -0.06458855409089669
      0.02775601692768474
      0.15120382884135317
      0.06392636975031145
      0.04805343647025799
      0.05097779763588679
     -0.01656719306048999
      0.18457721192458673
      0.24813567380735943
     -0.22240223729988207

The alternative model additive genetic variance

```julia
altmodel.Σ[1]
```

    1×1 Array{Float64,2}:
     0.36160628542343937

The alternative model environmental variance

```julia
altmodel.Σ[2]
```

    1×1 Array{Float64,2}:
     0.5266911593044307

### Likelihood ratio test

We use likelihood ratio test (LRT) to test the goodness-of-fit between two models.

Our likelihood ratio test statistic is around 15.7 (distributed chi-squared), with 11 degrees of freedom.

```julia
using Distributions
LRT = 2(altlogl - nulllogl)
```

    15.737586454517668

The associated p-value

```julia
pval = ccdf(Chisq(11), LRT)
```

    0.15115585964284117

We see that adding the SNPs as covariates to the model does not fit significantly better than the null model. In other words, the SNPs do not explain more of the variation in our trait.

### Exercise

Use minorization-maximization algorithm (`algo=:MM`) to find MLEs of both null model and alternative model. Then conduct the likelihood ratio test.


### 심파이를 사용한 함수 미분

- 예측 모형의 성능
     - 예측 오차가 가장 적은 최적의 예측 모형을 구하는 일
     - 함수의 계수를 바꾸는 과정
     - 선형 예측 모형의 가중치 값을 바꾸는 것을 고려
     - 모수는 항상 스칼라가 되어야 한다.
     - 일종의 다변수 함수를 구하는 것을 고려
     - 성능 함수라고 한다.
     - 손실함수, 비용함수, 오차함수라 부른다 => 목적함수

     - 최적화 : 모수를 구하는 과정, 목적 함수의 값을 구하기
     - 미분 : 최적화를 하기 위해 입력값에 따라 함수의 출력값이 어떻게 변하는지 확인해야 함. 미분이 그런 과정

         - 예제 : 조절나사 스피커 앰프 각도 찾는 문제

- 기울기
    - 수치적 최적화는 가장 적은 횟수로 여러가지 x값을 시도하여 최적의 값을 찾는 방법
    - 기울기, 민감도라 한다.
        - 예 : 조절나사 오른쪽 1도 이동시 소리가 작아지나 커지나
    - 입력변수 비율 변화 대비 출력 변수 비율 변화 -> 변화 값에 따라 변화율이 달라진다.
    - 기울기 : slope, dx값이 0으로 근접할 때의 변화율 의미
         - 함수의 그래프 관점에서 보는 경우
         부드러운 곡선의 형태로 나타난다. 한 점만 공통으로 가지는 접선으로 보는 경우, 기울기는 x 방향 이동거리와 y 방향 이동거리의 비율을 의미한다.

```python
def f(x):
    return x**3 - 3 * x**2 + x
```

```python
x = np.linspace(-1, 3, 400)
y = f(x)
```

```python
plt.plot(x, y)
plt.plot(0, 0, 'ro')
plt.plot(x, x, 'r:')
plt.plot(1, -1, 'go')
plt.plot(x, (3*1**2-6*1+1)*(x-1)-1, 'g--')

plt.xlim(-3.5, 5.5)
plt.ylim(-4, 2)
plt.xticks(np.arange(-3, 6))
plt.yticks(np.arange(-4, 2))

plt.annotate('', xy=(1, 0), xytext=(0, 0), arrowprops=dict(facecolor='gray'))
plt.annotate('', xy=(1, 1), xytext=(1, 0), arrowprops=dict(facecolor='gray'))

plt.annotate('', xy=(2, -1), xytext=(1, -1), arrowprops=dict(facecolor='gray'))
plt.annotate('', xy=(2, -3), xytext=(2, -1), arrowprops=dict(facecolor='gray'))

plt.xlabel("x")
plt.ylabel("f(x)")
plt.title("함수의 기울기")
plt.show()
```

- 수치미분
    - 기울기를 구하는 방법 - 대략적으로 계산해준다.
    - scipy.mis 패키지에서 derivative()명령 사용하여 수치적으로 대략적인 기울기 계산 가능
    - 함수 f, 기울기 x, 이동거리 dx
    - dx 도 임의로 입력해줘야 함. 너무 작으면 부동소수점의 오버플로우 오류 발생

```python
from scipy.misc import derivative

print(derivative(f, 0, dx=1e-6))
print(derivative(f, 1, dx=1e-10))
```

    1.000000000001
    -2.000000165480742

- 미분
    - 어떤 함수로부터 그 함수 기울기를 출력하는 새로운 함수를 만들어내는 작업
    - 도함수 : 미분으로 만들어진 함수
    - 수렴과 극한을 이용한 설명은 생략. 개념 이해와 암기
    - 도함수의 기호는 분수가 아니다.
    - f' -> f프라임, df/dx -> df 오버(over) dx

- 미분가능
    - 기울기를 구할 수 있는 경우

```python
def relu(x):
    return np.where(x > 0, x, 0)

xx = np.linspace(-2, 2, 100)
plt.plot(xx, relu(xx))
plt.plot(0, 0, marker='o', ms=10)
plt.title("ReLU")
plt.xlabel("$x$")
plt.ylabel("$ReLU(x)$")
plt.show()

```

- 미분공식
    - 기본 미분공식
    - 선형조합법칙
    - 곱셈법칙
    - 연쇄법칙

- 기본 미분공식
    - 상수, 거듭제곱, 로그함수, 지수함수 등 에 대한 미분공식
        - 상수 : 상수 미분하면 0이 된다.
        - 거듭제곱 : x의 n제곱을 미분하면 n-1 제곱으로 제곱수가 1씩 감소. n = 0 일 때 성립하지 않는다.
        - 로그함수 : 로그함수를 미분하면 1/x가 된다.
        - 지수 : 밑이 오일러 수인 지수함수는 미분해도 변하지 않는다.

- 선형조합법칙
    - 어떤 함수에 상수를 곱한 함수를 미분한 결과는 원래 함수의 도함수에 그 상수를 곱한 것과 같다.
    - 선형조합 : 어떤 함수에 각각 상수를 곱한 후 더한 것은 각 함수의 도함수를 선형조합한 것과 같다.

```python
def fprime(x):
    return 3 * x ** 2 - 6 * x + 1

# 2차 방정식의 근
x1, x2 = np.roots([3, -6, 1]) # 3x^2 - 6x +1 식의 근을 구해준다.
# 참고자료 : https://wikidocs.net/23285

x = np.linspace(-1, 3, 400)

plt.figure(figsize=(10, 7))
plt.subplot(211)
plt.plot(x, f(x))
plt.xlim(-2, 4)
plt.xticks(np.arange(-1, 4))
plt.yticks(np.arange(-5, 4))
plt.xlabel('x')
plt.ylabel('함수 f(x)')
plt.axvline(x1, c="b", ls="--")
plt.axvline(x2, c="b", ls="--")

plt.subplot(212) # 그래프 2개 두번째에 위치
plt.plot(x, fprime(x)) # 플롯을 그려준다. fprime 함수에 대한
plt.xlim(-2, 4) # x축에 대해 범위 설정 여기는 -2 ~ 4 까지
plt.xticks(np.arange(-1, 4)) # 그래프가 그려진다면 x축은 어디까지인지
plt.yticks(np.arange(-3, 11)) # 그래프가 그려진다면 y축은 어디까지인지
plt.xlabel('x') # x축에 대한 범례
plt.title("도함수 f'(x)") # y 축에 대한 범례
plt.axhline(0, c="r", ls="--") # 수평선 그러주기, 색깔은 red, 선 모양은 --
plt.axvline(x1, c="b", ls="--") # 구한 근에 대한 x1의 수직선 그려주기 색깔은 blue, 모양은 --
plt.axvline(x2, c="b", ls="--") # 구한 근에 대한 x2의 수직선 그려주기 색깔은 blue, 모양은 --

plt.tight_layout() # 적절하게 표현해줘라
plt.show() # Display a figure
```

- 곱셈법칙
    - 각 개별 함수의 도함수를 사용하여 원래 함수의 도함수를 구한다.

- 연쇄법칙
    - 미분하고자 하는 함수의 입력변수가 다른 함수의 출력 변수인 경우

- 2차 도함수
    - 도함수를 한번 더 미분하여 만들어진 함수
    - '' -> 두번 표시 한 것
    - 도함수의 기울기를 나타낸다.
    - 도함수 값 증가 -> 2차 도함수 값 양수 => 볼록(convex)
    - 도함수 값 감소 -> 2차 도함수 값 음수 => 오목(convexity)
    - 볼록도라고 한다.

```python
def fprime2(x):
    return 6*x - 6

# 2차 방정식의 근
x1, x2 - np.roots([3, -6, 1])

x = np.linspace(-1, 3, 400)

plt.figure(figsize=(10, 10))

plt.subplot(311)
plt.plot(x, f(x))
plt.xlim(-2, 4)
plt.xticks(np.arange(-1, 4))
plt.yticks(np.arange(-5, 4))
plt.title('함수 f(x)')
plt.xlabel('x')
plt.axvline(x1, c="b", ls="--")
plt.axvline(x2, c="b", ls="--")
plt.axvline(1, c="g", ls=":")

plt.subplot(312)
plt.plot(x, fprime(x))
plt.xlim(-2, 4)
plt.xticks(np.arange(-1, 4))
plt.yticks(np.arange(-3, 11))
plt.title("도함수 f'(x)")
plt.xlabel('x')
plt.axhline(0, c='r', ls="--")
plt.axvline(x1, c="b", ls="--")
plt.axvline(x2, c="b", ls="--")
plt.axvline(1, c="g", ls=":")

plt.subplot(313)
plt.plot(x, fprime2(x))
plt.xlim(-2, 4)
plt.xticks(np.arange(-1, 4))
plt.title("2차 도함수 f'(x)")
plt.axhline(0, c='r', ls="--")
plt.axvline(1, c='g', ls=":")

plt.tight_layout()
plt.show()

```

- 편미분
    - 기울기는 하나의 변수에 대해서 구할 수 있다.
    - 둘 이상의 독립변수를 가지는 다변수 함수 인 경우라도
    - 표시방법 : 독립 변수(미분해아하는 변수)를 함수의 아랫첨자로 표기, 라운드 기호를 사용하여 표기.
    - 어떤 하나의 독립 변수에 대해 미분할 때는 다른 독립 변수를 상수로 생각하여 한다.

- 다변수 함수의 연쇄법칙
    - 다변수 함수 미분도 연쇄법칙 적용

- 테일러 전개
     - 함수의 기울기(1차 미분값)를 안다면 함수의 모양 근사화 가능.

- 심파이
    - 심볼릭 연산 지원 패키지

```python
import sympy

sympy.init_printing(use_latex='mathjax')
```

```python
x = sympy.symbols('x')
x
type(x)
```

    sympy.core.symbol.Symbol

```python
f = x * sympy.exp(x)
f
```

$\displaystyle x e^{x}$

```python
sympy.diff(f)
```

$\displaystyle x e^{x} + e^{x}$

```python
sympy.simplify(sympy.diff(f))
```

$\displaystyle \left(x + 1\right) e^{x}$

```python
x, y = sympy.symbols('x y')
f = x ** 2 + 4 * x * y + 4 * y ** 2
f
```

$\displaystyle x^{2} + 4 x y + 4 y^{2}$

```python
sympy.diff(f, x)
```

$\displaystyle 2 x + 4 y$

```python
sympy.diff(f, y)
```

$\displaystyle 4 x + 8 y$

```python
x, mu, sigma = sympy.symbols('x mu sigma')
f = sympy.exp((x - mu) ** 2 / sigma ** 2)
f
```

$\displaystyle e^{\frac{\left(- \mu + x\right)^{2}}{\sigma^{2}}}$

```python
sympy.diff(f, x)
```

$\displaystyle \frac{\left(- 2 \mu + 2 x\right) e^{\frac{\left(- \mu + x\right)^{2}}{\sigma^{2}}}}{\sigma^{2}}$

```python
sympy.simplify(sympy.diff(f, x))
```

$\displaystyle \frac{2 \left(- \mu + x\right) e^{\frac{\left(\mu - x\right)^{2}}{\sigma^{2}}}}{\sigma^{2}}$

```python
x = sympy.symbols('x')
f = x ** 3 - 1
sympy.diff(f , x)
```

$\displaystyle 3 x^{2}$

```python
x, k = sympy.symbols('x k')
f = sympy.log(x ** 2 - 3 * k)
sympy.diff(f , x)
```

$\displaystyle \frac{2 x}{- 3 k + x^{2}}$

```python
x, a, b = sympy.symbols('x a b')
f = sympy.exp(a * x ** b)
sympy.simplify(sympy.diff(f, x))
```

$\displaystyle a b x^{b - 1} e^{a x^{b}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, x))
```

$\displaystyle 2 x e^{x^{2} + 2 y^{2}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, y))
```

$\displaystyle 4 y e^{x^{2} + 2 y^{2}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, x, x))
```

$\displaystyle \left(4 x^{2} + 2\right) e^{x^{2} + 2 y^{2}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, x, y))
```

$\displaystyle 8 x y e^{x^{2} + 2 y^{2}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, y, y))
```

$\displaystyle \left(16 y^{2} + 4\right) e^{x^{2} + 2 y^{2}}$

```python
x , y = sympy.symbols('x y')
f = sympy.exp(x ** 2 + 2 * y ** 2)
sympy.simplify(sympy.diff(f, y, x))
```

$\displaystyle 8 x y e^{x^{2} + 2 y^{2}}$


# Some Background on IPython

Let's start with some background on what IPython is,
then we can talk about what parallel IPython is,
and why it makes sense.

# What is IPython?

# Level 0: Interactive Python

```python
def foo():
    """docstring"""
    return 'hi'
foo()
```

```python
def bar():
    print 'hi'
```

```python
from __future__ import print_function

import time

for i in range(10):
    time.sleep(0.25)
    print(i, end=' ')

```

```python
import numpy as np
```

```python
import ctypes
libc = ctypes.CDLL('libc.dylib')
libc.time(-1)
```

```python
print("I'm okay!")
```

# Level 1: A bit more than Python

```python
import numpy as np
```

```python
np.
```

```python
np.linspace?
```

## Shell interaction

```python
!date
```

```python
ls
```

```python
files = !ls
```

```python
for f in files:
    print(repr(f))
```

## Extending Python with `%magics`

```python
biglist = range(1000000)
bigset = set(biglist)

item = biglist[0]
%timeit item in biglist
%timeit item in bigset
```

```python
def inner(x):
    return 1/x

def outer(y):
    return inner(1-y)

z = 2
outer(-z + 3)
```

```python
%debug
```

```python
%lsmagic
```

# Level 2: A bit more than the Terminal

## Rich Display

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.special import jn
```

```python
x = np.linspace(0,10)
for n in range(5):
    plt.plot(x, jn(n,x))
```

```latex
%%latex
\begin{eqnarray}
\nabla \times \vec{\mathbf{B}} &
    - \frac{1}{c} \frac{\partial\vec{\mathbf{E}}}{\partial t}
    & = & \frac{4 \pi}{c} \vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} && = & 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}} &
    + \frac{1}{c} \frac{\partial\vec{\mathbf{B}}}{\partial t}
    & = & \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} && = & 0
\end{eqnarray}

```

```python
import sympy
sympy.init_printing(use_latex='mathjax')
x = sympy.symbols('x')
expr = x**10-1
expr
```

```python
sympy.factor(expr)
```

```python
from IPython.display import display
from IPython.html.widgets import interact

@interact
def factorit(n=10):
    display(sympy.factor(x**n-1))

```

```python
from IPython.display import Image, YouTubeVideo
Image("figs/darts.png")
```

```python
YouTubeVideo('BROWqjuTM0g ')
```

## Level 3: Documents

## In the Notebook: not just code

We also have

- *markdown* and
- $\LaTeX$

```javascript
var foo = $(".class");
```

## What is a notebook?

```python
name = "Background"
!head -n 100 {name}.ipynb | pygmentize -l json
```

```python
!ipython nbconvert {name}.ipynb --to html

from IPython.display import display, HTML
display(HTML("<a href='{name}.html' target='_blank'>{name}.html</a>".format(name=name)))
```

# But what is IPython, *really*?

[*A Protocol*](http://ipython.org/ipython-doc/dev/development/messaging.html)

Message type: `execute_request`

```python
content = {
    # Source code to be executed by the kernel, one or more lines.
    'code' : str,

    # A boolean flag which, if True, signals the kernel to execute
    # this code as quietly as possible.
    'silent' : bool,

    # A boolean flag which, if True, signals the kernel to populate history
    # The default is True if silent is False.  If silent is True, store_history
    # is forced to be False.
    'store_history' : bool,
    ...
}
```

Message type: `execute_reply`

```python
content = {
  # One of: 'ok' OR 'error'
  'status' : str,

  # The global kernel counter that increases by one with each request that
  # stores history.  This will typically be used by clients to display
  # prompt numbers to the user.  If the request did not store history, this will
  # be the current value of the counter in the kernel.
  'execution_count' : int,
}
```

Message type: `display_data`

```python
content = {

    # The data dict contains key/value pairs, where the kids are MIME
    # types and the values are the raw data of the representation in that
    # format.
    'data' : dict,

    # Any metadata that describes the data
    'metadata' : dict,
}
```

```javascript
%%javascript
IPython.notebook.kernel.execute("a=1");
```

```python
print(a)
```

The protocol is [publicly documented](http://ipython.org/ipython-doc/dev/development/messaging.html), and [language agnostic](https://github.com/ipython/ipython/wiki/Projects-using-IPython#list-of-some-ipython-compatible-kernels).

## So what is IPython, really?

- Tools for interactive computing
  - documents
  - interfaces
    - terminal
    - qtconsole
    - notebook
  - introspection
  - debugging
- A message protocol for remote execution
  - language-agnostic
  - rich remote repl

## How does this become IPython.parallel?

Let's see a [quick demo](examples/Parallel%20image%20processing.ipynb), and then [find out](Overview.ipynb) in more detail.