# Binomial Tree

discrete-time models

- Single Period Binomial model
- Multiperiod Binomial model$\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\plim}{plim}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\asim}{\overset{\text{a}}{\sim}}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\abs}[1]{\left| #1 \right|}
\newcommand{\norm}[1]{\left\| #1 \right\|}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\Exp}{\mathrm{E}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\EE}{\mathbb{E}}
\newcommand{\II}{\mathbb{I}}
\newcommand{\NN}{\mathbb{N}}
\newcommand{\ZZ}{\mathbb{Z}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\PP}{\mathbb{P}}
\newcommand{\AcA}{\mathcal{A}}
\newcommand{\FcF}{\mathcal{F}}
\newcommand{\AsA}{\mathscr{A}}
\newcommand{\FsF}{\mathscr{F}}
\newcommand{\Var}[2][\,\!]{\mathrm{Var}_{#1}\left[#2\right]}
\newcommand{\Avar}[2][\,\!]{\mathrm{Avar}_{#1}\left[#2\right]}
\newcommand{\Cov}[2][\,\!]{\mathrm{Cov}_{#1}\left(#2\right)}
\newcommand{\Corr}[2][\,\!]{\mathrm{Corr}_{#1}\left(#2\right)}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathcal{N} \left( #1 \right)}
\newcommand{\ow}{\text{otherwise}}
\newcommand{\FSD}{\text{FSD}}$

## Single Period Binomial model

Suppose the market to be observed at just two times: $0$ and $T$, where it has two possible states at time $T$:

- $P\CB{S_t = S_0 \cdot u} = p$
- $P\CB{S_t = S_0 \cdot d} = 1-p$

Here, $p$ is the ***objective (actual) probability***.

$Remark$ Portfolio and Arbitrage

>Using two instruments: a bond and a stock; to build the portfolio, denoted as a vector $h\P{x,y}$ where $x$ and $y$ denote the number of bonds and number of units of the stock in the portfolio, respectively.
>
>$Def$ ***Value Process***
>
>The value process of the portfolio is $V_t^h = x \cdot B_t + y \cdot S_t$, where $B_t$ is the bond price at time $t$ and we let $B_t = e^{rt}$
>
>$Def$ **Arbitrage Portfolio**
>
>An arbitrage portfolio is a portfolio $h$ that satisfies
>
>- $V_0^h = 0\\[0.5em]$. At time $0$ there's nothing in your hand.
- $V_T^h\P{\omega} \geq 0\\[0.5em]$. $\omega$ is the possible states in time $T$.
- $\Exp\SB{V_T^h\P{\omega}} > 0$. $\Exp\SB\cdot$ is the expectation under the actual probability measure.

And in **Binomial Model**, $S_T = S_0d$ or $S_T = S_0u$ are the two states for $\omega$, with actual probability $d$ and $u$ respectively. $\Exp\SB{V_T^h\P{\omega}} = u\cdot V_T^h\P{S_T = S_0u} + d\cdot V_T^h\P{S_T = S_0d}$

$Lemma$

The single period binomial model is *free of arbitrage* $iff$ the $d < e^{rT} < u$

$Proof$

> $\Rightarrow)$: Here we're gonna first suppose that $e^{rT}$ is *not* between $d$ and $u$ and derive that the model is *not* free of arbitrage. Skipped, too easy man. The arbitrage strategy we're gonna construct is $h\P{S_0,-1}$ or $h\P{-S_0,1}$
>
>$\Leftarrow)$: Assume that there's an arbitrage portfolio $h \P{x,y}$ such that $V_0^h = 0$ then, $x = -yS_0$. Then at time $T$, we have

>$$\bspace \begin{align}
V_T^h &= -yS_0e^{rT} + yS_T \\[0.6em]
&= \begin{cases}
yS_0 \P{u - e^{rT}}, &\text{if } S_T = S_0 \cdot u \\[0.6em]
yS_0 \P{d - e^{rT}}, &\text{if } S_T = S_0 \cdot d
\end{cases}
\end{align}$$
>
>And no matter $y$ is greater of less than $0$, since $u$ and $d$ are on the different size of $e^{rT}$, $V_T^h$ can't be just bigger than $0$, thus free of arbitrage.

$Def$ **contingent claim**

A ***contingent claim (or financial derivative)*** is any stochastic variable $U$ of the form $\Phi(Z)$ (nonlinear or linear function), where $Z$ can be the stochastic variable driving the stock price process.

$Remark$

>$Z$ is a *stochastic variable* because it has two periods of time, $0$ and $T$, and at each time period the price can be treated as a $r.v.$ and it actually is at time $T$ for $S_T = S_0\cdot u$ with probability $u$ and $S_0\cdot d$ with probability $d$.

Say,  for an european call option, we have $U = \P{S_T - K}^+ = \Phi\P{S_T}$

That **claim** at time $T$, is ***attainable*** if it can be ***hedged/replicated***. That is, there exists a portfolio $h$ whose value at time $T$ is *exactly* $U$ ($i.e.$ $V_T^h = U$ $wp1$). Then $h$ is called a ***hedging portfolio*** or a ***replicating portfolio***.

There is NO difference between holding the claim and holding the portfolio.

$Proposition$

Suppose that a claim $U$ at time $T$ is attainable with replicating portfolio $h$. Denote the price of the claim at time $t$ by $\Pi\P{t, U}$, then $\Pi\P{t,U} = V_t^h$.

## No-arbitrage for options pricing

**e.g.**

A stock price is currently $20$, and it will be either $22$ or $18$ at the end of the next $3$ months. The risk-free interest rate is $12\%$ per annum. What is the price of a European call option to buy the stock for $21$ in $3$ months?

> We try to replicate the option with $x$ bonds and $y$ shares of stock. Then:
>
>$$V_t^h = x\cdot e^{rt} + y \cdot S_t\\[0.6em]
V_T^h = \begin{cases}
x \cdot e^{0.12*3/12} + 22\cdot y = \P{22-21}^+, & \text{state } 1\\
x \cdot e^{0.12*3/12} + 18\cdot y = \P{18-21}^+, & \text{state } 2
\end{cases}\\[1em]
\Longrightarrow x = -4.5 e^{-0.03}, y=0.25$$
>
>Thus we price the option as: $c = V_0^h = x + y\cdot S_0 = -4.5 e^{-0.03} + 0.25 \times 20 = 0.633$

$Remark$

>By solving the equation set of two equations, actually it's an implicaiton of the **binomial tree** model. For two states results in two equations and two unknown variables related to a bond and some certain cash, respectively.

**e.g.** **binomial option pricing formula**

Using the ***riskless hedging principle*** to derive that **formula**. The portfolio is consisted of a short position $\Delta$ shares of stock and a long position in one call option.

> First, how to say about the *risk*? Here it can be simply regarded as a variable with NO variance. Thus, $0$ changes. Then we have (following the last example):
>
>$\begin{cases}
\P{uS_0 - 21}^+ - \Delta u S_0 = \P{c - \Delta S_0} e^{rT}\\
\P{dS_0 - 21}^+ - \Delta d S_0 = \P{c - \Delta S_0} e^{rT}
\end{cases}$
>
>Solve it and that's all.

$Remark$

>Easy to see that this equation set is the same with the one before by letting $\Delta = y$.
>
> Mathematically, the ***Delta*** of stock option is the ratio of the change in the price of the stock option to the change in the price of the underlying stock. And financially, it's also the number of units of the stock we hold for each option shorted in order to create a riskless portfolio (Solving this **Delta** is called ***delta hedging***).

***

**Summary**

> let $c$ denote a European call option price at time $0$; and $c_u$ ($c_d$) the call option price at time $T$ corresponding to the up-move (down-move) of the asset price.
>
>Then we construct the portfolio $h$ that replicates the long position of a call option:
>
>$$\begin{cases}
x\cdot e^{rT} + y \cdot uS_0 = c_u = \P{uS_0 - K}^+\\
x\cdot e^{rT} + y \cdot dS_0 = c_d = \P{dS_0 - K}^+
\end{cases} \\[1em]
\Longrightarrow \boxed{x = \ffrac{u\cdot c_d - d\cdot c_u} {\P{u-d}e^{rT}}, y = \ffrac{c_u - c_d} {\P{u-d}S_0}}$$

$Remark$

>An interesting fact about the solution is that $x \leq 0$ and $y \geq 0$. Prove them by write $c_d = \max\P{dS_0 - k,0}$ and $c_u = \max\P{uS_0 - K,0}$
>
> Thus we substitute the value back and obtain:
>
>$$c = x + yS_0 = e^{-rT}\SB{q c_u + \P{1-q}c_d}, q = \ffrac{e^{rT}-d} {u-d}$$
>
>And this also has some connection with the previous lemma, that $d < e^{rT} < u$, thus $0 < q < 1$, could be considered as a probability and $c$, a expectation.

$Remark$

>To understand the relaiton of $p$ and $q$, here we consider $p$ as a sign of the symbol of what kinds of state the stock CAN have. Then $q$ is just a new probability measure.
>
>Then we have $c = e^{-rT}\SB{q c_u + \P{1-q}c_d} = e^{-rT} \Exp^Q\SB{\P{S_T-K}^+}$. Here $\Exp^Q$ is the  expectation with respect to the probability measure $Q$.
>
>Also, we can calculate the expeaction under $Q$ of $S_T$: $\Exp^Q\SB{S_T} = qS_0u + \P{1-q}S_0d$, then we substitute $q$ back, and then, Boom!
>
>$$S_0 = e^{-rT} \Exp^Q\SB{S_T}$$
>
>Thus, A probability measure $Q$ is called ***martingale measure*** (or ***risk neutral measure***) if $S_0 = e^{-rT} \Exp^Q\SB{S_T}$.
>
>And $e^{-rt}S_t$ is the martingale since $\Exp^Q\SB{e^{-rT}S_T} = e^{-r\cdot0}S_0$, true!

$Remark$

>In a risk-neutral world, the expected return on all securities is the risk-free interest rate, $r$. Also, once there's a question that why the money you put in the stock market will not expand with multiplicator $e^{rT}$, cause it's already in it's expecation value! $\Exp^Q\SB{S_T} = S_0\cdot e^{rT}$

## Risk-neutral valuation

Assume the existence of the risk-neutral probability values $q_u$ and $q_d$ corresponding to up and down movement of the price of the stock. Since:$S_0 = e^{-rT} \Exp^Q\SB{S_T}= e^{-rT}\P{q_uS_0u + q_dS_0d}$ and $q_u + q_d = 1$, if we let $q_u = q$, then we have:

$$c = e^{-rT} \Exp^Q\SB{\P{S_T - K}^+} = e^{-rT}\SB{qc_u + \P{1-q}c_d}$$

**e.g.** the same example

> $20 = e^{-0.12\times 3/12}\P{22q_u + 18q_d}$ and $q_u + q_d = 1$ then $q_u = 0.6523$. Thus:
>
> $\bspace c = e^{-0.12\times 3/12}\SB{0.6523 \times 1 + \P{1-0.6523}\times 0} = 0.633$

$Remark$

If *all* claims can be replicated, then the market is ***complete***.

$Proposition.4$

Assume that the general binomial model is free of arbitrage. Then it is also complete.

$Proof$

>For claim $U$ at time $T$, let the asset price be $U_d$ is moving down or $U_u$ is moving up. Construct a portfolio $h(x, y)$ that replicates any claim $U$:
>
>$$xe^{rT} + yuS_0 = U_u,\; xe^{rT}+ydS_0 = U_d \\[0.6em]
\Longrightarrow x = \ffrac{uU_d - dU_u} {\P{u-d}e^{rT}},\; y = \ffrac{U_u - U_d} {\P{u-d}S_0}$$

$Proposition.5$

If the binomial model if free of arbitrage. Then the arbitrage free price of a contingent claim $U$ at time zero is given by

$$\Pi\P{0,U} = e^{-rT}\Exp^Q\SB{U}$$

Here the martingale measure $Q$ is uniquely determined by the relation $S_0 = e^{-rT}\Exp^Q\SB{S_T}$

**Interpretation**

Price of the claim $U$: expected value of the discount payoff under the martingale measure.

For **forward**

$\begin{align}
0 &= \Pi\P{0,\P{S_T-K}} = e^{-rT}\Exp^Q\SB{S_T - K} \\
&\using{\text martingale} e^{-rT} \P{e^{rT}S_0 - K} \\
&= S_0 - Ke^{-rT}
\end{align}$

so that $K = S_0 e^{rT}$ is the price of the forward, verified!

***
For **European options**:

$\begin{align}
c = \Pi\P{0,\P{S_T-K}^+} = e^{-rT}\Exp^Q\SB{\P{S_T-K}^+} = e^{-rT}\SB{qc_u + \P{1-q}c_d}
\end{align}$

so that $c = e^{-rT}\SB{qc_u + \P{1-q}c_d}$ is the price of the european call, verified!

***

$Remark$

>This formula can be applied to calculate the price of american call, shown later.

## Multiperiod Binomial model 要考啦！

**e.g.**

Suppose that the stock prices are given by the tree above, The stock follows the binomial model over each time period $[t, t + 1]$, where $t = 0, 1, 2$. If the interest rates are zero, what is the cost of an option to buy the stock at price $100$ at time $3$?

>We will calculate from the last column to the first column. We first need to calculate $q$, **risk-neutral probability** by the stock price change. Since $e^{-rT}\Exp^Q\SB{S_3} = S_2$ we have
>
>$$e^{-0\times 1} \Exp^Q\SB{q\times160 + \P{1-q}\times 120} = 140 \Longrightarrow q = 0.5$$
>
>And since the interest rates are zero, $q = \ffrac{e^{rT} - d} {u-d}$ keeps constant!
>
> So that for the payoff: $e^{-0\times 1} \Exp^Q\SB{0.5\times\P{160-100}^+ + 0.5\times \P{120 - 100}^+} = 40$, on and on...
>
>
>
>***
>
>Alternative method: Path probabilities
>
>We can also let $V_t\P{k}$ denote the option price at the node $\P{t,k}$, where $k$ denotes the number of up-moves that have occurred and $k = 0 , 1 ,\dots,t$. Thus:
>
>$\bspace \begin{align}
c &= e^{-rT} \Exp^Q\SB{\P{S_T-K}^+}\\
&= e^{-0 \times 3}\SB{\binom{3}{3}q^3\P{160-100}^+ + \binom{3}{2}q^2\P{1-q}\P{120 - 100}^+ + \binom{3}{1}q\P{1-q}^2\P{80 - 100}^+ +\cdots} \\
&\using{q=0.5} 15
\end{align}$

$Remark$

>Here the $u$ and $d$ are calculated by $\ffrac{S_t - S_{t-1}}{ S_{t-1}}+1$. You can check that only when $r=0$ will the $q$ keeps constant while $u$ and $d$ change.

## Replication

Construct a portfolio that exactly replicates the previous claim at time $3$. Write $\P{x_t\P{k},y_t\P{k}}$ for the amount of stock and bond held in the portfolio at node $(t, k)$ over the time interval $[t, t + 1)$.

An intuitive thoughts is that since we're going to replicate the final state, it's better to start from the beginning. Suppose the price jumped up and up and down:

At time $0$, we calculate $y_0\P{0} = \ffrac{25-5} {120-80} = 0.5$. Note that we are given $15$ for selling the option, so we buy $0.5$ units of stock, which costs $50$ and then borrow $35$ cash bonds, $x_0\P{0} = -35$.

At time $1$, suppose that $S_1 = 120$. $y_1\P{1} = \ffrac{40-10} {140-100} = 0.75$, $x_1\P{1} = -35 - 0.25\times 120 = -65$ or $x_1\P{1} = 25 - 0.75\times 120 = -65$

At time $2$, suppose that $S_2 = 140$. $y_2\P{2} = \ffrac{60-20} {160-120} = 1$, $x_1\P{1} = -65 - 0.25\times 140 = -100$ or $x_1\P{1} = 40 - 1\times 100 = -100$

Finally, at time $3$, suppose that $S_3 = 120$. Our replication strategy has one share of stock and $-100$ debt. We will hand over unit of stock for strike price $100$, which is exactly enough to cancel our bond debt.

$Remark$

No extra input or output of cash, ***self-financing***.

$Proposition.6$

Consider a claim $U = \phi\P{S_T}$. Then this claim can be replicated using a self-financing portfolio. If $V_t\P{k}$ denotes the option price at the node $\P{t,k}$, where $k$ is the number of up-moves that have occurred, then $V_t\P{k}$ can be computed recursively by the scheme:

$$\begin{align}
V_T\P{k} &= \phi\P{S_T} \\
V_t\P{k} &= e^{-r \times 1}\SB{q V_{t+1}\P{t+1} + \P{1-q} V_{t+1}\P{k}}
\end{align}$$

and to hedge this, we construct the portfolio as

$$y_t\P{k} = \ffrac{V_{t+1}\P{k+1} - V_{t+1}\P{k}} {\P{u-d}S_t}, x_t\P{k} = V_{t}\P{k}-\ffrac{V_{t+1}\P{k+1} - V_{t+1}\P{k}} {u-d}$$

here $\P{x_t\P{k},y_t\P{k}}$ for the amount of stock and bond held in the portfolio at node $(t, k)$ over the time interval $[t, t + 1)$

**e.g.** Pricing American options under multiperiod binomial models

We don't need to consider the American call options. The American put options with strike price $52$ on a stock whose current price is $50$. We suppose that there are two time steps of $1$ year, and in each time step the stock price either moves up by $20\%$ or down $20\%$. The risk-free interest rate $r$ is $5\%$. Then what's the American put option price at time $0$?

> Strategy: Work back through the tree from the end to the beginning, testing at each node to see whether early exercise is optimal.
>
>At erlier nodes $t_i$, the option price is the greater of
>1. the value given by $e^{-r\Delta_t}\Exp^Q\SB{V\P{t_{i+1}}}$, the continuation value
>2. the payoff from early exercise
>
>First find the $q$: from $S_0 = e^{-r\times 1}\Exp^Q\SB{S_1} \Longrightarrow q  = \ffrac{e^{r\Delta_t}-d} {u-d} = \ffrac{e^{0.05}-0.8} {1.2-0.8} = 0.6282$ and $1-q = 0.3718$

>
>
>Then, as node $B$, the continuation value is $e^{-0.05}\P{0.6282 \times 0 + 0.3718 \times 4} = 1.4147$, while the early exercise payoff is $52-60 = -8$. Thus the option price is $1.4147$
>
>At node $C$, the continuation value is $e^{-0.05}\P{0.6282 \times 4 + 0.3718 \times 20} = 9.4636$, while the early exercise payoff is $52-40 =12$. Thus the option price is $12$
>
>Finally, at node $A$. The continuation value is $e^{-0.05}\P{0.6282 \times 1.4147 + 0.3718 \times 12} = 5.0894$, while the early exercise payoff is $52-50 =2$. Thus the option price is $5.0894$.

## Matching volatility with $u$ and $d$

Suppose that the expected return on a stock in the real world is $\mu$ and its volatility is $\sigma$. The time step is of length $\Delta_t$. Then we matching the expected return on the stock with the binomial tree's parameters, we have

$$\Exp\SB{S_{\Delta_t}} = pS_0 u + \P{1-p}S_0 d = S_0 e^{\mu \Delta_t} \Longrightarrow p = \ffrac{e^{\mu\Delta_t} - d} {u - d}$$

Matching the stock price volatility with the tree's parameters:

$$pu^2 + \P{1-p}d^2 - \SB{pu + \P{1-p}d}^2 = \Var{\ffrac{S_{\Delta_t}} {S_0}}\approx\sigma^2\Delta_t \\
\Downarrow p = \ffrac{e^{\mu\Delta_t} - d} {u - d}\\[0.7em]
e^{\mu\Delta_t} \P{u+d} -ud -e^{2\mu\Delta_t} = \sigma^2 \Delta_t
$$

And an extra condition: ***tree-symmetry condition***: $ud = 1$.

Then we can solve the equation and obtain that:

$$\begin{align}
u &= 1 + \sigma \sqrt{\Delta_t} + \ffrac{\sigma^2} {2} \Delta_t + \cdots = e^{\sigma\sqrt{\Delta_t}}\\
d &= e^{-\sigma\sqrt{\Delta_t}}
\end{align}$$

However in risk-neutral world, $\Exp^Q\SB{S_{\Delta_t}} = qS_0u + \P{1-q} S_0d = S_0 e^{r\Delta_t} \Rightarrow q = \ffrac{e^{r\Delta_t} -d} {u-d}$. And the volatility:

$$qu^2 + \P{1-q}d^2 -\SB{qu + \P{1-q}d}^2 = e^{r\Delta_t} \P{u+d} -ud -e^{2r\Delta_t} = \sigma^2\Delta_t + \P{ \text{higher powers of }\Delta_t}$$

$Remark$

When we move from the real world to the risk-neutral world, the expected return on the stock changes, but its volatility remains the same (at least in the limit as $\Delta_t$ tends to zero).

## Question

$A$ and $B$ are gambling, both of whom with infinite money. However $A$ got a higher chance to lose, so find the strategy that $A$ can at least win $\$1$ from $B$.

>Double the pool if lose, starting from $\$1$. So that if, any chances, that $A$ win (definitely gonna happen if only the event is not a not-possible event), $A$ will win $B$ $1$ dollar.

FinMath/Models and Pricing of Financial Derivatives/Chap_02_Binomial_Tree.ipynb


[[<- back to pattern_classification](https://github.com/rasbt/pattern_classification)]

```python
%load_ext watermark
```

```python
%watermark -a "Sebastian Raschka" -d -v
```

    Sebastian Raschka 07/17/2015

    CPython 3.4.3
    IPython 3.2.0

# Hierarchical Agglomerative Clustering - Complete Linkage Clustering

### - A quick Python tutorial

## Sections

- [Generating some Sample Data](#Generating-some-Sample-Data)
- [Pair-wise Distance Matrix, Rows](#Pair-wise-Distance-Matrix,-Rows)
- [Apply Clustering - Complete Linkage](#Apply-Clustering---Complete-Linkage)
- [Heatmaps](#Heatmaps)
    - [Heatmap of the Original Data](#Heatmap-of-the-Original-Data)
    - [Heatmap after Row-Clustering](#Heatmap-after-Row-Clustering)
    - [Heatmap plus Row-Dendrogram](#[Heatmap-plus-Row-Dendrogram)
    - [Adding a Column Dendrogram](#Adding-a-Column-Dendrogram)
- [Important Warning About Adding Dendrograms](#Important-Warning-About-Adding-Dendrograms)

**This is a more technical-oriented tutorial that was born out of necessity to plot a heatmap including the dendrograms from a complete linkage clustering. Since I couldn't find any good resource online, I thought that it might be worthwhile sharing it. **

**Therefore, the theoretical aspects are a little bit brief, but nonetheless, here is a short introduction of what it is all about.**

Complete linkage is one implementation of *hierarchical agglomerative clustering*. The principle of hierarchical agglomerative clustering is to start with a singleton cluster, and clusters are iteratively merged until one single cluster remains. This results in a "cluster tree," which is also called *dendrogram*. The opposite approach -- starting with one cluster and divide into clusters until only singleton clusters remain -- is called *divisive hierarchical clustering*.

The algorithm can be summarized via the following pseudocode

1: Compute a distance or similarity matrix.
2: Each data point is represented as a singleton cluster.
3: **Repeat**
4:         Merge two closest clusters (e.g., based on distance between most similar or dissimilar members).
5:         Update the distance (or similarity) matrix.
6: **Until** one single cluster remains.

Complete linkage compares the most dissimilar members between clusters in each iteration. The two clusters which have the most similar *dissimilar members* are merged into a new cluster.
\begin{equation}
d(C,D) = \max[dist(C_i, D_j)]
\end{equation}
for all $i$  points in cluster $C$ and $j$ points in cluster $D$.

In contrast, the *single linkage* algorithm compares the two most similar members instead of the most dissimilar ones.
\begin{equation}
d(C,D) = \min[dist(C_i, D_j)]
\end{equation}
for all $i$  points in cluster $C$ and $j$ points in cluster $D$.

For more implementations, e.g., `centroid`, `average` etc., please see the [`scipy.cluster.hierarchy.linkage` documentation](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage).

**(A more detailed article about prototype-based, hierarchical, and density-based clustering is in the planning stage)**

### Generating some Sample Data

[[back to top](#Sections)]

First, we generate some random sample data to work with. Like in a typical application, the rows represent different observations (Samples 1-5), and the columns are the different features (*X, Y, Z*).

```python
import pandas as pd
import numpy as np

np.random.seed(123)

variables = ['X', 'Y', 'Z']
labels = ['ID_0','ID_1','ID_2','ID_3','ID_4']

X = np.random.random_sample([5,3])*10
df = pd.DataFrame(X, columns=variables, index=labels)
df
```

      X
      Y
      Z

      ID_0
      6.964692
      2.861393
      2.268515

      ID_1
      5.513148
      7.194690
      4.231065

      ID_2
      9.807642
      6.848297
      4.809319

      ID_3
      3.921175
      3.431780
      7.290497

      ID_4
      4.385722
      0.596779
      3.980443

### Pair-wise Distance Matrix, Rows

[[back to top](#Sections)]

First, we calculate the pair-wise distances for every row (i.e, between every sample across the different variables). We will use the default euclidean distance measure. The other available distance measures are listed in the [`scipy.spatial.distance.pdist` documentation](http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.distance.pdist.html#scipy.spatial.distance.pdist) with a nice and short explanatory paragraph. In addition, we use the [`squareform`](http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.distance.squareform.html#scipy.spatial.distance.squareform) function to return a symmetrical matrix of the pair-wise distances.

```python
from scipy.spatial.distance import pdist,squareform

row_dist = pd.DataFrame(squareform(pdist(df, metric='euclidean')), columns=labels, index=labels)
row_dist
```

      ID_0
      ID_1
      ID_2
      ID_3
      ID_4

      ID_0
      0.000000
      4.973534
      5.516653
      5.899885
      3.835396

      ID_1
      4.973534
      0.000000
      4.347073
      5.104311
      6.698233

      ID_2
      5.516653
      4.347073
      0.000000
      7.244262
      8.316594

      ID_3
      5.899885
      5.104311
      7.244262
      0.000000
      4.382864

      ID_4
      3.835396
      6.698233
      8.316594
      4.382864
      0.000000

### Apply Clustering - Complete Linkage

[[back to top](#Sections)]

When we apply the complete linkage agglomeration to our clusters, the `linkage` function returns a so-called `linkage matrix`.
This `linkage matrix` consists of several rows where each row consists of 1 merge. The first and second column denote the most dissimilar members in each cluster, and the third row reports the distance between those members. The last column returns the count of members in the clusters.

However, before we call the `linkage` function, let us take a careful look at its [documentation](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage).

> Parameters:
**y** : ndarray
    A condensed or redundant distance matrix. A condensed distance matrix is a flat array containing the upper triangular of the distance matrix. This is the form that pdist returns. Alternatively, a collection of m observation vectors in n dimensions may be passed as an m by n array.

> **method** : str, optional
The linkage algorithm to use. See the Linkage Methods section below for full descriptions.

> **metric** : str, optional
The distance metric to use. See the distance.pdist function for a list of valid distance metrics.

> Returns:
**Z** : ndarray
The hierarchical clustering encoded as a linkage matrix.

Thus, we can either pass a condensed distance matrix (upper triangular) from the `pdist` function, or we can pass the "original" data array and define the `'euclidean'` metric as  function argument in `linkage`. However, we shouldn't pass the `squareform` distance metrics, which would yield incorrect distance values although the overall clustering could be the same.

#### a) Squareform distance matrix (wrong)

```python
from scipy.cluster.hierarchy import linkage

row_clusters = linkage(row_dist, method='complete', metric='euclidean')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      0
      4
      6.521973
      2

      cluster 2
      1
      2
      6.729603
      2

      cluster 3
      3
      5
      8.539247
      3

      cluster 4
      6
      7
      12.444824
      5

#### b) Condensed distance matrix (correct)

```python
row_clusters = linkage(pdist(df, metric='euclidean'), method='complete')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      0
      4
      3.835396
      2

      cluster 2
      1
      2
      4.347073
      2

      cluster 3
      3
      5
      5.899885
      3

      cluster 4
      6
      7
      8.316594
      5

#### c) Input sample matrix (correct)

```python
row_clusters = linkage(df.values, method='complete', metric='euclidean')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      0
      4
      3.835396
      2

      cluster 2
      1
      2
      4.347073
      2

      cluster 3
      3
      5
      5.899885
      3

      cluster 4
      6
      7
      8.316594
      5

```python
import matplotlib.pyplot as plt
%matplotlib inline
from scipy.cluster.hierarchy import dendrogram

row_dendr = dendrogram(row_clusters, labels=labels)
```

## Heatmaps

[[back to top](#Sections)]

This section is about the visualization of our hierarchical clustering. Here, we will plot simple heatmaps for each scenario: The original data, the data after row clustering, and eventually the data after we applied row and column clustering.

### Heatmap of the Original Data

[[back to top](#Sections)]

```python
fig = plt.figure()

ax = fig.add_subplot(111)

cax = ax.matshow(df, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)

ax.set_xticklabels([''] + list(df.columns))
ax.set_yticklabels([''] + list(df.index))

plt.show()
```

### Heatmap after Row-Clustering

[[back to top](#Sections)]

The `dendrogram` function returns a dictionary with various items which are explained in detail in the scipy documenation [here](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.dendrogram.html). The dendrogram leave order can then be accessed via the `'leaves'` key, e.g.,

```python
row_dendr['leaves']
```

    [1, 2, 3, 0, 4]

Thus, in order to sort the DataFrame according to the clustering, we can simply use the `'leaves'` as indices like so:

    df.ix[row_dendr['leaves']]

```python
# reorder rows with respect to the clustering
row_dendr = dendrogram(row_clusters, labels=labels, no_plot=True)
df_rowclust = df.ix[row_dendr['leaves']]

# plot
fig = plt.figure()
ax = fig.add_subplot(111)

cax = ax.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)

ax.set_xticklabels([''] + list(df_rowclust.columns))
ax.set_yticklabels([''] + list(df_rowclust.index))

plt.show()
```

### Heatmap plus Row-Dendrogram

[[back to top](#Sections)]

Now, we can rotate the dendrogram via the by setting the `orientation` parameter to `'right'`, but note that
we now have to sort the clustered data in reverse order to match the row labels in the heatmap via

    df.ix[row_dendr['leaves'][::-1]]

```python
from scipy.cluster import hierarchy
# makes dendrogram black (1)
hierarchy.set_link_color_palette(['black'])

# plot row dendrogram
fig = plt.figure(figsize=(8,8))
axd = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='right',
                   color_threshold=np.inf, ) # makes dendrogram black (2))

# reorder data with respect to clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

axd.set_xticks([])
axd.set_yticks([])

# remove axes spines from dendrogram
for i in axd.spines.values():
        i.set_visible(False)

# reorder rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

# plot heatmap
axm = fig.add_axes([0.26,0.1,0.6,0.6]) # x-pos, y-pos, width, height
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([''] + list(df_rowclust.index))

plt.show()
```

### Adding a Column Dendrogram

[[back to top](#Sections)]

Also, we can add a an additional dendrogram for the column clustering at the top. Here, we first sort the indexes after clustering the rows

    df_rowclust = df.ix[row_dendr['leaves'][::-1]]

And then we can use the indices from the column clustering to reorder the columns:

    df_rowclust.columns = [df_rowclust.columns[col_dendr['leaves']]]

```python
# Compute pairwise distances for columns
col_dists = pdist(df.T, metric='euclidean')
col_clusters = linkage(col_dists, method='complete')

# plot column dendrogram
fig = plt.figure(figsize=(8,8))

axd2 = fig.add_axes([0.38,0.74,0.36,0.10])
col_dendr = dendrogram(col_clusters, orientation='top',
                       color_threshold=np.inf) # makes dendrogram black)
axd2.set_xticks([])
axd2.set_yticks([])

# plot row dendrogram
axd1 = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='right',
                       count_sort='ascending',
                       color_threshold=np.inf) # makes dendrogram black
axd1.set_xticks([])
axd1.set_yticks([])

# remove axes spines from dendrogram
for i,j in zip(axd1.spines.values(), axd2.spines.values()):
        i.set_visible(False)
        j.set_visible(False)

# reorder columns and rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]
df_rowclust.columns = [df_rowclust.columns[col_dendr['leaves']]]

# plot heatmap
axm = fig.add_axes([0.26,0.1,0.6,0.6])
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([''] + list(df_rowclust.index))

plt.show()
```

# Important Warning About Adding Dendrograms

[[back to top](#Sections)]

It is very important that you always check that the labels on the heatmap and the dendrogram match when you create the first plot! For example, we can provide the `labels` to the `labels` parameter of the dendrogram and **DON'T set `axd.set_yticks([])`** to show the labels on the dendrogram to compare them to the heatmap labels.

```python
from scipy.cluster import hierarchy
# makes dendrogram black (1)
hierarchy.set_link_color_palette(['black'])

# plot row dendrogram
fig = plt.figure(figsize=(8,8))
axd = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='right',
                   labels=labels,
                   color_threshold=np.inf, ) # makes dendrogram black (2))
axd.set_xticks([])

# uncomment to hide dendrogram labels
#axd.set_yticks([])

# remove axes spines from dendrogram
for i in axd.spines.values():
        i.set_visible(False)

# reorder columns and rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

# plot heatmap
axm = fig.add_axes([0.26,0.1,0.6,0.6]) # x-pos, y-pos, width, height
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([''] + list(df_rowclust.index))

plt.show()
```


```python
import numpy as np
import pydae.build as db
import sympy as sym
```

```python
class cig:

    def __init__(self, data):

        self.f_list = []
        self.x_list = []
        self.g_list = []
        self.y_ini_list = []
        self.y_run_list = []
        self_u_dict = {}
        self_h_dict = {}
        self.params_dict = {}

        self.data = data
        self.cigs = data['cigs']

    def build(self):

        cigs = self.cigs
        for cig in cigs:
            self.vsc_lcl(cig)
            self.f_list += self.f_vsc_filter
            self.x_list += self.x_vsc_filter
            self.g_list += self.g_vsc_filter
            self.y_ini_list += self.y_vsc_filter
            self.y_run_list += self.y_vsc_filter

            self.transformations(cig)
            self.g_list += self.g_aux
            self.y_ini_list += self.y_aux
            self.y_run_list += self.y_aux

    def vsg2dae(data,grid_dae):
        '''

        Parameters
        ----------
        data : TYPE
            DESCRIPTION.
        grid_dae : TYPE
            DESCRIPTION.

        Returns
        -------
        dict
            DESCRIPTION.

        '''

        sin = sym.sin
        cos = sym.cos
        sqrt = sym.sqrt

        vsgs = data['vsgs']
        N_vsg = len(vsgs)

        # secondary control
        p_sec = {}
        q_sec = {}
        xi_f_sec,xi_v_sec = sym.symbols('xi_f_sec,xi_v_sec', real=True)
        K_f_sec,K_v_sec = sym.symbols('K_f_sec,K_v_sec', real=True)

        omega_coi_h = 0
        H_total = 0
        N_voltage = 0
        v_prom = 0
        for vsg in vsgs:
            name = vsg['name']

            omega_v_i  = sym.Symbol(f'omega_v_{name}', real=True)
            H = sym.Symbol(f'H_{name}', real=True)

            omega_coi_h += omega_v_i*H
            H_total += H

            v_s_filt_i = sym.symbols(f'v_s_filt_{name}', real=True)
            N_voltage += 1
            v_prom += v_s_filt_i/N_vsg

        omega_coi = omega_coi_h/H_total
        dxi_f_sec = 1 - omega_coi
        dxi_v_sec = 1 - v_prom

        for vsg in vsgs:
            name = vsg['name']
            p_sec.update({f'{name}':K_f_sec*xi_f_sec/N_vsg})
            q_sec.update({f'{name}':K_v_sec*xi_v_sec/N_vsg})

        f_ctrl_5 = [dxi_f_sec,dxi_v_sec]
        x_ctrl_5 = [ xi_f_sec, xi_v_sec]
        g_ctrl_5 = []
        y_ctrl_5 = []
        x_0_ctrl_5 = []
        y_0_ctrl_5 = []
        params_ctrl_5 = {'K_f_sec':0.001,'K_v_sec':0.01}
        u_ctrl_5 = {}
        h_ctrl_5 = {}

        f_vsg = []
        x_vsg = []
        g_vsg = []
        y_vsg = []
        x_0_vsg = []
        y_0_vsg = []
        params_vsg = {}
        u_vsg = {}
        h_vsg = {}

        for vsg in vsgs:
            name = vsg['name']
            bus = vsg['bus']

            U_b = vsg['U_b']
            S_b = vsg['S_b_kVA']*1000
            I_b = S_b/(np.sqrt(3)*U_b)

            U_bdq = U_b*(np.sqrt(2))
            V_bdq = U_bdq/np.sqrt(3)
            I_bdq = I_b*np.sqrt(2)

    def transformations(self):
            ## Transformations #########################################################################
            ## feedbacks:
            feedbacks = ['i_tD','i_tQ'] + ['v_mD','v_mQ'] + ['i_sD','i_sQ'] + ['v_sD','v_sQ'] + ['phi']
            for item in feedbacks:
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())

            v_md,v_mq = sym.symbols(f'v_md_{name},v_mq_{name}', real=True)
            v_sd,v_sq = sym.symbols(f'v_sd_{name},v_sq_{name}', real=True)
            i_sd,i_sq = sym.symbols(f'i_sd_{name},i_sq_{name}', real=True)
            i_td,i_tq = sym.symbols(f'i_td_{name},i_tq_{name}', real=True)
            phi,dum= sym.symbols(f'phi_{name},dum_{name}', real=True)

            eq_v_md = -v_md + v_mD*cos(phi) + v_mQ*sin(phi) # original park
            eq_v_mq = -v_mq - v_mD*sin(phi) + v_mQ*cos(phi) # original park

            eq_v_sd = -v_sd + v_sD*cos(phi) + v_sQ*sin(phi) # original park
            eq_v_sq = -v_sq - v_sD*sin(phi) + v_sQ*cos(phi) # original park

            eq_i_sd = -i_sd + i_sD*cos(phi) + i_sQ*sin(phi) # original park
            eq_i_sq = -i_sq - i_sD*sin(phi) + i_sQ*cos(phi) # original park

            # jmm: medimos i_t?
            eq_i_td = -i_td + i_tD*cos(phi) + i_tQ*sin(phi) # original park
            eq_i_tq = -i_tq - i_tD*sin(phi) + i_tQ*cos(phi) # original park

            g_aux = [eq_v_md,eq_v_mq,eq_v_sd,eq_v_sq,eq_i_td,eq_i_tq,eq_i_sd,eq_i_sq]
            y_aux = [   v_md,   v_mq,   v_sd,   v_sq,   i_td,   i_tq,   i_sd,   i_sq]
            y_0_aux = [  0.0,  V_bdq,    0.0,  V_bdq,       0,      0]

            #v_sd = v_md
            #v_sq = v_mq

           # S_b_kVA,U_b = sym.symbols(f'S_b_kVA_{name},U_b_{name}', real=True) # params

            ## per unit

            i_sd_pu = i_sd/I_bdq; # input in SI that is coverted to pu
            i_sq_pu = i_sq/I_bdq; # input in SI that is coverted to pu
            v_sd_pu = v_sd/V_bdq; # input in SI that is coverted to pu
            v_sq_pu = v_sq/V_bdq; # input in SI that is coverted to pu
            i_td_pu = i_td/I_bdq; # input in SI that is coverted to pu
            i_tq_pu = i_tq/I_bdq; # input in SI that is coverted to pu

            ## PLL #########################################################################

    def ctrl_4(self):
            # CTRL4 #########################################################################
            ## parameters:
            params_ctrl_4 = {}
            for item in ['T_vpoi','K_vpoi','T_f','K_f']:
                params_ctrl_4.update({f'{item}_{name}':vsg[item]})
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
            ## inputs:
            u_ctrl_4 = {}
            for item in ['v_s_ref','omega_ref','p_r','q_r']:
                u_ctrl_4.update({f'{item}_{name}':vsg[item]})
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
            ## dynamic states:
            x_list_ctrl_4 = ['omega_v_filt','v_s_filt']
            for item in x_list_ctrl_4:
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
            ## algebraic states
            y_list_ctrl_4 = ['p_m_ref','q_s_ref']
            for item in y_list_ctrl_4:
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
            ## feedbacks:
            feedbacks = ['omega_v']
            for item in feedbacks:
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())

            domega_v_filt = 1/T_f*(omega_v - omega_v_filt)
            dv_s_filt = 1/T_vpoi*((v_sd_pu**2+v_sq_pu**2)**0.5 - v_s_filt)

            eq_p_m_ref = -p_m_ref + p_r + K_f*(omega_ref - omega_v_filt) + p_sec[name]   # PFR and secondary input
            eq_q_s_ref = -q_s_ref + q_r + K_vpoi*(v_s_ref - v_s_filt)    + q_sec[name]

            # from derivatives to the integrator
            f_ctrl_4 = [domega_v_filt,dv_s_filt];
            x_ctrl_4 = [ omega_v_filt, v_s_filt];
            g_ctrl_4 = [ eq_p_m_ref, eq_q_s_ref] #eq_i_sd_ref, eq_i_sq_ref, ]
            y_ctrl_4 = [    p_m_ref,    q_s_ref] #   i_sd_ref,    i_sq_ref,    omega_v,    DV_sat];

            # CTRL3 #########################################################################

    def ctrl_3(self):
            if vsg['ctrl3'] == 'uvsg_i':

                ## parameters:
                params_ctrl_3 = {}
                for item in ['K_p','T_p','K_q','T_q','R_v','X_v','S_b_kVA','U_b','K_phi','H','D']:
                    params_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## inputs:
                u_ctrl_3 = {}
                for item in ['p_m_ref','q_s_ref']: # []
                    u_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## dynamic states:
                x_list_ctrl_3 = ['phi','omega_v','xi_q','omega_rads']
                for item in x_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## algebraic states
                y_list_ctrl_3 = ['DV_sat','p_s_pu','q_s_pu']
                for item in y_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## feedbacks:
                v_dc,dum = sym.symbols(f'v_dc_{name},dum_{name}', real=True)

                # equations:
                eq_omega_v = -omega_v + K_p*(epsilon_p + xi_p/T_p) + 1.0;

                dphi =  Omega_b*(omega_v-1.0) - K_phi*phi;
                dxi_p = epsilon_p;
                dxi_q = epsilon_q;
                domega_rads = 1.0/1.0*(Omega_b*omega_v - omega_rads);

            if vsg['ctrl3'] == 'sm2':

                ## parameters:
                params_ctrl_3 = {}
                for item in ['K_p','T_p','K_q','T_q','R_v','X_v','S_b_kVA','U_b','K_phi','H','D']:
                    params_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## inputs:
                u_ctrl_3 = {}
                for item in ['p_m_ref','q_s_ref']: # []
                    u_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## dynamic states:
                x_list_ctrl_3 = ['phi','omega_v','xi_q','omega_rads']
                for item in x_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## algebraic states
                y_list_ctrl_3 = ['DV_sat','p_s_pu','q_s_pu']
                for item in y_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## feedbacks:
                v_dc,dum = sym.symbols(f'v_dc_{name},dum_{name}', real=True)

                # equations:

                fault_flag = 0;
                e_0 = 1.0;
                eq_p_s_pu = -p_s_pu + i_sd_pu*v_sd_pu + i_sq_pu*v_sq_pu; # pu
                eq_q_s_pu = -q_s_pu + i_sd_pu*v_sq_pu - i_sq_pu*v_sd_pu; # pu

                # from the integrator to the states
                epsilon_p = (p_m_ref - p_s_pu)*(1.0-fault_flag);
                epsilon_q =  q_s_ref - q_s_pu;

                dphi =  Omega_b*(omega_v-omega_coi) - K_phi*phi;
                domega_v = 1/(2*H)*(p_m_ref - p_s_pu - D*(omega_v - 1.0))
                dxi_q = epsilon_q;
                domega_rads = 1.0/1.0*(Omega_b*omega_v - omega_rads);

                DV =   K_q*(epsilon_q + xi_q/T_q);

                eq_DV_sat = DV_sat - DV;
                #if DV_sat > 0.1
                #    DV_sat = 0.1;
                #    dxi_q = 0.0;
                #end
                #if DV_sat < -0.1
                #    DV_sat = -0.1;
                #    dxi_q = 0.0;
                #end

                e = e_0 + DV_sat;

                if (not 'ctrl1' in vsg) and (not 'ctrl2' in vsg):                      # CTRL3 over CTRL0
                    v_t_d_pu = 0.0
                    v_t_q_pu = e

                    v_t_d = (v_t_d_pu - R_v*i_sd_pu + X_v*i_sq_pu)*V_bdq
                    v_t_q = (v_t_q_pu - R_v*i_sq_pu - X_v*i_sd_pu)*V_bdq

                    eta_d_ref = v_t_d/v_dc*2
                    eta_q_ref = v_t_q/v_dc*2

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_v,dxi_q,domega_rads];
                    x_ctrl_3 = [ phi, omega_v, xi_q, omega_rads];
                    g_ctrl_3 = [ eq_DV_sat, eq_p_s_pu, eq_q_s_pu] #eq_i_sd_ref, eq_i_sq_ref, ]
                    y_ctrl_3 = [    DV_sat,    p_s_pu,    q_s_pu] #   i_sd_ref,    i_sq_ref,    omega_v,    DV_sat];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

                if ('ctrl1' in vsg) and (not 'ctrl2' in vsg):                          # CTRL3 over CTRL1
                    i_sd_pu = -(X_v*e + R_v*v_sd_pu + X_v*v_sq_pu)/(R_v**2 + X_v**2)
                    i_sq_pu = -(R_v*e + R_v*v_sq_pu - X_v*v_sd_pu)/(R_v**2 + X_v**2);
                    eq_i_sd_ref = -i_sd_ref + i_sd_pu*I_bdq
                    eq_i_sq_ref = -i_sq_ref + i_sq_pu*I_bdq

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_v,dxi_q,domega_rads];
                    x_ctrl_3 = [ phi, omega_v, xi_q, omega_rads];
                    g_ctrl_3 = [ eq_DV_sat, eq_p_s_pu, eq_q_s_pu, eq_i_sd_ref, eq_i_sq_ref]
                    y_ctrl_3 = [    DV_sat,    p_s_pu,    q_s_pu,    i_sd_ref,    i_sq_ref];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

                if ('ctrl1' in vsg) and ('ctrl2' in vsg):                          # CTRL3 over CTRL2
                    v_sd_ref_pu =    - R_v*i_sd_pu + X_v*i_sq_pu;
                    v_sq_ref_pu = -e - R_v*i_sq_pu - X_v*i_sd_pu;
                    eq_v_sd_ref = -v_sd_ref + v_sd_ref_pu*V_bdq
                    eq_v_sq_ref = -v_sq_ref + v_sq_ref_pu*V_bdq

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_v,dxi_q,domega_rads];
                    x_ctrl_3 = [ phi, omega_v, xi_q, omega_rads];
                    g_ctrl_3 = [ eq_DV_sat, eq_p_s_pu, eq_q_s_pu, eq_v_sd_ref, eq_v_sq_ref]
                    y_ctrl_3 = [    DV_sat,    p_s_pu,    q_s_pu,    v_sd_ref,    v_sq_ref];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

            if vsg['ctrl3'] == 'droop_pq':

                ## parameters:
                params_ctrl_3 = {}
                for item in ['Omega_b','K_p','T_p','K_q','T_q','R_v','X_v','S_b_kVA','U_b','K_phi','H','D','K_omega','K_v','T_omega']:
                    params_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## inputs:
                u_ctrl_3 = {}
                for item in ['p_m_ref','q_s_ref','e_ref']: # []
                    u_ctrl_3.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## dynamic states:
                x_list_ctrl_3 = ['phi','omega_v','xi_q','omega_rads']
                for item in x_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## algebraic states
                y_list_ctrl_3 = ['p_s_pu','q_s_pu']
                for item in y_list_ctrl_3:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## feedbacks:
                v_dc,dum = sym.symbols(f'v_dc_{name},dum_{name}', real=True)

                # equations:

                fault_flag = 0;
                e_0 = 1.0;
                eq_p_s_pu = -p_s_pu + i_sd_pu*v_sd_pu + i_sq_pu*v_sq_pu; # pu
                eq_q_s_pu = -q_s_pu + i_sd_pu*v_sq_pu - i_sq_pu*v_sd_pu; # pu

                # from the integrator to the states
                epsilon_p = (p_m_ref - p_s_pu)*(1.0-fault_flag);
                epsilon_q =  q_s_ref - q_s_pu;

                omega_v_ref = omega_ref + K_omega*epsilon_p
                e =   e_ref + K_v*epsilon_q

                dphi =  Omega_b*(omega_v-omega_coi) - K_phi*phi;
                domega_v = 1/T_omega*(omega_v_ref - omega_v)
                domega_rads = 1.0/1.0*(Omega_b*omega_v - omega_rads);

                if (not 'ctrl1' in vsg) and (not 'ctrl2' in vsg):                      # CTRL3 over CTRL0
                    v_t_d_pu = 0.0
                    v_t_q_pu = e

                    v_t_d = (v_t_d_pu - R_v*i_sd_pu + X_v*i_sq_pu)*V_bdq
                    v_t_q = (v_t_q_pu - R_v*i_sq_pu - X_v*i_sd_pu)*V_bdq

                    eta_d_ref = v_t_d/v_dc*2
                    eta_q_ref = v_t_q/v_dc*2

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_v,domega_rads];
                    x_ctrl_3 = [ phi, omega_v, omega_rads];
                    g_ctrl_3 = [ eq_p_s_pu, eq_q_s_pu] #eq_i_sd_ref, eq_i_sq_ref, ]
                    y_ctrl_3 = [    p_s_pu,    q_s_pu] #   i_sd_ref,    i_sq_ref,    omega_v,    DV_sat];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

                if ('ctrl1' in vsg) and (not 'ctrl2' in vsg):                          # CTRL3 over CTRL1
                    i_sd_pu = -(X_v*e + R_v*v_sd_pu + X_v*v_sq_pu)/(R_v**2 + X_v**2)
                    i_sq_pu = -(R_v*e + R_v*v_sq_pu - X_v*v_sd_pu)/(R_v**2 + X_v**2);
                    eq_i_sd_ref = -i_sd_ref + i_sd_pu*I_bdq
                    eq_i_sq_ref = -i_sq_ref + i_sq_pu*I_bdq

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_rads];
                    x_ctrl_3 = [ phi, omega_rads];
                    g_ctrl_3 = [ eq_DV_sat, eq_p_s_pu, eq_q_s_pu, eq_i_sd_ref, eq_i_sq_ref]
                    y_ctrl_3 = [    DV_sat,    p_s_pu,    q_s_pu,    i_sd_ref,    i_sq_ref];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

                if ('ctrl1' in vsg) and ('ctrl2' in vsg):                          # CTRL3 over CTRL2
                    v_sd_ref_pu =    - R_v*i_sd_pu + X_v*i_sq_pu;
                    v_sq_ref_pu = -e - R_v*i_sq_pu - X_v*i_sd_pu;
                    eq_v_sd_ref = -v_sd_ref + v_sd_ref_pu*V_bdq
                    eq_v_sq_ref = -v_sq_ref + v_sq_ref_pu*V_bdq

                    # from derivatives to the integrator
                    f_ctrl_3 = [dphi,domega_v,dxi_q,domega_rads];
                    x_ctrl_3 = [ phi, omega_v, xi_q, omega_rads];
                    g_ctrl_3 = [ eq_DV_sat, eq_p_s_pu, eq_q_s_pu, eq_v_sd_ref, eq_v_sq_ref]
                    y_ctrl_3 = [    DV_sat,    p_s_pu,    q_s_pu,    v_sd_ref,    v_sq_ref];
                    x_0_ctrl_3 = [ 0.0, 0.0, 0.0, 2*np.pi*50]
                    y_0_ctrl_3 = [ ] #     0.0,       V_bdq,          1]

            if vsg['ctrl3'] == 'genape':
                dphi =  Omega_b*(omega_v-omega_coi) - K_phi*phi;
                domega_v = RoCoF/F_b;

    def ctrl_0(self):
            # CTRL0 #########################################################################

            ## inputs:
            params_ctrl_0 = {}
            u_ctrl_0 = {}
            for item in []: # ['eta_d_ref','eta_q_ref','phi']: #+['eta_D','eta_Q']:
                u_ctrl_0.update({f'{item}_{name}':vsg[item]})
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
            ## algebraic states
            y_list_ctrl_0 = ['eta_d','eta_q'] + ['eta_D','eta_Q']
            for item in y_list_ctrl_0:
                exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())

            eq_eta_d = eta_d - eta_d_ref
            eq_eta_q = eta_q - eta_q_ref

            eq_eta_D = -eta_D + eta_d*cos(phi) - eta_q*sin(phi) # comment for test 1
            eq_eta_Q = -eta_Q + eta_d*sin(phi) + eta_q*cos(phi) # comment for test 1

            # from derivatives to the integrator
            f_ctrl_0 = [];
            x_ctrl_0 = [];
            g_ctrl_0 = [eq_eta_d,eq_eta_q,eq_eta_D,eq_eta_Q]
            y_ctrl_0 = [   eta_d,   eta_q,   eta_D,   eta_Q];
            x_0_ctrl_0 = [ ]
            y_0_ctrl_0 = [0.0,  0.8, 0.0,  0.8]

            ## VSC and Filters #########################################################################

            if vsg['filter'] == 'L':
                ## parameters:
                params_vsc_filter = {}
                for item in ['L_t','R_t','omega']:
                    params_vsc_filter.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## inputs:
                u_vsc_filter= {}
                for item in ['v_dc']: #+['eta_D','eta_Q']:
                    u_vsc_filter.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## dynamic states:
                x_list_vsc_filter = []
                for item in x_list_vsc_filter:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## algebraic states
                y_list_vsc_filter = ['i_tD','i_tQ'] + ['v_mD','v_mQ'] + ['i_sD','i_sQ'] + ['v_sD','v_sQ']
                for item in y_list_vsc_filter:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)")
                ## feedbacks:
                v_poiD,v_poiQ = sym.symbols(f'v_{bus}_D,v_{bus}_Q', real=True)
                i_poiD,i_poiQ = sym.symbols(f'i_{bus}_D,i_{bus}_Q', real=True)

                #eta_D = eta_D_ref # - Gv_in*(i_tD - i_sD)
                #eta_Q = eta_Q_ref #- Gv_in*(i_tQ - i_sQ)

                # LCL filter
                di_tD = 1/L_t*(eta_D/2*v_dc - R_t*i_tD + omega*L_t*i_tQ - v_mD)
                di_tQ = 1/L_t*(eta_Q/2*v_dc - R_t*i_tQ - omega*L_t*i_tD - v_mQ)
                dv_mD = 1/C_m*(i_tD + C_m*omega*v_mQ - G_d*v_mD - i_sD)
                dv_mQ = 1/C_m*(i_tQ - C_m*omega*v_mD - G_d*v_mQ - i_sQ)
                di_sD = 1/L_s*(v_mD - R_s*i_sD + omega*L_s*i_sQ - v_sD)
                di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - omega*L_s*i_sD - v_sQ)

                # Grid interaction
                eq_i_poiD =  i_sD - i_poiD
                eq_i_poiQ =  i_sQ - i_poiQ

                eq_v_sD =  v_sD - v_poiD
                eq_v_sQ =  v_sQ - v_poiQ

                grid_dae['params'].pop(f'i_{bus}_D')
                grid_dae['params'].pop(f'i_{bus}_Q')

                # DAE
                f_vsc_filter = []
                x_vsc_filter = []
                g_vsc_filter = [di_tD, di_tQ, eq_i_poiD, eq_i_poiQ, eq_v_sD, eq_v_sQ]
                y_vsc_filter = [ i_tD,  i_tQ,    i_poiD,    i_poiQ,    v_sD,    v_sQ]
                x_0_vsc_filter = [ ]
                y_0_vsc_filter = [ 0.0,  0.0,       0.0,         0,       0,   V_bdq]

            if vsg['filter'] == 'LCL':
                ## parameters:
                params_vsc_filter = {}
                for item in ['L_t','R_t','C_m','L_s','R_s','omega','G_d']:
                    params_vsc_filter.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## inputs:
                u_vsc_filter = {}
                for item in ['v_dc']: #+['eta_D','eta_Q']:
                    u_vsc_filter.update({f'{item}_{name}':vsg[item]})
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## dynamic states:
                x_list_vsc_filter = []
                for item in x_list_vsc_filter:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)")
                ## algebraic states
                y_list_vsc_filter = ['i_tD','i_tQ'] + ['v_mD','v_mQ'] + ['i_sD','i_sQ'] + ['v_sD','v_sQ']
                for item in y_list_vsc_filter:
                    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
                ## feedbacks:
                v_poiD,v_poiQ = sym.symbols(f'v_{bus}_D,v_{bus}_Q', real=True)
                i_poiD,i_poiQ = sym.symbols(f'i_{bus}_D,i_{bus}_Q', real=True)

                #eta_D = eta_D_ref # - Gv_in*(i_tD - i_sD)
                #eta_Q = eta_Q_ref #- Gv_in*(i_tQ - i_sQ)

                # LCL filter
                di_tD = 1/L_t*(eta_D/2*v_dc - R_t*i_tD + omega*L_t*i_tQ - v_mD)
                di_tQ = 1/L_t*(eta_Q/2*v_dc - R_t*i_tQ - omega*L_t*i_tD - v_mQ)
                dv_mD = 1/C_m*(i_tD + C_m*omega*v_mQ - G_d*v_mD - i_sD)
                dv_mQ = 1/C_m*(i_tQ - C_m*omega*v_mD - G_d*v_mQ - i_sQ)
                di_sD = 1/L_s*(v_mD - R_s*i_sD + omega*L_s*i_sQ - v_sD)
                di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - omega*L_s*i_sD - v_sQ)

                # Grid interaction
                eq_i_poiD =  i_sD - i_poiD
                eq_i_poiQ =  i_sQ - i_poiQ

                eq_v_sD =  v_sD - v_poiD
                eq_v_sQ =  v_sQ - v_poiQ

                grid_dae['params'].pop(f'i_{bus}_D')
                grid_dae['params'].pop(f'i_{bus}_Q')

                # DAE
                f_vsc_filter = []
                x_vsc_filter = []
                g_vsc_filter = [di_tD, di_tQ, dv_mD, dv_mQ, di_sD, di_sQ, eq_i_poiD, eq_i_poiQ, eq_v_sD, eq_v_sQ]
                y_vsc_filter = [ i_tD,  i_tQ,  v_mD,  v_mQ,  i_sD,  i_sQ,    i_poiD,    i_poiQ,    v_sD,    v_sQ]
                x_0_vsc_filter = [ ]
                y_0_vsc_filter = [ 0.0,  0.0,   0.0, V_bdq,     0,     0,         0,         0,       0,   V_bdq]

            ## Model integration
            f_vsg += f_vsc_filter + f_ctrl_0 + f_ctrl_3 + f_ctrl_4
            x_vsg += x_vsc_filter + x_ctrl_0 + x_ctrl_3 + x_ctrl_4
            g_vsg += g_vsc_filter + g_ctrl_0 + g_aux + g_ctrl_3 + g_ctrl_4
            y_vsg += y_vsc_filter + y_ctrl_0 + y_aux + y_ctrl_3 + y_ctrl_4
            params_vsg.update(params_vsc_filter)
            params_vsg.update(params_ctrl_0)
            params_vsg.update(params_ctrl_3)
            params_vsg.update(params_ctrl_4)
            u_vsg.update(u_vsc_filter)
            u_vsg.update(u_ctrl_0)
            u_vsg.update(u_ctrl_3)
            u_vsg.update(u_ctrl_4)

            h_vsg.update({f'i_sD_{name}':i_sD,f'i_sQ_{name}':i_sQ})

           # x_0_vsg += x_0_vsc_lc
           # y_0_vsg += y_0_vsc_lc

        f_vsg += f_ctrl_5
        x_vsg += x_ctrl_5
        params_vsg.update(params_ctrl_5)
        u_vsg.update(u_ctrl_5)

        return {'f_list':f_vsg,'g_list':g_vsg,
                'x_list':x_vsg,'y_list':y_vsg,
                'u_run_dict':u_vsg,'params_dict':params_vsg,'h_dict':h_vsg,
                'omega_coi':omega_coi}

```

```python
cig_obj = cig(data)
cig_obj.build()
cig_obj.x_list
cig_obj.y_ini_list
```

    [i_1_D,
     i_1_Q,
     v_sD_g01,
     v_sQ_g01,
     v_md_g01,
     v_mq_g01,
     v_sd_g01,
     v_sq_g01,
     i_td_g01,
     i_tq_g01,
     i_sd_g01,
     i_sq_g01,
     eta_D_g01,
     eta_Q_g01]

```python
Omega_b = 2*np.pi*50
L_t = 1.25e-3; #Converter side Filter Inductance H
R_t = L_t*Omega_b/10; #Converter side Filter resistance Ohms. Quality Factor of 10
L_s = 1.25e-3; #Grid Side Filter Inductance H
R_s = L_s*Omega_b/10; #Grid side Filter resistance Ohms. Quality Factor of 10
R_d = 20; #Dmaping resistor
C_m = 4e-6; #Filter capacitance
tau_ctrl_1 = 5e-3; #Time constant of CTRL 1
Ts_Power=50e-6; #Sample time of power plant
Ts_Control=50e-6; #Sample time of controller
K_pi = (L_t+L_s)/tau_ctrl_1; #Proportional gain of CTRL 1
K_ii = (R_t+R_s)/tau_ctrl_1; #Integral gain of CTRL 1
H = 5.0; # desired virtual inertia
K_p = 0.01; # active power proportinal gain
#H = T_p/K_p/2

T_p = K_p*2*H;  # active power integral time constant

K_q = 0.1; # reactive power proportinal gain
T_q = 0.1; # reactive power integral time constant

X_v = 0.05; # virtual reactance
R_v = 0.01;  # virtual resistance

K_pv = 0.0004
K_iv = 11.313708

g01 = {"name":"g01","bus": "1","U_b":400.0,"S_b_kVA":30,"v_dc":750,'type':'sm2_i',
           'R_t':R_t,'L_t':L_t,'R_s':R_s,'L_s':L_s,'C_m':C_m,'K_phi':0.00,'omega':Omega_b, 'G_d':1.0, #VSC
           'T_eta':0.01,   # CTRL0
           'K_pi':K_pi,'K_ii':K_ii,'Gv_in':0.1,   # CTRL1
           'K_pv':K_pv,'K_iv':K_iv,   # CTRL2
           'K_p':K_p,'T_p':T_p,'K_q':K_q,'T_q':T_q,'R_v':R_v,'X_v':X_v,'p_m_ref':0.5,'q_s_ref':0.2,   # CTRL3
           'H':5.0,'D':5.0,  # CTRL3
           'K_f':20.0, 'K_vr':20.0}   # CTRL4

data = {
"system":{"f_hz":50,"model_type":"ae"},
"buses": [{"bus":"1"},{"bus":"2"}],
"lines":     [  {"bus_j":"1", "bus_k":"2", "R":0.01, "X":0.1, "B":1e-3}
             ],
"loads" : [
    #    {"bus": "R06", "kVA": 55.0, "pf": 0.95, "T_i":0.01,"I_max":1200}
        ],
"grid_formers" : [
                  {"bus": "1","V_phph":400.0, "deg":0.0},
                  {"bus": "2","V_phph":400.0, "deg":0.0},
                 ],
"cigs" : [g01]}

```

```python
cos = sym.cos
sin = sym.sin

vsg = g01

name = vsg['name']
bus = vsg['bus']

U_b = vsg['U_b']
S_b = vsg['S_b_kVA']*1000
I_b = S_b/(np.sqrt(3)*U_b)

U_bdq = U_b*(np.sqrt(2))
V_bdq = U_bdq/np.sqrt(3)
I_bdq = I_b*np.sqrt(2)

eta_D,eta_Q =sym.symbols(f'eta_D_{name},eta_Q_{name}',real=True)

params_vsg = {}
u_vsg = {}

## Transformations #########################################################################
## feedbacks:
feedbacks = ['i_tD','i_tQ'] + ['v_mD','v_mQ'] + ['i_sD','i_sQ'] + ['v_sD','v_sQ'] + ['phi']
for item in feedbacks:
    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())

v_md,v_mq = sym.symbols(f'v_md_{name},v_mq_{name}', real=True)
v_sd,v_sq = sym.symbols(f'v_sd_{name},v_sq_{name}', real=True)
i_sd,i_sq = sym.symbols(f'i_sd_{name},i_sq_{name}', real=True)
i_td,i_tq = sym.symbols(f'i_td_{name},i_tq_{name}', real=True)
eta_d,eta_q = sym.symbols(f'eta_d_{name},eta_q_{name}', real=True)
phi,dum= sym.symbols(f'phi_{name},dum_{name}', real=True)

eq_v_md = -v_md + v_mD*cos(phi) + v_mQ*sin(phi) # original park
eq_v_mq = -v_mq - v_mD*sin(phi) + v_mQ*cos(phi) # original park

eq_v_sd = -v_sd + v_sD*cos(phi) + v_sQ*sin(phi) # original park
eq_v_sq = -v_sq - v_sD*sin(phi) + v_sQ*cos(phi) # original park

eq_i_sd = -i_sd + i_sD*cos(phi) + i_sQ*sin(phi) # original park
eq_i_sq = -i_sq - i_sD*sin(phi) + i_sQ*cos(phi) # original park

# jmm: medimos i_t?
eq_i_td = -i_td + i_tD*cos(phi) + i_tQ*sin(phi) # original park
eq_i_tq = -i_tq - i_tD*sin(phi) + i_tQ*cos(phi) # original park

eq_eta_D = -eta_D + eta_d*cos(phi) - eta_q*sin(phi) # original park
eq_eta_Q = -eta_Q + eta_d*sin(phi) + eta_q*cos(phi) # original park

g_aux = [eq_v_md,eq_v_mq,eq_v_sd,eq_v_sq,eq_i_td,eq_i_tq,eq_i_sd,eq_i_sq,eq_eta_D,eq_eta_Q]
y_aux = [   v_md,   v_mq,   v_sd,   v_sq,   i_td,   i_tq,   i_sd,   i_sq,   eta_D,   eta_Q]
y_0_aux = [  0.0,  V_bdq,    0.0,  V_bdq,       0,      0]

## parameters:
params_vsc_filter = {}
for item in ['L_t','R_t','C_m','L_s','R_s','omega','G_d']:
    params_vsc_filter.update({f'{item}_{name}':vsg[item]})
    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
## inputs:
u_vsc_filter = {}
for item in ['v_dc']:#,'eta_D','eta_Q']:
    u_vsc_filter.update({f'{item}_{name}':vsg[item]})
    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
## dynamic states:
x_list_vsc_filter = []
for item in x_list_vsc_filter:
    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)")
## algebraic states
y_list_vsc_filter = ['i_tD','i_tQ'] + ['v_mD','v_mQ'] + ['i_sD','i_sQ'] + ['v_sD','v_sQ']
for item in y_list_vsc_filter:
    exec(f"{item} = sym.Symbol('{item}_{name}', real=True)",globals())
## feedbacks:
v_poiD,v_poiQ = sym.symbols(f'v_{bus}_D,v_{bus}_Q', real=True)
i_poiD,i_poiQ = sym.symbols(f'i_{bus}_D,i_{bus}_Q', real=True)

#eta_D = eta_D_ref # - Gv_in*(i_tD - i_sD)
#eta_Q = eta_Q_ref #- Gv_in*(i_tQ - i_sQ)

# LCL filter
di_tD = 1/L_t*(eta_D/2*v_dc - R_t*i_tD + omega*L_t*i_tQ - v_mD)
di_tQ = 1/L_t*(eta_Q/2*v_dc - R_t*i_tQ - omega*L_t*i_tD - v_mQ)
dv_mD = 1/C_m*(i_tD + C_m*omega*v_mQ - G_d*v_mD - i_sD)
dv_mQ = 1/C_m*(i_tQ - C_m*omega*v_mD - G_d*v_mQ - i_sQ)
di_sD = 1/L_s*(v_mD - R_s*i_sD + omega*L_s*i_sQ - v_sD)
di_sQ = 1/L_s*(v_mQ - R_s*i_sQ - omega*L_s*i_sD - v_sQ)

# Grid interaction
eq_i_poiD =  i_sD - i_poiD
eq_i_poiQ =  i_sQ - i_poiQ

eq_v_sD =  v_sD - v_poiD
eq_v_sQ =  v_sQ - v_poiQ

#grid_dae['params'].pop(f'i_{bus}_D')
#grid_dae['params'].pop(f'i_{bus}_Q')

# DAE
f_vsc_filter = [di_tD, di_tQ, dv_mD, dv_mQ, di_sD, di_sQ]
x_vsc_filter = [ i_tD,  i_tQ,  v_mD,  v_mQ,  i_sD,  i_sQ]
g_vsc_filter = [ eq_i_poiD, eq_i_poiQ, eq_v_sD, eq_v_sQ]
y_vsc_filter = [    i_poiD,    i_poiQ,    v_sD,    v_sQ ]
x_0_vsc_filter = [ ]
y_0_vsc_filter = [ 0.0,  0.0,   0.0, V_bdq,     0,     0,         0,         0,       0,   V_bdq]

## Model integration
f_vsg = f_vsc_filter
x_vsg = x_vsc_filter
g_vsg = g_vsc_filter + g_aux
y_vsg = y_vsc_filter + y_aux
params_vsg.update(params_vsc_filter)
#params_vsg.update(params_ctrl_1)
u_vsg.update(u_vsc_filter)
#u_vsg.update(u_ctrl_0)
#u_vsg.update(u_ctrl_1)
u_vsg.update({'v_1_D':0.0,'v_1_Q':326,'phi_g01':0.0,'eta_d_g01':0.0,'eta_q_g01':0.8})
#u_vsg.update({'v_1_D':0.0,'v_1_Q':326,'eta_d_g01':0.0,'eta_q_g01':0.8})
```

```python
i_sd_g01,i_sq_g01 = sym.symbols(f'i_sd_g01,i_sq_g01', real=True)
sys_dict = {'name':'vsc_lcl',
           'params_dict':params_vsg,
           'f_list':[f_vsg],
           'g_list':[g_vsg],
           'x_list':[x_vsg],
           'y_ini_list':y_vsg,
           'y_run_list':y_vsg,
           'u_run_dict':u_vsg,
           'u_ini_dict':u_vsg,
           'h_dict':{'v_t_D':eta_D/2*v_dc,'v_t_Q':eta_Q/2*v_dc,'damp_D':G_d*v_mD,'damp_Q':G_d*v_mQ,
                     'i_sd_g01':i_sd,'i_sq_g01':i_sq,
                     'v_md_g01':v_md,'v_mq_g01':v_mq,
                     'i_td_g01':i_td,'i_tq_g01':i_tq,
                     'v_sd_g01':v_sd,'v_sq_g01':v_sq,}
           }

db.system(sys_dict)
db.sys2num(sys_dict)
```

```python
u_vsg
```

    {'v_dc_g01': 750,
     'v_1_D': 0.0,
     'v_1_Q': 326,
     'phi_g01': 0.0,
     'eta_d_g01': 0.0,
     'eta_q_g01': 0.8}

```python
f_vsg
```

    [(L_t_g01*i_tQ_g01*omega_g01 - R_t_g01*i_tD_g01 + eta_D_g01*v_dc_g01/2 - v_mD_g01)/L_t_g01,
     (-L_t_g01*i_tD_g01*omega_g01 - R_t_g01*i_tQ_g01 + eta_Q_g01*v_dc_g01/2 - v_mQ_g01)/L_t_g01,
     (C_m_g01*omega_g01*v_mQ_g01 - G_d_g01*v_mD_g01 - i_sD_g01 + i_tD_g01)/C_m_g01,
     (-C_m_g01*omega_g01*v_mD_g01 - G_d_g01*v_mQ_g01 - i_sQ_g01 + i_tQ_g01)/C_m_g01,
     (L_s_g01*i_sQ_g01*omega_g01 - R_s_g01*i_sD_g01 + v_mD_g01 - v_sD_g01)/L_s_g01,
     (-L_s_g01*i_sD_g01*omega_g01 - R_s_g01*i_sQ_g01 + v_mQ_g01 - v_sQ_g01)/L_s_g01]

```python
sys_dict['g'].jacobian(sys_dict['u_run'])
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0\\0 & -1 & 0 & 0 & 0 & 0\\0 & 0 & -1 & 0 & 0 & 0\\0 & 0 & 0 & - v_{mD g01} \sin{\left(\phi_{g01} \right)} + v_{mQ g01} \cos{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - v_{mD g01} \cos{\left(\phi_{g01} \right)} - v_{mQ g01} \sin{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - v_{sD g01} \sin{\left(\phi_{g01} \right)} + v_{sQ g01} \cos{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - v_{sD g01} \cos{\left(\phi_{g01} \right)} - v_{sQ g01} \sin{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - i_{tD g01} \sin{\left(\phi_{g01} \right)} + i_{tQ g01} \cos{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - i_{tD g01} \cos{\left(\phi_{g01} \right)} - i_{tQ g01} \sin{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - i_{sD g01} \sin{\left(\phi_{g01} \right)} + i_{sQ g01} \cos{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - i_{sD g01} \cos{\left(\phi_{g01} \right)} - i_{sQ g01} \sin{\left(\phi_{g01} \right)} & 0 & 0\\0 & 0 & 0 & - \eta_{d g01} \sin{\left(\phi_{g01} \right)} - \eta_{q g01} \cos{\left(\phi_{g01} \right)} & \cos{\left(\phi_{g01} \right)} & - \sin{\left(\phi_{g01} \right)}\\0 & 0 & 0 & \eta_{d g01} \cos{\left(\phi_{g01} \right)} - \eta_{q g01} \sin{\left(\phi_{g01} \right)} & \sin{\left(\phi_{g01} \right)} & \cos{\left(\phi_{g01} \right)}\end{matrix}\right]$

```python
sys_dict['x']
```

$\displaystyle \left[\begin{matrix}i_{tD g01}\\i_{tQ g01}\\v_{mD g01}\\v_{mQ g01}\\i_{sD g01}\\i_{sQ g01}\end{matrix}\right]$

```python

```


# Solving Stiff Equations
### Chris Rackauckas

This tutorial is for getting into the extra features for solving stiff ordinary
differential equations in an efficient manner. Solving stiff ordinary
differential equations requires specializing the linear solver on properties of
the Jacobian in order to cut down on the O(n^3) linear solve and the O(n^2)
back-solves. Note that these same functions and controls also extend to stiff
SDEs, DDEs, DAEs, etc.

## Code Optimization for Differential Equations

### Writing Efficient Code

For a detailed tutorial on how to optimize one's DifferentialEquations.jl code,
please see the
[Optimizing DiffEq Code tutorial](http://tutorials.juliadiffeq.org/html/introduction/03-optimizing_diffeq_code.html).

### Choosing a Good Solver

Choosing a good solver is required for getting top notch speed. General
recommendations can be found on the solver page (for example, the
[ODE Solver Recommendations](https://docs.juliadiffeq.org/latest/solvers/ode_solve)).
The current recommendations can be simplified to a Rosenbrock method
(`Rosenbrock23` or `Rodas5`) for smaller (<50 ODEs) problems, ESDIRK methods
for slightly larger (`TRBDF2` or `KenCarp4` for <2000 ODEs), and Sundials
`CVODE_BDF` for even larger problems. `lsoda` from
[LSODA.jl](https://github.com/rveltz/LSODA.jl) is generally worth a try.

More details on the solver to choose can be found by benchmarking. See the
[DiffEqBenchmarks](https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl) to
compare many solvers on many problems.

### Check Out the Speed FAQ

See [this FAQ](http://docs.juliadiffeq.org/latest/basics/faq.html#Performance-1)
for information on common pitfalls and how to improve performance.

### Setting Up Your Julia Installation for Speed

Julia uses an underlying BLAS implementation for its matrix multiplications
and factorizations. This library is automatically multithreaded and accelerates
the internal linear algebra of DifferentialEquations.jl. However, for optimality,
you should make sure that the number of BLAS threads that you are using matches
the number of physical cores and not the number of logical cores. See
[this issue for more details](https://github.com/JuliaLang/julia/issues/33409).

To check the number of BLAS threads, use:

```julia
ccall((:openblas_get_num_threads64_, Base.libblas_name), Cint, ())
```

    8

If I want to set this directly to 4 threads, I would use:

```julia
using LinearAlgebra
LinearAlgebra.BLAS.set_num_threads(4)
```

Additionally, in some cases Intel's MKL might be a faster BLAS than the standard
BLAS that ships with Julia (OpenBLAS). To switch your BLAS implementation, you
can use [MKL.jl](https://github.com/JuliaComputing/MKL.jl) which will accelerate
the linear algebra routines. Please see the package for the limitations.

### Use Accelerator Hardware

When possible, use GPUs. If your ODE system is small and you need to solve it
with very many different parameters, see the
[ensembles interface](https://docs.juliadiffeq.org/latest/features/ensemble)
and [DiffEqGPU.jl](https://github.com/JuliaDiffEq/DiffEqGPU.jl). If your problem
is large, consider using a [CuArray](https://github.com/JuliaGPU/CuArrays.jl)
for the state to allow for GPU-parallelism of the internal linear algebra.

## Speeding Up Jacobian Calculations

When one is using an implicit or semi-implicit differential equation solver,
the Jacobian must be built at many iterations and this can be one of the most
expensive steps. There are two pieces that must be optimized in order to reach
maximal efficiency when solving stiff equations: the sparsity pattern and the
construction of the Jacobian. The construction is filling the matrix
`J` with values, while the sparsity pattern is what `J` to use.

The sparsity pattern is given by a prototype matrix, the `jac_prototype`, which
will be copied to be used as `J`. The default is for `J` to be a `Matrix`,
i.e. a dense matrix. However, if you know the sparsity of your problem, then
you can pass a different matrix type. For example, a `SparseMatrixCSC` will
give a sparse matrix. Additionally, structured matrix types like `Tridiagonal`,
`BandedMatrix` (from
[BandedMatrices.jl](https://github.com/JuliaMatrices/BandedMatrices.jl)),
`BlockBandedMatrix` (from
[BlockBandedMatrices.jl](https://github.com/JuliaMatrices/BlockBandedMatrices.jl)),
and more can be given. DifferentialEquations.jl will internally use this matrix
type, making the factorizations faster by utilizing the specialized forms.

For the construction, there are 3 ways to fill `J`:

- The default, which uses normal finite/automatic differentiation
- A function `jac(J,u,p,t)` which directly computes the values of `J`
- A `colorvec` which defines a sparse differentiation scheme.

We will now showcase how to make use of this functionality with growing complexity.

### Declaring Jacobian Functions

Let's solve the Rosenbrock equations:

$$\begin{align}
dy_1 &= -0.04y₁ + 10^4 y_2 y_3 \\
dy_2 &= 0.04 y_1 - 10^4 y_2 y_3 - 3*10^7 y_{2}^2 \\
dy_3 &= 3*10^7 y_{3}^2 \\
\end{align}$$

In order to reduce the Jacobian construction cost, one can describe a Jacobian
function by using the `jac` argument for the `ODEFunction`. First, let's do
a standard `ODEProblem`:

```julia
using DifferentialEquations
function rober(du,u,p,t)
  y₁,y₂,y₃ = u
  k₁,k₂,k₃ = p
  du[1] = -k₁*y₁+k₃*y₂*y₃
  du[2] =  k₁*y₁-k₂*y₂^2-k₃*y₂*y₃
  du[3] =  k₂*y₂^2
  nothing
end
prob = ODEProblem(rober,[1.0,0.0,0.0],(0.0,1e5),(0.04,3e7,1e4))
sol = solve(prob,Rosenbrock23())

using Plots
plot(sol, xscale=:log10, tspan=(1e-6, 1e5), layout=(3,1))
```

```julia
using BenchmarkTools
@btime solve(prob)
```

Now we want to add the Jacobian. First we have to derive the Jacobian
$\frac{df_i}{du_j}$ which is `J[i,j]`. From this we get:

```julia
function rober_jac(J,u,p,t)
  y₁,y₂,y₃ = u
  k₁,k₂,k₃ = p
  J[1,1] = k₁ * -1
  J[2,1] = k₁
  J[3,1] = 0
  J[1,2] = y₃ * k₃
  J[2,2] = y₂ * k₂ * -2 + y₃ * k₃ * -1
  J[3,2] = y₂ * 2 * k₂
  J[1,3] = k₃ * y₂
  J[2,3] = k₃ * y₂ * -1
  J[3,3] = 0
  nothing
end
f = ODEFunction(rober, jac=rober_jac)
prob_jac = ODEProblem(f,[1.0,0.0,0.0],(0.0,1e5),(0.04,3e7,1e4))

@btime solve(prob_jac)
```

### Automatic Derivation of Jacobian Functions

But that was hard! If you want to take the symbolic Jacobian of numerical
code, we can make use of [ModelingToolkit.jl](https://github.com/JuliaDiffEq/ModelingToolkit.jl)
to symbolicify the numerical code and do the symbolic calculation and return
the Julia code for this.

```julia
using ModelingToolkit
de = modelingtoolkitize(prob)
ModelingToolkit.generate_jacobian(de...)[2] # Second is in-place
```

which outputs:

```julia
:((##MTIIPVar#376, u, p, t)->begin
          #= C:\Users\accou\.julia\packages\ModelingToolkit\czHtj\src\utils.jl:65 =#
          #= C:\Users\accou\.julia\packages\ModelingToolkit\czHtj\src\utils.jl:66 =#
          let (x₁, x₂, x₃, α₁, α₂, α₃) = (u[1], u[2], u[3], p[1], p[2], p[3])
              ##MTIIPVar#376[1] = α₁ * -1
              ##MTIIPVar#376[2] = α₁
              ##MTIIPVar#376[3] = 0
              ##MTIIPVar#376[4] = x₃ * α₃
              ##MTIIPVar#376[5] = x₂ * α₂ * -2 + x₃ * α₃ * -1
              ##MTIIPVar#376[6] = x₂ * 2 * α₂
              ##MTIIPVar#376[7] = α₃ * x₂
              ##MTIIPVar#376[8] = α₃ * x₂ * -1
              ##MTIIPVar#376[9] = 0
          end
          #= C:\Users\accou\.julia\packages\ModelingToolkit\czHtj\src\utils.jl:67 =#
          nothing
      end)
```

Now let's use that to give the analytical solution Jacobian:

```julia
jac = eval(ModelingToolkit.generate_jacobian(de...)[2])
f = ODEFunction(rober, jac=jac)
prob_jac = ODEProblem(f,[1.0,0.0,0.0],(0.0,1e5),(0.04,3e7,1e4))
```

### Declaring a Sparse Jacobian

Jacobian sparsity is declared by the `jac_prototype` argument in the `ODEFunction`.
Note that you should only do this if the sparsity is high, for example, 0.1%
of the matrix is non-zeros, otherwise the overhead of sparse matrices can be higher
than the gains from sparse differentiation!

But as a demonstration, let's build a sparse matrix for the Rober problem. We
can do this by gathering the `I` and `J` pairs for the non-zero components, like:

```julia
I = [1,2,1,2,3,1,2]
J = [1,1,2,2,2,3,3]
using SparseArrays
jac_prototype = sparse(I,J,1.0)
```

Now this is the sparse matrix prototype that we want to use in our solver, which
we then pass like:

```julia
f = ODEFunction(rober, jac=jac, jac_prototype=jac_prototype)
prob_jac = ODEProblem(f,[1.0,0.0,0.0],(0.0,1e5),(0.04,3e7,1e4))
```

### Automatic Sparsity Detection

One of the useful companion tools for DifferentialEquations.jl is
[SparsityDetection.jl](https://github.com/JuliaDiffEq/SparsityDetection.jl).
This allows for automatic declaration of Jacobian sparsity types. To see this
in action, let's look at the 2-dimensional Brusselator equation:

```julia
const N = 32
const xyd_brusselator = range(0,stop=1,length=N)
brusselator_f(x, y, t) = (((x-0.3)^2 + (y-0.6)^2) <= 0.1^2) * (t >= 1.1) * 5.
limit(a, N) = a == N+1 ? 1 : a == 0 ? N : a
function brusselator_2d_loop(du, u, p, t)
  A, B, alpha, dx = p
  alpha = alpha/dx^2
  @inbounds for I in CartesianIndices((N, N))
    i, j = Tuple(I)
    x, y = xyd_brusselator[I[1]], xyd_brusselator[I[2]]
    ip1, im1, jp1, jm1 = limit(i+1, N), limit(i-1, N), limit(j+1, N), limit(j-1, N)
    du[i,j,1] = alpha*(u[im1,j,1] + u[ip1,j,1] + u[i,jp1,1] + u[i,jm1,1] - 4u[i,j,1]) +
                B + u[i,j,1]^2*u[i,j,2] - (A + 1)*u[i,j,1] + brusselator_f(x, y, t)
    du[i,j,2] = alpha*(u[im1,j,2] + u[ip1,j,2] + u[i,jp1,2] + u[i,jm1,2] - 4u[i,j,2]) +
                A*u[i,j,1] - u[i,j,1]^2*u[i,j,2]
    end
end
p = (3.4, 1., 10., step(xyd_brusselator))
```

Given this setup, we can give and example `input` and `output` and call `sparsity!`
on our function with the example arguments and it will kick out a sparse matrix
with our pattern, that we can turn into our `jac_prototype`.

```julia
using SparsityDetection, SparseArrays
input = rand(32,32,2)
output = similar(input)
sparsity_pattern = sparsity!(brusselator_2d_loop,output,input,p,0.0)
jac_sparsity = Float64.(sparse(sparsity_pattern))
```

Let's double check what our sparsity pattern looks like:

```julia
using Plots
spy(jac_sparsity,markersize=1,colorbar=false,color=:deep)
```

That's neat, and would be tedius to build by hand! Now we just pass it to the
`ODEFunction` like as before:

```julia
f = ODEFunction(brusselator_2d_loop;jac_prototype=jac_sparsity)
```

Build the `ODEProblem`:

```julia
function init_brusselator_2d(xyd)
  N = length(xyd)
  u = zeros(N, N, 2)
  for I in CartesianIndices((N, N))
    x = xyd[I[1]]
    y = xyd[I[2]]
    u[I,1] = 22*(y*(1-y))^(3/2)
    u[I,2] = 27*(x*(1-x))^(3/2)
  end
  u
end
u0 = init_brusselator_2d(xyd_brusselator)
prob_ode_brusselator_2d = ODEProblem(brusselator_2d_loop,
                                     u0,(0.,11.5),p)

prob_ode_brusselator_2d_sparse = ODEProblem(f,
                                     u0,(0.,11.5),p)
```

Now let's see how the version with sparsity compares to the version without:

```julia
@btime solve(prob_ode_brusselator_2d,save_everystep=false)
@btime solve(prob_ode_brusselator_2d_sparse,save_everystep=false)
```

### Declaring Color Vectors for Fast Construction

If you cannot directly define a Jacobian function, you can use the `colorvec`
to speed up the Jacobian construction. What the `colorvec` does is allows for
calculating multiple columns of a Jacobian simultaniously by using the sparsity
pattern. An explanation of matrix coloring can be found in the
[MIT 18.337 Lecture Notes](https://mitmath.github.io/18337/lecture9/stiff_odes).

To perform general matrix coloring, we can use
[SparseDiffTools.jl](https://github.com/JuliaDiffEq/SparseDiffTools.jl). For
example, for the Brusselator equation:

```julia
using SparseDiffTools
colorvec = matrix_colors(jac_sparsity)
@show maximum(colorvec)
```

This means that we can now calculate the Jacobian in 12 function calls. This is
a nice reduction from 2048 using only automated tooling! To now make use of this
inside of the ODE solver, you simply need to declare the colorvec:

```julia
f = ODEFunction(brusselator_2d_loop;jac_prototype=jac_sparsity,
                                    colorvec=colorvec)
prob_ode_brusselator_2d_sparse = ODEProblem(f,
                                     init_brusselator_2d(xyd_brusselator),
                                     (0.,11.5),p)
@btime solve(prob_ode_brusselator_2d_sparse,save_everystep=false)
```

Notice the massive speed enhancement!

## Defining Linear Solver Routines and Jacobian-Free Newton-Krylov

A completely different way to optimize the linear solvers for large sparse
matrices is to use a Krylov subpsace method. This requires choosing a linear
solver for changing to a Krylov method. Optionally, one can use a Jacobian-free
operator to reduce the memory requirements.

### Declaring a Jacobian-Free Newton-Krylov Implementation

To swap the linear solver out, we use the `linsolve` command and choose the
GMRES linear solver.

```julia
@btime solve(prob_ode_brusselator_2d,TRBDF2(linsolve=LinSolveGMRES()),save_everystep=false)
@btime solve(prob_ode_brusselator_2d_sparse,TRBDF2(linsolve=LinSolveGMRES()),save_everystep=false)
```

For more information on linear solver choices, see the
[linear solver documentation](https://docs.juliadiffeq.org/latest/features/linear_nonlinear).

On this problem, handling the sparsity correctly seemed to give much more of a
speedup than going to a Krylov approach, but that can be dependent on the problem
(and whether a good preconditioner is found).

We can also enhance this by using a Jacobian-Free implementation of `f'(x)*v`.
To define the Jacobian-Free operator, we can use
[DiffEqOperators.jl](https://github.com/JuliaDiffEq/DiffEqOperators.jl) to generate
an operator `JacVecOperator` such that `Jv*v` performs `f'(x)*v` without building
the Jacobian matrix.

```julia
using DiffEqOperators
Jv = JacVecOperator(brusselator_2d_loop,u0,p,0.0)
```

and then we can use this by making it our `jac_prototype`:

```julia
f = ODEFunction(brusselator_2d_loop;jac_prototype=Jv)
prob_ode_brusselator_2d_jacfree = ODEProblem(f,u0,(0.,11.5),p)
@btime solve(prob_ode_brusselator_2d_jacfree,TRBDF2(linsolve=LinSolveGMRES()),save_everystep=false)
```

### Adding a Preconditioner

The [linear solver documentation](https://docs.juliadiffeq.org/latest/features/linear_nonlinear/#iterativesolvers-jl-1)
shows how you can add a preconditioner to the GMRES. For example, you can
use packages like [AlgebraicMultigrid.jl](https://github.com/JuliaLinearAlgebra/AlgebraicMultigrid.jl)
to add an algebraic multigrid (AMG) or [IncompleteLU.jl](https://github.com/haampie/IncompleteLU.jl)
for an incomplete LU-factorization (iLU).

```julia
using AlgebraicMultigrid
pc = aspreconditioner(ruge_stuben(jac_sparsity))
@btime solve(prob_ode_brusselator_2d_jacfree,TRBDF2(linsolve=LinSolveGMRES(Pl=pc)),save_everystep=false)
```

## Using Structured Matrix Types

If your sparsity pattern follows a specific structure, for example a banded
matrix, then you can declare `jac_prototype` to be of that structure and then
additional optimizations will come for free. Note that in this case, it is
not necessary to provide a `colorvec` since the color vector will be analytically
derived from the structure of the matrix.

The matrices which are allowed are those which satisfy the
[ArrayInterface.jl](https://github.com/JuliaDiffEq/ArrayInterface.jl) interface
for automatically-colorable matrices. These include:

- Bidiagonal
- Tridiagonal
- SymTridiagonal
- BandedMatrix ([BandedMatrices.jl](https://github.com/JuliaMatrices/BandedMatrices.jl))
- BlockBandedMatrix ([BlockBandedMatrices.jl](https://github.com/JuliaMatrices/BlockBandedMatrices.jl))

Matrices which do not satisfy this interface can still be used, but the matrix
coloring will not be automatic, and an appropriate linear solver may need to
be given (otherwise it will default to attempting an LU-decomposition).

## Sundials-Specific Handling

While much of the setup makes the transition to using Sundials automatic, there
are some differences between the pure Julia implementations and the Sundials
implementations which must be taken note of. These are all detailed in the
[Sundials solver documentation](https://docs.juliadiffeq.org/latest/solvers/ode_solve/#ode_solve_sundials-1),
but here we will highlight the main details which one should make note of.

Defining a sparse matrix and a Jacobian for Sundials works just like any other
package. The core difference is in the choice of the linear solver. With Sundials,
the linear solver choice is done with a Symbol in the `linear_solver` from a
preset list. Particular choices of note are `:Band` for a banded matrix and
`:GMRES` for using GMRES. If you are using Sundials, `:GMRES` will not require
defining the JacVecOperator, and instead will always make use of a Jacobian-Free
Newton Krylov (with numerical differentiation). Thus on this problem we could do:

```julia
using Sundials
# Sparse Version
@btime solve(prob_ode_brusselator_2d_sparse,CVODE_BDF(),save_everystep=false)
# GMRES Version: Doesn't require any extra stuff!
@btime solve(prob_ode_brusselator_2d,CVODE_BDF(linear_solver=:GMRES),save_everystep=false)
```

Details for setting up a preconditioner with Sundials can be found at the
[Sundials solver page](https://docs.juliadiffeq.org/latest/solvers/ode_solve/#ode_solve_sundials-1).

## Handling Mass Matrices

Instead of just defining an ODE as $u' = f(u,p,t)$, it can be common to express
the differential equation in the form with a mass matrix:

$$Mu' = f(u,p,t)$$

where $M$ is known as the mass matrix. Let's solve the Robertson equation.
At the top we wrote this equation as:

$$\begin{align}
dy_1 &= -0.04y₁ + 10^4 y_2 y_3 \\
dy_2 &= 0.04 y_1 - 10^4 y_2 y_3 - 3*10^7 y_{2}^2 \\
dy_3 &= 3*10^7 y_{3}^2 \\
\end{align}$$

But we can instead write this with a conservation relation:

$$\begin{align}
dy_1 &= -0.04y₁ + 10^4 y_2 y_3 \\
dy_2 &= 0.04 y_1 - 10^4 y_2 y_3 - 3*10^7 y_{2}^2 \\
1 &=  y_{1} + y_{2} + y_{3} \\
\end{align}$$

In this form, we can write this as a mass matrix ODE where $M$ is singular
(this is another form of a differential-algebraic equation (DAE)). Here, the
last row of `M` is just zero. We can implement this form as:

```julia
using DifferentialEquations
function rober(du,u,p,t)
  y₁,y₂,y₃ = u
  k₁,k₂,k₃ = p
  du[1] = -k₁*y₁+k₃*y₂*y₃
  du[2] =  k₁*y₁-k₂*y₂^2-k₃*y₂*y₃
  du[3] =  y₁ + y₂ + y₃ - 1
  nothing
end
M = [1. 0  0
     0  1. 0
     0  0  0]
f = ODEFunction(rober,mass_matrix=M)
prob_mm = ODEProblem(f,[1.0,0.0,0.0],(0.0,1e5),(0.04,3e7,1e4))
sol = solve(prob_mm,Rodas5())

plot(sol, xscale=:log10, tspan=(1e-6, 1e5), layout=(3,1))
```

Note that if your mass matrix is singular, i.e. your system is a DAE, then you
need to make sure you choose
[a solver that is compatible with DAEs](https://docs.juliadiffeq.org/latest/solvers/dae_solve/#dae_solve_full-1)