```python
import IPython as Ipy
Ipy.display.YouTubeVideo('https://www.youtube.com/watch?v=ypUqQWYZTJs&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=28',width= 600,height=400)
```

# [Linearity](https://www.youtube.com/watch?v=ypUqQWYZTJs&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=28)
> ### $ f(x) = \color{magenta}{a}x $
>> ### $
f(\vartriangle + \;\square) = a\vartriangle + a\; \square\\
f(\vartriangle + \; \square) = f(\vartriangle) + f(\square) \\
f(\vartriangle + \; 0) = f(\vartriangle) + f(0), \;s.t\; \square = 0 \\
f(0) = 0 \\
f(s\vartriangle) = sf(\vartriangle)
$
# Non linearity
> ### $ g(x) = ax + \color{magenta}{b}$
>> ### $
g(\vartriangle + \square) = a \vartriangle + a\square  + b \\
\therefore  g(\vartriangle + \square) \neq g(\vartriangle) + g(\square)
$

# Linearity of determinant
> ### row lineartity $
\begin{vmatrix}
\vartriangle_1 + \; \square_1 &  \vartriangle_2 + \; \square_2 \\
c & d
\end{vmatrix}
=
\begin{vmatrix}
\vartriangle_1 &  \vartriangle_2 \\
c & d
\end{vmatrix}
+ \begin{vmatrix}
\square_1 & \square_2 \\
c & d
\end{vmatrix}
$

> ### column lineartity $
\begin{vmatrix}
\color{magenta}{\vartriangle_1 + \; \square_1} & b \\
\color{magenta}{\vartriangle_2 + \; \square_2} & d \\
\end{vmatrix}
=
\begin{vmatrix}
\color{magenta}{\vartriangle_1} & b \\
\color{magenta}{\vartriangle_2} & d \\
\end{vmatrix}
+ \begin{vmatrix}
\color{magenta}{\square_1} & b \\
\color{magenta}{\square_2} & d \\
\end{vmatrix}
$

> ### non-lineartity of entry $
\begin{vmatrix}
\vartriangle_1 + \; \square_1 & b \\
\vartriangle_2 & d \\
\end{vmatrix}
\neq
\begin{vmatrix}
\vartriangle_1 & b \\
\vartriangle_2 & d \\
\end{vmatrix}
+ \begin{vmatrix}
\square_1 & b \\
\vartriangle_2 & d \\
\end{vmatrix}
\\
\because \text{ not curve}
$

> ### lineartity $
\begin{vmatrix}
\vartriangle_1 + \; \square_1 & b \\
\vartriangle_2 + \; \color{magenta}{0} & d \\
\end{vmatrix}
= \begin{vmatrix}
\vartriangle_1 & b \\
\vartriangle_2 & d \\
\end{vmatrix}
+ \begin{vmatrix}
\square_1 & b \\
\color{magenta}{0} & d \\
\end{vmatrix}
$

> ### non-lineartity - diagnonal $
\begin{vmatrix}
\vartriangle_1 + \; \square_1 & b \\
c & \vartriangle_2 + \; \square_2 \\
\end{vmatrix}
\neq
\begin{vmatrix}
\vartriangle_1 & b \\
c & \vartriangle_2 \\
\end{vmatrix}
+ \begin{vmatrix}
\square_1 & b \\
c & \square_2 \\
\end{vmatrix}
$

# multi-linear of determinant
> ### $
\big|A\big| = det(A) = \sum_{j} a_{ij}C_{ij} \\
\big|A\big| = det(A) = \sum_{j} a_{ji}C_{ji} \\
$
>> ### Minor Matrix
>>> ### $
\begin{bmatrix}
a_{11} & a_{12} & a_{13} & \cdots & a_{1n} \\
a_{21} & M_{11} & M_{12} & \cdots & M_{1,n-1} \\
a_{31} & M_{21} & M_{22} & \cdots & M_{2,n-1} \\
\vdots & \ddots & \ddots & \ddots & \vdots \\
a_{n1} & M_{n-1,1} & M_{n-1,2} & \cdots & M_{n-1,n-1} \\
\end{bmatrix}
$
>> ### Cofactor Matrix
>>> ### $
C_{ij} = (-1)^{i+j}M_{ij}
$

> ### arbitary i row determinant = arbitary j column determinant
>> ### $ det(A) = det\big[a_{i}\big] = \sum_{j}a_{ij}C_{ij} $
>>> ### $ det(A) = det\big[ 0 \big] = \sum_{j}(0)C_{ij} = 0 $
>>> ### $ det(A) = det\big[ sa_{i} \big] = \sum_{j}sa_{ij}C_{ij} = s\sum_{ij}C_{ij}= s\; \det(A) $

> ### determinant having a same row or column matrix
>> ### $
det\begin{bmatrix}
\check {\vartriangle}
\\ \vartriangle
\end{bmatrix}
= \sum_j \vartriangle_{i,j}(-1)^{i+j}M_{ij}
\\
det\begin{bmatrix}
\vartriangle
\\ \check{\vartriangle}
\end{bmatrix}
= \sum_j \vartriangle_{i+1,j}(-1)^{i+1+j}M_{i+1,j}
\\\qquad \quad = -\sum_j \vartriangle_{ij}(-1)^{i+j}M_{ij}
$
>>> ### 어떤 row 나 column을 선택해도 determimant는 같으므로
>>>> ### $
det \left[ \begin{array}{r} \vartriangle \\ \vartriangle \end{array} \right]
= - det \left[ \begin{array}{r} \vartriangle \\ \vartriangle \end{array} \right]
\\
\therefore det \left[
\begin{array}{c}\vartriangle \\ \vartriangle \end{array}\right] = 0
\\
\therefore det \left[
\begin{array}{c}\vartriangle \\ \vdots \\ \vartriangle \end{array}\right] = 0
$

> ### determinant having a same row or column matrix
>> ### $
0 = det\begin{bmatrix}
\vartriangle +\;\square
\\ \vartriangle +\; \square
\end{bmatrix}
=
det\begin{bmatrix}
\vartriangle
\\ \vartriangle +\; \square
\end{bmatrix}
+
det\begin{bmatrix}
\square
\\ \vartriangle +\; \square
\end{bmatrix}
$

>> ### $
0 =
\underbrace{
det\begin{bmatrix}
\vartriangle
\\ \vartriangle
\end{bmatrix}}_{0}
+
det\begin{bmatrix}
\vartriangle
\\ \square
\end{bmatrix}
+
det\begin{bmatrix}
\square
\\ \vartriangle
\end{bmatrix}
+
\underbrace{
det\begin{bmatrix}
\square
\\\square
\end{bmatrix}}_{0}
\\
\therefore 0 = \left[ \begin{array}{c}\vartriangle \\ \square \end{array} \right]
+ \left[ \begin{array}{c}\square \\ \vartriangle \end{array} \right]
\\ \therefore \left[ \begin{array}{c}\vartriangle \\ \square \end{array} \right]
= - \left[ \begin{array}{c}\square \\ \vartriangle \end{array} \right]
$
> ### 행교환
>> ### $
0 = det\begin{bmatrix}
\vartriangle +\;\square \\ \vdots
\\ \vartriangle +\; \square
\end{bmatrix}
\\
\therefore
\left[ \begin{array}{c}\vartriangle \\ \vdots \\ \square \end{array} \right]
= - \left[ \begin{array}{c}\square \\ \vdots \\ \vartriangle \end{array} \right]
$

# addtion of determinant
> ### $
det \left[
\begin{array}{c} \vartriangle \\ \vdots \\ \square + s \vartriangle \end{array} \right]
= det \left[
\begin{array}{c} \vartriangle \\ \vdots \\ \square
\end{array} \right]
+ det \left[
\begin{array}{c} \vartriangle \\ \vdots \\ s\vartriangle
\end{array} \right]
\\= det \left[
\begin{array}{c} \vartriangle \\ \vdots \\ \square
\end{array} \right]
+ s\; det \left[
\begin{array}{c} \vartriangle \\ \vdots \\ \vartriangle
\end{array} \right]
\\=
det \left[ \begin{array}{c}\vartriangle \\ \vdots \\ \square \end{array}\right]
$

# [Determinant](https://www.youtube.com/watch?v=FsCLrXjRCIM&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=29)
> $
det\begin{bmatrix}a & b \\ c & d\end{bmatrix} =
det \Big(\begin{bmatrix}a \\ c \end{bmatrix}, \begin{bmatrix} b \\ d \end{bmatrix}\Big)
\\ =
det \Big(
a \begin{bmatrix} 1 \\ 0 \end{bmatrix} +
c \begin{bmatrix} 0 \\ 1 \end{bmatrix},
b \begin{bmatrix} 1 \\ 0 \end{bmatrix} +
d \begin{bmatrix} 0 \\ 1 \end{bmatrix}
\Big)
\\ =
det \Big(
a \begin{bmatrix} 1 \\ 0 \end{bmatrix},
b \begin{bmatrix} 1 \\ 0 \end{bmatrix} +
d \begin{bmatrix} 0 \\ 1 \end{bmatrix}
\Big) +
det \Big(
c \begin{bmatrix} 0 \\ 1 \end{bmatrix},
b \begin{bmatrix} 1 \\ 0 \end{bmatrix} +
d \begin{bmatrix} 0 \\ 1 \end{bmatrix}
\Big)
\\
=
ab \det\Big(
\begin{bmatrix} 1 \\ 0 \end{bmatrix},
\begin{bmatrix} 1 \\ 0 \end{bmatrix}
\Big) +
ad \det\Big(
\begin{bmatrix} 1 \\ 0 \end{bmatrix},
\begin{bmatrix} 0 \\ 0 \end{bmatrix}
\Big) +
cb \det\Big(
\begin{bmatrix} 0 \\ 1 \end{bmatrix},
\begin{bmatrix} 1 \\ 0 \end{bmatrix}
\Big) +
cd \det\Big(
\begin{bmatrix} 0 \\ 1 \end{bmatrix},
\begin{bmatrix} 0 \\ 1 \end{bmatrix}
\Big)
\\
=
ab \det \begin{bmatrix} 1 & 1 \\ 0 & 0 \end{bmatrix}
+
ad \det \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}
+
cb \det \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}
+
cd \det \begin{bmatrix} 0 & 0 \\ 1 & 1 \end{bmatrix}
\\ = ad - bc \\
E^1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \quad
E^2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \\
det \big[E^1,E^2 \big] = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = 1, \quad
det \big[E^2,E^1 \big] = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = -1
\\
\therefore det \begin{bmatrix}a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix}
= det \Big[ A^1, A^2 \Big] = \sum_{ij} a_{i1}a_{j2}\; det\Big(E^i,E^j\Big)
= det
$

# determinant 3by 3
> ### $
\begin{vmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\
a_{31} & a_{32} & a_{33} \\
\end{vmatrix}
=
\Big|
A^1 , A^2 , A^3
\Big|
= \sum_{ijk}a_{i1}a_{j2} a_{k3}\Big| E^{i} E^{j} E^{k} \Big|
\\
\therefore \sum_{ijk} \epsilon \Big(i,j,k \Big)a_{i1}a_{j2}a_{k3}\; det \Big(E^1,E^2,E^3 \Big)
$

> ### $
det \Big( E^{1}, E^{2}, E^{3} \Big) =
\begin{vmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 \\
\end{vmatrix} = 1
\\
det \Big( E^{1}, E^{3}, E^{2} \Big) =
\begin{vmatrix}
1 & 0 & 0 \\
0 & 0 & 1 \\
0 & 1 & 0 \\
\end{vmatrix} = -1
$

# n by n determinant
> ### $
\begin{vmatrix}
a_{11} & a_{12} & \cdots & a_{1n} \\
a_{21} & a_{22} & \cdots & a_{2n} \\
\vdots & \vdots & \ddots & \vdots \\
a_{n1} & a_{n2} & \cdots & a_{nn} \\
\end{vmatrix} = det\Big[ A^1,A^2,\cdots,A^n \Big]
\\=
\sum_{s_1s_2\cdots s_n}A_{s_{1,1}}, \cdots a_{s_n,n}\; det\Big(E^{s_1},\cdots,E^{s_n}\Big)
\\
A^1 = \sum b_{j1}G^{j} = b_{11}G^1 + b_{21}G^2 + \cdots + b_{n1}G^n
\\ \vdots \\
A^k = \sum b_{jk}G^{j} = b_{1k}G^1 + b_{2k}G^2 + \cdots + b_{nk}G^n
\\ \vdots \\
A^n = \sum b_{jn}G^{j} = b_{1n}G^1 + b_{2n}G^2 + \cdots + b_{nn}G^n
\\
det\Big[A^1, A^2, \cdots, A^n\Big]
= \sum_{s_1s_2\cdots s_n} \epsilon\Big(s_1,s_2,\cdots, s_n \Big) b_{s_1,1}b_{s_2,2}\cdots b_{s_n,n}\; det\Big(G^1,G^2,\cdots,G^n\Big)
$

# Sumation
> ### $
AB = \sum_{ij}a_{ij}G_{ij} \sum_{lm}b_{lm}G_{lm}
\\ = \sum_{ijm}a_{ij}b{jm}G_{im}
\\ = \sum_{jm} b_{jm}(\sum_{}a_{}G_{})
$

# inverse
> ### $
T^{-1} = B, \quad Tx = h \\
B_{i} \cdot T^{j} = \delta_{i}^{j} \\
B_{i}\cdot h = x_{i} \\
\begin{bmatrix}
a & b \\ c & d
\end{bmatrix}
\begin{bmatrix}
a^{-1}(x) & b^{-1}(u) \\ c^{-1}(y) & d^{-1}(v)
\end{bmatrix}
=
\begin{bmatrix}
1 & 0 \\ 0 & 1
\end{bmatrix}
\iff  ax + by = 1  \quad cx + dy = 0,\quad   au + bv = 1 \quad  cu + dv = 0
$
>> ### $
(ax + by = 1)d - (cx + dy = 0)b, \to adx - bcx = d \to x(a^{-1}) = \frac{d}{ad-bc} \\
(ax + by = 1)c - (cx + dy = 0)a, \to bcy - adx = c \to y(c^{-1}) = \frac{c}{-(ad-bc)} = \frac{-c}{ad-bc} \\
(au + bv = 0)d - (cu + dv = 1)b, \to adu - bcu = -b \to u(b^{-1}) = \frac{-b}{ad-bc}\\
(au + bv = 0)c - (cu + dv = 1)a, \to bcv - dav = -a \to v(c^{-1}) = \frac{-a}{-(ad-bc)} = \frac{a}{ad-bc} \\
\therefore \begin{bmatrix}
\frac{d}{ad-bc} & \frac{-b}{ad-bc} \\ \frac{-c}{ad-bc} & \frac{a}{ad-bc}
\end{bmatrix} \iff
\therefore \frac{1}{ad-bc}\begin{bmatrix}
d & -b \\ -c & a
\end{bmatrix}
$

# determinant
> ### $
det(A) = det(A^{T}) \\
A = \begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \\
det(A) = \sum_{k=1}^{n}(-1)^{i+k}a_{ik} M_{ik} =  \sum_{k=1}^{n}a_{ik}C_{ik}
\\
C_{ij} = (-1)^{i+j}M_{ij}
$

# multi linear transformation of determinant
# 같은 행벡터가 두개 있을때
> ### $ det \begin{bmatrix} \vdots \\ \vartriangle \\ \vartriangle \\ \vdots \end{bmatrix} = 0 = det \begin{bmatrix} \vdots \\ \vartriangle \\ \vdots \\ \vartriangle \\ \vdots \end{bmatrix} $

# 행교환
> ### $ det \begin{bmatrix} \vdots \\ \square \\ \vartriangle \\ \vdots \end{bmatrix} = -det \begin{bmatrix} \vdots \\ \square \\ \vartriangle \\ \vdots \end{bmatrix} \iff det \begin{bmatrix} \vdots \\ \square \\ \vdots \\ \vartriangle \\ \vdots \end{bmatrix} = -det \begin{bmatrix} \vdots \\ \square \\ \vdots \\ \vartriangle \\ \vdots \end{bmatrix} $

# 상수배후 다른행에 더해주기
> ### $ det \begin{bmatrix} \vartriangle \\ \vdots \\ \square + s\vartriangle \end{bmatrix}
= det \begin{bmatrix} \vartriangle \\ \vdots \\ \square \end{bmatrix}
+ det \begin{bmatrix} \vartriangle \\ \vdots \\ s\vartriangle \end{bmatrix}
= det \begin{bmatrix} \vartriangle \\ \vdots \\ \square \end{bmatrix}
+ s\, det \begin{bmatrix} \vartriangle \\ \vdots \\ \vartriangle \end{bmatrix} $

# L (low triangle matrix) $\iff$ U (upper triangle matrix)
> ### $
det(L) = det
\begin{bmatrix}
a_{11} & 0 & \cdots & 0 \\ ? & a_{22} & \cdots & 0 \\ \vdots & \ddots & \ddots & \vdots  \\ ? & ? & \cdots & a_{nn}
\end{bmatrix}
= a_{11} \times a_{22} \times \cdots \times a_{nn}
$

```python
import numpy as np
import sympy as sm
import sympy.vector
N = sm.vector.CoordSys3D('N')

# v(3,1), w(1,2) \iff T([3,1],[1,2])
u = 3*N.i + N.j + 0*N.k
v = N.i + 2*N.j + 0*N.k
w = 0*N.i + 0*N.j + N.k
T = u.to_matrix(N).col_insert(1,v.to_matrix(N)).col_insert(2,w.to_matrix(N))
T = u.to_matrix(N).row_join(v.to_matrix(N)).row_join(w.to_matrix(N))
B = T.inv()
# x(x_0, x_1)
x0,x1,x2 = sm.symbols('x_0 x_1 x_2')
x = sm.Matrix([x0,x1,x2])
# sm.vector.matrix_to_vector(x,N)

# Tx = h(3,2)
h = 3*N.i + 2*N.j + 0*N.k

import matplotlib.pyplot as plt
%matplotlib widget
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot()
ax.grid()
ax.spines[['left','bottom']].set_position('zero')
ax.set_aspect('equal')
ax.set_xlim(xmin=-4,xmax=9)
ax.set_ylim(ymin=-4,ymax=9)
ax.set_xticks(np.arange(-4,9,1))
ax.set_yticks(np.arange(-4,9,1))
ax.quiver(0,0,3,1,scale=1,units='xy')
ax.quiver(0,0,-1,3,scale=1,units='xy')
ax.annotate("B'(3,1)",(2.2,1.1))
ax.quiver(0,0,1,2,scale=1,units='xy')
ax.quiver(0,0,2,-1,scale=1,units='xy')
ax.annotate('A(1,2)',(0.6,2.1))
ax.annotate("A'(0,0)",(0.2,-0.3))
ax.quiver(0,0,4,3,scale=1,units='xy')
ax.annotate('B(4,3)',(4.1,3))
ax.annotate('C(4,2)',(4.2,2))
ax.annotate("C'(3,0)",(3.1,0.1))
ax.annotate("D(3,3)",(2.8,3.1))
ax.annotate("D'(0,2)",(-0.5,2.1))
ax.plot([1,4],[2,2],color='r')
ax.plot([1,4],[2,3],color='r')
ax.plot([4,4],[2,3],color='r')
ax.plot([3,3],[0,1],color='r')
ax.plot([0,3],[0,0],color='r')

ax.plot([3,4],[1,3],color='b')
ax.plot([3,4],[3,3],color='b')
ax.plot([0,0],[0,2],color='b')
ax.plot([0,1],[2,2],color='b')
ax.plot([3,3],[2,3],color='b')
ax.plot([3,3],[1,2],color='k')

# solve
T.inv()*sm.Matrix([3,2,0])
(T.inv()*x).T*(h.to_matrix(N))
```

$\displaystyle \left[\begin{matrix}\frac{4 x_{0}}{5} + \frac{3 x_{1}}{5}\end{matrix}\right]$

        Figure

```python

```

# [determinant & inverse matrix meanning](https://www.youtube.com/watch?v=CLS3Z9jbZw4&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=33)

```python
# v(3,1), w(2,2) \iff T([3,1],[2,2]) , b=(3,2)
# https://www.youtube.com/watch?v=CLS3Z9jbZw4&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=33
u = 3*N.i + N.j + 0*N.k
v = 2*N.i + 2*N.j + 0*N.k
w = 0*N.i + 0*N.j + N.k
T = u.to_matrix(N).col_insert(1,v.to_matrix(N)).col_insert(2,w.to_matrix(N))
T = u.to_matrix(N).row_join(v.to_matrix(N)).row_join(w.to_matrix(N))

# x(x_0, x_1)
x1,x2,x3 = sm.symbols('x_1 x_2 x_3')
x = sm.Matrix([x1,x2,x3])

# Tx = h(3,2)
h = 3*N.i + 2*N.j + 0*N.k

import matplotlib.pyplot as plt
%matplotlib widget
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot()
ax.grid()
#ax.spines[['left','bottom']].set_position('zero')
ax.axhline()
ax.axvline()
ax.set_aspect('equal')
ax.set_xlim(xmin=-2,xmax=4)
ax.set_ylim(ymin=-2,ymax=4)
ax.set_xticks(np.arange(-2,4,1/4))
ax.set_xticklabels(ax.get_xticks(),rotation=45)
ax.set_yticks(np.arange(-2,4,1/4))
# u(3,1,0)
ax.quiver(0,0,3,1,scale=1,units='xy')
ax.annotate(r"$T^1=u(3,1)$",(3,1))
# v(2,2,0)
ax.quiver(0,0,2,2,scale=1,units='xy')
ax.annotate(r"$T^2v(2,2)$",(2,2))
# b(3,2)
ax.scatter(3,2,color='r')
ax.annotate("b(3,2)",(3,2))
# T.inv()  = [1/2, -1/2], [-1/4, 3/4]
# new axix of N.i (1/2, -1/2)
# new axix of N.j (-1/4, 3/4)
B = T.inv()
ax.quiver(0,0,1/2,-1/2,scale=1,units='xy')
ax.annotate(r"$B_{1}=\hat{e_1}( \frac{1}{2}, \frac{-1}{2})$",(1/2,-1/2))
ax.quiver(0,0,-1/4,3/4,scale=1,units='xy')
ax.annotate(r"$B_{2}=\hat{e_2}(\frac{-1}{4}, \frac{3}{4})$",(-1/4,3/4))

# B_i * h = x_i
# B_0 * h = x_0
B.row(0)*(h.to_matrix(N))
```

$\displaystyle \left[\begin{matrix}\frac{1}{2}\end{matrix}\right]$

        Figure

```python
#%%javascript
#MathJax.Hub.Config({
#    TeX: { equationNumbers: { autoNumber: "AMS" } }
#});
```

# 1 by 1 determinat
> ### $
Tx = h \\
x = \frac{b}{a}
$
# 2 by 2 determinant
> ### $
\begin{align}
& ax + by = u  \label{eq1}\tag{1}\\
& cx + dy = v  \label{eq2}\tag{2}\\
& adx + bdy = ud & \eqref{eq1} \times d \label{eq3}\tag{3}\\\
& bcx + bdy = bv & \eqref{eq2} \times b \label{eq4}\tag{4}\\
& (ad - bc)x = ud - bv & \eqref{eq3} - \eqref{eq4} \\
\therefore x = \frac{ud - bv}{ad - bc} & \\
\end{align}
$
> ### 행교환 $\iff$ 부호가 바뀜
>> ### $
cx + dy = v \\
ax + by = u \\
x = \frac{bv - ud}{bc - ad}
$

# 3 by 3 determinant
> ### $
a_1x + b_1y + c_1z = u\\
a_2x + b_2y + c_2z = v\\
a_3x + b_3y + c_3z = w\\
\therefore
|A| = det(A)$
>> ### $$
x =
\frac{
u\begin{vmatrix}b_2 & c_2 \\ b_3 & c_3\end{vmatrix} -
v\begin{vmatrix}b_1 & c_1 \\ b_3 & c_3\end{vmatrix} +
w\begin{vmatrix}b_1 & c_1 \\ b_2 & c_2\end{vmatrix}
}
{
a_1\begin{vmatrix}b_2 & c_2 \\ b_3 & c_3\end{vmatrix} -
a_2\begin{vmatrix}b_1 & c_1 \\ b_3 & c_3\end{vmatrix} +
a_3\begin{vmatrix}b_1 & c_1 \\ b_2 & c_2\end{vmatrix}
}
$$

# 행이나 열이 서로 비례하는 성분이 2개 행열식은 0이된다.
> ### 서로 다른 두점$(x_0,x_2),(y_0,y_1)$을 지나는 직선의 방정식은 행열식이 0이 되어야 함으로
>> ### $
det (A) \Rightarrow
\begin{align}
\begin{vmatrix}
1 & x & y \\ 1 & x_0 & y_0 \\ 1 & x_0 & y_1
\end{vmatrix} = 0
\end{align}
$
> ### 서로 다른 세점$(x_0,y_0),(x_1,y_1),(x_2,y_2)$을 지나는 직선의 방정식은 행열식이 0이 되어야 함으로
>> ### $
det (A) \Rightarrow
\begin{align}
\begin{vmatrix}
1 & x & x^2 & y \\ 1 & x_0 & x_0^2 & y_0 \\ 1 & x_1 & x_1^2 & y_1 \\ 1 & x_2 & x_2^2 & y_2
\end{vmatrix} = 0
\end{align}
$

```python
T
T1 = sm.Matrix([[1,2],[3,4]])
T1
```

$\displaystyle \left[\begin{matrix}1 & 2\\3 & 4\end{matrix}\right]$

```python
# Minor matrix (소행열)
T.minor_submatrix(0,0)
T.minorEntry(0,1)
T.minorMatrix(0,1)
T1.minorMatrix(0,0)
T1.minorEntry(0,0)
```

$\displaystyle 4$

```python
# cofactor (여인수)
T.cofactor(0,0)
T1.cofactor(0,0)
# cofactor Matrix(여인수 행열)
T1.cofactorMatrix()
T.cofactorMatrix()
```

$\displaystyle \left[\begin{matrix}2 & -1 & 0\\-2 & 3 & 0\\0 & 0 & 4\end{matrix}\right]$

```python
# elementary column operation(기본행 연산)
# elementary row operation(기본열 연산)
# n->n+km : n=col1, m=col2
T1.elementary_row_op(op="n->n+km",row=1,k=-3,row2=0)
```

$\displaystyle \left[\begin{matrix}1 & 2\\0 & -2\end{matrix}\right]$

```python
# adjoint (수반행열) = (cofactor matrix).transpose()
# adjugate in sympy
# jugate (쌍으로 되어있는)
T1.adjugate()
T.adjugate()
```

$\displaystyle \left[\begin{matrix}2 & -2 & 0\\-1 & 3 & 0\\0 & 0 & 4\end{matrix}\right]$

```python
T1.echelon_form()

# rref = reduce row echelon form
ref, pivots = T1.rref()
# 행사다리(echelon) 꼴(form)
T.echelon_form()
T.rref()
```

    (Matrix([
     [1, 0, 0],
     [0, 1, 0],
     [0, 0, 1]]),
     (0, 1, 2))

# adjoin matrix
> ### cofactor
>> ### $
C_{ij} = (-1)^{i+j}M_{ij} \in \mathbb R
$

```python

```


```python
import numpy as np
import sympy as sy
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from matplotlib.animation import FuncAnimation
from mpl_toolkits.mplot3d import Axes3D
from scipy import linalg as la
```

    Bad key "text.kerning_factor" on line 4 in
    /home/anybody/apps/anaconda3/envs/py37astro/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test_patch.mplstyle.
    You probably need to get an updated matplotlibrc file from
    https://github.com/matplotlib/matplotlib/blob/v3.1.3/matplotlibrc.template
    or from the matplotlib source distribution
    /home/anybody/apps/anaconda3/envs/py37astro/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
      return f(*args, **kwds)

```python
# Problem 2 (standing orbits)

# load in the data and assign to understandable variables
orb = np.load('orbits.npz',)
merc_orb = orb['mercury']
ven_orb = orb['venus']
earth_orb = orb['earth']
mars_orb = orb['mars']

# set the figure for 3d Plots
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.set_xlim((-1.3,1.3))
ax.set_ylim((-1.3,1.3))
ax.set_zlim((-1.3,1.3))
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")

# plot the sun at the origin
ax.scatter3D(0,0,0, color='yellow', marker='o', lw=8, label='Sun')

# plot the standing location of mercury and its orbit
ax.plot3D([merc_orb[0,0]],[merc_orb[0,1]],[merc_orb[0,2]],color='gray',marker='.',lw=2)
ax.plot3D(merc_orb[:,0],merc_orb[:,1],merc_orb[:,2], 'gray', lw=.75, label='Mercury')

# plot the standing location of Venus and its orbit
ax.plot3D([ven_orb[0,0]],[ven_orb[0,1]],[ven_orb[0,2]],color='orange',marker='.',lw=2)
ax.plot3D(ven_orb[:,0],ven_orb[:,1],ven_orb[:,2], 'orange', lw=.75, label='Venus')

# plot the standing location of the Earth and its orbit
ax.plot3D([earth_orb[0,0]],[earth_orb[0,1]],[earth_orb[0,2]],color='blue',marker='.',lw=2)
ax.plot3D(earth_orb[:,0],earth_orb[:,1],earth_orb[:,2], 'b', lw=.75, label='Earth')

# plot the standing location of Mars and its orbit
ax.plot3D([mars_orb[0,0]],[mars_orb[0,1]],[mars_orb[0,2]],color='red',marker='.',lw=2)
ax.plot3D(mars_orb[:,0],mars_orb[:,1],mars_orb[:,2], 'r', lw=.75, label='Mars')

ax.legend(loc='upper right', prop={'size': 5})
ax.set_title("Orbits of the inner Planets")
plt.show()
```

```python

```


# Bonus Tutorial: Extending the Wilson-Cowan Model
**Week 2, Day 4: Dynamic Networks**

**By Neuromatch Academy**

__Content creators:__ Qinglong Gu, Songtin Li, Arvind Kumar, John Murray, Julijana Gjorgjieva

__Content reviewers:__ Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Spiros Chavlis, Michael Waskom

__Content editors:__

__Production editors:__ Siddharth Suresh

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives
In the previous tutorial, you became familiar the **Wilson-Cowan** rate model. Here we will dive into some deeper analyses of this model.

Bonus steps:

- Find and plot the **fixed points** of the Wilson-Cowan model.
- Investigate the stability of the Wilson-Cowan model by linearizing its dynamics and examining the **Jacobian matrix**.
- Learn how the Wilson-Cowan model can reach an oscillatory state.

Applications of Wilson-Cowan model:
- Visualize the behavior of an Inhibition-stabilized network.
- Simulate working memory using the Wilson-Cowan model.

Reference paper:

Wilson H and Cowan J (1972) Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal 12, doi: [10.1016/S0006-3495(72)86068-5](https://doi.org/10.1016/S0006-3495(72)86068-5)

---
# Setup

```python
# Imports

import matplotlib.pyplot as plt
import numpy as np
import scipy.optimize as opt       # root-finding algorithm
```

```python
# @title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
# @title Plotting Functions

def plot_FI_inverse(x, a, theta):
  f, ax = plt.subplots()
  ax.plot(x, F_inv(x, a=a, theta=theta))
  ax.set(xlabel="$x$", ylabel="$F^{-1}(x)$")

def plot_FI_EI(x, FI_exc, FI_inh):
  plt.figure()
  plt.plot(x, FI_exc, 'b', label='E population')
  plt.plot(x, FI_inh, 'r', label='I population')
  plt.legend(loc='lower right')
  plt.xlabel('x (a.u.)')
  plt.ylabel('F(x)')
  plt.show()

def my_test_plot(t, rE1, rI1, rE2, rI2):

  plt.figure()
  ax1 = plt.subplot(211)
  ax1.plot(pars['range_t'], rE1, 'b', label='E population')
  ax1.plot(pars['range_t'], rI1, 'r', label='I population')
  ax1.set_ylabel('Activity')
  ax1.legend(loc='best')

  ax2 = plt.subplot(212, sharex=ax1, sharey=ax1)
  ax2.plot(pars['range_t'], rE2, 'b', label='E population')
  ax2.plot(pars['range_t'], rI2, 'r', label='I population')
  ax2.set_xlabel('t (ms)')
  ax2.set_ylabel('Activity')
  ax2.legend(loc='best')

  plt.tight_layout()
  plt.show()

def plot_nullclines(Exc_null_rE, Exc_null_rI, Inh_null_rE, Inh_null_rI):

  plt.figure()
  plt.plot(Exc_null_rE, Exc_null_rI, 'b', label='E nullcline')
  plt.plot(Inh_null_rE, Inh_null_rI, 'r', label='I nullcline')
  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')
  plt.legend(loc='best')
  plt.show()

def my_plot_nullcline(pars):
  Exc_null_rE = np.linspace(-0.01, 0.96, 100)
  Exc_null_rI = get_E_nullcline(Exc_null_rE, **pars)
  Inh_null_rI = np.linspace(-.01, 0.8, 100)
  Inh_null_rE = get_I_nullcline(Inh_null_rI, **pars)

  plt.plot(Exc_null_rE, Exc_null_rI, 'b', label='E nullcline')
  plt.plot(Inh_null_rE, Inh_null_rI, 'r', label='I nullcline')
  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')
  plt.legend(loc='best')

def my_plot_vector(pars, my_n_skip=2, myscale=5):
  EI_grid = np.linspace(0., 1., 20)
  rE, rI = np.meshgrid(EI_grid, EI_grid)
  drEdt, drIdt = EIderivs(rE, rI, **pars)

  n_skip = my_n_skip

  plt.quiver(rE[::n_skip, ::n_skip], rI[::n_skip, ::n_skip],
             drEdt[::n_skip, ::n_skip], drIdt[::n_skip, ::n_skip],
             angles='xy', scale_units='xy', scale=myscale, facecolor='c')

  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')

def my_plot_trajectory(pars, mycolor, x_init, mylabel):
  pars = pars.copy()
  pars['rE_init'], pars['rI_init'] = x_init[0], x_init[1]
  rE_tj, rI_tj = simulate_wc(**pars)

  plt.plot(rE_tj, rI_tj, color=mycolor, label=mylabel)
  plt.plot(x_init[0], x_init[1], 'o', color=mycolor, ms=8)
  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')

def my_plot_trajectories(pars, dx, n, mylabel):
  """
  Solve for I along the E_grid from dE/dt = 0.

  Expects:
  pars    : Parameter dictionary
  dx      : increment of initial values
  n       : n*n trjectories
  mylabel : label for legend

  Returns:
    figure of trajectory
  """
  pars = pars.copy()
  for ie in range(n):
    for ii in range(n):
      pars['rE_init'], pars['rI_init'] = dx * ie, dx * ii
      rE_tj, rI_tj = simulate_wc(**pars)
      if (ie == n-1) & (ii == n-1):
          plt.plot(rE_tj, rI_tj, 'gray', alpha=0.8, label=mylabel)
      else:
          plt.plot(rE_tj, rI_tj, 'gray', alpha=0.8)

  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')

def plot_complete_analysis(pars):
  plt.figure(figsize=(7.7, 6.))

  # plot example trajectories
  my_plot_trajectories(pars, 0.2, 6,
                       'Sample trajectories \nfor different init. conditions')
  my_plot_trajectory(pars, 'orange', [0.6, 0.8],
                     'Sample trajectory for \nlow activity')
  my_plot_trajectory(pars, 'm', [0.6, 0.6],
                     'Sample trajectory for \nhigh activity')

  # plot nullclines
  my_plot_nullcline(pars)

  # plot vector field
  EI_grid = np.linspace(0., 1., 20)
  rE, rI = np.meshgrid(EI_grid, EI_grid)
  drEdt, drIdt = EIderivs(rE, rI, **pars)
  n_skip = 2
  plt.quiver(rE[::n_skip, ::n_skip], rI[::n_skip, ::n_skip],
             drEdt[::n_skip, ::n_skip], drIdt[::n_skip, ::n_skip],
             angles='xy', scale_units='xy', scale=5., facecolor='c')

  plt.legend(loc=[1.02, 0.57], handlelength=1)
  plt.show()

def plot_fp(x_fp, position=(0.02, 0.1), rotation=0):
  plt.plot(x_fp[0], x_fp[1], 'ko', ms=8)
  plt.text(x_fp[0] + position[0], x_fp[1] + position[1],
           f'Fixed Point1=\n({x_fp[0]:.3f}, {x_fp[1]:.3f})',
           horizontalalignment='center', verticalalignment='bottom',
           rotation=rotation)
```

```python
# @title Helper functions

def default_pars(**kwargs):
  pars = {}

  # Excitatory parameters
  pars['tau_E'] = 1.     # Timescale of the E population [ms]
  pars['a_E'] = 1.2      # Gain of the E population
  pars['theta_E'] = 2.8  # Threshold of the E population

  # Inhibitory parameters
  pars['tau_I'] = 2.0    # Timescale of the I population [ms]
  pars['a_I'] = 1.0      # Gain of the I population
  pars['theta_I'] = 4.0  # Threshold of the I population

  # Connection strength
  pars['wEE'] = 9.   # E to E
  pars['wEI'] = 4.   # I to E
  pars['wIE'] = 13.  # E to I
  pars['wII'] = 11.  # I to I

  # External input
  pars['I_ext_E'] = 0.
  pars['I_ext_I'] = 0.

  # simulation parameters
  pars['T'] = 50.        # Total duration of simulation [ms]
  pars['dt'] = .1        # Simulation time step [ms]
  pars['rE_init'] = 0.2  # Initial value of E
  pars['rI_init'] = 0.2  # Initial value of I

  # External parameters if any
  for k in kwargs:
      pars[k] = kwargs[k]

  # Vector of discretized time points [ms]
  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])

  return pars

def F(x, a, theta):
  """
  Population activation function, F-I curve

  Args:
    x     : the population input
    a     : the gain of the function
    theta : the threshold of the function

  Returns:
    f     : the population activation response f(x) for input x
  """

  # add the expression of f = F(x)
  f = (1 + np.exp(-a * (x - theta)))**-1 - (1 + np.exp(a * theta))**-1

  return f

def dF(x, a, theta):
  """
  Derivative of the population activation function.

  Args:
    x     : the population input
    a     : the gain of the function
    theta : the threshold of the function

  Returns:
    dFdx  :  Derivative of the population activation function.
  """

  dFdx = a * np.exp(-a * (x - theta)) * (1 + np.exp(-a * (x - theta)))**-2

  return dFdx

def F_inv(x, a, theta):
  """
  Args:
    x         : the population input
    a         : the gain of the function
    theta     : the threshold of the function

  Returns:
    F_inverse : value of the inverse function
  """

  # Calculate Finverse (ln(x) can be calculated as np.log(x))
  F_inverse = -1/a * np.log((x + (1 + np.exp(a * theta))**-1)**-1 - 1) + theta

  return F_inverse

def get_E_nullcline(rE, a_E, theta_E, wEE, wEI, I_ext_E, **other_pars):
  """
  Solve for rI along the rE from drE/dt = 0.

  Args:
    rE    : response of excitatory population
    a_E, theta_E, wEE, wEI, I_ext_E : Wilson-Cowan excitatory parameters
    Other parameters are ignored

  Returns:
    rI    : values of inhibitory population along the nullcline on the rE
  """
  # calculate rI for E nullclines on rI
  rI = 1 / wEI * (wEE * rE - F_inv(rE, a_E, theta_E) + I_ext_E)

  return rI

def get_I_nullcline(rI, a_I, theta_I, wIE, wII, I_ext_I, **other_pars):
  """
  Solve for E along the rI from dI/dt = 0.

  Args:
    rI    : response of inhibitory population
    a_I, theta_I, wIE, wII, I_ext_I : Wilson-Cowan inhibitory parameters
    Other parameters are ignored

  Returns:
    rE    : values of the excitatory population along the nullcline on the rI
  """
  # calculate rE for I nullclines on rI
  rE = 1 / wIE * (wII * rI + F_inv(rI, a_I, theta_I) - I_ext_I)

  return rE

def EIderivs(rE, rI,
             tau_E, a_E, theta_E, wEE, wEI, I_ext_E,
             tau_I, a_I, theta_I, wIE, wII, I_ext_I,
             **other_pars):
  """Time derivatives for E/I variables (dE/dt, dI/dt)."""

  # Compute the derivative of rE
  drEdt = (-rE + F(wEE * rE - wEI * rI + I_ext_E, a_E, theta_E)) / tau_E

  # Compute the derivative of rI
  drIdt = (-rI + F(wIE * rE - wII * rI + I_ext_I, a_I, theta_I)) / tau_I

  return drEdt, drIdt

def simulate_wc(tau_E, a_E, theta_E, tau_I, a_I, theta_I,
                wEE, wEI, wIE, wII, I_ext_E, I_ext_I,
                rE_init, rI_init, dt, range_t, **other_pars):
  """
  Simulate the Wilson-Cowan equations

  Args:
    Parameters of the Wilson-Cowan model

  Returns:
    rE, rI (arrays) : Activity of excitatory and inhibitory populations
  """
  # Initialize activity arrays
  Lt = range_t.size
  rE = np.append(rE_init, np.zeros(Lt - 1))
  rI = np.append(rI_init, np.zeros(Lt - 1))
  I_ext_E = I_ext_E * np.ones(Lt)
  I_ext_I = I_ext_I * np.ones(Lt)

  # Simulate the Wilson-Cowan equations
  for k in range(Lt - 1):

    # Calculate the derivative of the E population
    drE = dt / tau_E * (-rE[k] + F(wEE * rE[k] - wEI * rI[k] + I_ext_E[k],
                                   a_E, theta_E))

    # Calculate the derivative of the I population
    drI = dt / tau_I * (-rI[k] + F(wIE * rE[k] - wII * rI[k] + I_ext_I[k],
                                   a_I, theta_I))

    # Update using Euler's method
    rE[k + 1] = rE[k] + drE
    rI[k + 1] = rI[k] + drI

  return rE, rI
```

The helper functions included:

- Parameter dictionary: `default_pars(**kwargs)`. You can use:
  - `pars = default_pars()` to get all the parameters, and then you can execute `print(pars)` to check these parameters.
  - `pars = default_pars(T=T_sim, dt=time_step)` to set a different simulation time and time step
  - After `pars = default_pars()`, use `par['New_para'] = value` to add a new parameter with its value
  - Pass to functions that accept individual parameters with `func(**pars)`
- F-I curve: `F(x, a, theta)`
- Derivative of the F-I curve: `dF(x, a, theta)`
- Inverse of F-I curve: `F_inv`
- Nullcline calculations: `get_E_nullcline`, `get_I_nullcline`
- Derivatives of E/I variables: `EIderivs`
- Simulate the Wilson-Cowan model: `simulate_wc`

---
# Section 1: Fixed points, stability analysis, and limit cycles in the Wilson-Cowan model

*Correction to video: this is now the first part of the second bonus tutorial, not the last part of the second tutorial*

```python
# @title Video 1: Fixed points and their stability
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Pf4y1d7dx", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="jIx26iQ69ps", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

As in Tutorial 2, we will be looking at the Wilson-Cowan model, with coupled equations representing the dynamics of the excitatory or inhibitory population:

\begin{align}
\tau_E \frac{dr_E}{dt} &= -r_E + F_E(w_{EE}r_E -w_{EI}r_I + I^{\text{ext}}_E;a_E,\theta_E)\\
\tau_I \frac{dr_I}{dt} &= -r_I + F_I(w_{IE}r_E -w_{II}r_I + I^{\text{ext}}_I;a_I,\theta_I)    \qquad (1)
\end{align}

$r_E(t)$ represents the average activation (or firing rate) of the excitatory population at time $t$, and $r_I(t)$ the activation (or firing rate) of the inhibitory population. The parameters $\tau_E$ and $\tau_I$ control the timescales of the dynamics of each population. Connection strengths are given by: $w_{EE}$ (E $\rightarrow$ E), $w_{EI}$ (I $\rightarrow$ E), $w_{IE}$ (E $\rightarrow$ I), and $w_{II}$ (I $\rightarrow$ I). The terms $w_{EI}$ and $w_{IE}$ represent connections from inhibitory to excitatory population and vice versa, respectively. The transfer functions (or F-I curves) $F_E(x;a_E,\theta_E)$ and $F_I(x;a_I,\theta_I)$ can be different for the excitatory and the inhibitory populations.

## Section 1.1: Fixed Points of the E/I system

The intersection points of the two nullcline curves are the fixed points of the Wilson-Cowan model in Equation $(1)$.

In the next exercise, we will find the coordinate of all fixed points for a given set of parameters.

We'll make use of two functions, similar to ones we saw in Tutorial 1, which use a root-finding algorithm to find the fixed points of the system with Excitatory and Inhibitory populations.

```python
# @markdown Execute to visualize nullclines

# Set parameters
pars = default_pars()
Exc_null_rE = np.linspace(-0.01, 0.96, 100)
Inh_null_rI = np.linspace(-.01, 0.8, 100)

# Compute nullclines
Exc_null_rI = get_E_nullcline(Exc_null_rE, **pars)
Inh_null_rE = get_I_nullcline(Inh_null_rI, **pars)

plot_nullclines(Exc_null_rE, Exc_null_rI, Inh_null_rE, Inh_null_rI)
```

```python
# @markdown *Execute the cell to define `my_fp` and `check_fp`*

def my_fp(pars, rE_init, rI_init):
  """
  Use opt.root function to solve Equations (2)-(3) from initial values
  """

  tau_E, a_E, theta_E = pars['tau_E'], pars['a_E'], pars['theta_E']
  tau_I, a_I, theta_I = pars['tau_I'], pars['a_I'], pars['theta_I']
  wEE, wEI = pars['wEE'], pars['wEI']
  wIE, wII = pars['wIE'], pars['wII']
  I_ext_E, I_ext_I = pars['I_ext_E'], pars['I_ext_I']

  # define the right hand of wilson-cowan equations
  def my_WCr(x):

    rE, rI = x
    drEdt = (-rE + F(wEE * rE - wEI * rI + I_ext_E, a_E, theta_E)) / tau_E
    drIdt = (-rI + F(wIE * rE - wII * rI + I_ext_I, a_I, theta_I)) / tau_I
    y = np.array([drEdt, drIdt])

    return y

  x0 = np.array([rE_init, rI_init])
  x_fp = opt.root(my_WCr, x0).x

  return x_fp

def check_fp(pars, x_fp, mytol=1e-6):
  """
  Verify (drE/dt)^2 + (drI/dt)^2< mytol

  Args:
    pars    : Parameter dictionary
    fp      : value of fixed point
    mytol   : tolerance, default as 10^{-6}

  Returns :
    Whether it is a correct fixed point: True/False
  """

  drEdt, drIdt = EIderivs(x_fp[0], x_fp[1], **pars)

  return drEdt**2 + drIdt**2 < mytol

help(my_fp)
```

### Coding Exercise 1.1: Find the fixed points of the Wilson-Cowan model

From the above nullclines, we notice that the system features  three fixed points with the parameters we used. To find their coordinates, we need to choose proper initial value to give to the `opt.root` function inside of the function `my_fp` we just defined, since the algorithm can only find fixed points in the vicinity of the initial value.

In this exercise, you will use the function `my_fp` to find each of the fixed points by varying the initial values. Note that you can choose the values near the intersections of the nullclines as the initial values to calculate the fixed points.

```python
pars = default_pars()

######################################################################
# TODO: Provide initial values to calculate the fixed points
# Check if x_fp's are the correct with the function check_fp(x_fp)
# Hint: vary different initial values to find the correct fixed points
raise NotImplementedError('student exercise: find fixed points')
######################################################################

my_plot_nullcline(pars)

# Find the first fixed point
x_fp_1 = my_fp(pars, ..., ...)
if check_fp(pars, x_fp_1):
  plot_fp(x_fp_1)

# Find the second fixed point
x_fp_2 = my_fp(pars, ..., ...)
if check_fp(pars, x_fp_2):
  plot_fp(x_fp_2)

# Find the third fixed point
x_fp_3 = my_fp(pars, ..., ...)
if check_fp(pars, x_fp_3):
  plot_fp(x_fp_3)
```

```python
# to_remove solution
pars = default_pars()

with plt.xkcd():
  my_plot_nullcline(pars)

  # Find the first fixed point
  x_fp_1 = my_fp(pars, 0.1, 0.1)
  if check_fp(pars, x_fp_1):
    plot_fp(x_fp_1)

  # Find the second fixed point
  x_fp_2 = my_fp(pars, 0.3, 0.3)
  if check_fp(pars, x_fp_2):
    plot_fp(x_fp_2)

  # Find the third fixed point
  x_fp_3 = my_fp(pars, 0.8, 0.6)
  if check_fp(pars, x_fp_3):
    plot_fp(x_fp_3)
```

## Section 1.2: Stability of a fixed point and eigenvalues of the Jacobian Matrix

First, let's first rewrite the system $1$ as:

\begin{align}
&\frac{dr_E}{dt} = G_E(r_E,r_I)\\[0.5mm]
&\frac{dr_I}{dt} = G_I(r_E,r_I)
\end{align}

where

\begin{align}
&G_E(r_E,r_I) = \frac{1}{\tau_E} [-r_E + F_E(w_{EE}r_E -w_{EI}r_I + I^{\text{ext}}_E;a,\theta)]\\[1mm]
&G_I(r_E,r_I) = \frac{1}{\tau_I} [-r_I + F_I(w_{IE}r_E -w_{II}r_I + I^{\text{ext}}_I;a,\theta)]
\end{align}

By definition, $\displaystyle\frac{dr_E}{dt}=0$ and $\displaystyle\frac{dr_I}{dt}=0$ at each fixed point. Therefore, if the initial state is exactly at the fixed point, the state of the system will not change as time evolves.

However, if the initial state deviates slightly from the fixed point, there are two possibilities
the trajectory will be attracted back to the

1. The trajectory will be attracted back to the fixed point
2. The trajectory will diverge from the fixed point.

These two possibilities define the type of fixed point, i.e., stable or unstable. Similar to the 1D system studied in the previous tutorial, the stability of a fixed point $(r_E^*, r_I^*)$ can be determined by linearizing the dynamics of the system (can you figure out how?). The linearization will yield a matrix of first-order derivatives called the Jacobian matrix:

 \begin{equation}
   J=
  \left[ {\begin{array}{cc}
   \displaystyle{\frac{\partial}{\partial r_E}}G_E(r_E^*, r_I^*) & \displaystyle{\frac{\partial}{\partial r_I}}G_E(r_E^*, r_I^*)\\[1mm]
   \displaystyle\frac{\partial}{\partial r_E} G_I(r_E^*, r_I^*) & \displaystyle\frac{\partial}{\partial r_I}G_I(r_E^*, r_I^*) \\
  \end{array} } \right] \quad (7)
\end{equation}

The eigenvalues of the Jacobian matrix calculated at the fixed point will determine whether it is a stable or unstable fixed point.

We can now compute the derivatives needed to build the Jacobian matrix. Using the chain and product rules the derivatives for the excitatory population are given by:

\begin{align}
&\frac{\partial}{\partial r_E} G_E(r_E^*, r_I^*) = \frac{1}{\tau_E} [-1 + w_{EE} F_E'(w_{EE}r_E^* -w_{EI}r_I^* +  I^{\text{ext}}_E;\alpha_E, \theta_E)] \\[1mm]
&\frac{\partial}{\partial r_I} G_E(r_E^*, r_I^*)= \frac{1}{\tau_E} [-w_{EI} F_E'(w_{EE}r_E^* -w_{EI}r_I^* +  I^{\text{ext}}_E;\alpha_E, \theta_E)]
\end{align}

The same applies to the inhibitory population.

### Coding Exercise 1.2: Compute the Jacobian Matrix for the Wilson-Cowan model

Here, you can use `dF(x,a,theta)` defined in the `Helper functions` to calculate the derivative of the F-I curve.

```python
def get_eig_Jacobian(fp,
                     tau_E, a_E, theta_E, wEE, wEI, I_ext_E,
                     tau_I, a_I, theta_I, wIE, wII, I_ext_I, **other_pars):
  """Compute eigenvalues of the Wilson-Cowan Jacobian matrix at fixed point."""
  # Initialization
  rE, rI = fp
  J = np.zeros((2, 2))

  ###########################################################################
  # TODO for students: compute J and disable the error
  raise NotImplementedError("Student exercise: compute the Jacobian matrix")
  ###########################################################################
  # Compute the four elements of the Jacobian matrix
  J[0, 0] = ...
  J[0, 1] = ...
  J[1, 0] = ...
  J[1, 1] = ...

  # Compute and return the eigenvalues
  evals = np.linalg.eig(J)[0]
  return evals

# Compute eigenvalues of Jacobian
eig_1 = get_eig_Jacobian(x_fp_1, **pars)
eig_2 = get_eig_Jacobian(x_fp_2, **pars)
eig_3 = get_eig_Jacobian(x_fp_3, **pars)

print(eig_1, 'Stable point')
print(eig_2, 'Unstable point')
print(eig_3, 'Stable point')
```

```python
# to_remove solution
def get_eig_Jacobian(fp,
                     tau_E, a_E, theta_E, wEE, wEI, I_ext_E,
                     tau_I, a_I, theta_I, wIE, wII, I_ext_I, **other_pars):
  """Compute eigenvalues of the Wilson-Cowan Jacobian matrix at fixed point."""
  # Initialization
  rE, rI = fp
  J = np.zeros((2, 2))

  # Compute the four elements of the Jacobian matrix
  J[0, 0] = (-1 + wEE * dF(wEE * rE - wEI * rI + I_ext_E,
                           a_E, theta_E)) / tau_E

  J[0, 1] = (-wEI * dF(wEE * rE - wEI * rI + I_ext_E,
                       a_E, theta_E)) / tau_E

  J[1, 0] = (wIE * dF(wIE * rE - wII * rI + I_ext_I,
                      a_I, theta_I)) / tau_I

  J[1, 1] = (-1 - wII * dF(wIE * rE - wII * rI + I_ext_I,
                           a_I, theta_I)) / tau_I

  # Compute and return the eigenvalues
  evals = np.linalg.eig(J)[0]
  return evals

# Compute eigenvalues of Jacobian
eig_1 = get_eig_Jacobian(x_fp_1, **pars)
eig_2 = get_eig_Jacobian(x_fp_2, **pars)
eig_3 = get_eig_Jacobian(x_fp_3, **pars)

print(eig_1, 'Stable point')
print(eig_2, 'Unstable point')
print(eig_3, 'Stable point')
```

As is evident, the stable fixed points correspond to the negative eigenvalues, while unstable point corresponds to at least one positive eigenvalue.

The sign of the eigenvalues is determined by the connectivity (interaction) between excitatory and inhibitory populations.

Below we investigate the effect of $w_{EE}$ on the nullclines and the eigenvalues of the dynamical system.

\* _Critical change is referred to as **pitchfork bifurcation**_.

## Section 1.3: Effect of `wEE` on the nullclines and the eigenvalues

### Interactive Demo 1.3: Nullclines position in the phase plane changes with parameter values

How do the nullclines move for different values of the parameter $w_{EE}$? What does this mean for fixed points and system activity?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def plot_nullcline_diffwEE(wEE):
  """
    plot nullclines for different values of wEE
  """

  pars = default_pars(wEE=wEE)

  # plot the E, I nullclines
  Exc_null_rE = np.linspace(-0.01, .96, 100)
  Exc_null_rI = get_E_nullcline(Exc_null_rE, **pars)

  Inh_null_rI = np.linspace(-.01, .8, 100)
  Inh_null_rE = get_I_nullcline(Inh_null_rI, **pars)

  plt.figure(figsize=(12, 5.5))
  plt.subplot(121)
  plt.plot(Exc_null_rE, Exc_null_rI, 'b', label='E nullcline')
  plt.plot(Inh_null_rE, Inh_null_rI, 'r', label='I nullcline')
  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')
  plt.legend(loc='best')

  plt.subplot(222)
  pars['rE_init'], pars['rI_init'] = 0.2, 0.2
  rE, rI = simulate_wc(**pars)
  plt.plot(pars['range_t'], rE, 'b', label='E population', clip_on=False)
  plt.plot(pars['range_t'], rI, 'r', label='I population', clip_on=False)
  plt.ylabel('Activity')
  plt.legend(loc='best')
  plt.ylim(-0.05, 1.05)
  plt.title('E/I activity\nfor different initial conditions',
            fontweight='bold')

  plt.subplot(224)
  pars['rE_init'], pars['rI_init'] = 0.4, 0.1
  rE, rI = simulate_wc(**pars)
  plt.plot(pars['range_t'], rE, 'b', label='E population', clip_on=False)
  plt.plot(pars['range_t'], rI, 'r', label='I population', clip_on=False)
  plt.xlabel('t (ms)')
  plt.ylabel('Activity')
  plt.legend(loc='best')
  plt.ylim(-0.05, 1.05)

  plt.tight_layout()
  plt.show()

_ = widgets.interact(plot_nullcline_diffwEE, wEE=(6., 10., .01))
```

```python
# to_remove explanation

"""

- For low values of wEE there is only one fixed point and it is stable so initial
conditions do not matter and the system always converge to the only fixed point

- For high values of wEE we have three fixed points of which two are stable and
one is unstable (or saddle). Now it matters where the initial conditions are. If
the initial conditions are in the attractor region os the high activity fixed
point then the system will converge to that (the bottom example).
""";
```

We can also investigate the effect of different $w_{EI}$, $w_{IE}$, $w_{II}$, $\tau_{E}$, $\tau_{I}$, and $I_{E}^{\text{ext}}$ on the stability of fixed points. In addition, we can also consider the perturbation of the parameters of the gain curve $F(\cdot)$.

## Section 1.4: Limit cycle - Oscillations

For some values of interaction terms ($w_{EE}, w_{IE}, w_{EI}, w_{II}$), the eigenvalues can become complex. When at least one pair of eigenvalues is complex, oscillations arise.
The stability of oscillations is determined by the real part of the eigenvalues (+ve real part oscillations will grow, -ve real part oscillations will die out). The size of the complex part determines the frequency of oscillations.

For instance, if we use a different set of parameters, $w_{EE}=6.4$, $w_{EI}=4.8$, $w_{IE}=6.$, $w_{II}=1.2$, and $I_{E}^{\text{ext}}=0.8$, then we shall observe that the E and I population activity start to oscillate! Please execute the cell below to check the oscillatory behavior.

```python
# @title

# @markdown Make sure you execute this cell to see the oscillations!

pars = default_pars(T=100.)
pars['wEE'], pars['wEI'] = 6.4, 4.8
pars['wIE'], pars['wII'] = 6.0, 1.2
pars['I_ext_E'] = 0.8
pars['rE_init'], pars['rI_init'] = 0.25, 0.25

rE, rI = simulate_wc(**pars)
plt.figure(figsize=(8, 5.5))
plt.plot(pars['range_t'], rE, 'b', label=r'$r_E$')
plt.plot(pars['range_t'], rI, 'r', label=r'$r_I$')
plt.xlabel('t (ms)')
plt.ylabel('Activity')
plt.legend(loc='best')
plt.show()
```

We can also understand the oscillations of the population behavior using the phase plane. By plotting a set of trajectories with different initial states, we can see that these trajectories will move in a circle instead of converging to a fixed point. This circle is called "limit cycle" and shows the periodic oscillations of the $E$ and $I$ population behavior under some conditions.

Let's plot the phase plane using the previously defined functions.

```python
# @markdown Execute to visualize phase plane

pars = default_pars(T=100.)
pars['wEE'], pars['wEI'] = 6.4, 4.8
pars['wIE'], pars['wII'] = 6.0, 1.2
pars['I_ext_E'] = 0.8

plt.figure(figsize=(7, 5.5))
my_plot_nullcline(pars)

# Find the correct fixed point
x_fp_1 = my_fp(pars, 0.8, 0.8)
if check_fp(pars, x_fp_1):
  plot_fp(x_fp_1, position=(0, 0), rotation=40)

my_plot_trajectories(pars, 0.2, 3,
                      'Sample trajectories \nwith different initial values')

my_plot_vector(pars)

plt.legend(loc=[1.01, 0.7])
plt.xlim(-0.05, 1.01)
plt.ylim(-0.05, 0.65)
plt.show()
```

### Interactive Demo 1.4: Limit cycle and oscillations.

From the above examples, the change of model parameters changes the shape of the nullclines and, accordingly, the behavior of the $E$ and $I$ populations from steady fixed points to oscillations. However, the shape of the nullclines is unable to fully determine the behavior of the network. The vector field also matters. To demonstrate this, here, we will investigate the effect of time constants on the population behavior. By changing the inhibitory time constant $\tau_I$, the nullclines do not change, but the network behavior changes substantially from steady state to oscillations with different frequencies.

Such a dramatic change in the system behavior is referred to as a **bifurcation**.

\\
Please execute the code below to check this out.

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def time_constant_effect(tau_i=0.5):

  pars = default_pars(T=100.)
  pars['wEE'], pars['wEI'] = 6.4, 4.8
  pars['wIE'], pars['wII'] = 6.0, 1.2
  pars['I_ext_E'] = 0.8

  pars['tau_I'] = tau_i

  Exc_null_rE = np.linspace(0.0, .9, 100)
  Inh_null_rI = np.linspace(0.0, .6, 100)

  Exc_null_rI = get_E_nullcline(Exc_null_rE, **pars)
  Inh_null_rE = get_I_nullcline(Inh_null_rI, **pars)

  plt.figure(figsize=(12.5, 5.5))

  plt.subplot(121)  # nullclines
  plt.plot(Exc_null_rE, Exc_null_rI, 'b', label='E nullcline', zorder=2)
  plt.plot(Inh_null_rE, Inh_null_rI, 'r', label='I nullcline', zorder=2)
  plt.xlabel(r'$r_E$')
  plt.ylabel(r'$r_I$')

  # fixed point
  x_fp_1 = my_fp(pars, 0.5, 0.5)
  plt.plot(x_fp_1[0], x_fp_1[1], 'ko', zorder=2)

  eig_1 = get_eig_Jacobian(x_fp_1, **pars)

  # trajectories
  for ie in range(5):
    for ii in range(5):
      pars['rE_init'], pars['rI_init'] = 0.1 * ie, 0.1 * ii
      rE_tj, rI_tj = simulate_wc(**pars)
      plt.plot(rE_tj, rI_tj, 'k', alpha=0.3, zorder=1)

  # vector field
  EI_grid_E = np.linspace(0., 1.0, 20)
  EI_grid_I = np.linspace(0., 0.6, 20)
  rE, rI = np.meshgrid(EI_grid_E, EI_grid_I)
  drEdt, drIdt = EIderivs(rE, rI, **pars)
  n_skip = 2
  plt.quiver(rE[::n_skip, ::n_skip], rI[::n_skip, ::n_skip],
              drEdt[::n_skip, ::n_skip], drIdt[::n_skip, ::n_skip],
              angles='xy', scale_units='xy', scale=10, facecolor='c')
  plt.title(r'$\tau_I=$'+'%.1f ms' % tau_i)

  plt.subplot(122)  # sample E/I trajectories
  pars['rE_init'], pars['rI_init'] = 0.25, 0.25
  rE, rI = simulate_wc(**pars)
  plt.plot(pars['range_t'], rE, 'b', label=r'$r_E$')
  plt.plot(pars['range_t'], rI, 'r', label=r'$r_I$')
  plt.xlabel('t (ms)')
  plt.ylabel('Activity')
  plt.title(r'$\tau_I=$'+'%.1f ms' % tau_i)
  plt.legend(loc='best')
  plt.tight_layout()
  plt.show()

_ = widgets.interact(time_constant_effect, tau_i=(0.2, 3, .1))
```

Both $\tau_E$ and $\tau_I$ feature in the Jacobian of the two population network (eq 7). So here is seems that the by increasing $\tau_I$ the eigenvalues corresponding to the stable fixed point are becoming complex.

Intuitively, when $\tau_I$ is smaller, inhibitory activity changes faster than excitatory activity. As inhibition exceeds above a certain value, high inhibition inhibits excitatory population but that in turns means that inhibitory population gets smaller input (from the exc. connection). So inhibition decreases rapidly. But this means that excitation recovers -- and so on ...

---
# Section 2: Inhibition-stabilized network (ISN)

## Section 2.1: Inhibition-stabilized network

As described above, one can obtain the linear approximation around the fixed point as

 \begin{equation}
   \frac{d}{dr}  \vec{R}=
  \left[ {\begin{array}{cc}
   \displaystyle{\frac{\partial G_E}{\partial r_E}} & \displaystyle{\frac{\partial G_E}{\partial r_I}}\\[1mm]
   \displaystyle\frac{\partial G_I}{\partial r_E} & \displaystyle\frac{\partial G_I}{\partial r_I} \\
  \end{array} } \right] \vec{R},
\end{equation}

where $\vec{R} = [r_E, r_I]^{\rm T}$ is the vector of the E/I activity.

Let's direct our attention to the excitatory subpopulation which follows:

\begin{equation}
\frac{dr_E}{dt} = \frac{\partial G_E}{\partial r_E}\cdot r_E + \frac{\partial G_E}{\partial r_I} \cdot r_I
\end{equation}

Recall that, around fixed point $(r_E^*, r_I^*)$:

\begin{align}
&\frac{\partial}{\partial r_E}G_E(r_E^*, r_I^*) = \frac{1}{\tau_E} [-1 + w_{EE} F'_{E}(w_{EE}r_E^* -w_{EI}r_I^* + I^{\text{ext}}_E; \alpha_E, \theta_E)] \qquad (8)\\[1mm]
&\frac{\partial}{\partial r_I}G_E(r_E^*, r_I^*) = \frac{1}{\tau_E} [-w_{EI} F'_{E}(w_{EE}r_E^* -w_{EI}r_I^* + I^{\text{ext}}_E; \alpha_E, \theta_E)] \qquad (9)\\[1mm]
&\frac{\partial}{\partial r_E}G_I(r_E^*, r_I^*) = \frac{1}{\tau_I} [w_{IE} F'_{I}(w_{IE}r_E^* -w_{II}r_I^* + I^{\text{ext}}_I; \alpha_I, \theta_I)] \qquad (10)\\[1mm]
&\frac{\partial}{\partial r_I}G_I(r_E^*, r_I^*) = \frac{1}{\tau_I} [-1-w_{II} F'_{I}(w_{IE}r_E^* -w_{II}r_I^* + I^{\text{ext}}_I; \alpha_I, \theta_I)] \qquad (11)
\end{align}

From Equation. (8), it is clear that $\displaystyle{\frac{\partial G_E}{\partial r_I}}$ is negative since the $\displaystyle{\frac{dF}{dx}}$ is always positive. It can be understood by that the recurrent inhibition from the inhibitory activity ($I$) can reduce the excitatory ($E$) activity. However, as described above, $\displaystyle{\frac{\partial G_E}{\partial r_E}}$ has negative terms related to the "leak" effect, and positive term related to the recurrent excitation. Therefore, it leads to two different regimes:

- $\displaystyle{\frac{\partial}{\partial r_E}G_E(r_E^*, r_I^*)}<0$, **noninhibition-stabilized
network (non-ISN) regime**

- $\displaystyle{\frac{\partial}{\partial r_E}G_E(r_E^*, r_I^*)}>0$, **inhibition-stabilized
network (ISN) regime**

### Coding Exercise 2.1: Compute $\displaystyle{\frac{\partial G_E}{\partial r_E}}$

Implement the function to calculate the $\displaystyle{\frac{\partial G_E}{\partial r_E}}$ for the default parameters, and the parameters of the limit cycle case.

```python
def get_dGdE(fp, tau_E, a_E, theta_E, wEE, wEI, I_ext_E, **other_pars):
  """
  Compute dGdE

  Args:
    fp   : fixed point (E, I), array
    Other arguments are parameters of the Wilson-Cowan model

  Returns:
    J    : the 2x2 Jacobian matrix
  """
  rE, rI = fp

  ##########################################################################
  # TODO for students: compute dGdrE and disable the error
  raise NotImplementedError("Student excercise: compute the dG/dE, Eq. (13)")
  ##########################################################################
  # Calculate the J[0,0]
  dGdrE = ...

  return dGdrE

# Get fixed points
pars = default_pars()
x_fp_1 = my_fp(pars, 0.1, 0.1)
x_fp_2 = my_fp(pars, 0.3, 0.3)
x_fp_3 = my_fp(pars, 0.8, 0.6)

# Compute dGdE
dGdrE1 = get_dGdE(x_fp_1, **pars)
dGdrE2 = get_dGdE(x_fp_2, **pars)
dGdrE3 = get_dGdE(x_fp_3, **pars)

print(f'For the default case:')
print(f'dG/drE(fp1) = {dGdrE1:.3f}')
print(f'dG/drE(fp2) = {dGdrE2:.3f}')
print(f'dG/drE(fp3) = {dGdrE3:.3f}')

print('\n')

pars = default_pars(wEE=6.4, wEI=4.8, wIE=6.0, wII=1.2, I_ext_E=0.8)
x_fp_lc = my_fp(pars, 0.8, 0.8)

dGdrE_lc = get_dGdE(x_fp_lc, **pars)

print('For the limit cycle case:')
print(f'dG/drE(fp_lc) = {dGdrE_lc:.3f}')
```

```python
# to_remove solution
def get_dGdE(fp, tau_E, a_E, theta_E, wEE, wEI, I_ext_E, **other_pars):
  """
  Compute dGdE

  Args:
    fp   : fixed point (E, I), array
    Other arguments are parameters of the Wilson-Cowan model

  Returns:
    J    : the 2x2 Jacobian matrix
  """
  rE, rI = fp

  # Calculate the J[0,0]
  dGdrE = (-1 + wEE * dF(wEE * rE - wEI * rI + I_ext_E, a_E, theta_E)) / tau_E

  return dGdrE

# Get fixed points
pars = default_pars()
x_fp_1 = my_fp(pars, 0.1, 0.1)
x_fp_2 = my_fp(pars, 0.3, 0.3)
x_fp_3 = my_fp(pars, 0.8, 0.6)

# Compute dGdE
dGdrE1 = get_dGdE(x_fp_1, **pars)
dGdrE2 = get_dGdE(x_fp_2, **pars)
dGdrE3 = get_dGdE(x_fp_3, **pars)

print(f'For the default case:')
print(f'dG/drE(fp1) = {dGdrE1:.3f}')
print(f'dG/drE(fp2) = {dGdrE2:.3f}')
print(f'dG/drE(fp3) = {dGdrE3:.3f}')

print('\n')

pars = default_pars(wEE=6.4, wEI=4.8, wIE=6.0, wII=1.2, I_ext_E=0.8)
x_fp_lc = my_fp(pars, 0.8, 0.8)

dGdrE_lc = get_dGdE(x_fp_lc, **pars)

print('For the limit cycle case:')
print(f'dG/drE(fp_lc) = {dGdrE_lc:.3f}')
```

**SAMPLE OUTPUT**
```
For the default case:
dG/drE(fp1) = -0.650
dG/drE(fp2) = 1.519
dG/drE(fp3) = -0.706

For the limit cycle case:
dG/drE(fp_lc) = 0.837
```

## Section 2.2: Nullcline analysis of the ISN

Recall that the E nullcline follows

\begin{align}
r_E = F_E(w_{EE}r_E -w_{EI}r_I + I^{\text{ext}}_E;a_E,\theta_E).
\end{align}

That is, the firing rate $r_E$ can be a function of $r_I$. Let's take the derivative of $r_E$ over $r_I$, and obtain

\begin{align}
&\frac{dr_E}{dr_I} = F_E' \cdot (w_{EE}\frac{dr_E}{dr_I} -w_{EI}) \iff \\
&(1-F_E'w_{EE})\frac{dr_E}{dr_I} = -F_E' w_{EI} \iff \\
&\frac{dr_E}{dr_I} = \frac{F_E' w_{EI}}{F_E'w_{EE}-1}.
\end{align}

That is, in the phase plane `rI-rE`-plane, we can obtain the slope along the E nullcline as

$$ \frac{dr_I}{dr_E} = \frac{F_E'w_{EE}-1}{F_E' w_{EI}} \qquad (12) $$

Similarly, we can obtain the slope along the I nullcline as

$$ \frac{dr_I}{dr_E} = \frac{F_I'w_{IE}}{F_I' w_{II}+1} \qquad (13) $$

Then, we can find that $\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm I-nullcline} >0$ in Equation (13).

However, in Equation (12), the sign of $\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm E-nullcline}$ depends on the sign of $(F_E'w_{EE}-1)$. Note that, $(F_E'w_{EE}-1)$ is the same as what we show above (Equation (8)). Therefore, we can have the following results:

- $\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm E-nullcline}<0$, **noninhibition-stabilized
network (non-ISN) regime**

- $\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm E-nullcline}>0$, **inhibition-stabilized
network (ISN) regime**

In addition, it is important to point out the following two conclusions:

**Conclusion 1:** The stability of a fixed point can determine the relationship between the slopes Equations (12) and (13). As discussed above, the fixed point is stable when the Jacobian matrix ($J$ in Equation (7)) has two eigenvalues with a negative real part, which indicates a positive determinant of $J$, i.e., $\text{det}(J)>0$.

From the Jacobian matrix definition and from Equations (8-11), we can obtain:

$$   J=
  \left[ {\begin{array}{cc}
   \displaystyle{\frac{1}{\tau_E}(w_{EE}F_E'-1)} & \displaystyle{-\frac{1}{\tau_E}w_{EI}F_E'}\\[1mm]
   \displaystyle {\frac{1}{\tau_I}w_{IE}F_I'}& \displaystyle {\frac{1}{\tau_I}(-w_{II}F_I'-1)} \\
  \end{array} } \right] $$

Note that, if we let

\begin{equation}
  T=
  \left[
  {
    \begin{matrix}
      \displaystyle{\tau_E} & \displaystyle{0} \\
      \displaystyle 0& \displaystyle \tau_I
  \end{matrix}
  } \right],
  F=
  \left[
  {
    \begin{matrix}
      \displaystyle{F_E'} & \displaystyle{0} \\
      \displaystyle 0& \displaystyle F_I'
    \end{matrix}
  }
  \right]
  \text{, and }
  W=
  \left[
  {
    \begin{matrix}
      \displaystyle{w_{EE}} & \displaystyle{-w_{EI}} \\
      \displaystyle w_{IE}& \displaystyle -w_{II}
    \end{matrix}
  }
  \right]
\end{equation}

then, using matrix notation, $J=T^{-1}(F W - I)$ where $I$ is the identity matrix, i.e., $I = \begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}.$

Therefore, $\det{(J)}=\det{(T^{-1}(F W - I))}=(\det{(T^{-1})})(\det{(F W - I)}).$

Since $\det{(T^{-1})}>0$, as time constants are positive by definition, the sign of $\det{(J)}$ is the same as the sign of $\det{(F W - I)}$, and so

$$\det{(FW - I)} = (F_E' w_{EI})(F_I'w_{IE}) - (F_I' w_{II} + 1)(F_E'w_{EE} - 1) > 0.$$

Then, combining this with Equations (12) and (13), we can obtain

\begin{equation}
\frac{\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm I-nullcline}}{\Big{(} \displaystyle{\frac{dr_I}{dr_E}} \Big{)}_{\rm E-nullcline}} > 1.
\end{equation}

Therefore, at the stable fixed point, I nullcline has a steeper slope than the E nullcline.

**Conclusion 2:** Effect of adding input to the inhibitory population.

While adding the input $\delta I^{\rm ext}_I$ into the inhibitory population, we can find that the E nullcline (Equation (5)) stays the same, while the I nullcline has a pure left shift: the original I nullcline equation,

\begin{equation}
r_I = F_I(w_{IE}r_E-w_{II}r_I + I^{\text{ext}}_I ; \alpha_I, \theta_I)
\end{equation}

remains true if we take $I^{\text{ext}}_I \rightarrow I^{\text{ext}}_I +\delta I^{\rm ext}_I$ and $r_E\rightarrow r_E'=r_E-\frac{\delta I^{\rm ext}_I}{w_{IE}}$ to obtain

\begin{equation}
r_I = F_I(w_{IE}r_E'-w_{II}r_I + I^{\text{ext}}_I +\delta I^{\rm ext}_I; \alpha_I, \theta_I)
\end{equation}

Putting these points together, we obtain the phase plane pictures shown below. After adding input to the inhibitory population, it can be seen in the trajectories above and the phase plane below that, in an **ISN**, $r_I$ will increase first but then decay to the new fixed point in which both $r_I$ and $r_E$ are decreased compared to the original fixed point. However, by adding $\delta I^{\rm ext}_I$ into a **non-ISN**, $r_I$ will increase while $r_E$ will decrease.

### Interactive Demo 2.2: Nullclines of Example **ISN** and **non-ISN**

In this interactive widget, we inject excitatory ($I^{\text{ext}}_I>0$) or inhibitory ($I^{\text{ext}}_I<0$) drive into the inhibitory population when the system is at its equilibrium (with parameters $w_{EE}=6.4$, $w_{EI}=4.8$, $w_{IE}=6.$, $w_{II}=1.2$, $I_{E}^{\text{ext}}=0.8$, $\tau_I = 0.8$, and $I^{\text{ext}}_I=0$). How does the firing rate of the $I$ population changes with excitatory vs inhibitory drive into the inhibitory population?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

pars = default_pars(T=50., dt=0.1)
pars['wEE'], pars['wEI'] = 6.4, 4.8
pars['wIE'], pars['wII'] = 6.0, 1.2
pars['I_ext_E'] = 0.8
pars['tau_I'] = 0.8

def ISN_I_perturb(dI=0.1):
  Lt = len(pars['range_t'])
  pars['I_ext_I'] = np.zeros(Lt)
  pars['I_ext_I'][int(Lt / 2):] = dI

  pars['rE_init'], pars['rI_init'] = 0.6, 0.26
  rE, rI = simulate_wc(**pars)

  plt.figure(figsize=(8, 1.5))

  plt.plot(pars['range_t'], pars['I_ext_I'], 'k')
  plt.xlabel('t (ms)')
  plt.ylabel(r'$I_I^{\mathrm{ext}}$')
  plt.ylim(pars['I_ext_I'].min() - 0.01, pars['I_ext_I'].max() + 0.01)
  plt.show()

  plt.figure(figsize=(8, 4.5))
  plt.plot(pars['range_t'], rE, 'b', label=r'$r_E$')
  plt.plot(pars['range_t'], rE[int(Lt / 2) - 1] * np.ones(Lt), 'b--')
  plt.plot(pars['range_t'], rI, 'r', label=r'$r_I$')
  plt.plot(pars['range_t'], rI[int(Lt / 2) - 1] * np.ones(Lt), 'r--')
  plt.ylim(0, 0.8)
  plt.xlabel('t (ms)')
  plt.ylabel('Activity')
  plt.legend(loc='best')
  plt.show()

_ = widgets.interact(ISN_I_perturb, dI=(-0.2, 0.21, .05))
```

```python
# to_remove explanation

"""
Discussion:

Here we observe a paradoxical effect; if we inject excitatory current to the I
population, the r_I goes down, whereas when we inject inhibitory current, the r_I
increases. Recall that we inject a constant excitatory current to the E population,
which also drives, indirectly, the I population. When Iext>0, the r_I increases
but this drives E to a low state, which in turn leads to rI decrease. Whereas,
when Iext<0, the effect is negative on I population for a short amount of time,
which is sufficient to drive the E population to a high steady state, and then due
to E to I connections, the I population activity is increased.
""";
```

---
# Section 3: Fixed point and working memory

The input into the neurons measured in the experiment is often very noisy ([links](http://www.scholarpedia.org/article/Stochastic_dynamical_systems)). Here, the noisy synaptic input current is modeled as an Ornstein-Uhlenbeck (OU)process, which has been discussed several times in the previous tutorials.

```python
# @markdown Make sure you execute this cell to enable the function my_OU and plot the input current!

def my_OU(pars, sig, myseed=False):
  """
  Expects:
  pars       : parameter dictionary
  sig        : noise amplitute
  myseed     : random seed. int or boolean

  Returns:
  I          : Ornstein-Uhlenbeck input current
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tau_ou = pars['tau_ou']  # [ms]

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # Initialize
  noise = np.random.randn(Lt)
  I_ou = np.zeros(Lt)
  I_ou[0] = noise[0] * sig

  # generate OU
  for it in range(Lt-1):
      I_ou[it+1] = (I_ou[it]
                    + dt / tau_ou * (0. - I_ou[it])
                    + np.sqrt(2 * dt / tau_ou) * sig * noise[it + 1])
  return I_ou

pars = default_pars(T=50)
pars['tau_ou'] = 1.  # [ms]
sig_ou = 0.1
I_ou = my_OU(pars, sig=sig_ou, myseed=2020)
plt.figure(figsize=(8, 5.5))
plt.plot(pars['range_t'], I_ou, 'b')
plt.xlabel('Time (ms)')
plt.ylabel(r'$I_{\mathrm{OU}}$')
plt.show()
```

With the default parameters, the system fluctuates around a resting state with the noisy input.

```python
# @markdown Execute this cell to plot activity with noisy input current
pars = default_pars(T=100)
pars['tau_ou'] = 1.  # [ms]
sig_ou = 0.1
pars['I_ext_E'] = my_OU(pars, sig=sig_ou, myseed=20201)
pars['I_ext_I'] = my_OU(pars, sig=sig_ou, myseed=20202)

pars['rE_init'], pars['rI_init'] = 0.1, 0.1
rE, rI = simulate_wc(**pars)

plt.figure(figsize=(8, 5.5))
ax = plt.subplot(111)
ax.plot(pars['range_t'], rE, 'b', label='E population')
ax.plot(pars['range_t'], rI, 'r', label='I population')
ax.set_xlabel('t (ms)')
ax.set_ylabel('Activity')
ax.legend(loc='best')
plt.show()
```

## Interactive Demo 3: Short pulse induced persistent activity
Then, let's use a brief 10-ms positive current to the E population when the system is at its equilibrium. When this amplitude (SE below) is sufficiently large, a persistent activity is produced that outlasts the transient input. What is the firing rate of the persistent activity, and what is the critical input strength? Try to understand the phenomena from the above phase-plane analysis.

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def my_inject(pars, t_start, t_lag=10.):
  """
  Expects:
  pars       : parameter dictionary
  t_start    : pulse starts [ms]
  t_lag      : pulse lasts  [ms]

  Returns:
  I          : extra pulse time
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # Initialize
  I = np.zeros(Lt)

  # pulse timing
  N_start = int(t_start / dt)
  N_lag = int(t_lag / dt)
  I[N_start:N_start + N_lag] = 1.

  return I

pars = default_pars(T=100)
pars['tau_ou'] = 1.  # [ms]
sig_ou = 0.1
pars['I_ext_I'] = my_OU(pars, sig=sig_ou, myseed=2021)
pars['rE_init'], pars['rI_init'] = 0.1, 0.1

# pulse
I_pulse = my_inject(pars, t_start=20., t_lag=10.)
L_pulse = sum(I_pulse > 0.)

def WC_with_pulse(SE=0.):
  pars['I_ext_E'] = my_OU(pars, sig=sig_ou, myseed=2022)
  pars['I_ext_E'] += SE * I_pulse

  rE, rI = simulate_wc(**pars)

  plt.figure(figsize=(8, 5.5))
  ax = plt.subplot(111)
  ax.plot(pars['range_t'], rE, 'b', label='E population')
  ax.plot(pars['range_t'], rI, 'r', label='I population')

  ax.plot(pars['range_t'][I_pulse > 0.], 1.0*np.ones(L_pulse), 'r', lw=3.)
  ax.text(25, 1.05, 'stimulus on', horizontalalignment='center',
          verticalalignment='bottom')
  ax.set_ylim(-0.03, 1.2)
  ax.set_xlabel('t (ms)')
  ax.set_ylabel('Activity')
  ax.legend(loc='best')
  plt.show()

_ = widgets.interact(WC_with_pulse, SE=(0.0, 1.0, .05))
```

```python
# to_remove explanation

"""
Discussion:

When a system has more than one fixed points, depending on the input strength,
the network will settle in one of the fixed points. In this case, we have two
fixed points, one of the fixed points corresponds to high activity. So when input
drives the network to the high activity fixed points, the network activity will
remain there -- it is a stable fixed point. Because the network retains its
activity (persistent activity) even after the input has been removed, we can
take the persistent activity as working memory.
""";
```

Explore what happened when a second, brief current is applied to the inhibitory population.


##### Copyright 2020 The OpenFermion Developers

```python
#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Introduction to the bosonic operators

    View on QuantumLib

    Run in Google Colab

    View source on GitHub

    Download notebook

## Setup

Install the OpenFermion package:

```python
try:
    import openfermion
except ImportError:
    !pip install git+https://github.com/quantumlib/OpenFermion.git@master#egg=openfermion
```

## The BosonOperator

Bosonic systems, like Fermionic systems, are expressed using the bosonic creation and annihilation operators $b^\dagger_k$ and $b_k$ respectively. Unlike fermions, however, which satisfy the Pauli exclusion principle and thus are distinguished by the canonical fermionic anticommutation relations, the bosonic ladder operators instead satisfy a set of commutation relations:

$$
\begin{align}
& [b_i^\dagger, b_j^\dagger] = 0, ~~~ [b_i, b_j] = 0, ~~~ [b_i, b^\dagger_j] = \delta_{ij}
\end{align}
$$

Any weighted sums of products of these operators are represented with the `BosonOperator` data structure in OpenFermion. Similarly to when we introduced the `FermionOperator`, the following are examples of valid `BosonOperator`s:

$$
\begin{align}
& a_1 \nonumber \\
& 1.7 b^\dagger_3 \nonumber \\
&-1.7 \, b^\dagger_3 b_1 \nonumber \\
&(1 + 2i) \, b^\dagger_3 b^\dagger_4 b_1 b_9 \nonumber \\
&(1 + 2i) \, b^\dagger_3 b^\dagger_4 b_1 b_9 - 1.7 \, b^\dagger_3 b_1 \nonumber
\end{align}
$$

The `BosonOperator` class is contained in `ops/_boson_operators.py`. The `BosonOperator` is derived from the `SymbolicOperator`, the same class that derives the `FermionOperator`. As such, the details of the class implementation are identical - as in the fermion case, the class is implemented as hash table (python dictionary). The keys of the dictionary encode the strings of ladder operators and values of the dictionary store the coefficients - the strings are subsequently encoded as a tuple of 2-tuples which we refer to as the "terms tuple".

Each ladder operator is represented by a 2-tuple. The first element of the 2-tuple is an int indicating the quantum mode on which the ladder operator acts. The second element of the 2-tuple is Boole: 1 represents raising and 0 represents lowering. For instance, $b^\dagger_8$ is represented in a 2-tuple as $(8, 1)$. Note that indices start at 0 and the identity operator is an empty list.

$$
\begin{align}
I & \mapsto () \nonumber \\
b_1 & \mapsto ((1, 0),) \nonumber \\
b^\dagger_3 & \mapsto ((3, 1),) \nonumber \\
b^\dagger_3 b_1 & \mapsto ((3, 1), (1, 0)) \nonumber \\
b^\dagger_3 b^\dagger_4 b_1  b_9 & \mapsto ((3, 1), (4, 1),  (1, 0), (9, 0)) \nonumber
\end{align}
$$

Alternatively, the `BosonOperator` supports the string-based syntax introduced in the `FermionOperator`; in this case, the terms are separated by spaces, with the integer corresponding to the quantum mode the operator acts on, and `'^'` indicating the Hermitian conjugate:

$$
\begin{align}
I & \mapsto \textrm{""} \nonumber \\
b_1 & \mapsto \textrm{"1"} \nonumber \\
b^\dagger_3 & \mapsto \textrm{"3^"} \nonumber \\
b^\dagger_3 b_1 & \mapsto \textrm{"3^}\;\textrm{1"} \nonumber \\
b^\dagger_3 b^\dagger_4  b_1 b_9  & \mapsto \textrm{"3^}\;\textrm{4^}\;\textrm{1}\;\textrm{9"} \nonumber
\end{align}
$$

   Note that, unlike the `FermionOperator`, the bosonic creation operators of different indices commute. As a result, the `BosonOperator` automatically sorts groups of annihilation and creation operators in ascending order of the modes they act on.

Let's initialize our first term! We do it two different ways below.

```python
from openfermion.ops import BosonOperator

my_term = BosonOperator(((3, 1), (5, 0), (4, 1), (1, 0)))
print(my_term)

my_term = BosonOperator('3^ 5 4^ 1')
print(my_term)
```

Note the printed order differs from the code, since bosonic operators of different indices commute past each other.

The preferred way to specify the coefficient in openfermion is to provide an optional coefficient argument. If not provided, the coefficient defaults to 1. In the code below, the first method is preferred. The multiplication in the second method actually creates a copy of the term, which introduces some additional cost. All inplace operands (such as +=) modify classes whereas binary operands such as + create copies.

The additive and multiplicative identities can also be created:

* `BosonOperator(())` and `BosonOperator('')` initialises the identity (`BosonOperator.identity()`).
* `BosonOperator()` and `BosonOperator()` initialises the zero operator (`BosonOperator.zero()`).

```python
good_way_to_initialize = BosonOperator('3^ 1', -1.7)
print(good_way_to_initialize)

bad_way_to_initialize = -1.7 * BosonOperator('3^ 1')
print(bad_way_to_initialize)

identity = BosonOperator('')
print(identity == BosonOperator.identity())
print(identity)

zero_operator = BosonOperator()
print(zero_operator == BosonOperator.zero())
print(zero_operator)
```

Note that `BosonOperator` has only one attribute: .terms. This attribute is the dictionary which stores the term tuples.

```python
my_operator = BosonOperator('4^ 1^ 3 9', 1. + 2.j)
print(my_operator)
print(my_operator.terms)
```

### Methods and functions that act on the BosonOperator

There are various functions and methods that act on the `BosonOperator`; these include the ability to normal order, double check if the operator is Hermitian, and calculate the Hermitian conjugate.

```python
from openfermion.utils import hermitian_conjugated, is_hermitian
from openfermion.transforms import normal_ordered
```

`normal_ordered_boson` applies the bosonic commutation relations to write the operator using only normal-ordered terms; that is, that all creation operators are to the left of annihilation operators:

```python
H = BosonOperator('0 0^', 1. + 2.j)
H.is_normal_ordered()
```

```python
normal_ordered(BosonOperator('0 0^', 1. + 2.j))
```

We can also use a boson operator method to check if the operator conserves the particle number - that is, for each qumode, the number of annihilation operators equals the number of creation operators.

```python
H.is_boson_preserving()
```

```python
H = BosonOperator('0 0^ 1^ ', 1. + 2.j)
H.is_boson_preserving()
```

The Hermitian conjugated function returns the Hermitian conjugate of the operator, and its hermiticity can be checked using `is_hermitian`:

```python
is_hermitian(H)
```

```python
hermitian_conjugated(H)
```

```python
H = BosonOperator('0 1^', 1/2.)
H += BosonOperator('1 0^', 1/2.)
print(is_hermitian(H))
print(hermitian_conjugated(H))
```

## The QuadOperator

Using the bosonic ladder operators, it is common to define the canonical position and momentum operators $\hat{q}$ and $\hat{p}$:

$$ \hat{q}_i = \sqrt{\frac{\hbar}{2}}(\hat{b}_i+\hat{b}^\dagger_i), ~~~ \hat{p}_i = -i\sqrt{\frac{\hbar}{2}}(\hat{b}_i-\hat{b}^\dagger_i)$$

These operators are Hermitian, and are referred to as the phase space quadrature operators. They satisfy the canonical commutation relation

$$ [\hat{q}_i, \hat{p}_j] = \delta_{ij}i\hbar$$

where the value of $\hbar$ depends on convention, often taking values $\hbar=0.5$, $1$, or $2$.

In OpenFermion, the quadrature operators are represented by the `QuadOperator` class, and stored as a dictionary of tuples (as keys) and coefficients (as values). For example, the multi-mode quadrature operator $q_0 p_1 q_3$ is represented internally as `((0, 'q'), (1, 'p'), (3, 'q'))`. Alternatively, `QuadOperators` also support string input - using string input, the same operator is described by `'q0 p1 q3'`.

```python
from openfermion.ops import QuadOperator

H = QuadOperator('q0 p1 q3')
print(H)
print(H.terms)

H2 = QuadOperator('q3 p4', 3.17)
H2 -= 77. * H
print('')
print(H2)
```

Note that quadrature operators of different indices commute; as such, like the `BosonOperator`, by default we sort quadrature operators such that the operators acting on the lowest numbered mode appear to the left.

### Methods and functions that act on the QuadOperator

Like the `BosonOperator`, there are various functions and methods that act on the `QuadOperator`; these include the ability to normal order, double check if the operator is Hermitian, and calculate the Hermitian conjugate.

```python
from openfermion.utils import hermitian_conjugated, is_hermitian
```

`normal_ordered_quad` is an arbitrary convention chosen in OpenFermion that allows us to compare two quadrature operators that might be equivalent, but written in different forms. It is simply defined as a quadrature operator that has all of the position operators $\hat{q}$ to the left of the momentum operators $\hat{q}$. All quadrature operators can be placed in this 'normal form' by making use of the canonical commutation relation.

```python
H = QuadOperator('p0 q0', 1. + 2.j)
H.is_normal_ordered()
```

```python
normal_ordered(H)
```

By default, we assume the value $\hbar=1$ in the canonical commutation relation, but this can be modified by passing the `hbar` keyword argument to the function:

```python
normal_ordered(H, hbar=2)
```

We can also use a quad operator method to check if the operator is **Gaussian** - that is, all terms in the quad operator are of quadratic order or lower:

```python
H = QuadOperator('p0 q0', 1. + 2.j)
H.is_gaussian()
```

```python
H = QuadOperator('p0 q0 q1', 1. + 2.j)
H.is_gaussian()
```

The Hermitian conjugated function returns the Hermitian conjugate of the operator, and its hermiticity can be checked using `is_hermitian`:

```python
H = QuadOperator('p0 q1 p1', 1-2j)
hermitian_conjugated(H)
```

```python
H = QuadOperator('p0 q0', 1/2.)
H += QuadOperator('q0 p0', -1/2.)
print(is_hermitian(H))
print(hermitian_conjugated(H))
```

```python
H = QuadOperator('p0 q0', 1/2.)
H += QuadOperator('q0 p0', 1/2.)
print(is_hermitian(H))
print(hermitian_conjugated(H))
```

```python
hermitian_conjugated(H)
```

## Converting between quadrature operators and bosonic operators

Converting between bosonic ladder operators and quadrature operators is simple - we just apply the definition of the $\hat{q}$ and $\hat{p}$ operators in terms of $\hat{b}$ and $\hat{b}^\dagger$. Two functions are provided to do this automatically; `get_quad_operator` and `get_boson_operator`:

```python
from openfermion.transforms import get_boson_operator, get_quad_operator
```

```python
H = QuadOperator('p0 q0', 1/2.)
H += QuadOperator('q0 p0', 1/2.)
H
```

```python
get_boson_operator(H)
```

Note that, since these conversions are dependent on the value of $\hbar$ chosen, both accept a `hbar` keyword argument. As before, if not specified, the default value of $\hbar$ is `hbar=1`.

```python
H = BosonOperator('0 0^')
normal_ordered(get_quad_operator(H, hbar=0.5), hbar=0.5)
```

## Weyl quantization and symmetric ordering

We also provide support for the Weyl quantization - this maps a polynomial function of the form

$$f(q_0,\dots,q_{N-1},p_0\dots,p_{N-1})=q_0^{m_0}\cdots q_{N-1}^{m_{N-1}} p_0^{m_0}\cdots p_{N-1}^{m_{N-1}}$$

on the phase space to the corresponding combination of quadrature operators $\hat{q}$ and $\hat{p}$. To do so, we make use of the McCoy formula,

$$q^m p^n \rightarrow \frac{1}{2^n} \sum_{r=0}^{n} \binom{n}{r} q^r p^m q^{n-r}.$$

```python
from openfermion.transforms import weyl_polynomial_quantization, symmetric_ordering
```

For `weyl_polynomial_quantization`, the polynomial function in the phase space is provided in the form of a string,    where 'q' or 'p' is the phase space quadrature variable, the integer directly following is the mode it is with respect to, and '^2' is the polynomial power. If the power is not provided, it is assumed to be '^1'.

```python
weyl_polynomial_quantization('q0 p0')
```

```python
weyl_polynomial_quantization('q0^2 p0^3 q1^3')
```

McCoy's formula is also used to provide a function that returns the symmetric ordering of a `BosonOperator` or `QuadOperator`, $S(\hat{O})$. Note that $S(\hat{O})\neq \hat{O}$:

```python
symmetric_ordering(QuadOperator('q0 p0'))
```

Consider the symmetric ordering of the square of the bosonic number operator, $\hat{n} = \hat{b}^\dagger \hat{b}$:

```python
from openfermion.hamiltonians import number_operator
n2 = number_operator(1, parity=1) * number_operator(1, parity=1)
```

```python
n2
```

```python
Sn2 = symmetric_ordering(n2)
Sn2
```

We can use `normal_ordered_boson` to simplify this result:

```python
Sn2 = normal_ordered(Sn2)
Sn2
```

Therefore $S(\hat{n}) = \hat{b}^\dagger \hat{b}^\dagger \hat{b}\hat{b} + 2\hat{b}^\dagger \hat{b} + 0.5$. This is equivalent to $\hat{n}^2+\hat{n}+0.5$:

```python
Sn2 == normal_ordered(n2 + number_operator(1, parity=1) + 0.5*BosonOperator.identity())
```

## Bose-Hubbard Hamiltonian

In addition to the bosonic operators discussed above, we also provide Bosonic Hamiltonians that describe specific models. The Bose-Hubbard Hamiltonian over a discrete lattice or grid described by nodes $V=\{0,1,\dots,N-1\}$ is described by:

$$H = - t \sum_{\langle i, j \rangle} b_i^\dagger b_{j + 1}
         + \frac{U}{2} \sum_{k=1}^{N-1} b_k^\dagger b_k (b_k^\dagger b_k - 1)
         - \mu \sum_{k=1}^N b_k^\dagger b_k
         + V \sum_{\langle i, j \rangle} b_i^\dagger b_i b_j^\dagger b_j.$$

where

- The indices $\langle i, j \rangle$ run over pairs
  $i$ and $j$ of adjacenct nodes (nodes that are connected) in the grid
- $t$ is the tunneling amplitude
- $U$ is the on-site interaction potential
- $\mu$ is the chemical potential
- $V$ is the dipole or nearest-neighbour interaction potential

The Bose-Hubbard Hamiltonian function provided in OpenFermion models a Bose-Hubbard model on a two-dimensional grid, with dimensions given by `[x_dimension, y_dimension]`. It has the form

```python
bose_hubbard(x_dimension, y_dimension, tunneling, interaction,
                 chemical_potential=0., dipole=0., periodic=True)
```

where

- `x_dimension` (int): The width of the grid.
- `y_dimension` (int): The height of the grid.
- `tunneling` (float): The tunneling amplitude $t$.
- `interaction` (float): The attractive local interaction $U$.
- `chemical_potential` (float, optional): The chemical potential $\mu$ at each site. Default value is 0.
- `periodic` (bool, optional): If True, add periodic boundary conditions. Default is True.
- `dipole` (float): The attractive dipole interaction strength $V$.

Below is an example of a Bose-Hubbard Hamiltonian constructed in OpenFermion.

```python
from openfermion.hamiltonians import bose_hubbard, fermi_hubbard
bose_hubbard(2, 2, 1, 1)
```

## Sparse bosonic operators

Like the fermionic operators, OpenFermion contains the capability to represent bosonic operators as a sparse matrix (`sparse.csc_matrix`). However, as the fermionic operators can be represented as finite matrices, this is not the case of bosonic systems, as they inhabit a infinite-dimensional Fock space. Instead, a integer truncation value $N$ need to be provided - the returned sparse operator will be of size $N^{M}\times N^{M}$, where $M$ is the number of modes in the system, and acts on the truncated Fock basis $\{\left|{0}\right\rangle, \left|{1}\right\rangle, \dots, \left|{N-1}\right\rangle\}$.

```python
from openfermion.linalg import boson_operator_sparse
```

The function `boson_operator_sparse` acts on both `BosonOperator`s and `QuadOperator`s:

```python
H = boson_operator_sparse(BosonOperator('0^ 0'), 5)
```

```python
H.toarray()
```

```python
H = boson_operator_sparse(QuadOperator('q0'), 5, hbar=1)
H.toarray()
```


```javascript
%%javascript
$('#appmode-leave').hide();
$('#copy-binder-link').hide();
$('#visit-repo-link').hide();
```

Copyright **Paolo Raiteri**, January 2022

# Numerical solution of equilibrium problems

Let's consider a generic chemical reaction

\begin{equation}
\nu_aA + \nu_bB + \dots \leftrightharpoons \nu_xX + \nu_yY + \dots \tag{1}
\end{equation}

if the concentrations of the species are not the equilibrium ones, the reaction will progress (either forward or backwards) to reach the equilibrium conditions, and the free energy of the system will decrease.

An infinitesimal change of the system's free energy can then be written in terms of the chemical potential, $\mu$, of all the species involved in the reaction.

\begin{equation}
\mathrm{d}G = \sum_i \mu_i \mathrm{d}n_i \tag{2}
\end{equation}

Because the concentrations of reactants and products are coupled through the reaction equation, they cannot change independently, _i.e._ if any products are formed, some reactants must be consumed.
We can therefore define a unique parameter, $\xi$, as the _progress of the reaction_

\begin{equation}
\mathrm{d}\xi = \frac{\mathrm{d}n_i}{ \nu_i } \tag{3}
\end{equation}

where $\nu_i$ is the stoichiometric coefficient of the species $i$ taken as a positive number for products (formed) and as a negative number for reactants (consumed), and $\mathrm{d}n_i$ is an infinitesimal change in the concentration of species $i$.
Thus, the free energy change due to the progress of the reaction (in any direction) becomes

\begin{equation}
\mathrm{d}G = \sum_i \nu_i \mu_i \mathrm{d}\xi \tag{4}
\end{equation}

At equilibrium, the concentrations of all species are constant, $\mathrm{d}n_i=0$, and the reaction is not _progressing_ anymore, $\mathrm{d}\xi=0$, hence the free energy is not changing anymore.
On the other hand, if the system is out of equilibrium, there is a driving force that pulls the system towards equilibrium.
A force can always be obtained as the negative of the derivative of the energy with respect to some coordinate.
In this case, the driving force that pushes the reaction towards equilibrium can be immediately obtained from equation (4).

\begin{equation}
\frac{\mathrm{d}G}{\mathrm{d}\xi} = \sum_i \nu_i \mu_i = -force \tag{5}
\end{equation}

If we then substitute the definition of the chemical potential, $\mu=\mu^0+RT\ln x$, where $x$ is the molar fraction of the species, we obtain

\begin{eqnarray}
\frac{\mathrm{d} G}{\mathrm{d}\xi} &=& \sum_i\nu_i\mu_i\\
&=& \sum_i\nu_i \big[\mu_i^0 +RT\ln x_i\big]\\
&=& \sum_i\nu_i\mu_i^0 +RT\sum_i\nu_i\ln x_i\\
&=& \Delta_r G^0 +RT\sum_i\ln x_i^{\nu_i}\\
&=& \Delta_r G^0 +RT\ln \prod_i x_i^{\nu_i}\\
&=& \Delta_r G^0 +RT\ln Q
\end{eqnarray}

here $Q$ is the reaction quotient. By remembering the definition of equilibrium constant, $-RT\ln K_{eq}=\Delta_r G^0$ we get

\begin{equation}
force = -\frac{\mathrm{d} G}{\mathrm{d}\xi} = RT\ln K_{eq} - RT\ln Q = RT\ln\ \frac{K_{eq}}{Q} \tag{6}
\end{equation}

This equation provides us with a conceptual framework to solve *numerically* any equilibrium problem.
Because the above equations are differential, they are valid only for infinitesimal changes in the conditions.
Hence each a small change in the concentrations, $\nu_i\delta c$ the driving force may change significantly.

Therefore we need to set up an iterative procedure where we compute the driving force, change the concentrations, compute the driving force\dots until the equilibrium is reached.

Given the concentration of all species, we can compute the driving force using the equation above and alter the concentrations of all species using

\begin{equation}
[C]_i^1 = [C]_i^0 + \mathrm{d}n_i \tag{7}
\end{equation}

where the superscript indicates a the passage of an infinitesimal amount of time, and $\delta n_i$ is

\begin{equation}
\mathrm{d}n_i = \nu_i \times force \times \delta c  \tag{7}
\end{equation}

where $\nu_i$ are the stoichiometric coefficients of the species and $\mathrm{d}c$ is an adjustable parameter.
$\mathrm{d}c$ must be small enough to maintain the validity of the approximation abover but not too small to allow for a quick convergence of the procedure.

We cen then compute the new reaction quotient, the new driving force and update the concentrations again. This procedure can then be repeated a number of times, until the force is negligible, which indicates that the calculation has converged.

## Example #1: Dimerisation reaction
In order to elucidate how that procedure works we can take a model dimerisation reaction

\begin{equation}
2A \leftrightharpoons B
\end{equation}

whose equilibrium constant can be written as
\begin{equation}
K_{eq} = \frac{[B]}{[A]^2}
\end{equation}

We now want to calculate the equilibrium concentrations of $[A]_{eq}$ and $[B]_{eq}$ given their initial concentrations $[A]_{0}$ and $[B]_{0}$.
Although this is a simple problem that can be solved analytically, in this workshop we will learn how we can use an iterative method to numerically solve it.
We will use a relatively simple minimisation procedure that can be applied to a large number of problems, for which it is not possible or it is too complicated to get an analytic solution.

Imagine mixing the reagents and then to be able to monitor the concentration of all the species in the system at very short discrete time intervals (*timesteps*). What you will see is that the concentrations will change and tend to the equilibrium value. As you have learnt in first year, the reaction quotient, $Q$, can be used to decided which way to reaction will proceed, and that at equilibrium the reaction quotient is equal to the equilibrium constant. Hence, as we have discussed in class, the reaction quotient and the equilibrium constant can be use to define a *driving force* that pulls the system towards equilibrium.

This *driving force* can then be used in conjunction with an *ICE* table to numerically compute the equilibrium concentration of reactant and products.

The working principle of the minimisation procedure that we will employ is

1. compute the reaction quotient
\begin{equation}
Q = \frac{\mathrm{[B]}_0}{\mathrm{[A]}^2_0}
\end{equation}

2. compute the driving force

\begin{equation}
F = RT \ln\bigg[\frac{K_{eq}}{Q}\bigg]
\end{equation}

3. evolve the concentrations using the ICE table

|      | [A]        | [B]
| :---: | :--------: |:---------:
| *I*  | [A]$_0$    | [B]$_0$
| *C*  | $-2F\delta c$        | $+F\delta c$
| *E*  | [A]$_0$-2$F\delta c$ | [B]$_0$+$F\delta c$

4. Repeat until the solution does not change

Let's try to implement this

# Working notebooks

```python
import ipywidgets as ipw
import os

from IPython.display import Javascript
import glob as glob
import nbformat as nbf

label_layout = ipw.Layout(width='300px')

##########
pfiles = ['.protectedFiles.txt' , '../.protectedFiles.txt']
for fff in pfiles:
    if os.path.isfile(fff):
        with open(fff) as f:
            protectedFiles = f.read().splitlines()
##########
def launchNotebook(filename):
    text = "   var name_of_the_notebook = '" + filename + "'"
    vv="""
    var url = window.location.href.split('/')
    var newurl = url[0] + '//'
    for (var i = 1; i < url.length - 1; i++) {
        console.log(url[i], newurl)
        newurl += url[i] + '/'
    }
    newurl += name_of_the_notebook
    window.open(newurl)
    """
    text = text + vv
    display(Javascript(text))

def openNewNotebook(btn):
    if os.path.exists(notebookeName.value):
        print("Filename exists - Please select a different name")
        return

    nb = nbf.v4.new_notebook()
    text = """# Click 'Edit App' to start coding"""

    code = """\
# python packages
import pandas as pd # Dataframes and reading CSV files
import numpy as np # Numerical libraries
import matplotlib.pyplot as plt # Plotting library
from lmfit import Model # Least squares fitting library
from scipy.optimize import curve_fit # Alternative curve fittting library"""

    nb['cells'] = [nbf.v4.new_markdown_cell(text),
                   nbf.v4.new_code_cell(code)]

    if notebookeName.value in protectedFiles or notebookeName.value in listOfFiles:
        print("File already exists, select a different filename")
    else:
        with open(notebookeName.value, 'w') as f:
            nbf.write(nb, f)
        launchNotebook(notebookeName.value)

##########
listOfFiles = []
files = glob.glob1("./","*.ipynb")
for f in files:
    if f in protectedFiles:
        continue
    listOfFiles.append(f)

def dropdown_filesHandler(change):
    for i in range(0,len(listOfFiles)):
        if listOfFiles[i] == change.new:
            oldNotebookeName[0] = listOfFiles[i]

def createMenuFiles(data):
    option_list = ["Choose one"]
    option_list.extend(data)

    dropdown = ipw.Dropdown(description="", options=option_list, layout=ipw.Layout(width="300px"))
    dropdown.observe(dropdown_filesHandler, names='value')

    return dropdown

##########
oldNotebookeName = ["None"]
def openOldNotebook(btn):
    if oldNotebookeName[0] == "None":
        print("Please select a filename")
    elif oldNotebookeName[0] in protectedFiles:
        print("Please select a different filename")
    else:
        launchNotebook(oldNotebookeName[0])

##########
actions0 = []

notebookeName = ipw.Text("Empty.ipynb")

btn_new = ipw.Button(description="Create a new notebook", layout=label_layout)
btn_new.on_click(openNewNotebook)

btn_old = ipw.Button(description="Open an old notebook", layout=label_layout)
btn_old.on_click(openOldNotebook)

actions0.append(ipw.HBox([btn_new,notebookeName]))
actions0.append(ipw.HBox([btn_old,createMenuFiles(listOfFiles)]))
ipw.VBox(actions0)
```

## Critical thinking questions

1. Verify that the final equilibrium concentrations are independent of the starting conditions, provided that the the mass is conserved. Can you explain why is that?
2. What is the effect of changing the values of $\delta c$ or the number of cycles?
3. Is the value of RT important?
4. In some cases, it is possible that the concentration of one (or more) species becomes negative, this could be due to the fact that the starting conditions are too far from equilibrium and/or $\delta c$ is too large. Can you think of how this problem can be fixed automatically in the implementation of these algorithm.

## Assignment #1
### Part a:  Dissociation of a mono-protic acid
Calculate the final pH of a solution of a 0.15M of acetic acid

\begin{equation}
\mathrm{CH_3COOH \rightleftharpoons CH_3COO^{-} + H^+} \qquad pK_{a} = 3.74
\end{equation}

Remember that the equilibrium constant for the reation
\begin{equation}
AH \leftrightharpoons A^- + H^+
\end{equation}
is $K_{eq}=10^{-pK_a}$ and the reaction quotient
\begin{equation}
Q = \frac{[A^-][H^+]}{[HA]}
\end{equation}

Here below you can see the ICE table for this problem

|     | [HA] | [A$^-$] | [H$^+$]
| :---: |:---------: |:---------: |:---------:
| *I*  | [HA]$_0$ | [A$^-$]$_0$ | [H$^+$]$_0$
| *C*  | $-F_1\delta c$ | $+F_1\delta c$ | $+F_1\delta c$
| *E*  | [HA]$_0-F\delta c$ | [A$^-$]$_0+F_1\delta c$ | [H$^+$]$_0+F\delta c$

### Part b: External titration
Let's now imagine that the pH of the solution of the previous example is kept constant by external titration.

1. What do you have to change in your program to account for this?
2. What is the concentration of [AH] if the pH is kept to values of 2, 4 and 6?
3. Try fixing the pH at a value greater than 9, does the procedure work? (see critical thinking question #4)

## Assignment #2:  Coupled reactions
Let's now imagine we have two coupled reactions in the system, _e.g._ oxalic acid in pure water.

\begin{eqnarray}
H_2C_2O_4 + H_2O &\leftrightharpoons& H_3O^+ + HC_2O_4^- &\qquad\qquad& pK_{a1} = 1.25\\
HC_2O_4^- + H_2O &\leftrightharpoons& H_3O^+ + C_2O_4^= &\qquad\qquad& pK_{a2} = 4.14 \\
\end{eqnarray}

Determine
1. What is the pH of 1 L solution prepared with 18 g of oxalic acid
2. What would be the equilibrium concentrations of $HC_2O_4$, $HC_2O_4^-$ and $C_2O_4^=$ if the pH of the same solution is maintained at a value of 3 using an extermal titration.

For brevity, let's call oxalic acid "H$_2$A".

You now have to compute two reaction quotients

\begin{eqnarray}
Q_1 = \frac{[HA^-][H^+]}{[H_2A]} &\qquad\qquad& Q_2 = \frac{[A^=][H^+]}{[HA^-]}\\
\end{eqnarray}

which will give rise to two forces, $F_1$ and $F_2$,

\begin{equation}
F_1 = RT \ln\bigg[\frac{K_{a1}}{Q_1}\bigg]
\end{equation}
\begin{equation}
F_2 = RT \ln\bigg[\frac{K_{a2}}{Q_2}\bigg]
\end{equation}

You can then use a "double" ICE table to update the concentrations.

|      | [H$_2$A]   | [HA$^-$] | [A$^=$] | [H$^+$]
| :---: | :--------: |:---------: |:---------: |:---------:
| *I*  | [H$_2$A]$_0$ | [HA$^-$]$_0$ | [A$^=$]$_0$ | [H$^+$]$_0$
| *C1*  | $-F_1\delta c$ | $+F_1\delta c$ | $-$ | $+F_1\delta c$
| *C2*  | $-$ | $-F_2\delta c$ | $+F_2\delta c$  | $+F_2\delta c$
| *E*  | [H$_2$A]$_0-F_1\delta c$ | [HA$^-$]$_0+(F_1-F_2)\delta c$ | [A$^=$]$_0+F_2\delta c$ | [H$^+$]$_0+(F_1+F_2)\delta c$

# Lab report

### Introduction:
1. Explain the working principle of this numerical procedure to solve chemical equilibrium problems
    1. Start from the definition of progress of a reaction and the change in free energy using the chemical potential
    2. define the driving force
    3. explain the procedure using a "pseudo code" or a flow chart

### Results:
1. Implement the numerical procedure in python/excel and solve the two assignment problems below

2. Show graphs of the concentrations of all the species _vs_ the number of cycles to show that your calculation is converged. In some cases a logarithmic scale would make the graph clearer.

3. Verify your numeric solutions by solving the problem analytically (show all the calculations).

### Critical thinking questions
Instead of the conclusions section, answer the "critical thinking" questions stated above.

### References
No references are needed for this lab.