Analisando a Estabilidade de Equilíbrios

Nos modelos que analisamos neste curso, nem sempre lineares mas autônomos e de primeira ordem, desejamos poder determinar a estabilidade local de seus Equilíbrios.

Dizer que um equilíbrio $f_e$ é localmente estável significa dizer que para uma qualquer solução f(t) com condições iniciais localizadas até uma distância $\varepsilon>0$ do equilíbrio em questão, permanece a uma disância $\|f(t)-f_e\|\leq\varepsilon$ para todo $t\geq t_0$.

Para estudar a estabilidade, podemos lançar mão da análise de estabilidade linear. Mesmo que o sistema em que estamos interessados não seja linear, podemos fazer uma aproximação linear na vizinhança do equilíbrio, e a partir do sistema linear resultante. estudar a estabilidade na vizinhança do(s) equilíbrio(s) do sistema.

Uma vez obtido um sistema linear, é preciso examinar a sua matriz de coeficientes, ou matriz Jacobiana, $J$.

    \begin{align}

    \end{align}

 Seja $p = Tr(J) = A+D$ o traço da matriz $J$ e $q=det(J)=AD-BC$, o determinante de $J$. A representação dos equilíbrio  plano $(Det J, Tr J)$ nos ajuda a classificar os equilíbrios:

A natureza dos autovalores (principalmente o autovalor dominante, $\lambda_d$) da matriz jacobiana determinam o tipo do equilíbrio do sistema.

Autovalores reais:

Sinais opostos ($Re(\lambda_d)>0$): Equilíbrio em ponto de sela

Diferentes e com sinais negativo: Equilíbrio pontual estável

Diferentes e com sinais Positivos: Equilibrio pontual instável

Autovalores Complexos:

Soluções oscilatórias.

$Re(\lambda_d) \lt 0$: oscilações com amplitude decrescente (estável)

$Re(\lambda_d) \gt 0$: Oscilações com amplitude crescente (instável)

$Re(\lambda_d) = 0$: Oscilações com amplitudes constantes: ciclo limite (estável)

```python
var('A B C D x y')
J = jacobian([A*x+B*y, C*x+B*y],[x,y])
J
```

```python
J.trace()
```

```python
J.det()
```

```python
J.characteristic_polynomial()
```

## EDO Cúbica Unidimensional
Em sistema unidimensionais é ainda mais simples entender a natureza dos equilíbrios. Um instrumento gráfico útil olhar para a o gráfico da EDO. Por exemplo seja a seguinte EDO cúbica:

$$\frac{dx}{dt}=r x - x^3$$

Lembrando que por definição, equilíbrios são estados em que $dx/dt=0$, vemos que os equilíbrios deste sistema dinâmico são as raízes desta funcão polinomial de 3⁰ grau. Neste caso são 3:

```python
%display typeset
var('r')
f(x) = r*x - x^3
raízes = solve(f,x)
show("Raízes=",raízes)
F=plot(f(r=2),(x,-2,2))
P=points([(-sqrt(2),0),(sqrt(2),0),(0,0)], pointsize=30, color='red')
F+P
```

Você consegue dizer apenas olhando para o gráfico acima, quais equilíbrios são estáveis ou instáveis?

E no caso do gráfico abaixo quando $r=0$?

```python
@interact
def cubica(r=(-2,2,1)):
    P = plot(f(r=r), (x,-1.5,1.5))
    raízes = solve(f,x)
    show("Raízes=",raízes)
    P2=points([(-sqrt(r),0),(sqrt(r),0),(0,0)], pointsize=30, color='red')
    show(P+P2)
```

    Interactive function <function cubica at 0x7fcaa7032040> with 1 widget
      r: IntSlider(value=0, description='r'…

No caso desta cúbica quando $r=0$, temos apenas um equilíbrio em $0$. Para obter a estabilidade do equilíbrio a chave é olhar para o sinal da derivada de $f(x)=rx-x^3$, com $r=0$ $f'(x)=-3x^2$. Podemos verificar na solução numérica de x(t)

```python
desolve_rk4(-x^3,x,ics=[0,.2],ivar='t', end_points=500, output='slope_field')
```

## Bifurcações

Em vários modelos alterações no valor de certos valores podem alterar o número de equilíbros ou mesmo a natureza destes. Retomando um modelo que já vimos:

$$\frac{dx}{dt}= c \left( x-\frac{x^3}{3}+A \right) \equiv f(x)$$

Já vimos que este modelo apresenta 3 equilíbrios: 2 estáveis separados por um instável, quando $A=0$, por exemplo.

```python
var('A')
f(x) = x-x^3/3+A
g = f(A=-1)
plot(g,(x,-2,2))
```

É fácil ver que para valores de $|A| \gt A_1$ dois dos equilíbrios desaparecem. Na figura abaixo, mostramos os pontos $A_1=-A_2$ em que os dois equilíbrios coalescem e como a partir destes limiares temos apenas um equilíbrio real e dois complexos iguais com sinais opostos na parte imaginária.

Esta mudança no comportamento qualitativo do modelo é chamada de bifurcação, e o parâmetro $A$, é o parêmetro de bifurcação. Como esta curva  de bifurcação parece dobrar-se sobre si mesma, este tipo de bifurcações que vemos é chamado de bifurcação em "dobra" ou "fold bifurcation".

```python
import numpy as np
pts = []
for v in np.linspace(-2,2,60):
    g = f(A=v)
    xvals = solve(g,x)
    pts.extend([[v,n(i.rhs().real_part())] for i in xvals])

show(points(pts),axes_labels=['$A$','$x$'],gridlines=True)
```

```python
sols = solve(f(A=0.7),x)
[s.rhs().n() for s in sols]
```

## Bi-Estabilidade e Histerese

Quando um sistema apresenta dois equilíbrios estáveis é denominado bi-estável. Esta bi-estabilidade  nos leva a um outro fenômeno que pode ser observado no modelo cúbico acima, denominado de Histerese. Histerese é quando a transição entre os equilíbrios ocorre em diferentes valores do parâmetro de bifurcação dependendo se nos aproximamos pela esquerda ou pela direita.
Bifurcações em um Parâmetro

Vamos examinar brevemente exemplos simples de alguns tipos de bifurcações locais mais comuns:
### Bifurcação em dobra

A equação diferencial mais simples que apresenta bifurcação em dobra é a seguinte:

$$\frac{dx}{dt}=f(x)=r+x^2$$

Se variarmos $r$ a parábola resultante ganhará ou perderá dois pontos de interseção com o eixo $x$. Logo a bifurcação ocorrerá quando $r=0$. Vejamos:

```python
def drawbif(func,l,u):
    ipts = []
    epts = []
    for v in np.linspace(l,u,100):
        g = func(r=v)
        xvals = solve(g,x)
        dg = diff(g,x)
        est = [(e.rhs().real_part(),dg(x=e.rhs().real_part())) for e in xvals]
        for e in est:
            if e[1]<0:
                epts.append((v,n(e[0])))
            elif e[1] == 0:
                pass
            else:
                ipts.append((v,n(e[0])))
    eplot = points(epts, color="blue")
    iplot = points(ipts, color="red")
    show(eplot+iplot,axes_labels=['$r$','$x$'],gridlines=True, xmax=2)
var('r')
f(x) = r - x^2
drawbif(f,-2,2)
```

### Bifurcação Transcrítica

Este tipo de bifurcação tem a seguinte equação como seu exemplo mais simples:

$$\frac{dx}{dt}= r x -x^2$$

à media que r varia, a função se transforma como podemos ver abaixo:

```python
f(x) = r*x-x^2
p1 = plot(f(r=-1),(-2,2), color='red', legend_label='$r=-1$')
p2 = plot(f(r=0),(-2,2), color='green', legend_label='$r=0$')
p3 = plot(f(r=1),(-2,2), color='blue', legend_label='$r=1$')
show(p1+p2+p3)
```

Para a maior parte dos valores de $r$ o sistema tem dois equilíbrios, mas quando $r=0$ o sistema perde um dos equilíbrios e troca de estabilidade. Vejamos:

```python
def drawbif(func,l,u):
    pts = []
    ipts = []
    epts = []
    for v in np.linspace(l,u,100):
        g = func(r=v)
#         dg = diff(g,x)
        xvals = solve(g,x)
        pts.extend([[v,n(i.rhs().real_part())] for i in xvals])
#         for e in pts:
#             if e[1]<0:
#                 epts.append((v,n(e[0])))
#             elif e[1] == 0:
#                 pass
#             else:
#                 ipts.append((v,n(e[0])))
#     eplot = points(epts, color="blue")
#     iplot = points(ipts, color="red")

    show(points(pts),axes_labels=['$r$','$x$'],gridlines=True)
var('r')
f(x) = r*x + x^2
drawbif(f,-2,2)
```

### Bifurcação Pitchfork (Garfo)

Um exemplo simples de bifurcação Pitchfork é:

$$\frac{dx}{dt}= r*x -x^3$$

Aqui novamente temos uma bifurcação quando o parâmtro de bifurcação $r$ cruza o valor de $r=0$. Na equação acima, quando $r$ torna-se positivo, dois novos equilíbrios aparecem:

```python
f(x) = r*x-x^3
p1 = plot(f(r=-5),(-2,2), color='red', legend_label='$r=-5$')
p2 = plot(f(r=0),(-2,2), color='green', legend_label='$r=0$')
p3 = plot(f(r=5),(-2,2), color='blue', legend_label='$r=5$')
show(p1+p2+p3)
```

```python
f(x) = r*x - x^3
drawbif(f,-2,2)
```

se mudarmos o sinal do termo cúbico obtemos uma bifurcação pitchfork subcrítica:

```python
f(x) = r*x + x^3
drawbif(f,-2,2)
```

### Bifurcação de Hopf

A [bifurcação de Hopf](https://en.wikipedia.org/wiki/Hopf_bifurcation) é importante por dar orígem a equilíbrios periódicos denominados limit cycles. Bifurcações de Hopf ocorrem em sistemas com autovalores complexos e se dá quando o sinal da parte real destes, muda. Numa bifurcação de Hopf, não surgem novos equilíbrios, mas surge uma solução periódica em torno do equilíbrio original.

Um exemplo simples de sistema apresentando uma bifurcação de Hopf é o modelo de Selkov.
Modelo de Selkov

Este modelo representa uma reação bioquímica importtante: a glicólise. Nesta reação a glicose é quebrada, dentro da célula, em piruvato.

$$\frac{dx}{dt} = -x + ay + x^2 y, $$

$$\frac{dy}{dt} = b - a y - x^2 y$$

[Selkov, E., Model of glycolytic oscillations, (1968) Eur. J. Biochem. 4, 79-86](http://onlinelibrary.wiley.com/doi/10.1111/j.1432-1033.1968.tb00175.x/full).

Vamos começar com uma simples simulação deste sistema:

```python
var('x y a b')
a = 0.1
b=0.5
P = desolve_system_rk4([-x+a*y+x**2*y, b-a*y-x**2*y], [x,y],ics=[0,.5,.5],ivar='t', end_points=[0,40])
X=[ [i,j] for i,j,k in P]
Y=[ [i,k] for i,j,k in P]
XP=list_plot(X, legend_label='$x$')
YP=list_plot(Y, legend_label='$y$', color='red')
XP+YP
```

```python
vf = plot_vector_field([-x+a*y+x**2*y, b-a*y-x**2*y],(x,0,3),(y,0,3),axes_labels=[r'$x$',r'$y$'])
traj = list_plot([ [j,k] for i,j,k in P], plotjoined=True, legend_label='$x$')
vf+traj
```

```python
var('x y a b')
J= jacobian([-x+a*y+x**2*y, b-a*y-x**2*y], [x,y])
J
```

```python
J.eigenvalues()
```

```python
J(a=0.1,b=0.4).eigenvalues()
```

```python
var('x y a b')
@interact
def _(a=input_box(default=0.1),
      b=(.4,(0.1,1)),
      fin = input_box(default=-x+a*y+x**2*y), gin=input_box(default=b-a*y-x**2*y),
      xmin=input_box(default=0), xmax=input_box(default=1.5),
      ymin=input_box(default=0), ymax=input_box(default=2.8),
      x_start=(0,(0,3,.1)), y_start=(2,(0,3,.1)), error=(0.5,(0,1)),
      t_length=(100,(0, 200)) , num_of_points = (1500,(5,2000)),
      algorithm = selector([
         ("rkf45" , "runga-kutta-felhberg (4,5)"),
         ("rk2" , "embedded runga-kutta (2,3)"),
         ("rk4" , "4th order classical runga-kutta"),
         ("rk8pd" , 'runga-kutta prince-dormand (8,9)'),
         ("rk2imp" , "implicit 2nd order runga-kutta at gaussian points"),
         ("rk4imp" , "implicit 4th order runga-kutta at gaussian points"),
         ("bsimp" , "implicit burlisch-stoer (requires jacobian)"),
         ("gear1" , "M=1 implicit gear"),
         ("gear2" , "M=2 implicit gear")
      ])):
    f(x,y)=fin(a=a, b=b)
    g(x,y)=gin(a=a, b=b)
    show(f)
    show(g)
    ff = f._fast_float_(*f.args())
    gg = g._fast_float_(*g.args())

    #solve
    path = []
    err = error
    xerr = 0
    for yerr in [-err, 0, +err]:
      T=ode_solver()
      T.algorithm=algorithm
      T.function = lambda t, yp: [ff(yp[0],yp[1]), gg(yp[0],yp[1])]
      T.jacobian = lambda t, yp: [[diff(fun,dval)(yp[0],yp[1]) for dval in [x,y]] for fun in [f,g]]
      T.ode_solve(y_0=[x_start + xerr, y_start + yerr],t_span=[0,t_length],num_points=num_of_points)
      path.append(line([p[1] for p in T.solution]))

    #plot
    xnull = implicit_plot(ff,(x,0.,1.5),(y,0.,2.5), color='green',axes_labels=['$x$','$y$'], legend_label='nuliclina de $x$',show_legend=True)
    ynull = implicit_plot(gg,(x,0.,1.5),(y,0.,2.5), color='red',axes_labels=['$x$','$y$'], legend_label='nuliclina de $y$',show_legend=True)
    vector_field = plot_vector_field( (f,g), (x,xmin,xmax), (y,ymin,ymax), plot_points=25 )
    starting_point = point([x_start, y_start], pointsize=50)
    show(vector_field + starting_point + sum(path)+xnull+ynull, aspect_ratio=1, figsize=[8,9],gridlines=True)
```

    Interactive function <function _ at 0x7fcaa6c93dc0> with 14 widgets
      a: EvalText(value='0.100000000000000', d…

Para encontrar o equilíbrio pontual do sistema, basta resolver o sistema:

```python
var('x y a b')
xdot(x,y) = -x+a*y+x**2*y
ydot(x,y) = -x^2*y - a*y + b
solve([xdot, ydot], [x,y])
```

Se nós substituirmos o valores de equilíbrio do sistema na matriz jacobiana:

```python
Jeq = J(x=b, y=b/(b^2+a))
Jeq
```

Podemos encontrar os autovalores do sistema:

```python
evs = Jeq(a=0.1).eigenvalues()
evs
```

Uma vez fixado o parâmetro $a$, os autovalores passam a ser polinômios em $b$:

```python
pol(b) = evs[0](a=0.1)
show(pol)
find_root(real_part(pol),0.3,0.5)
```

```python
pt = point((find_root(real_part(pol),0.3,0.5),0), color='red', pointsize=30)
P = plot(real_part(pol),(b,0.3,0.5))
P+pt
```

```python
import numpy as np
a=0.1

def drawbif_selkov(l,u):
    def selkov(t,S):
        x,y = S
        return [xdotf(x,y), ydotf(x,y)]

    T.function = selkov
    pts = []
    for b in np.linspace(l,u,500):
        xdotf=xdot(b=b, a=0.1)._fast_float_(*xdot.args())
        ydotf=ydot(b=b, a=0.1)._fast_float_(*ydot.args())
        T.ode_solve(y_0=[0,2],t_span=[0,50],num_points=500)
        sol = T.solution
        pts += [(b,j[1]) for i,j in sol[-100:] if abs(j[0]-b) <1e-3]

    show(points(pts),axes_labels=['$b$','$y$'],gridlines=True)

T = ode_solver()
T.algorithm="rk8pd"

drawbif_selkov(0.2,0.5)
```

```python

```

Planilhas Sage/Aula 3 - Análise de Estabilidade Linear.ipynb


# [ARE 212] Discussion Section - Python 02

- [Ethan's materials](https://github.com/ligonteaching/ARE212_Materials)
- [My github](https://github.com/bk-econ/share)

### Source Material

These notes are the fruits of arduous labor of others.  My contributions are minimual, but please expect to find some code (and even explanation) errors (and assume them all to be mine). If you find any mistakes or have questions, please [let me know](mailto:benjaminkrause@berkeley.edu).

The primary sources of these notes are:
- Ethan and in particular his EEP 153 Notes
- [Computational and Inferential Thinking: The Foundations of Data Science](https://www.inferentialthinking.com/chapters/intro.html) which is the textbook for UC Berkeley's [Data 8: The Foundations of Data Science](http://data8.org/) course.  All of the notes, readings, labs, and assignments are fully available online as well.  For instance, here is [Spring 2020](http://data8.org/sp20/).
- [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/)

### Follow-up from questions posed last week

- Discussions: See [Ethan's full response](https://bcourses.berkeley.edu/courses/1487913/discussion_topics/5737368)
    - *When is discussion participation "due"?*
        - Initial response by Wednesday
        - Further contributiioins can be later ni the week (including over the weekend)
- Final format: See [Ethan's full response](https://bcourses.berkeley.edu/courses/1487913/discussion_topics/5737448)
    - *Do we need to be able to code in Python?*
        - "My intention, rather, is to provide you with some simple working notebooks you can play with, and follow along with to achieve instructional ends which aren't principally focused on on new language acquisition"
        - "I'll make sure the final doesn't require python mastery."
    - *What should we expect in the final?*
        - "My intention is for the topics raised in the discussions to serve as likely jumping off points for the final.
        - ". . . students are responsible for the content of the lecture, and should be familiar with the specific readings I reference (not necessarily all the papers mentioned in the syllabus)."

## Introduction to `Python`

### Learning Goals Today

1.  Review [classical_regression](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/classical_regression.ipynb)
2.  Review [weighted_regression](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/weighted_regression.ipynb)
3.  Review [random_variables0](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/random_variables0.ipynb)

NB: If needed, review the introduction from last week at [ARE 212 Discussion Section - Python 01](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Discussion_Section/%5BARE%20212%5D%20Discussion%20Section%20-%20Python%2001.ipynb) or from [my github](https://github.com/bk-econ/share).

##### Open a `jupyter` notebook on `datahub.berkeley.edu`

1. Open the website [URL to Interact](https://url-to-interact.herokuapp.com/)
2. For `1. Choose your desired hub:` select `datahub.berkeley.edu`
3. Either navigate to [my github](https://github.com/bk-econ/share) page and select the `URL` of the correct section or simply copy it from right here: https://github.com/bk-econ/share/blob/master/%5BARE%20212%5D%20Discussion%20Section%20-%20Python%2001.ipynb
4. Paste the `URL` you copied in the previous step into `2. Paste the GitHub URL for your file or folder in the box below.`
5. Click `Convert to interact link!`
6. Copy the new `URL` generated in field `4. Your interact link URL will appear in the box below.`
7. Paste the new `URL` in any web browser and get to it!

### 1. [classical_regression](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/classical_regression.ipynb)

or see [classical regression on github](https://github.com/ligonteaching/ARE212_Materials/blob/master/classical_regression.ipynb)

#### Classical regression in =python=

The fact that $X$ and $u$ are “independent” variables means that
if we want to compute a “classical” regression we’d do it
something like this:

##### Define independent random variables

```python
# 1.1. ORIGINAL CODE
%matplotlib inline
import numpy as np
from scipy.stats import multivariate_normal

k = 2 # Number of observables

mu = [0]*k
Sigma=[[1,0.5],
       [0.5,2]]

X = multivariate_normal(mu,Sigma)

u = multivariate_normal(cov=0.2)
```

 *notes*:
- `%matplotlib inline`
    - `%` is used to call [magic functions](https://ipython.readthedocs.io/en/stable/interactive/tutorial.html#magics-explained) in `Python`.
    - [`matplotlib`](https://matplotlib.org/) - "comprehensive library for creating static, animated, and interactive visualizations in Python"
        - Here is a more comprehensive treatment of [Using matplotlib in jupyter notebooks](https://medium.com/@1522933668924/using-matplotlib-in-jupyter-notebooks-comparing-methods-and-some-tips-python-c38e85b40ba1)
        - Originally designed to enable MATLAB-style plotting via gnuplot from the [IPython](https://plotly.com/python/ipython-vs-python/) command line
        - Noteable because it plays well with many operating systems and graphics packages
    - `inline` simply means that any plots we create are going to appear in our notebook inline

```python
# check the available backends for matplotlib with --list
%matplotlib --list
```

    Available matplotlib backends: ['tk', 'gtk', 'gtk3', 'wx', 'qt4', 'qt5', 'qt', 'osx', 'nbagg', 'notebook', 'agg', 'svg', 'pdf', 'ps', 'inline', 'ipympl', 'widget']

 *notes*:
- `import numpy as np`
    - [numpy](https://numpy.org/): mathematical functions, generally used to create arrays
- `from scipy.stats import multivariate_normal`
    - `import X` vs `from X import x` (Explanation from this [Stack Exchange](https://stackoverflow.com/questions/9439480/from-import-vs-import))
        - `import X`: "Imports the module X, and creates a reference to that module in the current namespace. Then you need to define completed module path to access a particular attribute or method from inside the module (e.g.: X.name or X.attribute)"
        - `from X import x`: "Imports the module X, and creates references to all public objects defined by that module in the current namespace . . . after you've run this statement, you can simply use a plain (unqualified) name to refer to things defined in module X."
            - "But X itself is not defined, so X.name doesn't work."
            - "And if name was already defined, it is replaced by the new version."
            - "And if name in X is changed to point to some other object, your module won’t notice."
    - `scipy.stats`
        - [scipy](https://www.scipy.org/): scientific functions
        - `scipy.stats` is the set of statistical functions within `scipy`

```python
import scipy
# scipy?
#  A scientific computing package for Python . . .
## imports all the functions from the NumPy namespace, and in addition provides:
## Subpackages: Using any of these subpackages requires an explicit import.
## For example,``import scipy.cluster``.
```

```python
# scipy.stats?
# This module contains a large number of probability distributions
## as well as a growing library of statistical functions.
```

```python
# scipy.stats.multivariate_normal?
# A multivariate normal random variable.
### multivariate_normal(mean=None,
                    ### cov=1,
                    ### allow_singular=False,
                    ### seed=None,
                    ### )
```

```python
# 1.1. CODE ABOVE WITH COMMENTS AND PRINTING

# run magic function matplotlib to have visualizations inline in your notebook
%matplotlib inline
# see notes below

# import the NumPy module and call it "np"
import numpy as np # see notes below

# import just the multivariate_normal function from the "stats" functions of the SciPy module
from scipy.stats import multivariate_normal # see notes below

# k gets 2
k = 2 # Number of observables; R::k <- 2
print("k =", k)

# mu gets the 1x2 zero vector
mu = [0]*k
print("mu =", mu)

# sigma gets the following matrix (note the use double brackets)
Sigma=[[1,0.5],
       [0.5,2]] # R::matrix(c(1, .5, .5, 2), nrow=2, byrow=TRUE)
print("Sigma =", Sigma)

# X gets a multivariate normal random variable with mean = mu and cov = Sigma
X = multivariate_normal(mu,Sigma)
print("X =", X)

# u gets a multivariate normal random variable with mean = 0 and cov = .2
u = multivariate_normal(cov=0.2) #
print("u =", u)
```

    k = 2
    mu = [0, 0]
    Sigma = [[1, 0.5], [0.5, 2]]
    X = <scipy.stats._multivariate.multivariate_normal_frozen object at 0x7f7126071e48>
    u = <scipy.stats._multivariate.multivariate_normal_frozen object at 0x7f71447f94a8>

##### Construct Sample

To construct a sample of observables $(y,X)$ we just use the regression equation,
     plus an assumption about the value of $\beta$:

```python
# 1.2. ORIGINAL CODE

beta = [1/2,1]

N=1000 # Sample size

# Now: Transform r.v. X into a sample
X = X.rvs(N)

y = X@beta + u.rvs(N) # Note use of @ operator for matrix multiplication
```

 *notes*:
- `X.rvs(size = n)` yields a random sample of size n from the object X
- `@` represents matrix multiplication (as already stated above); R::%*%

```python
# 1.2. CODE ABOVE WITH COMMENTS AND PRINTING

# beta gets a list of 1/2 and 1
beta = [1/2,1] # R::list(1/2, 1)
print("beta =", beta)

# N gets 1000
N=1000 # Sample size
print("N =", N)

# Now: Transform random variable X into a sample of N draws
X = X.rvs(N)
print("X =", X)

# y gets (R::X %*% beta + U)  where U is a sample of N draws from u
y = X@beta + u.rvs(N) # Note use of @ operator for matrix multiplication
print("y =", y)
```

NB: Once you redefine your random variables using X.rvs(N), you must rerun the earlier code to run this cell again.

##### Turn to estimation

So, we now have data on *realizations* $(y,X)$ which take the
     Now forget that we know $\beta$ and let’s estimate it, using
     OLS.  As a numerical matter it’s better to avoid explicitly
     inverting the $(X^T X)$ matrix; instead we can solve the “normal”
     equations.

##### Numerical solution

```python
# 1.3. ORIGINAL CODE

from scipy.linalg import inv, sqrtm

b = np.linalg.solve(X.T@X,X.T@y)

e = y - X@b

vb = e.var()*inv(X.T@X)

print(b,end='\n\n')
print(sqrtm(vb))
```

    [0.54382267 0.99563508]

    [[ 0.01469681 -0.00224287]
     [-0.00224287  0.01048138]]

 *notes*:
- `X.T` produces the transpose of X; R::t(X)
- `np.linalg.solve(a, b)` "Solve a linear matrix equation, or system of linear scalar equations.  Computes the "exact" solution, `x`, of the well-determined, i.e., full rank, linear matrix equation `ax = b`."
- `print(x, end = '\n\n')` by default `end = '\n'` in the `print()` function, which starts a new line.  However, when `'\n\n'` is instead entered, your notebook will also skip a line between the printed information and the next output.  To avoid a new line being started, you can instead enter `' '`.
- `inv( )` is a function that calculates the inverse of a matrix
- `sqrtm( )` is the matrix square root function.

```python
# 1.3. CODE ABOVE WITH COMMENTS AND PRINTING

# import both the 'inv' and 'sqrtm' functions from the 'linalg' functions of the SciPy module
from scipy.linalg import inv, sqrtm

# solve for b in the equation X.T@X@b = X.T@y
b = np.linalg.solve(X.T@X,X.T@y)
print("b =", b)

# e gets R:: y - X %*% b
e = y - X@b
print("e =", e)

# An additional line simply to show what e.var() generates
evar = e.var()
print("evar =", evar)

# vb gets R:: var(e) * solve(t(X) %*% X)
vb = e.var()*inv(X.T@X)
print("vb =", vb)

# print b with an additional line after the results
print(b,end='\n\n')

# print the squareroot of the matrix vb
print(sqrtm(vb))
```

    b = [0.54382267 0.99563508]
    e = [ 2.63877568e-01 -4.05948961e-02 -3.06400265e-01  4.01680446e-01
      1.32770511e-01 -4.91137434e-01 -6.43808036e-01 -3.35490947e-01
     -4.38356243e-02  7.25539821e-01  8.12474308e-01  3.83179854e-01
      2.25032587e-01 -1.26480234e-01  4.25695337e-01  2.63712309e-01
      3.45894159e-01 -5.14810790e-02 -8.08247922e-01  1.54172789e-01
     -6.52153653e-01  6.55711474e-01  1.29100456e+00 -2.61035673e-01
     -6.66830892e-01  3.98032579e-01  5.82063626e-01 -9.20252685e-02
     -2.56009817e-01  1.05596879e-01  2.70580565e-01  2.06066938e-01
     -8.02597895e-01 -5.74631278e-02  4.76205281e-01 -2.60124223e-03
      4.09489955e-01 -2.64064129e-02  1.43363529e-01 -2.24597394e-01
      1.36951455e-01  1.17553824e-04  4.87108681e-01  9.39790221e-02
     -3.04193558e-01 -4.76847915e-01 -3.15018925e-01 -3.41043320e-01
      2.25761356e-02 -1.25327862e-01  1.70522404e-01 -3.46605233e-02
     -1.73270663e-01 -3.79867121e-01  1.00433291e+00 -2.19827376e-01
      1.70366034e-01  1.23304479e-01  4.50127754e-01  2.10111493e-01
      2.66430510e-02  1.27821991e-01  4.22613478e-01 -9.27713549e-02
     -1.72474907e-01 -4.05545111e-01  2.34916949e-01 -1.98781351e-01
      6.55753335e-02 -5.86366602e-01 -9.72017587e-01  1.19117952e-01
     -1.19907635e-01  4.24340928e-01 -1.13763317e-01 -2.32589384e-01
      5.72716999e-01 -1.28240074e-01 -6.69552563e-02 -4.44120017e-01
     -4.78815357e-01  3.33132573e-02 -1.78811760e-01 -2.87303904e-02
     -1.02555844e-02  2.45371390e-01 -9.76682574e-01 -1.44572171e-01
      7.06089721e-01  6.86646181e-02 -7.78989925e-02  1.09033172e-01
      1.86996989e-01 -3.44752411e-01 -8.04921346e-02  1.97666585e-01
     -7.74684879e-01 -5.54977454e-01 -2.65531996e-02 -2.51218741e-01
     -3.94259884e-02 -4.22922764e-01 -2.57757395e-01 -9.22054503e-02
     -2.28667699e-01 -9.08896883e-01 -4.90238650e-01 -9.31857940e-02
      1.65744043e-01  4.30140509e-01 -1.78973404e-01  1.66161152e-01
     -1.08156121e-01 -9.91300613e-02  2.17982403e-01 -3.51539245e-01
     -1.89785684e-01  1.04278876e+00 -3.04252427e-01 -4.56261392e-01
      2.39838632e-01 -8.85005680e-01 -1.65961078e-01 -3.68271229e-01
      1.82724033e-01  8.46092801e-01  1.94256962e-01 -7.46985426e-01
      4.55527820e-03 -4.31912179e-02  5.33204359e-01 -1.51512649e-01
     -1.82257608e-01 -7.00661888e-03  1.32656414e-01 -4.71202591e-01
     -7.46198167e-01 -4.55493085e-02 -4.31832064e-01  2.96884660e-01
     -3.42442432e-01 -3.75943564e-01  7.43496774e-01 -2.95100623e-03
      6.95697747e-01  1.03204197e-01  1.45695893e-01 -1.62550977e-02
      2.84926571e-02  2.88370493e-01  3.90802840e-01  8.08272110e-01
      3.23074405e-01  2.70858373e-01 -3.06631805e-02  4.53121898e-01
      1.00576573e+00  2.66310877e-02  3.39597007e-01  9.07505798e-04
      3.31636210e-01 -1.15504729e-01  7.77107939e-01  5.72437315e-01
     -1.67228105e-01  4.39012025e-01 -8.72886847e-02 -1.51944275e-01
     -3.99927386e-01  5.41186836e-01 -1.51517831e-01 -1.96119765e-01
     -7.40280065e-02  1.18646272e+00 -1.74010709e-01  6.49296925e-01
     -2.94631182e-01 -9.20458066e-02 -1.66675998e-01  6.67325027e-02
      4.70488918e-01 -8.07110162e-02  7.79354405e-02  1.29269289e-01
      2.99228943e-01  3.97911923e-01 -3.72696030e-01  2.96984873e-01
      7.76319136e-01 -3.72280118e-01 -3.89649558e-01 -2.10018373e-02
     -8.15702776e-02  1.28469046e+00  5.85078361e-03 -6.88925463e-02
      3.57402229e-01 -8.51969038e-01 -4.04606292e-01 -6.01638502e-01
     -2.28475518e-01 -1.27361662e-01  7.89749570e-01  1.16387719e-01
      6.49948844e-02 -4.45451345e-01  3.05541742e-01  5.32385259e-01
      3.40364302e-01  3.66757042e-01  1.35097168e-01 -2.96340048e-01
      9.05146884e-01 -1.79471470e-01  8.95699726e-01 -6.67609652e-02
      2.11982521e-01 -5.28056386e-03  7.40656805e-02 -5.98822059e-01
      5.37766325e-02 -2.29255203e-01  8.32870992e-01 -2.03354233e-01
      1.67250124e-01 -2.55504713e-01 -2.38160744e-01  4.75876655e-01
     -4.05150853e-01  4.06316512e-01  7.46971102e-01  2.53537877e-01
     -1.34274708e-01  3.47676909e-01 -2.59678622e-01  3.52874244e-01
     -6.02037625e-01  2.17891297e-01 -2.29784370e-01  2.83988829e-01
      1.52376160e-02 -4.79281779e-01  7.18912172e-02  4.77150954e-01
      1.41344921e-01  3.16865985e-01  8.95965838e-01  5.33562565e-01
      4.55485001e-01  3.56934258e-01  1.26431022e-01  3.27373978e-01
     -4.41788735e-01 -7.34061455e-01  6.92023719e-01  7.17840282e-02
     -2.76670975e-01  5.66586012e-01 -4.23789241e-01  8.23551615e-01
     -1.72983914e-01  1.57242371e-01  4.82079125e-01 -1.97095277e-01
     -5.35010092e-01 -5.08079517e-02  2.25927939e-01  2.43222042e-01
      4.49542964e-01 -2.71460486e-01  3.25139814e-01 -3.07966446e-01
      3.99709830e-02  1.22019371e-01  8.29039892e-02 -5.48007028e-01
      6.73955152e-02 -1.40149092e-01  5.95001249e-01 -2.66207668e-01
     -5.31370666e-01  1.16268038e-01 -2.48030420e-01 -5.67867803e-01
      6.08700169e-02 -7.23434645e-01  2.22733021e-01 -6.26456048e-01
     -1.44658762e-01  5.54840113e-01  4.52856250e-01  1.61871554e-01
     -1.56471402e-01  5.44629139e-01 -2.88045816e-01 -2.04388207e-01
      8.70316486e-03 -3.40615023e-01 -9.92715504e-02 -4.57816246e-01
     -1.60196644e-01  3.75549029e-01 -1.53314777e-01 -6.89419420e-01
      4.59223484e-01 -5.48720677e-01 -8.17532403e-01  1.05212030e-01
     -7.99130819e-01 -1.25423777e-01  1.19499762e+00  2.54800500e-01
     -2.23812217e-01 -8.30787811e-01 -5.47170077e-02 -6.04464109e-01
     -2.16128442e-01  3.01196436e-01  2.92753181e-01 -3.34553355e-01
     -1.15581887e-02  3.36014702e-01 -9.28278947e-02  1.22383188e-01
      4.23953870e-01 -8.90009517e-02 -2.05954889e-01 -8.21029529e-01
      8.48646123e-01 -3.47546046e-01 -7.46837911e-01 -4.08917719e-01
     -6.92065983e-01  2.88890783e-01 -8.51164783e-01 -4.77213788e-01
     -5.19248536e-01 -4.53262707e-01 -1.65780287e-01  7.08587850e-01
      8.96951843e-02 -3.12579887e-02 -8.01425747e-01  7.31305121e-01
      5.51550677e-01 -1.80798910e-01  6.20757466e-01 -5.26420353e-01
     -4.29287077e-01  5.54782433e-01  6.47418181e-01 -9.29126496e-01
      4.24504028e-01  1.65503369e-02 -5.65660466e-01 -4.48750113e-01
     -1.42606894e-01 -3.58370739e-01 -5.49768014e-01  2.26715665e-01
      7.59567279e-01 -1.70390385e-01 -1.29147338e-01  5.18420667e-01
      7.26289874e-02 -1.24110182e-01  1.67595367e-01 -2.27579958e-01
     -1.29237707e-01  5.04398210e-01  5.63896973e-01  3.75649626e-01
      2.41262429e-01  4.35319598e-01  7.81600400e-01 -1.90788864e-01
      7.04286722e-01 -2.57834723e-01  1.66002172e-02  4.99771199e-01
     -6.98076615e-02  1.99368404e-02  1.69557759e-02  6.72450372e-01
     -2.61657103e-01 -2.48276488e-01 -2.54648130e-01  2.66933237e-01
     -1.11269025e-01 -3.04349994e-01 -1.86486737e-01  1.04088083e-01
     -2.04991710e-01 -1.42062548e-01  4.09745553e-01  7.18902173e-01
     -4.17237082e-01 -4.66106213e-01 -6.05041031e-01  1.64477269e-01
     -9.98371489e-02 -7.57197577e-01 -4.11716842e-01  2.56659569e-01
     -2.85581727e-02 -4.08223158e-01  9.74102364e-02  5.77573246e-02
     -1.31596888e+00  5.46928560e-01 -9.35323971e-02  2.05374086e-01
     -4.37611102e-02 -7.70637753e-02 -5.03931351e-01  6.40366578e-01
      1.15417791e-01  3.13548164e-01 -3.39258281e-02  1.04231413e-01
      1.08770838e-01  3.59229554e-01 -9.22998517e-01 -2.86048562e-01
      9.38375787e-01  1.78813781e-01  1.18366524e-01 -6.14083065e-01
     -8.29878175e-01  4.19755810e-01 -1.21809581e+00  6.41759979e-01
     -9.65455514e-02  3.64455811e-01 -1.02804621e+00 -3.53208640e-02
     -1.03144707e+00  9.32580357e-01  8.23175859e-01  9.60187628e-01
     -2.16701232e-01  1.26213434e-01  4.95934365e-01 -1.27071310e-01
     -5.04781494e-01  3.49727187e-01 -6.29277147e-01  7.99042616e-02
      5.85150614e-01  2.83053716e-01  3.33636807e-01 -2.31929197e-01
      1.79124852e-01  6.55336859e-01 -1.07746340e-01  9.12889253e-02
      4.85613160e-02  5.65558603e-01 -1.32538441e-01 -4.07309306e-01
     -7.39164225e-01 -1.44958260e-01  5.14331740e-01 -4.19178789e-01
     -6.17395206e-01  1.92174074e-01  1.43629079e-01  2.35306438e-01
     -1.19681252e-01 -1.27121678e-01 -5.24828011e-01  7.60670972e-02
      4.77107528e-01 -8.04589467e-02 -3.14832881e-02  3.06522625e-01
      3.47568260e-02  5.12749489e-02 -4.07137523e-01 -3.87385384e-01
     -1.46852578e+00 -6.58151176e-01  2.48718070e-01 -1.94882282e-01
      2.64563168e-01  2.26893337e-02 -1.36494071e-02  6.70483809e-01
     -3.69417330e-01 -3.49950819e-01 -4.92873149e-01 -7.15787022e-01
     -3.17314753e-01  5.79370207e-01  4.10791235e-01 -2.22901891e-01
     -9.39052311e-02 -3.23345754e-01 -8.98380451e-01 -8.11961070e-01
     -7.65390043e-02 -4.18046381e-01 -1.14842232e-01  7.53970653e-01
      1.48573694e-01 -1.61635122e-01  1.20741170e+00 -4.84098967e-01
     -7.21156498e-01 -4.23841558e-01  5.12569105e-01  4.52451103e-02
     -5.11933603e-01  7.15931619e-02  4.94201382e-02 -4.46899367e-01
      4.90353413e-01 -7.09226797e-01 -3.84952120e-01  1.23118965e+00
      3.36611615e-01  5.67357713e-01 -4.95878571e-05  1.58354035e-01
     -2.95291591e-01 -1.14594640e-01 -3.04209433e-01  4.69896295e-01
      3.31545789e-01 -8.76424349e-01 -4.82473406e-01 -8.18235309e-01
      4.32477410e-01 -3.63102457e-01  4.66329519e-01 -4.53863576e-01
     -2.93951723e-01 -6.66000909e-01 -4.87558124e-01 -4.41846995e-01
      4.96821030e-01 -3.10662542e-02  2.13451011e-01  9.05079329e-02
      1.29044298e+00 -2.97525029e-01 -2.25945685e-01 -3.36254071e-01
     -3.66180622e-01  4.99177793e-01 -3.22461898e-01 -4.66937016e-01
     -4.89694131e-01 -1.97023363e-01  9.08692937e-02  1.29666527e-01
     -2.83889726e-01  8.46348981e-01 -5.68197516e-03 -1.12335395e+00
     -4.58935087e-03  9.67042913e-01 -4.30536409e-02  5.34533339e-01
      1.78918418e-01  3.08098665e-02 -9.29172959e-02  2.92325011e-01
     -1.62568998e-01  8.80541608e-01 -4.48681862e-01 -5.75439689e-01
     -7.22994053e-01  3.02101886e-01  5.53241847e-01 -2.42860868e-01
     -2.13638830e-01  1.16191108e-01 -4.98910318e-01 -1.12985940e-01
     -6.31196740e-01 -9.13022973e-02  1.59965714e-01 -1.40762096e-01
      5.78395894e-01  3.36282651e-01  5.06198653e-02  2.58769945e-01
     -2.26502261e-01  4.36728870e-01  3.03810487e-01  2.84214687e-01
     -9.41278124e-01 -5.05889515e-01  8.88895043e-01  1.91829105e-01
     -3.20548870e-01 -4.85140199e-01  1.78983068e-01  5.61696007e-01
      3.99585007e-01  6.91680815e-01 -5.03002401e-01  1.18248748e-03
     -8.68985637e-03 -4.60605216e-01 -1.91084821e-01 -5.47101368e-01
     -1.56857025e-01 -3.08400290e-01  1.28314505e-01 -1.81846508e-01
      1.32242612e-01  3.76062467e-01  8.39747879e-02  1.16342757e+00
      8.58758319e-02 -5.17395732e-01  2.64374124e-01  1.38705450e-01
      4.11598465e-01  1.17992855e-01 -2.14628480e-01 -6.32495887e-02
      1.80189236e-01 -2.70389180e-01 -4.56622897e-01 -5.13061200e-01
     -2.21554394e-01 -2.73122592e-01 -4.23982815e-01 -8.08261569e-02
     -8.68289499e-02  4.06223279e-01 -2.21297854e-01 -1.52100208e-01
      5.91684806e-01 -1.07426111e+00 -1.03952264e+00  1.06922884e-01
     -4.81476511e-01  3.81098294e-01 -7.10130707e-01  6.05865431e-01
      2.04383135e-01  2.44198108e-01 -3.92704853e-02 -1.31952713e-01
     -1.86963104e-01 -1.88055223e-01 -2.20793737e-01 -1.92903922e-01
      6.54025736e-01 -5.41557143e-01 -1.85344038e-01  3.63267418e-01
      1.00763268e+00 -3.36192703e-01 -2.77419354e-01  3.18013217e-01
     -2.68699060e-01  2.26648835e-01  4.26840600e-01 -7.10919304e-01
      7.79262335e-01 -1.80936772e-01  8.52031116e-02  5.52879730e-01
     -1.22891162e-01 -4.96696627e-01  8.69903786e-01 -1.35449076e-01
      7.10146469e-01  2.80935304e-01  5.08636577e-02 -2.76480333e-01
      9.01454208e-01 -7.60839432e-01  6.54694626e-02  1.02519842e-01
      5.87732932e-01  1.45446234e-01 -7.98058744e-01 -8.28560616e-02
     -9.03052020e-02  1.96828910e-01 -9.98858424e-02  9.87128911e-01
      1.01932497e+00 -1.97028545e-01  5.22520790e-01 -9.76295928e-02
      8.42951239e-03  1.54506985e-01  9.39765732e-01  7.09895246e-01
     -7.20021562e-02  6.85995191e-01 -2.79425071e-01 -1.04200619e+00
      3.94423115e-01 -2.77642111e-01 -3.73887814e-01 -2.00338499e-01
      1.89229528e-01  3.86589855e-01  2.94999102e-01 -6.23616263e-01
     -2.00015258e-01  4.10311569e-01 -6.31068902e-01 -2.64131095e-02
      1.54114951e-01 -1.35020338e-01  1.06644896e-02  2.54419244e-01
      1.02614337e-01 -1.77283122e-01  3.61994902e-01  1.18849741e-01
      6.05408391e-01 -2.27066271e-01  7.26812174e-03  8.94169469e-01
     -7.40066726e-01  1.95494267e-01  1.63821605e-01  4.85091139e-01
      1.53605552e-01  1.69554036e-03 -7.14041453e-02 -3.12239607e-01
      2.56626511e-01 -4.30274427e-01 -1.98261136e-01 -3.96190190e-01
      2.27956251e-01 -6.01665401e-01  1.33824155e-01  4.60709800e-01
      2.34052781e-01 -5.40829753e-01  3.00862675e-02 -9.21616656e-02
     -9.13855428e-03  5.24536285e-01 -1.79873844e-01 -4.56126637e-02
     -5.02222266e-01  1.89865223e-01 -2.64438927e-01 -1.17349023e-02
     -3.79106235e-01  2.27509213e-01 -1.45048298e-02  6.84377322e-01
      1.10483265e-01  4.54681543e-03 -1.49566952e-01  6.96938927e-01
      2.01170527e-02  1.08359819e+00  1.03849625e-01 -2.39635686e-01
     -5.55481530e-01  8.86897468e-01 -5.37124074e-01  3.88105920e-01
      2.52045276e-01 -2.95252410e-01  7.21688849e-01  7.28888715e-01
     -5.04646175e-01 -5.84103715e-01 -3.47669415e-01 -2.21376542e-01
      4.69886340e-01 -1.63519362e-01 -4.98273682e-01  2.60713978e-01
     -4.19159054e-01  4.49733200e-01  7.90860439e-02 -1.93001485e-01
     -9.95299538e-02  4.37180863e-01 -1.02703797e-01  2.99256351e-01
     -6.28769475e-01 -2.03119907e-01  2.58282510e-02  2.85796101e-01
     -2.61871361e-01  7.26654512e-02  3.81760192e-01 -4.89833220e-01
     -3.15414452e-01 -5.12365391e-01 -2.17835877e-01  4.36965348e-01
      4.78269793e-01 -2.58983789e-01 -3.60825449e-02 -2.90276475e-01
      1.68247319e-01  1.50274451e-01 -4.92114052e-01 -1.71109587e-01
      5.95222386e-01  2.19927983e-02  7.09060727e-01  7.57294117e-01
      9.86289565e-03  4.49347556e-02 -6.04850387e-01  8.78827798e-01
      4.83546845e-01  1.16888089e-01  2.37730782e-01 -7.43210429e-02
     -1.52788671e+00  4.83454532e-01  2.86372317e-01  2.98315123e-01
      7.03468053e-01  1.72018726e-01 -7.37456624e-01 -1.67396478e-01
      2.37154887e-01  2.83393007e-01  2.92465709e-01 -5.56140000e-01
      3.25931918e-01  4.98526724e-01 -2.36680563e-02 -1.23692203e-01
      3.67912429e-01  6.41056106e-01 -1.66290827e-01 -3.13008117e-01
      4.86140828e-01 -3.51323230e-01  8.08113554e-02  7.54404625e-01
     -4.67058021e-01  1.97175597e-01  5.40324345e-01  6.15009649e-01
     -5.54746308e-01 -5.92198502e-02 -5.89137582e-01  1.56360658e-01
     -1.12436901e+00 -1.83964476e-02 -2.02689352e-01 -5.85244927e-01
      4.51212852e-01  3.25074930e-01 -7.28416986e-01  1.20225572e+00
     -1.08402565e-01  9.42259931e-01  2.33073079e-01 -3.07242930e-01
     -4.86158453e-01  2.92766399e-02 -3.75791621e-01 -3.37389668e-01
      1.85800217e-01  9.25698483e-02  5.46935079e-01  3.84258192e-01
     -4.42376643e-01 -1.77511569e-01 -7.67198332e-02  1.53392868e+00
     -1.59577503e-01 -5.33264451e-01  9.85817409e-01  1.17133546e-01
     -7.45235160e-02 -5.85406213e-02 -1.83090846e-02  3.62439357e-01
     -4.02303065e-02 -5.85207067e-01  5.73838556e-02 -6.97896674e-02
      5.50305125e-01  1.06047544e+00 -3.74787888e-01 -3.97798152e-02
     -3.69383388e-01  5.83164117e-02  7.26660556e-01  4.51110263e-01
     -8.64392196e-01 -3.42058766e-01 -6.46368495e-01  4.47849760e-01
     -5.33362229e-01 -1.02238185e+00 -1.51369375e-01  1.16738899e-02
     -1.78136808e-01 -3.61839031e-01 -6.27537912e-01  1.68887579e-01
      1.00185195e-01  1.06061821e-01  2.72162052e-02  6.89372243e-01
     -1.49192941e-01  2.32902171e-01  3.80879746e-01 -1.54883356e-01
      7.98018338e-02  1.53106020e-01  3.22816503e-01 -3.59226870e-03
      7.02620238e-01  2.49984997e-01 -1.11615757e-01 -3.76020666e-02
      9.30952984e-02 -1.77315392e-01  2.56090203e-01 -3.10588316e-01
      1.17570820e-01 -6.91300974e-01 -4.84673243e-01  2.55603220e-01
     -6.00401576e-01  3.15782314e-01  4.92286873e-02 -5.63238372e-01
      3.94972795e-01 -2.85083066e-02 -9.64706615e-02 -2.12482205e-01
     -2.06218543e-01 -3.78395638e-01 -2.87899323e-01  1.18904259e-01
     -3.37341105e-01 -5.83081608e-01 -2.00792068e-01  1.99686060e-01
     -6.36235594e-01  7.99985261e-02  3.58830755e-01 -2.46203951e-01
     -2.44210596e-01  7.07563450e-02 -9.91180649e-02  3.00371476e-01
     -6.35707961e-01 -4.25883290e-01 -1.62351459e-01  3.59430704e-01
     -3.77525224e-01  6.56125973e-01  6.12750186e-01  4.12282004e-01
     -6.11836910e-01  4.73770311e-01  2.81090917e-01  9.47251654e-02
     -9.78861843e-01  1.55302261e-03  1.02338908e+00 -5.40835134e-01
      1.04709457e+00  1.28156135e-01  5.66489253e-01 -1.75454662e-01
      7.73474107e-03  5.29807555e-01 -8.45997573e-01  1.74728420e-01
      3.77456109e-01 -4.62021740e-02  1.54230711e-01  2.62556817e-01
      1.48740491e+00  7.71568423e-01 -6.72188268e-01  5.28113340e-02
      6.02600701e-01  4.34653346e-01  5.13777559e-01 -7.37552451e-01
     -1.14333316e-02 -1.68049684e-01  9.23128793e-02 -7.43284951e-01
      4.01099343e-01 -4.22745554e-01  5.72656396e-01 -5.02418806e-01]
    evar = 0.20678342154546583
    vb = [[ 2.21026819e-04 -5.64715199e-05]
     [-5.64715199e-05  1.14889709e-04]]
    [0.54382267 0.99563508]

    [[ 0.01469681 -0.00224287]
     [-0.00224287  0.01048138]]

### 2. [Weighted Regression in =python=](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/weighted_regression.ipynb)

or see [weighted_regressions on github](https://github.com/ligonteaching/ARE212_Materials/blob/master/weighted_regression.ipynb)

The fact that $T$ and $u$ are “independent” (or at least
orthogonal) variables means that if we want to compute a
“classical” regression we’d do it something like this:

##### Define independent random variables

```python
# 2.1. ORIGINAL CODE

%matplotlib inline
import numpy as np
from scipy.stats import multivariate_normal

k = 3 # Number of observables in T

mu = [0]*k
Sigma=[[1,0.5,0],
       [0.5,2,0],
       [0,0,3]]

T = multivariate_normal(mu,Sigma)

u = multivariate_normal(cov=0.2)
```

 *notes*:
- `len(x)` returns the length of an array; R::length(x)
- `np.shape(x)` returns the dimensions of a matrix; R::dim(x)

```python
# 2.1. CODE ABOVE WITH COMMENTS AND PRINTING

%matplotlib inline
import numpy as np
from scipy.stats import multivariate_normal

k = 3 # Number of observables in T
print("k =", k)

mu = [0]*k
print("mu =", mu)
print("Length mu =", len(mu))

Sigma=[[1,0.5,0],
       [0.5,2,0],
       [0,0,3]]
print("Sigma =", Sigma)
print("Shape Sigma =", np.shape(Sigma))

T = multivariate_normal(mu,Sigma)
print("T =", T)

u = multivariate_normal(cov=0.2)
print("T =", T)
```

    k = 3
    mu = [0, 0, 0]
    Length mu = 3
    Sigma = [[1, 0.5, 0], [0.5, 2, 0], [0, 0, 3]]
    Shape Sigma = (3, 3)
    T = <scipy.stats._multivariate.multivariate_normal_frozen object at 0x7f712600cda0>
    T = <scipy.stats._multivariate.multivariate_normal_frozen object at 0x7f712600cda0>

##### Define =X=

Recall that $X$ can depend on $T$ and $u$.  This dependence needn’t be
linear!  For example, suppose $X=T^3D + u$, where $D$ is an
$\ell\times k$ matrix.

##### Construct Sample

To construct a sample of observables $(y,X,T)$ we just use the regression equation,
      plus an assumption about the value of $\beta$:

```python
# 2.2. ORIGINAL CODE

beta = [1/2,1]

D = np.random.random(size=(3,2)) # Generate random 3x2 matrix

N=1000 # Sample size

# Now: Transform rvs into a sample
T = T.rvs(N)

u = u.rvs(N) # Replace u with a sample

X = (T**3)@D  # Note use of ** operator for exponentiation

y = X@beta + u # Note use of @ operator for matrix multiplication
```

NB: Once you redefine your random variables using X.rvs(N), you must rerun the earlier code to run this cell again.

 *notes*:
- `**` for exponents so that `3**2` = $3^2$; R::3^2 or 3**2 (i.e. the same)

```python
# 2.2. CODE ABOVE WITH COMMENTS AND PRINTING

beta = [1/2,1]
print("beta =", beta)
print("Length beta =", len(beta))

D = np.random.random(size=(3,2)) # Generate random 3x2 matrix
print("D =", D)
print("Shape D =", np.shape(D))

N=1000 # Sample size

# Now: Transform rvs into a sample
T = T.rvs(N)
print("T =", T)
print("Shape T =", np.shape(T))

u = u.rvs(N) # Replace u with a sample
print("u =", u)
print("Shape u =", np.shape(u))
print("Length u =", len(u))

X = (T**3)@D  # Note use of ** operator for exponentiation
print("Shape X =", np.shape(X))

y = X@beta + u # Note use of @ operator for matrix multiplication
print("Shape y =", np.shape(y))
```

##### Turn to estimation

So, we now have data on *realizations* $(y,X,T)$.  Now forget
     that we know $\beta$ and let’s estimate it, using weighted least
     squares.  As a numerical matter it’s better to avoid explicitly
     inverting the $(T^T X)$ matrix; instead we can solve the “normal”
     equations

\begin{align*}
   X'y &= X' X b + X' u\\
   \mbox{E}(T'u) = 0
\end{align*}

##### Numerical solution

In the classical case we were trying to solve a linear system that
 took the form $Ab=0$, with $A$ a square matrix.  In the present case
 we’re also trying to solve a linear system, but with a matrix $A$
 that may have more rows than columns.  Provided the rows are linearly
 independent, this implies that we have an **overidentified** system of
 equations.  We’ll return to the implications of this later, but for
 now this also calls for a different numerical approach, using
 `np.linalg.lstsq` instead of `np.linalg.solve`.

```python
# 2.3. ORIGINAL CODE

from scipy.linalg import inv, sqrtm

b = np.linalg.lstsq(T.T@X,T.T@y)[0] # lstsqs returns several results

e = y - X@b

print(b)

TXplus = np.linalg.pinv(T.T@X) # Moore-Penrose pseudo-inverse

# Covariance matrix of b
vb = e.var()*TXplus@T.T@T@TXplus.T  # u is known to be homoskedastic

print(vb)
```

    [0.49668521 1.00142071]
    [[ 4.38645065e-06 -2.08749324e-06]
     [-2.08749324e-06  3.17039612e-06]]

    /srv/app/venv/lib/python3.6/site-packages/ipykernel_launcher.py:5: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
    To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
      """

```python
# np.linalg.lstsq?
# Return the least-squares solution to a linear matrix equation.
## Solves the equation `a x = b` by computing a vector `x` that
## minimizes the Euclidean 2-norm `|| b - a x ||^2`.  The equation may
## be under-, well-, or over- determined (i.e., the number of
## linearly independent rows of `a` can be less than, equal to, or
## greater than its number of linearly independent columns).  If `a`
## is square and of full rank, then `x` (but for round-off error) is
## the "exact" solution of the equation.
```

```python
# np.linalg.pinv?
# Compute the (Moore-Penrose) pseudo-inverse of a matrix.
## Calculate the generalized inverse of a matrix using its
## singular-value decomposition (SVD) and including all
## *large* singular values.
```

```python
# 2.3. CODE ABOVE WITH COMMENTS AND PRINTING

# import both the 'inv' and 'sqrtm' functions from the 'linalg' functions of the SciPy module
from scipy.linalg import inv, sqrtm

# solve for b in the equation T.T@X@b = T.T@y
b = np.linalg.lstsq(T.T@X,T.T@y, rcond=-1)[0] # lstsqs returns several results
# note that I'm also passing it rcond=-1 to mute the error above and maintain the old parameter

# e gets R:: y - X %*% b
e = y - X@b
print

print("b =", b)

# calculate the Moore-Penrose pseudo-inverse
TXplus = np.linalg.pinv(T.T@X) # Moore-Penrose pseudo-inverse

# Covariance matrix of b
## vb gets var(e) * TXplus %*% t(T) %*% T %*% t(TXplus)
vb = e.var()*TXplus@T.T@T@TXplus.T  # u is known to be homoskedastic

print("vb =",vb)
```

    b = [0.49668521 1.00142071]
    vb = [[ 4.38645065e-06 -2.08749324e-06]
     [-2.08749324e-06  3.17039612e-06]]

### 3. [random_variables0](https://datahub.berkeley.edu/user/benjaminkrause/notebooks/ARE212_Materials/random_variables0.ipynb)

or see [random_variables0 on github](https://github.com/ligonteaching/ARE212_Materials/blob/master/random_variables0.ipynb)

### Defining Random Variables in =python=                       :code_example:

```python
# Define "tag" the display of which hides the cell
from IPython.display import HTML
from IPython.display import display

# Taken from https://stackoverflow.com/questions/31517194/how-to-hide-one-specific-cell-input-or-output-in-ipython-notebook
def tag(marker):
    s = HTML('''
<a href="javascript:code_toggle()">%s</a>.''' % marker)
    return s
display(tag(''))
```

.

The distinguishing feature of variables in a field such as the
   reals or the complex plane is their *value*; the distinguishing
   feature of random variables is their *distribution*.  The `python`
   package `scipy.stats` is well-engineered and offers many different
   distributions, and tools to construct others, while the package
   `pacal` is perhaps less well engineered, but defines arithmetic
   operations over random variables which allows for more elegant
   semantics.

There are two main classes of random variables to consider: discrete
and continuous.  The distinction is worth drawing because different
classes are handled differently in many mathematical operations.

For example, here we instantiate a scalar $\rvx$:

```python
# 3.2. ORIGINAL CODE

from scipy.stats import distributions as iid

x = iid.norm()
```

```python
# iid.norm?
# A normal continuous random variable.
```

```python
# 3.2. CODE ABOVE WITH COMMENTS AND PRINTING

# import just the distributions function (and call it "iid") from the "stats" functions of the SciPy module
from scipy.stats import distributions as iid

# x gets a normal continuous random variable.
x = iid.norm()
```

And here we instantiate a discrete random variable which is defined
 over an event space $\{-1,0,1\}$ with corresponding probabilities $(1/3,1/2,1/6)$:

```python
# 3.3. ORIGINAL CODE

Omega = (-1,0,1)
Pr = (1/3.,1/2.,1/6.)

s = iid.rv_discrete(values=(Omega,Pr))
```

 *notes*:
- `.` after numbers specifies the type as `float` (as opposed to an `integer`)

```python
# iid.rv_discrete?
# `rv_discrete` is a base class to construct specific distribution classes
## and instances for discrete random variables. It can also be used
## to construct an arbitrary distribution defined by a list of support
## points and corresponding probabilities.
```

```python
# 3.3. CODE ABOVE WITH COMMENTS AND PRINTING

# Omega gets R::list(-1, 0, 1)
Omega = (-1,0,1)
print("Omega =", Omega)

# Pr gets R::list(1/3,1/2,1/6)
Pr = (1/3.,1/2.,1/6.)
print("Pr =", Pr)

# generate a discrete random variable of defined over event space Omega with corresponding probabilities Pr
s = iid.rv_discrete(values=(Omega,Pr))
print("s =", s)
```

    Omega = (-1, 0, 1)
    Pr = (0.3333333333333333, 0.5, 0.16666666666666666)
    s = <scipy.stats._distn_infrastructure.rv_sample object at 0x7f712600c860>

Now, here are some things we can do with these random variables.
 First, the continuous  $\rvx$:

```python
# 3.4. ORIGINAL CODE

print("E(x) = %6.4f" % x.mean())
print()
print("Some (central) moments of x:")
print([(m,x.moment(m)) for m in [1,2,3,4]])
print()
print("95%% confidence interval: (%f,%f)" % x.interval(0.95))
print()
print(x.cdf(0),x.pdf(0))
```

    E(x) = 0.0000

    Some (central) moments of x:
    [(1, 0.0), (2, 1.0), (3, 0.0), (4, 3.0)]

    95% confidence interval: (-1.959964,1.959964)

    0.5 0.3989422804014327

 *notes*:
- `%` operator used to specify formats in strings, see [A Guide to the Newer Python String Format Techniques](https://realpython.com/python-formatted-output/)
- `%6.4f` sets the formatting to round and show float numbers to 4 decimal places
- `%f` sets formatting as a float
- `x.moment(y)` calls the yth moment of x
- `x.interval(y)` calls the y (as a decimal point) interval of x
- `x.cdf(y)` Cumulative distribution function of x evaluated at y
- `x.pdf(y)` probability density function of x evaluated at y
- `print()` just returns blank lines, similar to including the `end` specification `print( , end = '\n\n')`

```python
# 3.4. CODE ABOVE WITH COMMENTS AND PRINTING

print("E(x) = %6.4f" % x.mean())
print()
print("E(x) = %6.4f" % x.mean(), end = '\n\n')
print("Some (central) moments of x:")
print([(m,x.moment(m)) for m in [1,2,3,4]]) # note the equivalent of an for loop passing the range 1-4 through m in x.moment(m)
print()
print("Just the first moment = ", x.moment(1))
print()
print("Just the fourth moment = ", x.moment(4))
print()
print("Some (central) moments of x called as a range:")
print([(m,x.moment(m)) for m in range(1,4)])
print()
print("95%% confidence interval: (%f,%f)" % x.interval(0.95))
print()
print(x.cdf(0),x.pdf(0))
```

    E(x) = 0.0000

    E(x) = 0.0000

    Some (central) moments of x:
    [(1, 0.0), (2, 1.0), (3, 0.0), (4, 3.0)]

    Just the first moment =  0.0

    Just the fourth moment =  3.0

    Some (central) moments of x called as a range:
    [(1, 0.0), (2, 1.0), (3, 0.0)]

    95% confidence interval: (-1.959964,1.959964)

    0.5 0.3989422804014327

Next, the discrete r.v., \rv{s}:

```python
# 3.5. ORIGINAL CODE

print("E(s) = %6.4f" % s.mean())
print()
print("Some moments of x:")
print([(m,s.moment(m)) for m in [1,2,3,4]])
print()
print("95%% confidence interval: (%f,%f)" % s.interval(0.95))
print()
# Note! Not pdf, but pmf for discrete rv.
print(s.cdf(0),s.pmf(0))
```

    E(s) = -0.1667

    Some moments of x:
    [(1, -0.16666666666666666), (2, 0.5), (3, -0.16666666666666666), (4, 0.5)]

    95% confidence interval: (-1.000000,1.000000)

    0.8333333333333333 0.5

 *notes*:
- `s.pmf(y)` probability mass function of s evaluated at y

If we want *realizations* of these random variables:

```python
# 3.6. ORIGINAL CODE

N=3
print(x.rvs(N)) # N realizations; no longer random
```

    [ 1.02792504 -0.26372605  0.86607785]

We’d like to be able to combine different random variables, say
by addition, yielding a new random variable.  For instance, we’d like
to be able to construct

```python
# 3.7. ORIGINAL CODE

y = x + s
```

But this fails.  Can you explain why?  What do you suppose the cdf of
$\rvy$ looks like?  Does it have a density, or does the addition of a
random variable that *lacks* a density ($\rv{s}$) to a random variable
that has one ($\rvx$) mess things up?

```python
# 3.8. ORIGINAL CODE

display(tag("+")
# Code to convolve a random variable with a pmf and another having a cdf
# Exploits =scipy.stats= base rv_continuous class.

class ConvolvedContinuousAndDiscrete(iid.rv_continuous):

    """Convolve (add) a continuous rv x and a discrete rv s,
       returning the resulting cdf."""

    def __init__(self,f,s):
        self.continuous_rv = f
        self.discrete_rv = s
        super(ConvolvedContinuousAndDiscrete, self).__init__(name="ConvolvedContinuousAndDiscrete")

    def _cdf(self,z):
        F=0
        s = self.discrete_rv
        x = self.continuous_rv

        for k in range(len(s.xk)):
            F = F + x.cdf(z-s.xk[k])*s.pk[k]
        return F

    def _pdf(self,z):
        f=0
        s = self.discrete_rv
        x = self.continuous_rv

        for k in range(len(s.xk)):
            f = f + x.pdf(z-s.xk[k])*s.pk[k]
        return f

# Create new convolved rv:
y = ConvolvedContinuousAndDiscrete(x,s)
```

```python
# 3.9. ORIGINAL CODE

import plotly.graph_objects as go
import numpy as np

X = np.linspace(-4,4,100).tolist()

fig = go.Figure(data=go.Scatter(x=X, y=[y.pdf(z) for z in X]))
fig.show()
```

#### Exercise

Prove that $\rvy$ is continuous (in the sense that it has a density),
     as suggested by the figure *or* establish that the figure is
     wrong or misleading.

#### Proof

Let $F_x$ denote the cdf of \rvx.  We want to establish that the cdf
of $\rvy$, say $F_y(y)=\Pr(\rvy\leq y)$ is a continuously differentiable
function of $y$.  We use the fact that the distribution of $\rvy$ is a
convolution of $\rvx$ and $\rvs$, so that

\begin{equation}
\begin{split}
    \Pr(\rvy\leq y) &= \Pr(\rv{s} + \rv{x}\leq y ) \\
                    &= \sum_{s\in\Omega}\Pr(\rvx\leq y-s|s)\pi_s\\
                    &= \sum_{s\in\Omega}F_x(y-s)\pi_s,
\end{split}
\end{equation}

which is continuously differentiable in $y$, as required.

### Final Word


$G_i + P \quad[k^{b_P}_i]<->[k^{u_P}_i] \quad  G_i:P \\
G_i:P \quad  --> [k_i^{tx}] \quad  G_i + P + T_i \\
T_i + R \quad  [k^{b_R}_i]<->[k^{u_R}_i]\quad   T_i:R \\
T_i:R  \quad -->[k_i^{tl}]\quad T_i + R + X_i \\
T_i + E \quad  [k^{b_E}_i]<->[k^{u_E}_i] \quad  T_i:E \\
T_i:E \quad-->[\delta_i]\quad E \\
T \quad-->[d_T]\quad \\
X \quad-->[d]\quad $

```python

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
from autoreduce import *
import numpy as np
from sympy import symbols
```

```python
# Post conservation law and other approximations phenomenological model at the RNA level
n = 8 # Number of states : P, C1, T, R, C2, E, C3, X
nouts = 1 # Number of outputs, X_i

# Inputs by user
x_init = np.zeros(n)
x_init[0] = 100
x_init[3] = 400
x_init[5] = 20
C = np.zeros((nouts,n), dtype=int)
C[0][7] = 1

nstates_tol_max = 3
nsatees_tol_min = 2
error_tol = 3000
# System dynamics symbolically

# k_bp, k_up, k_tx, k_br, k_ur, k_tl, k_be, k_ue, d_i, d = params, len(params) = 10

x0 = symbols('P')
x1 = symbols('C1') # G:P
x2 = symbols('T')
x3 = symbols('R')
x4 = symbols('C2') # T:R
x5 = symbols('E')
x6 = symbols('C3') # T:E
x7 = symbols('X')

x = [x0, x1, x2, x3, x4, x5, x6, x7]

G = symbols('G')
k_bp = symbols('k_bp')
k_up = symbols('k_up')
k_tx = symbols('k_tx')
k_br = symbols('k_br')
k_ur = symbols('k_ur')
k_tl = symbols('k_tl')
k_be = symbols('k_be')
k_ue = symbols('k_ue')
d_i = symbols('d_i')
d = symbols('d')
d_T = symbols('d_T')

E_tot = symbols('E_tot')
P_tot = symbols('P_tot')
R_tot = symbols('R_tot')
params = [k_bp, k_up, k_tx, k_br, k_ur, k_tl, k_be, k_ue, d_i, d, d_T, E_tot, P_tot, R_tot, G]
# f0 = (k_bp + k_tx) * x1 - k_up * G * x0
f0 = (k_up + k_tx) * x1 - k_bp * G * x0
f1 = k_bp * G * x0 - (k_up + k_tx)*x1
f2 = k_tx * x1 + (k_ur + k_tl) * x4 + k_ue * x6 - k_br * x2 * x3 - k_be * x2 * x5 - d_T * x2
f3 = (k_ur + k_tl) * x4 - k_br * x2 * x3
f4 = k_br * x2 * x3 - (k_ur + k_tl) * x4
f5 = (k_ue + d_i) * x6 - k_be * x2 * x5
f6 = k_be * x2 * x5 - (k_ue + d_i) * x6
f7 = k_tl * x4 - d * x7

f = [f0,f1,f2,f3,f4,f5,f6,f7]
# parameter values
# E_tot = 20
# P_tot = 100
# R_tot = 400
params_values = [30, 10, 0.50, 80, 2, 8, 10, 2, 1, 0.5, 0.01, 20, 100, 400, 10]
# params_values = [100, 10, 4, 10, 0.25, 2, 10, 0.5, 1, 1, 1000, 1000, 1000, 10]
sys = System(x, f, params = params, params_values = params_values, C = C, x_init = x_init)
```

```python
from autoreduce.utils import get_ODE
timepoints_ode = np.linspace(0, 20, 100)
sys_ode = get_ODE(sys, timepoints_ode)
sol = sys_ode.solve_system().T
try:
    import matplotlib.pyplot as plt
    plt.plot(timepoints_ode, np.transpose(np.array(C)@sol))
    plt.xlabel('Time')
    plt.ylabel('[Outputs]')
    plt.show()
except:
    print('Plotting libraries missing.')
```

```python
from autoreduce.utils import get_reducible
timepoints_ssm = np.linspace(0,2,10)
timepoints_ode = np.linspace(0,2,100)
sys_reduce = get_reducible(sys, timepoints_ode, timepoints_ssm)
sys_reduce.nstates_tol_min = 2
sys_reduce.nstates_tol_max = 3
```

```python
P, C1, T, R, C2, E, C3, X = sys.x
conserved_quantities = [P + C1 - P_tot, R + C2 - R_tot, E + C3 - E_tot]
states_to_eliminate = [C1, C2, C3]
f_cons = sys_reduce.set_conservation_laws(conserved_quantities, states_to_eliminate)
```

```python
f_cons
```

    [-G*P*k_bp + (-P + P_tot)*(k_tx + k_up),
     -E*T*k_be - R*T*k_br - T*d_T + k_tx*(-P + P_tot) + k_ue*(-E + E_tot) + (-R + R_tot)*(k_tl + k_ur),
     -R*T*k_br + (-R + R_tot)*(k_tl + k_ur),
     -E*T*k_be + (-E + E_tot)*(d_i + k_ue),
     -X*d + k_tl*(-R + R_tot)]

```python
from autoreduce.utils import get_ODE
timepoints_ode = np.linspace(0, 20, 100)
# params = [k_bp, k_up, k_tx, k_br, k_ur, k_tl, k_be, k_ue, d_i, d, E_tot, P_tot, R_tot, G]
# params_values = [100, 10, 4, 10, 0.25, 2, 10, 0.5, 1, 10, 10000, 10000, 10000, 0.01]
# params_values = [30, 10, 0.50, 80, 2, 8, 10, 2, 1, 0.5, 0.01, 20, 100, 400, 10]
# sys_reduce.params_values = params_values
sys_ode = get_ODE(sys_reduce, timepoints_ode)
sol = sys_ode.solve_system().T
try:
    import matplotlib.pyplot as plt
    fig, ax = plt.subplots()
    plt.plot(timepoints_ode, np.transpose(np.array(sys_reduce.C)@sol), linewidth = 3)
    plt.xlabel('Time', FontSize = 18)
    plt.ylabel('[X]', FontSize = 18)
    ax.tick_params(axis='both', which='major', labelsize=14)
    # plt.legend()
    plt.show()
except:
    print('Plotting libraries missing.')
```

```python
sys_reduce.f
```

    [-G*P*k_bp + (-P + P_tot)*(k_tx + k_up),
     -E*T*k_be - R*T*k_br - T*d_T + k_tx*(-P + P_tot) + k_ue*(-E + E_tot) + (-R + R_tot)*(k_tl + k_ur),
     -R*T*k_br + (-R + R_tot)*(k_tl + k_ur),
     -E*T*k_be + (-E + E_tot)*(d_i + k_ue),
     -X*d + k_tl*(-R + R_tot)]

```python
sys_reduce.x
```

    [P, T, R, E, X]

```python
reduced_sys, fast_ss = sys_reduce.solve_timescale_separation([T,R,X], fast_states = [E, P], debug = False)

```

    Successful time-scale separation solution obtained with states: [T, R, X]!

```python
reduced_sys.f[1]
```

$\displaystyle - R T k_{br} + \left(- R + R_{tot}\right) \left(k_{tl} + k_{ur}\right)$

```python
try:
    fig, ax = plt.subplots()
    # params = [k_bp, k_up, k_tx, k_br, k_ur, k_tl, k_be, k_ue, d_i, d, d_T, E_tot, P_tot, R_tot, G]
    params_values_new = [80, 2, 0.5, 80, 2, 5, 80, 2, 0.1, 0.5, 0.05, 100, 100, 400, 10]
    sys_reduce.params_values = params_values_new

    sys_ode = get_ODE(sys_reduce, timepoints_ode)
    sol = sys_ode.solve_system().T
    _ = plt.plot(timepoints_ode, np.transpose(np.array(sys_reduce.C)@sol), 'k--', label = 'Full CRN model', linewidth = 5)
    reduced_ode = get_ODE(reduced_sys, timepoints_ode)
    reduced_sol = reduced_ode.solve_system().T
    _ = plt.plot(timepoints_ode, np.transpose(np.array(reduced_sys.C)@reduced_sol), 'r', label = 'Reduced model', linewidth = 2)
    plt.show()
except:
    print('Plotting libraries missing.')
```

```python
# results = sys_reduce.reduce_simple(skip_numerical_computations = True, debug = True)
reduced_sys, fast_ss = sys_reduce.solve_timescale_separation([T,R,X], debug = False)
# se1 = sys_reduce.get_robustness_metric(reduced_sys)
# e1 = sys_reduce.get_error_metric(reduced_sys)

```

    Successful time-scale separation solution obtained with states: [T, R, X]!

```python
reduced_sys, fast_ss = sys_reduce.solve_timescale_separation([T, R, X])
reduced_sys.f[1]
# from auto_reduce.utils import get_ODE
x_sol_hat = get_ODE(sys_reduce.get_system(), timepoints_ode).solve_system()
```

    Successful time-scale separation solution obtained with states: [T, R, X]!

$\displaystyle - R T k_{br} + \left(- R + R_{tot}\right) \left(k_{tl} + k_{ur}\right)$


This notebook verifies `doc/python/*.py`

# BACCAB.py

```python
import sympy
```

```python
from __future__ import absolute_import, division
from __future__ import print_function
from galgebra.printer import Format, xpdf
from galgebra.ga import Ga
```

```python
Format()
```

```python
g4d = Ga('a b c d')
```

```python
(a, b, c, d) = g4d.mv()
```

```python
g4d.g
```

$\displaystyle \left[\begin{matrix}(a.a) & (a.b) & (a.c) & (a.d)\\(a.b) & (b.b) & (b.c) & (b.d)\\(a.c) & (b.c) & (c.c) & (c.d)\\(a.d) & (b.d) & (c.d) & (d.d)\end{matrix}\right]$

```python
a | (b * c)
```

\begin{equation*} - \left ( a\cdot c\right )  \boldsymbol{b} + \left ( a\cdot b\right )  \boldsymbol{c} \end{equation*}

```python
a | (b ^ c)
```

\begin{equation*} - \left ( a\cdot c\right )  \boldsymbol{b} + \left ( a\cdot b\right )  \boldsymbol{c} \end{equation*}

```python
a | (b ^ c ^ d)
```

\begin{equation*} \left ( a\cdot d\right )  \boldsymbol{b}\wedge \boldsymbol{c} - \left ( a\cdot c\right )  \boldsymbol{b}\wedge \boldsymbol{d} + \left ( a\cdot b\right )  \boldsymbol{c}\wedge \boldsymbol{d} \end{equation*}

```python
(a | (b ^ c)) + (c | (a ^ b)) + (b | (c ^ a))
```

\begin{equation*}  0  \end{equation*}

```python
a * (b ^ c) - b * (a ^ c) + c * (a ^ b)
```

\begin{equation*} 3 \boldsymbol{a}\wedge \boldsymbol{b}\wedge \boldsymbol{c} \end{equation*}

```python
a * (b ^ c ^ d) - b * (a ^ c ^ d) + c * (a ^ b ^ d) - d * (a ^ b ^ c)
```

\begin{equation*} 4 \boldsymbol{a}\wedge \boldsymbol{b}\wedge \boldsymbol{c}\wedge \boldsymbol{d} \end{equation*}

```python
(a ^ b) | (c ^ d)
```

\begin{equation*} - \left ( a\cdot c\right )  \left ( b\cdot d\right )  + \left ( a\cdot d\right )  \left ( b\cdot c\right )  \end{equation*}

```python
((a ^ b) | c) | d
```

\begin{equation*} - \left ( a\cdot c\right )  \left ( b\cdot d\right )  + \left ( a\cdot d\right )  \left ( b\cdot c\right )  \end{equation*}

```python
Ga.com(a ^ b, c ^ d)
```

\begin{equation*} - \left ( b\cdot d\right )  \boldsymbol{a}\wedge \boldsymbol{c} + \left ( b\cdot c\right )  \boldsymbol{a}\wedge \boldsymbol{d} + \left ( a\cdot d\right )  \boldsymbol{b}\wedge \boldsymbol{c} - \left ( a\cdot c\right )  \boldsymbol{b}\wedge \boldsymbol{d} \end{equation*}

# Dirac.py

```python
from __future__ import absolute_import, division
from __future__ import print_function
import sys
from sympy import symbols, sin, cos
from galgebra.printer import Format, xpdf, Get_Program, Print_Function
from galgebra.ga import Ga
```

```python
Format()
coords = symbols('t x y z', real=True)
coords
```

$\displaystyle \left( t, \  x, \  y, \  z\right)$

```python
(st4d, g0, g1, g2, g3) = Ga.build(
    'gamma*t|x|y|z', g=[1, -1, -1, -1], coords=coords)
```

```python
g0
```

\begin{equation*}  \boldsymbol{\gamma }_{t} \end{equation*}

```python
g1
```

\begin{equation*}  \boldsymbol{\gamma }_{x} \end{equation*}

```python
g2
```

\begin{equation*}  \boldsymbol{\gamma }_{y} \end{equation*}

```python
g3
```

\begin{equation*}  \boldsymbol{\gamma }_{z} \end{equation*}

```python
I = st4d.i
I
```

\begin{equation*}  \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} \end{equation*}

```python
(m, e) = symbols('m e')
```

```python
m
```

$\displaystyle m$

```python
e
```

$\displaystyle e$

```python
# 4-Vector Potential
A = st4d.mv('A', 'vector', f=True)
A
```

\begin{equation*} A = A^{t}  \boldsymbol{\gamma }_{t} + A^{x}  \boldsymbol{\gamma }_{x} + A^{y}  \boldsymbol{\gamma }_{y} + A^{z}  \boldsymbol{\gamma }_{z} \end{equation*}

```python
# 8-componentrealspinor
psi = st4d.mv('psi', 'spinor', f=True)
psi
```

\begin{equation*} psi = \psi    + \psi ^{tx}  \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x} + \psi ^{ty}  \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{y} + \psi ^{tz}  \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{z} + \psi ^{xy}  \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y} + \psi ^{xz}  \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{z} + \psi ^{yz}  \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} + \psi ^{txyz}  \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} \end{equation*}

```python
sig_z = g3 * g0
sig_z
```

\begin{equation*} - \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{z} \end{equation*}

Dirac Equation $\newcommand{bm}[1]{\boldsymbol #1}
\nabla \bm{\psi} I \sigma_{z}-e\bm{A}\bm{\psi}-m\bm{\psi}\gamma_{t} = 0$

```python
dirac_eq = (st4d.grad * psi) * I * sig_z - e * A * psi - m * psi * g0
dirac_eq
```

\begin{equation*} \left ( - e A^{t}  \psi   - e A^{x}  \psi ^{tx}  - e A^{y}  \psi ^{ty}  - e A^{z}  \psi ^{tz}  - m \psi   - \partial_{y} \psi ^{tx}  - \partial_{z} \psi ^{txyz}  + \partial_{x} \psi ^{ty}  + \partial_{t} \psi ^{xy} \right ) \boldsymbol{\gamma }_{t} + \left ( - e A^{t}  \psi ^{tx}  - e A^{x}  \psi   - e A^{y}  \psi ^{xy}  - e A^{z}  \psi ^{xz}  + m \psi ^{tx}  + \partial_{y} \psi   - \partial_{t} \psi ^{ty}  - \partial_{x} \psi ^{xy}  + \partial_{z} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x} + \left ( - e A^{t}  \psi ^{ty}  + e A^{x}  \psi ^{xy}  - e A^{y}  \psi   - e A^{z}  \psi ^{yz}  + m \psi ^{ty}  - \partial_{x} \psi   + \partial_{t} \psi ^{tx}  - \partial_{y} \psi ^{xy}  - \partial_{z} \psi ^{xz} \right ) \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{tz}  + e A^{x}  \psi ^{xz}  + e A^{y}  \psi ^{yz}  - e A^{z}  \psi   + m \psi ^{tz}  + \partial_{t} \psi ^{txyz}  - \partial_{z} \psi ^{xy}  + \partial_{y} \psi ^{xz}  - \partial_{x} \psi ^{yz} \right ) \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{xy}  + e A^{x}  \psi ^{ty}  - e A^{y}  \psi ^{tx}  - e A^{z}  \psi ^{txyz}  - m \psi ^{xy}  - \partial_{t} \psi   + \partial_{x} \psi ^{tx}  + \partial_{y} \psi ^{ty}  + \partial_{z} \psi ^{tz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{xz}  + e A^{x}  \psi ^{tz}  + e A^{y}  \psi ^{txyz}  - e A^{z}  \psi ^{tx}  - m \psi ^{xz}  + \partial_{x} \psi ^{txyz}  + \partial_{z} \psi ^{ty}  - \partial_{y} \psi ^{tz}  - \partial_{t} \psi ^{yz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{yz}  - e A^{x}  \psi ^{txyz}  + e A^{y}  \psi ^{tz}  - e A^{z}  \psi ^{ty}  - m \psi ^{yz}  - \partial_{z} \psi ^{tx}  + \partial_{y} \psi ^{txyz}  + \partial_{x} \psi ^{tz}  + \partial_{t} \psi ^{xz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{txyz}  - e A^{x}  \psi ^{yz}  + e A^{y}  \psi ^{xz}  - e A^{z}  \psi ^{xy}  + m \psi ^{txyz}  + \partial_{z} \psi   - \partial_{t} \psi ^{tz}  - \partial_{x} \psi ^{xz}  - \partial_{y} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} \end{equation*}

```python
dirac_eq.Fmt(2)
```

 \begin{align*}  & \left ( - e A^{t}  \psi   - e A^{x}  \psi ^{tx}  - e A^{y}  \psi ^{ty}  - e A^{z}  \psi ^{tz}  - m \psi   - \partial_{y} \psi ^{tx}  - \partial_{z} \psi ^{txyz}  + \partial_{x} \psi ^{ty}  + \partial_{t} \psi ^{xy} \right ) \boldsymbol{\gamma }_{t} + \left ( - e A^{t}  \psi ^{tx}  - e A^{x}  \psi   - e A^{y}  \psi ^{xy}  - e A^{z}  \psi ^{xz}  + m \psi ^{tx}  + \partial_{y} \psi   - \partial_{t} \psi ^{ty}  - \partial_{x} \psi ^{xy}  + \partial_{z} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x} + \left ( - e A^{t}  \psi ^{ty}  + e A^{x}  \psi ^{xy}  - e A^{y}  \psi   - e A^{z}  \psi ^{yz}  + m \psi ^{ty}  - \partial_{x} \psi   + \partial_{t} \psi ^{tx}  - \partial_{y} \psi ^{xy}  - \partial_{z} \psi ^{xz} \right ) \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{tz}  + e A^{x}  \psi ^{xz}  + e A^{y}  \psi ^{yz}  - e A^{z}  \psi   + m \psi ^{tz}  + \partial_{t} \psi ^{txyz}  - \partial_{z} \psi ^{xy}  + \partial_{y} \psi ^{xz}  - \partial_{x} \psi ^{yz} \right ) \boldsymbol{\gamma }_{z} \\  &  + \left ( - e A^{t}  \psi ^{xy}  + e A^{x}  \psi ^{ty}  - e A^{y}  \psi ^{tx}  - e A^{z}  \psi ^{txyz}  - m \psi ^{xy}  - \partial_{t} \psi   + \partial_{x} \psi ^{tx}  + \partial_{y} \psi ^{ty}  + \partial_{z} \psi ^{tz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{xz}  + e A^{x}  \psi ^{tz}  + e A^{y}  \psi ^{txyz}  - e A^{z}  \psi ^{tx}  - m \psi ^{xz}  + \partial_{x} \psi ^{txyz}  + \partial_{z} \psi ^{ty}  - \partial_{y} \psi ^{tz}  - \partial_{t} \psi ^{yz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{yz}  - e A^{x}  \psi ^{txyz}  + e A^{y}  \psi ^{tz}  - e A^{z}  \psi ^{ty}  - m \psi ^{yz}  - \partial_{z} \psi ^{tx}  + \partial_{y} \psi ^{txyz}  + \partial_{x} \psi ^{tz}  + \partial_{t} \psi ^{xz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{txyz}  - e A^{x}  \psi ^{yz}  + e A^{y}  \psi ^{xz}  - e A^{z}  \psi ^{xy}  + m \psi ^{txyz}  + \partial_{z} \psi   - \partial_{t} \psi ^{tz}  - \partial_{x} \psi ^{xz}  - \partial_{y} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z}  \end{align*}

```python
dirac_eq = dirac_eq.simplify()
dirac_eq
```

 \begin{align*}  & \left ( - e A^{t}  \psi   - e A^{x}  \psi ^{tx}  - e A^{y}  \psi ^{ty}  - e A^{z}  \psi ^{tz}  - m \psi   - \partial_{y} \psi ^{tx}  - \partial_{z} \psi ^{txyz}  + \partial_{x} \psi ^{ty}  + \partial_{t} \psi ^{xy} \right ) \boldsymbol{\gamma }_{t} + \left ( - e A^{t}  \psi ^{tx}  - e A^{x}  \psi   - e A^{y}  \psi ^{xy}  - e A^{z}  \psi ^{xz}  + m \psi ^{tx}  + \partial_{y} \psi   - \partial_{t} \psi ^{ty}  - \partial_{x} \psi ^{xy}  + \partial_{z} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x} + \left ( - e A^{t}  \psi ^{ty}  + e A^{x}  \psi ^{xy}  - e A^{y}  \psi   - e A^{z}  \psi ^{yz}  + m \psi ^{ty}  - \partial_{x} \psi   + \partial_{t} \psi ^{tx}  - \partial_{y} \psi ^{xy}  - \partial_{z} \psi ^{xz} \right ) \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{tz}  + e A^{x}  \psi ^{xz}  + e A^{y}  \psi ^{yz}  - e A^{z}  \psi   + m \psi ^{tz}  + \partial_{t} \psi ^{txyz}  - \partial_{z} \psi ^{xy}  + \partial_{y} \psi ^{xz}  - \partial_{x} \psi ^{yz} \right ) \boldsymbol{\gamma }_{z} \\  &  + \left ( - e A^{t}  \psi ^{xy}  + e A^{x}  \psi ^{ty}  - e A^{y}  \psi ^{tx}  - e A^{z}  \psi ^{txyz}  - m \psi ^{xy}  - \partial_{t} \psi   + \partial_{x} \psi ^{tx}  + \partial_{y} \psi ^{ty}  + \partial_{z} \psi ^{tz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{xz}  + e A^{x}  \psi ^{tz}  + e A^{y}  \psi ^{txyz}  - e A^{z}  \psi ^{tx}  - m \psi ^{xz}  + \partial_{x} \psi ^{txyz}  + \partial_{z} \psi ^{ty}  - \partial_{y} \psi ^{tz}  - \partial_{t} \psi ^{yz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{yz}  - e A^{x}  \psi ^{txyz}  + e A^{y}  \psi ^{tz}  - e A^{z}  \psi ^{ty}  - m \psi ^{yz}  - \partial_{z} \psi ^{tx}  + \partial_{y} \psi ^{txyz}  + \partial_{x} \psi ^{tz}  + \partial_{t} \psi ^{xz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{txyz}  - e A^{x}  \psi ^{yz}  + e A^{y}  \psi ^{xz}  - e A^{z}  \psi ^{xy}  + m \psi ^{txyz}  + \partial_{z} \psi   - \partial_{t} \psi ^{tz}  - \partial_{x} \psi ^{xz}  - \partial_{y} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z}  \end{align*}

```python
dirac_eq.Fmt(2)
```

 \begin{align*}  & \left ( - e A^{t}  \psi   - e A^{x}  \psi ^{tx}  - e A^{y}  \psi ^{ty}  - e A^{z}  \psi ^{tz}  - m \psi   - \partial_{y} \psi ^{tx}  - \partial_{z} \psi ^{txyz}  + \partial_{x} \psi ^{ty}  + \partial_{t} \psi ^{xy} \right ) \boldsymbol{\gamma }_{t} + \left ( - e A^{t}  \psi ^{tx}  - e A^{x}  \psi   - e A^{y}  \psi ^{xy}  - e A^{z}  \psi ^{xz}  + m \psi ^{tx}  + \partial_{y} \psi   - \partial_{t} \psi ^{ty}  - \partial_{x} \psi ^{xy}  + \partial_{z} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x} + \left ( - e A^{t}  \psi ^{ty}  + e A^{x}  \psi ^{xy}  - e A^{y}  \psi   - e A^{z}  \psi ^{yz}  + m \psi ^{ty}  - \partial_{x} \psi   + \partial_{t} \psi ^{tx}  - \partial_{y} \psi ^{xy}  - \partial_{z} \psi ^{xz} \right ) \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{tz}  + e A^{x}  \psi ^{xz}  + e A^{y}  \psi ^{yz}  - e A^{z}  \psi   + m \psi ^{tz}  + \partial_{t} \psi ^{txyz}  - \partial_{z} \psi ^{xy}  + \partial_{y} \psi ^{xz}  - \partial_{x} \psi ^{yz} \right ) \boldsymbol{\gamma }_{z} \\  &  + \left ( - e A^{t}  \psi ^{xy}  + e A^{x}  \psi ^{ty}  - e A^{y}  \psi ^{tx}  - e A^{z}  \psi ^{txyz}  - m \psi ^{xy}  - \partial_{t} \psi   + \partial_{x} \psi ^{tx}  + \partial_{y} \psi ^{ty}  + \partial_{z} \psi ^{tz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y} + \left ( - e A^{t}  \psi ^{xz}  + e A^{x}  \psi ^{tz}  + e A^{y}  \psi ^{txyz}  - e A^{z}  \psi ^{tx}  - m \psi ^{xz}  + \partial_{x} \psi ^{txyz}  + \partial_{z} \psi ^{ty}  - \partial_{y} \psi ^{tz}  - \partial_{t} \psi ^{yz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{yz}  - e A^{x}  \psi ^{txyz}  + e A^{y}  \psi ^{tz}  - e A^{z}  \psi ^{ty}  - m \psi ^{yz}  - \partial_{z} \psi ^{tx}  + \partial_{y} \psi ^{txyz}  + \partial_{x} \psi ^{tz}  + \partial_{t} \psi ^{xz} \right ) \boldsymbol{\gamma }_{t}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z} + \left ( - e A^{t}  \psi ^{txyz}  - e A^{x}  \psi ^{yz}  + e A^{y}  \psi ^{xz}  - e A^{z}  \psi ^{xy}  + m \psi ^{txyz}  + \partial_{z} \psi   - \partial_{t} \psi ^{tz}  - \partial_{x} \psi ^{xz}  - \partial_{y} \psi ^{yz} \right ) \boldsymbol{\gamma }_{x}\wedge \boldsymbol{\gamma }_{y}\wedge \boldsymbol{\gamma }_{z}  \end{align*}

```python

```


```python
import numpy as np
import sympy as sp
import matplotlib.pyplot as plt
%matplotlib widget

x,y = sp.symbols('x y')
f = 2 * x**2 + 6 * x * y + 7 * y**2 - 26 * x - 54 * y + 107
fx = sp.lambdify((x,y),f,'numpy')

dx = sp.diff(f,x)
dfx = sp.lambdify((x,y),dx,'numpy')
dy = sp.diff(f,y)
dfy = sp.lambdify((x,y),dy,'numpy')

xx = np.linspace(1, 16, 100)
yy = np.linspace(-3, 6, 90)
X, Y = np.meshgrid(xx, yy)
Z = fx(X, Y)

xx2 = np.linspace(1, 16, 15)
yy2 = np.linspace(-3, 6, 9)
X2, Y2 = np.meshgrid(xx2, yy2)
Z2 = fx(X2,Y2)

GZ = np.ones_like(Z2)
GX = dfx(X2, Y2)
GY = dfy(X2, Y2)
plt.figure(figsize=(10, 5))
plt.contour(X, Y, Z, levels=np.logspace(0, 3, 10))
plt.quiver(X2, Y2, GX, GY, color='blue', scale=400, minshaft=2)

plt.xlabel('x')
plt.ylabel('y')
plt.title("quiver plot")
plt.show()
```

        Figure

```python
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
ax.plot_surface(X,Y,Z,alpha=0.5)
ax.quiver(X2,Y2,Z2,GX,GY,GZ,color='r',normalize=True)
#ax.quiver(X2,Y2,Z2,GX,GY,GZ,color='r')
```

    <mpl_toolkits.mplot3d.art3d.Line3DCollection at 0x7f74c1253730>

        Figure

```python

```


Note on Bayesian Sequential Probability Test

Haixi Li

## The Problem
Consider a dynamic regression model in the following form:

\begin{equation}
y_{t} = x_{t}'\beta_{t}+\epsilon_{t}, \tag{1}
\end{equation}
where $x_{t}$ is a $k$ dimensional vector of regressors and $\beta_{t}$ is a $k\times 1$ vector of regression coefficients, and the error terms $\epsilon_{t}$ is i.i.d $\mathcal{N}(0,\sigma^{2})$. Assume in the history $t \in \{ 1,...,n \}$, the coefficients in Equation [(1)](#OLS) are constant and equal to $\beta_{0}$. We want to monitor new data from $n+1$ onward to test if there is any structural break happening since. The null hypothesis is
\begin{equation}
    \beta_{t} = \beta_{0} \, \forall t
\end{equation}
against the alternative that from some unknown time $\kappa>n$, the coefficients changes to $\beta_{1} \neq \beta_{0}$. To be more precise, we assume the following statistical process:
\begin{eqnarray*}
    H_{0}: y_{t} &=& x_{t}'\beta_{0} + \epsilon_{t} \text{ for } t = 0,1,2,...,\kappa-1,\\
    H_{1}: y_{t} &=& x_{t}'\beta_{1} + \epsilon_{t} \text{ for } t = \kappa, \kappa+1,...
\end{eqnarray*}
Correspondingly we have the following hypotheses test, define stochastic process $\zeta_{t}$:
\begin{eqnarray*}
    H_{0}: \varepsilon_{t} &=& \epsilon_{t} \text{ for } t = 0,1,2,...,\kappa-1,\\
    H_{1}: \varepsilon_{t} &=& x_{t}'(\beta_{1}-\beta_{0}) + \epsilon_{t} \text{ for } t = \kappa, \kappa+1,...
\end{eqnarray*}
To simplify the problem we assume the unconditional mean $E(\varepsilon_{t})=\mu$.
So we arrived the following stochastic process:
\begin{eqnarray*}
    H_{0}: \zeta_{t} &=& \epsilon_{t} \text{ for } t = 0,1,2,...,\kappa-1,\\
    H_{1}: \zeta_{t} &=& \mu + \epsilon_{t} \text{ for } t = \kappa, \kappa+1,...
\end{eqnarray*}
The reason to use unconditional mean is because $\beta_{1}$ is unknown. The covariance $cov(x_{t},\beta_{t})=0$ due to standard exogeneity assumption of regression model.
In practice, practitioners estimate the model with historical data, and start monitoring the out of sample performance of the estimated model with newly arrived data. Let $\hat{\beta}^{n}$ denote the OLS estimates with historical data up to $t=n$. Under the null, the residuals can be approximated as
\begin{eqnarray*}
    \zeta_{t} &=&  \epsilon_{t} - x_{t}'(\hat{\beta}^{n} - \beta_{0})\\
    &=& \epsilon_{t}-x_{t}'((\sum_{i=0}^{n}x_{i}'x_{i})^{-1}\sum_{i=0}^{n}x_{i}'\epsilon_{i})\\
    &\approx& \epsilon_{t}.
\end{eqnarray*}
The approximation is reasonable when $\hat{\beta}^{n}$ is estimated consistently. Under the alternative, the residuals can be approximated as
\begin{eqnarray*}
    \zeta_{t} &=&  \epsilon_{t} - x_{t}'(\hat{\beta}^{n} - \beta_{1})\\
    &=& \epsilon_{t} - x_{t}'(\beta_{0} -\beta_{1}+(\sum_{i=0}^{n}x_{i}'x_{i})^{-1}\sum_{i=0}^{n}x_{i}'\epsilon_{i})\\
    &\approx& \epsilon_{t} - x_{t}'(\beta_{0}-\beta_{1})\\
    &\approx& \epsilon_{t} - \mu.
\end{eqnarray*}

```python

```


# Simulating the Heston Process

## Introduction
In today's post, I will examine a few issues related to numerical simulation of the Heston model.
As a quick reminder, the Heston process is a stochastic volatility model defined by the
following sde:

$$
\begin{align}
dS_t =& (r-d) S_t dt + \sqrt{V_t} S_t dW^s_t \\
dV_t =& \kappa (\theta - V_t) dt + \sigma \sqrt{V_t} dW^\upsilon_t \\
dW^s_t dW^\upsilon_t =& \rho dt
\end{align}
$$

for asset price $S_t$. $dW_t^s$ and $dW_t^{\nu}$ are Wiener processes with correlation $\rho$.  As usual, $r$ and $d$ are the risk free and dividend rates respectively.

The Heston model is an excellent benchmarking model due to the suprising fact that
that it's probability distribution function (pdf) has a closed formed expression.  It's a one
of five such processes in 1 or 2 dimensions [PHL](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=773568).  With the closed form pdf, we can easily compare the performance of the Heston model's numerical simulation by
comparing PV's of a Monte Carlo pricer with an analytic pricer.

To carry out a numerical simulation we first need to choose a discretization scheme. That is, we need to approximate the infinitesimal diffusion process with a finite scheme to be used in the simulation code.  Any such scheme may potentially introduce bias into the simulation that could alter the realized moments of the pdf.

The simplest possible scheme is a first order expansion, i.e. an Euler Mayorana (or just ''Euler'') scheme.  Transforming to $\ln S$ and discretizing $t$ to step size $\Delta$ gives

$$
\ln S(t + \Delta) = \ln S(t) + \left(r - d - \frac{1}{2}V(t)\right)\Delta +\sqrt{V(t)} Z_S\sqrt{\Delta}
$$
$$
V(t + \Delta) = V(t) + \kappa\left(\theta - V(t)\right)\Delta + \sigma \sqrt{V(t)} Z_V \sqrt{\Delta}
$$

However, it turns out that this scheme does introduce bias.  Furthermore, in this scheme it's possible to drive the variance process to negative values, which is outside the range of the pdf and causes the simulation to stop as $\sqrt{V}$ becomes imaginary.

One immediate fix to this problem is the floor V at 0, giving the so called "Truncated" scheme.
Other schemes are based on higher order expansions of the sde or utilizing the closed form
expression for the pdf. These are documented extensively in the literature [Anderson](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=946405).  We list some of the schemes below

1. Euler Mayorana
2. Truncated
3. Quadratic Exponential
4. Broadie Kaya (Exact Simulation)

Broadie Kaya is based off direct sampling from the pdf but is rather difficult to implement numerically.  The Quadratic Exponential (developed in [Anderson](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=946405)) approximates the pdf using simplified functions and moment matching.  The pdf for high and low values of the variance are matched against two different functions and threshold is implemented to switch between the two functions during a diffusion.  This technique allows one to minimize variance while also easing the numerical implementation.  As such, this scheme it is often the preferred scheme for Heston simulations.

## QuantLib Implementation

QuantLib includes several of these discretization schemes in their
Heston Process class.  Let's see what a simulation looks like.

Quick note: Although the HestonProcess class is available through the SWIG bindings,
it does not expose the discretizations argument.  Instead, we use [PyQL](https://github.com/enthought/pyql),
which has all the discretizations available.

```python
from quantlib.processes.heston_process import *
from quantlib.quotes import SimpleQuote
from quantlib.settings import Settings
from quantlib.termstructures.yields.flat_forward import FlatForward
from quantlib.time.api import today, TARGET, ActualActual, Date, Period, Years
from quantlib.models.equity.heston_model import (HestonModel,
                                                 HestonModelHelper)
from quantlib.pricingengines.api import (AnalyticHestonEngine)
from quantlib.pricingengines.vanilla.mceuropeanhestonengine import MCEuropeanHestonEngine
from quantlib.instruments.api import (PlainVanillaPayoff,
                                      EuropeanExercise,
                                      VanillaOption,
                                      EuropeanOption)
```

Defining the Heston Process
-------------------------------

```python
def flat_rate(forward, daycounter):
    return FlatForward(
        forward = forward,
        settlement_days = 0,
        calendar = TARGET(),
        daycounter = daycounter
    )

settings = Settings.instance()
settlement_date = today()
settings.evaluation_date = settlement_date

day_counter = ActualActual()
interest_rate = 0.7
dividend_yield = 0.4

risk_free_ts = flat_rate(interest_rate, day_counter)
dividend_ts = flat_rate(dividend_yield, day_counter)

maturity = Period(10, Years)
exercise_date = settlement_date + maturity

# spot
s0 = SimpleQuote(100.0)

# Available descritizations
#PARTIALTRUNCATION
#FULLTRUNCATION
#REFLECTION
#NONCENTRALCHISQUAREVARIANCE
#QUADRATICEXPONENTIAL
#QUADRATICEXPONENTIALMARTINGALE
#BROADIEKAYAEXACTSCHEMELOBATTO
#BROADIEKAYAEXACTSCHEMELAGUERRE
#BROADIEKAYAEXACTSCHEMETRAPEZOIDAL

# Heston Model params
v0 = 0.05
kappa = 5.0
theta = 0.05
sigma = 1.0e-4
rho = -0.5

def gen_process(desc):
    process = HestonProcess(risk_free_ts,
                            dividend_ts,
                            s0,
                            v0,
                            kappa,
                            theta,
                            sigma,
                            rho,
                            desc)
    return process

processes = {"REFLECTION" : gen_process(REFLECTION),
             "PARTIALTRUNCATION" : gen_process(PARTIALTRUNCATION),
             "QUADRATICEXPONENTIAL" : gen_process(QUADRATICEXPONENTIAL),
             "QUADRATICEXPONENTIALMARTINGALE" : gen_process(QUADRATICEXPONENTIALMARTINGALE),
}
```

## Visualizing the Simulation

Note: The *simulate* function is not part of Quantlib. It has been added to the pyQL interface (see folder quantlib/sim).

```python
from quantlib.sim.simulate import simulate_process
from quantlib.time_grid import TimeGrid
```

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set(style="white", rc={"axes.facecolor": (0, 0, 0, 0)})
```

```python
# simulate and plot Heston paths
paths = 200
steps = 100
horizon = 2
seed = 154

grid = TimeGrid(horizon, steps)

fig, axs = plt.subplots(figsize=(14, 12), nrows=2, ncols=2)
flat_axs = axs.reshape(-1)

for i, key in enumerate(processes.keys()):
    flat_axs[i].plot(list(grid), simulate_process(processes[key], paths, grid, seed))
    flat_axs[i].set_xlabel('Time')
    flat_axs[i].set_ylabel('Stock Price')
    flat_axs[i].set_title('%s' % key)

```

Let's look at of the diffusions in detail, for example the evolution of the Heston process using Partial Truncation looks like this

```python
res = simulate_process(processes["PARTIALTRUNCATION"], paths, grid, seed)
x = res[[25,50,75,100],:].reshape(4*paths)

g = np.repeat(["T=0.5","T=1.0","T=1.5","T=2.0"], paths)
df = pd.DataFrame(dict(x=x, g=g))

# Initialize the FacetGrid object
pal = sns.cubehelix_palette(10, rot=-.25, light=.7)
g = sns.FacetGrid(df, row="g", hue="g", aspect=10, size=1.6, palette=pal)

g.map(sns.distplot, "x", bins=40, kde=True, rug=False)
g.map(plt.axhline, y=0, lw=2, clip_on=False)

def label(x, color, label):
    ax = plt.gca()
    ax.text(0, .3, label, fontweight="bold", color=color,
            ha="left", va="center", transform=ax.transAxes)

g.map(label, "x")
g.fig.subplots_adjust(hspace=.25)
g.set_titles("")
g.set(yticks=[])
g.despine(bottom=True, left=True)
```

## Comparing the PV's of different engines

To examine the performance of the numerical schemes, we simply compare the PV's
the same Call Option priced with an analytic and Monte Carlo engine.  The results
for different time steps and schemes are given in the table below

```python
# Build a Call Option
payoff = PlainVanillaPayoff('Call', 105)
exercise = EuropeanExercise(exercise_date)
option = EuropeanOption(payoff, exercise)

results = {"REFLECTION" : [],
           "PARTIALTRUNCATION" : [],
           "QUADRATICEXPONENTIAL" : [],
           "QUADRATICEXPONENTIALMARTINGALE" : [],
}

# steps per year
steps = [1, 4, 8, 16]

for k, v in processes.items():
    for step in steps:
        mc_engine = MCEuropeanHestonEngine(v,
                                           steps_per_year=step,
                                           required_samples=500,
                                           seed=1234,
                                           )
        option.set_pricing_engine(mc_engine)
        results[k].append((np.around(option.npv, 5), np.around(option.error_estimate, 5)))

results["time steps per year"] = steps
results_df = pd.DataFrame(data = results)
cols = results_df.columns.tolist()
cols = cols[-1:] + cols[-2:-1] +  cols[:-2]
results_df = results_df[cols]
```

```python
results_df
```

      time steps per year
      REFLECTION
      PARTIALTRUNCATION
      QUADRATICEXPONENTIAL
      QUADRATICEXPONENTIALMARTINGALE

      0
      1
      (1.64404, 0.05392)
      (1.64447, 0.06608)
      (1.98966, 0.04768)
      (1.80227, 0.03563)

      1
      4
      (1.74109, 0.03607)
      (1.74109, 0.03607)
      (1.77399, 0.03692)
      (1.75864, 0.0351)

      2
      8
      (1.71144, 0.02744)
      (1.71144, 0.02744)
      (1.70905, 0.02727)
      (1.70581, 0.02687)

      3
      16
      (1.71527, 0.02811)
      (1.71527, 0.02811)
      (1.73049, 0.02829)
      (1.72956, 0.02818)

## Conclusion

As we saw, there are many ways to simulate the Heston model, each with strengths and
weaknesses with regards to bias/accuracy and runtime performance.  In general, the Quadratic Exponential scheme performed better than Euler-type schemes, especially for simulations with fewer time steps per year.

In a future post, I will visit the local stochastic volatility version of the Heston model to see how it's simulation
works.

## Is there a better way?

If you take a look at the HestonProcess class file in QuantLib you will see that it,
for each discretization, there is a different diffusion implementation.
The user chooses the discretization from an enumerated list. The downside to this approach is that
, in addition to creating bloated code, the list is always finite.
 What happens when someone wants to create a new discretization scheme that's not on the list? She has to add the diffusion implementation directly into the HestonProcess class.

A better approach might be to allow the user to build her own discretization in
Python and then "compile-down" to C++ in the similar way to how the finite
difference stencils are created in Devito [see my earlier post]
This may just be hopeful speculation on my part :).  Until next time.


참고문헌 : Pytel 외 저, 이주성 외 역, 재료역학, 2판, 한티미디어, 2013.
Ref: Pytel, Kiusalaas, Sharma, Mechanics of Materials, 2nd Ed., Cengege Learning, 2013.

`python` 기능을 확장해 주는 `module`을 불러 들임 (일부 기능만 사용될 수도 있음)

```python
import numpy as np  # 배열, 행렬 관련 기능
import numpy.linalg as na  # 선형대수 (벡터, 행렬) 관련 기능
import matplotlib.pyplot as plt  # 그래프 관련 기능
import scipy.integrate as si  # 적분 관련 기능
import sympy as sy  # 기호 연산 기능
import sympy.plotting as splot
import IPython.display as disp  # 웹페이지 표시 기능
sy.init_printing()  # 기호 연산 결과 표시 기능 준비
```

## 예제 07.008
ex07.008

부정정보: 중첩법
Statically Indeterminate Beam : Superposition

p. 314

### 문제에서 주어진 변수
Given Parameters

#### 보의 길이
Length of the beam

```python
L_AB_m = sy.symbols('L_AB_m', real=True, nonnegative=True)

s_d = {
    L_AB_m: 10,
}

```

#### 하중
Load

```python
w0_N_m = sy.symbols('w0_N_m', real=True)

s_d.update(
    {
        w0_N_m: -1,
    }
)

```

#### 재료와 단면 특성
Material & section properties

```python
E_Pa, I_m4 = sy.symbols('E_Pa, I_m4', positive=True)

s_d.update(
    {
        E_Pa: 200e9,
        I_m4: 20e6 * (1e-3) ** 4,
    }
)

```

#### 자유물체도
Free body diagram

```python
x_m = sy.symbols('x_m', nonnegative=True)
x_A_m = 0
x_B_m = L_AB_m

```

### 아직 알지 못하는 반력
Reaction forces unknown yet

```python
R_A_N, M_A_Nm, R_B_N = sy.symbols('R_A_N, M_A_Nm, R_B_N', real=True)
```

### 중첩법
Superposition

#### 외팔보 전체에 가해지는 분포하중 $w_0$에 의한 끝점의 처짐
Deflection of a cantilever's end point by distributed load $w_0$ over the whole length

p. 279 Table 6.2

$$
\delta_{Bw_0} = \frac{w_0L^4}{8EI}
$$

```python
delta_Bw0_m = (w0_N_m * L_AB_m ** 4 / (8 * E_Pa * I_m4))
```

```python
delta_Bw0_m
```

#### 외팔보 $L$ 지점에 가해지는 반력 $R$에 의한 끝점의 처짐
Deflection of a cantilever's end point by reaction force $R$ at $L$

p. 279 Table 6.2

$$
\delta_{BR} = \frac{RL^3}{3EI}
$$

```python
delta_BR_m = (R_B_N * L_AB_m ** 3 / (3 * E_Pa * I_m4))
```

```python
delta_BR_m
```

#### 두 처짐을 중첩함
Superpose two $\delta$'s

```python
super_eq = sy.Eq(delta_Bw0_m + delta_BR_m, 0)
```

```python
super_eq
```

#### $R$에 관하여 풂
Solve for $R$

```python
R_B_N_sol = sy.solve(super_eq, R_B_N)[0]
```

```python
R_B_N_sol
```

### 평형방정식
Equilibrium equation

수직방향
Vertical direction

$$
R_A + R_B + w_0 L = 0
$$

```python
fy_eq = sy.Eq(R_A_N + R_B_N + w0_N_m * L_AB_m)
```

```python
fy_eq
```

```python
fy_eq.subs(R_B_N, R_B_N_sol)
```

```python
R_A_N_sol = sy.solve(fy_eq.subs(R_B_N, R_B_N_sol), R_A_N)[0]
```

```python
R_A_N_sol
```

$B$점 중심 회전방향
Rotation direction around $B$

$$
M_A - R_A \cdot L - P_N \cdot (L - a)=0
$$

```python
m_eq = sy.Eq(M_A_Nm - R_A_N * L_AB_m - sy.Rational(1, 2) * w0_N_m * L_AB_m ** 2)
```

```python
m_eq
```

```python
sy.simplify(m_eq.subs(R_A_N, R_A_N_sol))
```

```python
M_A_Nm_sol = sy.solve(m_eq.subs(R_A_N, R_A_N_sol), M_A_Nm)[0]
```

```python
sy.simplify(M_A_Nm_sol)
```