# Scenario A - Noise Level Variation (results evaluation)

This file is used to evaluate the inference (numerical) results.

The model used in the inference of the parameters is formulated as follows:

\begin{equation}
\large y = f(x) = \sum\limits_{m=1}^M \big[A_m \cdot e^{-\frac{(x-\mu_m)^2}{2\cdot\sigma_m^2}}\big] + \epsilon
\end{equation}

```python
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import pymc3 as pm
import arviz as az

#az.style.use('arviz-darkgrid')

print('Running on PyMC3 v{}'.format(pm.__version__))
```

    WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

    Running on PyMC3 v3.8

## Load results summary

```python
# load results from disk
df = pd.read_csv('./scenario_noise.csv')
df.index += 1
#df.sort_values(by=['waic'], ascending=False)
df
```

      r_hat
      mcse
      ess
      bfmi
      r2
      waic
      epsilon
      epsilon_real
      run

      1
      1.792
      17.6343
      2.7
      0.983629
      0.819621
      5224.546374
      1.363282
      0.05
      1

      2
      1.046
      2.6305
      6995.3
      0.797742
      0.906804
      5396.323351
      1.449872
      0.05
      1

      3
      1.497
      1.2642
      101.8
      1.264172
      0.999211
      -3299.410427
      0.079342
      0.05
      1

      4
      1.000
      0.0000
      35333.5
      1.010989
      0.999781
      -3282.649651
      0.080546
      0.05
      1

      5
      1.010
      1.2969
      2590.5
      0.505033
      0.720291
      7475.069122
      2.909367
      0.05
      1

      6
      1.375
      0.5484
      1384.7
      1.251146
      0.876761
      5318.837708
      1.421364
      0.05
      1

      7
      1.345
      1.1971
      291.5
      1.154194
      0.995257
      1016.922391
      0.338878
      0.05
      1

      8
      1.763
      3.4459
      10.2
      0.497526
      0.699698
      7306.713303
      1.415655
      0.05
      1

      9
      1.168
      3.6623
      31.6
      0.441019
      0.934793
      4777.161077
      1.161842
      0.05
      1

      10
      2.308
      0.2235
      34.0
      0.003836
      0.993789
      1066.920371
      0.361880
      0.05
      1

      11
      1.735
      14.4972
      23.0
      0.986735
      0.977436
      3291.500964
      0.711856
      0.10
      1

      12
      1.000
      0.0393
      29583.1
      1.008336
      0.799142
      6492.833368
      2.099084
      0.10
      1

      13
      1.999
      7.4211
      2.7
      1.393198
      0.998259
      171.074332
      0.162321
      0.10
      1

      14
      2.739
      0.7501
      3.3
      1.003822
      0.999518
      1309.262642
      0.302636
      0.10
      1

      15
      2.015
      8.2185
      1218.0
      0.001468
      0.707638
      7196.037268
      1.450688
      0.10
      1

      16
      1.000
      0.0353
      41440.1
      0.976178
      0.636774
      7525.263090
      2.955174
      0.10
      1

      17
      1.833
      9.2352
      4.7
      0.442542
      0.757043
      6118.781046
      1.620072
      0.10
      1

      18
      1.060
      3.0758
      400.1
      0.434032
      0.757816
      6632.806131
      2.164517
      0.10
      1

      19
      1.980
      11.4349
      2.5
      0.918732
      0.986219
      3106.783921
      0.520355
      0.10
      1

      20
      2.131
      9.1774
      7.5
      0.486471
      0.265548
      10582.633054
      2.771358
      0.10
      1

      21
      1.827
      1.6465
      20.7
      0.494267
      0.096097
      7580.319020
      1.563907
      0.25
      1

      22
      1.000
      0.0983
      11285.1
      0.861854
      0.779369
      7063.682095
      2.534114
      0.25
      1

      23
      1.015
      1.6263
      6300.3
      0.001088
      0.725909
      5757.857579
      1.571888
      0.25
      1

      24
      1.660
      3.1826
      4250.5
      0.992823
      0.932047
      7226.362048
      0.632385
      0.25
      1

      25
      2.199
      3.5816
      20.4
      0.531355
      0.964774
      3814.780677
      0.942123
      0.25
      1

      26
      1.455
      5.5307
      42.7
      0.001519
      0.757377
      6417.173372
      2.050267
      0.25
      1

      27
      1.938
      3.8205
      11.4
      0.514028
      0.513889
      8918.640812
      2.004500
      0.25
      1

      28
      2.002
      0.1386
      26.7
      0.016020
      0.989207
      1289.054065
      0.396308
      0.25
      1

      29
      1.000
      0.0008
      30679.2
      0.996755
      0.911607
      4596.484693
      1.114364
      0.25
      1

      30
      1.000
      0.1927
      14839.5
      0.917105
      0.960027
      3584.065631
      0.793608
      0.25
      1

```python
#suc = df.loc[(df['r_hat'] <= 1.1) & (df['r2'] >= 0.99)]
suc = df.loc[(df['r_hat'] <= 1.1) & (df['r2'] >= 0.99) & (df['run'].astype(str) == '1')]
suc
```

      r_hat
      mcse
      ess
      bfmi
      r2
      waic
      epsilon
      epsilon_real
      run

      4
      1.0
      0.0
      35333.5
      1.010989
      0.999781
      -3282.649651
      0.080546
      0.05
      1

```python
run = '1'
suc_l = df.loc[(df['r_hat'] <= 1.1) & (df['r2'] >= 0.99) & (df['run'].astype(str) == run)
               & (df['epsilon_real'].astype(str) == '0.05')]
suc_m = df.loc[(df['r_hat'] <= 1.1) & (df['r2'] >= 0.99) & (df['run'].astype(str) == run)
               & (df['epsilon_real'].astype(str) == '0.1')]
suc_h = df.loc[(df['r_hat'] <= 1.1) & (df['r2'] >= 0.99) & (df['run'].astype(str) == run)
               & (df['epsilon_real'].astype(str) == '0.25')]

print("1%: ", len(suc_l))
print("2%: ", len(suc_m))
print("5%: ", len(suc_h))
print("t : ", len(suc_l)+len(suc_m)+len(suc_h))
```

    1%:  1
    2%:  0
    5%:  0
    t :  1

```python

```


# 7. Gyakorlat - 1 DoF negyed-jármű modell
2021.03.22.

## Feladat:

```python
from IPython.display import Image
Image(filename='gyak7_1.png',width=600)
```

A mellékelt ábrán egy negyed-jármű modell látható, mellyel a jármű függőleges dinamikáját lehet vizsgáni. A gyakorlatban a lengéscsillapító merevségét és csillapítási tényezőjét az útviszonyoknak megfelelően, a komfort szempotokat kielégítve szokták beállítani. A negyed-jármű tömegét $m$, a lengéscsilapító egyenértékű merevségét $k$, csillapítási tényezőjét $c$ jelöli. A jármű longitudinális sebesség komponense $v$ konstans, míg a harmonikus útprofil miatt a függőleges sebesség komponenst az $r(t) = R\sin(\omega t)$ függvénnyel lehet megadni, ahol $R$ az út egyenetlenségének amplitúdója, $\omega$ pedig a gerjesztési frekvencia, melyet a $v$ sebesség és az $L$ hullámhossz határoz meg. A mozgás leírásáshoz az $y(t)$ általános koordinátát használjuk. A gravitációs hatást elhanyagoljuk és feltételezzük, hogy a kerék sosem emelkedik el a talajtól.

### Adatok:
|||
|---------------------------|---------------------------|
| $m$ = 300 kg              | $v$ = 36 km/h = 10 m/s    |
| $k$ = 2$\cdot$10$^5$ N/m  | $R$ = 0,04 m              |
| $c$ = 9800 Ns/m           | $L$ = 1,2 m               |
| $r(t)$ = $R\sin(\omega t)$|

### Részfeladatok:

1. Írja fel a negyed-jármű modell mozgásegyenletét!
2. Számítsa ki a csillapított és a csillapítatlan rendszer sajátkörfrekvenciáját, a relatív csillapítási tényezőt, a frekvencia hányadost és a statikus deformációt!
3. Határozza meg az állandósult állapotot leíró mozgástörvényt $y_p(t) = Y\sin(\omega t+\delta-\vartheta)$ alakban!
4. Határozza meg a stacionárius állapotban a csillapító erő maximumát ($F_{d,max}$)!

## Megoldás:

```python
from IPython.display import Image
Image(filename='gyak7_2.png',width=500)
```

## 1. Feladat:
A fenti ábrán a negyed-jármű modell szabadtest ábrája látható egy kitérített pozícióban. A mozgás egyenlet Newton II alapján:

\begin{equation}
    \dot{\mathbf{I}}=\mathbf{F}.
\end{equation}

Mivel most csak a függőleges mozgásra vagyunk kíváncsiak, ezért a mozgásegyenlet a következő alakban adódik

\begin{equation}
    m\ddot{y} = -F_s(t)-F_d(t),
\end{equation}

ahol $F_s(t)$ és $F_d(t)$ a rugóerő és a csillapítóerő

\begin{equation}
    F_s(t) = k(y-r(t)),
\end{equation}
\begin{equation}
    F_d(t) = c(\dot{y}-\dot{r}(t)).
\end{equation}

```python
import sympy as sp
from IPython.display import display, Math

sp.init_printing()
```

```python
## Függvények, szimbólumok definiálása

m, k, c, v, R, L, ω= sp.symbols("m, k, c, v, R, L, ω", real=True)

# Készítsünk behelyettesítési listát az adatok alapján, SI-ben
adatok = [(m, 300), (k, 2*10**5), (c, 9300), (v,10), (R, 0.04), (L, 1.2)]

#általános koordináta
t = sp.symbols("t",real=True, positive=True)
y = sp.Function('y')(t)

# Útgerjesztés
r = R*sp.sin(ω*t)

# Rugóerő, csillapítóerő
F_s = k*(y-r)
F_c = c*(y.diff(t)-r.diff(t))
display(Math('F_s = {}'.format(sp.latex(F_s))),Math('F_c = {}'.format(sp.latex(F_c))))
```

$\displaystyle F_s = k \left(- R \sin{\left(t ω \right)} + y{\left(t \right)}\right)$

$\displaystyle F_c = c \left(- R ω \cos{\left(t ω \right)} + \frac{d}{d t} y{\left(t \right)}\right)$

```python
# A gerjesztési frekvencia a hullámhosszal és a longitudinális sebességgel fejezhető ki

# periódusidő
T_g = L/v

# gerjesztési körfrekvencia
# Ennek a mennyiségnek hozzunk létre egy új változót és ne a korábban használt ω-t írjuk felül
# (Ez azért fontos, mert az ω szimbólum már szerepel az útgerjesztés kifejezésében és ott nem fog felülíródni)
ω_g = 2*sp.pi/T_g
display(Math('ω_g = {}'.format(sp.latex(ω_g))))

# Numerikusan
ω_g_num = ω_g.subs(adatok).evalf(6)
display(Math('ω_g = {}'.format(sp.latex(ω_g_num))))

# [rad/s]
```

$\displaystyle ω_g = \frac{2 \pi v}{L}$

$\displaystyle ω_g = 52.3599$

```python
# mozgásegyenlet nullára rendezve
mozgegy = (m*y.diff(t,2)+F_s+F_c).apart(y)
mozgegy
```

A mozgásegyenletben a gerjesztés az alább egyszerűsített alakra hozható:

\begin{equation}
    kR\sin(\omega t)+cR\omega\cos(\omega t) = F_0\sin(\omega t+\delta),
\end{equation}

ahol $\delta$ a fáziskésés és $F_0$ a gerjesztés erőamplitúdója. Trigonometrikus azonosságot alkalmazva ez a kifejezés kibontható:

\begin{equation}
    F_0\sin(\omega t+\delta) = F_0\sin(\omega t)\cos(\delta) + F_0\cos(\omega t)\sin(\delta),
\end{equation}

majd összevetve a gerjesztés eredeti alakjavál, az egyenlet két oldalán a szinusz és a koszinusz függvény együtthatói meg kell egyezzenek

\begin{equation}
    kR\sin(\omega t)+cR\omega\cos(\omega t) = F_0\sin(\omega t)\cos(\delta) + F_0\cos(\omega t)\sin(\delta),
\end{equation}

Így ebből $F_0$ és $\delta$ paraméterek meghatározhatók.

```python
δ, F_0 = sp.symbols("δ, F_0")
# A sympy jól ismert expand() függvénye a trigonometrikus azonosságokra nem alkalmazható.
# Szerencsére erre van egy külön függvény, az expand_trig()
# (Megjegyzés: ugyanúgy a simplify() sem alkalmazható trigonometrikus kifejezések egyszerűsétésére,
# abban az esetben a trigsimp() függvény nyújthat segítséget)
gerj_expand = sp.expand_trig(F_0*sp.sin(ω*t+δ)).expand()
gerj_expand
```

```python
# Most szedjük ki a mozgásegyenletből a gerjesztést
gerj = mozgegy.coeff(y,0) # y nulladik hatványának az együtthatója (a diffegyenlet inhomogenitását okozó tagja)
gerj
```

```python
# És most oldjuk meg az adódó egyenletrendszert az együtthatókra
egyutthatok = sp.solve([gerj.coeff(sp.sin(ω*t))-gerj_expand.coeff(sp.sin(ω*t)), gerj.coeff(sp.cos(ω*t))-gerj_expand.coeff(sp.cos(ω*t))],F_0,δ)
F_0_val = sp.ratsimp(egyutthatok[1][0]) # A két megoldás közül a pozitívat vesszük ki
δ_val = sp.simplify(egyutthatok[0][1])
display(Math('F_0 = {}'.format(sp.latex(F_0_val))), Math('δ = {}'.format(sp.latex(δ_val))))

# numerikusan
F_0_num = F_0_val.subs(ω, ω_g).subs(adatok).evalf(8)
δ_num = δ_val.subs(ω, ω_g).subs(adatok).evalf(5)
display(Math('F_0 = {}'.format(sp.latex(F_0_num))), Math('δ = {}'.format(sp.latex(δ_num))))

# [N]
# [rad]
```

$\displaystyle F_0 = \frac{R c^{2} ω^{2}}{k + \sqrt{c^{2} ω^{2} + k^{2}}} + R k$

$\displaystyle δ = - 2 \operatorname{atan}{\left(\frac{k - \sqrt{c^{2} ω^{2} + k^{2}}}{c ω} \right)}$

$\displaystyle F_0 = 21056.771$

$\displaystyle δ = 1.1811$

## 2. Feladat

```python
# Mozgásegyenlet leosztása a főegyütthatóval
mozgegy = (mozgegy/mozgegy.coeff(y.diff(t,2))).expand()

# csillapítatlan sajátkörfrekvencia
ω_n = sp.sqrt(mozgegy.coeff(y))

# frekvenciahányados
λ = ω_g/ω_n

# Relatív csillapítási tényező
ζ = mozgegy.coeff(y.diff(t))/(2*ω_n)

# csillapított sajátkörfrekvencia
ω_d = ω_n*sp.sqrt(1-ζ**2)

# statikus deformáció
f_0 = F_0_val/(ω_n**2*m)

## Numerikusan

ω_n_num = ω_n.subs(ω, ω_g).subs(adatok).evalf(6)
λ_num = λ.subs(ω, ω_g).subs(adatok).evalf(5)
ζ_num = ζ.subs(ω, ω_g).subs(adatok).evalf(4)
ω_d_num = ω_d.subs(ω, ω_g).subs(adatok).evalf(6)
f_0_num = f_0.subs(ω, ω_g).subs(adatok).evalf(4)

display(Math('ω_n= {}'.format(sp.latex(ω_n_num))), Math('λ = {}'.format(sp.latex(λ_num))))
display(Math('ζ = {}'.format(sp.latex(ζ_num))), Math('ω_d = {}'.format(sp.latex(ω_d_num))))
display(Math('f_0 = {}'.format(sp.latex(f_0_num))))

# [rad/s]
# [1]
# [1]
# [rad/s]
# [m]
```

$\displaystyle ω_n= 25.8199$

$\displaystyle λ = 2.0279$

$\displaystyle ζ = 0.6003$

$\displaystyle ω_d = 20.6499$

$\displaystyle f_0 = 0.1053$

## 3. Feladat
Az állandósult állapotban a rendszert leíró mozgástörvényt a differenciálegyenlet partikuláris megoldása adja, melyet az alábbi alakra lehet rendezni

\begin{equation}
     y_p(t) = Y_1\sin(\omega t+\delta_1-\vartheta_1),
\end{equation}

ahol $Y$ az állandósult állapotban a rezgés amplitúdó, $\vartheta$ az erőgerjesztéshez képesti fáziskésés, $\delta-\vartheta$ pedig az útgerjesztéshez képesti fáziskésés.

```python
# Az Y rezgésamplitúdó a nagyítási függvény segítségével fejezhető ki
# Nagyítási függvény
N = 1/(sp.sqrt((1-λ**2)**2+4*ζ**2*λ**2))
N_num = N.subs(adatok).evalf(4)
display(Math('N = {}'.format(sp.latex(N_num))))

## Rezgésamplitúdó
Y = N*f_0
Y_num = Y.subs(ω,ω_g).subs(adatok).evalf(3)
display(Math('Y = {}'.format(sp.latex(Y_num))))

# [1]
# [m]
```

$\displaystyle N = 0.2531$

$\displaystyle Y = 0.0266$

```python
# A fáziskésés
ϑ =  sp.atan2(2*ζ*λ,(1-λ**2))
ϑ_num = (ϑ).subs(adatok).evalf(5)
display(Math('ϑ = {}'.format(sp.latex(ϑ_num))))

# [rad]
```

$\displaystyle ϑ = 2.4777$

```python
# A partikuláris megoldás
yp =Y*sp.sin(ω*t+δ-ϑ)
yp_num = Y_num*sp.sin(ω*t+δ_num-ϑ_num)
display(Math('y_p = {}'.format(sp.latex(yp_num))))

# [m]
```

$\displaystyle y_p = 0.0266 \sin{\left(t ω - 1.2967 \right)}$

## 4. Feladat

A csillíptási erő az alábbi kondenzált formára hozható

\begin{equation}
    F_c(t) = c(\dot{y}_p(t)-\dot{r}(t)) = c \omega(Y\cos(\omega t+\delta-\vartheta)-R\cos(\omega t))
\end{equation}

\begin{equation}
    (1) \hspace{20pt} c\omega(Y\cos(\omega t+\delta-\vartheta)-R\cos(\omega t)) = c\omega Y^*\cos(\omega t+\delta^*)
\end{equation}

A csillapítási erő amplitúdója ($Y^*$) az első feladatban ismeretett módon most is meghatározható az (1) egyenlet trigonometrikus bővítésével, majd az együtthatókra adódó egyenletrendszer megoldássával. Nézzük meg hogyan lehetne megkeresni az erőmaximumot numerikus módszer segítségével. Ehhez először célszerű kirajzoltatni a függvényünket.

```python
# Csillapítási erő az állandósult állapotban
F_c_stac = F_c.subs(y, yp).simplify()

# numerikusan az idő függvényeként megadva
F_c_stac_num = (F_c_stac.subs(ω, ω_g).subs(δ, δ_num).subs(adatok)).evalf(5)
```

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
### Csillapítási erőidőfüggvényének ábrázolása

t_val = np.linspace(0,0.5,1000) # lista létrehozása a [0 ; 0,5] intervallum 1000 részre való bontásával
F_c_val = np.zeros(len(t_val)) # nulla lista létrehozása (ugyanannyi elemszámmal)
# for ciklus segítségével írjuk felül a nulla listában szereplő elemeket az adott x értékhez tartozó y értékekkel
for i in range(len(t_val)):
    F_c_val[i] = F_c_stac_num.subs(t,t_val[i]).evalf()

# rajzterület létrehozása
plt.figure(figsize=(30/2.54,25/2.54))

# függvény kirajzolása az x és y kordináta értékeket tartalmazó listák megadásásval
plt.plot(t_val,F_c_val,color='r',label=r'num_sim')

# tengelyek
axes = plt.gca()
axes.set_xlim([0,t_val[-1]])
axes.set_ylim([-25000, 25000])

# rácsozás
plt.grid()

# tengelyfeliratozás
plt.xlabel(r'$ t [s] $',fontsize=30)
plt.ylabel(r'$ F_c(t) [N] $',fontsize=30)

plt.show()
```

```python
# Az erőmaximum az erőértékeket tartalmazó listából könnyen kiszedhető a max() függvénnyel
F_c_max = np.around(max(F_c_val),decimals=2) # két tizedesre kerekítve

display(Math('F_{{c,max}} = {}'.format(sp.latex(F_c_max))))

# [N]
```

$\displaystyle F_{c,max} = 20270.5$

Készítette:
    Juhos-Kiss Álmos (Alkalmazott Mechanika Szakosztály)
    Takács Dénes (BME MM) kidolgozása és ábrái alapján.

        Hibák, javaslatok:
        amsz.bme@gmail.com
        csuzdi02@gmail.com
        almosjuhoskiss@gmail.com

            2021.03.22


```python

```

# Maintaining voltage setpoints via PandaModels.jl
### PandaModels.jl: Interfacing PowerModels with pandapower

This tutorial describes how to run the optimization for maintaining voltage setpoints via [PandaModels.jl](https://e2niee.github.io/PandaModels.jl/dev/).
Maintaining voltage setpoits (MVSP) is an optimization problem which is exclusively defined in [PandaModels.jl](https://e2niee.github.io/PandaModels.jl/dev/) and
needs the extra user defined parameters from pandapower.

The general mathematical model of MVSP is defined as follows:

\begin{align}
& \underset{\mathcal{X} = [q, ...]}{\text{minimize}}
& &  \sum_{i\in \mathcal{BI}} [v_i - setpoint\_v]^2 \\
& \text{subject to}
& & g(\mathcal{X})=0 \\
& & & h(\mathcal{X}) \leq 0
\end{align}

where $v_{i}$ is the voltage variable of bus $i$ in $\mathcal{BI}$ which denotes the set of buses located at the DSO-TSO interfaces. The $g(\mathcal{X})$ and $h(\mathcal{X})$, denote equality and inequality constraints, respectively. The $\mathcal{X}$ denotes the set of variables decisions, such as reactive power, $q$.

### Let's get started

So here is an example of how it works. First, we create cigre grid with pv and wind DERs(distribution energy resource) from pandapower's network database.

```python
import pandapower as pp
import pandapower.networks as nw
from copy import deepcopy

net = nw.create_cigre_network_mv(with_der="pv_wind")
```

then we need to run powerflow to get the initial values:

```python
net.sgen.p_mw = net.sgen.p_mw * 8
net.sgen.sn_mva = net.sgen.sn_mva * 8
pp.runpp(net)
```

Let's have a look at the grid we created with pandapower plotting module:

```python
import pandapower.plotting as plot
%matplotlib inline
plot.simple_plot(net)
```

let's keep a copy of the net for further comparision:

```python
net_org = deepcopy(net)
```

Although VD is not a function of PowerModels and is exclusively modeled in Pandamodels, PandaModels similar to PowerModels follows the data structures from [InfrastructureModels.jl](https://github.com/lanl-ansi/InfrastructureModels.jl).
 In this data structure DERs are not defined separately unlike pandapower,
 then the uncontrollable load and sgens are converted to load and controllable elements are converted to the generator.

Accordingly, we need to set loads as uncontrollable elements and set DERs (sgens) as controllable elements.

```python
net.load['controllable'] = False
net.sgen['controllable'] = True
```

now, lets set the optimization boundaries:

```python
# lower and upper bounds for buses
net.bus["max_vm_pu"] = 1.1
net.bus["min_vm_pu"] = 0.9

# lower and upper bounds for external grid
net.ext_grid["max_q_mvar"] = 10000.0
net.ext_grid["min_q_mvar"] = -10000.0
net.ext_grid["max_p_mw"] = 10000.0
net.ext_grid["min_p_mw"] = -10000.0
```

in VD we need to keep the active power value for DER and generators constant to the calculated value from power flow.

```python
# lower and upper bounds for DERs
net.sgen["max_p_mw"] = net.sgen.p_mw.values
net.sgen["min_p_mw"] = net.sgen.p_mw.values
net.sgen["max_q_mvar"] = net.sgen.p_mw.values * 0.328
net.sgen["min_q_mvar"] = -net.sgen.p_mw.values * 0.328

# lower and upper bounds for generators
net.gen["max_p_mw"] = net.gen.p_mw.values
net.gen["min_p_mw"] = net.gen.p_mw.values
net.gen["max_q_mvar"] = 10000.0
net.gen["min_q_mvar"] = -10000.0
```

Let's set a high upper bound for line and transformers to avoid congestion issue:

```python
# lower and upper bounds for lines
net.trafo["max_loading_percent"] = 500.0
net.line["max_loading_percent"] = 500.0
```

I also we could set costs for sgens, gens and external grids in order to not get warnings during data correctness in julia side.

```python
for idx in net.sgen.index:
    pp.create_poly_cost(net, idx, "sgen", 1.0)
for idx in net.gen.index:
    pp.create_poly_cost(net, idx, "gen", 1.0)
for idx in net.ext_grid.index:
    pp.create_poly_cost(net, idx, "ext_grid", 1.0)
```

Finally, lets add our user defined parameter "setpoint_v". at first we need to add extra column called "pm_param/setpoint_v" in bus data,
then we set values for buses contain DERs(sgens):

```python
net.bus["pm_param/setpoint_v"] = None
net.bus["pm_param/setpoint_v"].loc[net.sgen.bus] = 0.99
```

Now lets run an MVSP through PandaModels and look at the results (Note that the first time the function is called, Julia is started in the background, which may take some time):

```python
pp.runpm_vstab(net)
```

    hp.pandapower.opf.run_powermodels - INFO: Optimization ('run_pandamodels_vstab') is finished in 0.09 seconds:

Also, there more parameters and options that you can add as input while calling the Optimization Problem from PandaModles:

| parameter | description | type | default |
| :--- | :--- | :---: | :--- |
| correct_pm_network_data | checks if network data is correct. If not tries to correct it | bool | True |
| silence | Suppresses information and warning messages output by PowerModels | bool | True |
| pm_model | PowerModels.jl model to use | str | "ACPPowerModel" |
| pm_solver | "main" solver| str | "ipopt" |
| pm_mip_solver | mixed integer solver| str | "cbc" |
| pm_nl_solver | nonlinear solver| str | "ipopt" |
| pm_tol | default desired convergence tolerance for solver to use | float | 1e-8 |
| pm_log_level | solver log level in power models | int | 0 |
| delete_buffer_file | If True, the .json file used by PandaModels will be deleted after optimization. | bool | True |

lets check the result values for voltage and reactive power at buses contains DERs before and after VD:

```python
# results from power flow
net_org.res_bus.vm_pu[net_org.sgen.bus]
```

    3     1.039473
    4     1.038293
    5     1.037487
    6     1.036603
    8     1.053060
    9     1.052446
    10    1.051725
    11    1.051589
    7     1.075642
    Name: vm_pu, dtype: float64

```python
# results from VD
net.res_bus.vm_pu[net.sgen.bus]
```

    3     0.984793
    4     0.983774
    5     0.983079
    6     0.982361
    8     0.992054
    9     0.991521
    10    0.990934
    11    0.990804
    7     1.006539
    Name: vm_pu, dtype: float64

as exected the results are close to the defined setpoints, and as we set the active power constants we expect to get different results from power flow for reactive power:

```python
# results from normal power flow
net_org.res_sgen.q_mvar
```

    0    0.0
    1    0.0
    2    0.0
    3    0.0
    4    0.0
    5    0.0
    6    0.0
    7    0.0
    8    0.0
    Name: q_mvar, dtype: float64

```python
# results from MVSP
net.res_sgen.q_mvar

```

    0    0.052467
    1    0.052470
    2    0.078711
    3    0.078712
    4    0.078694
    5    0.078693
    6    0.104932
    7    0.026212
    8   -3.135065
    Name: q_mvar, dtype: float64

```python

```


     Histograms of Oriented Gradients (HOG)

Introduction

As we saw with the ORB algorithm, we can use keypoints in images to do keypoint-based matching to detect objects in images. These type of algorithms work great when you want to detect objects that have a lot of consistent internal features that are not affected by the background. For example, these algorithms work well for facial detection because faces have a lot of consistent internal features that don’t get affected by the image background, such as the eyes, nose, and mouth. However, these type of algorithms don’t work so well when attempting to do more general object recognition, say for example, pedestrian detection in images. The reason is that people don’t have consistent internal features, like faces do, because the body shape and style of every person is different (see Fig. 1). This means that every person is going to have a different set of internal features, and so we need something that can more generally describe a person.

Fig. 1. - Pedestrians.

One option is to try to detect pedestrians by their contours instead.  Detecting objects in images by their contours (boundaries) is very challenging because we have to deal with the difficulties brought about by the contrast between the background and the foreground. For example, suppose you wanted to detect a pedestrian in an image that is walking in front of a white building and she is wearing a white coat and black pants (see Fig. 2). We can see in Fig. 2, that since the background of the image is mostly white, the black pants are going to have a very high contrast, but the coat, since it is white as well, is going to have very low contrast. In this case, detecting the edges of pants is going to be easy but detecting the edges of the coat is going to be very difficult. This is where **HOG** comes in. HOG stands for **Histograms of Oriented Gradients** and it was first introduced by Navneet Dalal and Bill Triggs in 2005.

Fig. 2. - High and Low Contrast.

The HOG algorithm works by creating histograms of the distribution of gradient orientations in an image and then normalizing them in a very special way. This special normalization is what makes HOG so effective at detecting the edges of objects even in cases where the contrast is very low. These normalized histograms are put together into a feature vector, known as the HOG descriptor, that can be used to train a machine learning algorithm, such as a Support Vector Machine (SVM), to detect objects in images based on their boundaries (edges). Due to its great success and reliability, HOG has become one of the most widely used algorithms in computer vison for object detection.

In this notebook, you will learn:

* How the HOG algorithm works
* How to use OpenCV to create a HOG descriptor
* How to visualize the HOG descriptor.

# The HOG Algorithm

As its name suggests, the HOG algorithm, is based on creating histograms from the orientation of image gradients. The HOG algorithm is implemented in a series of steps:

1. Given the image of particular object, set a detection window (region of interest) that covers the entire object in the image (see Fig. 3).

2. Calculate the magnitude and direction of the gradient for each individual pixel in the detection window.

3. Divide the detection window into connected *cells* of pixels, with all cells being of the same size (see Fig. 3). The size of the cells is a free parameter and it is usually chosen so as to match the scale of the features that want to be detected. For example, in a 64 x 128 pixel detection window, square cells 6 to 8 pixels wide are suitable for detecting human limbs.

4. Create a Histogram for each cell, by first grouping the gradient directions of all pixels in each cell into a particular number of orientation (angular) bins; and then adding up the gradient magnitudes of the gradients in each angular bin (see Fig. 3). The number of bins in the histogram is a free parameter and it is usually set to 9 angular bins.

5. Group adjacent cells into *blocks* (see Fig. 3). The number of cells in each block is a free parameter and all blocks must be of the same size. The distance between each block (known as the stride) is a free parameter but it is usually set to half the block size, in which case you will get overlapping blocks (*see video below*).  The HOG algorithm has been shown empirically to work better with overlapping blocks.

6. Use the cells contained within each block to normalize the cell histograms in that block (see Fig. 3). If you have overlapping blocks this means that most cells will be normalized with respect to different blocks (*see video below*). Therefore, the same cell may have several different normalizations.

7. Collect all the normalized histograms from all the blocks into a single feature vector called the HOG descriptor.

8. Use the resulting HOG descriptors from many images of the same type of object to train a machine learning algorithm, such as an SVM, to detect those type of objects in images. For example, you could use the HOG descriptors from many images of pedestrians to train an SVM to detect pedestrians in images. The training is done with both positive a negative examples of the object you want detect in the image.

9. Once the SVM has been trained, a sliding window approach is used to try to detect and locate objects in images.  Detecting an object in the image entails finding the part of the image that looks similar to the HOG pattern learned by the SVM.

Fig. 3. - HOG Diagram.

Vid. 1. - HOG Animation.

# Why The HOG Algorithm Works

As we learned above, HOG creates histograms by adding the magnitude of the gradients in particular orientations in localized portions of the image called *cells*.  By doing this we guarantee that stronger gradients will contribute more to the magnitude of their respective angular bin, while the effects of weak and randomly oriented gradients resulting from noise are minimized. In this manner the histograms tell us the dominant gradient orientation of each cell.

### Dealing with contrast

Now, the magnitude of the dominant orientation can vary widely due to variations in local illumination and the contrast between the background and the foreground.

To account for the background-foreground contrast differences, the HOG algorithm tries to detect edges locally.  In order to do this, it defines groups of cells, called **blocks**, and normalizes the histograms using this local group of cells. By normalizing locally, the HOG algorithm can detect the edges in each block very reliably; this is called **block normalization**.

In addition to using block normalization, the HOG algorithm also uses overlapping blocks to increase its performance. By using overlapping blocks, each cell contributes several independent components to the final HOG descriptor, where each component corresponds to a cell being normalized with respect to a different block. This may seem redundant but, it has been shown empirically that by normalizing each cell several times with respect to different local blocks, the performance of the HOG algorithm increases dramatically.

### Loading Images and Importing Resources

The first step in building our HOG descriptor is to load the required packages into Python and to load our image.

We start by using OpenCV to load an image of a triangle tile. Since, the `cv2.imread()` function loads images as BGR we will convert our image to RGB so we can display it with the correct colors. As usual we will convert our BGR image to Gray Scale for analysis.

```python
import cv2
import numpy as np
import matplotlib.pyplot as plt

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Load the image
image = cv2.imread('./images/triangle_tile.jpeg')

# Convert the original image to RGB
original_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Convert the original image to gray scale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Print the shape of the original and gray scale images
print('The original image has shape: ', original_image.shape)
print('The gray scale image has shape: ', gray_image.shape)

# Display the images
plt.subplot(121)
plt.imshow(original_image)
plt.title('Original Image')
plt.subplot(122)
plt.imshow(gray_image, cmap='gray')
plt.title('Gray Scale Image')
plt.show()
```

    The original image has shape:  (250, 250, 3)
    The gray scale image has shape:  (250, 250)

    <matplotlib.figure.Figure at 0x7f01c77d3e10>

# Creating The HOG Descriptor

We will be using OpenCV’s `HOGDescriptor` class to create the HOG descriptor. The parameters of the HOG descriptor are setup using the `HOGDescriptor()` function. The parameters of the `HOGDescriptor()` function and their default values are given below:

`cv2.HOGDescriptor(win_size = (64, 128),
                  block_size = (16, 16),
                  block_stride = (8, 8),
                  cell_size = (8, 8),
                  nbins = 9,
                  win_sigma = DEFAULT_WIN_SIGMA,
                  threshold_L2hys = 0.2,
                  gamma_correction = true,
                  nlevels = DEFAULT_NLEVELS)`

Parameters:

* **win_size** – *Size*
Size of detection window in pixels (*width, height*). Defines the region of interest. Must be an integer multiple of cell size.

* **block_size** – *Size*
Block size in pixels (*width, height*). Defines how many cells are in each block. Must be an integer multiple of cell size and it must be smaller than the detection window. The smaller the block the finer detail you will get.

* **block_stride** – *Size*
Block stride in pixels (*horizontal, vertical*). It must be an integer multiple of cell size. The `block_stride` defines the distance between adjecent blocks, for example, 8 pixels horizontally and 8 pixels vertically. Longer `block_strides` makes the algorithm run faster (because less blocks are evaluated) but the algorithm may not perform as well.

* **cell_size** – *Size*
Cell size in pixels (*width, height*). Determines the size fo your cell. The smaller the cell the finer detail you will get.

* **nbins** – *int*
Number of bins for the histograms. Determines the number of angular bins used to make the histograms. With more bins you capture more gradient directions. HOG uses unsigned gradients, so the angular bins will have values between 0 and 180 degrees.

* **win_sigma** – *double*
Gaussian smoothing window parameter. The performance of the HOG algorithm can be improved by smoothing the pixels near the edges of the blocks by applying a Gaussian spatial window to each pixel before computing the histograms.

* **threshold_L2hys** – *double*
L2-Hys (Lowe-style clipped L2 norm) normalization method shrinkage. The L2-Hys method is used to normalize the blocks and it consists of an L2-norm followed by clipping and a renormalization. The clipping limits the maximum value of the descriptor vector for each block to have the value of the given threshold (0.2 by default). After the clipping the descriptor vector is renormalized as described in *IJCV*, 60(2):91-110, 2004.

* **gamma_correction** – *bool*
Flag to specify whether the gamma correction preprocessing is required or not. Performing gamma correction slightly increases the performance of the HOG algorithm.

* **nlevels** – *int*
Maximum number of detection window increases.

As we can see, the  `cv2.HOGDescriptor()`function supports a wide range of parameters. The first few arguments (`block_size, block_stride, cell_size`, and `nbins`) are probably the ones you are most likely to change. The other parameters can be safely left at their default values and you will get good results.

In the code below, we will use the `cv2.HOGDescriptor()`function to set the cell size, block size, block stride, and the number of bins for the histograms of the HOG descriptor. We will then use `.compute(image)`method to compute the HOG descriptor (feature vector) for the given `image`.

```python
# Specify the parameters for our HOG descriptor

# Cell Size in pixels (width, height). Must be smaller than the size of the detection window
# and must be chosen so that the resulting Block Size is smaller than the detection window.
cell_size = (6, 6)

# Number of cells per block in each direction (x, y). Must be chosen so that the resulting
# Block Size is smaller than the detection window
num_cells_per_block = (2, 2)

# Block Size in pixels (width, height). Must be an integer multiple of Cell Size.
# The Block Size must be smaller than the detection window
block_size = (num_cells_per_block[0] * cell_size[0],
              num_cells_per_block[1] * cell_size[1])

# Calculate the number of cells that fit in our image in the x and y directions
x_cells = gray_image.shape[1] // cell_size[0]
y_cells = gray_image.shape[0] // cell_size[1]

# Horizontal distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (x_cells - num_cells_per_block[0]) / h_stride = integer.
h_stride = 1

# Vertical distance between blocks in units of Cell Size. Must be an integer and it must
# be set such that (y_cells - num_cells_per_block[1]) / v_stride = integer.
v_stride = 1

# Block Stride in pixels (horizantal, vertical). Must be an integer multiple of Cell Size
block_stride = (cell_size[0] * h_stride, cell_size[1] * v_stride)

# Number of gradient orientation bins
num_bins = 9

# Specify the size of the detection window (Region of Interest) in pixels (width, height).
# It must be an integer multiple of Cell Size and it must cover the entire image. Because
# the detection window must be an integer multiple of cell size, depending on the size of
# your cells, the resulting detection window might be slightly smaller than the image.
# This is perfectly ok.
win_size = (x_cells * cell_size[0] , y_cells * cell_size[1])

# Print the shape of the gray scale image for reference
print('\nThe gray scale image has shape: ', gray_image.shape)
print()

# Print the parameters of our HOG descriptor
print('HOG Descriptor Parameters:\n')
print('Window Size:', win_size)
print('Cell Size:', cell_size)
print('Block Size:', block_size)
print('Block Stride:', block_stride)
print('Number of Bins:', num_bins)
print()

# Set the parameters of the HOG descriptor using the variables defined above
hog = cv2.HOGDescriptor(win_size, block_size, block_stride, cell_size, num_bins)

# Compute the HOG Descriptor for the gray scale image
hog_descriptor = hog.compute(gray_image)
```

    The gray scale image has shape:  (250, 250)

    HOG Descriptor Parameters:

    Window Size: (246, 246)
    Cell Size: (6, 6)
    Block Size: (12, 12)
    Block Stride: (6, 6)
    Number of Bins: 9

# Number of Elements In The HOG Descriptor

The resulting HOG Descriptor (feature vector), contains the normalized histograms from all cells from all blocks in the detection window concatenated in one long vector. Therefore, the size of the HOG feature vector will be given by the total number of blocks in the detection window, multiplied by the number of cells per block, times the number of orientation bins:

\begin{equation}
\mbox{total_elements} = (\mbox{total_number_of_blocks})\mbox{ } \times \mbox{ } (\mbox{number_cells_per_block})\mbox{ } \times \mbox{ } (\mbox{number_of_bins})
\end{equation}

If we don’t have overlapping blocks (*i.e.* the `block_stride`equals the `block_size`), the total number of blocks can be easily calculated by dividing the size of the detection window by the block size. However, in the general case we have to take into account the fact that we have overlapping blocks. To find the total number of blocks in the general case (*i.e.* for any `block_stride` and `block_size`), we can use the formula given below:

\begin{equation}
\mbox{Total}_i = \left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right)\left( \frac{\mbox{window_size}_i}{\mbox{block_size}_i} \right) - \left [\left( \frac{\mbox{block_size}_i}{\mbox{block_stride}_i} \right) - 1 \right]; \mbox{  for  } i = x,y
\end{equation}

Where Total$_x$, is the total number of blocks along the width of the detection window, and Total$_y$, is the total number of blocks along the height of the detection window. This formula for Total$_x$ and Total$_y$, takes into account the extra blocks that result from overlapping. After calculating Total$_x$ and Total$_y$, we can get the total number of blocks in the detection window by multiplying Total$_x$  $\times$ Total$_y$. The above formula can be simplified considerably because the `block_size`, `block_stride`, and `window_size`are all defined in terms of the `cell_size`. By making all the appropriate substitutions and cancelations the above formula reduces to:

\begin{equation}
\mbox{Total}_i = \left(\frac{\mbox{cells}_i - \mbox{num_cells_per_block}_i}{N_i}\right) + 1\mbox{  }; \mbox{  for  } i = x,y
\end{equation}

Where cells$_x$ is the total number of cells along the width of the detection window, and cells$_y$, is the total number of cells along the height of the detection window. And $N_x$ is the horizontal block stride in units of `cell_size` and $N_y$ is the vertical block stride in units of `cell_size`.

Let's calculate what the number of elements for the HOG feature vector should be and check that it matches the shape of the HOG Descriptor calculated above.

```python
# Calculate the total number of blocks along the width of the detection window
tot_bx = np.uint32(((x_cells - num_cells_per_block[0]) / h_stride) + 1)

# Calculate the total number of blocks along the height of the detection window
tot_by = np.uint32(((y_cells - num_cells_per_block[1]) / v_stride) + 1)

# Calculate the total number of elements in the feature vector
tot_els = (tot_bx) * (tot_by) * num_cells_per_block[0] * num_cells_per_block[1] * num_bins

# Print the total number of elements the HOG feature vector should have
print('\nThe total number of elements in the HOG Feature Vector should be: ',
      tot_bx, 'x',
      tot_by, 'x',
      num_cells_per_block[0], 'x',
      num_cells_per_block[1], 'x',
      num_bins, '=',
      tot_els)

# Print the shape of the HOG Descriptor to see that it matches the above
print('\nThe HOG Descriptor has shape:', hog_descriptor.shape)
print()
```

    The total number of elements in the HOG Feature Vector should be:  40 x 40 x 2 x 2 x 9 = 57600

    The HOG Descriptor has shape: (57600, 1)

# Visualizing The HOG Descriptor

We can visualize the HOG Descriptor by plotting the histogram associated with each cell as a collection of vectors. To do this, we will plot each bin in the histogram as a single vector whose magnitude is given by the height of the bin and its orientation is given by the angular bin that its associated with. Since any given cell might have multiple histograms associated with it, due to the overlapping blocks, we will choose to average all the histograms for each cell to produce a single histogram for each cell.

OpenCV has no easy way to visualize the HOG Descriptor, so we have to do some manipulation first in order to visualize it. We will start by reshaping the HOG Descriptor in order to make our calculations easier.  We will then compute the average histogram of each cell and finally we will convert the histogram bins into vectors. Once we have the vectors, we plot the corresponding vectors for each cell in an image.

The code below produces an interactive plot so that you can interact with the figure. The figure contains:
* the grayscale image,
* the HOG Descriptor (feature vector),
* a zoomed-in portion of the HOG Descriptor, and
* the histogram of the selected cell.

**You can click anywhere on the gray scale image or the HOG Descriptor image to select a particular cell**. Once you click on either image a *magenta* rectangle will appear showing the cell you selected. The Zoom Window will show you a zoomed in version of the HOG descriptor around the selected cell; and the histogram plot will show you the corresponding histogram for the selected cell. The interactive window also has buttons at the bottom that allow for other functionality, such as panning, and giving you the option to save the figure if desired. The home button returns the figure to its default value.

**NOTE**: If you are running this notebook in the Udacity workspace, there is around a 2 second lag in the interactive plot. This means that if you click in the image to zoom in, it will take about 2 seconds for the plot to refresh.

```python
%matplotlib notebook

import copy
import matplotlib.patches as patches

# Set the default figure size
plt.rcParams['figure.figsize'] = [9.8, 9]

# Reshape the feature vector to [blocks_y, blocks_x, num_cells_per_block_x, num_cells_per_block_y, num_bins].
# The blocks_x and blocks_y will be transposed so that the first index (blocks_y) referes to the row number
# and the second index to the column number. This will be useful later when we plot the feature vector, so
# that the feature vector indexing matches the image indexing.
hog_descriptor_reshaped = hog_descriptor.reshape(tot_bx,
                                                 tot_by,
                                                 num_cells_per_block[0],
                                                 num_cells_per_block[1],
                                                 num_bins).transpose((1, 0, 2, 3, 4))

# Print the shape of the feature vector for reference
print('The feature vector has shape:', hog_descriptor.shape)

# Print the reshaped feature vector
print('The reshaped feature vector has shape:', hog_descriptor_reshaped.shape)

# Create an array that will hold the average gradients for each cell
ave_grad = np.zeros((y_cells, x_cells, num_bins))

# Print the shape of the ave_grad array for reference
print('The average gradient array has shape: ', ave_grad.shape)

# Create an array that will count the number of histograms per cell
hist_counter = np.zeros((y_cells, x_cells, 1))

# Add up all the histograms for each cell and count the number of histograms per cell
for i in range (num_cells_per_block[0]):
    for j in range(num_cells_per_block[1]):
        ave_grad[i:tot_by + i,
                 j:tot_bx + j] += hog_descriptor_reshaped[:, :, i, j, :]

        hist_counter[i:tot_by + i,
                     j:tot_bx + j] += 1

# Calculate the average gradient for each cell
ave_grad /= hist_counter

# Calculate the total number of vectors we have in all the cells.
len_vecs = ave_grad.shape[0] * ave_grad.shape[1] * ave_grad.shape[2]

# Create an array that has num_bins equally spaced between 0 and 180 degress in radians.
deg = np.linspace(0, np.pi, num_bins, endpoint = False)

# Each cell will have a histogram with num_bins. For each cell, plot each bin as a vector (with its magnitude
# equal to the height of the bin in the histogram, and its angle corresponding to the bin in the histogram).
# To do this, create rank 1 arrays that will hold the (x,y)-coordinate of all the vectors in all the cells in the
# image. Also, create the rank 1 arrays that will hold all the (U,V)-components of all the vectors in all the
# cells in the image. Create the arrays that will hold all the vector positons and components.
U = np.zeros((len_vecs))
V = np.zeros((len_vecs))
X = np.zeros((len_vecs))
Y = np.zeros((len_vecs))

# Set the counter to zero
counter = 0

# Use the cosine and sine functions to calculate the vector components (U,V) from their maginitudes. Remember the
# cosine and sine functions take angles in radians. Calculate the vector positions and magnitudes from the
# average gradient array
for i in range(ave_grad.shape[0]):
    for j in range(ave_grad.shape[1]):
        for k in range(ave_grad.shape[2]):
            U[counter] = ave_grad[i,j,k] * np.cos(deg[k])
            V[counter] = ave_grad[i,j,k] * np.sin(deg[k])

            X[counter] = (cell_size[0] / 2) + (cell_size[0] * i)
            Y[counter] = (cell_size[1] / 2) + (cell_size[1] * j)

            counter = counter + 1

# Create the bins in degress to plot our histogram.
angle_axis = np.linspace(0, 180, num_bins, endpoint = False)
angle_axis += ((angle_axis[1] - angle_axis[0]) / 2)

# Create a figure with 4 subplots arranged in 2 x 2
fig, ((a,b),(c,d)) = plt.subplots(2,2)

# Set the title of each subplot
a.set(title = 'Gray Scale Image\n(Click to Zoom)')
b.set(title = 'HOG Descriptor\n(Click to Zoom)')
c.set(title = 'Zoom Window', xlim = (0, 18), ylim = (0, 18), autoscale_on = False)
d.set(title = 'Histogram of Gradients')

# Plot the gray scale image
a.imshow(gray_image, cmap = 'gray')
a.set_aspect(aspect = 1)

# Plot the feature vector (HOG Descriptor)
b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
b.invert_yaxis()
b.set_aspect(aspect = 1)
b.set_facecolor('black')

# Define function for interactive zoom
def onpress(event):

    #Unless the left mouse button is pressed do nothing
    if event.button != 1:
        return

    # Only accept clicks for subplots a and b
    if event.inaxes in [a, b]:

        # Get mouse click coordinates
        x, y = event.xdata, event.ydata

        # Select the cell closest to the mouse click coordinates
        cell_num_x = np.uint32(x / cell_size[0])
        cell_num_y = np.uint32(y / cell_size[1])

        # Set the edge coordinates of the rectangle patch
        edgex = x - (x % cell_size[0])
        edgey = y - (y % cell_size[1])

        # Create a rectangle patch that matches the the cell selected above
        rect = patches.Rectangle((edgex, edgey),
                                  cell_size[0], cell_size[1],
                                  linewidth = 1,
                                  edgecolor = 'magenta',
                                  facecolor='none')

        # A single patch can only be used in a single plot. Create copies
        # of the patch to use in the other subplots
        rect2 = copy.copy(rect)
        rect3 = copy.copy(rect)

        # Update all subplots
        a.clear()
        a.set(title = 'Gray Scale Image\n(Click to Zoom)')
        a.imshow(gray_image, cmap = 'gray')
        a.set_aspect(aspect = 1)
        a.add_patch(rect)

        b.clear()
        b.set(title = 'HOG Descriptor\n(Click to Zoom)')
        b.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 5)
        b.invert_yaxis()
        b.set_aspect(aspect = 1)
        b.set_facecolor('black')
        b.add_patch(rect2)

        c.clear()
        c.set(title = 'Zoom Window')
        c.quiver(Y, X, U, V, color = 'white', headwidth = 0, headlength = 0, scale_units = 'inches', scale = 1)
        c.set_xlim(edgex - cell_size[0], edgex + (2 * cell_size[0]))
        c.set_ylim(edgey - cell_size[1], edgey + (2 * cell_size[1]))
        c.invert_yaxis()
        c.set_aspect(aspect = 1)
        c.set_facecolor('black')
        c.add_patch(rect3)

        d.clear()
        d.set(title = 'Histogram of Gradients')
        d.grid()
        d.set_xlim(0, 180)
        d.set_xticks(angle_axis)
        d.set_xlabel('Angle')
        d.bar(angle_axis,
              ave_grad[cell_num_y, cell_num_x, :],
              180 // num_bins,
              align = 'center',
              alpha = 0.5,
              linewidth = 1.2,
              edgecolor = 'k')

        fig.canvas.draw()

# Create a connection between the figure and the mouse click
fig.canvas.mpl_connect('button_press_event', onpress)
plt.show()
```

    The feature vector has shape: (57600, 1)
    The reshaped feature vector has shape: (40, 40, 2, 2, 9)
    The average gradient array has shape:  (41, 41, 9)

    <IPython.core.display.Javascript object>

# Understanding The Histograms

Let's take a look at a couple of snapshots of the above figure to see if the histograms for the selected cell make sense. Let's start looking at a cell that is inside a triangle and not near an edge:

Fig. 4. - Histograms Inside a Triangle.

In this case, since the triangle is nearly all of the same color there shouldn't be any dominant gradient in the selected cell. As we can clearly see in the Zoom Window and the histogram, this is indeed the case. We have many gradients but none of them clearly dominates over the other.

Now let’s take a look at a cell that is near a horizontal edge:

Fig. 5. - Histograms Near a Horizontal Edge.

Remember that edges are areas of an image where the intensity changes abruptly. In these cases, we will have a high intensity gradient in some particular direction. This is exactly what we see in the corresponding histogram and Zoom Window for the selected cell.  In the Zoom Window, we can see that the dominant gradient is pointing up, almost at 90 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 90-degree bin in the histogram to dominate strongly over the others. This is in fact what we see.

Now let’s take a look at a cell that is near a vertical edge:

Fig. 6. - Histograms Near a Vertical Edge.

In this case we expect the dominant gradient in the cell to be horizontal, close to 180 degrees, since that’s the direction in which there is a sharp change in intensity. Therefore, we should expect to see the 170-degree bin in the histogram to dominate strongly over the others. This is what we see in the histogram but we also see that there is another dominant gradient in the cell, namely the one in the 10-degree bin. The reason for this, is because the HOG algorithm is using unsigned gradients, which means 0 degrees and 180 degrees are considered the same. Therefore, when the histograms are being created, angles between 160 and 180 degrees, contribute proportionally to both the 10-degree bin and the 170-degree bin. This results in there being two dominant gradients in the cell near the vertical edge instead of just one.

To conclude let’s take a look at a cell that is near a diagonal edge.

Fig. 7. - Histograms Near a Diagonal Edge.

To understand what we are seeing, let’s first remember that gradients have an *x*-component, and a *y*-component, just like vectors. Therefore, the resulting orientation of a gradient is going to be given by the vector sum of its components. For this reason, on vertical edges the gradients are horizontal, because they only have an x-component, as we saw in Figure 4. While on horizontal edges the gradients are vertical, because they only have a y-component, as we saw in Figure 3. Consequently, on diagonal edges, the gradients are also going to be diagonal because both the *x* and *y* components are non-zero.  Since the diagonal edges in the image are close to 45 degrees, we should expect to see a dominant gradient orientation in the 50-degree bin. This is in fact what we see in the histogram but, just like in Figure 4., we see there are two dominant gradients instead of just one. The reason for this is that when the histograms are being created, angles that are near the boundaries of bins, contribute proportionally to the adjacent bins.  For example, a gradient with an angle of 40 degrees, is right in the middle of the 30-degree and 50-degree bin. Therefore, the magnitude of the gradient is split evenly into the 30-degree and 50-degree bin. This results in there being two dominant gradients in the cell near the diagonal edge instead of just one.

Now that you know how HOG is implemented, in the workspace you will find a notebook named *Examples*. In there, you will be able set your own paramters for the HOG descriptor for various images. Have fun!

```python

```


```python
import numpy as np
from sympy import Matrix
import copy
import math
import importlib.util
try:
	import thewalrus
except:
	print("Unable to import `thewalrus`. Using (slower) permanent backup function." )

```

    Unable to import `thewalrus`. Using (slower) permanent backup function.

# Simulating an Interferometer.

Simulating the quantum state evolution through an interferometer is simulating boson sampling, and as such cannot be done efficiently classically.

The simplest example of boson sampling is that of simulating a beamsplitter. It's instructive to look at this example in detail as it highlights why one must move to the second quantized picture (Fock space), as the first quantised picture fails to accurately predict the correct behaviour.

## Naive first quantised beamsplitter treatment:

Consider the case shown in the diagram here of two incident photons on a beamsplitter. In the first quantised picture we would multiply the input state by the unitary of the beamsplitter:

\begin{equation}
\begin{pmatrix} a_{out} \\ b_{out} \end{pmatrix}= \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & i \\ i & 1  \end{pmatrix} \begin{pmatrix} a_{in} \\ b_{in} \end{pmatrix}
\end{equation}

For the case of $\begin{pmatrix} a_{in} \\ b_{in} \end{pmatrix} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$, this gives$\begin{pmatrix} a_{out} \\ b_{out} \end{pmatrix} =  \frac{1}{\sqrt{2}} \begin{pmatrix} 1+i \\ 1+i \end{pmatrix}$

So the first quantised picture gives us this superposition state as an output, but fails to capture the full behaviour as it does not properly deal with interference effects. In this case, it results in the failure to describe a **HOM dip**.

## Second quantised beamsplitter treatment:
*For an intro to Fock space see pages 9-15 of https://arxiv.org/pdf/1812.10732.pdf* [1]

In the second quantised picture every mode is described by a simple harmonic oscillator each with infinite occupation levels. When working in the second quantised picture we move away from describing transforms on states and instead work out how the creation **$\hat{a}^\dagger$** and annihilation **$\hat{a}$** operators of each mode transform under the action of the given unitary.
<div>

</div>

In the case of the general 2x2 unitary above, the creation and annihilation operators for each mode transform under the linear mapping:
\begin{equation}
\begin{bmatrix} \hat{a}_{2} \\ \hat{a}_{3} \end{bmatrix}= \frac{1}{\sqrt{2}}\begin{bmatrix} t' & r \\ r' & t  \end{bmatrix} \begin{bmatrix} \hat{a_0} \\ \hat{a_1} \end{bmatrix}
\end{equation}

Where $ \hat{a}_{2}, \hat{a}_{3}$ are the output modes. Subbing in the unitary for a 50:50 beamsplitter we can calculate how the modes transform in the case of two input photons, as in the first part.

\begin{align}
\hat{a_0}^\dagger \hat{a_1}^\dagger \ket{0_{a0}0_{a1}} \rightarrow (\hat{a_2}^\dagger+ i\hat{a_3}^\dagger)(\hat{a_3}^\dagger+ i\hat{a_2}^\dagger)\ket{0_{a2}0_{a3}} \\
=\frac{i}{2}(\hat{a_2}^\dagger\hat{a_2}^\dagger+\hat{a_3}^\dagger\hat{a_3}^\dagger)\ket{0_{a2}0_{a3}} \\
=\frac{i}{2}(\ket{2_{a2}0_{a3}}+\ket{0_{a2}2_{a3}})
\end{align}

When inputting two photons into a two port interferometer like this there are in theory 3 possible photon number preserving outcomes. $\ket{2 0}, \ket{0 2} , \ket{1 1}$. The above is telling us that in a 50:50 beamsplitter, the interference effects are such that we never see the $\ket{1 1}$ case.

## Generalising to an arbritrary interferometer:

*Follows the notation and arguments of:https://arxiv.org/pdf/quant-ph/0406127.pdf* [2]

### Notation
Ultimatley what we want to calculate is the effect of our interferometer on a given input fock state $ \ket{n_1, n_2, ...,n_N}$, and return some output quantum state.

Let our generalised interferomter be represented by the operator $ \hat{U}$, with associated unitary matrix $ \Lambda $. Hence the fock space mode transformations are:
\begin{equation}
\hat{a} \rightarrow \Lambda^\dagger \hat{a} \hspace{3em} \hat{a}^\dagger \rightarrow \Lambda^T \hat{a}^\dagger
\end{equation}

Here it is also relevant to introduce some more obscure notation taken verbatim from [2]:

"Let $\Lambda[k_1, . . . , k_m|l_1, . . . , l_m]$ be the (m × m)- matrix whose matrix elements are those of the original matrix Λ with row indices $k_1, . . . , k_m$ and column indices $l_1, . . . , l_m$. For example:

\begin{equation}
\Lambda[k_1, . . . , k_m|l_1, . . . , l_m]= \begin{pmatrix} \Lambda_{k_1l_1} & \Lambda_{k_1l_2} & \Lambda_{k_1l_3} \\ \Lambda_{k_2l_1} & \Lambda_{k_2l_2} & \Lambda_{k_2l_3} \\ \Lambda_{k_3l_1} & \Lambda_{k_3l_2} & \Lambda_{k_3l_3}  \end{pmatrix}
\end{equation}

The object $\Lambda[1^{m1},2^{m2} . . . )|1^{n1},2^{n2} . . . ]$ denotes a matrix whose entries are taken from the matrix $\Lambda$ and whose row index i occurs exactly $m_i$ times and whose column index j occurs exactly $n_j$ times, for example:
\begin{equation}
\Lambda[1^{1},2^{1},3^{1} )|1^{0},2^{2},3^{1} ]= \begin{pmatrix} \Lambda_{12} & \Lambda_{12} & \Lambda_{13} \\ \Lambda_{22} & \Lambda_{22} & \Lambda_{23} \\ \Lambda_{32} & \Lambda_{32} & \Lambda_{33}  \end{pmatrix}
\end{equation}
"

### Caculating the output state

For a state with N input modes, there are N modes to transform. The creation operators being transformed are applied $n_i$ times, where $n_i$ is the occupation number of each mode. The output state obiously also depends on the relevant entires of the unitary $\Lambda$. With this in mind it is hopefully clear that the state after transformation can be written as:

\begin{equation}
\hat{U}\ket{n_1, n_2, ...,n_N} = \prod_{i=1}^N \frac{1}{\sqrt{n_i!}} \left(  \sum^N_{k_i=1} \Lambda_{ki,}i\hat{a_{ki}}^\dagger\right)
\end{equation}

With a significant amount of algebra (shown in above reference), one can show that the transition amplitude from the input state to a given output state $ \ket{m_1, m_2, ...,m_N}$ is found via:

\begin{equation}
\bra{m_1, m_2, ...,m_N}\hat{U}\ket{n_1, n_2, ...,n_N} = \left( \prod_i n_i!\right)^{-\frac{1}{\sqrt{2}}} \left( \prod_i n_i!\right)^{-\frac{1}{\sqrt{2}}} per(\Lambda[\Omega'][\Omega])
\end{equation}

Here we have introduced:
\begin{align}
\Omega= (1^{n_1},2^{n_2},...,N^{n_N}) \\
\Omega'=(1^{m_1},2^{m_2},...,N^{m_N}
\end{align}

The two product coefficients within the the equation are just combinatorical factors which can be found directly from the state transition being looked at. The term $per(\Lambda[\Omega'][\Omega])$ requires slightly more peicing apart. Following from the definitions in the notation section, $\Lambda[\Omega'][\Omega]$ is a rerformulation of the matrix of the unitary operator $\Lambda$, with several replacements made so that it describes the transition from an individual term of the input vector to a specific output state being looked at. By taking the permanent of this *transition matrix*, we can calculate the RHS of the equation. For a given output state, we must take find the product of the permanent of the transition matrix corresponding to every term of the input state. The mod^2 of this then gives us the probability of this transition occcuring. This is perhaps seen more clearly in the example below.

So we have a method of calculating the transition amplitude from our input state to *a given output state $ \ket{m_1, m_2, ...,m_N}$*. In order to calculate the full output state we must evaluate the transition amplitude from our input state to every possible output state. In order to do this we enforce photon number conservation by limiting the maximum occupation of each output mode to the total number of input photons across all input modes.

### Coded example with explicit steps

Lets simulate a HOM dip using the above formalism. Firstly we define our input state and unitary matrix. Note the max mode occupation for each mode is 2 (sum of input photons).

```python
#Unitary beamsplitter matrix
bs_mat= Matrix([[1,1j],[1j,1]])*(1/math.sqrt(2))

#Input state with one photon in each mode
input_state={(0, 0): 0j, (0, 1): 0j, (0, 2): 0j, (1, 0): 0j, (1, 1): 1, (1, 2): 0j, (2, 0): 0j, (2, 1): 0j, (2, 2): 0j}

#Empty output state.
output_state={(0, 0): 0j, (0, 1): 0j, (0, 2): 0j, (1, 0): 0j, (1, 1): 0j, (1, 2): 0j, (2, 0): 0j, (2, 1): 0j, (2, 2): 0j}
```

Now define a function which calculates the transition matrix for a single term in the input state vector, and a given term in the output state $ \ket{m_1, m_2, ...,m_N}$. (i.e $\ket{010} \rightarrow \ket{100})$

```python
def create_transition_matrix(unitary,input_vector,output_vector, d=complex):
		""" Function to make appropriate changes to unitary so that it represents the desired transition
			from this we can then find the permanent representing the probability of this transition.
			This function must be called for every transition probability required to be calculated.
		"""
		no_photons=int(np.sum(input_vector))
		col_swapped_matrix=np.zeros(shape=(no_photons,no_photons),dtype=d)

		#If there are more or less input photons than output channels we must reshape the matrix slightly for the following to work
		#Definitely exists a more efficient way to do this

		reshaped_unitary=np.zeros(shape=(no_photons,no_photons),dtype=d)
		col_count=0
		row_count=0

		for i in range(len(input_vector)):
			for j in range(len(input_vector)):

				if (no_photons-len(input_vector))>=0:
					reshaped_unitary[i,j]=unitary[i,j]

				elif (no_photons-len(input_vector))<0:

					if input_vector[i]!=0 and output_vector[j]!=0:
						reshaped_unitary[row_count,col_count]=unitary[i,j]
						col_count+=1
						row_count+=1

		#Special case of matrix with only 1 photon in and out
		if len(reshaped_unitary)==1:
			return reshaped_unitary[0]

		#Make the column swaps required for the given input vector.
		col_counter=0
		for k in range(len(input_vector)):
			if input_vector[k]==0:
				continue
			else:
				for j in range(input_vector[k]):
					col_swapped_matrix[:,col_counter+j]=copy.deepcopy(reshaped_unitary[:,k])
				col_counter+=1+j

		#Make the row swaps required for a given output vector
		transition_matrix=copy.deepcopy(col_swapped_matrix)
		row_counter=0
		for p in range(len(output_vector)):
			if output_vector[p]==0:
				continue
			else:
				for r in range(output_vector[p]):
					transition_matrix[row_counter+r,:]=copy.deepcopy(col_swapped_matrix[p,:])
				row_counter+=1+r

		return transition_matrix
```

Write a function which calculates the permanent of a given matrix: https://github.com/scipy/scipy/issues/7151

```python
def calculate_permanent(M):
		""" Manual permanent function for cases where thewalrus
		fails to install. As of 04/02/21 no thewalrus wheel
		for python 3.9. Slower than thewalrus, taken from:
		https://github.com/scipy/scipy/issues/7151"""

		n = M.shape[0]
		d = np.ones(n)
		j =  0
		s = 1
		f = np.arange(n)
		v = M.sum(axis=0)
		p = np.prod(v)

		while (j < n-1):
			v -= 2*d[j]*M[j]
			d[j] = -d[j]
			s = -s
			prod = np.prod(v)
			p += s*prod
			f[0] = 0
			f[j] = f[j+1]
			f[j+1] = j+1
			j = f[0]
		return p/2**(n-1)
```

Now we need to loop over every element in the input state with a non 0 amplitude and calculate every transition probability for that given element. i.e for if we have a  $\ket{11}$ input term we must evaulate the transition amplitudes to all number preserving outputs, $\ket{11},\ket{02},\ket{20}$. Each component of the output state will have transition amplitudes from multiple parts of the input state and these must be summed. For example $\ket{10}$ will have contributions from $\ket{10},\ket{01}$ input terms. Once we have done this looping over every input term, for every possible output term we are returned our output state vector.

```python
def calculate_output_amplitudes(unitary, input_vector):
		"""Using the probability expression in 'Permanents in linear optical networks' Scheel 2004,
		we calculate the probability of each transition and store it in an array.
		In the fully quantum case we need to calculate all possible contributions to the output state
		that is we need to loop over every element in the input state with a non 0 amplitude
		and calculate every transition probability for that element.
		"""
		state_vector_elements=[list(key) for key in input_vector]
		input_amplitudes=list(input_vector.values() )
		output_amplitudes=np.zeros(shape=(len(input_amplitudes)), dtype=complex)

		#If the walrus not installed use manual permanent calc
		is_walrus_alive = importlib.util.find_spec(name='thewalrus')

		#For every element of the input state vector

		for i in range(len(state_vector_elements)):
			input_element=state_vector_elements[i]
			#Loop over every possible outcome
			for k in range(len(state_vector_elements)):
				element=state_vector_elements[k]

				#If it has a non zero amplitude
				#only consider photon number preserving transitions as these should evaluate to 0 anyway (true?)
				if input_amplitudes[i] != 0 and np.sum(input_element)==np.sum(element):

					#print('The transition being calculated is ', input_element, element )

					trans_matrix=create_transition_matrix(unitary, input_element, element)

					if len(trans_matrix)==1:
						output_amplitudes[i]+=(np.abs(trans_matrix[0])**2)*input_amplitudes[i]

					else:
						prefactor=1

						if is_walrus_alive is None:
							perm=calculate_permanent(trans_matrix)
						else:
							perm=thewalrus.perm(trans_matrix)

						for m in range(len(input_element)):
							prefactor=prefactor*(1/math.sqrt(math.factorial(input_element[m])))*(1/math.sqrt(math.factorial(element[m])))

						output_amplitudes[k]+=np.around(perm*prefactor, decimals=6)*input_amplitudes[i]

		return output_amplitudes
```

Now we feed out input state of one photon in each mode into out beamsplitter and calculate the output state:

```python
output_amplitudes=calculate_output_amplitudes(bs_mat, input_state)

#update the output state dictionary with the new amplitudes
it=0
for key in output_state:
		output_state[key]=output_amplitudes[it]
		it+=1

print('The output state is:', output_state)
```

    The output state is: {(0, 0): 0j, (0, 1): 0j, (0, 2): 0.707107j, (1, 0): 0j, (1, 1): 0j, (1, 2): 0j, (2, 0): 0.707107j, (2, 1): 0j, (2, 2): 0j}

What you see above should hopefully be $\frac{i}{\sqrt(2)}(\ket{20}+\ket{02})$. We've simulated a HOM dip!

Hopefully its clear that this is an extremely expensive calculation for large interferometers. The complexity of this calculation in general is exactly the complexity of classically simulating boson sampling (exactly what this example is).


```python
%load_ext autoreload
%autoreload 2

import copy

import numpy as np
import pandas as pd
import scipy.stats as st

import re #regex

import cmdstanpy
import arviz as az

import bebi103
import bokeh_catplot

import bokeh.io
import bokeh.plotting
# import bokeh.models.mappers
import bokeh.palettes

import holoviews as hv
import holoviews.operation.datashader
hv.extension('bokeh')
bebi103.hv.set_defaults()

import panel as pn
pn.extension()

bokeh.io.output_notebook()

%load_ext blackcellmagic
```

### Munging
Load data from Brewster, pre-tidied by Manuel, and drop the spurious column that was the index in csv.
See `code/exploratory/fish_munging.ipynb` for details. TL;DR: don't use the regulated csv, the one below has all the FISH data. mRNA_cell is the data we want, not spots_totals (some of the repressed strains have higher spots_totals than UV5, so that's clearly not the readout we want).

```python
df_fish = pd.read_csv("../../data/jones_brewster_2014.csv")
del df_fish['Unnamed: 0']
df_fish.head()
```

Next, let's get the energies from the supplement of Brewster/Jones 2012 paper.

```python
df_energies = pd.read_csv("../../data/brewster_jones_2012.csv")
df_energies.head()
```

All the promoters in the 2012 dataset are in the 2014 fish dataset (verified in `code/exploratory/fish_munging.ipynb`). These are the only constitutive promoters I'm interested in (this only excludes a couple, and they are useless without more metadata).

#### Splitting into regulated & constitutive data
Some of these datasets are not of interest right now so let's split it into multiple dataframes for easier downstream handling. The regulated datasets start with O1, O2, or O3. Everything else doesn't. From that everything else, grab the ones that we have energies for, and set aside the rest. Use regex to parse.

```python
raw_expt_labels = df_fish['experiment'].unique()
raw_expt_labels.sort()

# put all strings that start w/ 'O' in one list
regulated_labels = [label for label in raw_expt_labels if re.match('^O', label)]
# from that, split out those we have energies for
constitutive_labels = [label for label in raw_expt_labels if label in tuple(df_energies.Name)]
```

Without more metadata, I don't really know what to do with the leftover labels data, e.g., what good does the aTc concentration do me if I don't know what promoter it was for?

Now that we've got labels we want, let's slice dataframes accordingly.

```python
df_reg = df_fish[df_fish['experiment'].isin(regulated_labels)]
df_unreg = df_fish[df_fish['experiment'].isin(constitutive_labels)]
```

Also separate UV5 for testing convenience.

```python
df_UV5 = df_unreg[df_unreg["experiment"] == "UV5"]
```

## Analyzing constitutive UV5 with Stan

Stan model borrows from JB's tutorial 7a, 2018. Stan parametrizes the negative binomial with $\alpha$ and $\beta$, where $\alpha$ is the burst frequency (dimensionless, nondimensionalized by mRNA lifetime) and $\beta = 1/b$ where $b$ is the mean burst size.

### Prior predictive checks

```python
sm_prior_predictive = cmdstanpy.CmdStanModel(
    stan_file="stan/constitutive_prior_predictive_v01.stan"
)
# print(sm_prior_predictive.code())
```

```python
# data_prior_pred = dict(
#     N=len(df_UV5),
#     log_alpha_loc=0.0,
#     log_alpha_scale=2.0,
#     log_b_loc=0.5,
#     log_b_scale=1.0
# )
data_prior_pred = dict(
    N=len(df_UV5),
    log_alpha_loc=0.0,
    log_alpha_scale=0.5,
    log_b_loc=0.5,
    log_b_scale=0.5
)
```

```python
prior_pred_samples = sm_prior_predictive.sample(
    data=data_prior_pred,
    fixed_param=True,
    sampling_iters=1000,
    output_dir="./stan/stan_samples",
)
```

```python
# Convert to ArviZ InferenceData
prior_pred_samples = az.from_cmdstanpy(
    posterior=prior_pred_samples,
    prior=prior_pred_samples,
    prior_predictive=['mRNA_counts']
)
```

```python
p = bebi103.viz.predictive_ecdf(
    prior_pred_samples.prior_predictive['mRNA_counts'],
    frame_height=250,
    frame_width=350,
    discrete=True,
    percentiles=(95, 90, 75, 50),
    x_axis_label='mRNA counts',
    x_axis_type='log'
)
bokeh.io.show(p)
```

### Simulation-based calibration

Next up: simulation-based calibration (SBC). Quoting JB, this checks "that the sampler can effectively sample the entire space of parameters covered by the prior." We'll go ahead and set up the data for the posterior, even though we won't be sampling the posterior right now. Then we can set up the model.

```python
data_UV5 = copy.deepcopy(data_prior_pred)
data_UV5["N"] = len(df_UV5)
data_UV5["mRNA_counts"] = df_UV5["mRNA_cell"].values.astype(int)
data_UV5["ppc"] = 0

sm = cmdstanpy.CmdStanModel(stan_file="stan/constitutive_v01.stan")
# print(sm.code())
```

```python
try:
    sbc_output = pd.read_csv("stan/sbc_minimal_1state_stan_analysis.csv")
except:
    sbc_output = bebi103.stan.sbc(
        prior_predictive_model=sm_prior_predictive,
        posterior_model=sm,
        prior_predictive_model_data=data_prior_pred,
        posterior_model_data=data_UV5,
        measured_data=["mRNA_counts"],
        parameters=["alpha", "b"],
        sampling_kwargs={"thin": 10},
        cores=7,
        N=400,
        progress_bar=True,
    )

    sbc_output.to_csv("stan/sbc_minimal_1state_stan_analysis.csv", index=False)
```

Plot ECDFs of the rank statistics, which should be ~ uniform. Color according to warning code, which will also let us know if there were any issues with divergences, R-hat, EBFMI, effective number of steps, or tree depth.

```python
plots = [
    bokeh_catplot.ecdf(
        data=sbc_output.loc[sbc_output["parameter"] == param, :],
        val="rank_statistic",
        cats="warning_code",
        kind="colored",
        frame_width=400,
        frame_height=150,
        title=param,
        conf_int=True,
    )
    for param in sbc_output["parameter"].unique()
]

bokeh.io.show(bokeh.layouts.gridplot(plots, ncols=1))
```

Very uniform. Warning code 2 means R-hat failure. Let's look at the R-hat values.

```python
bokeh.io.show(
    bokeh_catplot.ecdf(data=sbc_output, val="Rhat", cats="parameter")
)
```

Not terrible - about 15% are above the brightline 1.01, but not way over. JB suggests in this case that more samples from the posterior might be wise.

JB also has a nice helper function to plot the difference from uniform of rank ECDFs, with a confidence interval for the uniform distribution to guide the eye. Let's take a look.

```python
bokeh.io.show(bebi103.viz.sbc_rank_ecdf(sbc_output))
```

Finally let's check shrinkage and z-scores.

```python
hv.Points(
    data=sbc_output,
    kdims=['shrinkage', 'z_score'],
    vdims=['parameter', 'ground_truth', 'mean', 'sd'],
).opts(
    color='parameter',
    alpha=0.3,
    xlim=(0, 1.05),
    tools=['hover']
)
```

z-scores are great. There are a couple of points with terrible shrinkage that may be worth investigating. Generally $\alpha$ shrinkage is great, $b$ less so. This may just reflect that my prior on $b$ was pretty tight; less than ~5% have shrinkage less than 0.9, and $\gtrsim 90\%$ have shrinkage above 0.98. I wonder too if the outliers are datasets that just had very very few mRNAs, which would make it hard to inform the posterior. Most of the $b$ points with poor shrinkage have $b >1$, which, if they also had very small $\alpha$, is not inconsistent with my above guess. I should probably relax my priors on $\alpha$ and especially $b$, now that I realized my mistake in plotting the prior predictives (forgot the `discrete` option).

_Update_: after modifying my priors to be weaker, this looks perfectly fine.

### Sampling the Posterior
We already finished building the model in order to do SBC. Now we just run it.

```python
# We do want posterior predictive checks this time
data_UV5["ppc"] = 1

with bebi103.stan.disable_logging():
    posterior_samples = sm.sample(data=data_UV5, cores=6)
posterior_samples = az.from_cmdstanpy(
    posterior_samples, posterior_predictive=["mRNA_counts_ppc"]
)
```

```python
bebi103.stan.check_all_diagnostics(posterior_samples)
```

Good, no sampler warnings. Let's visualize the posterior.

```python
bokeh.io.show(
    bebi103.viz.corner(
        posterior_samples,
        pars=["alpha", "b"],
        alpha=0.1,
        xtick_label_orientation=np.pi / 4,
    )
)
```

That looks quite reasonable. The transcripts per burst & burst frequency are both comparable to what we would have inferred from Manuel's MCMC, but now both parameters are actually identifiable!

### Posterior predictive checks

Even though the posterior looks ok, the model could still be wrong: it is identifiable, but is it consistent with the data? Posterior predictive checks address this by asking whether the model could plausibly generate the observed data. (Function borrowed from JB's finch beak tutorial for bebi103b 2020 TAs.)

```python
def ppc_ecdfs(posterior_samples, df):
    """Plot posterior predictive ECDFs."""
    n_samples = (
        posterior_samples.posterior_predictive.dims["chain"]
        * posterior_samples.posterior_predictive.dims["draw"]
    )

    p1 = bebi103.viz.predictive_ecdf(
        posterior_samples.posterior_predictive["mRNA_counts_ppc"].values.reshape(
            (n_samples, len(df))
        ),
        data=df["mRNA_cell"],
        discrete=True,
        x_axis_label="mRNA counts per cells",
        frame_width=200,
        frame_height=200
    )

    p2 = bebi103.viz.predictive_ecdf(
        posterior_samples.posterior_predictive["mRNA_counts_ppc"].values.reshape(
            (n_samples, len(df))
        ),
        data=df["mRNA_cell"],
        percentiles=[95, 90, 80, 50],
        diff=True,
        discrete=True,
        x_axis_label="mRNA counts per cells",
        frame_width=200,
        frame_height=200
    )
    p1.x_range = p2.x_range

    return [p1, p2]

bokeh.io.show(bokeh.layouts.gridplot(ppc_ecdfs(posterior_samples, df_UV5), ncols=2))
```

The model is not obviously wrong, but depending what percentiles I choose to plot, I might say that a few too many of the observed datapoints are outside the posterior predictive bands.

Still, we should be systematic and plot the posterior and ppc for all the promoters.

### Sampling all the data
Let's repeat for all the constitutive promoters! Since we have so many, do separate loops to generate the samples and generate viz (so we can tweak viz without pointlessly rerunning the sampling).

To supress output, check out JB's disable_logging wrapper in bebi103.stan.

```python
all_samples = {}
for gene in df_unreg['experiment'].unique():
    temp_df = df_unreg[df_unreg['experiment'] == gene]
    data = copy.deepcopy(data_prior_pred)
    data["N"] = len(temp_df)
    data["mRNA_counts"] = temp_df["mRNA_cell"].values.astype(int)
    data["ppc"] = 1

    with bebi103.stan.disable_logging():
        posterior_samples = sm.sample(data=data, cores=6)
    all_samples[gene] = az.from_cmdstanpy(
        posterior_samples, posterior_predictive=["mRNA_counts_ppc"]
    )
```

It's always wise to check diagnostics (though for saving, I might supress the output).

```python
# for gene in all_samples:
#     print(gene)
#     bebi103.stan.check_all_diagnostics(all_samples[gene])
```

There's some run-to-run variation. Sometimes one or two promoters give Rhat warnings a bit over 1.01, but generally this looks ok.

Now that we've sampled we can plot. (I can't figure out how to add titles to the bokeh layouts that `viz.corner` returns, so I'm doing the poor man's version for now.)

```python
for gene in all_samples:
    print(gene)
    # plot posterior as corner plot
    bokeh.io.show(
        bebi103.viz.corner(
            all_samples[gene],
            pars=["alpha", "b"],
            alpha=0.1,
            xtick_label_orientation=np.pi / 4,
        )
    )
    # plot post pred ecdf
    obs_data = df_unreg[df_unreg["experiment"] == gene]
    bokeh.io.show(
        bokeh.layouts.gridplot(ppc_ecdfs(all_samples[gene], obs_data), ncols=2)
    )
```

The posterior predictive checks make it pretty clear that this most simple of models is not entirely capable of producing the observed data for all the promoters. The observed data has a pretty stereotyped pattern on the difference-in-ECDF plots. This strongly suggests there is something systematic our model fails to capture. Nevertheless, looking at the full ecdfs, I'm actually pleasantly surprised that the model does as well as it does. We haven't explicitly put in anything like cell cycle & gene copy number variability, and yet it captures quite a lot. Maybe it's already robust enough to attempt inference on simple repression data?

### Correlation with predicted promoter binding energies

We would like to plot all 18 promoters together to compare their inferred burst parameters at a glance. Plotting all the posterior samples would be visually illegible.
Instead we can compute a contour that encloses 95% of the samples. (Default smoothing in the contour calculator occasionally breaks and totally misses the HPD, so I increased it slightly.) Further, we can color code the contours by the predicted binding energies from Brewster/Jones 2012.

First compute all the contours coords with JB's utility. (Note that `hv.Contours` prefers its input as a dictionary: `"x"` and `"y"` keys must be labeled as such and provide the coords of the contour, and a 3rd key providing a scalar for the contour level, here the promoter binding energy which we lookup.) Then we can overlay all the contours, colored by their corresponding promoter binding energy (fat lines make it easier to perceive the color).

```python
contour_list = []
for gene in all_samples:
    alpha_samples = all_samples[gene].posterior.alpha.values.flatten()
    b_samples = all_samples[gene].posterior.b.values.flatten()
    x_contour, y_contour = bebi103.viz.contour_lines_from_samples(
        alpha_samples, b_samples, levels=0.95, smooth=0.025
    )
    contour_list.append(
        {
            "x": x_contour[0],
            "y": y_contour[0],
            "Energy (kT)": df_energies.loc[
                df_energies["Name"] == gene, "Energy (kT)"
            ].values[0],
            "Promoter": gene
        }
    )
p = (
    hv.Contours(contour_list, vdims=["Energy (kT)", "Promoter"])
    .opts(logx=True, logy=True)
    .opts(line_width=2)
)
p.opts(
    hv.opts.Contours(
        cmap="viridis",
        colorbar=True,
        tools=["hover"],
        width=500,
        height=500,
        xlabel="α (bursts per mRNA lifetime)",
        ylabel="b (transcripts per burst)",
        padding=0.03,
    )
)
```

Very interesting. By eye I'd say there's little to no correlation between burst size $b$ and binding energy, and maybe linear scaling between the log of burst frequency $\alpha$ and binding energy, though that correlation is quite noisy. Can we intuit that relation theoretically?

### First attempt at theory understanding
Manuel observed how to relate burst parameters to thermodynamic model: just equate $\langle m\rangle$ in the two pictures. In the 2-state constitutive picture, we would have
\begin{align}
\langle m\rangle = \frac{r}{\gamma}\frac{k_{on}}{k_{on} + k_{off}}
    \approx \frac{r}{\gamma}\frac{k_{on}}{k_{off}}
\end{align}
in the limit $k_{off} \gg k_{on}$ suggested by Manuel's inference. Taking the limit to the bursty one-state picture amounts to taking $r, k_{off}\rightarrow\infty$ while holding $r/k_{off}$ constant, and this ratio becomes the mean burst size $b$. In other words then
\begin{align}
\langle m\rangle \approx \frac{r}{\gamma}\frac{k_{on}}{k_{off}}
    \approx b\frac{k_{burst}}{\gamma},
\end{align}
where we have reinterpreted $k_{on}$ as the burst frequency $k_{burst}$ in the new picture.

In the states-and-weights picture, we would have
\begin{align}
\langle m\rangle =
    \frac{r}{\gamma}\frac{\frac{P}{N_{NS}}e^{-\beta\Delta\epsilon}}
            {1+\frac{P}{N_{NS}}e^{-\beta\Delta\epsilon}}
    \approx \frac{r}{\gamma}\frac{P}{N_{NS}}e^{-\beta\Delta\epsilon}
\end{align}
taking the usual weak promoter limit.

Now we equate these two expressions for $\langle m\rangle$, i.e.,
\begin{align}
\langle m\rangle \approx b\frac{k_{burst}}{\gamma}
    \approx \frac{r}{\gamma}\frac{P}{N_{NS}}e^{-\beta\Delta\epsilon}.
\end{align}
One could imagine several plausible associations between the various parameters. A post-facto argument for the "right" choice is the following.

In the parametrization of negative binomial we are using, the Fano factor has the especially simple form $1+b$ (independent of $k_{burst}$). It would therefore be quite strange if $b\ll1$ or $b\gg10$, the former meaning the data is Poisson distributed to an absurd precision and the latter meaning it is absurdly overdisperse relative to Poisson. But $P/N_{NS}\sim10^{-3}$, $r/\gamma\gg1$, and for various promoters we have anywhere from $e^{-\beta\Delta\epsilon}\lesssim10$ to $e^{-\beta\Delta\epsilon}\gtrsim10^3$. So then the only comibination that is anywhere near unity is $r/\gamma \times P/N_{NS}$ and thus the only associations that make any sense are
\begin{align}
b &\sim \frac{r}{\gamma}\frac{P}{N_{NS}}
\\
\frac{k_{burst}}{\gamma} &\sim e^{-\beta\Delta\epsilon}.
\end{align}

For a first check of this, set aside the variation in burst size $b$. Recalling we nondimensionalized $k_{burst}/\gamma\equiv\alpha$, let us plot $\log\alpha$ vs the predicted $\Delta G$ for each promoter.

```python
log_alphas = np.zeros((len(df_energies), 3))
for i, promoter in enumerate(df_energies.Name):
    samples = all_samples[promoter].posterior.alpha.values.flatten()
    log_alphas[i] = np.percentile(samples, (5, 50, 95))

df_energies['log_alpha'] = np.log(log_alphas[:,1])
df_energies['log_alpha_lbnd'] = df_energies['log_alpha'] - np.log(log_alphas[:,0])
df_energies['log_alpha_ubnd'] = np.log(log_alphas[:,2]) - df_energies['log_alpha']
```

```python
# holoviews ErrorBars wants a goofy format, can't take straight from dataframe
err_bars = np.array(
    (
        df_energies["Energy (kT)"],
        df_energies["log_alpha"],
        df_energies["log_alpha_lbnd"],
        df_energies["log_alpha_ubnd"],
    )
).transpose()

(
    hv.Points(
        df_energies,
        kdims=["Energy (kT)", "log_alpha"],
        vdims=["Name", "log_alpha_lbnd", "log_alpha_ubnd"],
    ).opts(tools=["hover"])
    * hv.ErrorBars(err_bars)
)
```

That resembles a straight line and it most certainly has the correct slope. The error bars are merely the uncertainty in our inference, and we shouldn't expect perfect agreement here since we've coarse-grained away so much detail. Realistically there should probably be horizontal error bars of at least 0.5 to 1 $kT$ on the predicted energies also.


 ILI285 - Computación Científica I  / INF285 - Computación Científica

 Floating Point Arithmetic

 [[S]cientific [C]omputing [T]eam](#acknowledgements)

 Version: 1.13

## Table of Contents
* [Introduction](#intro)
* [The nature of floating point numbers](#nature)
* [Visualization of floating point numbers](#visualization)
* [Loss of significance](#loss)
* [Loss of significance in funcion evaluation](#func)
* [Another analysis (example from textbook)](#another)
* [Acknowledgements](#acknowledgements)

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

## Introduction

Hello! This notebook is an introduction to how our computers handle the representation of real numbers using double-presicion floating-point format. To understand the contents of this notebook you should have at least a basic notion of how binary numbers work.

The aforementioned format occupies 64 bits which are divided as follows:

* 1 bit for the sign
* 11 bits for the exponent
* 52 bits for the mantissa

This means that the very next representable number after $1$ is $1 + 2^{-52}$, and their difference, $2^{-52}$, is the $\epsilon _{mach}$.

Additionally, if you'd like to quickly go from a base-2 integer to a base-10 integer and viceversa, Python has some functions that can help you with that.

```python
int('0b11', 2)
```

```python
bin(9)
```

```python
bin(2**53)
```

## The nature of floating point numbers

As we know until now, float representations of real numbers are just a finite and bounded represetation of them. But another interesting thing, is that these floating numbers are distributed across the real numbers.

To see that, it's really important to keep in mind the following property:

\begin{equation} \left|\frac{\text{fl}(x)-x}{x}\right| \leq \frac{1}{2} \epsilon_{\text{mach}} \end{equation}

where $\text{fl}(x)$ means the float representation of $x \in R$. What it says is that **the relative error in representing any non-zero real number x, is bounded by a quantity that depends on the system representation** ($\epsilon_{\text{mach}}$).

Maybe now you're thinking: what relationship does this have with the distribution of floating point numbers? So if we rewrite the previous property like this:

\begin{equation} |\text{fl}(x)-x| \leq \frac{1}{2} \epsilon_{\text{mach}} |x| \end{equation}

it's clearer: **The absolute error (distance) between a real number and its floating point representation is proportional to the real number's magnitude.**

Intuitively speaking, if the representation error of a number increases as its magnitude increases, then it's quite natural that **the distance between a floating point number and the next representable floating point number will increase as the magnitude of such number increases (and conversely)**. Could you prove that?  For now we will prove it experimentally.

We will use a library named **bitstring** to handle different number representations. You can install it with:
```
pip install bitstring
```

```python
import bitstring as bs
```

The next two functions are self-explanatory:

1. `next_float(f)` computes the next representable float number.
2. `epsilon(f)` computes the difference between f and the next representable float number

```python
def next_float(f):
    #packing double-precision foat
    b = bs.pack('>d', f)

    #extracting mantisa as unsigned int
    #and adding up 1
    m = b[12:].uint
    m += 1

    #putting the result in his place
    b[12:] = m

    return b.float

def epsilon(f):
    next_f = next_float(f)
    return next_f - f
```

So if we compute `epsilon(1)` we should get the epsilon machine number. Let's try it:

```python
epsilon(1)
```

In order to prove our hypotesis, we will create an array of values: [1e-32, 1e-31, ..., 1e31, 1e32] and compute their corresponding epsilon.

```python
#values between 10**-32 and 10**+32
values = np.array([10**i for i in range(-32,32)]).astype(float)

#corresponding epsilons
vepsilon = np.vectorize(epsilon)
eps = vepsilon(values)
```

```python
plt.plot(values, eps,'.',markersize=20)
plt.xlabel('Values')
plt.ylabel('Corresponding Epsilons')
plt.title('Epsilons v/s Values')
plt.grid(True)
plt.show()
```

```python
plt.loglog(values, eps,'.')
plt.xlabel('Values')
plt.ylabel('Corresponding Epsilons')
plt.title('Epsilons v/s Values')
plt.grid(True)
plt.show()
```

As you can see, the hypotesis was right. In other words: Floating point numbers are not linearly distributed across the real numbers, and the distance between them is proportional to their magnitude. **Tiny numbers (~ 0) are closer between each other than bigger numbers are.**

## Visualization of floating point numbers

With the help of `bitstring` library we could write a function to visualize floating point numbers in his binary representation

```python
def to_binary(f):
    b = bs.pack('>d', f)
    b = b.bin
    #show sign + exponent + mantisa
    print(b[0]+' '+b[1:12]+ ' '+b[12:])
```

Let's see some intereseting examples

```python
to_binary(1.)
```

```python
int('0b01111111111', 2)
```

```python
to_binary(1.+epsilon(1.))
```

```python
to_binary(+0.)
```

```python
to_binary(-0.)
```

```python
to_binary(np.inf)
```

```python
to_binary(-np.inf)
```

```python
to_binary(np.nan)
```

```python
to_binary(-np.nan)
```

```python
to_binary(2.**-1074)
```

```python
print(2.**-1074)
```

```python
to_binary(2.**-1075)
```

```python
print(2.**-1075)
```

```python
to_binary(9.4)
```

## Loss of significance

As we mentioned, there's a small leap between 1 and the next representable number, which means that if you want to represent a number between those two, you won't be able to do so; that number is nonexistent as it is for the computer, so it'll have to round it to a representable number before storing it in memory.

```python
a = 1.
b = 2.**(-52) #emach
result_1 = a + b     # arithmetic result is 1.0000000000000002220446049250313080847263336181640625
result_1b = result_1-1.0
print("{0:.1000}".format(result_1))
print(result_1b)
print(b)
```

```python
c = 2.**(-53)
result_2 = a + c     # arithmetic result is 1.00000000000000011102230246251565404236316680908203125
np.set_printoptions(precision=16)
print("{0:.1000}".format(result_2))
print(result_2-a)
```

```python
to_binary(result_2)
to_binary(result_2-a)
```

```python
d = 2.**(-53) + 2.**(-54)

result_3 = a + d     # arithmetic result is 1.000000000000000166533453693773481063544750213623046875
print("{0:.1000}".format(result_3))
to_binary(result_3)
to_binary(d)
```

As you can see, if you try to save a number between $1$ and $1 + \epsilon _{mach}$, it will have to be rounded (according to some criteria) to a representable number before being stored, thus creating a difference between the real number and the stored number. This situation is an example of loss of significance.

Does that mean that the "leap" between representable numbers is always going to be $\epsilon _{mach}$? Of course not! Some numbers will require smaller leaps, and some others will require bigger leaps.

In any interval of the form $[2^n,2^{n+1}]$ for $n\in \mathbb{Z}$ and representables, is constant. For example, all the numbers between $2^{-1}$ and $2^0$ (but excluding $2^0$) have a distance of $\epsilon _{mach}/2$ between them. All the numbers between $2^0$ and $2^1$ (excluding $2^1$) have a distance of $\epsilon _{mach}$ between them. Those between $2^1$ and $2^2$ (not including $2^2$) have a distance of $2\,\epsilon _{mach}$ between them, and so on and so forth.

```python
e = 2.**(-1)
f = b/2. # emach/2

result_4 = e + f     # 0.50000000000000011102230246251565404236316680908203125
print("{0:.1000}".format(result_4))

result_5 = e + b     # 0.5000000000000002220446049250313080847263336181640625
print("{0:.1000}".format(result_5))
```

```python
g = b/4.

result_5 = e + g     # 0.500000000000000055511151231257827021181583404541015625
print("{0:.1000}".format(result_5))
```

We'll let the students find some representable numbers and some non-representable numbers. It's important to note that loss significance can occur in many more operations and functions other that the simple addition of two numbers.

```python
num_1 = a
num_2 = b
result = a + b
print("{0:.1000}".format(result))
```

## Loss of significance in function evaluation

Loss of Significance is present too in the representation of **functions**. A classical example (which you can see in the guide book), is the next function:

\begin{equation}f(x)= \frac{1 - \cos x}{\sin^{2}x} \end{equation}

Applying trigonometric identities, we can obtain the 'equivalent' function:

\begin{equation}f(x)= \frac{1}{1 + \cos x} \end{equation}

Both of these functions are apparently equals. Nevertheless, its graphics say to us another thing when $x$ is equal to zero.

```python
x = np.arange(-10,10,0.1)
y = (1-np.cos(x))/np.sin(x)**2
plt.figure()
plt.plot(x,y,'.')
plt.grid(True)
plt.show()
```

```python
x = np.arange(-10,10,0.1)
y = 1/(1+np.cos(x))
plt.figure()
plt.plot(x,y,'.')
plt.grid(True)
plt.show()
```

```python
x = np.arange(-1,1,0.01)
y = (1-np.cos(x))/np.sin(x)**2
plt.figure()
plt.plot(x,y,'.',markersize=20)
plt.grid(True)
plt.show()
```

```python
y = 1/(1+np.cos(x))
plt.figure()
plt.plot(x,y,'.',markersize=20)
plt.grid(True)
plt.show()
```

This happens because when $x$ is equal to zero, the first function has an indetermination, but previously, the computer makes a subtraction between numbers that are almost equals. This generates a loss of significance, turning the expression close to this point to be zero. However, modifying this expression to make the second function, eliminates this substraction, fixing the error in its calculation when $x=0$.

In conclusion, for us, two representations of a function can be equals, but for the computer they can be different!

## Another analysis (example from textbook)

```python
f1 = lambda x: (1.-np.cos(x))/(np.sin(x)**2)
f2 = lambda x: 1./(1+np.cos(x))
x = np.logspace(-19,0,20)[-1:0:-1]
o1 = f1(x)
o2 = f2(x)

print("x,                 f1(x),             f2(x)")
for i in np.arange(len(x)):
    print("%1.15f, %1.15f, %1.15f" % (x[i],o1[i],o2[i]))
```

## Libraries
### Please make sure you make all of them your BFF!!

* Numpy - IEEE 754 Floating Point Special Values: https://docs.scipy.org/doc/numpy-1.10.0/user/misc.html
* Matplotlib: http://matplotlib.org/examples/pylab_examples/simple_plot.html
* Nice Trick: https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

# Acknowledgements
* _Material originally created by professor Claudio Torres_ (`ctorres@inf.utfsm.cl`) _and assistants: Laura Bermeo, Alvaro Salinas, Axel Simonsen and Martín Villanueva. v.1.1. DI UTFSM. March 2016._

```python

```