```python
#remove cell visibility
from IPython.display import HTML
tag = HTML('''
Promijeni vidljivost <a href="javascript:code_toggle()">ovdje</a>.''')
display(tag)
```

Promijeni vidljivost ovdje.

```python

```

## Tempomat vozila

Dinamika brzine vozila može se modelirati (za detalje pogledajte ICCT priručnik) kao:

$$
\begin{cases}
\dot{x}=\underbrace{\begin{bmatrix}-\frac{b}{m}&\frac{\tau_{\text{max}}\eta}{mr}\\0&-\frac{1}{2}\end{bmatrix}}_{A}x+\underbrace{\begin{bmatrix}0\\\frac{1}{2}\end{bmatrix}}_{B}t_r \\
y = \underbrace{\begin{bmatrix}3.6&0\end{bmatrix}}_{C}x
\end{cases}
$$

gdje je$y$  brzina automobila (u km/h), masa automobila $m$ je $1000$ kg, maksimalni okretni moment motora $\tau_{\text{max}}$ je 150 Nm, radijus kotača $r$ je 25 cm (s konstantom prijenosnog omjera $\eta$ i 4:1 motor:kotač), faktor aerodinamičnog prigušenja brzine $b$ iznosi $60$ Ns/m, a $\tau_{\%}$ je naredba za dodavanje gasa (s vrijednostima unutar 0 i 1). Sustav je upravljiv i osmotriv.

Cilj je dizajnirati sustav upravljanja koji regulira brzinu vozila djelujući na motor, a prema sljedećim specifikacijama:
- nulta pogreška u stacionarnom stanju u odzivu na promjenu željene brzine,
- nema prekoračenja ili je ono barem manje od 1%,
- vrijeme smirivanja za pojas tolerancije od 5% kraće je od 4 sekunde.

### Dizajn regulatora
Važno je primijetiti da nije moguće ispuniti zahtjev o vremenu smirivanja za sve moguće referentne vrijednosti brzine. To je lako razumjeti fizičkim rezoniranjem: zračni otpor je mnogo veći pri većim brzinama, pa vozilo ne može ubrzati pri većim brzinama na isti način kao što to može pri manjim brzinama. Nadalje, lako je pokazati da, čak i ako je zračni otpor zanemariv ($b=0$), maksimalna promjena brzine koja se može postići pri punom gasu u 5 sekundi iznosi 43,2 km/h.
Stoga, sustav upravljanja se razvija na način da se ispoštuju prva dva zahtjeva i da sustav bude čim brži.

#### Dizajn povratne veze stanja

Da bi se postigla nulta pogreška u stacionarnom stanju u odzivu na komandni korak, sustav se proširuje dodavanjem stanja $x_3 = v-v_{\text{ref}}$, pri čemu $v_{\text{ref}}$ postaje referentni ulaz u km/h. Jednadžbe proširenog sustava tada postaju:

$$
\begin{cases}
\dot{x_a}=\underbrace{\begin{bmatrix}-\frac{b}{m}&\frac{\tau_{\text{max}}\eta}{mr}&0\\0&-\frac{1}{2}&0\\3.6&0&0\end{bmatrix}}_{A_a}x_a+\underbrace{\begin{bmatrix}0\\\frac{1}{2}\\0\end{bmatrix}}_{B_a}t_r+\underbrace{\begin{bmatrix}0\\0\\-1\end{bmatrix}}_{B_{\text{ref}}}v_{\text{ref}} \\
y_a = \underbrace{\begin{bmatrix}3.6&0&0\\0&0&1\end{bmatrix}}_{C_a}x_a
\end{cases}
$$

Imajte na umu da sustav ostaje upravljiv.

Moguće rješenje je postaviti svojstvene vrijednosti od $A-BK$ u $-0.06$, $-2.43-2.41i$ i $-2.43+2.41i$, uz $K_a=\begin{bmatrix}9.75&8.73&0.15\end{bmatrix}$.

#### Dizajn promatrača

Promatrač je razvijen na temelju početnog sustava kako bi se procijenili $x_1$ i $x_2$. Polovi dinamike pogreške ($A-LC$) smješteni su u $-10$rad/s uz $L=\begin{bmatrix}5.4&10.45\end{bmatrix}^T$.

U nastavku je simuliran sustav s kontrolerom. Također ga je moguće simulirati uz prisustvo zasićenja na ulaznoj strani.

```python
%matplotlib inline
import control as control
import numpy
import sympy as sym
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    def dummychangecallback(self,change):
        pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(control.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        control.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass
```

```python
# Preparatory cell

Aa = numpy.matrix([[-3/50,2.4,0],[0,-0.5,0],[3.6,0,0]])
Ba = numpy.matrix('0; 0.5; 0')
A = numpy.matrix([[-3/50, 2.4],[0,-0.5]])
B = numpy.matrix('0; 0.5')
C = numpy.matrix('3.6 0')
X0 = numpy.matrix('0.; 0.')
K = numpy.matrix([9.75,8.73,0.15])
L = numpy.matrix([[5.4],[10.45]])

# Aw = matrixWidget(3,3)
# Aw.setM(A)
# Bw = matrixWidget(3,1)
# Bw.setM(B)
# Cw = matrixWidget(1,3)
# Cw.setM(C)
X0w = matrixWidget(2,1)
X0w.setM(X0)
Kw = matrixWidget(1,3)
Kw.setM(K)
Lw = matrixWidget(2,1)
Lw.setM(L)

eig1c = matrixWidget(1,1)
eig2c = matrixWidget(2,1)
eig3c = matrixWidget(1,1)
eig1c.setM(numpy.matrix([-0.06]))
eig2c.setM(numpy.matrix([[-2.43],[-2.41]]))
eig3c.setM(numpy.matrix([-0.06]))

eig1o = matrixWidget(1,1)
eig2o = matrixWidget(2,1)
eig1o.setM(numpy.matrix([-10.]))
eig2o.setM(numpy.matrix([[-10.],[0.]]))
```

```python
# Misc

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

# Define type of method
selm = widgets.Dropdown(
    options= ['Postavi K i L', 'Postavi svojstvene vrijednosti'],
    value= 'Postavi K i L',
    description='',
    disabled=False
)

# Define the number of complex eigenvalues
sele = widgets.Dropdown(
    options= ['0 kompleksnih svojstvenih vrijednosti', '2 kompleksne svojstvene vrijednosti'],
    value= '2 kompleksne svojstvene vrijednosti',
    description='Kompleksne svojstvene vrijednosti:',
    style = {'description_width': 'initial'},
    disabled=False
)

#define type of ipout
selu = widgets.Dropdown(
    options=['impuls', 'step', 'sinus', 'Pravokutni val'],
    value='step',
    description='Tip referentnog signala:',
    style = {'description_width': 'initial'},
    disabled=False
)
# Define the values of the input
u = widgets.FloatSlider(
    value=30.,
    min=1.,
    max=150.0,
    step=1,
    description='Referentni signal:',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.1f',
)
period = widgets.FloatSlider(
    value=30,
    min=1,
    max=80,
    step=1,
    description='Period: ',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.2f',
)
time = widgets.BoundedIntText(
    value=120,
    min=1,
    max=200,
    step=1,
    description='Vrijeme:',
    disabled=False
)
sim = widgets.Checkbox(
    value=False,
    description='Zasićenje',
    disabled=False,
    indent=False
)
```

```python
# Support functions

def eigen_choice(sele):
    if sele == '0 kompleksnih svojstvenih vrijednosti':
        eig1c.children[0].children[0].disabled = False
        eig2c.children[1].children[0].disabled = True
        eig1o.children[0].children[0].disabled = False
        eig2o.children[1].children[0].disabled = True
        eig = 0
    if sele == '2 kompleksne svojstvene vrijednosti':
        eig1c.children[0].children[0].disabled = True
        eig2c.children[1].children[0].disabled = False
        eig1o.children[0].children[0].disabled = True
        eig2o.children[1].children[0].disabled = False
        eig = 2
    return eig

def method_choice(selm):
    if selm == 'Postavi K i L':
        method = 1
        sele.disabled = True
    if selm == 'Postavi svojstvene vrijednosti':
        method = 2
        sele.disabled = False
    return method

def simulation(sys_G, sys_K, sys_int, x0, x0k, time, Ts, ref, lim):
    '''Function that simulate a siso feedback system with a saturation in the actuator and integral action.
       Input to sys_K [u_sys_G; y_sys_G].
       sys_int must have k_3 as gain in output and B = [1, -1] associated to y and ref.'''
    if len(time) != len(ref):
        print('Pogreška: T i ref moraju biti iste veličine!')
        return

    # Initialization and preparation...
    K_temp = control.append(sys_K,sys_int)
    K = sss(K_temp.A,
            K_temp.B*sym.Matrix([[1,0,0],[0,1,0],[0,1,0],[0,0,1]]),
            sym.Matrix([1,1]).T*K_temp.C,
            sym.Matrix([1,1]).T*K_temp.D*sym.Matrix([[1,0,0],[0,1,0],[0,1,0],[0,0,1]]))
    sys_temp = control.append(sys_G,K)
    sys_temp1 = sss(sys_temp.A,
                    sys_temp.B*sym.Matrix([[1,0,0],[1,0,0],[0,1,0],[0,0,1]]),
                    sys_temp.C,
                    sys_temp.D*sym.Matrix([[1,0,0],[1,0,0],[0,1,0],[0,0,1]]))
    #sys = control.connect(sys_temp1,[[2,1]],[1,3],[1,2])
    sys = sss(sys_temp1.A+sys_temp1.B[:,1]*sys_temp1.C[0,:],
              sym.Matrix(sys_temp1.B[:,0]).row_join(sym.Matrix(sys_temp1.B[:,2])),
              sys_temp1.C,
              [[0,0],[0,0]])
    sys_d = control.c2d(sys,Ts)
    A, B, C, D = sys_d.A, sys_d.B, sys_d.C, sys_d.D

    # Initializing system's variables
    x = [x0.col_join(x0k).col_join(sym.Matrix([0])) for t in range(0,len(time))]
    yinit = C*x[0]
    y = [yinit for t in range(0,len(time))]
    Input = [sym.Matrix([yinit[1],ref[t]]) for t in range(0,len(time))]

    for i in range(1,len(time)):
        Input[i-1][0] = -y[i-1][1]
        if Input[i-1][0] > lim:
            Input[i-1][0] = lim
        elif Input[i-1][0] < 0:
            Input[i-1][0] = 0.

        x[i] = A*x[i-1] + B*Input[i-1]
        y[i] = C*x[i-1] + D*Input[i-1]

    if Input[-1][0] > lim:
            Input[-1][0] = lim
    elif Input[-1][0] < 0:
            Input[-1][0] = 0.

    y  = [y[t][0] for t in range(len(y))]
    U  = [Input[t][0] for t in range(len(Input))]
    x  = [[x[i][t] for i in range(len(x))] for t in range(A.shape[0])]
    return (y, U, x)

def simulation_ideal(sys_G, sys_K, sys_int):
    K_temp = control.append(sys_K,sys_int)
    Ka = sss(K_temp.A,
            K_temp.B*sym.Matrix([[1,0,0],[0,1,0],[0,1,0],[0,0,1]]),
            sym.Matrix([1,1]).T*K_temp.C,
            sym.Matrix([1,1]).T*K_temp.D*sym.Matrix([[1,0,0],[0,1,0],[0,1,0],[0,0,1]]))
    sys_temp = control.append(sys_G,Ka)
    sys_temp1 = sss(sys_temp.A,
                    sys_temp.B*sym.Matrix([[1,0,0],[1,0,0],[0,1,0],[0,0,1]]),
                    sys_temp.C,
                    sys_temp.D*sym.Matrix([[1,0,0],[1,0,0],[0,1,0],[0,0,1]]))
    #sys = control.connect(sys_temp1,[[2,1],[1,-2]],[3],[1,2])
    sys = sss(sys_temp1.A+sys_temp1.B[:,1]*sys_temp1.C[0,:]-sys_temp1.B[:,0]*sys_temp1.C[1,:],
              sys_temp1.B[:,2],
              sys_temp1.C,
              [[0],[0]])
    return sys
```

```python
sols = numpy.linalg.eig(A)
x0 = sym.Matrix([0,0])
lim = 1.

def main_callback(X0w, K, L, eig1c, eig2c, eig3c, eig1o, eig2o, u, period, selm, sele, selu, sim, time, DW):
    eige = eigen_choice(sele)
    method = method_choice(selm)

    if method == 1:
        solc = numpy.linalg.eig(Aa-Ba*K)
        solo = numpy.linalg.eig(A-L*C)
    if method == 2:
        if eige == 0:
            K = control.acker(Aa, Ba, [eig1c[0,0], eig2c[0,0], eig3c[0,0]])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [eig1o[0,0], eig2o[0,0]]).T
            Lw.setM(L)
        if eige == 2:
            K = control.acker(Aa, Ba, [eig3c[0,0],
                                     numpy.complex(eig2c[0,0],eig2c[1,0]),
                                     numpy.complex(eig2c[0,0],-eig2c[1,0])])
            Kw.setM(K)
            L = control.acker(A.T, C.T, [numpy.complex(eig2o[0,0],eig2o[1,0]),
                                         numpy.complex(eig2o[0,0],-eig2o[1,0])]).T
            Lw.setM(L)

    # Simulation without saturation
    sys_G   = sss(A,B,C,0)
    sys_K   = sss(A-L*C,sym.Matrix(B).row_join(sym.Matrix(L)),K[0][0,:2],[0,0])
    sys_int = sss(0,sym.Matrix([1,-1]).T,K[0][0,2],[0,0])
    x0 = sym.Matrix([0.,0.])
    sys = simulation_ideal(sys_G, sys_K, sys_int)
    solc = sym.Matrix(sys.A).eigenvals(multiple=True,rational=False)
    solc = numpy.matrix(solc,dtype=numpy.complex128)
    print('Svojstvene vrijednosti sustava su:', round(sols[0][0],2),'and', round(sols[0][1],2))
    print('Svojstvene vrijednosti upravljivog sustava u zatvorenoj petlji su:',
          round(solc[0,0],2),',',
          round(solc[0,1],2),',',
          round(solc[0,2],2),',',
          round(solc[0,3],2),'and',
          round(solc[0,4],2))

    X0w1 = [float(x0[0,0]),float(x0[1,0]),float(X0w[0,0]),float(X0w[1,0]),0.]
    T, Td = numpy.linspace(0,time,1000,retstep=True)

    if selu == 'impuls': #selu
        U = [0 for t in range(0,len(T))]
        U[0] = u
        if sim == False:
            T, yout, xout = control.forced_response(sys,T,U,X0w1)
    if selu == 'step':
        U = [u for t in range(0,len(T))]
        if sim == False:
            T, yout, xout = control.forced_response(sys,T,U,X0w1)
    if selu == 'sinus':
        U = u*numpy.sin(2*numpy.pi/period*T)+u
        if sim == False:
            T, yout, xout = control.forced_response(sys,T,U,X0w1)
    if selu == 'Pravokutni val':
        U = u*numpy.sign(numpy.sin(2*numpy.pi/period*T))+u
        if sim == False:
            T, yout, xout = control.forced_response(sys,T,U,X0w1)

    if sim:
        y_sat, U_sat, x_sat = simulation(sys_G, sys_K, sys_int, x0, sym.Matrix(X0w), T, Td, U, lim)

    fig = plt.figure(num='Simulation', figsize=(14,12))
    fig.add_subplot(224)
    plt.title('Izlazni odziv')
    plt.ylabel('Izlaz')
    if sim:
        plt.plot(T,y_sat,T,U,'r--')
        plt.legend(['$y_{sat}$','Referentni signal'])
    else:
        plt.plot(T,yout[0],T,U,'r--')
        plt.legend(['$y$','Referentni signal'])
    plt.xlabel('$t$ [s]')
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(221)
    plt.title('Odziv prvog stanja')
    plt.ylabel('$x_1$')
    if sim:
        plt.plot(T,x_sat[0],T,x_sat[2])
        plt.legend(['$x_{1sat}$','$x_{1est}$'])
    else:
        plt.plot(T,xout[0],T,xout[2])
        plt.legend(['$x_{1}$','$x_{1est}$'])
    plt.xlabel('$t$ [s]')
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(222)
    plt.title('Odziv drugog stanja')
    plt.ylabel('$x_2$')
    if sim:
        plt.plot(T,x_sat[1],T,x_sat[3])
        plt.legend(['$x_{2sat}$','$x_{2est}$'])
    else:
        plt.plot(T,xout[1],T,xout[3])
        plt.legend(['$x_{2}$','$x_{2est}$'])
    plt.xlabel('$t$ [s]')
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(223)
    plt.title('Ulaz')
    plt.ylabel('Ulaz')
    if sim:
        plt.plot(T[:-1],U_sat[:-1],T,[1 for t in range(len(T))],'r--')
        plt.legend(['$Gas$','$limit$'])
    else:
        plt.plot(T,-yout[1],T,[1 for t in range(len(T))],'r--')
        plt.legend(['$Gas$','$limit$'])
    plt.xlabel('$t$ [s]')
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()
    if selu == 'step':
        if sim:
            print('Maksimalno prekoračenje je:',round((max(y_sat)-U[0])/U[0]*100,3),'postotak')
        else:
            print('Maksimalno prekoračenje je:',round((max(yout[0])-U[0])/U[0]*100,3),'postotak')

alltogether = widgets.VBox([widgets.HBox([selm,
                                          sele,
                                          selu]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('K:',border=3), Kw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Svojstvene vrijednosti:',border=3),
                                          eig1c,
                                          eig2c,
                                          eig3c,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('X0 est.:',border=3), X0w]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('L:',border=3), Lw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Svojstvene vrijednosti:',border=3),
                                          eig1o,
                                          eig2o,
                                          widgets.Label(' ',border=3),
                                          time]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([u,
                                          period,
                                          widgets.Label(' ',border=3),widgets.Label(' ',border=3),sim]),
                            widgets.HBox([widgets.Label('Klikni za K, L ili svojstvene vrijednosti ->',border=3),
                            START])])
out = widgets.interactive_output(main_callback, {'X0w':X0w, 'K':Kw, 'L':Lw,
                                                 'eig1c':eig1c, 'eig2c':eig2c, 'eig3c':eig3c,
                                                 'eig1o':eig1o, 'eig2o':eig2o,
                                                 'u':u, 'period':period, 'selm':selm, 'sele':sele, 'selu':selu,
                                                 'sim':sim, 'time':time, 'DW':DW})
out.layout.height = '840px'
display(out, alltogether)
```

    Output(layout=Layout(height='840px'))

    VBox(children=(HBox(children=(Dropdown(options=('Postavi K i L', 'Postavi svojstvene vrijednosti'), value='Pos…

```python

```


# E. coli Glycolytic Network Construction
## Growth Medium: Glucose
Growth data obtained from the following sources:
- Gerosa, Luca et al. “Pseudo-transition Analysis Identifies the Key Regulators of Dynamic Metabolic Adaptations from Steady-State Data.” Cell systems vol. 1,4 (2015): 270-82. doi:10.1016/j.cels.2015.09.008

- Volkmer, Benjamin, and Matthias Heinemann. “Condition-dependent cell volume and concentration of Escherichia coli to facilitate data conversion for systems biology modeling.” PloS one vol. 6,7 (2011): e23126. doi:10.1371/journal.pone.0023126

### Import packages

```python
# Disable gurobi logging output for this notebook.
try:
    import gurobipy
    gurobipy.setParam("OutputFlag", 0)
except ImportError:
    pass

import numpy as np
import pandas as pd
import sympy as sym
import matplotlib.pyplot as plt

import cobra
from cobra.io.json import load_json_model as load_cobra_json_model

import mass
from mass import MassConfiguration, MassModel, MassMetabolite, MassReaction, Simulation
from mass.io.json import save_json_model as save_mass_json_model
from mass.visualization import plot_comparison, plot_time_profile

print(f"COBRApy version: {cobra.__version__}")
print(f"MASSpy version: {mass.__version__}")
```

    Set parameter Username
    Academic license - for non-commercial use only - expires 2022-01-21
    COBRApy version: 0.22.1
    MASSpy version: 0.1.5

### Set solver

```python
MASSCONFIGURATION = MassConfiguration()
MASSCONFIGURATION.solver = "gurobi"
```

## Load COBRA model

```python
cobra_model = load_cobra_json_model(f"./models/cobra/iML1515.json")
```

## Obtain Flux State
### Load growth data

```python
medium = "Glucose"

flux_data = pd.read_excel(
    io="./data/growth_data.xlsx",
    sheet_name="flux_data",
    index_col=0
)
flux_data = flux_data.loc[lambda x: x['Growth Medium'] == medium]
flux_data = flux_data.drop("Growth Medium", axis=1)
flux_data
```

      Flux (mmol * gDW-1 * h-1)

      ID

      EX_ac_e
      6.827019e+00

      ACt2rpp
      -6.827019e+00

      ACKr
      -6.827019e+00

      PTAr
      6.827019e+00

      ACS
      0.000000e+00

      ...
      ...

      ME1
      -4.778621e-08

      ME2
      -4.778621e-08

      ICL
      9.654038e-10

      MALS
      9.654038e-10

      BIOMASS_Ec_iML1515_core_75p37M
      6.500000e-01

68 rows × 1 columns

### Set bounds
#### Growth rate and media

```python
biomass_rxn = cobra_model.reactions.BIOMASS_Ec_iML1515_core_75p37M
growth_rate = flux_data.loc[biomass_rxn.id][0]
biomass_rxn.bounds = (growth_rate, growth_rate)
biomass_rxn.bounds
```

    (0.65, 0.65)

```python
EX_glc__D_e = cobra_model.reactions.EX_glc__D_e
medium_uptake = flux_data.loc[EX_glc__D_e.id][0]
EX_glc__D_e.bounds = (medium_uptake, medium_uptake)
EX_glc__D_e.bounds
```

    (-9.654, -9.654)

### Formulate QP minimization for fluxes

```python
v_vars = []
v_data = []

# For irreversible enzyme pairs, flux data is given as Enzyme1 - Enzyme2 = value.
# To ensure all enzymes have some flux, add a percentage of the net flux for each enzyme
# The netflux will still remain the same value.
reverse_flux_percent = 0.1

irreversible_enzyme_pairs = [["PFK", "FBP"], ["PYK", "PPS"]]

for rid, flux in flux_data.itertuples():
    # Make adjustments to net flux of PFK/FBP and PYK/PPS to ensure
    # no target flux value is 0 in order to create an enzyme module.
    for irreversible_enzyme_pair in irreversible_enzyme_pairs:
        if rid in irreversible_enzyme_pair:
            flux1, flux2 = flux_data.loc[irreversible_enzyme_pair, "Flux (mmol * gDW-1 * h-1)"].values
            if flux1 == 0:
                flux += reverse_flux_percent * flux2 # mmol*gDW^-1*hr^-1
            if flux2 == 0:
                flux += reverse_flux_percent * flux1 # mmol*gDW^-1*hr^-1
            print(rid, flux)
    v_vars.append(sym.Symbol(rid))
    v_data.append(flux)

# Make symbolic for optlang objective
v_vars = sym.Matrix(v_vars)
v_data = sym.Matrix(v_data)
F = sym.Matrix(2 * sym.eye(len(v_vars)))

objective = 0.5 * v_vars.T * F * v_vars  - (2 * v_data).T * v_vars
cobra_model.objective = objective[0]
cobra_model.objective_direction = "min"

flux_solution = cobra_model.optimize()
flux_solution
```

    PFK 7.76432470140912
    FBP 0.7058477001281019
    PYK 2.7360389076214697
    PPS 0.24873080978377

Optimal solution with objective value -1075.383

      fluxes
      reduced_costs

      CYTDK2
      0.000000
      0.000000

      XPPT
      0.000000
      0.000000

      HXPRT
      0.000000
      0.045104

      NDPK5
      0.017561
      0.000000

      SHK3Dr
      0.247727
      0.000000

      ...
      ...
      ...

      MPTS
      0.000000
      0.000000

      MOCOS
      0.000000
      0.000000

      BMOGDS2
      0.000000
      0.000000

      FESD2s
      0.000000
      0.000000

      OCTNLL
      0.000000
      2.611245

2712 rows × 2 columns

```python
flux_comparison_fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(5, 5))
plot_comparison(
    x=flux_data["Flux (mmol * gDW-1 * h-1)"], y=flux_solution, compare="fluxes",
    observable=[rid for rid in flux_data.index], ax=ax,
    legend="right outside", plot_function="plot",
    xlim=(-16.5, 16.5), ylim=(-16.5, 16.5),
    xy_line=True,
    xy_legend="best", xlabel="Measured [mmol/(gDW * h)]", ylabel="Adjusted [mmol/(gDW * h)]")

flux_comparison_fig.tight_layout()
```

#### Export data for analysis

```python
flux_data_for_comparison = pd.concat(objs=(flux_data, flux_solution.fluxes), axis=1).dropna()
flux_data_for_comparison.index.name = "ID"
flux_data_for_comparison.columns = ["Initial", "Adjusted"]
flux_data_for_comparison.to_csv("./data/analysis_data/fluxes.csv")
```

## Create MASS Model

```python
# Create MassModel
mass_model = MassModel("Glycolysis", array_type="DataFrame")

# Reactions to extract into subnetwork
reaction_list = [
    "PGI",
    "PFK",
    "FBP",
    "FBA",
    "TPI",
    "GAPD",
    "PGK",
    "PGM",
    "ENO",
    "PYK",
    "PPS",
    "LDH_D",
]

cobra_reactions = cobra_model.reactions.get_by_any(reaction_list)
mass_model.add_reactions([MassReaction(rxn) for rxn in cobra_reactions])
mass_model
```

        Name	Glycolysis

        Memory address	0x07ff0d1ebc5b0

        Stoichiometric Matrix
        19x12

        Matrix Rank
        12

        Number of metabolites
        19

        Initial conditions defined
        0/19

        Number of reactions
        12

        Number of genes
        12

        Number of enzyme modules
        0

        Number of groups
        0

        Objective expression
        0

        Compartments
        c

### Convert flux units to M/s

```python
T = 313.15
gas_constant = 0.008314
e_coli_density = 1.1 # g / mL assumption
volume = 3.2  # femtoliter

# Perform conversions
doubling_time_per_minute = np.log(2) / growth_rate * 60
cell_gDW = 42000 * doubling_time_per_minute**-1.232 * 1e-15
real_cell_total_weight = e_coli_density * (volume * 1e-12) # fL --> mL

# Assume water is 70%
adj_volume = volume * 0.7
gDW_L_conversion_factor = real_cell_total_weight / (adj_volume * 1e-15)

for reaction in mass_model.reactions.get_by_any(reaction_list):
    flux = flux_solution[reaction.id]
    reaction.steady_state_flux = flux * gDW_L_conversion_factor * 0.001 / 3600
```

## Set equilibrium constants

```python
Keq_data = pd.read_excel(
    io="./data/growth_data.xlsx",
    sheet_name="Keq_data",
    index_col=0
)
for reaction in mass_model.reactions.get_by_any(reaction_list):
    reaction.Keq = Keq_data.loc[reaction.Keq_str][0]
```

```python
conc_data = pd.read_excel(
    io="./data/growth_data.xlsx",
    sheet_name="conc_data",
    index_col=0
)
conc_data = conc_data.loc[lambda x: x['Growth Medium'] == "Glucose"]
conc_data = conc_data.drop("Growth Medium", axis=1)
conc_data
```

      Concentration (mol * L-1)

      ID

      adp_c
      0.002185

      amp_c
      0.001743

      atp_c
      0.012466

      nad_c
      0.007636

      nadh_c
      0.000099

      13dpg_c
      0.000184

      f6p_c
      0.001214

      dhap_c
      0.004772

      g6p_c
      0.003431

      pep_c
      0.001276

      pyr_c
      0.005079

      2pg_c
      0.002876

      3pg_c
      0.002876

      fdp_c
      0.004196

      gdp_c
      0.001206

### Add PFK1 activator GDP

```python
gdp_c = MassMetabolite(cobra_model.metabolites.gdp_c)
# Set the activator as a constant
gdp_c.fixed = True

mass_model.add_metabolites(gdp_c)
```

### Set initial concentrations from growth data

```python
mass_model.update_initial_conditions({
    mid: value for mid, value in conc_data.itertuples()
})

# Fix hydrogen and water as constants and set concentration to 1.
for metabolite in mass_model.metabolites.get_by_any(["h2o_c", "h_c"]):
    metabolite.fixed = True
    metabolite.initial_condition = 1

missing_ics = mass_model.metabolites.query(lambda m: m.initial_condition is None)
# Provide initial guesses for missing metabolites (pi_c, g3p_c, and lac__D_c)
print(missing_ics)
for metabolite in missing_ics:
    metabolite.initial_condition = 0.001
```

    [<MassMetabolite pi_c at 0x7ff0d1ebcfa0>, <MassMetabolite g3p_c at 0x7ff100dd8280>, <MassMetabolite lac__D_c at 0x7ff100dd8eb0>]

### Formulate QP minimization for concentrations

```python
from mass.thermo import (
    ConcSolver, sample_concentrations,
    update_model_with_concentration_solution)
```

```python
conc_solver = ConcSolver(
    mass_model,
    excluded_metabolites=["h_c", "h2o_c"],
    constraint_buffer=1,
    equilibrium_reactions=[x.id for x in mass_model.reactions if x.steady_state_flux == 0]

)

conc_solver.setup_feasible_qp_problem(
    fixed_conc_bounds=list(mass_model.fixed),
)

conc_solution = conc_solver.optimize()
conc_solution
```

Optimal solution with objective value 0.000

      variables
      reduced_costs

      f6p_c
      0.001711
      0.0

      g6p_c
      0.002435
      0.0

      adp_c
      0.002185
      0.0

      atp_c
      0.012466
      0.0

      fdp_c
      0.009390
      0.0

      ...
      ...
      ...

      Keq_PGM
      2.178140
      0.0

      Keq_ENO
      5.190000
      0.0

      Keq_PYK
      21739.130400
      0.0

      Keq_PPS
      2.410000
      0.0

      Keq_LDH_D
      0.000240
      0.0

30 rows × 2 columns

```python
conc_comparison_fig, ax = plt.subplots(nrows=1, ncols=1,
                                       figsize=(5, 5))

plot_comparison(
    x=conc_data["Concentration (mol * L-1)"], y=conc_solution,
    compare="concentrations",
    observable=[mid for mid in conc_data.index], ax=ax,
    legend="right outside", plot_function="loglog",
    xlim=(1e-5, 1e-1), ylim=(1e-5, 1e-1), xy_line=True,
    xy_legend="best", xlabel="Initial [mol/L]", ylabel="Adjusted [mol/L]")

conc_comparison_fig.tight_layout()
update_model_with_concentration_solution(
    mass_model, conc_solution, concentrations=True, Keqs=True, inplace=True);
```

#### Export data for analysis

```python
conc_data_for_comparison = pd.concat(objs=(conc_data, conc_solution.concentrations), axis=1).dropna()
conc_data_for_comparison.index.name = "ID"
conc_data_for_comparison.columns = ["Initial", "Adjusted"]
conc_data_for_comparison.to_csv("./data/analysis_data/concentrations.csv")
conc_data_for_comparison
```

      Initial
      Adjusted

      ID

      adp_c
      0.002185
      0.002185

      amp_c
      0.001743
      0.001743

      atp_c
      0.012466
      0.012466

      nad_c
      0.007636
      0.035436

      nadh_c
      0.000099
      0.000021

      13dpg_c
      0.000184
      0.000141

      f6p_c
      0.001214
      0.001711

      dhap_c
      0.004772
      0.004062

      g6p_c
      0.003431
      0.002435

      pep_c
      0.001276
      0.001276

      pyr_c
      0.005079
      0.001420

      2pg_c
      0.002876
      0.001182

      3pg_c
      0.002876
      0.006999

      fdp_c
      0.004196
      0.009390

      gdp_c
      0.001206
      0.001206

```python
Keq_comparison_fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(5, 5))

plot_comparison(
    x=Keq_data["Equilibrium Constant"], y=conc_solution,
    compare="Keqs", ax=ax,
    legend="right outside", plot_function="loglog",
    xlim=(1e-5, 1e5), ylim=(1e-5, 1e5), xy_line=True,
    xy_legend="best", xlabel="Initial", ylabel="Adjusted")

conc_comparison_fig.tight_layout()
update_model_with_concentration_solution(
    mass_model, conc_solution, concentrations=True, Keqs=True, inplace=True);
```

#### Export data for analysis

```python
Keq_data_for_comparison = pd.concat(objs=(Keq_data, conc_solution.Keqs), axis=1).dropna()
Keq_data_for_comparison.index.name = "ID"
Keq_data_for_comparison.columns = ["Initial", "Adjusted"]
Keq_data_for_comparison.to_csv("./data/analysis_data/equilibrium_constants.csv")
Keq_data_for_comparison
```

      Initial
      Adjusted

      ID

      Keq_PGI
      0.364000
      0.258394

      Keq_PFK
      2000.000000
      2000.000000

      Keq_FBP
      41.500000
      41.500000

      Keq_FBA
      0.000160
      0.000358

      Keq_TPI
      0.106952
      0.203736

      Keq_GAPD
      0.452000
      0.586631

      Keq_PGK
      0.000530
      0.000530

      Keq_PGM
      5.300000
      2.178140

      Keq_ENO
      5.190000
      5.190000

      Keq_PYK
      21739.130400
      21739.130400

      Keq_PPS
      2.410000
      2.410000

      Keq_LDH_D
      0.000067
      0.000240

```python
# Fix Metabolite IDs as SBML compatible before next step
for metabolite in mass_model.metabolites:
    if metabolite.id[0].isdigit():
        metabolite.id = f"_{metabolite.id}"
mass_model.repair()
```

```python
n_models = 5
```

```python
conc_solver = ConcSolver(
    mass_model,
    excluded_metabolites=["h_c", "h2o_c"],
    constraint_buffer=1,
    equilibrium_reactions=[x.id for x in mass_model.reactions if x.steady_state_flux == 0]

)

conc_solver.setup_sampling_problem(
    fixed_conc_bounds=list(mass_model.fixed),
)
for variable in conc_solver.variables:
    try:
        met = mass_model.metabolites.get_by_id(variable.name)
        variable.lb, variable.ub = np.log([met.ic / 10, met.ic * 10])
    except:
        pass
conc_samples = sample_concentrations(conc_solver, n=n_models, seed=4)
conc_samples
```

      f6p_c
      g6p_c
      adp_c
      atp_c
      fdp_c
      pi_c
      dhap_c
      g3p_c
      _13dpg_c
      nad_c
      nadh_c
      _3pg_c
      _2pg_c
      pep_c
      pyr_c
      amp_c
      lac__D_c
      gdp_c

      0
      0.010470
      0.010661
      0.013486
      0.105633
      0.024770
      0.004267
      0.005646
      0.000339
      0.000149
      0.073327
      0.000050
      0.011918
      0.001586
      0.001170
      0.000168
      0.000468
      0.000491
      0.000121

      1
      0.000553
      0.000743
      0.003268
      0.001477
      0.005574
      0.004123
      0.002493
      0.000171
      0.000108
      0.112689
      0.000071
      0.038825
      0.005840
      0.002756
      0.001839
      0.008645
      0.004783
      0.000456

      2
      0.003326
      0.004251
      0.003567
      0.001996
      0.011121
      0.011735
      0.003586
      0.000109
      0.000027
      0.147767
      0.000113
      0.019942
      0.000497
      0.000485
      0.000204
      0.004712
      0.000692
      0.000146

      3
      0.001739
      0.002718
      0.009105
      0.001749
      0.002127
      0.008857
      0.002809
      0.000071
      0.000029
      0.043257
      0.000055
      0.038715
      0.000583
      0.000700
      0.000203
      0.002591
      0.001066
      0.000122

      4
      0.001519
      0.001621
      0.004541
      0.002020
      0.003339
      0.012093
      0.000720
      0.000049
      0.000028
      0.033218
      0.000030
      0.028609
      0.000541
      0.000468
      0.000170
      0.001633
      0.000637
      0.000121

### Balance network with pseudoreactions and calculate PERCs

```python
models_for_ensemble = []
for idx, conc_sample in conc_samples.iterrows():
    # Make copy of new model
    new_model = mass_model.copy()
    new_model.id += "_C{0:d}".format(idx)
    print(f"Creating model {new_model.id}")
    # Get concentration sample and update model with sample
    new_model.update_initial_conditions(conc_sample.to_dict())
    fluxes = np.array(list(new_model.steady_state_fluxes.values()))
    imbalanced_metabolites = new_model.S.dot(fluxes)
    # Iterate through metabolites
    for mid, imbalance in imbalanced_metabolites.iteritems():
        # Ignore balanced metabolites
        if imbalance == 0:
            continue
        # Get metabolite object
        met = new_model.metabolites.get_by_id(mid)

        # Add boundary reactions for imbalanced metabolites
        boundary_type = "sink"
        # Add boundary reaction with imbalance as flux value
        boundary_reaction = new_model.add_boundary(
            mid, boundary_type, boundary_condition=met.ic)

        boundary_reaction.Keq = 1
        if imbalance < 0:
            boundary_reaction.reverse_stoichiometry(inplace=True)
            imbalance = -imbalance

        boundary_reaction.kf = imbalance / met.ic
        boundary_reaction.steady_state_flux = imbalance
        try:
            # Update PERCs
            new_model.calculate_PERCs(
                fluxes={
                    r: v for r, v in new_model.steady_state_fluxes.items()
                    if not r.boundary},
                update_reactions=True)
        except:
            print("Negative PERCs for {0}".format(new_model.id))
            continue
    models_for_ensemble.append(new_model)
print("Number of models in ensemble: {0:d}".format(
    len(models_for_ensemble)))
```

    Creating model Glycolysis_C0
    Creating model Glycolysis_C1
    Creating model Glycolysis_C2
    Creating model Glycolysis_C3
    Creating model Glycolysis_C4
    Number of models in ensemble: 5

```python
reference_model = models_for_ensemble[0].copy()
reference_model.id = "Glycolysis_Ref_wo_Enzymes"
sim = Simulation(reference_model)
sim.integrator.absolute_tolerance = 1e-15
sim.integrator.relative_tolerance = 1e-9

tfinal = 1e6
conc_sol, flux_sol = sim.simulate(reference_model, time=(0, tfinal))
conc_sol.view_time_profile()

```

#### Save a reference MASS model w/o enzymes

```python
conc_sol, flux_sol = sim.find_steady_state(
    models=reference_model, strategy="simulate",
    tfinal=tfinal)
if conc_sol and flux_sol:
    reference_model.update_initial_conditions(conc_sol)
    reference_model.update_parameters({f"v_{k}": v for k, v in flux_sol.items()})
    # Save a reference MASS model
    save_mass_json_model(
        mass_model=reference_model,
        filename=f"./models/mass/without_enzymes/{reference_model.id}.json")
    print(f"Saving {reference_model.id}")
reference_model
```

    Saving Glycolysis_Ref_wo_Enzymes

        Name	Glycolysis_Ref_wo_Enzymes

        Memory address	0x07ff0f115da30

        Stoichiometric Matrix
        20x27

        Matrix Rank
        19

        Number of metabolites
        20

        Initial conditions defined
        20/20

        Number of reactions
        27

        Number of genes
        12

        Number of enzyme modules
        0

        Number of groups
        0

        Objective expression
        0

        Compartments
        c

## Create Enzyme Modules

Assume 90% of flux goes through the major isozyme, the remaining through the minor isozyme

```python
from construction_functions import make_enzyme_module_from_dir
```

```python
isozyme1_percent = 0.9
isozyme2_percent = 0.1

# Isozymes and flux split percentages,
isozymes_and_flux_splits = {
    "PFK": {
        "PFK1": isozyme1_percent,
        "PFK2": isozyme2_percent,
    },
    "FBP": {
        "FBP1": isozyme1_percent,
        "FBP2": isozyme2_percent,
    },
    "FBA": {
        "FBA1": isozyme1_percent,
        "FBA2": isozyme2_percent,
    },
    "PGM": {
        "PGMi": isozyme1_percent,
        "PGMd": isozyme2_percent,
    },
    "PYK": {
        "PYK1": isozyme1_percent,
        "PYK2": isozyme2_percent,
    },
}

isozymes_and_flux_splits
```

    {'PFK': {'PFK1': 0.9, 'PFK2': 0.1},
     'FBP': {'FBP1': 0.9, 'FBP2': 0.1},
     'FBA': {'FBA1': 0.9, 'FBA2': 0.1},
     'PGM': {'PGMi': 0.9, 'PGMd': 0.1},
     'PYK': {'PYK1': 0.9, 'PYK2': 0.1}}

```python
final_ensemble = []
for model in models_for_ensemble:
    enzyme_modules = {}
    for reaction in model.reactions.get_by_any(reaction_list):
        # PGM & PGK needs flux flipped since enzyme module stoichiometry
        # is reversed compared to lone reaction.
        if reaction.id in ["PGK", "PGM"]:
            flux = -reaction.steady_state_flux
        else:
            flux = reaction.steady_state_flux
        # Make isozymes
        if reaction.id in isozymes_and_flux_splits:
            isozymes_and_flux_split = isozymes_and_flux_splits[reaction.id]
            isozyme_modules = []
            for isozyme, flux_split in isozymes_and_flux_split.items():
                enzyme_module = make_enzyme_module_from_dir(
                    enzyme_id=isozyme,
                    steady_state_flux=flux * flux_split, # Split flux for isozymes
                    metabolite_concentrations=model.initial_conditions,
                    path_to_dir="./data/enzyme_module_data",
                    kcluster=1,
                    enzyme_gpr=reaction.gene_reaction_rule,
                    zero_tol=1e-10)
                isozyme_modules += [enzyme_module]
            enzyme_modules[reaction] = isozyme_modules
        else:
            enzyme_module = make_enzyme_module_from_dir(
                enzyme_id=reaction.id,
                steady_state_flux=flux,
                metabolite_concentrations=new_model.initial_conditions,
                path_to_dir="./data/enzyme_module_data",
                kcluster=1,
                enzyme_gpr=reaction.gene_reaction_rule,
                zero_tol=1e-10)
            enzyme_modules[reaction] = [enzyme_module]
    for reaction_to_remove, enzymes_to_add in enzyme_modules.items():
        model.remove_reactions([reaction_to_remove])
        for enzyme in enzymes_to_add:
            model = model.merge(enzyme, inplace=True)
    final_ensemble += [model]
    print(f"Finished {model.id}")
```

    Finished Glycolysis_C0
    Finished Glycolysis_C1
    Finished Glycolysis_C2
    Finished Glycolysis_C3
    Finished Glycolysis_C4

# Inspect a model

```python
reference_model = models_for_ensemble[0].copy()
reference_model.id = "Glycolysis_Ref_w_Enzymes"
sim = Simulation(reference_model)
sim.integrator.absolute_tolerance = 1e-15
sim.integrator.relative_tolerance = 1e-9

tfinal = 1e6
conc_sol, flux_sol = sim.simulate(reference_model, time=(0, tfinal))
conc_sol.view_time_profile(plot_function="semilogx")

```

#### Save a reference MASS model w/ enzymes

```python
conc_sol, flux_sol = sim.find_steady_state(
    models=reference_model, strategy="simulate", update_values=True,
    tfinal=tfinal)
if conc_sol and flux_sol:
    # Save a reference MASS model
    save_mass_json_model(
        mass_model=reference_model,
        filename=f"./models/mass/with_enzymes/{reference_model.id}.json")
    print(f"Saving {reference_model.id}")
reference_model
```

    Saving Glycolysis_Ref_w_Enzymes

        Name	Glycolysis_Ref_w_Enzymes

        Memory address	0x07ff0c1f03880

        Stoichiometric Matrix
        120x121

        Matrix Rank
        102

        Number of metabolites
        120

        Initial conditions defined
        120/120

        Number of reactions
        121

        Number of genes
        19

        Number of enzyme modules
        17

        Number of groups
        0

        Objective expression
        0

        Compartments
        c

## Simulate to steady state and export ensemble

```python
sim = Simulation(final_ensemble[0])
sim.integrator.absolute_tolerance = 1e-15
sim.integrator.relative_tolerance = 1e-9
sim.integrator.
sim.add_models(final_ensemble[1:], disable_safe_load=True)
for model in final_ensemble:
    # Attempt to determine steady state
    conc_sol, flux_sol = sim.find_steady_state(
        models=model, strategy="simulate", update_values=True,
        tfinal=tfinal)
    if conc_sol and flux_sol:
        # Save a reference MASS model
        save_mass_json_model(
            mass_model=model,
            filename=f"./models/mass/with_enzymes/{model.id}.json")
        print(f"Saving {model.id}")
    else:
        print(f"No steady state for {model.id}.")
```

    Saving Glycolysis_C0
    Saving Glycolysis_C1
    Saving Glycolysis_C2
    Saving Glycolysis_C3
    Saving Glycolysis_C4

```python

```

Module4_Kinetics/Part2_Exercise/COBRA to MASS model workflow.ipynb


```python
import sys
sys.path.append('..')
import os
import torch
import numpy as np
import matplotlib.pyplot as plt
from sympy import simplify_logic
import time
from sklearn.metrics import accuracy_score
import pandas as pd
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.tree import _tree, export_text
from torchvision import datasets, transforms
from torch.utils.data import TensorDataset, DataLoader
from torch.utils.data.sampler import SubsetRandomSampler
import torch.nn.functional as F
from sklearn.preprocessing import OneHotEncoder, MinMaxScaler, LabelEncoder
from sklearn.model_selection import StratifiedKFold

from lens.utils.base import validate_network, set_seed, tree_to_formula
from lens.utils.relunn import get_reduced_model, prune_features
from lens.utils.sigmoidnn import prune_equal_fanin
from lens import logic

results_dir = 'results/omalizumab_full'
if not os.path.isdir(results_dir):
    os.makedirs(results_dir)

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
n_rep = 10
tot_epochs = 6001
prune_epochs = 3001
```

```python
gene_expression_matrix = pd.read_csv('data/omalizumab/w_1/data_0.csv', index_col=None, header=None)
labels = pd.read_csv('data/omalizumab/w_1/tempLabels_W-1.csv', index_col=None, header=None)
genes = pd.read_csv('data/omalizumab/w_1/features_0.csv', index_col=None, header=None)
gene_expression_matrix
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      ...
      28392
      28393
      28394
      28395
      28396
      28397
      28398
      28399
      28400
      28401

      0
      14.622486
      11.162004
      3.320000
      3.320000
      3.320000
      12.788433
      6.143456
      3.320000
      4.876620
      3.320000
      ...
      3.32
      3.320000
      3.885589
      3.914260
      3.320000
      3.32
      3.320000
      4.465420
      3.320000
      4.973620

      1
      14.398743
      11.000080
      3.320000
      3.320000
      3.320000
      12.845914
      6.147482
      3.320000
      4.484223
      3.320000
      ...
      3.32
      3.575025
      4.236519
      4.047825
      3.320000
      3.32
      4.176269
      3.320000
      4.553796
      4.967418

      2
      14.692079
      11.100175
      3.320000
      4.171535
      3.320000
      12.712544
      5.583210
      3.320000
      3.478171
      3.320000
      ...
      3.32
      3.320000
      3.992331
      4.865538
      3.320000
      3.32
      3.488281
      3.406285
      3.320000
      6.676063

      3
      14.613382
      11.023209
      3.320000
      3.320000
      3.320000
      12.750496
      5.688023
      3.320000
      4.464426
      3.320000
      ...
      3.32
      3.855643
      3.320000
      4.905350
      3.320000
      3.32
      4.158393
      4.433457
      3.874214
      5.981160

      4
      14.482065
      10.989851
      3.320000
      3.992726
      4.574745
      12.878702
      6.195418
      4.177962
      3.872567
      3.320000
      ...
      3.32
      3.320000
      3.320000
      4.879493
      3.320000
      3.32
      4.571869
      3.320000
      4.982136
      6.145585

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      56
      14.565031
      11.699843
      3.320000
      3.320000
      3.320000
      12.789212
      6.504027
      3.320000
      3.320000
      6.182912
      ...
      3.32
      3.320000
      3.320000
      4.338456
      3.771718
      3.32
      3.320000
      3.320000
      3.320000
      6.100644

      57
      14.624502
      11.918757
      3.320000
      4.292406
      3.430485
      10.728709
      6.197159
      3.320000
      4.089918
      5.201608
      ...
      3.32
      3.320000
      3.320000
      4.790134
      3.320000
      3.32
      4.985474
      4.444057
      3.580523
      6.301926

      58
      14.585190
      11.090112
      3.320000
      3.674768
      3.320000
      12.877485
      6.326960
      3.320000
      3.320000
      3.320000
      ...
      3.32
      3.320000
      3.320000
      4.547342
      3.320000
      3.32
      4.064473
      3.320000
      4.254152
      5.964505

      59
      14.449554
      10.805855
      3.320000
      3.320000
      3.320000
      12.660038
      6.261395
      3.320000
      4.125096
      3.320000
      ...
      3.32
      3.784260
      3.644823
      4.546974
      3.427441
      3.32
      4.666265
      3.888525
      3.765754
      5.452018

      60
      14.439020
      11.080826
      4.152651
      3.320000
      3.320000
      12.187504
      6.154358
      3.320000
      4.247009
      3.320000
      ...
      3.32
      3.671102
      3.519157
      4.771458
      3.320000
      3.32
      5.115644
      3.320000
      3.410361
      6.449961

61 rows × 28402 columns

```python
encoder = LabelEncoder()
labels_encoded = encoder.fit_transform(labels.values)
labels_encoded_noncontrols = labels_encoded[labels_encoded!=0] - 1

data_controls = gene_expression_matrix[labels_encoded==0]
data = gene_expression_matrix[labels_encoded!=0]

gene_signature = data_controls.mean(axis=0)
data_scaled = data - gene_signature

scaler = MinMaxScaler((0, 1))
scaler.fit(data_scaled)
data_normalized = scaler.transform(data_scaled)

x = torch.FloatTensor(data_normalized)
y = torch.LongTensor(labels_encoded_noncontrols)
print(x.shape)
print(y.shape)
```

    c:\users\pietr\anaconda3\envs\deep-logic\lib\site-packages\sklearn\utils\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
      return f(*args, **kwargs)

    torch.Size([40, 28402])
    torch.Size([40])

```python
concepts = list(genes.values.squeeze())
concepts[:10]
```

    ['ILMN_1343291',
     'ILMN_1343295',
     'ILMN_1651199',
     'ILMN_1651209',
     'ILMN_1651221',
     'ILMN_1651228',
     'ILMN_1651229',
     'ILMN_1651230',
     'ILMN_1651232',
     'ILMN_1651237']

```python
n_splits = 10
skf = StratifiedKFold(n_splits=n_splits, shuffle=True, random_state=0)
```

```python
def train_nn(x_train, y_train, need_pruning, seed, device, relu=False):
    set_seed(seed)
    x_train = x_train.to(device)
    y_train = y_train.to(device)
    layers = [
        torch.nn.Linear(x_train.size(1), 50),
        torch.nn.ReLU() if relu else torch.nn.LeakyReLU(),
        torch.nn.Linear(50, 20),
        torch.nn.ReLU() if relu else torch.nn.LeakyReLU(),
        torch.nn.Linear(20, 5),
        torch.nn.ReLU() if relu else torch.nn.LeakyReLU(),
        torch.nn.Linear(5, 2),
        torch.nn.Softmax(dim=1),
    ]
    model = torch.nn.Sequential(*layers).to(device)

    optimizer = torch.optim.AdamW(model.parameters(), lr=0.0001)
    loss_form = torch.nn.CrossEntropyLoss()
    model.train()
    for epoch in range(tot_epochs):
        # forward pass
        optimizer.zero_grad()
        y_pred = model(x_train)
        # Compute Loss
        loss = loss_form(y_pred, y_train)

        for module in model.children():
            if isinstance(module, torch.nn.Linear):
                loss += 0.0001 * torch.norm(module.weight, 1)
                loss += 0.0001 * torch.norm(module.bias, 1)
                break

        # backward pass
        loss.backward()
        optimizer.step()

        if epoch > prune_epochs and need_pruning and epoch % 1000 == 1:
            prune_features(model, n_classes=1, device=device)
            need_pruning = True

        # compute accuracy
        if epoch % 500 == 0:
            y_pred_d = torch.argmax(y_pred, dim=1)
            accuracy = y_pred_d.eq(y_train).sum().item() / y_train.size(0)
            print(f'\t Epoch {epoch}: train accuracy: {accuracy:.4f}')

    return model
```

# General pruning

```python
need_pruning = True
method = 'pruning'
methods = []
splits = []
explanations = []
explanations_inv = []
model_accuracies = []
explanation_accuracies = []
explanation_accuracies_inv = []
elapsed_times = []
elapsed_times_inv = []

for split, (train_index, test_index) in enumerate(skf.split(x.cpu().detach().numpy(), y.cpu().detach().numpy())):
    print(f'Split [{split+1}/{n_splits}]')
    x_train, x_test = torch.FloatTensor(x[train_index]), torch.FloatTensor(x[test_index])
    y_train, y_test = torch.LongTensor(y[train_index]), torch.LongTensor(y[test_index])

#     if split not in [5]: continue

    model = train_nn(x_train, y_train, need_pruning, split, device)

    y_preds = model(x_test.to(device)).cpu().detach().numpy()
    model_accuracy = accuracy_score(y_test.cpu().detach().numpy(), y_preds.argmax(axis=1))
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    # positive class
    target_class = 1
    start = time.time()
    global_explanation, _, counter = logic.relunn.combine_local_explanations(model,
                                                                       x_train.to(device), y_train.to(device),
                                                                       target_class=target_class,
                                                                       topk_explanations=2,
                                                                       method=method, device=device)
    elapsed_time = time.time() - start
    if global_explanation:
        explanation_accuracy, _ = logic.base.test_explanation(global_explanation, target_class, x_test, y_test)
        explanation = logic.base.replace_names(global_explanation, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation}" - Accuracy: {explanation_accuracy:.4f}')
    print(f'\t Elapsed time {elapsed_time}')

    # negative class
    target_class = 0
    start = time.time()
    global_explanation_inv, _, counter_inv = logic.relunn.combine_local_explanations(model,
                                                                           x_train.to(device), y_train.to(device),
                                                                           target_class=target_class,
                                                                           topk_explanations=2,
                                                                           method=method, device=device)
    elapsed_time_inv = time.time() - start
    if global_explanation_inv:
        explanation_accuracy_inv, _ = logic.base.test_explanation(global_explanation_inv, target_class, x_test, y_test)
        explanation_inv = logic.base.replace_names(global_explanation_inv, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation_inv}" - Accuracy: {explanation_accuracy_inv:.4f}')
    print(f'\t Elapsed time {elapsed_time_inv}')

    methods.append(method)
    splits.append(split)
    explanations.append(explanation)
    explanations_inv.append(explanation_inv)
    model_accuracies.append(model_accuracy)
    explanation_accuracies.append(explanation_accuracy)
    explanation_accuracies_inv.append(explanation_accuracy_inv)
    elapsed_times.append(elapsed_time)
    elapsed_times_inv.append(elapsed_time_inv)
```

```python
results_pruning = pd.DataFrame({
    'method': methods,
    'split': splits,
    'explanation': explanations,
    'explanation_inv': explanations_inv,
    'model_accuracy': model_accuracies,
    'explanation_accuracy': explanation_accuracies,
    'explanation_accuracy_inv': explanation_accuracies_inv,
    'elapsed_time': elapsed_times,
    'elapsed_time_inv': elapsed_times_inv,
})
results_pruning.to_csv(os.path.join(results_dir, 'results_pruning.csv'))
results_pruning
```

# LIME

```python
need_pruning = False
method = 'lime'
methods = []
splits = []
explanations = []
explanations_inv = []
model_accuracies = []
explanation_accuracies = []
explanation_accuracies_inv = []
elapsed_times = []
elapsed_times_inv = []
for seed in range(n_rep):
    print(f'Seed [{seed+1}/{n_rep}]')

    model = train_nn(x_train, y_train, need_pruning, seed, device)

    y_preds = model(x_test.to(device)).cpu().detach().numpy()
    model_accuracy = accuracy_score(y_test.cpu().detach().numpy(), y_preds.argmax(axis=1))
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    # positive class
    target_class = 1
    start = time.time()
    global_explanation, _, _ = logic.relunn.combine_local_explanations(model,
                                                                       x_train.to(device), y_train.to(device),
                                                                       topk_explanations=2,
                                                                       target_class=target_class,
                                                                       method=method, device=device)
    elapsed_time = time.time() - start
    if global_explanation:
        explanation_accuracy, _ = logic.base.test_explanation(global_explanation, target_class, x_test, y_test)
        explanation = logic.base.replace_names(global_explanation, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation}" - Accuracy: {explanation_accuracy:.4f}')
    print(f'\t Elapsed time {elapsed_time}')

    # negative class
    target_class = 0
    start = time.time()
    global_explanation_inv, _, _ = logic.relunn.combine_local_explanations(model,
                                                                           x_train.to(device), y_train.to(device),
                                                                           topk_explanations=2,
                                                                           target_class=target_class,
                                                                           method=method, device=device)
    elapsed_time_inv = time.time() - start
    if global_explanation_inv:
        explanation_accuracy_inv, _ = logic.base.test_explanation(global_explanation_inv, target_class, x_test, y_test)
        explanation_inv = logic.base.replace_names(global_explanation_inv, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation_inv}" - Accuracy: {explanation_accuracy_inv:.4f}')
    print(f'\t Elapsed time {elapsed_time_inv}')

    methods.append(method)
    splits.append(seed)
    explanations.append(explanation)
    explanations_inv.append(explanation_inv)
    model_accuracies.append(model_accuracy)
    explanation_accuracies.append(explanation_accuracy)
    explanation_accuracies_inv.append(explanation_accuracy_inv)
    elapsed_times.append(elapsed_time)
    elapsed_times_inv.append(elapsed_time_inv)
```

```python
results_lime = pd.DataFrame({
    'method': methods,
    'split': splits,
    'explanation': explanations,
    'explanation_inv': explanations_inv,
    'model_accuracy': model_accuracies,
    'explanation_accuracy': explanation_accuracies,
    'explanation_accuracy_inv': explanation_accuracies_inv,
    'elapsed_time': elapsed_times,
    'elapsed_time_inv': elapsed_times_inv,
})
results_lime.to_csv(os.path.join(results_dir, 'results_lime.csv'))
results_lime
```

# Weights

```python
need_pruning = False
method = 'weights'
methods = []
splits = []
explanations = []
explanations_inv = []
model_accuracies = []
explanation_accuracies = []
explanation_accuracies_inv = []
elapsed_times = []
elapsed_times_inv = []

for split, (train_index, test_index) in enumerate(skf.split(x.cpu().detach().numpy(), y.cpu().detach().numpy())):
    print(f'Split [{split+1}/{n_splits}]')
    x_train, x_test = torch.FloatTensor(x[train_index]), torch.FloatTensor(x[test_index])
    y_train, y_test = torch.LongTensor(y[train_index]), torch.LongTensor(y[test_index])

    model = train_nn(x_train, y_train, need_pruning, split, device, relu=True)

    y_preds = model(x_test.to(device)).cpu().detach().numpy()
    model_accuracy = accuracy_score(y_test.cpu().detach().numpy(), y_preds.argmax(axis=1))
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    # positive class
    target_class = 1
    start = time.time()
    global_explanation, _, _ = logic.relunn.combine_local_explanations(model,
                                                                       x_train.to(device), y_train.to(device),
                                                                       topk_explanations=2,
                                                                       target_class=target_class,
                                                                       method=method, device=device)
    elapsed_time = time.time() - start
    if global_explanation:
        explanation_accuracy, _ = logic.base.test_explanation(global_explanation, target_class, x_test, y_test)
        explanation = logic.base.replace_names(global_explanation, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation}" - Accuracy: {explanation_accuracy:.4f}')
    print(f'\t Elapsed time {elapsed_time}')

    # negative class
    target_class = 0
    start = time.time()
    global_explanation_inv, _, _ = logic.relunn.combine_local_explanations(model,
                                                                           x_train.to(device), y_train.to(device),
                                                                           topk_explanations=2,
                                                                           target_class=target_class,
                                                                           method=method, device=device)
    elapsed_time_inv = time.time() - start
    if global_explanation_inv:
        explanation_accuracy_inv, _ = logic.base.test_explanation(global_explanation_inv, target_class, x_test, y_test)
        explanation_inv = logic.base.replace_names(global_explanation_inv, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation_inv}" - Accuracy: {explanation_accuracy_inv:.4f}')
    print(f'\t Elapsed time {elapsed_time_inv}')

    methods.append(method)
    splits.append(split)
    explanations.append(explanation)
    explanations_inv.append(explanation_inv)
    model_accuracies.append(model_accuracy)
    explanation_accuracies.append(explanation_accuracy)
    explanation_accuracies_inv.append(explanation_accuracy_inv)
    elapsed_times.append(elapsed_time)
    elapsed_times_inv.append(elapsed_time_inv)
```

```python
results_weights = pd.DataFrame({
    'method': methods,
    'split': splits,
    'explanation': explanations,
    'explanation_inv': explanations_inv,
    'model_accuracy': model_accuracies,
    'explanation_accuracy': explanation_accuracies,
    'explanation_accuracy_inv': explanation_accuracies_inv,
    'elapsed_time': elapsed_times,
    'elapsed_time_inv': elapsed_times_inv,
})
results_weights.to_csv(os.path.join(results_dir, 'results_weights.csv'))
results_weights
```

# Psi network

```python
def train_psi_nn(x_train, y_train, need_pruning, seed, device):
    set_seed(seed)
    x_train = x_train.to(device)
    y_train = y_train.to(device).to(torch.float)
    layers = [
        torch.nn.Linear(x_train.size(1), 10),
        torch.nn.Sigmoid(),
        torch.nn.Linear(10, 4),
        torch.nn.Sigmoid(),
        torch.nn.Linear(4, 1),
        torch.nn.Sigmoid(),
    ]
    model = torch.nn.Sequential(*layers).to(device)

    optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
    loss_form = torch.nn.BCELoss()
    model.train()
    for epoch in range(tot_epochs):
        # forward pass
        optimizer.zero_grad()
        y_pred = model(x_train).squeeze()
        # Compute Loss
        loss = loss_form(y_pred, y_train)

        for module in model.children():
            if isinstance(module, torch.nn.Linear):
                loss += 0.0001 * torch.norm(module.weight, 1)

        # backward pass
        loss.backward()
        optimizer.step()

        if epoch > 1500 and need_pruning:
            model = prune_equal_fanin(model, 2, validate=True, device=device)
            need_pruning = False

        # compute accuracy
        if epoch % 500 == 0:
            y_pred_d = y_pred > 0.5
            accuracy = y_pred_d.eq(y_train).sum().item() / y_train.size(0)
            print(f'\t Epoch {epoch}: train accuracy: {accuracy:.4f}')

    return model
```

```python
need_pruning = True
method = 'psi'
methods = []
splits = []
explanations = []
explanations_inv = []
model_accuracies = []
explanation_accuracies = []
explanation_accuracies_inv = []
elapsed_times = []
elapsed_times_inv = []
for seed in range(n_rep):
    print(f'Seed [{seed+1}/{n_rep}]')

    # positive class
    target_class = 1
    model = train_psi_nn(x_train, y_train, need_pruning, seed, device)

    y_preds = model(x_test.to(device)).cpu().detach().numpy()
    model_accuracy = accuracy_score(y_test.cpu().detach().numpy(), y_preds > 0.5)
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    start = time.time()
    global_explanation = logic.generate_fol_explanations(model, device)[0]
    elapsed_time = time.time() - start
    explanation_accuracy, _ = logic.base.test_explanation(global_explanation, target_class, x_test, y_test)
    explanation = logic.base.replace_names(global_explanation, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation}" - Accuracy: {explanation_accuracy:.4f}')
    print(f'\t Elapsed time {elapsed_time}')

    # negative class
    target_class = 0
    model = train_psi_nn(x_train, y_train.eq(target_class), need_pruning, seed, device)

    y_preds = model(x_test.to(device)).cpu().detach().numpy()
    model_accuracy = accuracy_score(y_test.eq(target_class).cpu().detach().numpy(), y_preds > 0.5)
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    start = time.time()
    global_explanation_inv = logic.generate_fol_explanations(model, device)[0]
    elapsed_time_inv = time.time() - start
    explanation_accuracy_inv, _ = logic.base.test_explanation(global_explanation_inv,
                                                              target_class, x_test, y_test)
    explanation_inv = logic.base.replace_names(global_explanation_inv, concepts)
    print(f'\t Class {target_class} - Global explanation: "{explanation_inv}" - Accuracy: {explanation_accuracy_inv:.4f}')
    print(f'\t Elapsed time {elapsed_time_inv}')

    methods.append(method)
    splits.append(seed)
    explanations.append(explanation)
    explanations_inv.append(explanation_inv)
    model_accuracies.append(model_accuracy)
    explanation_accuracies.append(explanation_accuracy)
    explanation_accuracies_inv.append(explanation_accuracy_inv)
    elapsed_times.append(elapsed_time)
    elapsed_times_inv.append(elapsed_time_inv)
```

```python
results_psi = pd.DataFrame({
    'method': methods,
    'split': splits,
    'explanation': explanations,
    'explanation_inv': explanations_inv,
    'model_accuracy': model_accuracies,
    'explanation_accuracy': explanation_accuracies,
    'explanation_accuracy_inv': explanation_accuracies_inv,
    'elapsed_time': elapsed_times,
    'elapsed_time_inv': elapsed_times_inv,
})
results_psi.to_csv(os.path.join(results_dir, 'results_psi.csv'))
results_psi
```

# Decision tree

```python
need_pruning = False
method = 'decision_tree'
methods = []
splits = []
explanations = []
explanations_inv = []
model_accuracies = []
explanation_accuracies = []
explanation_accuracies_inv = []
elapsed_times = []
elapsed_times_inv = []

for split, (train_index, test_index) in enumerate(skf.split(x.cpu().detach().numpy(), y.cpu().detach().numpy())):
    print(f'Split [{split+1}/{n_splits}]')
    x_train, x_test = x[train_index], x[test_index]
    y_train, y_test = y[train_index], y[test_index]

    classifier = DecisionTreeClassifier(random_state=split)
    classifier.fit(x_train.cpu().detach().numpy(), y_train.cpu().detach().numpy())
    y_preds = classifier.predict(x_test.cpu().detach().numpy())
    model_accuracy = accuracy_score(y_test.cpu().detach().numpy(), y_preds)
    print(f'\t Model\'s accuracy: {model_accuracy:.4f}')

    target_class = 1
    start = time.time()
    explanation = tree_to_formula(classifier, concepts, target_class)
    elapsed_time = time.time() - start
    print(f'\t Class {target_class} - Global explanation: {explanation}')
    print(f'\t Elapsed time {elapsed_time}')

    target_class = 0
    start = time.time()
    explanation_inv = tree_to_formula(classifier, concepts, target_class)
    elapsed_time_inv = time.time() - start
    print(f'\t Class {target_class} - Global explanation: {explanation_inv}')
    print(f'\t Elapsed time {elapsed_time_inv}')

    methods.append(method)
    splits.append(split)
    explanations.append(explanation)
    explanations_inv.append(explanation_inv)
    model_accuracies.append(model_accuracy)
    explanation_accuracies.append(model_accuracy)
    explanation_accuracies_inv.append(model_accuracy)
    elapsed_times.append(0)
    elapsed_times_inv.append(0)
```

```python
results_tree = pd.DataFrame({
    'method': methods,
    'split': splits,
    'explanation': explanations,
    'explanation_inv': explanations_inv,
    'model_accuracy': model_accuracies,
    'explanation_accuracy': explanation_accuracies,
    'explanation_accuracy_inv': explanation_accuracies_inv,
    'elapsed_time': elapsed_times,
    'elapsed_time_inv': elapsed_times_inv,
})
results_tree.to_csv(os.path.join(results_dir, 'results_tree.csv'))
results_tree
```

# Summary

```python
cols = ['model_accuracy', 'explanation_accuracy', 'explanation_accuracy_inv', 'elapsed_time', 'elapsed_time_inv']
mean_cols = [f'{c}_mean' for c in cols]
sem_cols = [f'{c}_sem' for c in cols]

# pruning
df_mean = results_pruning[cols].mean()
df_sem = results_pruning[cols].sem()
df_mean.columns = mean_cols
df_sem.columns = sem_cols
summary_pruning = pd.concat([df_mean, df_sem])
summary_pruning.name = 'pruning'

# lime
df_mean = results_lime[cols].mean()
df_sem = results_lime[cols].sem()
df_mean.columns = mean_cols
df_sem.columns = sem_cols
summary_lime = pd.concat([df_mean, df_sem])
summary_lime.name = 'lime'

# weights
df_mean = results_weights[cols].mean()
df_sem = results_weights[cols].sem()
df_mean.columns = mean_cols
df_sem.columns = sem_cols
summary_weights = pd.concat([df_mean, df_sem])
summary_weights.name = 'weights'

# psi
df_mean = results_psi[cols].mean()
df_sem = results_psi[cols].sem()
df_mean.columns = mean_cols
df_sem.columns = sem_cols
summary_psi = pd.concat([df_mean, df_sem])
summary_psi.name = 'psi'

# tree
df_mean = results_tree[cols].mean()
df_sem = results_tree[cols].sem()
df_mean.columns = mean_cols
df_sem.columns = sem_cols
summary_tree = pd.concat([df_mean, df_sem])
summary_tree.name = 'tree'

summary = pd.concat([summary_pruning,
                     summary_lime,
                     summary_weights,
                     summary_psi,
                     summary_tree], axis=1).T
summary.columns = mean_cols + sem_cols
summary
```

```python
summary.to_csv(os.path.join(results_dir, 'summary.csv'))
```


**Para execução deste tutorial é necessária a instalação de algumas dependências:**

1. [Savime](https://github.com/hllustosa/Savime/). Execute o script `examples/test_savime.sh` para verificar se a instalação foi bem sucedida.
2. Algumas bibliotecas em Python dispostas em `tutorial.yml`. A maneira mais fácil de as instalar é obter o gerenciador [Anaconda](https://www.anaconda.com/) e criar um ambiente com o seguinte comando:
    - `conda env create -f tutorial.yml`
    - Para ativar o ambiente basta executar `conda activate tutorial`
3. [pysavime](https://github.com/dnasc/PySavime): Utilize o ambiente criado anteriormente durante o processo de compilação; não será necessário efetuar a instalação dos pacotes sugeridos no repositório PySavime.
4. [TFX](https://www.tensorflow.org/tfx/serving/setup): No Ubuntu você deve executar `sudo apt install tensorflow-model-server`. Caso esteja utilizando outro sistema operacional siga o [tutorial](https://www.tensorflow.org/tfx/serving/setup).

# Parte 01

Nesta parte serão gerados múltiplos modelos (adaptação da [Elastic Net](https://en.wikipedia.org/wiki/Elastic_net_regularization)). Esses serão utilizados posteriormente
para se testar o uso conjunto de SAVIME e TFX. Para efetuar essa construção, os seguintes passos serão executados:

1. Definir a função alvo de aprendizado (onde $d$ é a quantidade de *features*):
$$\begin{align}y &= f(x)\\f: X &\to Y, X \subset \mathbb{R}^d, Y \subset \mathbb{R}\end{align}$$
2. Gerar uma amostra uniforme (dataset) $D_X \subset X$.
3. Produzir $D_Y$ a partir de $D_X$ e $D = concat(D_X, D_Y)$.
3. Treinar $m$ modelos (regressores) $m_i: X \to Y$. Cada modelo $m_i$ deve ser
treinado (treino / teste/ validação) em uma partição de $D^{(i)} = (D_X^{(i)}, D_Y^{(i)}) \subset D$ tal que:
$$D = \bigcup_{i=1}^{m} D^{(i)} \;,\; \bigcap_{i=1}^{m} D^{(i)} = \emptyset \; \land \;  i < j \implies \forall x_i \in D_X^{(i)}, x_j \in D_X^{(j)} \left(x_i < x_j \right)$$
4. Utilizar uma métrica $\phi$ e avaliar cada modelo $m_i$ em cada partição $D^{(j)}$.
No caso desse exemplo $\phi$ é o erro  quadrático médio entre a predição e o valor
verdadeiro.
5. Salvar os modelos em um formato adequado para execução posterior no tensorflow,
assim como $D$ em disco a fim de ser registrado no SAVIME.

## Definição de $f$:

A construção de $f$ depende do estabecimento de alguns parâmetros:

- $X = [\rm{low}, \rm{high}]$;
- $D_X = (x_1, ..., x_n), x_i \sim U(X)$, onde $n$ é o número de observações.
- $D_Y = (f(x),  x \in D_X)$.
- $D = (D_X, D_Y)$ é particionado em $m$ particões $D^{(i)}$ cada qual associada a um modelo $m_i$.
- $A = \rm{linspace}(\alpha_{min}=1, \alpha_{max}=3, \rm{num\_partitions} = m)$ é um vetor ordenado e igualmente
espaçado de números entre 1 e 3 e de tamanho igual a $m$. Cada $d_i$ é associado a um $\alpha_i \in A$.
- $g(x)$ indica qual o $\alpha$ deve ser associado a $x$. Por exemplo, se $X = [-1, 1]$,
e $m = 2$ então $A = [1, 3]$, $g(x=-.5) = 1$ e $g(x=.5) = 3$.
- $f(x) = g(x_1)x_1 + ... + g(x_d)x_d + \varepsilon$, onde
$\varepsilon \sim \mathcal{N}(\mu, \sigma)$ (e.g., $\mu = 0$ e $\sigma=1$).
- *Note que $f$ é uma coletânea de funções lineares cada qual associada a uma partição de $X$.

```python
%load_ext autoreload
%autoreload 2

import inspect
import os
import random as rn

if not 'notebooks' in os.listdir('.'):
    current_dir = os.path.abspath(os.getcwd())
    parent_dir = os.path.dirname(current_dir)
    os.chdir(parent_dir)

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
import tensorflow as tf

from src.functions import f2_x as f
from src.model import ElasticNet, ModelFactory
from src.util import plot_heatmap

# Reprodutibilidade
import numpy as np
import random as rn
import tensorflow as tf

seed = 32
rn.seed(seed)
np.random.seed(seed)
tf.random.set_seed(seed)

sns.set_context('notebook')
sns.set_style('whitegrid')
sns.set_palette(sns.color_palette("Paired"))

tf.get_logger().setLevel('ERROR')
```

A critério de exemplo, a seguir é demonstrado o comportamento de $f$. Note que (i) o domínio da função é $[-1, 1]$, (ii) a quantidade de partições é 5, e (iii) os cinco alphas (coeficientes) associados às partições são [-1, -0.5, 0, 0.5,1]. Perceba ainda que a função tem uma componente estocástica ($\varepsilon \in \mathcal{N}\left(0, \frac{1}{10}\right)$):

```python
low_plot, high_plot, num_partitions_plot = -1, 1, 5
x_plot = np.random.uniform(low_plot, high_plot, 10000).reshape(-1, 1)
y_plot = f(x_plot, low_plot, high_plot, num_partitions_plot, 0, .01,
              alpha_min=-1, alpha_max=1)
sns.regplot(x_plot.ravel(), y_plot.ravel(), fit_reg=False)
```

Já para o treinamento dos modelos, utilizaremos as seguintes configurações:
- **Quantidade de features** = 2
- **Quantidade de modelos/partições** = 25
- **Observações**:
 - Quantidade = 100000
 - Intervalo = [-1, 1]
- **Ajuste de modelo**:
  - Percentual de treino, validação e teste respectivamente igual a .5, .3 e .2.
- **Função**:
  - Alphas variando entre 1 e 3.
  - Erro $\varepsilon \in \mathcal{N}(0, 0.01)$

```python
# número de features, observações e modelos
d = 2
n = 100000
m = 25

# Input shape do modelo
input_shape = (d, )

# intervalo de X
low = -1
high = 1

# amostra uniforme entre low e high de tamanho n*d
x = np.random.uniform(low, high, n * d)

# Ordenação de x a fim de garantir o item .4
x = np.reshape(np.sort(x), newshape=(n, d))

# Configurações para o treinamento dos modelos
val_size = .3
test_size = .2

# Prefixo de nome associado a cada modelo
model_name_prefix = 'my_elastic_net'

# Onde os modelos e dados serão salvos
output_dir = os.path.join(parent_dir, 'saved_models_elastic_net')

# Configuração da função
f_alpha_min = 1
f_alpha_max = 3
f_mean = 0
f_std = 10e-2
```

**Na célula abaixo é instanciada uma fábrica de modelos. A fábrica ('model_factory') ficará responsável por:**
1. Criar o vetor y com base na função f descrita anteriormente;
2. Dividir x em m partições;
3. Treinar cada modelo na sua devida partição;
4. Treinar um modelo em todo o domínio;
5. Salvar os modelos;
6. Salvar os dados.

```python
f_config = dict(low=low, high=high, num_partitions=m, mean=f_mean, std=f_std,
                alpha_min=f_alpha_min, alpha_max=f_alpha_max)

model_config = dict(input_shape=input_shape, loss=tf.keras.losses.mean_squared_error,
                    metrics=[tf.keras.metrics.mean_squared_error],epochs=10)

model_factory = ModelFactory(model_class=ElasticNet, x=x, f_x=f, num_models=m,
                             test_size=test_size, val_size=val_size,
                             model_name_prefix=model_name_prefix,
                             f_kwargs=f_config, model_kwargs=model_config)

model_factory.build_models()
model_factory.fit_models()
```

    Building models: 100%|██████████| 25/25 [00:01<00:00, 22.62it/s]
    Training models by partition: 25it [00:41,  1.66s/it]
    Training a model in the whole domain: 100%|██████████| 1/1 [00:34<00:00, 34.51s/it]

Abaixo é computado o erro quadrático médio de cada modelo em cada partição e no domínio completo.

```python
# Salvar modelos e dados
info = model_factory.get_metric_info()

model_factory.save_models(output_dir)
model_factory.save_data(output_dir, info)
```

    Building metric matrices: 26it [01:40,  3.88s/it]

```python
fig, ax = plt.subplots()
fig.set_size_inches(15, 9)

matrix = info['mean_squared_error']

plot_heatmap(matrix=matrix, title=f'Heatmap for MSE', x_label='Model ID', y_label='Split ID', ax=ax,
             heatmap_kwargs=dict(linewidths=.1, cmap='Blues', annot=True, fmt='.2f'))
fig.tight_layout()
```

Obs.: A coluna mais à direita e linha mais abaixo dizem respeito respectivamente ao conjunto de teste e modelo treinado no domínio completo.

Note no plot acima que quanto mais distante da partição em que o modelo foi treinado, maior o erro que o modelo obtém na predição. Por exemplo, o modelo 24 obtém MSE de mais de 50 para partição 1 enquanto menor que 1 para partição 24.


<p align="center">

</p>

## Data Analytics

### Monte Carlo Simulation in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

Monte Carlo Simulation in Python

Here's a simple workflow, demonstration of Monte Carlo simulation for subsurface uncertainty modeling workflows. This should help you get started with building subsurface models that integrate uncertainty sources.

#### Monte Carlo Simulation

Definition: random sampling from a distribution

Procedure:

1. Model the representative distribution (CDF)
2. Draw a random value from a uniform [0,1] distribution (p-value)
3. Apply the inverse of the CDF to calculate the associated realization

In practice, Monte Carlo simulation refers to the workflow with multiple realizations drawn to buld an uncertainty model.

\begin{equation}
X^\ell = F_x(p^\ell),  \, \forall \, \ell = 1,\ldots, L
\end{equation}

where $X^\ell$ is the realization of the variable $X$ drawn from its CDF, $F_x$, with cumulative probability, p-value, $p^\ell$.

#### Monter Carlo Simulation for Uncertainty Modeling

It would be trivial to apply Monte Carlo simulation to a single random variable, after many realizations one would get back the original distribution. The general approach is to:

1. Model all distributions for the input, variables of interest $F_{x_1},\ldots,F_{x_m}$.
2. For each realization draw $p^\ell_{1},\ldots,p^\ell_{m}$, p-values
3. Apply the inverse of each distribution to calculate a realization of each variable, $X^\ell_j = F_{x^\ell_j}^{-1}(p^\ell_j),  \, \forall \, j = 1,\ldots$, $m$ variables.
4. Apply each set of variables for a $\ell$ realization to the transfer function to calculate the ouptput realization, $Y^\ell = F(X_1^\ell,\ldots,X_m^\ell)$.

Monte Carlo Simulation (MCS) is extremely powerful

* Possible to easily simulate uncertainty models for complicated systems
* Simulations are conducted by drawing values at random from specified uncertainty distributions for each variable
* A single realization of each variable, $X_1^\ell, X_2^\ell,\ldots,X_m^\ell$ is applied to the transfer function to calculate the realization of the variable of interest (output, decision criteria):

\begin{equation}
Y^\ell = f(X_1^\ell,\ldots,X_m^\ell), \, \forall \, \ell = 1,\ldots, L
\end{equation}

* The MCS method builds empirical uncertainty models by random sampling

How many realizations, $L$?

The answer is enough! If the MCS computational cost is low then **many** is the right answer. If too few realizations are calculated then the summary statistics and the entire CDF of the output, decision criteria may be incorrect. This is caused by fluctuations due to not enough samples (see the 'Law of Small Numbers').

The MCS method is very powerful. You can simulate output distributions that could not be calculated analytically.

#### Limitations

The MCS method above assumes:
1. **representativity** - the distribution is representative
2. **independence** - the variables are independent of eachother
3. **stationarity** - all realizations for each variable are from the same distribution

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

There are examples below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

```python
import geostatspy.GSLIB as GSLIB          # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats    # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                        # ndarrys for gridded data
import pandas as pd                       # DataFrames for tabular data
import os                                 # set working directory, run executables
import matplotlib.pyplot as plt           # for plotting
from scipy import stats                   # summary statistics
import math                               # trig etc.
import random
from ipywidgets import interactive        # widgets and interactivity
from ipywidgets import widgets
from ipywidgets import Layout
from ipywidgets import Label
from ipywidgets import VBox, HBox
```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
# interactive calculation of the random sample set (control of source parametric distribution and number of samples)
l = widgets.Text(value='                                      Monte Carlo Simulation Demonstration, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))

operator = widgets.RadioButtons(options=['add', 'mult'],description='Operator:',disabled=False,layout=Layout(width='230px', height='50px'))

L = widgets.IntSlider(min=1, max = 10000, value = 50, description = '$L$:',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
L.style.handle_color = 'gray'

uiL = widgets.VBox([L,operator])

dist1 = widgets.Dropdown(
    options=['Uniform','Triangular','Gaussian'],
    value='Gaussian',
    description='$X_1$:',
    disabled=False,
    layout=Layout(width='200px', height='30px')
)
min1 = widgets.FloatSlider(min=0.0, max = 100.0, value = 10.0, description = 'Min',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
min1.style.handle_color = 'blue'
max1 = widgets.FloatSlider(min=0.0, max = 100.0, value = 30.0, description = 'Max',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
max1.style.handle_color = 'blue'
ui1 = widgets.VBox([dist1,min1,max1],kwargs = {'justify_content':'center'})

dist2 = widgets.Dropdown(
    options=['Triangular', 'Uniform', 'Gaussian'],
    value='Gaussian',
    description='$X_2$:',
    disabled=False,
    layout=Layout(width='200px', height='30px')
)
min2 = widgets.FloatSlider(min=0.0, max = 100.0, value = 10.0, description = 'Min',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
min2.style.handle_color = 'red'
max2 = widgets.FloatSlider(min=0.0, max = 100.0, value = 30.0, description = 'Max',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
max2.style.handle_color = 'red'
ui2 = widgets.VBox([dist2,min2,max2],kwargs = {'justify_content':'center'})

dist3 = widgets.Dropdown(
    options=['Triangular', 'Uniform', 'Gaussian'],
    value='Gaussian',
    description='$X_3$:',
    disabled=False,
    layout=Layout(width='200px', height='30px')
)
min3 = widgets.FloatSlider(min=0.0, max = 100.0, value = 10.0, description = 'Min',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
min3.style.handle_color = 'yellow'
max3 = widgets.FloatSlider(min=0.0, max = 100.0, value = 30.0, description = 'Max',orientation='horizontal',layout=Layout(width='230px', height='50px'),continuous_update=False)
max3.style.handle_color = 'yellow'
ui3 = widgets.VBox([dist3,min3,max3],kwargs = {'justify_content':'center'})

ui = widgets.HBox([uiL,ui1,ui2,ui3])
ui2 = widgets.VBox([l,ui],)

def make_dist(dist,zmin,zmax,L):
    if dist == 'Triangular':
        z = np.random.triangular(left=zmin, mode=(zmax+zmin)*0.5, right=zmax, size=L)
        pdf = stats.triang.pdf(np.linspace(0.0,100.0,1000), loc = zmin, c = 0.5, scale = zmax-zmin)* 2 * L
    if dist == 'Uniform':
        z = np.random.uniform(low=zmin, high=zmax, size=L)
        pdf = stats.uniform.pdf(np.linspace(0.0,100.0,1000), loc = zmin, scale = zmax-zmin) * 2 * L
    if dist == 'Gaussian':
        mean = (zmax + zmin)*0.5; sd = (zmax - zmin)/6.0
        z = np.random.normal(loc = mean, scale = sd, size=L)
        pdf = stats.norm.pdf(np.linspace(0.0,100.0,1000), loc = mean, scale = sd) * 2 * L
    return z, pdf

def f_make(L,operator,dist1,min1,max1,dist2,min2,max2,dist3,min3,max3):
    np.random.seed(seed = 73073)
    x1, pdf1 = make_dist(dist1,min1,max1,L)
    x2, pdf2 = make_dist(dist2,min2,max2,L)
    x3, pdf3 = make_dist(dist3,min3,max3,L)

    xvals = np.linspace(0.0,100.0,1000)
    plt.subplot(241)
    plt.hist(x1,density = False,bins=np.linspace(0,100,50),weights=None,color='blue',alpha=0.3,edgecolor='black')
    plt.plot(xvals,pdf1,'--',color='black')
    plt.xlim(0,100); plt.xlabel("$X_1$"); plt.title("First Feature, $X_1$"); plt.ylabel('Frequency')

    plt.subplot(242)
    plt.hist(x2,density = False,bins=np.linspace(0,100,50),weights=None,color='red',alpha=0.3,edgecolor='black')
    plt.plot(xvals,pdf2,'--',color='black')
    plt.xlim(0,100); plt.xlabel("$X_1$"); plt.title("Second Feature, $X_2$"); plt.ylabel('Frequency')

    plt.subplot(243)
    plt.hist(x3,density = False,bins=np.linspace(0,100,50),weights=None,color='yellow',alpha=0.3,edgecolor='black')
    plt.plot(xvals,pdf3,'--',color='black')
    plt.xlim(0,100); plt.xlabel("$X_1$"); plt.title("Third Feature, $X_3$"); plt.ylabel('Frequency')

    y = np.zeros(L)
    ymin = 0.0
    if operator == "add":
        y = x1 + x2 + x3
    elif operator == "mult":
        y = x1 * x2 * x3

    ymax = max(round((np.max(y)+50)/100)*100,100) # round up to nearest hundreds to avoid the chart jumping around

    plt.subplot(244)
    plt.hist(y,density = False,bins=np.linspace(ymin,ymax,50),weights=None,color='black',alpha=0.5,edgecolor='black')
    plt.xlabel("$Y$"); plt.title("Response Feature, $Y$"); plt.ylabel('Frequency')
    plt.xlim(ymin,ymax)

    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.3, hspace=0.2)
    plt.show()

interactive_plot = widgets.interactive_output(f_make, {'L':L,'operator':operator,'dist1':dist1,'min1':min1,'max1':max1,'dist2':dist2,'min2':min2,'max2':max2,'dist3':dist3,'min3':min3,'max3':max3})
interactive_plot.clear_output(wait = True)                # reduce flickering by delaying plot updating

```

### Monte Carlo Simulation Demonstration

* specify the distributions for 3 Random Variables, $X_1$, $X_2$, and $X_3$ and select the operator $y = f(X_1,X_2,X_3)$

* observe the distribution of the resulting Monte Carlos Simulation realization histograms of $X_1^{\ell}$, $X_2^{\ell}$, $X_3^{\ell}$, and $y^{\ell}$

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

### The Inputs

* **$L$**: number of realizations, **Operator**: addition for $y = X_1 + X_2 + X_3$, multiplication for $y = X_1 \times X_2 \times X_3$

* **$X_1$, $X_2$, and $X_3$**: distribution type, min and max. Assume mode or mean is centered and 3 st.dev. for Gaussian

```python
display(ui2, interactive_plot)                            # display the interactive plot
```

    VBox(children=(Text(value='                                      Monte Carlo Simulation Demonstration, Michael…

    Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '<Figure size 432x288 with 4 Axes>', 'i…

#### Comments

This was a basic demonstration of Monte Carlo simulation for uncertainty analysis. A lot more could be done, for example, more complicated transfer functions and a combination of non-parametric and parametric distributions. Also, one could integrate relationships between the variables (we assumed independent here).

I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling, multivariate analysis and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```