<p align="center">

</p>

## Uncertaint Model Checking Demonstration

### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### The Interactive Workflow

Here's a simple workflow for checking the uncertainty model from simple kriging estimates and the estimation variance

* we assume a Gaussian local uncertainty model

#### Spatial Estimation

Consider the case of making an estimate at some unsampled location, $𝑧(\bf{u}_0)$, where $z$ is the property of interest (e.g. porosity etc.) and $𝐮_0$ is a location vector describing the unsampled location.

How would you do this given data, $𝑧(\bf{𝐮}_1)$, $𝑧(\bf{𝐮}_2)$, and $𝑧(\bf{𝐮}_3)$?

It would be natural to use a set of linear weights to formulate the estimator given the available data.

\begin{equation}
z^{*}(\bf{u}) = \sum^{n}_{\alpha = 1} \lambda_{\alpha} z(\bf{u}_{\alpha})
\end{equation}

We could add an unbiasedness constraint to impose the sum of the weights equal to one.  What we will do is assign the remainder of the weight (one minus the sum of weights) to the global average; therefore, if we have no informative data we will estimate with the global average of the property of interest.

\begin{equation}
z^{*}(\bf{u}) = \sum^{n}_{\alpha = 1} \lambda_{\alpha} z(\bf{u}_{\alpha}) + \left(1-\sum^{n}_{\alpha = 1} \lambda_{\alpha} \right) \overline{z}
\end{equation}

We will make a stationarity assumption, so let's assume that we are working with residuals, $y$.

\begin{equation}
y^{*}(\bf{u}) = z^{*}(\bf{u}) - \overline{z}(\bf{u})
\end{equation}

If we substitute this form into our estimator the estimator simplifies, since the mean of the residual is zero.

\begin{equation}
y^{*}(\bf{u}) = \sum^{n}_{\alpha = 1} \lambda_{\alpha} y(\bf{u}_{\alpha})
\end{equation}

while satisfying the unbaisedness constraint.

#### Kriging

Now the next question is what weights should we use?

We could use equal weighting, $\lambda = \frac{1}{n}$, and the estimator would be the average of the local data applied for the spatial estimate. This would not be very informative.

We could assign weights considering the spatial context of the data and the estimate:

* **spatial continuity** as quantified by the variogram (and covariance function)
* **redundancy** the degree of spatial continuity between all of the available data with themselves
* **closeness** the degree of spatial continuity between the avaiable data and the estimation location

The kriging approach accomplishes this, calculating the best linear unbiased weights for the local data to estimate at the unknown location.  The derivation of the kriging system and the resulting linear set of equations is available in the lecture notes.  Furthermore kriging provides a measure of the accuracy of the estimate!  This is the kriging estimation variance (sometimes just called the kriging variance).

\begin{equation}
\sigma^{2}_{E}(\bf{u}) = C(0) - \sum^{n}_{\alpha = 1} \lambda_{\alpha} C(\bf{u}_0 - \bf{u}_{\alpha})
\end{equation}

What is 'best' about this estimate? Kriging estimates are best in that they minimize the above estimation variance.

#### Properties of Kriging

Here are some important properties of kriging:

* **Exact interpolator** - kriging estimates with the data values at the data locations
* **Kriging variance** can be calculated before getting the sample information, as the kriging estimation variance is not dependent on the values of the data nor the kriging estimate, i.e. the kriging estimator is homoscedastic.
* **Spatial context** - kriging takes into account, furthermore to the statements on spatial continuity, closeness and redundancy we can state that kriging accounts for the configuration of the data and structural continuity of the variable being estimated.
* **Scale** - kriging may be generalized to account for the support volume of the data and estimate. We will cover this later.
* **Multivariate** - kriging may be generalized to account for multiple secondary data in the spatial estimate with the cokriging system. We will cover this later.
* **Smoothing effect** of kriging can be forecast. We will use this to build stochastic simulations later.

#### Spatial Continuity

**Spatial Continuity** is the correlation between values over distance.

* No spatial continuity – no correlation between values over distance, random values at each location in space regardless of separation distance.

* Homogenous phenomenon have perfect spatial continuity, since all values as the same (or very similar) they are correlated.

We need a statistic to quantify spatial continuity! A convenient method is the Semivariogram.

#### The Semivariogram

Function of difference over distance.

* The expected (average) squared difference between values separated by a lag distance vector (distance and direction), $h$:

\begin{equation}
\gamma(\bf{h}) = \frac{1}{2 N(\bf{h})} \sum^{N(\bf{h})}_{\alpha=1} (z(\bf{u}_\alpha) - z(\bf{u}_\alpha + \bf{h}))^2
\end{equation}

where $z(\bf{u}_\alpha)$ and $z(\bf{u}_\alpha + \bf{h})$ are the spatial sample values at tail and head locations of the lag vector respectively.

* Calculated over a suite of lag distances to obtain a continuous function.

* the $\frac{1}{2}$ term converts a variogram into a semivariogram, but in practice the term variogram is used instead of semivariogram.
* We prefer the semivariogram because it relates directly to the covariance function, $C_x(\bf{h})$ and univariate variance, $\sigma^2_x$:

\begin{equation}
C_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

Note the correlogram is related to the covariance function as:

\begin{equation}
\rho_x(\bf{h}) = \frac{C_x(\bf{h})}{\sigma^2_x}
\end{equation}

The correlogram provides of function of the $\bf{h}-\bf{h}$ scatter plot correlation vs. lag offset $\bf{h}$.

\begin{equation}
-1.0 \le \rho_x(\bf{h}) \le 1.0
\end{equation}

#### Accuracy Plots

The accuracy plot was developed by Deutsch (1996) and described in Pyrcz and Deutsch (2014).

* a method for checking uncertainty models

* based on calculating the percentiles of withheld testing data in the estimated local uncertainty distributions, $Z(\bf{u}_{\alpha})$, describes by CDFs, $F_Z(z,\bf{u}_{\alpha})$, at all testing locations $\alpha = 1,\ldots,n_{test}$.

The accuracy plot is the proportion of data within symmetric probability intervals vs. the probability interval, $p$.

* for example, 20% of withheld testing data should fall between the P40 and P60 probability interval

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data.csv at https://git.io/fh4gm.

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import geostatspy.GSLIB as GSLIB                       # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                 # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
%matplotlib inline
import os                                               # to set current working directory
from sklearn.model_selection import train_test_split    # train and test data split by random selection of a proportion
from scipy.stats import norm                            # Gaussian distribution assumed for local uncertainty
import sys                                              # supress output to screen for interactive variogram modeling
import io                                               # set the working directory
import numpy as np                                      # arrays and matrix math
import pandas as pd                                     # DataFrames
import matplotlib.pyplot as plt                         # plotting
from matplotlib.pyplot import cm                        # color maps
from matplotlib.patches import Ellipse                  # plot an ellipse
import math                                             # sqrt operator
from ipywidgets import interactive                      # widgets and interactivity
from ipywidgets import widgets
from ipywidgets import Layout
from ipywidgets import Label
from ipywidgets import VBox, HBox
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Simple, Simple Kriging Function

Let's write a fast Python function to take data points and unknown location and provide the:

* **simple kriging estimate**

* **simple kriging variance / estimation variance**

* **simple kriging weights**

This provides a fast method for small datasets, with less parameters (no search parameters) and the ability to see the simple kriging weights

```python
def simple_simple_krige(df,xcol,ycol,vcol,dfl,xlcol,ylcol,vario,skmean):
# load the variogram
    nst = vario['nst']; pmx = 9999.9
    cc = np.zeros(nst); aa = np.zeros(nst); it = np.zeros(nst)
    ang = np.zeros(nst); anis = np.zeros(nst)
    nug = vario['nug']; sill = nug
    cc[0] = vario['cc1']; sill = sill + cc[0]
    it[0] = vario['it1']; ang[0] = vario['azi1'];
    aa[0] = vario['hmaj1']; anis[0] = vario['hmin1']/vario['hmaj1'];
    if nst == 2:
        cc[1] = vario['cc2']; sill = sill + cc[1]
        it[1] = vario['it2']; ang[1] = vario['azi2'];
        aa[1] = vario['hmaj2']; anis[1] = vario['hmin2']/vario['hmaj2'];

# set up the required matrices
    rotmat, maxcov = geostats.setup_rotmat(nug,nst,it,cc,ang,pmx)
    ndata = len(df); a = np.zeros([ndata,ndata]); r = np.zeros(ndata); s = np.zeros(ndata); rr = np.zeros(ndata)
    nest = len(dfl)

    est = np.zeros(nest); var = np.full(nest,sill); weights = np.zeros([nest,ndata])

# Make and solve the kriging matrix, calculate the kriging estimate and variance
    for iest in range(0,nest):
        for idata in range(0,ndata):
            for jdata in range(0,ndata):
                a[idata,jdata] = geostats.cova2(df[xcol].values[idata],df[ycol].values[idata],df[xcol].values[jdata],df[ycol].values[jdata],
                                        nst,nug,pmx,cc,aa,it,ang,anis,rotmat,maxcov)
            r[idata] = geostats.cova2(df[xcol].values[idata],df[ycol].values[idata],dfl[xlcol].values[iest],dfl[ylcol].values[iest],
                                        nst,nug,pmx,cc,aa,it,ang,anis,rotmat,maxcov)
            rr[idata] = r[idata]

        s = geostats.ksol_numpy(ndata,a,r)
        sumw = 0.0
        for idata in range(0,ndata):
            sumw = sumw + s[idata]
            weights[iest,idata] = s[idata]
            est[iest] = est[iest] + s[idata]*df[vcol].values[idata]
            var[iest] = var[iest] - s[idata]*rr[idata]
        est[iest] = est[iest] + (1.0-sumw)*skmean
    return est,var,weights
```

#### Set Global Parameters

These impact the look and results of this demonstration.

```python
seed = 73073                                            # random number seed for train and test split and added error term
cmap = plt.cm.plasma                                    # color map
vmin = 0.0; vmax = 0.20                                 # feature min and max for plotting
error_std = 0.0                                         # error standard deviation
bins = 20                                               # number of bins for the accuracy plots
```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")                                   # set the working directory
```

#### Load and Visualize the Spatial Data

Here's the code to load our comma delimited data file in to a Pandas' DataFrame object and to visualize it.

```python
df = pd.read_csv("sample_data_biased.csv")              # read a .csv file in as a DataFrame
df['Porosity'] = df['Porosity']+norm.rvs(0.0,error_std,random_state = seed,size=len(df))

plt.subplot(111)
im = plt.scatter(df['X'],df['Y'],c=df['Porosity'],marker='o',cmap=cmap,vmin=vmin,vmax=vmax,alpha=0.8,linewidths=0.8,
        verts=None,edgecolors="black",label="train")
plt.title("Subset of the Data for the Demonstration")
plt.xlim([0,1000]); plt.ylim([0,1000])
plt.xlabel('X(m)'); plt.ylabel('Y(m)'); plt.legend()
cbar = plt.colorbar(im, orientation="vertical", ticks=np.linspace(vmin, vmax, 10),format='%.2f')
cbar.set_label('Porosity (fraction)', rotation=270, labelpad=20)

plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

#### Interactive Uncertainty Checking with Simple Kriging

The following code includes:
* dashboard with variogram model, number of data and the proportion of testing data
* plots of variogram model, train and test data locations, accuracy plot and training data with testing data percentiles

```python
import warnings; warnings.simplefilter('ignore')

# build the dashboard
style = {'description_width': 'initial'}
l = widgets.Text(value='                                              Simple Kriging, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))
nug = widgets.FloatSlider(min = 0, max = 1.0, value = 0.0, step = 0.1, description = 'nug',orientation='vertical',continuous_update=False,
                          layout=Layout(width='50px', height='200px'))
nug.style.handle_color = 'gray'
it1 = widgets.Dropdown(options=['Spherical', 'Exponential', 'Gaussian'],value='Spherical',continuous_update=False,
    description='Type1:',disabled=False,layout=Layout(width='180px', height='30px'), style=style)

azi = widgets.FloatSlider(min=0, max = 360, value = 0, step = 22.5, description = 'azi',continuous_update=False,
                        orientation='vertical',layout=Layout(width='80px', height='200px'))
azi.style.handle_color = 'gray'
hmaj1 = widgets.FloatSlider(min=0.01, max = 10000.0, value = 100.0, step = 25.0, description = 'hmaj1',continuous_update=False,
                        orientation='vertical',layout=Layout(width='80px', height='200px'))
hmaj1.style.handle_color = 'gray'
hmin1 = widgets.FloatSlider(min = 0.01, max = 10000.0, value = 100.0, step = 25.0, description = 'hmin1',continuous_update=False,
                        orientation='vertical',layout=Layout(width='80px', height='200px'))
hmin1.style.handle_color = 'gray'

ptest = widgets.FloatSlider(min = 0.01, max = 0.9, value = 100.0, step = 0.1, description = 'prop test',continuous_update=False,
                        orientation='vertical',layout=Layout(width='80px', height='200px'))
ptest.style.handle_color = 'gray'

ndata = widgets.IntSlider(min = 1, max = len(df), value = 100, step = 10, description = 'number data',continuous_update=False,
                        orientation='vertical',layout=Layout(width='80px', height='200px'))
ndata.style.handle_color = 'gray'

uikvar = widgets.HBox([nug,it1,azi,hmaj1,hmin1,ptest,ndata],)

uipars = widgets.HBox([uikvar],)
uik = widgets.VBox([l,uipars],)

# convenience function ot convert variogram model type to a integer
def convert_type(it):
    if it == 'Spherical':
        return 1
    elif it == 'Exponential':
        return 2
    else:
        return 3

# calculate the kriging-based uncertainty distributions and match truth values to percentiles and product plots
def f_make_krige(nug,it1,azi,hmaj1,hmin1,ptest,ndata):
    text_trap = io.StringIO()
    sys.stdout = text_trap
    it1 = convert_type(it1)

    train, test = train_test_split(df.iloc[len(df)-ndata:,[0,1,3,]], test_size=ptest, random_state=73073)

    nst = 1; xlag = 10; nlag = int(hmaj1/xlag); c1 = 1.0-nug
    vario = GSLIB.make_variogram(nug,nst,it1,c1,azi,hmaj1,hmin1) # make model object
    index_maj,h_maj,gam_maj,cov_maj,ro_maj = geostats.vmodel(nlag,xlag,azi,vario) # project the model in the major azimuth                                                  # project the model in the 135 azimuth
    index_min,h_min,gam_min,cov_min,ro_min = geostats.vmodel(nlag,xlag,azi+90.0,vario) # project the model in the minor azimuth
    skmean = np.average(train['Porosity'])              # calculate the input mean and sill for simple kriging
    sill = np.var(train['Porosity'])

    sk_est, sk_var, sk_weights =  simple_simple_krige(train,'X','Y','Porosity',test,'X','Y',vario,skmean=skmean) # data, esitmation locations
    sk_std = np.sqrt(sk_var*sill)                       # standardize estimation variance by the sill and convert to std. dev.

    percentiles = norm.cdf(test['Porosity'],sk_est,sk_std) # calculate the percentiles of truth in the uncertainty models
    test["Percentile"] = percentiles

    xlag = 10.0; nlag = int(hmaj1/xlag)                 # lags for variogram plotting

    plt.subplot(221)                                    # plot variograms
    plt.plot([0,hmaj1*1.5],[1.0,1.0],color = 'black')
    plt.plot(h_maj,gam_maj,color = 'black',label = 'Major ' + str(azi))
    plt.plot(h_min,gam_min,color = 'black',label = 'Minor ' + str(azi+90.0))
    deltas = [22.5, 45, 67.5];
    ndelta = len(deltas); hd = np.zeros(ndelta); gamd = np.zeros(ndelta);
    color=iter(cm.plasma(np.linspace(0,1,ndelta)))
    for delta in deltas:
        index,hd,gamd,cov,ro = geostats.vmodel(nlag,xlag,azi+delta,vario);
        c=next(color)
        plt.plot(hd,gamd,color = c,label = 'Azimuth ' + str(azi+delta))
    plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
    plt.ylabel(r'$\gamma \bf(h)$')
    plt.title('Interpolated NSCORE Porosity Variogram Models')
    plt.xlim([0,hmaj1*1.5])
    plt.ylim([0,1.4])
    plt.legend(loc='upper left')

    plt.subplot(222)                                    # plot the train and test data
    im = plt.scatter(train['X'],train['Y'],c=train['Porosity'],marker='o',cmap=cmap,vmin=vmin,vmax=vmax,alpha=0.8,linewidths=0.8,
        verts=None,edgecolors="black",label="train")
    plt.scatter(test['X'],test['Y'],c=test['Porosity'],marker='^',cmap=cmap,vmin=vmin,vmax=vmax,alpha=0.8,linewidths=0.8,
        verts=None,edgecolors="black",label="test")
    plt.title("Training and Testing Data")
    plt.xlim([0,1000]); plt.ylim([0,1000])
    plt.xlabel('X(m)'); plt.ylabel('Y(m)'); plt.legend()
    cbar = plt.colorbar(im, orientation="vertical", ticks=np.linspace(vmin, vmax, 10),format='%.2f')
    cbar.set_label('Porosity (fraction)', rotation=270, labelpad=20)

    fraction_in = np.zeros(bins)                        # calculate and plot the accuracy plot
    p_intervals = np.linspace(0.0,1.0,bins)
    for i,p in enumerate(p_intervals):
        test_result = (test['Percentile'] > 0.5-0.5*p) & (test['Percentile'] < 0.5+0.5*p)
        fraction_in[i] = test_result.sum()/len(test)

    plt.subplot(223)
    plt.scatter(p_intervals,fraction_in,c='black',marker='o',alpha=0.8)
    plt.plot([0.0,1.0],[0.0,1.0],c='black')
    plt.xlim([0.0,1.0]); plt.ylim([0,1.0])
    plt.title('Uncertainty Model at Unknown Location')
    plt.xlabel('Probability Interval'); plt.ylabel('Fraction In the Interval')

    plt.subplot(224)                                      # plot the testing percentiles with the training data
    plt.scatter(train['X'],train['Y'],s=20,c='black',marker='o',cmap=cmap,vmin=vmin,vmax=vmax,alpha=0.8,linewidths=0.8,
        verts=None,edgecolors="black",label="train")
    im = plt.scatter(test['X'],test['Y'],s=80.0,c=test['Percentile'],marker='^',cmap=cmap,vmin=0.0,vmax=1.0,alpha=0.8,linewidths=0.8,
        verts=None,edgecolors="black",label="test")
    plt.title("Cross Validation Percentiles")
    plt.xlim([0,1000]); plt.ylim([0,1000])
    plt.xlabel('X(m)'); plt.ylabel('Y(m)'); plt.legend()
    cbar = plt.colorbar(im, orientation="vertical", ticks=np.linspace(0.0, 1.0, 10),format='%.2f')
    cbar.set_label('Porosity (fraction)', rotation=270, labelpad=20)

    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.2, top=2.2, wspace=0.3, hspace=0.3)
    plt.show()

# connect the function to make the samples and plot to the widgets
interactive_plot = widgets.interactive_output(f_make_krige, {'nug':nug, 'it1':it1, 'azi':azi, 'hmaj1':hmaj1, 'hmin1':hmin1, 'ptest':ptest, 'ndata':ndata})
interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

### Interactive Uncertianty Checking Kriging Demostration

* select the variogram model for simple kriging and observe the impact on the uncertainty model

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

### The Inputs

Select the variogram model and the proportion of data withheld for testing.

* **nug**: nugget effect

* **c1**: contributions of the sill

* **hmaj1 / hmin1**: range in the major and minor direction

* **(x1, y1),...(x3,y3)**: spatial data locations

* **test proportion**: proportion of data withheld for testing

```python
display(uik, interactive_plot)                            # display the interactive plot
```

    VBox(children=(Text(value='                                              Simple Kriging, Michael Pyrcz, Associ…

    Output()

#### Comments

This was an interactive demonstration of simple kriging for spatial data analytics. Much more could be done, I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python
norm.cdf(0,0,1)
```

    0.5

```python

```

```python

```


# Cardelli, Clayton, & Mathis 1989 interstellar extinction curve

Figure 6.3 from Chapter 6 of *Interstellar and Intergalactic Medium* by Ryden & Pogge, 2021,
Cambridge University Press.

Plot the [Cardelli, Clayton, & Mathis 1989, ApJ, 345, 245](https://ui.adsabs.harvard.edu/abs/1989ApJ...345..245C/abstract) interstellar extinction curve, using the
updated optical/IR coefficients of [O'Donnell 1994, ApJ, 422, 158](https://ui.adsabs.harvard.edu/abs/1994ApJ...422..158O/abstract).

We use the parameterization of the extinction curve as a set of polynomials of order n.

```python
%matplotlib inline

import os
import sys
import math
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from matplotlib.ticker import MultipleLocator, LogLocator, NullFormatter

import warnings
warnings.filterwarnings('ignore',category=UserWarning, append=True)
```

## Standard Plot Format

Setup the standard plotting format and make the plot. Fonts and resolution adopted follow CUP style.

```python
figName = 'Fig6_3'

# graphic aspect ratio = width/height

aspect = 4.0/3.0 # 4:3

# Text width in inches - don't change, this is defined by the print layout

textWidth = 6.0 # inches

# output format and resolution

figFmt = 'png'
dpi = 600

# Graphic dimensions

plotWidth = dpi*textWidth
plotHeight = plotWidth/aspect
axisFontSize = 10
labelFontSize = 6
lwidth = 0.5
axisPad = 5
wInches = textWidth
hInches = wInches/aspect

# Plot filename

plotFile = f'{figName}.{figFmt}'

# LaTeX is used throughout for markup of symbols, Times-Roman serif font

plt.rc('text', usetex=True)
plt.rc('font', **{'family':'serif','serif':['Times-Roman'],'weight':'bold','size':'16'})

# Font and line weight defaults for axes

matplotlib.rc('axes',linewidth=lwidth)
matplotlib.rcParams.update({'font.size':axisFontSize})

# axis and label padding

plt.rcParams['xtick.major.pad'] = f'{axisPad}'
plt.rcParams['ytick.major.pad'] = f'{axisPad}'
plt.rcParams['axes.labelpad'] = f'{axisPad}'
```

## Extinction curve calculation

Compute the extinction curve for $R_V=3.1$ following CCM89 Equation 1:
\begin{equation}
   A_\lambda/A_V = a(x) + b(x)/R_V
\end{equation}
where $x=1/\lambda$, broken into three wavelength regions: IR, Optical/IR, and UV.

We adopt the O'Donnell polynomial coefficients $a(x)$ and $b(x)$ for the Optical/IR regime, otherwise use the
CCM89 parameterization.

```python
# Ratio of total to selective extinction

Rv = 3.1

# x has units of 1/um

# Infrared: x=0.3 to 1.1/um: CCM89 Eqn 2a and 2b

xIR = np.linspace(0.3,1.1,100)
aIR =  0.574*(xIR**1.61)
bIR = -0.527*(xIR**1.61)

AlamIR = aIR + bIR/Rv

# Optical/NIR: x=1.1 to 3.3/um and y=x-1.82 with the O'Donnel 1994 coefficients

xOIR = np.linspace(1.1,3.3,100)
y = xOIR-1.82

ca = [1.0,0.104,-0.609, 0.701, 1.137,-1.718,-0.827,  1.647,-0.505]
cb = [0.0,1.952, 2.908,-3.989,-7.985,11.102, 5.491,-10.805, 3.347]

aOIR = np.polynomial.polynomial.polyval(y,ca)
bOIR = np.polynomial.polynomial.polyval(y,cb)

AlamOIR = aOIR + bOIR/Rv

# Mid-UV

xMUV = np.linspace(3.3,8.0,100)

aMUV = np.zeros(100)
bMUV = np.zeros(100)
for i, x in enumerate(xMUV):
    if x < 5.9:
        Fa = 0.0
        Fb = 0.0
    else:
        Fa = -0.04473*(x-5.9)**2 - 0.009779*(x-5.9)**3
        Fb = 0.2130*(x-5.9)**2 + 0.1207*(x-5.9)**3
    aMUV[i] = 1.752 - 0.316*x -0.104/((x-4.67)**2 + 0.341) + Fa
    bMUV[i] = -3.090 + 1.825*x + 1.206/((x-4.62)**2 + 0.263) + Fb

AlamMUV = aMUV + bMUV/Rv

# Far-UV

xFUV = np.linspace(8.0,10.0,100)
y = xFUV - 8.0
ca = [-1.073,-0.628, 0.137,-0.070 ]
cb = [13.670, 4.257,-0.420, 0.374 ]

aFUV = np.polynomial.polynomial.polyval(y,ca)
bFUV = np.polynomial.polynomial.polyval(y,cb)

AlamFUV = aFUV + bFUV/Rv

```

## Plot the extinction curve

```python
xMin = 0.1
xMax = 1.0
yMin = 0.0
yMax = 5.5

fig,ax = plt.subplots()

fig.set_dpi(dpi)
fig.set_size_inches(wInches,hInches,forward=True)

ax.tick_params('both',length=6,width=lwidth,which='major',direction='in',top='on',right='on')
ax.tick_params('both',length=3,width=lwidth,which='minor',direction='in',top='on',right='on')

# Limits

plt.xlim(xMin,xMax)
ax.xaxis.set_major_locator(MultipleLocator(0.1))
ax.xaxis.set_minor_locator(MultipleLocator(0.05))
plt.xlabel(r'$\lambda$ [$\mu$m]',fontsize=axisFontSize)

plt.ylim(yMin,yMax)
ax.yaxis.set_major_locator(MultipleLocator(1))
ax.yaxis.set_minor_locator(MultipleLocator(0.5))
plt.ylabel(r'A$_\lambda$/A$_V$',fontsize=axisFontSize)

plt.plot(1./xIR,AlamIR,color='black',ls='-',lw=1.5,zorder=10)
plt.plot(1./xOIR,AlamOIR,color='black',ls='-',lw=1.5,zorder=10)
plt.plot(1./xMUV,AlamMUV,color='black',ls='-',lw=1.5,zorder=10)
plt.plot(1./xFUV,AlamFUV,color='black',ls='-',lw=1.5,zorder=10)

plt.plot([0.4,0.4],[0,5.5],'--',color='black',lw=0.5,zorder=8)
plt.plot([0.7,0.7],[0,5.5],'--',color='black',lw=0.5,zorder=8)
plt.plot([0.7,0.7],[0,5.5],'--',color='black',lw=0.5,zorder=8)

plt.text((0.1+0.4)/2.0,5,r'UV',fontsize=axisFontSize,ha='center',color='black')
plt.text((0.4+0.7)/2.0,5,r'Visible',fontsize=axisFontSize,ha='center',color='black')
plt.text((0.7+1.0)/2.0,5,r'Infrared',fontsize=axisFontSize,ha='center',color='black')

plt.plot()
plt.savefig(plotFile,bbox_inches='tight')
```


## Subsurface Data Analytics

### Support Vector Machine for Multivariate for Subsurface Modeling in Python

#### Wendi Liu, Graduate Student, The University of Texas at Austin

##### [LinkedIn](https://www.linkedin.com/in/wendi-liu-8a3023127/) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

Appreciation to Stephanie Forstner for a workflow bug fix (Nov. 22, 2019).

### PGE 383 Exercise: Support Vector Machine for Subsurface Modeling in Python

Here's a simple workflow, demonstration of subsurface vector machines for subsurface modeling workflows. This should help you get started with building subsurface models that data analytics and machine learning. Here's some basic details about support vector machines.

#### Support Vector Machines

Machine learning method for supervised learning for classification and regression analysis.  Here are some key aspects of support vector machines.

**Prediction**

* estimate a function $\hat{f}$ such that we predict a response feature $Y$ from a set of predictor features $X_1,\ldots,X_m$.

* the prediction is of the form $\hat{Y} = \hat{f}(X_1,\ldots,X_m)$

**Suppervised learning**

* the response feature label, $Y$, is available over the training and testing data

**Unsupervised learning**

* is avaiable with **suport vector clustering**

**Binary separator**

* given the data within a mD feature space, $X_1,\ldots,X_m$, find that boundary maximizes the gap between the two categories

* new cases are classified based on where they fall relative to this boundary

**Linear and Non-linear**

* **maximum margin classfier**, assignment of a m-1 dimensional hyperplane that maximizes the gap (margin) between the two categories

* **mapped to a higher dimensional space** based on a kernel function, e.g. polygonal and gaussian radial basis function shown below

##### Applications to subsurface modeling

Support the segmentation of facies categories from a combination of features (porosity, permeability, acoustic impedance etc.).

* we work with just 2 predictor features for the example below for easy visualization

* flexibility through selection of kernel (hyperparameter) and associated kernel and cost parameters

#### Workflow Goals

Learn the basics of support vector machine in python to segment facies given petrophysical properties. This includes:

* Loading and visualizing sample data
* Trying out support vector machine with different kernels (linear, polynomial, Gaussian radial basis function)
* Tuning the SVM model parameters and results evaluation

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - 12_sample_data.csv found [here](https://github.com/GeostatsGuy/GeoDataSets/blob/master/12_sample_data.csv).

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Import Required Packages

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import os                                                   # to set current working directory
import numpy as np                                          # arrays and matrix math
import pandas as pd                                         # DataFrames
import matplotlib.pyplot as plt                             # for plotting
import seaborn as sns                                       # for data visualization
from sklearn.svm import SVC                                 # support vector machine methods
from sklearn.model_selection import train_test_split        # training and testing datasets
from sklearn.metrics import confusion_matrix                # for sumarizing model performance
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.model_selection import GridSearchCV
scaler = StandardScaler()
```

#### Declare Functions

Here's a couple of functions that we will use in our workflow to support visualization of the model results.

```python
def visualize_model(model,nxfeature,nyfeature,xfeature,yfeature,response):# plots the data points and the decision tree prediction
    n_classes = 10
    cmap = plt.cm.RdYlBu
    plot_step = 500.0
    x_min, x_max = min(nxfeature), max(nxfeature)
    y_min, y_max = min(nyfeature) , max(nyfeature)
    resp_min = round(min(response)); resp_max = round(max(response));
    xx, yy = np.meshgrid(np.linspace(x_min, x_max, plot_step),
                     np.linspace(y_min, y_max, plot_step))

    z_min = round(min(response)); z_max = round(max(response))
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])

    Z = Z.reshape(xx.shape)
    trans = scaler.inverse_transform(np.c_[xx.ravel(), yy.ravel()])
    trans_x = trans[:,0].reshape(xx.shape)
    trans_y = trans[:,1].reshape(xx.shape)
    cs = plt.contourf(trans_x, trans_y, Z, cmap=cmap,vmin=z_min, vmax=z_max)

    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=cmap, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar = plt.colorbar(im, orientation = 'vertical')
    cbar.set_label(response.name, rotation=270, labelpad=20)
    return(plt)

def plot_svc_decision_function(model,plt, plot_support=True): # modified from Jake VanderPlas's Python Data Science Handbook
    """Plot the decision function for a 2D SVC"""
    xlim = [-2.5,2.5]
    ylim = [-2.5,2.5]
    # plot support vectors
    sv = scaler.inverse_transform(model.support_vectors_)
    if plot_support:
        plt.scatter(sv[:, 0],
                   sv[:, 1],
                   s=3, linewidth=8,alpha = 0.6, facecolors='black');
```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("/Users/stephanierae/Box Sync/Grad School/Machine Learning/DataSets")                     # set the working directory
```

#### Loading Data
Let's load the provided multivariate, spatial dataset '12_sample_data.csv'.  It is a comma delimited file with:

* X and Y coordinates ($m$)
* facies 0 and 1
* porosity (fraction)
* permeability ($mD$)
* acoustic impedance ($\frac{kg}{m^3} \cdot \frac{m}{s} \cdot 10^3$).

We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it to make sure it loaded correctly.

**Python Tip: using functions from a package** just type the label for the package that we declared at the beginning:

```python
import pandas as pd
```

so we can access the pandas function 'read_csv' with the command:

```python
pd.read_csv()
```

but read csv has required input parameters. The essential one is the name of the file. For our circumstance all the other default parameters are fine. If you want to see all the possible parameters for this function, just go to the docs [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html).

* The docs are always helpful
* There is often a lot of flexibility for Python functions, possible through using various inputs parameters

also, the program has an output, a pandas DataFrame loaded from the data.  So we have to specficy the name / variable representing that new object.

```python
df = pd.read_csv("12_sample_data.csv")
```

Let's run this command to load the data and then look at the resulting DataFrame to ensure that we loaded it.

```python
df = pd.read_csv("12_sample_data.csv")                      # read a .csv file in as a DataFrame
```

* Let's get a quick preview of our new DataFrame, 'df' with the built in function.

```python
df.head()
```

```python
df.head(n=13)
```

      X
      Y
      Facies
      Porosity
      Perm
      AI

      0
      50
      900
      1
      0.220761
      140.021266
      3413.063944

      1
      50
      850
      1
      0.237154
      39.837129
      3074.562617

      2
      50
      800
      1
      0.234352
      84.992437
      2292.783358

      3
      50
      750
      1
      0.244553
      90.632307
      2494.848885

      4
      50
      700
      1
      0.231787
      811.547979
      2522.063995

      5
      50
      650
      1
      0.233280
      426.992456
      3964.185956

      6
      50
      550
      1
      0.234423
      2398.406492
      3318.885844

      7
      50
      500
      1
      0.219657
      1637.224971
      3030.874323

      8
      50
      450
      1
      0.200389
      265.636019
      3454.389302

      9
      50
      350
      0
      0.165908
      7.951511
      5025.286221

      10
      50
      300
      0
      0.155199
      1.476967
      4939.312852

      11
      50
      150
      0
      0.175103
      5.476548
      4770.744199

      12
      100
      950
      1
      0.202124
      357.481641
      3383.118632

#### Summary Statistics for Tabular Data

The table includes X and Y coordinates (meters), Facies 1 and 0 (1 is sandstone and 0 interbedded sand and mudstone), Porosity (fraction), and permeability as Perm (mDarcy).

There are a lot of efficient methods to calculate summary statistics from tabular data in DataFrames. The describe command provides count, mean, minimum, maximum, and quartiles all in a nice data table. We use transpose just to flip the table so that features are on the rows and the statistics are on the columns.

```python
df.describe().transpose()
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X
      480.0
      430.187500
      263.832692
      0.000000
      200.000000
      390.000000
      630.000000
      980.000000

      Y
      480.0
      522.166667
      284.293420
      19.000000
      279.000000
      539.000000
      759.000000
      999.000000

      Facies
      480.0
      0.616667
      0.486706
      0.000000
      0.000000
      1.000000
      1.000000
      1.000000

      Porosity
      480.0
      0.189440
      0.031702
      0.117562
      0.165884
      0.185443
      0.216513
      0.261091

      Perm
      480.0
      520.932093
      1226.207190
      0.005776
      6.539988
      49.451463
      369.470756
      10319.904850

      AI
      480.0
      3758.879653
      779.990582
      1746.387548
      3212.900120
      3719.883000
      4236.160395
      6194.573653

#### Visualization of Loaded Data

In this excises, we want to use support vector machine to help segment, classify, facies based on acoustic impedance and porosity predictor features.

First we need to visualize the dataframe we loaded.

* We will use the Seabon package has built-in function ```scatterplot ``` to do scatter plot with categorical data like facies.

```python
sns.scatterplot(x = 'Porosity', y = 'AI', hue='Facies',data=df,legend='full',palette = "Set1")
```

Segmentation looks possible, but it will not be easy as there is a lot of overlap.

#### Split Training and Testing Data

As a supervised learning classifier, support vector machine requires:

* **training data** to train the model parameters

* **testing data** to check the model performance and tune the hyperparameters.

First we split loaded data into training (70%) and testing data (30%) randomly using a function from sklean

```python
train, test = train_test_split()
```

The function parameters include:

* **df** - the data frame
* **random_state** - the random number seed for reproducibility
* **test_size** - the fraction of the data withheld for the testing data set

```python
train, test = train_test_split(df,random_state=4, test_size=0.3)
```

Check the training and testing set to see if split fairly.

* not too few data
* not extremely clustered
* limited extrapolation from training to testing

Of course, there is a lot of complexity related to a 'fair' testing dataset. We will not get into this now.

Here's the training dataset.

```python
### plots the training data set
sns.scatterplot(x = 'Porosity', y = 'AI', hue='Facies',data=train,legend='full',palette = "Set1")
train.describe().transpose()
```

and here is the testing dataset.

```python
###Testing data set
sns.scatterplot(x = 'Porosity', y = 'AI', hue='Facies',data=test,legend='full',palette = "Set1")

test.describe().transpose()
```

We need to reformat our data to inputs to train the support vector machine.

* we extract the features (Acoustic impedance and Porosity) into a $n \times 2$ array and the cateogrical response (facies) into a separate $n \times 1$ array

where $n$ is the number of data samples

We will show the first 10 samples to check our reformatting.

```python
train_X = train.loc[:,['Porosity','AI']]
train_X = train_X.reset_index(drop = True)

train_y = train['Facies']
train_y = train_y.reset_index(drop = True)
print('The training predictor features:')
print(train_X.head())
print('\nThe training response features:')
print(train_y.head())
```

    The training predictor features:
       Porosity           AI
    0  0.204313  4373.187870
    1  0.228370  2975.483280
    2  0.166260  5016.841662
    3  0.237154  3074.562617
    4  0.233895  2386.160099

    The training response features:
    0    1
    1    1
    2    0
    3    1
    4    1
    Name: Facies, dtype: int64

SVMs assume that the data it works with is in a standard range, usually -1 to 1 (roughly). So we need to normalize the features prior to fit them to the SVM. Otherwise one feature would be scaled more important than the other and will skew the results.

We will transfor the normalized data back when visualize the result.

```python
scaler = StandardScaler()
xx = scaler.fit_transform(train_X)
train_X['nPorosity'] = xx[:,0]
train_X['nAI'] = xx[:,1]
train_X.head()
```

      Porosity
      AI
      nPorosity
      nAI

      0
      0.204313
      4373.187870
      0.445012
      0.806662

      1
      0.228370
      2975.483280
      1.218698
      -0.960185

      2
      0.166260
      5016.841662
      -0.778816
      1.620308

      3
      0.237154
      3074.562617
      1.501206
      -0.834938

      4
      0.233895
      2386.160099
      1.396390
      -1.705152

#### Support Vector Machine Model with Linear Kernel

Let's train and plot linear support vector machine models over our solution space.

* This will provide a linear spatial classification model for facies 0 and 1 as a function of AI and porosity.

We use the scikit-learn function $SVC$ substantiate the support vector machine:

```python
svm_linear = SVC()
```

The parameters include:

* **kernel** the kernel type that is applied to project the data to a potentially higher dimensional space
* **C** penalty for misclassification
* **random_state** random number see for random shuffling data for probability estimates

We then use the command"

```python
svm_linear.fit()
```
to train the model with the training dataset.

The parameters include:

* **X** - the $n \times m$ array with the predictor features for the training dataset
* **y** - the $n \times 1$ array with the response feature for the training dataset

We will also

```python
C = 0.1
svm_linear = SVC(kernel = 'linear',C = C, random_state = 0)    # Create a SVC classifier using a linear kernel
svm_linear.fit(train_X.loc[:,['nPorosity','nAI']], train_y)    # Train the classifier

C10 = 10
svm_linear10 = SVC(kernel = 'linear',C = C10, random_state = 0)    # Create a SVC classifier using a linear kernel
svm_linear10.fit(train_X.loc[:,['nPorosity','nAI']], train_y)    # Train the classifier
```

    SVC(C=10, cache_size=200, class_weight=None, coef0=0.0,
      decision_function_shape='ovr', degree=3, gamma='auto_deprecated',
      kernel='linear', max_iter=-1, probability=False, random_state=0,
      shrinking=True, tol=0.001, verbose=False)

Looks like it ran ok!  Let's visualize the results using the convenient visualization functions that we previously defined.

```python
plt.subplot(121)
plt = visualize_model(svm_linear,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plot_svc_decision_function(svm_linear,plt,plot_support=True)
plt.title('Suport Vector Machine Linear Kernel with C=0.1')

plt.subplot(122)
plt = visualize_model(svm_linear10,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plot_svc_decision_function(svm_linear10,plt,plot_support=True)
plt.title('Suport Vector Machine Linear Kernel with C=10')
plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.0, wspace=0.2, hspace=0.2)
```

The above plot shows the linear kernel support vector machine classification model (Facies 0 is red and facies 1 is blue), the training dataset and the resulting support vectors circled.

* Linear kernel only provide a straight decision boundary.

* It is hard to tune the model to fit more complicated situation.

Let's try some more flexible classifiers with a different kernel.Recall, there is a trade off:

* more complexity reduces model bias (error due to a simple model not fitting the actual data)
* but increases model variance (sensitivity to training data).

#### Support Vector Machine Model with Polynomial Kernel

The polynomial kernel is defined as

\begin{equation}
K(x,x') = (x^Tx)^d,
\end{equation}

where $d$ is the degree of polynomials.

While using the SVC function in scikit-learn package,```C```is a parameter of the SVC learner and is the penalty for misclassifying a data point. When C is smaller, the classifier is more tolerant with misclassified data points (high bias, low variance).

```Degree``` is to control the order of the polynomial kernel function.

```python
C = 1
svm_poly3_C1 = SVC(kernel = 'poly', degree = 3, random_state=0,C = C)
svm_poly3_C1.fit(train_X.loc[:,['nPorosity','nAI']],train_y)

C = 1
svm_poly4_C1 = SVC(kernel = 'poly', degree = 4,random_state=0, C = C)
svm_poly4_C1.fit(train_X.loc[:,['nPorosity','nAI']],train_y)

C = 100
svm_poly3_C100 = SVC(kernel = 'poly', degree = 3,random_state=0, C = C)
svm_poly3_C100.fit(train_X.loc[:,['nPorosity','nAI']],train_y)

C = 100
svm_poly4_C100 = SVC(kernel = 'poly', degree = 4, random_state=0, C = C)
svm_poly4_C100.fit(train_X.loc[:,['nPorosity','nAI']],train_y)
```

    SVC(C=100, cache_size=200, class_weight=None, coef0=0.0,
      decision_function_shape='ovr', degree=4, gamma='auto_deprecated',
      kernel='poly', max_iter=-1, probability=False, random_state=0,
      shrinking=True, tol=0.001, verbose=False)

Looks like it ran again!  Let's visualize the results again with our plotting function.

```python
plt.subplot(221)
plt = visualize_model(svm_poly3_C1,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('3rd order polynomial kernel with C=1')
plot_svc_decision_function(svm_poly3_C1,plt, plot_support=True)

plt.subplot(222)
plt = visualize_model(svm_poly4_C1,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('4th order polynomial kernel with C=1')
plot_svc_decision_function(svm_poly4_C1,plt, plot_support=True)

plt.subplot(223)
plt = visualize_model(svm_poly3_C100,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('3rd order polynomial kernel with C=100')
plot_svc_decision_function(svm_poly3_C100,plt, plot_support=True)

plt.subplot(224)
plt = visualize_model(svm_poly4_C100,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('4th order polynomial kernel with C=100')
plot_svc_decision_function(svm_poly4_C100,plt, plot_support=True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=2.5, wspace=0.2, hspace=0.2)
```

The 3rd order polynomial kernal performs quite well.  With a low $C$, penalty parameter, we see more local fit, while the high penatly parameter result in a less specific fit.

The 4th order and higher order polynomial does not perform well.

#### Support Vector Machine Model with Radial Basis Function Kernel

Radial Basis Function (RBF) is another commonly used kernel in SVC:

\begin{equation}
K(x,x') = e^{- \gamma ||x-x'||^2},
\end{equation}

where $||x-x'||^2$ is the squared Euclidean distance between two data points x and x'.

Gaussian kernel is a special case of RBF, where:

\begin{equation}
K(x,x') = e^{- \frac {||x-x'||^2} {2 \sigma^2}}.
\end{equation}

By changing the value of $\gamma$ and C, the classifier with an RBF kernel can be tuned.

$\gamma$ can be thought of as the spread of the kernel.

* When $\gamma$ is low, the curve of the decision boundary is low and lead to a broad decision region (low variance, high bias)

* The $\gamma$ parameter can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.

```python
C_2d_range = [1e-1, 1, 1e2]
gamma_2d_range = [1e-1, 1, 1e1]
classifiers = []
for C in C_2d_range:
    for gamma in gamma_2d_range:
        clf = SVC(kernel = 'rbf',C=C, gamma=gamma)
        clf.fit(train_X.loc[:,['nPorosity','nAI']], train_y)
        classifiers.append((C, gamma, clf))
for (k, (C, gamma, clf)) in enumerate(classifiers):
    plt.subplot(len(C_2d_range), len(gamma_2d_range), k + 1)
    plt = visualize_model(clf,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
    plot_svc_decision_function(clf,plt,plot_support=True)
    plt.title("gamma=10^%d, C=10^%d" % (np.log10(gamma), np.log10(C)))
plt.subplots_adjust(left=0.0, bottom=0.0, right=3.5, top=3.5, wspace=0.2, hspace=0.2)
```

Although two facies seem to be classified properly in some cases above, there is a risk of overfitting.

#### Evaluating Performance Over Multiple Parameters

Now we try to evaluate the result and find the proper parameter values for all the dataset.

* Draw heatmap of the cross-validation accuracy as a function of gamma and C.

* The higher score indicates better testing performance

First we must normalize all of the data.

```python
xx = scaler.fit_transform(df.loc[:,['Porosity','AI']])  ### normalize all the data
df['nPorosity'] = xx[:,0]
df['nAI'] = xx[:,1]
df.head()
```

Now we will iterative over multiple parameter cases.

* Part of the code was modified from scikit-learn RBF SVM parameter example
* This Loop will take a while to run

```python
C_range = np.logspace(-2, 10, 13)
gamma_range = np.logspace(-9, 3, 13)
param_grid = dict(gamma=gamma_range, C=C_range)
cv = StratifiedShuffleSplit(n_splits=5, test_size=0.3, random_state=4)
grid = GridSearchCV(SVC(), param_grid=param_grid, cv=cv)
grid.fit(df.loc[:,['nPorosity','nAI']], df['Facies'])
scores = grid.cv_results_['mean_test_score'].reshape(len(C_range),
                                                     len(gamma_range))
```

Now we can visualize the cross validation accuracy for all of the parameter combinations.

```python
plt.figure(figsize=(8, 6))
plt.imshow(scores, cmap=plt.cm.hot)
plt.xlabel('gamma')
plt.ylabel('C')
plt.colorbar()
plt.xticks(np.arange(len(gamma_range)), gamma_range, rotation=45)
plt.yticks(np.arange(len(C_range)), C_range)
plt.title('Validation accuracy')
```

#### Visualizing High, Mid and Low Performing Models

Now we show examples of high, mid and low performing model based on validation accuracy from the demonstration above.

* we will use parameter combinations, $C$ and $\gamma$, from the plot above to select and rerun the cases.

```python
C=1e6;gamma=0.01                                                # High
clf_c0_g01 = SVC(kernel = 'rbf',C=C, gamma=gamma)
clf_c0_g01.fit(train_X.loc[:,['nPorosity','nAI']], train_y)

C=100;gamma=10                                                  # Mid
clf_c100_g10 = SVC(kernel = 'rbf',C=C, gamma=gamma)
clf_c100_g10.fit(train_X.loc[:,['nPorosity','nAI']], train_y)

C=100;gamma=100                                                 # Low
clf_c100_g100 = SVC(kernel = 'rbf',C=C, gamma=gamma)
clf_c100_g100.fit(train_X.loc[:,['nPorosity','nAI']], train_y)
```

Now let's visualize the low, mid and high performing support vector machine classification models for our facies assignment.

```python
plt.subplot(131)
plt = visualize_model(clf_c0_g01,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('High Cross Validation Accuracy')
plot_svc_decision_function(clf_c0_g01,plt, plot_support=True)

plt.subplot(132)
plt = visualize_model(clf_c100_g10,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('Mid Cross Validation Accuracy')
plot_svc_decision_function(clf_c100_g10,plt, plot_support=True)

plt.subplot(133)
plt = visualize_model(clf_c100_g100,train_X["nPorosity"],train_X["nAI"],train_X["Porosity"],train_X["AI"],train_y)
plt.title('Low Cross Validation Accuracy')
plot_svc_decision_function(clf_c100_g100,plt, plot_support=True)
plt.subplots_adjust(left=0.0, bottom=0.0, right=3, top=1, wspace=0.2, hspace=0.2)
```

### Summary

Here's some observations from this workflow.

* Linear kernel only works for linear decision boundary problems while polynomial and RBF kernels perform better for more complicated boundaries.

* By tuning parameters like C and $\gamma$, it is possible to use RBF model to segment facies with a good bias-variance trade-off.

* SVM does not work well when there is overlap among each category and it is sensitive to outliers.

* Computationally intensive when there are a lot of data.

#### Comments

This was a basic demonstration of support vector machines.

The Texas Center for Geostatistics has many other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available [here](https://github.com/GeostatsGuy/PythonNumericalDemos), along with a package for geostatistics in Python called [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy).

We hope this was helpful,

*Wendi* and *Michael*

***

#### More on Michael Pyrcz and the Texas Center for Geostatistics:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


```python
!pip install python-snap7==0.5
# va en version 1.1
import snap7.client as c
from snap7.util import * #set_int   set_bool
from snap7.snap7types import * # areas
import time
# !pip install keyboard
import keyboard  # using module keyboard
import os
import math
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
```

    Requirement already satisfied: python-snap7==0.5 in c:\users\usuario\.conda\envs\sistdin\lib\site-packages (0.5)

# Excitación del módulo  multivariable SIEMENS del PCJIC

En el laboratorio de automática del Politécnico Colombiano Jaime Isaza Cadavid se encuentra un módulo multivariable que está controlado con un PLC  SIEMENS. Por esta razón, se utiliza el módulo `SNAP7` para interactuar con el PLC desde el notebook.

Para instalar `SNAP7`:
- Use la instrucción
~~~
!pip install python-snap7==0.5
~~~

- Descargue la librería completa de [https://sourceforge.net/projects/snap7/files/1.2.1/](https://sourceforge.net/projects/snap7/files/1.2.1/)

- Copie `snap7.dll` y `snap7.lib` a su carpeta `System32`.

## 1. Establezca conexión con el PLC.

El PLC tiene configurada la dirección IP **192.168.0.1**. Este funcionará como un servidor al cuál debe conectar su PC configurándole una dirección estática dentro de la subred del PLC.  Su dirección puede ser **192.168.0.2**, por ejemplo. Una vez su equipo haya establecido comunicación con el PLC, establezca conexión desde el notebook con las siguientes instrucciones.

```python
#Conexión con el PLC
plc = c.Client()
IP_PLC = '192.168.0.1'
```

```python
try:
    plc.connect(IP_PLC,0,1)
    print("Conectado")
except:
    print("Algo no funcionó")
```

    Conectado

## 2. Identifique actuadores y sensores.

Con `SNAP7` puede manipular los actuadores del sistema y capturar los datos de los sensores. Se han construido algunas funciones para facilitar la interacción con el PLC.

Tenga en cuenta la siguiente información, que le orientará en cómo se ha conectado el sistema al PLC.

|Elemento   |  Punto  |
|-----------|---------|
| Presión 	|    IW96 |
| Flujo		|    IW98 |
| Nivel		|   IW100 |
| Run Bomba	|    Q0.5 |
| Bomba		|    QW80 |
| Válvula H20 |  QW96 |

```python
#ESCRIBIR SALIDA BOOLEANA
def escr_sal_bool(byte, bit, valor):
    lectura = plc.ab_read(byte, bit)
    set_bool(lectura, byte, bit, valor)
    plc.ab_write(0, lectura)
#escr_sal_bool(0,1,1)

#ESCRIBIR SALIDA ENTERO
def escr_sal_ent(byte,valor):
    lectura = plc.read_area(areas['PA'], 0, byte, 2) #PA: salidas, 0: bloque de datos, dirección, # bytes.
    # print(lectura)
    set_int(lectura, 0, valor)  # se da formato al valor deseado, en este caso entero
    plc.write_area(areas['PA'], 0, byte, lectura)  # Escribe en la dirección definida
#escr_sal_ent(90,9000)

#LEER MARCA ENTERA
def leer_ent_ent(byte):
    leer = plc.read_area(areas['PE'],0,byte,2) #PE: entradas, 0: bloque de datos, dirección, # bytes.
    leer_ent = get_int(leer,0) #Comando get_int(_bytearray, byte_index)
    return leer_ent

def AbrirValvulaQW96(valvulap):
    # Abre la valvula en QW96 a un porcentaje determinado
    valvula = ((7800 / 71.5) * (valvulap - 7.4)) + 6200
    escr_sal_ent(96,valvula)

def BombaQW80(motobombaHz):
    #Poner bomba en 60Hz salida QW80
    motobomba = motobombaHz * (22118 / 60) + 5530
    escr_sal_ent(80,motobomba)

def leerNivelPLCIW100():
    nivelplc = leer_ent_ent(100)
    nivelcm = ((60 / 15105) * (nivelplc - 10125)) + 20
    # print(nivelplc, nivelcm)
    return nivelcm

def leerPresionPLCIW96():
    presionplc = leer_ent_ent(96)
    # 0.64 5679
    # 0.4  5623
    mp = (0.64-0.4)/(5679-5623)
    presion =  mp* presionplc - mp *5623+0.4
    # print(presionplc, presion)
    return presion

def leerFlujoPLCIW98():
    flujoplc = leer_ent_ent(98)
    # 4.67 8773
    # 16.52  16957
    mf = (4.67-16.52)/(8773-16957)
    flujo =  mf* flujoplc - mf *16957+16.52
    # print(flujoplc, flujo)
    return flujo

def leerNada():
    return np.random.rand(1)[0]
```

## 3. Inicie el experimento.

El módulo multivariable tiene:
- 2 actuadores:
    - Motobomba
    - Válvula
- 3 sensores:
    - Flujo
    - Presión
    - Nivel

3.1 Escoja una variable como salida y un actuador para su experimento.

3.2 Ponga un punto de operación para el otro actuador.

**Establecer un punto para la bomba**
~~~
BombaQW80(60)
~~~

**Establecer un punto para la válvula**
~~~
AbrirValvulaQW96(100)
~~~

```python
escr_sal_bool(0,5,1) #Habilitar RUN motobomba
# Complete para establecer un punto de operación
BombaQW80(20)
AbrirValvulaQW96(100)
```

    bytearray(b']\x99')
    bytearray(b'\x15\x10')

## 5. Defina la entrada del sistema

Esta vez se configura una entrada en forma de un escalón.

\begin{equation}
U(t) = \begin{cases} V_0 & \forall t<t_0 \\  V_f &  \forall t > t_0 \end{cases}
\end{equation}

Defina:
- Tiempo de muestreo $t_m$
- Duración para el experimento $t_f$
- Valor inicial para el actuador $V_0$
- Valor final para el actuador $V_f$

**Lista de tiempos $t$**

```python
tm = 1
tf = 500.0
t = np.linspace(0.0 ,tf, round(tf/tm) + 1)

```

**Señal de entrada $U(t)$**

```python
t0 = 30
v0 = 20
vf = 60
U = np.where(t>=t0,vf,v0)

```

**Gráfica de la señal**

```python
plt.plot(t,U)
```

## 6. Realice el experimento y guarde

Entregue los valores definidos para $U(t)$ en el actuador cada $t_m$ segundos, lea el sensor apropiado para su caso y guarde la información necesaria. Para esto puede crear un archivo `csv` a partir de un **dataframe**.

Para leer los sensores se han definido las siguientes funciones:

~~~
leerNivelPLCIW100()
leerPresionPLCIW96()
leerFlujoPLCIW98()
~~~

```python
Y = []
print('Para abortar el experimento presione ESPACIO')
abortado = False
for idt,ti in enumerate(t):
    Entrada = U[idt]
    # Poner el valor en el actuador
    #AbrirValvulaQW96(Entrada)
    BombaQW80(Entrada)
    Y.append(leerPresionPLCIW96())
    if keyboard.is_pressed(' '):
        print('Abortando el experimento a los ' +
              str(ti) +' segundos')
        abortado = True
        escr_sal_bool(0,5,0)
        BombaQW80(5)
        AbrirValvulaQW96(0)

        break
    time.sleep(tm)

if abortado:
    Y = Y + [np.nan]*(len(U)-len(Y))
d = {'Tiempo': t, 'Entrada': U, 'Salida': Y}

df = pd.DataFrame(data=d)
df.head()
escr_sal_bool(0,5,0)
AbrirValvulaQW96(0)

```

    Para abortar el experimento presione ESPACIO
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'2f')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    bytearray(b'l\x00')
    Abortando el experimento a los 359.0 segundos
    bytearray(b'l\x00')
    bytearray(b'?\xad')
    bytearray(b'\x15\x10')

```python
# Gráfica de entrada y salida
plt.plot(df["Tiempo"],df["Entrada"],
         df["Tiempo"],10*df["Salida"])
```

```python
# Guarda datos
archivo = "Datos/sebastianmartinezospina.csv"
df.to_csv(archivo, index=False)
```

```python

```


# Run Scores II for many ships (as input to Ikeda)

```python
# %load imports.py
"""
These is the standard setup for the notebooks.
"""

%matplotlib inline
%load_ext autoreload
%autoreload 2

from jupyterthemes import jtplot
jtplot.style(theme='onedork', context='notebook', ticks=True, grid=False)

import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
import os
import matplotlib.pyplot as plt
from collections import OrderedDict
#plt.style.use('paper')

#import data
import copy
from mdldb.run import Run

from sklearn.pipeline import Pipeline
from rolldecayestimators.transformers import CutTransformer, LowpassFilterDerivatorTransformer, ScaleFactorTransformer, OffsetTransformer
from rolldecayestimators.direct_estimator_cubic import EstimatorQuadraticB, EstimatorCubic
from rolldecayestimators.ikeda_estimator import IkedaQuadraticEstimator
import rolldecayestimators.equations as equations
import rolldecayestimators.lambdas as lambdas
from rolldecayestimators.substitute_dynamic_symbols import lambdify
import rolldecayestimators.symbols as symbols
import sympy as sp

from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex

from sklearn.metrics import r2_score
from src.data import database
from mdldb import tables
import shipflowmotionshelpers.shipflowmotionshelpers as helpers

```

    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 461 ('figure.figsize   : 5, 3   ## figure size in inches')
    Duplicate key in file WindowsPath('C:/Users/maa/.matplotlib/stylelib/paper.mplstyle'), line 462 ('figure.dpi       : 100        ## figure dots per inch')

```python
from pyscores2.indata import Indata
from pyscores2.runScores2 import Calculation
from pyscores2.output import OutputFile
from pyscores2 import TDPError
import pyscores2
from rolldecayestimators.ikeda import Ikeda, IkedaR
from rolldecayestimators.simplified_ikeda_class import SimplifiedIkeda
import subprocess
from rolldecayestimators import measure
```

```python
df_all_sections_id = pd.read_csv('../data/external/all_sections.csv', sep=';')
df_all_sections_id.head()
```

      b
      area
      t
      x
      SHIP
      loading_condition_id

      0
      0.00
      0.0
      0.0
      0.00
      68
      NaN

      1
      26.86
      19.5
      6.6
      8.61
      68
      NaN

      2
      28.17
      54.7
      6.6
      17.22
      68
      NaN

      3
      28.40
      96.0
      6.6
      25.83
      68
      NaN

      4
      28.40
      129.4
      6.6
      34.44
      68
      NaN

```python
section_groups=df_all_sections_id.groupby(by='loading_condition_id')
```

```python
loading_condition_ids = df_all_sections_id['loading_condition_id'].unique()
mask=pd.notnull(loading_condition_ids)
loading_condition_ids=loading_condition_ids[mask]
```

```python
df_rolldecay = database.load(rolldecay_table_name='rolldecay_quadratic_b', limit_score=0.99,
                             exclude_table_name='rolldecay_exclude')
```

```python
mask=df_rolldecay['loading_condition_id'].isin(loading_condition_ids)
df=df_rolldecay.loc[mask].copy()
```

```python
df['BKB'].fillna(0, inplace=True)
```

```python
def add_cScores(sections):
    sections=sections.copy()
    sections['cScores']=sections['area']/(sections['b']*sections['t'])
    mask=sections['cScores']>1
    sections.loc[mask,'cScores']=1
    return sections

def cut_sections(sections, draught):
    sections=sections.copy()
    mask = sections['t']>draught
    sections.loc[mask,'t']=draught
    sections.loc[mask,'area']-=draught*sections['b'].max()  # Assuming rectangular shape
    return sections

def remove_duplicate_sections(sections):
    sections=sections.copy()
    mask=~sections['x'].duplicated()
    sections=sections.loc[mask]
    assert sections['x'].is_unique
    return sections

def too_small_sections(sections):
    sections=sections.copy()
    small = 0.1
    mask=sections['b']==0
    sections.loc[mask,'b']=small
    mask=sections['t']==0
    sections.loc[mask,'t']=small
    mask=sections['area']==0
    sections.loc[mask,'area']=small
    return sections
```

```python
from scipy.integrate import simps
def calculate_lcb(x, area, **kwargs):
    """
    Calculate lcb from AP
    """
    return simps(y=area*x,x=x)/np.trapz(y=area,x=x)

def calculate_dispacement(x, area, **kwargs):
    """
    Calculate displacement
    """
    return np.trapz(y=area,x=x)
```

```python
class DraughtError(ValueError): pass

def define_indata(row, sections, rho=1000,  g=9.81):

    indata = Indata()

    draught=(row.TA+row.TF)/2
    indata.draught=draught
    if draught<=sections['t'].max():
        sections = cut_sections(sections, draught)
    else:
        raise DraughtError('Draught is too large for sections')

    sections=add_cScores(sections)

    indata.cScores=np.array(sections['cScores'])
    indata.ts=np.array(sections['t'])
    indata.bs=np.array(sections['b'])
    indata.zbars=np.zeros_like(sections['b'])  # Guessing...

    beam=sections['b'].max()
    indata.lpp=sections['x'].max()-sections['x'].min()
    #indata.displacement=row.Volume
    indata.displacement=calculate_dispacement(**sections)

    indata.g=g
    indata.kxx=row.KXX
    indata.kyy=row.lpp*0.4
    lcb=calculate_lcb(x=sections['x'], area=sections['area'])
    indata.lcb=lcb-row.lpp/2
    indata.lpp=row.lpp
    indata.projectName='loading_condition_id_%i' % row.loading_condition_id

    indata.rho=rho
    indata.zcg=row.kg-draught
    #indata.waveFrequenciesMin=0.2
    #indata.waveFrequenciesMax=0.5
    #indata.waveFrequenciesIncrement=0.006
    w=row.omega0/np.sqrt(row.scale_factor)
    indata.waveFrequenciesMin=w*0.5
    indata.waveFrequenciesMax=w*2.0
    N=40
    indata.waveFrequenciesIncrement=(indata.waveFrequenciesMax-indata.waveFrequenciesMin)/N
    indata.runOptions["IE"].set_value(1)

    return indata,sections
```

```python
results = pd.DataFrame()
fi_as = np.deg2rad([1,10])
for run_name, row in df.iterrows():
    loading_condition_id=row['loading_condition_id']
    sections = section_groups.get_group(loading_condition_id)
    sections=remove_duplicate_sections(sections)
    sections=too_small_sections(sections)

    try:
        indata,sections_ = define_indata(row, sections)
    except DraughtError as e:
        print('Draught is too large for sections, this loading condition is skipped.')
        continue

    save_name='%s.in' % row.loading_condition_id
    save_path=os.path.join('scores',save_name)
    indata.save(save_path)

    calculation = Calculation(outDataDirectory='scores/result')

    # Run scoresII:
    try:
        calculation.run(indata=indata, b_div_t_max=None, timeout=1.0)
    except TDPError:
        print('Dissregarding the TDPError')
        continue
    except pyscores2.LcgError as e:
        print('Disregarded')
        print(e)
        continue
    except subprocess.TimeoutExpired:
        print('Disregarded, scoresII got stuck...')
        continue

    output_file = OutputFile(filePath=calculation.outDataPath)
```

    Running Scores2 for loading_condition_id_107

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_14

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_34

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_14

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_14

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_34

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_34

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_34

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_89
    Disregarded, scoresII got stuck...
    Running Scores2 for loading_condition_id_119

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_89
    Disregarded, scoresII got stuck...
    Running Scores2 for loading_condition_id_10

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_119

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_99
    Disregarded, scoresII got stuck...
    Running Scores2 for loading_condition_id_10

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_10

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_6

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_3
    Dissregarding the TDPError
    Running Scores2 for loading_condition_id_3
    Dissregarding the TDPError
    Running Scores2 for loading_condition_id_6

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_6

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_37

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_15

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_20

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_106

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_15

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_15

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_106

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_106

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Draught is too large for sections, this loading condition is skipped.
    Running Scores2 for loading_condition_id_100

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Draught is too large for sections, this loading condition is skipped.
    Running Scores2 for loading_condition_id_8

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_8

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Draught is too large for sections, this loading condition is skipped.
    Draught is too large for sections, this loading condition is skipped.
    Running Scores2 for loading_condition_id_118
    Disregarded, scoresII got stuck...
    Running Scores2 for loading_condition_id_13

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_1

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_13

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_13

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_4

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_4

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_49
    Disregarded
    <class 'pyscores2.LcgError'>: The calculated longitudinal centre of bouyancy differ from the nomonal value ((LCB-LCG)/L max 0.5%) Calculated LCB = 4.271000 m (FWD. OF MIDSHIPS)
    Running Scores2 for loading_condition_id_22

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_7

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_7

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_7

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_107

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_78

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_32

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_78

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_78

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_78

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_107

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_21

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_21

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_19

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_107

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_79

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_79

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)

    Running Scores2 for loading_condition_id_79

    c:\python36-64\lib\re.py:212: FutureWarning: split() requires a non-empty pattern match.
      return _compile(pattern, flags).split(string, maxsplit)