###  2D Stochastic Gaussian Simulation in Python for Engineers and Geoscientists

### with GSLIB's SGSIM Program Converted to Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

#### Contacts: [Twitter/@GeostatsGuy](https://twitter.com/geostatsguy) | [GitHub/GeostatsGuy](https://github.com/GeostatsGuy) | [www.michaelpyrcz.com](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446)

This is a tutorial for / demonstration of **Sequential Gaussian Simulation in Python with GSLIB's SGSIM program translated to Python from the original FORTRAN GSLIB: Geostatistical Library methods** (Deutsch and Journel, 1997).

### PGE 383 Exercise: Methods for Spatial Simulation with GeostatsPy

Here's a simple workflow for spatial simulation with SGSIM. This method is ciritical for:

* Building spatial models that honor the univariate and spatial distribution of the property of interest

First let's explain the concept of spatial simulation.

#### Spatial Simulation

We can define and contrast estimation and simulation.

* Estimation:
    * a method to calculate the best estimate at each location focus on local accuracy, globally too smooth

* Simulation:
    * a method to calculate a good / reasonable estimate at each location focus on global accuracy, sacrifice local accuracy

What is wrong with kriging?

Kriging is locally accurate and smooth, appropriate for visualizing trends, inappropriate for any applications where heterogeneity is important

The ''variance'' of the kriged estimates is too small:

\begin{equation}
Var\{Y^*(\bf{u})\} = \sigma^2 - \sigma^2_{SK}(\bf{u})
\end{equation}

$\sigma^2$ is variance of the property, $𝑉𝑎𝑟{𝑌^∗(\bf{𝐮})}$ is the variance of the estimates

Consider the following:

* $\sigma^2_{SK}(\bf{u})$ is zero at the data locations → no smoothing

* $\sigma^2_{SK}(\bf{u})$ is the variance $\sigma^2$ beyond range of data locations, this is complete smoothing

* spatial variations of $\sigma^2_{SK}(\bf{u})$ depend on the variogram and data spacing

Missing variance in the estimates, $Var\{Y^*(\bf{u})\}$, is the kriging variance, Monte Carlo Simulation corrects the variance (get right histogram)

* we will add a random residual, $Y_s(\bf{u}) = Y^*(\bf{u}) + R(\bf{u})$, where $R(\bf{u})$ is the random residual that adds back in the missing variance

and correct the covariance (get the right variogram)

* sequential simulation – add simulated values to data to enforce the correct covariance between estimates

Simulation reproduces histogram, honors spatial variability (variogram); therefore, it is appropriate for process evaluation where heterogeneity is important

Also, the simulation approach allows an assessment of uncertainty with alternative realizations

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data_MV_biased.csv available at https://git.io/fhgu0.

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import os                                                 # to set current working directory
import numpy as np                                        # arrays and matrix math
import pandas as pd                                       # DataFrames
import matplotlib.pyplot as plt                           # plotting
import math
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Load the Required Libraries

The following code loads the required libraries.

```python
import geostatspy.geostats as geostats
import geostatspy.GSLIB as GSLIB
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see above) GSLIB executables in this directory or a location identified in the environmental variable *Path*.

```python
os.chdir("c:/PGE383")                                     # set the working directory
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object. We will also extra a limited sample so that the spatial samples are not too dense.  This way we can observe more of the heterogeneity from the simulation with the spatial continuity model, rather than mostly data driven heterogeneity.

```python
df = pd.read_csv("sample_data_MV_biased.csv")             # read a .csv file in as a DataFrame
df.describe()                                             # summary statistics
df = df.sample(50)                                        # extract 50 samples
df = df.reset_index()                                     # reset the record index
df['logPerm'] = np.log(df['Perm'].values)
df.head()
```

      index
      Unnamed: 0
      X
      Y
      Facies
      Porosity
      Perm
      AI
      logPerm

      0
      331
      523
      280.0
      799.0
      1.0
      0.161938
      395.717857
      4212.827564
      5.980701

      1
      206
      339
      680.0
      859.0
      1.0
      0.139132
      15.031775
      4684.316559
      2.710166

      2
      291
      458
      420.0
      959.0
      1.0
      0.137036
      46.273841
      4460.227665
      3.834577

      3
      285
      452
      860.0
      889.0
      1.0
      0.116910
      1.446133
      3931.954570
      0.368893

      4
      95
      156
      540.0
      799.0
      1.0
      0.126067
      24.399946
      4906.490088
      3.194581

#### Sequential Gaussian Simulation

Let's jump right to building multiple realizations with simulation and visualizing the results.  We will start with a test, comparision of simulation with simple and ordinary kriging.

```python
# Demonstration of Multiple Realizations - Short Continuity
nx = 100; ny = 100; xsiz = 10.0; ysiz = 10.0; xmn = 5.0; ymn = 5.0; nxdis = 1; nydis = 1
ndmin = 0; ndmax = 20; radius = 400; ktype = 0; skmean = 0
vario = GSLIB.make_variogram(nug=0.0,nst=1,it1=1,cc1=1.0,azi1=45.0,hmaj1=100,hmin1=60)
tmin = -999; tmax = 999

sim = geostats.sgsim(df,'X','Y','Porosity',wcol=-1,scol=-1,tmin=tmin,tmax=tmax,itrans=1,ismooth=0,dftrans=0,tcol=0,twtcol=0,
            zmin=0.0,zmax=0.3,ltail=1,ltpar=0.0,utail=1,utpar=0.3,nsim=1,
            nx=nx,xmn=xmn,xsiz=xsiz,ny=ny,ymn=ymn,ysiz=ysiz,seed=73073,
            ndmin=ndmin,ndmax=ndmax,nodmax=20,mults=0,nmult=2,noct=-1,radius=radius,radius1=1,sang1=0,
            mxctx=10,mxcty=10,ktype=ktype,colocorr=0.0,sec_map=0,vario=vario)

sim2 = geostats.sgsim(df,'X','Y','Porosity',wcol=-1,scol=-1,tmin=tmin,tmax=tmax,itrans=1,ismooth=0,dftrans=0,tcol=0,
            twtcol=0,zmin=0.0,zmax=0.3,ltail=1,ltpar=0.0,utail=1,utpar=0.3,nsim=1,
            nx=nx,xmn=xmn,xsiz=xsiz,ny=ny,ymn=ymn,ysiz=ysiz,seed=73072,
            ndmin=ndmin,ndmax=ndmax,nodmax=20,mults=0,nmult=2,noct=-1,radius=radius,radius1=1,sang1=0,
            mxctx=10,mxcty=10,ktype=ktype,colocorr=0.0,sec_map=0,vario=vario)

sim3 = geostats.sgsim(df,'X','Y','Porosity',wcol=-1,scol=-1,tmin=tmin,tmax=tmax,itrans=1,ismooth=0,dftrans=0,tcol=0,
            twtcol=0,zmin=0.0,zmax=0.3,ltail=1,ltpar=0.0,utail=1,utpar=0.3,nsim=1,
            nx=nx,xmn=xmn,xsiz=xsiz,ny=ny,ymn=ymn,ysiz=ysiz,seed=73071,
            ndmin=ndmin,ndmax=ndmax,nodmax=20,mults=0,nmult=2,noct=-1,radius=radius,radius1=1,sang1=0,
            mxctx=10,mxcty=10,ktype=ktype,colocorr=0.0,sec_map=0,vario=vario)

xmin = 0.0; xmax = 1000.0; ymin = 0.0; ymax = 1000.0; cmap = plt.cm.inferno
plt.subplot(131)
GSLIB.locpix_st(sim,xmin,xmax,ymin,ymax,xsiz,0.05,0.25,df,'X','Y','Porosity','Sequential Gaussian Simulation - Realization 1','X(m)','Y(m)','Porosity',cmap)

plt.subplot(132)
GSLIB.locpix_st(sim2,xmin,xmax,ymin,ymax,xsiz,0.05,0.25,df,'X','Y','Porosity','Sequential Gaussian Simulation - Realization 2','X(m)','Y(m)','Porosity',cmap)

plt.subplot(133)
GSLIB.locpix_st(sim3,xmin,xmax,ymin,ymax,xsiz,0.05,0.25,df,'X','Y','Porosity','Sequential Gaussian Simulation - Realization 3','X(m)','Y(m)','Porosity',cmap)

plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()
```

Now we have 3 equiprobable realizations for porosity over the area of interest.  See the difference?  Away from the data at the range the full range of porosity values is possible!  Let's check the realizations.

#### Realization Checking

All of the theory of sequential Gaussian simulation supports the calculation of realizations that honor the input statistics.  Let' confirm that by checking the:

* histograms
* variogram

```python
np.linspace(0.0,0.4,20)
```

    array([0.        , 0.02105263, 0.04210526, 0.06315789, 0.08421053,
           0.10526316, 0.12631579, 0.14736842, 0.16842105, 0.18947368,
           0.21052632, 0.23157895, 0.25263158, 0.27368421, 0.29473684,
           0.31578947, 0.33684211, 0.35789474, 0.37894737, 0.4       ])

```python
# Check the histograms
plt.subplot(121)
plt.hist(sim.flatten(), facecolor='blue',bins=np.linspace(0.0,0.4,40),alpha=0.2,density=True,edgecolor='black',label = 'Sim 1')
plt.hist(sim2.flatten(), facecolor='yellow',bins=np.linspace(0.0,0.4,40),alpha=0.2,density=True,edgecolor='black',label = 'Sim 2')
plt.hist(sim3.flatten(), facecolor='green',bins=np.linspace(0.0,0.4,40),alpha=0.2,density=True,edgecolor='black',label = 'Sim 3')
plt.hist(df['Porosity'].values, facecolor='black',bins=np.linspace(0.0,0.4,40),alpha=0.2,density=True,edgecolor='black',label = 'Data Distribution')
plt.xlim([0.0,0.4]); plt.ylim([0,20.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity Data vs. Realizations')
plt.legend(loc='upper right')
plt.grid(True)

plt.subplot(122)
plt.hist(sim.flatten(), color='red',lw = 2, bins=np.linspace(0.0,0.4,50),histtype="step",alpha=1.0,density=True,cumulative=True,edgecolor='blue',label='Sim 1')
plt.hist(sim2.flatten(), color='blue',lw = 2, bins=np.linspace(0.0,0.4,50),histtype="step",alpha=1.0,density=True,cumulative=True,edgecolor='yellow',label='Sim 2')
plt.hist(sim3.flatten(), color='green',lw = 2, bins=np.linspace(0.0,0.4,50),histtype="step",alpha=1.0,density=True,cumulative=True,edgecolor='green',label='Sim 3')
plt.hist(df['Porosity'].values, color='black',lw = 2,bins=np.linspace(0.0,0.4,50),histtype="step",alpha=1.0,density=True,cumulative=True,edgecolor='black',label='Data Distribution')
plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Data vs. Realizations')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

The histograms look quite good.  The three realizations have

Let's look at 4 realizations of porosity all with short range isotropic variograms.

```python
lag045sim1, vario045sim1, npar045 = geostats.gam(sim,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = -1, nlag = 80, isill = 1.0)
lag045sim2, vario045sim2, npar045 = geostats.gam(sim2,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = -1, nlag = 80, isill = 1.0)
lag045sim3, vario045sim3, npar045 = geostats.gam(sim3,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = -1, nlag = 80, isill = 1.0)

index,lag045mod,vario045mod,cov,ro  = geostats.vmodel(nlag = 80, xlag = 10.0, azm = 45.0, vario = vario)

lag135sim1, vario135sim1, npar045 = geostats.gam(sim,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = 1, nlag = 80, isill = 1.0)
lag135sim2, vario135sim2, npar045 = geostats.gam(sim2,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = 1, nlag = 80, isill = 1.0)
lag135sim3, vario135sim3, npar045 = geostats.gam(sim3,tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = 1, nlag = 80, isill = 1.0)

index,lag135mod,vario135mod,cov,ro  = geostats.vmodel(nlag = 80, xlag = 10.0, azm = 135.0, vario = vario)

plt.subplot(121)
plt.plot(lag045sim1,vario045sim1,'x',color = 'red',label = 'Sim 1')
plt.plot(lag045sim2,vario045sim2,'x',color = 'blue',label = 'Sim 2')
plt.plot(lag045sim3,vario045sim3,'x',color = 'green',label = 'Sim 3')
plt.plot(lag045mod,vario045mod,'-',color = 'black',label = 'Mod')

plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional Azimuth 045 Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(122)
plt.plot(lag135sim1,vario135sim1,'x',color = 'red',label = 'Sim 1')
plt.plot(lag135sim2,vario135sim2,'x',color = 'blue',label = 'Sim 2')
plt.plot(lag135sim3,vario135sim3,'x',color = 'green',label = 'Sim 3')
plt.plot(lag135mod,vario135mod,'-',color = 'black',label = 'Mod')

plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional Azimuth 135 Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

#### Minimum Acceptance Checks

We will make more realizations and dmeonstrate the approach of minimum acceptance checks.

With this approach we should check:

* the simulated values at the data locations
* the histograms of the simulated realizations
* the spatial continuity of the simulated realizations

We will skip the the first check as with our 'simple' workflow the data are directly assigned to the model cells.

You can specify how many realizations as 'nreal' below.

```python
nreal = 20

nx = 100; ny = 100; xsiz = 10.0; ysiz = 10.0; xmn = 5.0; ymn = 5.0; nxdis = 1; nydis = 1
ndmin = 0; ndmax = 20; radius = 400; ktype = 0; skmean = 0
vario = GSLIB.make_variogram(nug=0.0,nst=1,it1=1,cc1=1.0,azi1=45.0,hmaj1=100,hmin1=60)
tmin = -999; tmax = 999

realizations = np.zeros((nreal,ny,nx))
for ireal in range(0,nreal):
    realizations[ireal] = geostats.sgsim(df,'X','Y','Porosity',wcol=-1,scol=-1,tmin=tmin,tmax=tmax,itrans=1,ismooth=0,dftrans=0,tcol=0,twtcol=0,
            zmin=0.0,zmax=0.3,ltail=1,ltpar=0.0,utail=1,utpar=0.3,nsim=1,
            nx=nx,xmn=xmn,xsiz=xsiz,ny=ny,ymn=ymn,ysiz=ysiz,seed=73073+ireal,
            ndmin=ndmin,ndmax=ndmax,nodmax=20,mults=0,nmult=2,noct=-1,radius=radius,radius1=1,sang1=0,
            mxctx=10,mxcty=10,ktype=ktype,colocorr=0.0,sec_map=0,vario=vario)

```

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0111 (close to 0.0?)
                                       variance = 0.9696 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0165 (close to 0.0?)
                                       variance = 0.8806 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0037 (close to 0.0?)
                                       variance = 0.9748 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.001 (close to 0.0?)
                                       variance = 0.8929 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.125 (close to 0.0?)
                                       variance = 0.9932 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0164 (close to 0.0?)
                                       variance = 0.9726 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0112 (close to 0.0?)
                                       variance = 0.9639 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.1126 (close to 0.0?)
                                       variance = 1.021 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.1699 (close to 0.0?)
                                       variance = 0.8708 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.019 (close to 0.0?)
                                       variance = 1.051 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0133 (close to 0.0?)
                                       variance = 0.9965 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0038 (close to 0.0?)
                                       variance = 1.1016 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.1349 (close to 0.0?)
                                       variance = 0.9483 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0413 (close to 0.0?)
                                       variance = 0.9677 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0767 (close to 0.0?)
                                       variance = 0.9271 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0068 (close to 0.0?)
                                       variance = 0.9985 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.1161 (close to 0.0?)
                                       variance = 0.9427 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0365 (close to 0.0?)
                                       variance = 1.0217 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = 0.0297 (close to 0.0?)
                                       variance = 0.9704 (close to gammabar(V,V)? approx. 1.0)

     Data for SGSIM: Number of acceptable data     = 50
                     Number trimmed                = 0
                     Weighted Average              = 0.1246
                     Weighted Variance             = 0.0007
                     Weighted Transformed Average  = 0.0
                     Weighted Transformed Variance = 0.9749
    Setting up rotation matrices for variogram and search
    Working on realization number 0
       currently on node 0
       currently on node 1000
       currently on node 2000
       currently on node 3000
       currently on node 4000
       currently on node 5000
       currently on node 6000
       currently on node 7000
       currently on node 8000
       currently on node 9000
    Reassigning data to nodes

     Realization 0: number   = 10000
                                       mean     = -0.0038 (close to 0.0?)
                                       variance = 0.9783 (close to gammabar(V,V)? approx. 1.0)

Now let's check all of the distributions.

```python
plt.subplot(111)
for ireal in range(0,nreal):
    if ireal == 0:
        plt.hist(realizations[ireal].flatten(), color='red',lw = 2, bins=np.linspace(0.0,0.4,1000),histtype="step",alpha=0.2,density=True,cumulative=True,edgecolor='red',label='Sim')
    else:
        plt.hist(realizations[ireal].flatten(), color='red',lw = 2, bins=np.linspace(0.0,0.4,1000),histtype="step",alpha=0.2,density=True,cumulative=True,edgecolor='red')

plt.hist(df['Porosity'].values, color='black',lw = 2,bins=np.linspace(0.0,0.4,1000),histtype="step",alpha=0.8,density=True,cumulative=True,edgecolor='black',label='Data Distribution')
plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Data vs. Realizations')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

Now let's check the experimental variograms vs. the input model of spatial continuity.

```python
nlag = 80

lag045sim = np.zeros((nreal,nlag)); vario045sim = np.zeros((nreal,nlag)); npar045sim = np.zeros((nreal,nlag));
lag135sim = np.zeros((nreal,nlag)); vario135sim = np.zeros((nreal,nlag)); npar135sim = np.zeros((nreal,nlag));

for ireal in range(0,nreal):
    lag045sim[ireal], vario045sim[ireal], npar045sim[ireal] = geostats.gam(realizations[ireal],tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = -1, nlag = nlag, isill = 1.0)
    lag135sim[ireal], vario135sim[ireal], npar135sim[ireal] = geostats.gam(realizations[ireal],tmin = -999, tmax = 999, xsiz = 10.0, ysiz = 10.0, ixd = 1, iyd = 1, nlag = nlag, isill = 1.0)

index,lag045mod,vario045mod,cov,ro  = geostats.vmodel(nlag = 80, xlag = 10.0, azm = 45.0, vario = vario)
index,lag135mod,vario135mod,cov,ro  = geostats.vmodel(nlag = 80, xlag = 10.0, azm = 135.0, vario = vario)

plt.subplot(121)
for ireal in range(0,nreal):
    if ireal == 0:
        plt.plot(lag045sim[ireal],vario045sim[ireal],'x',color = 'red',label = 'Sim', alpha = 0.2)
    else:
        plt.plot(lag045sim[ireal],vario045sim[ireal],'x',color = 'red', alpha = 0.2)

plt.plot(lag045mod,vario045mod,'-',color = 'black',label = 'Mod')

plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional Azimuth 045 Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(122)
for ireal in range(0,nreal):
    if ireal == 0:
        plt.plot(lag135sim[ireal],vario135sim[ireal],'x',color = 'red',label = 'Sim',alpha = 0.2)
    else:
        plt.plot(lag135sim[ireal],vario135sim[ireal],'x',color = 'red',alpha = 0.2)

plt.plot(lag135mod,vario135mod,'-',color = 'black',label = 'Mod')

plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional Azimuth 135 Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

The variogram reproduction looks good.  There are ergodic fluctuations, but the model is at the average for each lag distance.

#### Comments

This was a basic demonstration of spatial simulation. I know the simulation program is rough and the code needs some clean up, but we are getting to the point where we have a function package for Geostatistics in Python.

Much more could be done, I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


## The inviscid Burgers' Equation - A different approach

#### Problem Setup

Setting $u(x,t) = \frac{x}{1+t}$, we expect $\nu = 0$ as a parameter.

Then $u_0(x) := u(x,0) = x$.

Using the forward Euler scheme, the equation can be re-written as:

and setting the factor $u_{n-1} = \mu_{n-1}$ (analogously to the previous subchapter this is the mean of $u_{n-1}$) to deal with the non-linearity:

Consider $u_{n-1}$ to be a Gaussian process.

$u_{n-1} \sim \mathcal{GP}(0, k_{uu}(x_i, x_j; \theta, l))$

And the linear operator:

$\mathcal{L}_x^\nu = \cdot + \tau \nu \frac{d}{dx}\cdot - \tau \mu_{n-1} \frac{d}{dx} \cdot$

so that

$\mathcal{L}_x^\nu u_{n-1} = u_n$

Problem at hand: Estimate $\nu$ (should be $\nu = 0$ in the end).

For the sake of simplicity, take $u := u_{n-1}$ and $f := u_n$.

#### Step 1: Simulate data

Take data points at $t = 0$ for $(u_{n-1})$ and $t = \tau$ for $(u_n)$, where $\tau$ is the time step.

```python
import numpy as np
import sympy as sp
from scipy.optimize import minimize
import matplotlib.pyplot as plt
import warnings
import time
```

$x \in [0, 1], \; t \in \{0, \tau \}$

```python
tau = 0.001
def get_simulated_data(tau, n=20):
    x = np.random.rand(n)
    y_u = x
    y_f = x/(1+tau)
    return (x, y_u, y_f)

(x, y_u, y_f) = get_simulated_data(tau)
```

```python
def show_1(x, y_u, y_f):
    f, (ax1, ax2) = plt.subplots(ncols=2, nrows=1, sharey=True, figsize=(10,3))
    f.suptitle("Data for u(x) in red and f(x) in blue")
    ax1.plot(x, y_u, 'ro')
    ax1.set(xlabel= r"x", ylabel= "u(x)")
    ax2.plot(x, y_f, 'bo')
    ax2.set(xlabel= r"x", ylabel= "f(x)");
```

```python
show_1(x, y_u, y_f)
```

#### Step 2: Evaluate kernels

1) $k_{uu}(x_i, x_j; \theta, l) = \theta exp(-\frac{1}{2l}(x_i-x_j)^2)$

```python
x_i, x_j, theta, l, nu = sp.symbols('x_i x_j theta l nu')
mu = np.mean(y_u)
kuu_sym = theta*sp.exp(-1/(2*l)*((x_i - x_j)**2))
kuu_fn = sp.lambdify((x_i, x_j, theta, l), kuu_sym, "numpy")
def kuu(x, theta, l):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kuu_fn(x[i], x[j], theta, l)
    return k
```

2) $k_{ff}(x_i,x_j;\theta, l,\nu)
= \mathcal{L}_{x_i}^\nu \mathcal{L}_{x_j}^\nu k_{uu}(x_i, x_j; \theta, l) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau \mu_{n-1} \frac{d}{dx_i}k_{uu} + \tau \nu \frac{d}{dx_j}k_{uu} + \tau^2 \nu^2 \frac{d}{dx_i} \frac{d}{dx_j}k_{uu} - \tau^2 \nu \mu_{n-1}\frac{d^2}{dx_i dx_j} k_{uu} - \tau \mu_{n-1} \frac{d}{dx_j}k_{uu} - \tau^2 \nu \mu_{n-1} \frac{d^2}{dx_i dx_j} k_{uu} + \tau^2 \mu_{n-1}^2 \frac{d^2}{dx_i dx_j}k_{uu}$

```python
kff_sym = kuu_sym \
        + tau*nu*sp.diff(kuu_sym, x_i) \
        - tau*mu*sp.diff(kuu_sym, x_i) \
        + tau*nu*sp.diff(kuu_sym, x_j) \
        + tau**2*nu**2*sp.diff(kuu_sym, x_j, x_i) \
        - tau**2*nu*mu*sp.diff(kuu_sym, x_j, x_i) \
        - tau*mu*sp.diff(kuu_sym, x_j) \
        - tau**2*nu*mu*sp.diff(kuu_sym, x_j, x_i) \
        + tau**2*mu**2*sp.diff(kuu_sym, x_j, x_i)
kff_fn = sp.lambdify((x_i, x_j, theta, l, nu), kff_sym, "numpy")
def kff(x, theta, l, nu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kff_fn(x[i], x[j], theta, l, nu)
    return k
```

3) $k_{fu}(x_i,x_j;\theta,l,\nu)
= \mathcal{L}_{x_i}^\nu k_{uu}(x_i, x_j; \theta, l) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau \mu_{n-1}\frac{d}{dx_i}k_{uu}$

```python
kfu_sym = kuu_sym + tau*nu*sp.diff(kuu_sym, x_i) - tau*mu*sp.diff(kuu_sym, x_i)
kfu_fn = sp.lambdify((x_i, x_j, theta, l, nu), kfu_sym, "numpy")
def kfu(x, theta, l, nu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kfu_fn(x[i], x[j], theta, l, nu)
    return k
```

4) $k_{uf}(x_i,x_j;\theta, l, \nu)$ is given by the transpose of $k_{fu}(x_i,x_j;\theta, l, \nu)$.

```python
def kuf(x, theta, l, nu):
    return kfu(x,theta, l, nu).T
```

#### Steps 3 and 4: Compute NLML and optimize the hyperparameters

```python
def nlml(params, x, y1, y2, s):
    theta_exp = np.exp(params[0])
    l_exp = np.exp(params[1])
    K = np.block([
        [kuu(x, theta_exp, l_exp) + s*np.identity(x.size), kuf(x, theta_exp, l_exp, params[2])],
        [kfu(x, theta_exp, l_exp, params[2]), kff(x, theta_exp, l_exp, params[2]) + s*np.identity(x.size)]
    ])
    y = np.concatenate((y1, y2))
    val = 0.5*(np.log(abs(np.linalg.det(K))) + np.mat(y) * np.linalg.inv(K) * np.mat(y).T)
    return val.item(0)
```

```python
m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7), method=\
             "Nelder-Mead", options = {'maxiter' : 1000})
```

```python
m.x[2]   # This is our inferred value for \nu
```

    0.00021347647791778476

```python
m
```

     final_simplex: (array([[6.98888116e+00, 8.08598514e+00, 2.13476478e-04],
           [6.98887122e+00, 8.08597556e+00, 2.13481534e-04],
           [6.98882034e+00, 8.08592923e+00, 2.13858738e-04],
           [6.98884831e+00, 8.08595542e+00, 2.13548057e-04]]), array([-291.94040781, -291.94038991, -291.94038888, -291.940384  ]))
               fun: -291.9404078103601
           message: 'Optimization terminated successfully.'
              nfev: 209
               nit: 106
            status: 0
           success: True
                 x: array([6.98888116e+00, 8.08598514e+00, 2.13476478e-04])

#### Step 5: Analysis w.r.t. the number of data points (up to 25):

In this section we want to analyze the error of our algorithm and plot its time complexity.

```python
res = np.zeros((5,25))
timing = np.zeros((5,25))
warnings.filterwarnings("ignore")
for k in range(5):
    for n in range(25):
        start_time = time.time()
        (x, y_u, y_f) = get_simulated_data(tau, n)
        m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7), method="Nelder-Mead")
        res[k][n] = m.x[2]
        timing[k][n] = time.time() - start_time
```

**1. Plotting the error in our estimate:**

The error is given by $| \nu_{estimate} - \nu_{true} |$.

```python
lin = np.linspace(1, res.shape[1], res.shape[1])

for i in range(res.shape[0]):
    plt.plot(lin, np.abs(res[i,:]))
    plt.ylabel('Error')
    plt.xlabel('Number of data points')
    plt.show()
```

We plot the error with respect to the number of data samples for five runs of the program:

```python
%matplotlib inline
import matplotlib.ticker as ticker

# Setting global figsize to (5,3)
import matplotlib
matplotlib.rcParams["figure.figsize"] = (5,3)

lin = np.linspace(5, res.shape[1] - 1, res.shape[1] - 5)

def show_2(lin, res):

    plt.figure(figsize=(5,3))
    to_del = np.linspace(0, 4, 5)

    ax = plt.figure().gca()
    ax.xaxis.set_major_locator(ticker.MaxNLocator(integer=True))

    plt.ylabel('Error')
    plt.xlabel('Number of data points')
    plt.suptitle(r'Error in our estimate for $\nu$')

    for i in range(res.shape[0]):
        a_i = np.delete(np.abs(res[i,:]), to_del)
        plt.plot(lin, a_i)

    est = np.repeat(0.16, len(lin))
    plt.plot(lin, est, color='blue', linestyle='dashed')
    plt.show()
```

```python
show_2(lin, res)
```

We see that for n sufficiently large (in this case $n \geq 8$), we can assume the error to be bounded by 0.16.

**2. Plotting the error between the solution and the approximative solution:**

**3. Plotting the execution time:**

The blue-dashed line follows $f(x) = 0.032 x^2$.

```python
lin = np.linspace(1, timing.shape[1], timing.shape[1])

for i in range(timing.shape[0]):
    plt.plot(lin, timing[i,:])
    plt.ylabel('Execution time in seconds')
    plt.xlabel('Number of data points')
    plt.show()
```

```python
lin = np.linspace(1, timing.shape[1], timing.shape[1])

def show_3(lin, timing):

    plt.suptitle('Execution time of the algorithm')
    plt.ylabel('Seconds')
    plt.xlabel('Number of data points')

    for i in range(timing.shape[0]):
        plt.plot(lin, timing[i,:])

    est = 0.032*lin**(2)
    plt.plot(lin, est, color='blue', linestyle='dashed')
    plt.show()
```

```python
show_3(lin, timing)
```

We observe a time complexity of roughly $\mathcal{O}(n^2)$ (blue-dashed line).

#### Comparison with a no-mean version

$\tau \nu \frac{d^2}{dx^2}u_{n-1} - \tau x \frac{d}{dx}u_{n-1} + u_{n-1} = u_{n}$

The linear operator looks just slightly different:

$\mathcal{L}_x^\nu = \cdot + \tau \nu \frac{d}{dx}\cdot - \tau x \frac{d}{dx} \cdot$

so that

$\mathcal{L}_x^\nu u_{n-1} = u_n$.

Our kernels evaluate to:

1) $k_{uu}(x_i, x_j; \theta, l) = \theta exp(-\frac{1}{2l}(x_i-x_j)^2)$

2) $k_{ff}(x_i,x_j;\theta, l,\nu)
= \mathcal{L}_{x_i}^\nu \mathcal{L}_{x_j}^\nu k_{uu}(x_i, x_j; \theta, l) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau x \frac{d}{dx_i}k_{uu} + \tau \nu \frac{d}{dx_j}k_{uu} + \tau^2 \nu^2 \frac{d}{dx_i} \frac{d}{dx_j}k_{uu} - \tau^2 \nu x\frac{d^2}{dx_i dx_j} k_{uu} - \tau x \frac{d}{dx_j}k_{uu} - \tau^2 \nu x \frac{d^2}{dx_i dx_j} k_{uu} + \tau^2 x^2 \frac{d^2}{dx_i dx_j}k_{uu}$

3) $k_{fu}(x_i,x_j;\theta,l,\nu)
= \mathcal{L}_{x_i}^\nu k_{uu}(x_i, x_j; \theta, l) \\
= k_{uu} + \tau \nu \frac{d}{dx_i}k_{uu} - \tau x\frac{d}{dx_i}k_{uu}$

4) $k_{uf}(x_i,x_j;\theta, l, \nu)$ is given by the transpose of $k_{fu}(x_i,x_j;\theta, l, \nu)$.

After constructing the *nlml* we get:

```python
%matplotlib inline
import matplotlib.ticker as ticker

# Setting global figsize to (5,3)
import matplotlib
matplotlib.rcParams["figure.figsize"] = (5,3)
```

```python
x_i, x_j, theta, l, nu = sp.symbols('x_i x_j theta l nu')
kuu_sym = theta*sp.exp(-1/(2*l)*((x_i - x_j)**2))
kuu_fn = sp.lambdify((x_i, x_j, theta, l), kuu_sym, "numpy")
def kuu(x, theta, l):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kuu_fn(x[i], x[j], theta, l)
    return k
```

```python
kff_sym = kuu_sym \
        + tau*nu*sp.diff(kuu_sym, x_i) \
        - tau*x_i*sp.diff(kuu_sym, x_i) \
        + tau*nu*sp.diff(kuu_sym, x_j) \
        + tau**2*nu**2*sp.diff(kuu_sym, x_j, x_i) \
        - tau**2*nu*x_i*sp.diff(kuu_sym, x_j, x_i) \
        - tau*x_j*sp.diff(kuu_sym, x_j) \
        - tau**2*nu*x_j*sp.diff(kuu_sym, x_j, x_i) \
        + tau**2*x_i*x_j*sp.diff(kuu_sym, x_j, x_i)
kff_fn = sp.lambdify((x_i, x_j, theta, l, nu), kff_sym, "numpy")
def kff(x, theta, l, nu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kff_fn(x[i], x[j], theta, l, nu)
    return k
```

```python
kfu_sym = kuu_sym + tau*nu*sp.diff(kuu_sym, x_i) - tau*x_i*sp.diff(kuu_sym, x_i)
kfu_fn = sp.lambdify((x_i, x_j, theta, l, nu), kfu_sym, "numpy")
def kfu(x, theta, l, nu):
    k = np.zeros((x.size, x.size))
    for i in range(x.size):
        for j in range(x.size):
            k[i,j] = kfu_fn(x[i], x[j], theta, l, nu)
    return k
```

```python
def kuf(x, theta, l, nu):
    return kfu(x,theta, l, nu).T
```

```python
def nlml(params, x, y1, y2, s):
    theta_exp = np.exp(params[0])
    l_exp = np.exp(params[1])
    K = np.block([
        [kuu(x, theta_exp, l_exp) + s*np.identity(x.size), kuf(x, theta_exp, l_exp, params[2])],
        [kfu(x, theta_exp, l_exp, params[2]), kff(x, theta_exp, l_exp, params[2]) + s*np.identity(x.size)]
    ])
    y = np.concatenate((y1, y2))
    val = 0.5*(np.log(abs(np.linalg.det(K))) + np.mat(y) * np.linalg.inv(K) * np.mat(y).T)
    return val.item(0)
```

```python
m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7), method=\
             "Nelder-Mead", options = {'maxiter' : 1000})
m.x[2] # This is our prediction for \nu
```

    0.00042724953581625225

We can analyze the error in multiple runs and look at the execution time:

```python
res = np.zeros((5,25))
timing = np.zeros((5,25))
warnings.filterwarnings("ignore")
for k in range(5):
    for n in range(25):
        start_time = time.time()
        (x, y_u, y_f) = get_simulated_data(tau, n)
        m = minimize(nlml, np.random.rand(3), args=(x, y_u, y_f, 1e-7), method="Nelder-Mead")
        res[k][n] = m.x[2]
        timing[k][n] = time.time() - start_time
```

**1. Plotting the error in our estimate for** $\nu$

The error is given by $| \nu_{estimate} - \nu_{true} |$.

```python
lin  =  np.linspace(1, res.shape[1], res.shape[1])
est = np.repeat(0.01, len(lin))

f, (ax1, ax2) = plt.subplots(ncols=2, nrows=2, figsize=(13,7))
ax1[0].plot(lin, np.abs(res[0,:]), color = 'green')
ax1[0].plot(lin, est, color='blue', linestyle='dashed')
ax1[0].set(xlabel= r"Number of data points", ylabel= "Error")
ax1[1].plot(lin, np.abs(res[1,:]), color = 'green')
ax1[1].plot(lin, est, color='blue', linestyle='dashed')
ax1[1].set(xlabel= r"Number of data points", ylabel= "Error")
ax2[0].plot(lin, np.abs(res[2,:]), color = 'green')
ax2[0].plot(lin, est, color='blue', linestyle='dashed')
ax2[0].set(xlabel= r"Number of data points", ylabel= "Error")
ax2[1].plot(lin, np.abs(res[3,:]), color = 'green')
ax2[1].plot(lin, est, color='blue', linestyle='dashed')
ax2[1].set(xlabel= r"Number of data points", ylabel= "Error");
```

```python
lin = np.linspace(5, res.shape[1] - 1, res.shape[1] - 5)
est = np.repeat(0.0009, len(lin))

def show_4(lin, est, res):

    to_del = np.linspace(0, 4, 5)

    ax = plt.figure().gca()
    ax.xaxis.set_major_locator(ticker.MaxNLocator(integer=True))

    plt.suptitle(r'Error in our estimate for $\nu$')
    plt.ylabel('Error')
    plt.xlabel('Number of data points')

    for i in range(res.shape[0]):
        a_i = np.delete(np.abs(res[i,:]), to_del)
        plt.plot(lin, a_i)

    plt.plot(lin, est, color='blue', linestyle='dashed')
```

```python
show_4(lin, est, res)
```

We see that for n sufficiently large (in this case $n \geq 5$), we can assume the error to be bounded by 0.0009. <br>

**2. Plotting the execution time:**

```python
lin = np.linspace(1, timing.shape[1], timing.shape[1])

def show_5(lin, timing):
    for i in range(timing.shape[0]):
        plt.plot(lin, timing[i,:])
        plt.ylabel('Seconds')
        plt.xlabel('Number of data points')
        plt.suptitle('Execution time of our algorithm')

    est = 0.032*lin**(2)
    plt.plot(lin, est, color='blue', linestyle='dashed')
    plt.show()
```

```python
show_5(lin, timing)
```

The time complexity seems to be as before roughly $\mathcal{O}(n^2)$.


On Galerkin approximations for the QG equations

Supplementary material for subsection on the $\beta-$Eady model

Wave structure for Charney mode

Cesar B. Rocha*

, William R. Young, and Ian Grooms

Winter 2015

*Scripps Institution of Oceanography, University of California, San Diego, 9500 Gilman Dr. MC 0213, La Jolla, CA/USA, <crocha@ucsd.edu>

```python
from __future__ import division

import numpy as np
from numpy import pi, sqrt,cos

import matplotlib.pyplot as plt
plt.rcParams.update({'font.size': 25, 'legend.handlelength'  : 1.25})
%matplotlib inline

import seaborn as sns
#sns.set(style="darkgrid")
sns.set_context("paper", font_scale=5, rc={"lines.linewidth": 1.5})
```

### A function to compute difference matrices

```python
def D_matrices(N):
    ''' create an N x N difference matrices

    D1 :: first-order, centered difference
    D2 :: second-order, centered difference

                                            '''
    D2 = np.zeros((N,N))
    D1 = np.zeros((N,N))
    for i in range(N):
        D2[i,i] = -2.
        if i<N-1:
            D2[i,i+1],D1[i,i+1] = 1.,-1
        if i>0:
            D2[i,i-1],D1[i,i-1] = 1.,1.
    return D1,D2
```

### Load data

```python
data_path = 'linear_charney_num_kappa_8.npz'
charney = np.load(data_path)

kappa = charney['kappa']
phi_max = charney['e_num'][1:-1]  # do no consider ghost points
N = charney['N']

# the critical level
zc = charney['c_num'].real - 1.   # recall the domain has depth 1
```

### set up domain

```python
# vertical coordinate
dz = 1./N   # vertical resolution
z = np.arange(-dz/2,-1.-dz/2.,-dz)  # level array

# horizontal coordinate
x = np.linspace(0,np.pi,100)

# grid
X,Z = np.meshgrid(x,z)
```

### compute wavestructure
We solved the problem for the streamfunction $\phi$ vertical structure. The streamfunction is them

\begin{equation}
\psi = |\phi(z)|\cos{(i k x +P_{\psi}(z))}
\end{equation}

The associated PV is

\begin{equation}
q = \partial^2_{xx}\psi + \partial^2_{zz}\psi = (-k^2\,|\phi(z)| + \partial^2_{zz}\,\,|\phi(z)| - |\phi(z)|(\partial_zP_{\psi}(z))^2)) \cos{(i k x +P_{\psi}(z)}) -(2\partial_z|\phi(z)|\partial_z P_{\psi}(z)) \sin{(i k x +P_{\psi}(z)})\,
\end{equation}

The phase is
\begin{equation}
P_{\psi}(z) = \text{tan$^{-1}$}\frac{\text{Im}(\hat{\psi})}{\text{Re}(\hat{\psi})}\,.
\end{equation}

```python
# wave structure in xz-plane
phi_max_abs = np.abs(phi_max)
phi_max_phase = np.arctan2(phi_max.imag,phi_max.real)

phase = np.repeat(phi_max_phase,x.size).reshape(z.size,x.size)
mag = np.repeat(phi_max_abs,x.size).reshape(z.size,x.size)

# wave structure
PSI = mag*np.cos( kappa*X + phase )

phi = charney['e_num'][:]
phi_abs = np.abs(phi)
phi_phase = np.arctan2(phi.imag,phi.real)

D1,D2 = D_matrices(N+2)
D1,D2 = np.matrix(D1),np.matrix(D2)

phi_abs_prime = np.array(D1*np.matrix(phi_abs).T)[1:-1]/(2*dz)
phi_abs_dprime = np.array(D2*np.matrix(phi_abs).T)[1:-1]/(dz**2)
phi_phase_prime = np.array(D1*np.matrix(phi_phase).T)[1:-1]/(2*dz)
phi_phase_dprime = np.array(D2*np.matrix(phi_phase).T)[1:-1]/(dz**2)

mag_prime =  np.repeat(phi_abs_prime,x.size).reshape(z.size,x.size)
mag_dprime =  np.repeat(phi_abs_dprime,x.size).reshape(z.size,x.size)

phase_prime =  np.repeat(phi_phase_prime,x.size).reshape(z.size,x.size)
phase_dprime =  np.repeat(phi_phase_dprime,x.size).reshape(z.size,x.size)

cost = np.cos( kappa*X + phase)
sint = np.sin( kappa*X + phase)

PV = (-(kappa**2)*mag + mag_dprime - mag*(phase_prime**2) )*cost \
        - (2.*mag_prime*phase_prime + mag*phase_dprime)*sint

```

```python
lw = 2.
aph = .5

# PV and psi wave structure
plt.figure(figsize=(12,9))
plt.contour(X,Z,1e2*PSI,np.linspace(-10,10,9),colors='k')
plt.contourf(X,Z,PV,np.linspace(-6.,6.,9),cmap='RdBu_r',extend='both')
#plt.plot(x,np.ones(x.size)*zc,'w--',linewidth=lw,alpha=1)
plt.text(-0.375,zc-.01,r' $z_c \rightarrow$',fontsize=35)

cb = plt.colorbar(extend='both',shrink=.9)
cb.ax.text(.0,1.075,'PV',rotation=0,fontsize=30)

plt.text(2.4, -.075, r"$\beta-$Eady Problem, $\kappa = 8$", size=25, rotation=0.,\
            ha="center", va="center",\
            bbox = dict(boxstyle="round",ec='k',fc='w'))

plt.xticks([0.,pi/4,pi/2,3*pi/4,pi],[r'$0$',r'$\pi/4$',r'$\pi/2$',\
            r'$3\,\pi/4$',r'$\pi$'])

plt.ylabel('$z/H$')
plt.xlabel(r'$x/L_d$')

plt.savefig('figs/wave-structure_pv_psi_kappa_8_num.eps')
```

```python

```

```python

```


```python
from __future__ import division
from sympy import *
x, y, z, t = symbols('x y z t')
k, m, n = symbols('k m n', integer=True)
f, g, h = symbols('f g h', cls=Function)
init_printing()
```

```python
Integral(sqrt(1/x), x)
```

```python
!which python
```

    /app/.conda/bin/python

```python
!ls
```

    Untitled.ipynb	readme.txt

```python
!pip install mathjax
```

    Collecting mathjax
      Could not find a version that satisfies the requirement mathjax (from versions: )
    No matching distribution found for mathjax

```python
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

# Data for plotting
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2 * np.pi * t)

# Note that using plt.subplots below is equivalent to using
# fig = plt.figure() and then ax = fig.add_subplot(111)
fig, ax = plt.subplots()
ax.plot(t, s)

ax.set(xlabel='time (s)', ylabel='voltage (mV)',
       title='About as simple as it gets, folks')
ax.grid()

fig.savefig("test.png")
plt.show()
```

```python

```


# Workforce Scheduling Problem

## Objective and Prerequisites

In this example, you’ll learn how to solve a critical, central problem in the services industry: workforce scheduling. We’ll demonstrate how you can use mathematical optimization to generate an optimal workforce schedule that meets your business requirements, maximizes employee fairness and satisfaction, and minimizes the number of temporary workers your company needs to hire.

This modeling example is at the advanced level, where we assume that you know Python and the Gurobi Python API and that you have advanced knowledge of building mathematical optimization models. Typically, the objective function and/or constraints of these examples are complex or require advanced features of the Gurobi Python API.

**Download the Repository**

You can download the repository containing this and other examples by clicking [here](https://github.com/Gurobi/modeling-examples/archive/master.zip).

**Gurobi License**

In order to run this Jupyter Notebook properly, you must have a Gurobi license. If you do not have one, you can request an [evaluation license](https://www.gurobi.com/downloads/request-an-evaluation-license/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-PRO-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_workforce-scheduling_COM_EVAL_GITHUB_&utm_term=workforce-scheduling-problem&utm_content=C_JPM) as a *commercial user*, or download a [free license](https://www.gurobi.com/academia/academic-program-and-licenses/?utm_source=3PW&utm_medium=OT&utm_campaign=WW-MU-PRO-OR-O_LEA-PR_NO-Q3_FY20_WW_JPME_workforce-scheduling_ACADEMIC_EVAL_GITHUB_&utm_term=workforce-scheduling-problem&utm_content=C_JPM) as an *academic user*.

## Motivation
People are the most important asset for companies in the services industry as well as their largest source of costs.
Workforce allocation and personnel scheduling deal with the arrangement of work schedules  and the assignment of personnel shifts in order to cover the demand for resources that vary over time.

These problems are very important for companies in the services industries who rely on workforce resources such as:
- Telephone operators
- Hospital nurses
- Policemen
- Transportation personnel (plane crews, bus drivers, etc.)
- Hospitality personnel
- Restaurant personnel

## Problem Description

Consider a service business, like a restaurant, that develops its workforce plans for the next two weeks (considering a 7-day week). The service requires only one set of skills. There are a number of employed workers with the same set of skills and with identical productivity that are available to work on some of the days during the two-week planning horizon. There is only one shift per workday. Each shift may have different resource (worker) requirements on each workday. The service business may hire extra (temp) workers from an agency to satisfy shift requirements. The service business wants to minimize the number of extra workers that it needs to hire, and as a secondary objective, it wants to balance the workload of employed workers to ensure fairness.

## Model Formulation
Mathematical programming is a declarative approach where the modeler formulates a mathematical optimization problem that captures the key features of a complex decision problem. The Gurobi Optimizer solves the mathematical optimization problem using state-of-the-art mathematics and computer science.

A mathematical optimization model has five components:
- Sets.
- Parameters.
- Decision variables.
- Constraints.
- Objective function(s).

### Sets and Indices

$s \in \text{shifts}$: Index and set of shifts.

$w \in workers$: Index and set of employed workers.

$availability= \{(w,s): w \in workers \; \text{is available for} \; s \in \text{shifts} \}$: This set determines that the employed worker  $w$ is available to work on shift $s$.

### Parameters

$\text{shiftRequirements}(s) \in \mathbb{N}$: This parameter represents the number of workers required at each shift
$s \in \text{shifts}$.

$pay(w) \in \mathbb{R^{+}}$: This parameter is the salary per day of each worker $w \in workers$.

$relTol$: This relative tolerance parameter  makes it possible to deviate from the optimal objective function value of the primary objective, when solving the model for the secondary objective. (See explanation below of the multi-objective optimization process).

### Computed Parameters
These parameters are derived from the input parameters and/or from the optimization process.

$Opt(totSlack)$: This parameter is the minimum value of extra workers required to fill demand.

### Decision Variables

$x(w,s) \in \{0,1\}$: This variable equals 1 if worker $w \in workers$ is assigned to shift $s \in \text{shifts}$,
and 0 otherwise.
This decision variable is defined over the $availability$ set.

$Slack(s) \geq 0$: This variable is the number of extra workers required to satisfy the requirements of shift $s$.

### Auxiliary Variables
Auxiliary variables are derived from the values of the decision variables.

$totSlack$: This variable counts the total number of extra workers required to satisfy the demand for shift workers.

$\text{totShifts}(w)$: This variable counts the total number of shifts worked by each worker $w \in workers$.

$\text{minShift}$: This variable determines the minimum number of shifts allocated to workers.

$\text{maxShift}$: This variable  determines the maximum number of shifts allocated to workers.

## Optimization Process

In this modeling example, we tackle a multi-objective model where the primary objective is to minimize the number of extra workers required to satisfy shift requirements, and the secondary objective is to minimize the maximum difference in the number of shifts worked between any pair of workers. This secondary objective balances the workload allocated to workers.

The hierarchical multi-objective optimization approach will first solve the following model with the primary objective.

### Objective Function

- **Primary objective:** The objective function is to minimize the number of extra workers allocated.

\begin{equation}
\text{Min} \quad totSlack
\tag{0}
\end{equation}

### Constraints

- **Shift requirements:**  All shifts requirements most be satisfied.

\begin{equation}
\sum_{w \: \in \: workers} x(w,s)  +  Slack(s) = \text{shiftRequirements}(s) \quad \forall \; s \in \text{shifts}
\tag{1}
\end{equation}

- **Temps:** Compute the total number of extra workers required.

\begin{equation}
\sum_{s \: \in \: \text{shifts}} Slack(s) =  totSlack
\tag{2}
\end{equation}

- **Shifts:**  Compute the total number of shifts for each worker.

\begin{equation}
\sum_{s \: \in \: \text{shifts}} x(w,s)  = \text{totShifts}(w) \quad \forall \; w \in workers
\tag{3}
\end{equation}

- **Binary and non-negativity constraints:**

$$
x(w,s) \in \{0, 1 \}  \quad \forall \; (w,s) \in availability
$$

$$
Slack(s) \geq 0 \quad  \forall \; s \in \text{shifts}
$$

$$
\text{totShifts}(w) \geq 0 \quad \forall \; w \in workers
$$

$$
totSlack  \geq 0
$$

The parameter $Opt(totSlack)$ is set to be equal to the minimum number of extra workers required to satisfy demand. That is, $Opt(totSlack) = totSlack^{*} $, where $totSlack^{*} $ is the optimal value of the variable $totSlack$.

For a given relative tolerance of primary objective $relTol$, we now solve the following model where we minimize the secondary objective.

### Objective Function

- **Secondary objective:** The objective function is to minimize the difference between the maximum number of shifts allocated to a worker, and the minimum number of shifts allocated to a worker.

\begin{equation}
\text{Min} \quad ( \text{maxShift} -  \text{minShift} )
\tag{4}
\end{equation}

### Constraints

- **Shift requirements:**  All shifts requirements most be satisfied.

\begin{equation}
\sum_{w \: \in \: workers} x(w,s)  +  Slack(s) = \text{shiftRequirements}(s) \quad \forall \; s \in \text{shifts}
\tag{1}
\end{equation}

- **Relative tolerance:**  Ensure that the total number of extra workers (totSlack) minimized by the primary objective is satisfied up to a tolerance percentage.

\begin{equation}
\sum_{s \: \in \: shifts} Slack(s) \leq (1+relTol)*Opt(totSlack)
\tag{5}
\end{equation}

- **Shifts:**  Compute the total number of shifts for each worker.

\begin{equation}
\sum_{s \: \in \: \text{shifts}} x(w,s)  = \text{totShifts}(w) \quad \forall \; w \in workers
\tag{3}
\end{equation}

- **Minimum shifts:**  Compute the minimum number of shifts allocated to an employed worker.

\begin{equation}
\text{minShift} = \text{Min} \{\text{totShifts}(w): \ w \; in \; workers \}
\tag{6}
\end{equation}

- **Maximum shifts:**  Compute the maximum number of shifts allocated to an employed worker.

\begin{equation}
\text{maxShift} = \text{Max} \{\text{totShifts}(w): \ w \; in \; workers \}
\tag{7}
\end{equation}

- **Binary and non-negativity constraints:**

$$
x(w,s) \in \{0, 1 \}  \quad \forall \; (w,s) \in availability
$$

$$
Slack(s) \geq 0 \quad  \forall \; s \in \text{shifts}
$$

$$
\text{totShifts}(w) \geq 0 \quad   \forall \; w \in workers
$$

$$
\text{maxShift},  \text{minShift}   \geq 0
$$

This optimization process is done automatically by the Gurobi Python API when using the following method:

"Model.setObjectiveN()"

## Python Implementation

We now import the Gurobi Python Module. Then, we initialize the data structures with the given data.

```python
# This command imports the Gurobi functions and classes.

import gurobipy as gp
from gurobipy import GRB

import pandas as pd
from pylab import *
import matplotlib
import matplotlib.pyplot as plt
```

## Input Data

We define all the input data of the model.

### Parameter

$\text{shiftRequirements}(s) \in \mathbb{N}$: This parameter represents the number of workers required at each shift
$s \in \text{shifts}$.

```python
# Number of workers required for each shift.
shifts, shiftRequirements = gp.multidict({
  "Mon1":  3,
  "Tue2":  2,
  "Wed3":  4,
  "Thu4":  4,
  "Fri5":  5,
  "Sat6":  6,
  "Sun7":  5,
  "Mon8":  2,
  "Tue9":  2,
  "Wed10": 3,
  "Thu11": 4,
  "Fri12": 6,
  "Sat13": 7,
  "Sun14": 5 })
```

### Parameter

$pay(w) \in \mathbb{R^{+}}$: This parameter is the salary per day of each worker $w \in workers$.

```python
# Amount each worker is paid to work one shift.
workers, pay = gp.multidict({
  "Amy":   10,
  "Bob":   12,
  "Cathy": 10,
  "Dan":   8,
  "Ed":    8,
  "Fred":  9,
  "Gu":    11 })
```

### Set

$availability= \{(w,s): w \in workers \; \text{is available for} \; s \in \text{shifts} \}$: This set determines that the employed worker  $w$ is available to work on shift $s$.

```python
# Worker availability: defines on which day each employed worker is available.

availability = gp.tuplelist([
('Amy', 'Tue2'), ('Amy', 'Wed3'), ('Amy', 'Fri5'), ('Amy', 'Sun7'),
('Amy', 'Tue9'), ('Amy', 'Wed10'), ('Amy', 'Thu11'), ('Amy', 'Fri12'),
('Amy', 'Sat13'), ('Amy', 'Sun14'), ('Bob', 'Mon1'), ('Bob', 'Tue2'),
('Bob', 'Fri5'), ('Bob', 'Sat6'), ('Bob', 'Mon8'), ('Bob', 'Thu11'),
('Bob', 'Sat13'), ('Cathy', 'Wed3'), ('Cathy', 'Thu4'), ('Cathy', 'Fri5'),
('Cathy', 'Sun7'), ('Cathy', 'Mon8'), ('Cathy', 'Tue9'), ('Cathy', 'Wed10'),
('Cathy', 'Thu11'), ('Cathy', 'Fri12'), ('Cathy', 'Sat13'),
('Cathy', 'Sun14'), ('Dan', 'Tue2'), ('Dan', 'Wed3'), ('Dan', 'Fri5'),
('Dan', 'Sat6'), ('Dan', 'Mon8'), ('Dan', 'Tue9'), ('Dan', 'Wed10'),
('Dan', 'Thu11'), ('Dan', 'Fri12'), ('Dan', 'Sat13'), ('Dan', 'Sun14'),
('Ed', 'Mon1'), ('Ed', 'Tue2'), ('Ed', 'Wed3'), ('Ed', 'Thu4'),
('Ed', 'Fri5'), ('Ed', 'Sun7'), ('Ed', 'Mon8'), ('Ed', 'Tue9'),
('Ed', 'Thu11'), ('Ed', 'Sat13'), ('Ed', 'Sun14'), ('Fred', 'Mon1'),
('Fred', 'Tue2'), ('Fred', 'Wed3'), ('Fred', 'Sat6'), ('Fred', 'Mon8'),
('Fred', 'Tue9'), ('Fred', 'Fri12'), ('Fred', 'Sat13'), ('Fred', 'Sun14'),
('Gu', 'Mon1'), ('Gu', 'Tue2'), ('Gu', 'Wed3'), ('Gu', 'Fri5'),
('Gu', 'Sat6'), ('Gu', 'Sun7'), ('Gu', 'Mon8'), ('Gu', 'Tue9'),
('Gu', 'Wed10'), ('Gu', 'Thu11'), ('Gu', 'Fri12'), ('Gu', 'Sat13'),
('Gu', 'Sun14')
])
```

## Model Deployment

```python
# Create initial model.
m = gp.Model("workforce5")
```

    Using license file c:\gurobi\gurobi.lic

### Decision Variables

$x(w,s) \in \{0,1\}$: This variable equals 1 if worker $w \in workers$ is assigned to shift $s \in \text{shifts}$,
and 0 otherwise.
This decision variable is defined over the $availability$ set.

$Slack(s) \geq 0$: This variable is the number of extra workers required to satisfy the requirements of shift $s$.

```python
# Initialize assignment decision variables.

x = m.addVars(availability, vtype=GRB.BINARY, name="x")
```

```python
# Slack decision variables determine the number of extra workers required to satisfy the requirements
# of each shift
slacks = m.addVars(shifts, name="Slack")
```

### Auxiliary Variables
Auxiliary variables are derived from the values of the decision variables.

$totSlack$: This variable counts the total number of extra workers required to satisfy the demand for shift workers.

$\text{totShifts}(w)$: This variable counts the total number of shifts worked by each worker $w \in workers$.

$\text{minShift}$: This variable determines the minimum number of shifts allocated to workers.

$\text{maxShift}$: This variable  determines the maximum number of shifts allocated to workers.

```python
# Auxiliary variable totSlack to represent the total number of extra workers required to satisfy the
# requirements of all the shifts.
totSlack = m.addVar(name='totSlack')
```

```python
# Auxiliary variable totShifts counts the total shifts worked by each employed worker
totShifts = m.addVars(workers, name="TotShifts")
```

### Constraint

$$\sum_{w \: \in \: workers} x(w,s) \; + \; Slack(s) \; = \; \text{shiftRequirements}(s) \; \; \; \forall \; s \in shifts $$

```python
# Constraint: All shifts requirements most be satisfied.

shift_reqmts = m.addConstrs((x.sum('*',s) + slacks[s] == shiftRequirements[s] for s in shifts), name='shiftRequirement')

```

### Constraint

$$ \sum_{s \: \in \: shifts} Slack(s) \; = \; totSlack  $$

```python
# Constraint: set the auxiliary variable (totSlack) equal to the total number of extra workers
# required to satisfy shift requirements

num_temps = m.addConstr(totSlack == slacks.sum(), name='totSlack')

```

### Constraint

$$\sum_{s \: \in \: shifts} x(w,s) \; = \; \text{totShifts}(w) \; \; \forall \ w \; in \; workers$$

```python
# Constraint: compute the total number of shifts for each worker

num_shifts = m.addConstrs((totShifts[w] == x.sum(w,'*') for w in workers), name='totShifts')

```

The variable $\text{minShift}$ determines the minimum numbers of shifts allocated to workers. This variable is calculated as follows:
$\text{minShift} = Min \{\text{totShifts}(w): \ w \; in \; workers \} $

The variable $\text{maxShift}$ determines the maximum numbers of shifts allocated to workers. This variable is calculated as follows:
$\text{maxShift} = Max \{\text{totShifts}(w): \ w \; in \; workers \} $

```python
# Auxiliary variables.
# minShift is the minimum number of shifts allocated to workers
# maxShift is the maximum number of shifts allocated to workers

minShift = m.addVar(name='minShift')

maxShift = m.addVar(name='maxShift')

# Constraint:
# The addGenConstrMin() method of the model object m adds a new general constraint that
# determines the minimum value among a set of variables.
# The first argument is the variable whose value will be equal to the minimum of the other variables,
# minShift in this case.
# The second argument is the set variables over which the minimum will be taken, (totShifts) in
# this case.
# Recall that the totShifts variable is defined over the set of worker and determines the number of
# shifts that an employed worker will work. The third argument is the name of this constraint.

min_constr = m.addGenConstrMin(minShift, totShifts, name='minShift')

# Constraint:
# Similarly, the addGenConstrMax() method of the model object m adds a new general
# constraint that determines the maximum value among a set of variables.

max_constr = m.addGenConstrMax(maxShift, totShifts, name='maxShift')

```

We have a primary and a secondary objective which both aim to minimize.

```python
# Set global sense for ALL objectives.
# This means that all objectives of the model object m are going to be minimized
m.ModelSense = GRB.MINIMIZE
```

## Objective Function

The primary objective is to minimize the total number of extra workers required to satisfy shift requirements.
$$
\text{Min} \; totSlack \; = \; \sum_{s \: \in \: shifts} Slack(s)
$$

For this objective we allow a degradation tolerance of up to certain percentage. If reltol = 0.2, then we allow a degradation
tolerance of 20%.

```python
# Set up primary objective.

# The setObjectiveN() method of the model object m allows to define multiple objectives.
# The first argument is the linear expression defining the most important objective, called primary
# objective, in this case it is the minimization of extra workers required to satisfy shift
# requirements.
# The second argument is the index of the objective function, we set the index of the primary
# objective to be equal to 0.
# The third argument is the priority of the objective.
# The fourth argument is the relative tolerance to degrade this objective when a lower priority
# objective is optimized. The fifth argument is the name of this objective.
# A hierarchical or lexicographic approach assigns a priority to each objective, and optimizes
# for the objectives in decreasing priority order.
# For this problem, we have two objectives, and the primary objective has the highest priority
# which is equal to 2.
# When the secondary objective is minimized, since the relative tolerance is 0.2, we can only
# increase the minimum number of extra workers up to 20%.
# For example if the minimum number extra workers is 10, then when optimizing the secondary objective
# we can have up to 12 extra workers.

m.setObjectiveN(totSlack, index=0, priority=2, reltol=0.2, name='TotalSlack')
```

The secondary objective is to balance workload among workers. To do so, we minimize the difference between the maximum
number of shifts allocated to a worker, and the minimum number of shifts allocated to a worker. That is
$$
\text{Min} \; (\text{maxShift}  -  \text{minShift})
$$

```python
# Set up secondary objective.

# The secondary objective is called fairness and its goal is to balance the workload assigned
# to the employed workers.
# To balance the workload assigned to the employed workers, we can minimize the difference
# between the maximum number of shifts assigned to an employed worker and the minimum number
# of shifts assigned to an employed worker.

m.setObjectiveN(maxShift - minShift, index=1, priority=1, name='Fairness')
```

```python
# Save model formulation for inspection

m.write('workforce.lp')
```

```python
# Optimize
# This method runs the optimization engine to solve the MIP problem in the model object m
m.optimize()
```

    Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (win64)
    Thread count: 4 physical cores, 8 logical processors, using up to 8 threads
    Optimize a model with 22 rows, 96 columns and 180 nonzeros
    Model fingerprint: 0x8f5c4720
    Model has 2 general constraints
    Variable types: 24 continuous, 72 integer (72 binary)
    Coefficient statistics:
      Matrix range     [1e+00, 1e+00]
      Objective range  [1e+00, 1e+00]
      Bounds range     [1e+00, 1e+00]
      RHS range        [2e+00, 7e+00]

    ---------------------------------------------------------------------------
    Multi-objectives: starting optimization with 2 objectives ...
    ---------------------------------------------------------------------------

    Multi-objectives: applying initial presolve ...
    ---------------------------------------------------------------------------

    Presolve added 10 rows and 22 columns
    Presolve time: 0.00s
    Presolved: 32 rows and 118 columns
    ---------------------------------------------------------------------------

    Multi-objectives: optimize objective 1 (TotalSlack) ...
    ---------------------------------------------------------------------------

    Presolve added 7 rows and 0 columns
    Presolve removed 0 rows and 24 columns
    Presolve time: 0.00s
    Presolved: 39 rows, 94 columns, 184 nonzeros
    Presolved model has 7 SOS constraint(s)
    Variable types: 16 continuous, 78 integer (64 binary)
    Found heuristic solution: objective 11.0000000
    Found heuristic solution: objective 6.0000000

    Root relaxation: cutoff, 0 iterations, 0.00 seconds

    Explored 0 nodes (0 simplex iterations) in 0.03 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 6 11

    Optimal solution found (tolerance 1.00e-04)
    Best objective 6.000000000000e+00, best bound 6.000000000000e+00, gap 0.0000%
    ---------------------------------------------------------------------------

    Multi-objectives: optimize objective 2 (Fairness) ...
    ---------------------------------------------------------------------------

    Loaded user MIP start with objective 10

    Presolve added 7 rows and 0 columns
    Presolve removed 0 rows and 15 columns
    Presolve time: 0.00s
    Presolved: 40 rows, 103 columns, 241 nonzeros
    Presolved model has 7 SOS constraint(s)
    Variable types: 16 continuous, 87 integer (80 binary)

    Root relaxation: objective 3.333333e-01, 52 iterations, 0.00 seconds

        Nodes    |    Current Node    |     Objective Bounds      |     Work
     Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

         0     0    0.33333    0   16   10.00000    0.33333  96.7%     -    0s
    H    0     0                       1.0000000    0.33333  66.7%     -    0s
         0     0     cutoff    0         1.00000    1.00000  0.00%     -    0s

    Cutting planes:
      Gomory: 8
      Implied bound: 1
      RLT: 6
      Relax-and-lift: 1

    Explored 1 nodes (79 simplex iterations) in 0.07 seconds
    Thread count was 8 (of 8 available processors)

    Solution count 2: 1 10

    Optimal solution found (tolerance 1.00e-04)
    Best objective 1.000000000000e+00, best bound 1.000000000000e+00, gap 0.0000%

    ---------------------------------------------------------------------------
    Multi-objectives: solved in 0.08 seconds, solution count 3

```python
# The Status attribute  provides current optimization status of the model object m
# In workforce model, we check if the model is infeasible or unbounded and report this situation
status = m.Status
if status == GRB.Status.INF_OR_UNBD or status == GRB.Status.INFEASIBLE  or status == GRB.Status.UNBOUNDED:
    print('The model cannot be solved because it is infeasible or unbounded')
    sys.exit(0)
# If the optimization status of the model is not optimal for some other reason, we report that
# situation.
if status != GRB.Status.OPTIMAL:
    print('Optimization was stopped with status ' + str(status))
    sys.exit(0)
```

```python
# Print total slack and the number of shifts worked for each worker
# The KPIs for this optimization number is the number of extra worked required to satisfy
# demand and the number of shifts that each employed worker is working.
solution = {}
shifts_sol = {}
solution['Total slack required'] = str(totSlack.X)
assignments_all = {}
gant={}

assignments = dict()
for [w, s] in availability:
    if x[w, s].x == 1:
        if w in assignments:
            assignments[w].append(s)
        else:
            assignments[w] = [s]

print(pd.DataFrame.from_records(list(solution.items()), columns=['KPI', 'Value']))
print('-'*50)

for w in workers:
    shifts_sol[w]=totShifts[w].X
    assignments_all[w]=assignments.get(w, [])

print('Shifts')
print(pd.DataFrame.from_records(list(shifts_sol.items()), columns=['Worker', 'Number of shifts']))

y_pos = np.arange(len(shifts_sol.keys()))
plt.bar(y_pos,shifts_sol.values() , align='center')
plt.xticks(y_pos, shifts_sol.keys())
plt.show()

print('-'*50)
for w in assignments_all:
    gant[w] = [w]
    for d in shifts:
        gant[w].append('*' if d in assignments_all[w] else '-')

print('Assigments')
print('Symbols: \'-\': not working, \'*\': working')
pd.set_option('display.width', 1000)
print(pd.DataFrame.from_records(list(gant.values()), columns=['worker']+shifts))
```

## Analysis

The total number of extra workers required to satisfy the shift requirements during the planning horizon is seven.

The bar chart shows the number of shifts that each employed worker will work during the planning horizon. Observe that with seven extra workers, the workload of employed workers is quite balanced.

The last table shows which day during the planning horizon each employed worker is working.

## Conclusion

In this Jupyter Notebook, we discussed a workforce scheduling optimization problem that is common in the services industry.
Consider a service business that develops its workforce plans for the next two weeks. There is only one shift per workday. Each shift may have different workers requirements on each workday. The service business may hire extra workers from an agency to satisfy shift requirements. The service business wants to minimize the number of extra workers that needs to hire, and as a secondary objective, it wants to balance the workload of employed workers.

The problem was formulated as a multi-objective mixed-integer-programming (MIP) model,
and implemented using the Gurobi Python API and solved using the Gurobi Optimizer.

The hierarchical  approach to tackling multi-objective function problems was explained in detail.

Copyright © 2020 Gurobi Optimization, LLC