## GeostatsPy: Vairogram Modeling for Subsurface Data Analytics in Python

### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### PGE 383 Exercise: Variogram Modeling with GeostatsPy

Here's a simple workflow on detecting the major spatial continuity directions in a spatial dataset with variogram analysis. This information is essential to optimum well placement and prectiction away from wells.  First let's explain the concept of spatial continuity and the variogram.

#### Spatial Continuity

**Spatial Continuity** is the correlation between values over distance.

* No spatial continuity – no correlation between values over distance, random values at each location in space regardless of separation distance.

* Homogenous phenomenon have perfect spatial continuity, since all values as the same (or very similar) they are correlated.

We need a statistic to quantify spatial continuity! A convenient method is the Semivariogram.

#### The Semivariogram

Function of difference over distance.

* The expected (average) squared difference between values separated by a lag distance vector (distance and direction), $h$:

\begin{equation}
\gamma(\bf{h}) = \frac{1}{2 N(\bf{h})} \sum^{N(\bf{h})}_{\alpha=1} (z(\bf{u}_\alpha) - z(\bf{u}_\alpha + \bf{h}))^2
\end{equation}

where $z(\bf{u}_\alpha)$ and $z(\bf{u}_\alpha + \bf{h})$ are the spatial sample values at tail and head locations of the lag vector respectively.

* Calculated over a suite of lag distances to obtain a continuous function.

* the $\frac{1}{2}$ term converts a variogram into a semivariogram, but in practice the term variogram is used instead of semivariogram.
* We prefer the semivariogram because it relates directly to the covariance function, $C_x(\bf{h})$ and univariate variance, $\sigma^2_x$:

\begin{equation}
C_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

Note the correlogram is related to the covariance function as:

\begin{equation}
\rho_x(\bf{h}) = \frac{C_x(\bf{h})}{\sigma^2_x}
\end{equation}

The correlogram provides of function of the $\bf{h}-\bf{h}$ scatter plot correlation vs. lag offset $\bf{h}$.

\begin{equation}
-1.0 \le \rho_x(\bf{h}) \le 1.0
\end{equation}

#### Variogram Observations

The following are common observations for variograms that should assist with their practical use.

##### Observation \#1 - As distance increases, variability increase (in general).

This is common since in general, over greater distance offsets, there is often more difference between the head and tail samples.

In some cases, such as with spatial cyclicity of the hole effect variogram model the variogram may have negative slope over somelag distance intervals

Negative slopes at lag distances greater than half the data extent are often caused by too few pairs for a reliable variogram calculation

##### Observation \#2 - Calculated with over all possible pairs separated by lag vector, $\bf{𝐡}$.

We scan through the entire data set, searching for all possible pair combinations with all other data.  We then calculate the variogram as one half the expectation of squared difference between all pairs.

More pairs results in a more reliable measure.

##### Observation \#3 - Need to plot the sill to know the degree of correlation.

**Sill** is the variance, $\sigma^2_x$

Given stationarity of the variance, $\sigma^2_x$, and variogram $\gamma(\bf{h})$:

we can define the covariance function:

\begin{equation}
C_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

The covariance measure is a measure of similarity over distance (the mirror image of the variogram as shown by the equation above).

Given a standardized distribution $\sigma^2_x = 1.0$, the covariance, $C_x(\bf{h})$, is equal to the correlogram, $\rho_x(\bf{h})$:

\begin{equation}
\rho_x(\bf{h}) = \sigma^2_x - \gamma(\bf{h})
\end{equation}

##### Observation \#4 - The lag distance at which the variogram reaches the sill is know as the range.

At the range, knowing the data value at the tail location provides no information about a value at the head location of the lag distance vector.

##### Observation \#5 - The nugget effect, a discontinuity at the origin

Sometimes there is a discontinuity in the variogram at distances less than the minimum data spacing.  This is known as **nugget effect**.

The ratio of nugget / sill, is known as relative nugget effect (%). Modeled as a discontinuity with no correlation structure that at lags, $h \gt \epsilon$, an infinitesimal lag distance, and perfect correlation at $\bf{h} = 0$.
Caution when including nuggect effect in the variogram model as measurement error, mixing populations cause apparent nugget effect

This exercise demonstrates the semivariogram calculation with GeostatsPy. The steps include:

1. generate a 2D model with sequential Gaussian simulation
2. sample from the simulation
3. calculate and visualize experimental semivariograms

#### Detecting Directions of Continuity

Spatial continuity can be described with nested spatial continuity models:

\begin{equation}
\Gamma_x(\bf{h}) = \sum_{i=1}^{nst} \gamma_i(\bf{h})
\end{equation}

where $\Gamma_x(\bf{h})$ is the nested variogram model resulting from the summation of $nst$ nested variograms  $\gamma_i(\bf{h})$.

Each one of these variogram structures, $\gamma_i(\bf{h})$, is based on a geometric anisotropy model parameterized by the orientation and range in the major and minor directions.  In 2D this is simply an azimuth and ranges, $azi$, $a_{maj}$ and $a_{min}$. Note, the range in the minor direction (orthogonal to the major direction.

The geometric anisotropy model assumes that the range in all off-diagonal directions is based on an ellipse with the major and minor axes alligned with and set to the major and minor for the variogram.

\begin{equation}
\bf{h}_i = \sqrt{\left(\frac{r_{maj}}{a_{maj_i}}\right)^2 + \left(\frac{r_{maj}}{a_{maj_i}}\right)^2}
\end{equation}

Therefore, if we know the major direction, range in major and minor directions, we may completely describe each nested componnent of the complete spatial continuity of the variable of interest, $i = 1,\dots,nst$.

In this workflow we will explore methods to detect directionality from a spatial dataset.

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - sample_data.csv at https://git.io/fh4gm.

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import geostatspy.GSLIB as GSLIB                       # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                 # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import os                                               # to set current working directory
import numpy as np                                      # arrays and matrix math
import pandas as pd                                     # DataFrames
import matplotlib.pyplot as plt                         # plotting
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see above) GSLIB executables in this directory or a location identified in the environmental variable *Path*.

```python
os.chdir("d:/PGE383")                                   # set the working directory
```

#### Loading Tabular Data

Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.

```python
df = pd.read_csv("sample_data_MV_biased.csv")                     # read a .csv file in as a DataFrame
#print(df.iloc[0:5,:])                                  # display first 4 samples in the table as a preview
df.head()                                               # we could also use this command for a table preview
```

      Unnamed: 0
      X
      Y
      Facies
      Porosity
      Perm
      AI

      0
      0
      100.0
      900.0
      0.0
      0.101319
      1.996868
      5590.417154

      1
      1
      100.0
      800.0
      1.0
      0.147676
      10.711789
      3470.845666

      2
      2
      100.0
      700.0
      1.0
      0.145912
      17.818143
      3586.988513

      3
      3
      100.0
      600.0
      1.0
      0.186167
      217.109365
      3732.114787

      4
      4
      100.0
      500.0
      1.0
      0.146088
      16.717367
      2534.551236

We will work with all facies pooled together. I wanted to simplify this workflow and focus more on spatial continuity direction detection. Finally, by not using facies we do have more samples to support our statistical inference. Most often facies are essential in the subsurface model. Don't worry we will check if this is reasonable in a bit.

You are welcome to repeat this workflow on a by-facies basis.  The following code could be used to build DataFrames ('df_sand' and 'df_shale') for each facies.

```p
df_sand = pd.DataFrame.copy(df[df['Facies'] == 1]).reset_index()  # copy only 'Facies' = sand records
df_shale = pd.DataFrame.copy(df[df['Facies'] == 0]).reset_index() # copy only 'Facies' = shale records
```

Let's look at summary statistics for all facies combined:

```python
df.describe().transpose()                          # summary table of sand only DataFrame statistics
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Unnamed: 0
      368.0
      293.260870
      169.058258
      0.000000
      150.500000
      296.000000
      439.500000
      586.000000

      X
      368.0
      499.565217
      289.770794
      0.000000
      240.000000
      500.000000
      762.500000
      990.000000

      Y
      368.0
      520.644022
      277.412187
      9.000000
      269.000000
      539.000000
      769.000000
      999.000000

      Facies
      368.0
      0.597826
      0.491004
      0.000000
      0.000000
      1.000000
      1.000000
      1.000000

      Porosity
      368.0
      0.127026
      0.030642
      0.041122
      0.103412
      0.125842
      0.148623
      0.210258

      Perm
      368.0
      85.617362
      228.362654
      0.094627
      2.297348
      10.377292
      50.581288
      1991.097723

      AI
      368.0
      4791.736646
      974.560569
      1981.177309
      4110.728374
      4713.325533
      5464.043562
      7561.250336

Let's transform the porosity and permeaiblity data to standard normal (mean = 0.0, standard deviation = 1.0, Gaussian shape). This is required for sequential Gaussian simulation (common target for our variogram models) and the Gaussian transform assists with outliers and provides more interpretable variograms.

Let's look at the inputs for the GeostatsPy nscore program.  Note the output include an ndarray with the transformed values (in the same order as the input data in Dataframe 'df' and column 'vcol'), and the transformation table in original values and also in normal score values.

```python
geostats.nscore                                         # see the input parameters required by the nscore function
```

    <function geostatspy.geostats.nscore(df, vcol, wcol=None, ismooth=False, dfsmooth=None, smcol=0, smwcol=0)>

The following command will transform the Porosity and Permeabilty to standard normal.

```python
#Transform to Gaussian by Facies
df['NPor'], tvPor, tnsPor = geostats.nscore(df, 'Porosity') # nscore transform for all facies porosity
df['NPerm'], tvPermSand, tnsPermSand = geostats.nscore(df, 'Perm')  # nscore transform for all facies permeability
```

Let's look at the updated DataFrame to make sure that we now have the normal score porosity and permeability.

```python
df.head()                                               # preview sand DataFrame with nscore transforms
```

      Unnamed: 0
      X
      Y
      Facies
      Porosity
      Perm
      AI
      NPor
      NPerm

      0
      0
      100.0
      900.0
      0.0
      0.101319
      1.996868
      5590.417154
      -0.749088
      -0.767247

      1
      1
      100.0
      800.0
      1.0
      0.147676
      10.711789
      3470.845666
      0.653263
      0.017030

      2
      2
      100.0
      700.0
      1.0
      0.145912
      17.818143
      3586.988513
      0.611663
      0.336607

      3
      3
      100.0
      600.0
      1.0
      0.186167
      217.109365
      3732.114787
      1.993601
      1.211919

      4
      4
      100.0
      500.0
      1.0
      0.146088
      16.717367
      2534.551236
      0.628172
      0.279461

That looks good! One way to check is to see if the relative magnitudes of the normal score transformed values match the original values.  e.g. that the normal score transform of 0.10 porosity normal score is less than the normal score transform of 0.14 porsity.  Also, the normal score transform of values close to the mean value should be close to 0.0

Let's also check the original and transformed sand and shale porosity distributions.

```python
plt.subplot(221)                                        # plot original sand and shale porosity histograms
plt.hist(df['Porosity'], facecolor='red',bins=np.linspace(0.0,0.25,1000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label='Original')
plt.xlim([0.05,0.25]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(222)
plt.hist(df['NPor'], facecolor='blue',bins=np.linspace(-3.0,3.0,1000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label = 'Trans')
plt.xlim([-3.0,3.0]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Nscore Porosity')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(223)                                        # plot nscore transformed sand and shale histograms
plt.hist(df['Perm'], facecolor='red',bins=np.linspace(0.0,1000.0,100000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label='Original')
plt.xlim([0.0,1000.0]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Permeability')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(224)                                        # plot nscore transformed sand and shale histograms
plt.hist(df['NPerm'], facecolor='blue',bins=np.linspace(-3.0,3.0,100000),histtype="stepfilled",alpha=0.2,density=True,cumulative=True,edgecolor='black',label = 'Trans')
plt.xlim([-3.0,3.0]); plt.ylim([0,1.0])
plt.xlabel('Permeability (mD)'); plt.ylabel('Frequency'); plt.title('Nscore Permeability')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=2.2, wspace=0.2, hspace=0.3)
plt.show()
```

The normal score transform has correctly transformed the porosity and permeability to standard normal.

#### Method \#1: Ocular Inspection of Posted Data

Data visualization is very useful to detect patterns. Our brains are very good at pattern detection. I promote quantitative methods and recognize issues with cognitive bias, but it is important to recognize the value is expert intepretation based on data visualization.

* This data visualization will also be important to assist with parameter selection for the quantitative methods later.

Let's plot the location maps of normal score transforms of porosity and permeability for all facies. We will also include a cross plot of the nscore permeability vs. porosity colored by facies to aid with comparison in spatial features between the porosity and permeability data.

```python
cmap = plt.cm.plasma                    # color map
plt.subplot(131)
GSLIB.locmap_st(df,'X','Y','NPor',0,1000,0,1000,-3,3,'Nscore Porosity - All Facies','X (m)','Y (m)','Nscore Porosity',cmap)

plt.subplot(132)
GSLIB.locmap_st(df,'X','Y','NPerm',0,1000,0,1000,-3,3,'Nscore Permeability - All Facies','X (m)','Y (m)','Nscore Permeability',cmap)

plt.subplot(133)
facies = df['Facies'].values +0.01
plt.scatter(df['NPor'],df['NPerm'],c = facies,edgecolor = 'black',cmap = plt.cm.inferno)
#plt.plot([-3,3],[-3,3],color = 'black')
plt.xlabel(r'Nscore Porosity')
plt.ylabel(r'Nscore Permeability')
plt.title('Nscore Permeability vs. Porosity')
plt.xlim([-3,3])
plt.ylim([-3,3])
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=0.8, wspace=0.5, hspace=0.3)
plt.show()
```

What do you see?  Here's my observations:

* there is a high degree of spatial agreement between porosity and permeability, this is supported by the high correlation evident in the cross plot.
* there are no discontinuities that could suggest that facies represent a distinct change, rather the porosity and permeability seem continuous and the assigned facies are a truncation of their continous behavoir, we doing 'ok' with no facies
* suspect a 045 azimuth major direction of continuity (up - right)
* there may be cycles in the 135 azimuth
* there will not likely be a nugget effect, but there is an hint of some short scale discontinuity?

**Do you agree?** If you have a different observations, drop me a line at mpyrcz@austin.utexas.edu and I'll add to this lesson with credit.

#### Method \#2: Variogram Maps

Let's try out variogram maps.

* I realized that I had not coded variogram maps in Python, so I just did it and added it to GeostatsPy.  I have added if here temporarily to support my students completely their assignments. I'm teaching this week at a company and it is safer not to update the package until I have time to test it.

The inputs include:

* **input data** - DataFrame, 'df', and columns for x, y and property of interest, 'x', 'y' and 'vcol',
* **variogram map parameters** - number of cells in each direction to search, 'nxlag', nylag', the cell size / lag distance, 'dxlag' and 'dylag'''
* **search** - the minimum number of pairs reuqired to assign a result, 'mnpairs'
* **normalization** - 1 for standardize variance to 1.0 and 0 for not

The output is a 2D ndarray with the variogram map and the number of pairs.

```python
vmap, npmap = geostats.varmapv(df,'X','Y','NPor',tmin=-999,tmax=999,nxlag=11,nylag=11,dxlag=50,dylag=50,minnp=1,isill=1)

plt.subplot(121)
GSLIB.pixelplt_st(vmap,-575,575,-575,575,50.0,0,1.6,'Nscore Porosity Variogram Map','X(m)','Y(m)','Nscore Variogram',cmap)

plt.subplot(122)
GSLIB.pixelplt_st(npmap,-575,575,-575,575,50.0,0,900,'Nscore Porosity Variogram Map Number of Pairs','X(m)','Y(m)','Number of Pairs',cmap)

plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=1.2, wspace=0.2, hspace=0.2)
plt.show()

print('The shape of the output is ' + str(vmap.shape))
```

Notice that the output ndarray is 23, 23 cells? We asked for the number of cells to extend in each direction, 11 and 11 in x and y.  The map has an origin (zero distance) cell in the middle and extends 11 in both positive and negative directions.  So we have 2*$nx$ + 1, 2*$ny$ + 1 cells in the resulting variogram map and the $xmin = -1 * (nx * x_{cellsize} + \frac{1}{2} x_{cellsize})$ and the $xmax = nx * x_{cellsize} + \frac{1}{2} x_{cellsize}$

What do you think of this variogram map? These are my observations:

* major continuity direction is at azimuth 045
* there is a high degree of geometric anisotropy
* there is cyclicity in the 135 direction
* there may be some cyclicity in the 045 direction

From this variogram map we can immediately see directionality in our spatial data.

#### Method \#3: Experimental Variograms

Another method for exploring spatial data directionality is the calculation of multiple experimental variograms for a variety of directions.

We can use the location maps to help determine good variogram calculation parameters.

```p
tmin = -9999.; tmax = 9999.;
lag_dist = 100.0; lag_tol = 50.0; nlag = 7; bandh = 9999.9; azi = azi; atol = 22.5; isill = 1
```
* **tmin**, **tmax** are trimming limits - set to have no impact, no need to filter the data
* **lag_dist**, **lag_tol** are the lag distance, lag tolerance - set based on the common data spacing (100m) and tolerance as 100% of lag distance for additonal smoothing
* **nlag** is number of lags - set to extend just past 50 of the data extent
* **bandh** is the horizontal band width - set to have no effect
* **azi** is the azimuth -  it has not effect since we set atol, the azimuth tolerance, to 90.0
* **isill** is a boolean to standardize the distribution to a variance of 1 - it has no effect since the nscore transform sets the variance to 1.0

```python
tmin = -9999.; tmax = 9999.                             # no trimming
lag_dist = 100.0; lag_tol = 100.0; nlag = 7;            # maximum lag is 700m and tolerance > 1/2 lag distance for smoothing
bandh = 9999.9; atol = 22.5                             # no bandwidth, directional variograms
isill = 1                                               # standardize sill
azi_mat = [0,22.5,45,67.5,90,112.5,135,157.5]           # directions in azimuth to consider
```

Let's try running these variograms and visualizing them on separate plots.  I'll demonstrate a method to promgramtically loop over each direction for efficiency (and code brevity).

* we have the direction in the list called 'azi_mat'
* we use the command:
```p
for iazi in range(0,len(azi_mat)):
```
to loop over all the elements in the list with index 'iazi'

* we run the variogram calculation and store the reuslts in 2D arrays, iy is direction, ix is the lag
* we use subplots with the 'iazi' index to add each plot

```p
    plt.subplot(4,2,iazi+1)
```
we add one because the plot index must be $1,\ldots,n$, but arrays / list index as $0,\ldots,n-1$ in Python.

```python
# Arrays to store the results
lag = np.zeros((len(azi_mat),nlag+2)); gamma = np.zeros((len(azi_mat),nlag+2)); npp = np.zeros((len(azi_mat),nlag+2));

for iazi in range(0,len(azi_mat)):                      # Loop over all directions
    lag[iazi,:], gamma[iazi,:], npp[iazi,:] = geostats.gamv(df,"X","Y","NPor",tmin,tmax,lag_dist,lag_tol,nlag,azi_mat[iazi],atol,bandh,isill)
    plt.subplot(4,2,iazi+1)
    plt.plot(lag[iazi,:],gamma[iazi,:],'x',color = 'black',label = 'Azimuth ' +str(azi_mat[iazi]))
    plt.plot([0,2000],[1.0,1.0],color = 'black')
    plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
    plt.ylabel(r'$\gamma \bf(h)$')
    plt.title('Directional NSCORE Porosity Variogram')
    plt.xlim([0,700])
    plt.ylim([0,1.8])
    plt.legend(loc='upper left')
    plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=4.2, wspace=0.2, hspace=0.3)
plt.show()
```

The directional variograms provide a very clear image of directionality. The ranges vary from 300m, 500m to zonal anisotropy, to 500m, 350m, 280m, 250m and finally 280m.  We are observing the actually spatial continuity ellipse by exploring a variety of directions!

* We can observe that Azimuth 045 is the major direction and Azimuth 135 is the minor direction.

This is a very powerful tool for exploring directionality in spatial datasets.

#### Variogram Modeling

From above we can identify 045 as the major direction and 135 as the minor direction. We must build a single 2D nested variogram model based on the experiential variograms in each 045 and 135 directions.  First, let's look at these two experimental variograms again.

```python
plt.subplot(1,2,1)
plt.plot(lag[2,:],gamma[2,:],'x',color = 'black',label = 'Azimuth ' +str(azi_mat[2]))
plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional NSCORE Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')

plt.subplot(1,2,2)
plt.plot(lag[6,:],gamma[6,:],'x',color = 'black',label = 'Azimuth ' +str(azi_mat[6]))
plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional NSCORE Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.2, top=1.6, wspace=0.2, hspace=0.3)
plt.show()
```

We will assume these models will be applied in sequential Gaussian simulation; therefore, we must model to the sill (or the global distribution will not be reproduced over the realizations).  We will also not use cyclicity for now, as we are just getting started.

* Let's build a reasonable model to the sill.

##### make_variogram function

We use the make_variogram function to make a variogram model

* a dictionary for compact storage of the variogram model parameters to pass into plotting (below), kriging and simulation methods

The variogram model parameter include:

* **nug** - nugget effect contribution to sill
* **nst** - number of nested structures (1 or 2)
* **it** - type for this nested structure (1 - spherical, 2 - exponential, 3 - Gaussian)
* **cc** - contribution of each nested structure (contributions + nugget must sum to the sill)
* **azi** - the azimuth for this nested structure of the major direction, the minor is orthogonal
* **hmaj** - the range for this nested structure in the major direction
* **hmin** - the range for this nested structure in the minor direction

We increment it, cc, azi, hmaj, and hmin for the 1st and 2nd structures

* for only 1 structure plus optional nugget, omit the 2nd structure parameters and they will default to $cc2 = 0$, no contribution to the model

Here's my model:

```p
nug = 0.0; nst = 2
it1 = 1; cc1 = 0.6; azi1 = 45; hmaj1 = 350; hmin1 = 350                # first structure
it2 = 1; cc2 = 0.4; azi2 = 45; hmaj2 = 9999.9; hmin2 = 400             # second structure
```

Some comments on our model:

* we model to the sill of 1.0, since we applied the normal score transform ($nug + cc1 + cc2 = 1.0$)

* we used 2 spherical structures to capture zontal anisotropy in the 045 azimuth

* since the experimental variogram exceeds the sill with trend or cyclicity we could have attempted trend modeling and then worked with the residual, but we will not do this for workflow brevity and simplicity

We input these model parameters to make a variogram model dictionary with the make_variogram function as follows:

```p
vario = GSLIB.make_variogram(nug,nst,it1,cc1,azi1,hmaj1,hmin1,it2,cc2,azi2,hmaj2,hmin2)
```

##### vmodel function

To plot the variogram we use the vmodel function to project the model in the major and minor directions

The inputs for vmodel are:

* **nlag** - the number of points along the variogram to calculate for the projection

* **xlag** - the size of a lag for the projection

* **azm** - the direction of the projection in azimuth (this is all we need since we are working in 2D)

* **vario** - the variogram model dictionary from the make_variogram function (above)

Note: this function is just for visualization by projecting the variogram model in a direction, so the convention is to use a very small **xlag** and large **nlag** for a high resolution display of the variogram model

The outputs from the vmodel program include:

* **index** - the lag number for the projection

* **lag distance** - the distance offset along the projection (the **h** in the variogram plot)

* **variogram** - the variogram value at the lag distance for the projection (the $\gamma$(**h**) in the variogram plot)

* **covariance function** - the covariance function at the lag distance for the projection (for the C(**h**) plot)

* **correlogram** - the correlogram at the lag distance for the projection (for the $\rho$(**h**) plot)

We have 2 structures and no nugget effect.  We needed the 2nd structure to capture the zonal anisotropy in the 045 direction.  Let's calculate the variogram model in these directions and plot them with the experimental variograms.

```python
nug = 0.0; nst = 2                                             # 2 nest structure variogram model parameters
it1 = 1; cc1 = 0.6; azi1 = 45; hmaj1 = 350; hmin1 = 350
it2 = 1; cc2 = 0.4; azi2 = 45; hmaj2 = 9999.9; hmin2 = 400

vario = GSLIB.make_variogram(nug,nst,it1,cc1,azi1,hmaj1,hmin1,it2,cc2,azi2,hmaj2,hmin2) # make model object
nlag = 70; xlag = 10; azm = 45;                                # project the model in the 045 azimuth
index45,h45,gam45,cov45,ro45 = geostats.vmodel(nlag,xlag,azm,vario)
azm = 135                                                      # project the model in the 135 azimuth
index135,h135,gam135,cov135,ro135 = geostats.vmodel(nlag,xlag,azm,vario)

plt.subplot(1,2,1)
plt.plot(lag[2,:],gamma[2,:],'x',color = 'black',label = 'Azimuth ' +str(azi_mat[2]))
plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.plot(h45,gam45,color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional NSCORE Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')

plt.subplot(1,2,2)
plt.plot(lag[6,:],gamma[6,:],'x',color = 'black',label = 'Azimuth ' +str(azi_mat[6]))
plt.plot([0,2000],[1.0,1.0],color = 'black')
plt.plot(h135,gam135,color = 'black')
plt.xlabel(r'Lag Distance $\bf(h)$, (m)')
plt.ylabel(r'$\gamma \bf(h)$')
plt.title('Directional NSCORE Porosity Variogram')
plt.xlim([0,700])
plt.ylim([0,1.8])
plt.legend(loc='upper left')

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.2, top=1.6, wspace=0.2, hspace=0.3)
plt.show()
```

#### Comments

This was a basic demonstration of vairogram modeling for spatial continuity analysis. Much more could be done, I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


```python
import sympy

x, y, z = sympy.symbols("x y z")
c11, c22, c33, c44, c55, c66 = sympy.symbols("c11 c22 c33 c44 c55 c66")
c12, c13, c16, c22, c23, c26, c36, c45 = sympy.symbols("c12 c13 c16 c22 c23 c26 c36 c45")
s1, s2, s3, p = sympy.symbols("s1 s2 s3 p")
y11, y12, y13, y22, y23, y33, y31, y32, y21 = sympy.symbols("y11 y12 y13 y22 y23 y33, y31, y32, y21")

y11 = c11*s1**2 + c66*s2**2 + c55*s3**2 + 2*c16*s1*s2
y22 = c66*s1**2 + c22*s2**2 + c44*s3**2 + 2*c26*s1*s2
y33 = c55*s1**2 + c44*s2**2 + c33*s3**2 + 2*c45*s1*s2
y12 = c16*s1**2 + c26*s2**2 + c45*s3**2 + (c12+c66)*s1*s2
y21 = y12
y13 = (c13+c55)*s1*s3 + (c36+c45)*s2*s3
y31 = y13
y23 = (c36+c45)*s1*s3 + (c23+c44)*s2*s3
y32 = y23

A = sympy.Matrix([[y11, y12, y13], [y21, y22, y23], [y31, y32, y33]])
I = sympy.eye(3)

B = A - p*I

S = sympy.collect(sympy.simplify(B.det()), s3)
print(S)
print('Done Solve')
```

    c11*c22*c44*s1**2*s2**4 + 2*c11*c22*c45*s1**3*s2**3 + c11*c22*c55*s1**4*s2**2 - c11*c22*p*s1**2*s2**2 + 2*c11*c26*c44*s1**3*s2**3 + 4*c11*c26*c45*s1**4*s2**2 + 2*c11*c26*c55*s1**5*s2 - 2*c11*c26*p*s1**3*s2 + c11*c44*c66*s1**4*s2**2 - c11*c44*p*s1**2*s2**2 + 2*c11*c45*c66*s1**5*s2 - 2*c11*c45*p*s1**3*s2 + c11*c55*c66*s1**6 - c11*c55*p*s1**4 - c11*c66*p*s1**4 + c11*p**2*s1**2 - c12**2*c44*s1**2*s2**4 - 2*c12**2*c45*s1**3*s2**3 - c12**2*c55*s1**4*s2**2 + c12**2*p*s1**2*s2**2 - 2*c12*c16*c44*s1**3*s2**3 - 4*c12*c16*c45*s1**4*s2**2 - 2*c12*c16*c55*s1**5*s2 + 2*c12*c16*p*s1**3*s2 - 2*c12*c26*c44*s1*s2**5 - 4*c12*c26*c45*s1**2*s2**4 - 2*c12*c26*c55*s1**3*s2**3 + 2*c12*c26*p*s1*s2**3 - 2*c12*c44*c66*s1**2*s2**4 - 4*c12*c45*c66*s1**3*s2**3 - 2*c12*c55*c66*s1**4*s2**2 + 2*c12*c66*p*s1**2*s2**2 - c16**2*c44*s1**4*s2**2 - 2*c16**2*c45*s1**5*s2 - c16**2*c55*s1**6 + c16**2*p*s1**4 + 2*c16*c22*c44*s1*s2**5 + 4*c16*c22*c45*s1**2*s2**4 + 2*c16*c22*c55*s1**3*s2**3 - 2*c16*c22*p*s1*s2**3 + 2*c16*c26*c44*s1**2*s2**4 + 4*c16*c26*c45*s1**3*s2**3 + 2*c16*c26*c55*s1**4*s2**2 - 2*c16*c26*p*s1**2*s2**2 - 2*c16*c44*p*s1*s2**3 - 4*c16*c45*p*s1**2*s2**2 - 2*c16*c55*p*s1**3*s2 + 2*c16*p**2*s1*s2 + c22*c44*c66*s2**6 - c22*c44*p*s2**4 + 2*c22*c45*c66*s1*s2**5 - 2*c22*c45*p*s1*s2**3 + c22*c55*c66*s1**2*s2**4 - c22*c55*p*s1**2*s2**2 - c22*c66*p*s2**4 + c22*p**2*s2**2 - c26**2*c44*s2**6 - 2*c26**2*c45*s1*s2**5 - c26**2*c55*s1**2*s2**4 + c26**2*p*s2**4 - 2*c26*c44*p*s1*s2**3 - 4*c26*c45*p*s1**2*s2**2 - 2*c26*c55*p*s1**3*s2 + 2*c26*p**2*s1*s2 - c44*c66*p*s1**2*s2**2 - c44*c66*p*s2**4 + c44*p**2*s2**2 - 2*c45*c66*p*s1**3*s2 - 2*c45*c66*p*s1*s2**3 + 2*c45*p**2*s1*s2 - c55*c66*p*s1**4 - c55*c66*p*s1**2*s2**2 + c55*p**2*s1**2 + c66*p**2*s1**2 + c66*p**2*s2**2 - p**3 + s3**6*(c33*c44*c55 - c33*c45**2) + s3**4*(c11*c33*c44*s1**2 - 2*c12*c33*c45*s1*s2 - c13**2*c44*s1**2 + 2*c13*c23*c45*s1*s2 - 2*c13*c36*c44*s1*s2 + 2*c13*c36*c45*s1**2 - 2*c13*c44*c55*s1**2 + 2*c13*c45**2*s1**2 + 2*c16*c33*c44*s1*s2 - 2*c16*c33*c45*s1**2 + c22*c33*c55*s2**2 - c23**2*c55*s2**2 + 2*c23*c36*c45*s2**2 - 2*c23*c36*c55*s1*s2 - 2*c23*c44*c55*s2**2 + 2*c23*c45**2*s2**2 - 2*c26*c33*c45*s2**2 + 2*c26*c33*c55*s1*s2 + c33*c44*c66*s2**2 - c33*c44*p - 2*c33*c45*c66*s1*s2 + c33*c55*c66*s1**2 - c33*c55*p - c36**2*c44*s2**2 + 2*c36**2*c45*s1*s2 - c36**2*c55*s1**2 - 4*c36*c44*c55*s1*s2 + 4*c36*c45**2*s1*s2 - c44*c55*p + c45**2*p) + s3**2*(c11*c22*c33*s1**2*s2**2 - c11*c23**2*s1**2*s2**2 - 2*c11*c23*c36*s1**3*s2 - 2*c11*c23*c44*s1**2*s2**2 - 2*c11*c23*c45*s1**3*s2 + 2*c11*c26*c33*s1**3*s2 + c11*c33*c66*s1**4 - c11*c33*p*s1**2 - c11*c36**2*s1**4 - 2*c11*c36*c44*s1**3*s2 - 2*c11*c36*c45*s1**4 + c11*c44*c55*s1**4 - c11*c44*p*s1**2 - c11*c45**2*s1**4 - c12**2*c33*s1**2*s2**2 + 2*c12*c13*c23*s1**2*s2**2 + 2*c12*c13*c36*s1**3*s2 + 2*c12*c13*c44*s1**2*s2**2 + 2*c12*c13*c45*s1**3*s2 - 2*c12*c16*c33*s1**3*s2 + 2*c12*c23*c36*s1*s2**3 + 2*c12*c23*c45*s1*s2**3 + 2*c12*c23*c55*s1**2*s2**2 - 2*c12*c26*c33*s1*s2**3 - 2*c12*c33*c66*s1**2*s2**2 + 2*c12*c36**2*s1**2*s2**2 + 2*c12*c36*c44*s1*s2**3 + 4*c12*c36*c45*s1**2*s2**2 + 2*c12*c36*c55*s1**3*s2 + 2*c12*c44*c55*s1**2*s2**2 - 2*c12*c45**2*s1**2*s2**2 + 2*c12*c45*p*s1*s2 - c13**2*c22*s1**2*s2**2 - 2*c13**2*c26*s1**3*s2 - c13**2*c66*s1**4 + c13**2*p*s1**2 + 2*c13*c16*c23*s1**3*s2 + 2*c13*c16*c36*s1**4 + 2*c13*c16*c44*s1**3*s2 + 2*c13*c16*c45*s1**4 - 2*c13*c22*c36*s1*s2**3 - 2*c13*c22*c45*s1*s2**3 - 2*c13*c22*c55*s1**2*s2**2 + 2*c13*c23*c26*s1*s2**3 + 2*c13*c23*c66*s1**2*s2**2 - 2*c13*c26*c36*s1**2*s2**2 + 2*c13*c26*c44*s1*s2**3 - 2*c13*c26*c45*s1**2*s2**2 - 4*c13*c26*c55*s1**3*s2 + 2*c13*c36*p*s1*s2 + 2*c13*c44*c66*s1**2*s2**2 + 2*c13*c45*p*s1*s2 - 2*c13*c55*c66*s1**4 + 2*c13*c55*p*s1**2 - c16**2*c33*s1**4 + 2*c16*c22*c33*s1*s2**3 - 2*c16*c23**2*s1*s2**3 - 2*c16*c23*c36*s1**2*s2**2 - 4*c16*c23*c44*s1*s2**3 - 2*c16*c23*c45*s1**2*s2**2 + 2*c16*c23*c55*s1**3*s2 + 2*c16*c26*c33*s1**2*s2**2 - 2*c16*c33*p*s1*s2 - 2*c16*c36*c44*s1**2*s2**2 + 2*c16*c36*c55*s1**4 + 4*c16*c44*c55*s1**3*s2 - 2*c16*c44*p*s1*s2 - 4*c16*c45**2*s1**3*s2 + 2*c16*c45*p*s1**2 + c22*c33*c66*s2**4 - c22*c33*p*s2**2 - c22*c36**2*s2**4 - 2*c22*c36*c45*s2**4 - 2*c22*c36*c55*s1*s2**3 + c22*c44*c55*s2**4 - c22*c45**2*s2**4 - c22*c55*p*s2**2 - c23**2*c66*s2**4 + c23**2*p*s2**2 + 2*c23*c26*c36*s2**4 + 2*c23*c26*c45*s2**4 + 2*c23*c26*c55*s1*s2**3 + 2*c23*c36*p*s1*s2 - 2*c23*c44*c66*s2**4 + 2*c23*c44*p*s2**2 + 2*c23*c45*p*s1*s2 + 2*c23*c55*c66*s1**2*s2**2 - c26**2*c33*s2**4 - 2*c26*c33*p*s1*s2 + 2*c26*c36*c44*s2**4 - 2*c26*c36*c55*s1**2*s2**2 + 4*c26*c44*c55*s1*s2**3 - 4*c26*c45**2*s1*s2**3 + 2*c26*c45*p*s2**2 - 2*c26*c55*p*s1*s2 - c33*c66*p*s1**2 - c33*c66*p*s2**2 + c33*p**2 + c36**2*p*s1**2 + c36**2*p*s2**2 + 2*c36*c44*p*s1*s2 + 2*c36*c45*p*s1**2 + 2*c36*c45*p*s2**2 + 2*c36*c55*p*s1*s2 + 4*c44*c55*c66*s1**2*s2**2 - c44*c55*p*s1**2 - c44*c55*p*s2**2 - c44*c66*p*s2**2 + c44*p**2 - 4*c45**2*c66*s1**2*s2**2 + c45**2*p*s1**2 + c45**2*p*s2**2 + 2*c45*c66*p*s1*s2 - c55*c66*p*s1**2 + c55*p**2)
    Done Solve

```python

c11*c22*c44*s1**2*s2**4 + 2*c11*c22*c45*s1**3*s2**3 + c11*c22*c55*s1**4*s2**2 -
c11*c22*p*s1**2*s2**2 + 2*c11*c26*c44*s1**3*s2**3 + 4*c11*c26*c45*s1**4*s2**2 +
2*c11*c26*c55*s1**5*s2 - 2*c11*c26*p*s1**3*s2 + c11*c44*c66*s1**4*s2**2 -
c11*c44*p*s1**2*s2**2 + 2*c11*c45*c66*s1**5*s2 - 2*c11*c45*p*s1**3*s2 +
c11*c55*c66*s1**6 - c11*c55*p*s1**4 - c11*c66*p*s1**4 +
c11*p**2*s1**2 - c12**2*c44*s1**2*s2**4 - 2*c12**2*c45*s1**3*s2**3 -
c12**2*c55*s1**4*s2**2 + c12**2*p*s1**2*s2**2 - 2*c12*c16*c44*s1**3*s2**3 -
4*c12*c16*c45*s1**4*s2**2 - 2*c12*c16*c55*s1**5*s2 + 2*c12*c16*p*s1**3*s2 -
2*c12*c26*c44*s1*s2**5 - 4*c12*c26*c45*s1**2*s2**4 - 2*c12*c26*c55*s1**3*s2**3 +
2*c12*c26*p*s1*s2**3 - 2*c12*c44*c66*s1**2*s2**4 - 4*c12*c45*c66*s1**3*s2**3 -
2*c12*c55*c66*s1**4*s2**2 + 2*c12*c66*p*s1**2*s2**2 - c16**2*c44*s1**4*s2**2 -
2*c16**2*c45*s1**5*s2 - c16**2*c55*s1**6 + c16**2*p*s1**4 +
2*c16*c22*c44*s1*s2**5 + 4*c16*c22*c45*s1**2*s2**4 + 2*c16*c22*c55*s1**3*s2**3 -
2*c16*c22*p*s1*s2**3 + 2*c16*c26*c44*s1**2*s2**4 + 4*c16*c26*c45*s1**3*s2**3 +
2*c16*c26*c55*s1**4*s2**2 - 2*c16*c26*p*s1**2*s2**2 - 2*c16*c44*p*s1*s2**3 -
4*c16*c45*p*s1**2*s2**2 - 2*c16*c55*p*s1**3*s2 + 2*c16*p**2*s1*s2 +
c22*c44*c66*s2**6 - c22*c44*p*s2**4 + 2*c22*c45*c66*s1*s2**5 -
2*c22*c45*p*s1*s2**3 + c22*c55*c66*s1**2*s2**4 - c22*c55*p*s1**2*s2**2 -
c22*c66*p*s2**4 + c22*p**2*s2**2 - c26**2*c44*s2**6 -
2*c26**2*c45*s1*s2**5 - c26**2*c55*s1**2*s2**4 + c26**2*p*s2**4 -
2*c26*c44*p*s1*s2**3 - 4*c26*c45*p*s1**2*s2**2 - 2*c26*c55*p*s1**3*s2 +
2*c26*p**2*s1*s2 - c44*c66*p*s1**2*s2**2 - c44*c66*p*s2**4 +
c44*p**2*s2**2 - 2*c45*c66*p*s1**3*s2 - 2*c45*c66*p*s1*s2**3 +
2*c45*p**2*s1*s2 - c55*c66*p*s1**4 - c55*c66*p*s1**2*s2**2 +
c55*p**2*s1**2 + c66*p**2*s1**2 + c66*p**2*s2**2 - p**3 +

s3**6*(c33*c44*c55 - c33*c45**2) +

s3**4*(
c11*c33*c44*s1**2 - 2*c12*c33*c45*s1*s2 - c13**2*c44*s1**2 +
    2*c13*c23*c45*s1*s2 - 2*c13*c36*c44*s1*s2 + 2*c13*c36*c45*s1**2 -
    2*c13*c44*c55*s1**2 + 2*c13*c45**2*s1**2 + 2*c16*c33*c44*s1*s2 -
    2*c16*c33*c45*s1**2 + c22*c33*c55*s2**2 - c23**2*c55*s2**2 +
    2*c23*c36*c45*s2**2 - 2*c23*c36*c55*s1*s2 - 2*c23*c44*c55*s2**2 +
    2*c23*c45**2*s2**2 - 2*c26*c33*c45*s2**2 + 2*c26*c33*c55*s1*s2 +
    c33*c44*c66*s2**2 - c33*c44*p - 2*c33*c45*c66*s1*s2 +
    c33*c55*c66*s1**2 - c33*c55*p - c36**2*c44*s2**2 +
    2*c36**2*c45*s1*s2 - c36**2*c55*s1**2 - 4*c36*c44*c55*s1*s2 +
    4*c36*c45**2*s1*s2 - c44*c55*p + c45**2*p) +

s3**2*(c11*c22*c33*s1**2*s2**2 - c11*c23**2*s1**2*s2**2 - 2*c11*c23*c36*s1**3*s2 -
       2*c11*c23*c44*s1**2*s2**2 - 2*c11*c23*c45*s1**3*s2 + 2*c11*c26*c33*s1**3*s2 +
       c11*c33*c66*s1**4 - c11*c33*p*s1**2 - c11*c36**2*s1**4 -
       2*c11*c36*c44*s1**3*s2 - 2*c11*c36*c45*s1**4 + c11*c44*c55*s1**4 -
       c11*c44*p*s1**2 - c11*c45**2*s1**4 - c12**2*c33*s1**2*s2**2 +
       2*c12*c13*c23*s1**2*s2**2 + 2*c12*c13*c36*s1**3*s2 + 2*c12*c13*c44*s1**2*s2**2 +
       2*c12*c13*c45*s1**3*s2 - 2*c12*c16*c33*s1**3*s2 + 2*c12*c23*c36*s1*s2**3 +
       2*c12*c23*c45*s1*s2**3 + 2*c12*c23*c55*s1**2*s2**2 - 2*c12*c26*c33*s1*s2**3 -
       2*c12*c33*c66*s1**2*s2**2 + 2*c12*c36**2*s1**2*s2**2 + 2*c12*c36*c44*s1*s2**3 +
       4*c12*c36*c45*s1**2*s2**2 + 2*c12*c36*c55*s1**3*s2 + 2*c12*c44*c55*s1**2*s2**2 -
       2*c12*c45**2*s1**2*s2**2 + 2*c12*c45*p*s1*s2 - c13**2*c22*s1**2*s2**2 -
       2*c13**2*c26*s1**3*s2 - c13**2*c66*s1**4 + c13**2*p*s1**2 +
       2*c13*c16*c23*s1**3*s2 + 2*c13*c16*c36*s1**4 + 2*c13*c16*c44*s1**3*s2 +
       2*c13*c16*c45*s1**4 - 2*c13*c22*c36*s1*s2**3 - 2*c13*c22*c45*s1*s2**3 -
       2*c13*c22*c55*s1**2*s2**2 + 2*c13*c23*c26*s1*s2**3 + 2*c13*c23*c66*s1**2*s2**2 -
       2*c13*c26*c36*s1**2*s2**2 + 2*c13*c26*c44*s1*s2**3 - 2*c13*c26*c45*s1**2*s2**2 -
       4*c13*c26*c55*s1**3*s2 + 2*c13*c36*p*s1*s2 + 2*c13*c44*c66*s1**2*s2**2 +
       2*c13*c45*p*s1*s2 - 2*c13*c55*c66*s1**4 + 2*c13*c55*p*s1**2 -
       c16**2*c33*s1**4 + 2*c16*c22*c33*s1*s2**3 - 2*c16*c23**2*s1*s2**3 -
       2*c16*c23*c36*s1**2*s2**2 - 4*c16*c23*c44*s1*s2**3 - 2*c16*c23*c45*s1**2*s2**2 +
       2*c16*c23*c55*s1**3*s2 + 2*c16*c26*c33*s1**2*s2**2 - 2*c16*c33*p*s1*s2 -
       2*c16*c36*c44*s1**2*s2**2 + 2*c16*c36*c55*s1**4 + 4*c16*c44*c55*s1**3*s2 -
       2*c16*c44*p*s1*s2 - 4*c16*c45**2*s1**3*s2 + 2*c16*c45*p*s1**2 +
       c22*c33*c66*s2**4 - c22*c33*p*s2**2 - c22*c36**2*s2**4 -
       2*c22*c36*c45*s2**4 - 2*c22*c36*c55*s1*s2**3 + c22*c44*c55*s2**4 -
       c22*c45**2*s2**4 - c22*c55*p*s2**2 - c23**2*c66*s2**4 +
       c23**2*p*s2**2 + 2*c23*c26*c36*s2**4 + 2*c23*c26*c45*s2**4 +
       2*c23*c26*c55*s1*s2**3 + 2*c23*c36*p*s1*s2 - 2*c23*c44*c66*s2**4 +
       2*c23*c44*p*s2**2 + 2*c23*c45*p*s1*s2 + 2*c23*c55*c66*s1**2*s2**2 -
       c26**2*c33*s2**4 - 2*c26*c33*p*s1*s2 + 2*c26*c36*c44*s2**4 -
       2*c26*c36*c55*s1**2*s2**2 + 4*c26*c44*c55*s1*s2**3 - 4*c26*c45**2*s1*s2**3 +
       2*c26*c45*p*s2**2 - 2*c26*c55*p*s1*s2 - c33*c66*p*s1**2 -
       c33*c66*p*s2**2 + c33*p**2 + c36**2*p*s1**2 +
       c36**2*p*s2**2 + 2*c36*c44*p*s1*s2 + 2*c36*c45*p*s1**2 +
       2*c36*c45*p*s2**2 + 2*c36*c55*p*s1*s2 + 4*c44*c55*c66*s1**2*s2**2 -
       c44*c55*p*s1**2 - c44*c55*p*s2**2 - c44*c66*p*s2**2 +
       c44*p**2 - 4*c45**2*c66*s1**2*s2**2 + c45**2*p*s1**2 +
       c45**2*p*s2**2 + 2*c45*c66*p*s1*s2 - c55*c66*p*s1**2 + c55*p**2)
```


# Métodos avanzados para EDOs

## Métodos tipo Runge-Kutta

En la clase pasada, vimos el ejemplo del **método (explícito) del punto medio** y del **método de Heun** en los que se utilizaba una **variable auxiliar** para hacer más sencilla la notación.

Sin embargo, esa variable auxiliar también puede interpretarse como un **punto auxiliar** que utilizamos para evaluar la función $\mathbf{f}$. Es decir, ya no solo utilizamos las variables $\mathbf{x}(t),t$ y $\mathbf{f}(\mathbf{x}(t),t)$ para calcular $\mathbf{x}(t+h)$, si no hacemos combinaciones de ellas o utilizamos tiempos intermedios $t^* \in [t,t+h]$, aproximaciones de $\mathbf{x}(t^*)$ y combinaciones lineales de estas para obtener $\mathbf{x}(t+h)$. A todos los métodos que utilizan uno o más puntos auxiliares para después calcular $\mathbf{x}(t+h)$ se les llama **métodos tipo Runge-Kutta**.

Se dice que el **número de pasos** de un método de Runge-Kutta es el número de cálculos de puntos auxiliares que debemos hacer para obtener $\mathbf{x}(t+h)$ a partir de $\mathbf{x}(t)$ (debemos aclarar que a $\mathbf{x}(t)$ también se le cuenta como un punto auxiliar). Así, es obvio que el método de Euler resulta ser un método de un paso ($\mathbf{x}(t)$), mientras que el de punto medio o el de Heun son métodos de dos pasos ($\mathbf{k}_1,\mathbf{x}(t)$).

Normalmente, entre mayor número de pasos tenga un método de Runge-Kutta, mayor será su precisión. Un método ampliamente usado en el cómputo científico (podría decirse que es el método estándar) es el método de [Runge-Kutta de cuatro pasos](https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods) (denotado comúnmente como  **RK4**) obedece la siguiente regla:

$$
\begin{align}
\mathbf{k}_1 &= \mathbf{f}(\mathbf{x}_{i-1},t_{i-1})\\
\mathbf{k}_2 &= \mathbf{f}(\mathbf{x}_{i-1} + (h/2)\mathbf{k}_1,t_{i-1}+h/2)\\
\mathbf{k}_3 &= \mathbf{f}(\mathbf{x}_{i-1} + (h/2)\mathbf{k}_2,t_{i-1}+h/2)\\
\mathbf{k}_4 &= \mathbf{f}(\mathbf{x}_{i-1} + h\mathbf{k}_3,t_{i-1}+h)\\
 & \\
\mathbf{x}_{i} &= \mathbf{x}_{i-1} + \frac{h}{6}\left( \mathbf{k}_1 + 2 \mathbf{k}_2 + 2 \mathbf{k}_3 + \mathbf{k}_4 \right)
\end{align}
$$

Notemos que el método ya se vuelve mucho más elaborado, por lo que deducirlo es algo que escapa al alcance del curso. Estamos interesados simplemente en utilizarlo.

### Ejercicio 1

Implementa una función `RK4(f,x_0,t_0,t_f,h)` con `f` la función **vectorial** tal que $\dot{\mathbf{x}}(t) = \mathbf{f}(\mathbf{x}(t),t)$, `x_0` (un vector de $n$ entradas) y `t_0` la condición inicial, `t_f` el tiempo máximo al que queremos obtener la solución y `h` el tamaño de paso. La función debe de implementar el método de Runge Kutta de orden 4 y debe de regresar el arreglo de los tiempos `[t_0,t_1, t_2,....,t_n]` con $t_{n} \leq t_f < t_{n} + h$ y el de la solución a la ecuación diferencial `[x_0,x_1,...,x_n]`.

### Ejercicio 2

Utiliza la función `RK4` para resolver el sistema del péndulo simple y del oscilador armónico simple de la clase 16. ¿La solución obtenida es mejor que la que se obtiene con los otros métodos ya vistos?

## Sistema para ejemplificar: Péndulo doble

Tenemos un péndulo de longitud $l_1$ con un objeto de masa $m_1$, y al final de él está sujeto otr péndulo de longitud $l_2$ y masa $m_2$. Ambos se describen por los ángulos $\theta_1$ y $\theta_2$ como se muestra en la siguiente figura

Aunque es posible tratar el problema de manera general, no buscamos complicarnos por lo que revisaremos el caso $m_1 = m_2 = m$  y $l_1 = l_2 = l$. La manera más sencilla de obtener las ecuaciones de movimiento es mdiante mecánica Hamiltoniana. La derivación completa se puede revisar en el capítulo 6 del Stickler (*Basic Concepts in Computational Physics*).

Sin ahondar en más detalles, el Hamiltoniano del sistema, en este caso, la energía mecánica total se puede escribir en términos de 4 variables $\theta_1,\theta_2,p_1,p_2$.

$p_1$, $p_2$ son los **momentos generalizados** de $\theta_1$ y $\theta_2$, respectivamente.

La energía cinética $K$ se escribe entonces como

$$
K = \frac{1}{2ml^2} \frac{p_1^2+2p_2^2-2p_1p_2\cos{(\theta_1-\theta_2)}}{1+\sin^2{(\theta_1-\theta_2)}}
$$

Mientras que la energía potencial  $V$

$$
V = mgl\left( 4-2\cos{\theta_1}-\cos{\theta_2} \right)
$$

El Hamiltoniano explícitamente es:

$$
H = K + V = \frac{1}{2ml^2} \frac{p_1^2+2p_2^2-2p_1p_2\cos{(\theta_1-\theta_2)}}{1+\sin^2{(\theta_1-\theta_2})} + mgl\left( 4-2\cos{\theta_1}-\cos{\theta_2} \right)
$$

Las ecuaciones de movimiento de evolución para $\dot{\theta}_1$, $\dot{\theta}_2$, $\dot{p}_1$ y $\dot{p}_2$ se obtienen mediante las **ecuaciones de Hamilton**

$$
\dot{\theta}_i = \frac{\partial H}{\partial p_i}
$$

$$
\dot{p}_i = -\frac{\partial H}{\partial \theta_i}
$$

Explícitamente, **las ecuaciones de movimiento son**

$$
\dot{\theta}_1 = \frac{1}{ml^2} \frac{p_1 - p_2 \cos{(\theta_1-\theta_2)}}{1+\sin^2{(\theta_1-\theta_2)}}
$$

$$
\dot{\theta}_2 = \frac{1}{ml^2} \frac{2p_2 - p_1 \cos{(\theta_1-\theta_2)}}{1+\sin^2{(\theta_1-\theta_2)}}
$$

$$
\begin{align}
\dot{p}_1 &= \frac{1}{ml^2} \frac{1}{1+\sin^2{(\theta_1-\theta_2)}} \big[ -p_1p_2 \sin{(\theta_1 - \theta_2)} \\
&+ \frac{p_1^2 + 2p_2^2 - 2p_1p_2 \cos{(\theta_1 - \theta_2)}}{1+\sin^2{(\theta_1-\theta_2)}} \cos{(\theta_1 - \theta_2)} \sin{(\theta_1 - \theta_2)} \big] \\
&- 2mgl\sin{\theta_1}
\end{align}
$$

$$
\begin{align}
\dot{p}_2 &= \frac{1}{ml^2} \frac{1}{1+\sin^2{(\theta_1-\theta_2)}} \Big[ p_1p_2 \sin{(\theta_1 - \theta_2)} \\
&- \frac{p_1^2 + 2p_2^2 - 2p_1p_2 \cos{(\theta_1 - \theta_2)}}{1+\sin^2{(\theta_1-\theta_2)}} \cos{(\theta_1 - \theta_2)} \sin{(\theta_1 - \theta_2)} \Big] \\
&- mgl\sin{\theta_2}
\end{align}
$$

### Ejercicio 3

Antes de resolver las ecuaciones de movimiento del péndulo doble, debemos tomar los siguientes pasos:

(i) Sea $\boldsymbol{\theta}(t) = (\theta_1(t),p_1(t),\theta_2(t),p_2(t))$, identifica la función $\mathbf{f}(\boldsymbol{\theta}(t),t)$ para la cual el sistema de ecuaciones del péndulo doble se escribe, de forma compacta, como:

$$
\dot{\boldsymbol{\theta}} = \mathbf{f}(\boldsymbol{\theta}(t),t)
$$

(ii) Implementa una función `penduloDoble(Theta,t)` con `Theta` un arreglo de 4 entradas que representa al vector $(\theta_1,p_1,\theta_2,p_2)$ y `t` un flotante que representa el tiempo. Tu función debe de regresa un vector correspondiente a la función $\mathbf{f}(\boldsymbol{\theta}(t),t)$ del inciso anterior.

**Sugerencia:** nota que en las formulas de las ecuaciones de movimiento, y por lo tanto, en la función $\mathbf{f}(\boldsymbol{\theta}(t),t)$, hay muchos términos que se repiten ($1+\sin^2{(\theta_1-\theta_2)}$, por ejemplo).  Puedes usar variables auxiliares para calcular esos términos y así ir calculando la función poco a poco en lugar que de forma abrupta.

### Ejercicio 4

(i) Resuelve el sistema de ecuaciones obtenido en el inciso anterior utilizando tu función `RK4`. Fija condiciones iniciales arbitrarias **que tengan sentido físico**, al igual que dale valores arbitrarios a los parámetros $m$ y $l$. Grafica $\theta_1$ y $\theta_2$ como función del tiempo, así como el espacio fase de cada coordenada ($(\theta_1(t),p_1(t))$ y $(\theta_2(t),p_2(t))$, respectivamente) y los espacios $(\theta_1(t),\theta_2(t))$ y $(p_1(t),p_2(t))$. Haz una animación de la trayectoria del péndulo.

(ii) Grafica la energía potencial, cinética y mecánica del sistema como función del tiempo. ¿La energía se conserva?

(iii) En las ecuaciones, nunca se hace una aproximación de ángulo pequeño. Resuelve el sistema tomando una condición inicial $\theta_1(t_0) > \pi/2$ y realiza las mismas gráficas que en el inciso (i).

(iv) Fija las condiciones iniciales, al igual que el parámetro $m$. Resuelve el sistema para distintos valores de $l$. ¿Cómo cambia la trayectoria del péndulo?. Muestra distintas trayectorias en la misma animación.

### Ejercicio 5

(i) Fija los parámetros $m = 1$, $l = 1$ y fija unas condiciones iniciales arbitrarias $\theta_1(t_0)=a$, $p_1(t_0)=b$, $\theta_2(t_0)=c$, $p_2(t_0)=d$.

Toma un número arbitrario $\epsilon$ tal que $\epsilon << c$ y resuelve dos sistemas: uno para las condiciones iniciales $\theta_2(t_0)=c$ y otro con $\theta_2(t_0)=c+\epsilon$. Toma un tiempo final **muy grande** (que lo soporte tu computadora, claramente). Muestras las dos trayectorias en una misma animación. ¿Las dos trayectorias son parecidas?

(ii) Haz una gráfica de la diferencia (norma) de las posiciones de la segunda masa $m_2$ entre un sistema y el otro como función del tiempo. Recordemos que la posición de la masa $m_2$ está dada por

$$
\mathbf{r_{2}}(t) = (l \sin{(\theta_1(t)) + l \sin{(\theta_2(t)})},l \cos{(\theta_1(t)) + l \cos{(\theta_2(t)})}) \in \mathbb{R}^2
$$

(iii) Viendo la gráfica del inciso (ii) ¿Qué puedes decir del comportamiento de la diferencia de las trayectorias como función del tiempo? ¿Es el comportamiento que esperas para dos condiciones iniciales muy cercanas?

## Métodos implícitos

Todos los métodos para resolver EDOs que hemos visto hasta ahora cumplen con la propiedad de que, independientemente de los pasos auxiliares que hagan, pueden hacer el cálculo de $\mathbf{x}(t+h)$ utilizando como base solamente $\mathbf{f}(\mathbf{x}(t),t)$, $\mathbf{x}(t)$, $t$ y $h$. Es decir, son métodos en los cuales $\mathbf{x}(t+h)$ esta expresado de manera **explícita** como función de valores ya conocidos y puede clacularse directamente.

Sin embargo, podemos realizar aproximaciones de tal forma que $\mathbf{x}(t+h)$ esté definido en términos de de algo que depende de sí mismo, por ejemplo, de algo como $\mathbf{f}(\mathbf{x}(t+h),t)$. En ese caso decimos que $\mathbf{x}(t+h)$ esta expresado de manera **implícita**.

Por ejemplo, si utilizamos una diferencia hacia atrás para aproximar la derivada en la ecuación $\dot{\mathbf{x}}(t) = \mathbf{f}(\mathbf{x}(t),t)$, podríamos expresar $\mathbf{x}(t+h)$ de la siguiente forma:

$$
\mathbf{x}(t+h) = \mathbf{x}(t) + h \; \mathbf{f}(\mathbf{x}(t+h),t+h)
$$

Por simplicidad, supongamos que tenemos el caso $d=1$, es decir, $\mathbf{x}(t) = x(t)$, por lo que no tenemos un sistema de ecuaciones diferenciales, si no solo una ecuación diferencial en la que la aproximación queda:

$$
x(t+h) = x(t) + h f(x(t+h),t+h)
$$

Hagamos un cambio de variable y definamos $z = x(t+h)$. Así, la regla queda

$$
z = x(t) + h f(z,t+h)
$$

Pasando $z$ de un lado, tenemos

$$
 x(t) + h f(z,t+h) - z =0
$$

Si ya conocemos los valores de $t$, $h$ y $x(t)$, entonces la expresión anterior es una **ecuación trascendental para $z$** de la forma $g(z) = 0$. Así, para conocer el valor de $z = x(t+h)$ debemos **resolver la ecuación trascendental utilizando un método ya conocido** (Newton-Rhapson (preferentemente), Bisección, etc).

Todos los métodos en los cuales expresamos $x(t+h)$ de manera implícita y lo calculamos resolviendo una ecuación trascendental son llamados **métodos implícitos**. El ya visto, que esta dado por la regla:

$$
x(t+h) = x(t) + h f(x(t+h),t+h)
$$

Usando índices para la dependencia temporal:

$$
x_i = x_{i-1} + h f(x_{i},t_{i})
$$

se conoce como [**método de Euler hacia atrás**](https://en.wikipedia.org/wiki/Backward_Euler_method). Otro método importante es el [**método trapezoidal**](https://en.wikipedia.org/wiki/Trapezoidal_rule_(differential_equations)), dado por la regla.

$$
x(t+h) = x(t) + \frac{h}{2} \left(  f(x(t),t) + f(x(t+h),t+h) \right)
$$

Usando índices para la dependencia temporal:

$$
x_i = x_{i-1} + \frac{h}{2} \left( f(x_{i-1},t_{i-1}) + f(x_{i},t_{i}) \right)
$$

A la hora de resolver las EDOs, los métodos implícitos son **más tardados** que los explícitos, pues deben de resolver una ecuación trascendental en cada paso, que puede tardar. Sin embargo, pueden resultar más precisos que los métodos explícitos.

A la hora de resolver **sistemas de EDOs** con métodos implícitos, en cada paso se deberá de resolver un sistema de ecuaciones trascendentales, cosa que se vuelve más complicada que el caso 1D (y que además no vimos en el curso). Así, nos restringiremos al caso 1D.

### Ejercicio 6

Implementa funciones `eulerAtras(f,x_0,t_0,t_f,h)` y `trapezoide(f,x_0,t_0,t_f,h)` con `f` la función **vectorial** tal que $\dot{x}(t) = f(x(t),t)$, `x_0`, `t_0` la condición inicial, `t_f` el tiempo máximo al que queremos obtener la solución y `h` el tamaño de paso.

Las funciones deben de implementar, respectivamente, el método de Euler hacia atrás y el método del trapezoide. Utiliza el método de Newtown-Raphson con diferencias finitas y tolerancia $\epsilon$ fija para resolver la ecuación trascendental en cada paso.

Tus funciones deben de regresar el arreglo de los tiempos `[t_0,t_1, t_2,....,t_n]` con $t_{n}\leq t_f < t_{n} + h$ y el de la solución a la ecuación diferencial `[x_0,x_1,...,x_n]`.

**Sugerencia 1:** La ecuación trascendental será distinta para cada $x_{i}$, por lo que deberás definir una función en cada paso del ciclo for para dársela como argumento al método de Newton-Raphson.

### Ejercicio 7

Utiliza tus funciones `eulerAtrás` y `trapezoide` para resolver la ecuación diferencial

$$\dot{x}(t) = x(t)(1-x(t))$$

Con condiciones iniciales arbitrarias **que tengan sentido físico**. ¿La solución mejora, comparada con la solución obtenida con el método de euler normal?. Resuélvela también con  `RK4` ¿Cómo se comparan los métodos implícitos con el método RK4?

## Métodos adaptativos

En todos los métodos que hemos revisado anteriormente, sean explícitos o implícitos, siempre tenemos un **paso de tiempo fijo** $h$ con el cual vamos obteniendo, a partir de $x(t_0)$, primero $x(t_0+h)$, luego  $x(t_0+2h)$, y así hasta un $x(t_f) = x(t_0 + nh)$.

Sin embargo, es posible que nuestra solución real  $x(t)$ sea una función que no varía de la misma manera en todo su dominio. Es decir, que para algunos $t$ el método que usamos sea demasiado impreciso mientras que para otros sea suficientemente preciso.

Eso es muy común cuando nuestras ecuaciones diferenciales son exóticas o usan funciones muy complicadas para definir sus derivadas. Así, en general es mala idea que nuestro paso de tiempo $h$ sea igual en todos los pasos. Lo que conviene es utilizar un **paso de tiempo adaptativo** $h(t)$ que varie en cada paso dependiendo de alguna regla.

Los métodos que utilizan pasos de tiempo adaptativos se conocen como [**métodos adaptativos**](https://en.wikipedia.org/wiki/Adaptive_step_size). Cualquier método explícito o implícito puede extenderse, en principio, a un método adaptativo. Para nuestro ejemplo, tomaremos el método más simple: crearemos un método de Euler adaptativo.

Sabemos que la regla del método de euler es

$$
\mathbf{x}(t+h) = \mathbf{x}(t) + h \; \mathbf{f}(\mathbf{x}(t),t)
$$

Podemos considerar a $\mathbf{x}(t+h)$ obtenida así una *primera* estimación de $\mathbf{x}(t+h)$, y la denotamos $\mathbf{x}^{(1)}$. Por otro lado, podemos obtener otra estimación, denotada $\mathbf{x}^{(2)}$, utilizando dos pasos de euler con paso $h/2$: primero obtenemos $\mathbf{x}(t + h/2) $ como

$$
\mathbf{x}(t+h/2) = \mathbf{x}(t) + \frac{h}{2} \mathbf{f}(\mathbf{x}(t),t)
$$

Para posteriormente obtener

$$
\mathbf{x}^{(2)} = \mathbf{x}(t+h) = \mathbf{x}(t+h/2) + \frac{h}{2} \mathbf{f}(\mathbf{x}(t+h/2),t+h/2)
$$

Definimos ahora el **error entre aproximaciones** $\epsilon$ como

$$
\epsilon = \Vert \mathbf{x}^{(1)} - \mathbf{x}^{(2)} \Vert
$$

En principio,  si la función $\mathbf{x}(t)$ no tiene cambios abruptos, ambas aproximaciones se parecerán y $\epsilon$ será pequeño. Sin embargo, si la función real $\mathbf{x}(t)$ tiene cambios muy abruptos, entonces ambas aproximaciones serán malas y la $\epsilon$ será grande. Así, ver el tamaño o magnitud de $\epsilon$ nos permite saber cuando el paso $h$ es demasiado grande y cuando es suficientemente bueno y podríamos aumentarlo.

Supongamos que definimos una tolerancia del error denotada $tol$. $tol >0$ es un número real positivo y pequeño. Proponemos entonces la siguiente regla de actualización:

$$
\begin{align}
\mathbf{x}^{(1)} &= \mathbf{x}(t) + h(t) \mathbf{f}(\mathbf{x}(t),t) \\
\mathbf{x}(t+h(t)/2) &= \mathbf{x}(t) + \frac{h(t)}{2} \mathbf{f}(\mathbf{x}(t),t) \\
\mathbf{x}^{(2)} &= \mathbf{x}(t+h(t)/2) + \frac{h(t)}{2} \mathbf{f}(\mathbf{x}(t+h(t)/2),t+h(t)/2) \\
\epsilon &= \Vert \mathbf{x}^{(1)} - \mathbf{x}^{(2)} \Vert \\
& \\
\mathbf{x}(t+h(t)) &= \mathbf{x}^{(2)} \\
h(t+h(t)) &= 0.9 h(t) \; \frac{tol}{\epsilon}
\end{align}
$$

Notemos que si $\epsilon > tol$, $h(t+h(t)) < h(t)$, es decir, nuestro tamaño de paso decrece. Por otro lado, si $\epsilon < tol$, $h(t+h(t)) > h(t)$ el paso aumentará en proporción al cociente entre $tol$ y $\epsilon$.

Utilizando ahora índices para la dependencia temporal y usando una variable $\mathbf{k}_1$ para denotar a $\mathbf{x}(t+h(t)/2)$, finalmente obtenemos

$$
\begin{align}
\mathbf{x}^{(1)} &= \mathbf{x}_{i-1} + h_{i-1} \mathbf{f}(\mathbf{x}_{i-1},t_{i-1}) \\
\mathbf{k}_1 &= \mathbf{x}_{i-1} + \frac{h_{i-1}}{2} \mathbf{f}(\mathbf{x}_{i-1},t_{i-1}) \\
\mathbf{x}^{(2)} &= \mathbf{k}_1 + \frac{h_{i-1}}{2} \mathbf{f}(\mathbf{k}_1,t_{i-1}+h_{i-1}/2) \\
\epsilon &= \Vert \mathbf{x}^{(1)} - \mathbf{x}^{(2)} \Vert \\
& \\
\mathbf{x}_{i} &= \mathbf{x}^{(2)} \\
h_{i} &= 0.9 h_{i-1} \; \frac{tol}{\epsilon}
\end{align}
$$

### Ejercicio 8

Implementa una función `eulerAdaptativo(f,x_0,t_0,t_f,h_0,tol)` con `f` la función **vectorial** tal que $\dot{\mathbf{x}}(t) = \mathbf{f}(\mathbf{x}(t),t)$, `x_0` (un vector de $n$ entradas) y `t_0` la condición inicial, `t_f` el tiempo máximo al que queremos obtener la solución, `h_0` el tamaño de paso inicial y `tol` la tolerancia del error en cada paso. La función debe de implementar el método de Euler adaptativo

Tu función debe de regresar el arreglo de los pasos de tiempo `[h_0,..,h_n]`, el de los tiempos `[t_0,t_1, t_2,....,t_n]` con $t_{n}< t_f < t_{n} + h_{n}$ y el de la solución a la ecuación diferencial `[x_0,x_1,...,x_n]`.

**Sugerencia 1** esto es muy parecido a un método de Euler normal, así que puedes tomar tu función `eulerVec` como punto de partida.

### Ejercicio 9

Utiliza tu función `eulerAdaptativo` para resolver el problema del péndulo simple. ¿Le sucede el mismo problema que el método de Euler normal? Grafica el ángulo como función del tiempo, el espacio fase y las energías potencial, cinética y mecánica. ¿El método conserva la energía?

```julia

```


 INF285 - Computación Científica

 Polynomial Interpolation: Vandermonde matrix, Lagrange interpolation, Barycentric interpolation, Newton's divided differences, Chebyshev points.

  [S]cientific [C]omputing [T]eam

 Version: 1.30

## Table of Contents
* [Introduction](#intro)
* [Vandermonde Matrix](#vander)
* [Lagrange Interpolation](#lagrange)
    * [Plotting $L_i(x)$](#LiLagrange)
* [Barycentric interpolation](#barycentric)
* [Newton's Divided Difference](#DDN)
* [Interpolation Error](#Error)
* [Runge Phenomenon](#runge)
* [Chebyshev Interpolation](#cheby)
* [Python Modules and Functions](#py)
* [Acknowledgements](#acknowledgements)

```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
from functools import reduce
import matplotlib as mpl
mpl.rcParams['font.size'] = 14
mpl.rcParams['axes.labelsize'] = 20
mpl.rcParams['xtick.labelsize'] = 14
mpl.rcParams['ytick.labelsize'] = 14
%matplotlib inline
from ipywidgets import interact, fixed, IntSlider, interact_manual, Checkbox, RadioButtons
sym.init_printing()
```

## Introduction
[Back to TOC](#toc)

Hello! In this notebook we will learn how to interpolate 1D data with polynomials.
A polynomial interpolation consists in finding a polynomial that fits a discrete set of known data points, allowing us to construct new data points within the range of the data.
Formally, a polynomial $p(x)$ interpolate the data $(x_1,y_1),...,(x_n,y_n)$ if $p(x_i)=y_i$ for all $i$ in $1,...,n$.

```python
# Function that evaluates the xi's points in the polynomial
def EvaluateInterpolant(D, xx):
    if D['Method']=='Vandermonde':
        p = lambda x: np.dot(D['PrecomputedData'],np.power(x,np.arange(D['n'])))
    elif D['Method']=='Lagrange':
        p = lambda x: np.dot(D['PrecomputedData'],[np.prod(x - np.delete(D['x'],j)) for j in range(D['n'])])
    elif D['Method']=='Barycentric':
        wi = D['PrecomputedData']
        xi = D['x']
        yi = D['y']
        p = lambda x: np.sum(yi*wi/(x-xi))/np.sum(wi/(x-xi)) if len(xi[xi == x])==0 else np.ndarray.item(yi[xi == x])
    elif D['Method']=='Newton':
        p = lambda x: np.dot(D['PrecomputedData'],np.append([1],[np.prod(x-D['x'][:j]) for j in range(1,D['n'])]))
    return np.array([p(x) for x in xx])

# Function that shows the data points and the function that interpolates them.
def PlotInterpolant(D,ylim=None):
    xi = np.linspace(min(D['x']),max(D['x']),1000)
    yi = EvaluateInterpolant(D,xi)
    plt.figure(figsize=(8,8))
    plt.plot(D['x'],D['y'],'ro',label='Interpolation points')
    plt.plot(xi,yi,'b-',label='$p(x)$')
    plt.xlim(min(xi)-0.5, max(xi)+0.5)
    if ylim:
        plt.ylim(ylim[0], ylim[1])
    else:
        plt.ylim(min(yi)-0.5, max(yi)+0.5)
    plt.grid(True)
    plt.legend(loc='best')
    plt.xlabel('$x$')
    #plt.ylabel('$P(x)$')
    plt.show()
```

## Vandermonde Matrix
[Back to TOC](#toc)

First, we are going to learn the Vandermonde matrix method.
This is a $n \times n$ matrix (with $n$ being the cardinality of the set of known data points) with the terms of a geometric progression in each row.
It allows us to construct a system of linear equations with the objective of find the coefficients of the polynomial function that interpolates our data.

Example:

Given the set of known data points: $(x_1,y_1),(x_2,y_2),(x_3,y_3)$

Our system of linear equations will be:
$$ \begin{bmatrix}
    1 & x_1 & x_1^2  \\[0.3em]
    1 & x_2 & x_2^2  \\[0.3em]
    1 & x_3 & x_3^2  \end{bmatrix}
    \begin{bmatrix}
    a_0  \\[0.3em]
    a_1  \\[0.3em]
    a_2  \end{bmatrix} =
    \begin{bmatrix}
    y_1  \\[0.3em]
    y_2  \\[0.3em]
    y_3  \end{bmatrix}$$
And by solving it we will find the coefficients $a_0,a_1,a_2$ that we need to construct the polynomial, $$p(x)=a_0+a_1\,x+a_2\,x^2,$$
that interpolates our data.

```python
def Vandermonde(x, y, show=False):
    # We construct the matrix and solve the system of linear equations
    # A = np.array([xi**np.arange(len(x)) for xi in x]) # OLDER VERSION
    A = np.vander(x, increasing=True)
    b = y
    coefficients = np.linalg.solve(A,b)
    n = len(x)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(n)])
        print('A = '); print(np.array_str(A, precision=2, suppress_small=True))
        print("cond(A) = "+str(np.linalg.cond(A)))
        print('b = '); print(np.array_str(b, precision=2, suppress_small=True))
        print('x = '); print(np.array_str(coefficients, precision=2, suppress_small=True))
        xS = sym.Symbol('x')
        F = np.dot(xS**np.arange(len(x)),coefficients)
        print('Interpolation Function: ')
        print('F(x) = ')
        print(F)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method': 'Vandermonde',
         'PrecomputedData': coefficients,
         'x': x,
         'y': y,
         'n': n}
    return D
```

The following example shows the output of a polynomial interpolation with the provided data.
We use an epsilon value on the thrid interpolation point ($x_3=3+\varepsilon$) and then we control it with a intercat widget, this way we can evaluate what will happen to the interpolation when two collocation points gets close to each other.
This behavior is obtained as soon as the third point is to close to $x_2=2$ by setting the value of $\varepsilon$ equal to $-1$, or making it close to the forth point $x_4$ by setting $\varepsilon$ equal to $2$.

```python
def show_time_Vandermonde(epsilon=0):
    x = np.array([0.0,2.0,3.0+epsilon,4.0,5.0,6.0])
    y = np.array([1.0,3.0,0.0,6.0,8.0,4.0])
    D = Vandermonde(x,y,True)
    PlotInterpolant(D)
interact(show_time_Vandermonde,epsilon=(-1,1,0.1))
```

    interactive(children=(FloatSlider(value=0.0, description='epsilon', max=1.0, min=-1.0), Output()), _dom_classe…

    <function __main__.show_time_Vandermonde(epsilon=0)>

As the numerical experiment show, the interpolated polynomial shows larger and larger oscilations as two points get close to each other.
Notice however that the function still interpolates the data.

## Lagrange Interpolation
[Back to TOC](#toc)

With this method, we can interpolate data thanks to the Lagrange basis polynomials. Given a set of $n$ data points $(x_1,y_1),...,(x_n,y_n)$, the Lagrange interpolation polynomial is the following:
$$ p(x) = \sum^n_{i=1} y_i\,L_i(x),$$
where $L_i(x)$ are the Lagrange basis polynomials:
$$ L_i(x) = \prod^n_{j=1,j \neq i} \frac{x-x_j}{x_i-x_j} = \frac{x-x_1}{x_i-x_1} \cdot ... \cdot \frac{x-x_{i-1}}{x_i-x_{i-1}} \cdot \frac{x-x_{i+1}}{x_i-x_{i+1}} \cdot ... \cdot \frac{x-x_n}{x_i-x_n}$$
or simply $L_i(x)=\dfrac{l_i(x)}{l_i(x_i)}$, where $l_i(x)=\displaystyle{\prod^n_{j=1,j \neq i} (x-x_j)}$.
The key properties of these basis polynomials are the followings:
$$ L_i(x_i) = 1 $$
$$ L_{j \neq i}(x_i) = 0 $$
So, we assure that $L(x_i) = y_i$, which indeed interpolates the data.

```python
def Lagrange(x, y, show=False):
    # We calculate the li's
    n = len(x)
    p = np.array([y[i]/np.prod(x[i] - np.delete(x,i)) for i in range(n)])

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(n)])
        xS = sym.Symbol('x')
        L = np.dot(np.array([np.prod(xS - np.delete(x,i))/np.prod(x[i] - np.delete(x,i)) for i in range(n)]),y)
        print('Interpolation Function: ');
        print(L)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method': 'Lagrange',
         'PrecomputedData': p,
         'x': x,
         'y': y,
         'n': n}
    return D
```

The following numerical experiment follows the same idea of the previous numerical experiment with the Vandermonde matrix.

```python
def show_time_Lagrange(epsilon=0):
    x = np.array([0.0,2.0,3.0+epsilon,4.0,5.0,6.0])
    y = np.array([1.0,3.0,0.0,6.0,8.0,4.0])
    D = Lagrange(x,y,True)
    PlotInterpolant(D)
interact(show_time_Lagrange,epsilon=(-1,1,0.1))
```

    interactive(children=(FloatSlider(value=0.0, description='epsilon', max=1.0, min=-1.0), Output()), _dom_classe…

    <function __main__.show_time_Lagrange(epsilon=0)>

### Plotting $L_i(x)$
[Back to TOC](#toc)

The next cell shows what $L_i(x)$ looks like.
It basically shows how a polynomial looks like when it is one at $x_i$ and zero at all the other interpolation nodes.

```python
def show_time_Li(i=0, N=7):
    x = np.arange(N+1)
    y = np.zeros(N+1)
    y[i]=1
    D = Lagrange(x,y,True)
    PlotInterpolant(D,[-1,2])

i_widget = IntSlider(min=0, max=7, step=1, value=0)
N_widget = IntSlider(min=1, max=20, step=1, value=7)
def update_i_range(*args):
    i_widget.max = N_widget.value
N_widget.observe(update_i_range, 'value')

interact(show_time_Li,i=i_widget,N=N_widget)
```

    interactive(children=(IntSlider(value=0, description='i', max=7), IntSlider(value=7, description='N', max=20, …

    <function __main__.show_time_Li(i=0, N=7)>

The following example show all the $L_i(x)$ for $i\in\{1,2,\dots,N\}$ at once.
This was included thank to an inclass question!

```python
def show_time_all_Li(N=3, ShowLegend=False):
    x = np.arange(1,N+1)
    xx = np.linspace(1,N,1000)
    for i in range(N):
        y = np.zeros(N)
        y[i]=1
        D = Lagrange(x,y)
        yy=EvaluateInterpolant(D, xx)
        plt.plot(xx,yy,label=r'$L_{0}(x)$'.format(i+1))
    if ShowLegend:
        plt.legend(loc='best')
    plt.grid(True)
interact(show_time_all_Li,N=(2,20,1), ShowLegend=Checkbox(value=False, description="Include legend"))
```

    interactive(children=(IntSlider(value=3, description='N', max=20, min=2), Checkbox(value=False, description='I…

    <function __main__.show_time_all_Li(N=3, ShowLegend=False)>

Here you get some questions about Lagrange Interpolation:
- Explain what happens with the interpolator polynomial when you add a new point to the set of points to interpolate. **Answer: We need to recalculate the polynomial**
- Why it is not a good idea to use Lagrange interpolation for a set of points which is constantly changing? **A: Because we need to compute the whole interpolation again**
- What is the operation count of obtaining the interpolator polynomial using Lagrange? What happens with the error?

#### As an additional example we show the $\text{sinc}(x)$ function.
This function is a relative of $L_i(x)$ from Lagrange interpolation, the only difference is that it is one at one point and zero over an infinite grid.
This could be alse use for interpolation, you only need to shift it a bit and you good to go!
This is not an polynomial interpolation but it may be a good alternative.

```python
xD = np.arange(-10,11)
yD = np.zeros_like(xD)
yD[10] = 1
xi = np.linspace(-10,10,1000)
yi = np.sinc(xi)
plt.figure(figsize=(8,8))
plt.plot(xD,yD,'ro',label='Interpolation points')
plt.plot(xi,yi,'b-',label='sinc$(x)$')
plt.xlim(min(xi)-0.5, max(xi)+0.5)
plt.ylim(min(yi)-0.5, max(yi)+0.5)
plt.grid(True)
plt.legend(loc='best')
plt.xlabel('$x$')
#plt.ylabel('$P(x)$')
plt.show()
```

## Barycentric interpolation
[Back to TOC](#toc)

The barycentric interpolation for interpolation $n$ data points $(x_1,y_1),...,(x_n,y_n)$ is of the following form:
$$ p(x) = \dfrac{\displaystyle{\sum_{i=1}^n y_i\dfrac{w_i}{(x-x_i)}}}{\displaystyle{\sum_{i=1}^n \dfrac{w_i}{(x-x_i)}}},$$
where $w_i=\dfrac{1}{l_i(x)}$ and $l_i(x)=\displaystyle{\prod_{k=1,i\neq k}^n(x-x_k)}$.

```python
def Barycentric(x, y, show=False):
    W  = np.subtract.outer(x, x)
    wi  = 1/np.prod(W, axis=1, where = W!=0)
    n = len(x)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(n)])
        xS = sym.Symbol('x')
        N = np.sum(y*wi/(xS-x))/np.sum(wi/(xS-x))
        print('Interpolation Function: ');
        print(N)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method':'Barycentric',
         'PrecomputedData': wi,
         'x': x,
         'y': y,
         'n': n}
    return D
```

```python
def show_time_Barycentric(epsilon=0):
    x = np.array([0.0,2.0,3.0+epsilon,4.0,5.0,6.0])
    y = np.array([1.0,3.0,0.0,6.0,8.0,4.0])
    D = Barycentric(x,y,True)
    PlotInterpolant(D)
interact(show_time_Barycentric,epsilon=(-1,1,0.1))
```

    interactive(children=(FloatSlider(value=0.0, description='epsilon', max=1.0, min=-1.0), Output()), _dom_classe…

    <function __main__.show_time_Barycentric(epsilon=0)>

## Newton's Divided Difference
[Back to TOC](#toc)

In this interpolation method we will use divided differences to calculate the coefficients of our interpolation polynomial. Given a set of $n$ data points $(x_1,y_1),...,(x_n,y_n)$, the Newton polynomial is:
$$ p(x) = \sum^n_{i=1} (f[x_1 ... x_i] \cdot \prod^{i-1}_{j=1} (x-x_j)) ,$$
where $ \prod^{0}_{j=1} (x-x_j) = 0 $, and:
$$ f[x_i] = y_i $$
$$ f[x_j...x_i] = \frac{f[x_{j+1}...x_i]-f[x_j...x_{i-1}]}{x_i-x_j}$$

```python
def Divided_Differences(x, y):
    dd = np.array([y])
    n = len(x)
    for i in range(n-1):
        ddi = []
        for a in range(n-i-1):
            ddi.append((dd[i][a+1]-dd[i][a])/(x[a+i+1]-x[a]))
        ddi = np.append(ddi,np.full((n-len(ddi),),0.0))
        dd = np.append(dd,[ddi],axis=0)
    return np.array(dd)

def Newton(x, y, show=False):
    # We calculate the divided differences and store them in a data structure
    dd = Divided_Differences(x,y)
    n = len(x)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(n)])
        xS = sym.Symbol('x')
        N = np.dot(dd[:,0],np.append([1],[np.prod(xS-x[:i]) for i in range(1,n)]))
        print('Interpolation Function: ');
        print(N)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method':'Newton',
         'PrecomputedData': dd[:,0],
         'x': x,
         'y': y,
         'n': n}
    return D
```

```python
def show_time_Newton(epsilon=0):
    x = np.array([0.0,2.0,3.0+epsilon,4.0,5.0,6.0])
    y = np.array([1.0,3.0,0.0,6.0,8.0,4.0])
    D = Newton(x,y,True)
    PlotInterpolant(D)
interact(show_time_Newton,epsilon=(-1,1,0.1))
```

    interactive(children=(FloatSlider(value=0.0, description='epsilon', max=1.0, min=-1.0), Output()), _dom_classe…

    <function __main__.show_time_Newton(epsilon=0)>

Questions about Newton's DD:
- What is the main problem using this method (and Lagrange)? How can you fix it? **A: A problem with polynomial interpolation with equispaced date is the Runge phenomenon and can be handle with Chebyshev points**
- What to do when a new point is added? **A: Pro, is not necessary re-calculate the whole polynomial only a small piece**

## Polynomial Interpolation Error
[Back to TOC](#toc)

The interpolation error is given by:

$$ f(x)-p(x) = \frac{(x-x_1) \, (x-x_2) \, \dots \, (x-x_n)}{n!} \, f^{(n)}(c),$$

where $c$ is within the interval defined by $[\min(x,x_1,\dots,x_n),\max(x,x_1,\dots,x_n)]$.

```python
def Error(f, n, xmin, xmax, method=Lagrange, points=np.linspace, plot_flag=True):
    # This function plots f(x), the interpolating polynomial, and the associated error
    # points can be np.linspace to equidistant points or Chebyshev to get Chebyshev points

    # This two lines generate the data that will be used in the interpolation
    x = points(xmin,xmax,n)
    y = f(x)

    # This two lines compute the exact value of the function being interpolated on a finer grid
    xe = np.linspace(xmin,xmax,100)
    ye = f(xe)

    # This two lines build the interpolant chosen, where the name of the method is
    # passed as parameter usign the variable 'method'.
    D = method(x,y)
    yi = EvaluateInterpolant(D, xe)

    if plot_flag:
        plt.figure(figsize=(5,10))
        f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5), sharey = False)

        ax1.plot(xe, ye,'r-', label='f(x)')
        ax1.plot(x, y,'ro', label='Interpolation points')
        ax1.plot(xe, yi,'b-', label='Interpolation')
        ax1.set_xlim(xmin-0.5,xmax+0.5)
        ax1.set_ylim(min(yi)-0.5,max(yi)+0.5)
        ax1.set_title('Interpolation')
        ax1.grid(True)
        ax1.set_xlabel('$x$')
        ax1.legend(loc='best')

        ax2.semilogy(xe, abs(ye-yi),'b-', label='Absolute Error')
        ax2.set_xlim(xmin-0.5,xmax+0.5)
        ax2.set_title('Absolute Error')
        ax2.set_xlabel('$x$')
        ax2.grid(True)
        plt.show()

    return max(abs(ye-yi))
```

```python
def test_error_Newton(n=5):
    # Example 1
    #me = Error(lambda x: np.sin(x)**3, n, 1, 7, Newton)
    # Example 2
    me = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, Newton)
    print("Max Error:", me)
```

```python
interact(test_error_Newton,n=(5,25))
```

    interactive(children=(IntSlider(value=5, description='n', max=25, min=5), Output()), _dom_classes=('widget-int…

    <function __main__.test_error_Newton(n=5)>

##  **Runge's Phenomenon**: It is a problem of oscillation of polynomials at the edges of the interval.
[Back to TOC](#toc)

We are interpolating a data that is 0 almost everywhere and 1 at the middle point.
Notice that as $n$ increases the oscilations increase and all the red dots seems to be at 0 in the plot but this is just an artifact due to the scale on the left.
The oscillations you see at the end of the interval is the Runge phenomenon.
This is due to the use of equalspaced point.

```python
def Runge(n=9):
    x = np.linspace(0,1,n)
    y = np.zeros(n)
    y[int((n-1.0)/2.)]=1
    D = Newton(x,y,False)
    PlotInterpolant(D)
interact(Runge,n=(5,25,2))
```

    interactive(children=(IntSlider(value=9, description='n', max=25, min=5, step=2), Output()), _dom_classes=('wi…

    <function __main__.Runge(n=9)>

## Chebyshev Interpolation
[Back to TOC](#toc)

From the interpolation error defined previously, we notice on the right hand side that the only term we can actually control (besides $n$) is the actual value used for the $x_i$, for $i\in\{1,2,\dots,n\}$.
Thus, with the objective of reducing the error of the polynomial interpolation, we need to find the values of $x_1,x_2,...,x_n$ that minimize the following expression:
$$(x-x_1) \, (x-x_2) \, ... \, (x-x_n).$$

To choose these values we select the canonical interval $[-1,1]$, in particular, we expect the points to be in $-1 \leq x_1,x_2,...,x_n \leq 1$ (to use another interval we just need to do a change of variables).
This probles was already studied by Chebyshev, thus we will use the roots of the Chebyshev polynomials, also called **Chebyshev nodes** (of the first kind), which are defined by:

$$ x_i = \cos\left(\frac{(2i-1)\pi}{2n}\right), i\in\{1,2,\dots,n\} $$

```python
def Chebyshev(xmin,xmax,n=5):
    # This function calculates the n Chebyshev points and plots or returns them depending on ax
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    y = np.sin((2*ns-1)*np.pi/(2*n))
    plt.figure(figsize=(10,5))
    plt.ylim(-0.1,1.1)
    plt.xlim(-1.1,1.1)
    plt.plot(np.cos(np.linspace(0,np.pi)),np.sin(np.linspace(0,np.pi)),'k-')
    plt.plot([-2,2],[0,0],'k-')
    plt.plot([0,0],[-1,2],'k-')
    for i in range(len(y)):
        plt.plot([x[i],x[i]],[0,y[i]],'r-')
        plt.plot([0,x[i]],[0,y[i]],'r-')
    plt.plot(x,[0]*len(x),'bo',label='Chebyshev points')
    plt.plot(x,y,'ro')
    plt.xlabel('$x$')
    plt.title('n = '+str(n))
    plt.grid(True)
    plt.legend(loc='best')
    plt.show()
def Chebyshev_points(xmin,xmax,n):
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    #y = np.sin((2*ns-1)*np.pi/(2*n))
    return (xmin+xmax)/2 + (xmax-xmin)*x/2
def Chebyshev_points_histogram(n=50,nbins=20):
    xCheb=Chebyshev_points(-1,1,n)
    #plt.figure()
    #plt.hist(xCheb,bins=nbins,density=True,cumulative=True)
    #plt.grid(True)
    #plt.show()

    plt.figure(figsize=(5,10))
    f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5), sharey = False)
    ax1.hist(xCheb,bins=nbins,density=True)
    ax1.grid(True)
    ax2.hist(xCheb,bins=nbins,density=True,cumulative=True)
    ax2.grid(True)
    plt.show()
```

The next plot shows a graphical interpretation of the Chebyshev nodes, in particular, they are the projection on the $x$-axis from equalspaced point on the unit sphere.
The angle used for defining the equalspaced point on the unit sphere are: $\theta_i=\frac{(2i-1)\pi}{2n}$, for $i\in\{1,2,\dots,n\}$.

```python
interact(Chebyshev,xmin=fixed(-1),xmax=fixed(1),n=(2,50))
```

    interactive(children=(IntSlider(value=5, description='n', max=50, min=2), Output()), _dom_classes=('widget-int…

    <function __main__.Chebyshev(xmin, xmax, n=5)>

The next plot shows an histogram to explicitly show that the Chebychev nodes cluster at each end of the interval.
The left plot is an explicit histogram and the plot on the right is the cumulative distribution function.

```python
interact_manual(Chebyshev_points_histogram,n=(20,10000),nbins=(20,200))
```

    interactive(children=(IntSlider(value=50, description='n', max=10000, min=20), IntSlider(value=20, description…

    <function __main__.Chebyshev_points_histogram(n=50, nbins=20)>

Moreover, these points have the an explicit expresion for the term we want to minimize,
this means,
$$ (x-x_1) \, (x-x_2) \, ... \, (x-x_n) = \dfrac{1}{2^{n-1}} \, T_n(x), $$
where $T_n(x) = \cos (n \, \arccos (x))$ is the n-th Chebyshev polynomial.
$$
\begin{align*}
T_0(x) &= 1, \\
T_1(x) &= x, \\
T_2(x) &= 2x^2 -1,\\
\vdots
\end{align*}
$$

It also has a recurrence relation,
$$ T_{n+1}(x) = 2 \cdot x \cdot T_n(x) - T_{n-1}(x).$$
The importance of the first identity showed in this cell is that we can bound the left-hand-side when using Chebychev points.
This is not possible when using equalspaced points.
For instance, when using Chebychehv points we have the following bound,
$$|(x-x_1) \, (x-x_2) \, ... \, (x-x_n)| = \left|\dfrac{1}{2^{n-1}} \, T_n(x)\right|\leq \dfrac{1}{2^{n-1}}.$$
This is true since $T(x) = \cos (n \, \arccos (x))$ and $|\cos(x)|\leq 1$.
This is useful when one wants to find the number of points required to interpolate a funcion to a required error.
In this case the error would become,
$$
\begin{align*}
|f(x)-p(x)| &= \left|\frac{(x-x_1) \, (x-x_2) \, \dots \, (x-x_n)}{n!} \, f^{(n)}(c)\right|\\
            &\leq \frac{1}{2^{n-1}\,n!} \, \left|f^{(n)}(c)\right|,
\end{align*}
$$
which is very nice!
We only need to pay attention to find a bound for $\left|f^{(n)}(c)\right|$ on the interval we want to interpolate the function.

In the next cell, for completeness, we plot the Chebyshev polynomials $T_n(x)$.

```python
# Recursive function that returns the n-th Chebyshev polynomial evaluated at x
def T(n,x):
    if n == 0:
        return x**0
    elif n == 1:
        return x
    else:
        return 2*x*T(n-1,x)-T(n-2,x)

# This function plots the first n Chebyshev polynomials
def Chebyshev_Polynomials(n=2, Flag_All_Tn=False):
    x = np.linspace(-1,1,1000)
    plt.figure(figsize=(10,5))
    plt.xlim(-1, 1)
    plt.ylim(-1.1, 1.1)
    if Flag_All_Tn:
        for i in np.arange(n+1):
            y = T(i,x)
            plt.plot(x,y,label='$T_{'+str(i)+'}(x)$')
    else:
        y = T(n,x)
        plt.plot(x,y,label='$T_{'+str(n)+'}(x)$')
    plt.legend(loc='right')
    plt.grid(True)
    plt.xlabel('$x$')
    plt.show()
```

```python
interact(Chebyshev_Polynomials,n=(0,12),Flag_All_Tn=True)
```

    interactive(children=(IntSlider(value=2, description='n', max=12), Checkbox(value=True, description='Flag_All_…

    <function __main__.Chebyshev_Polynomials(n=2, Flag_All_Tn=False)>

The follwing function compares the error obtain by interpolation a function with equalspaced points and Chebychev points.

```python
def test_error_chebyshev(n=5):
    mee = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, Lagrange)
    mec = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, method=Lagrange, points=Chebyshev_points)
    print("Max error (equidistants points):", mee)
    print("Max error (Chebyshev nodes):", mec)
```

The plots on the left show the actual function and the corresponding interpolation.
The plots in the right show the error in $\log_{10}$ scale.
Notice that we interpolation polynomial goes by the interpolation nodes exactly, this induce an error equal to $0$ and these nodes.
This behaviour explain the down spikes on the logaritmic scales, but we care about the worst case.
In the worst case the equalspaced polynomial behaves very badly, this is way polynomial interpolation with Chebyshev points is so useful.

```python
interact(test_error_chebyshev,n=(5,100,2))
```

    interactive(children=(IntSlider(value=5, description='n', min=5, step=2), Output()), _dom_classes=('widget-int…

    <function __main__.test_error_chebyshev(n=5)>

Questions about Chebyshev:
- How can you calculate the Chebyshev points in the interval [a,b] instead of [-1,1]? **A: Using a change of variable**

## Convergence analysis

The next set of experiments show a convergence analysis comparing equalspaced points to Chebychev points.
The $x$-axis is the number of points used and the $y$-axis is the maxmimum error obtain on the interval in logarithmic scale.
The experiment are the followings:
1. $f(x)=x^{10}$. In this case both sets of points perform similarly, however there is a significant drop in the error when $n$ is equal to $11$. This means, the error reach a minimum of $10^{-14}$ app at that number points. Why?
2. $f(x)=|x^3|$. This looks like a naive function but it is not. The equalspace points start to decrease from an error of $10^{0}$ until an error app $10^{-3}$ with approximately 10 points, it then start to increase the error. On the other hand, the Chebychev points show a decresing behavior upto $n=50$ but it only reaches an error of app $10^{-5}$, which is too high. What is the issue with this funcion?
3. $f(x)=\exp(-(x^{-2}))$. This is also a tricky function bevcause involves a division by $0$ when we evaluate it at $x=0$, but it is well defined at that points. Similarly to the first example, the equalspaced points reduce the error upto $n=10$, and then start to increase. For the Chebychev points, they decrease upto 40 points and the also show a slight increment in the error. What is the value of $f(x)$ at $x=0$?
4. $f(x)=\dfrac{1}{1+x^2}$. In this case both cases decrease the error but faster for the Chebyshev points. The main difference is that the minimum error reached for equalspaced points is about $10^{-8}$ and $10^{-13}$ for the Chebyshev points, in both cases about $n=37$.
5. $f(x)=\sin^3(x)$. In this case both cases reached an error $10^{-14}$ with $n=20$, then for equalspaced points start to increase the error right away but for Chebyshev points it starts to increase at $n=40$.

Answers to the previous questions:
1. The reason is that when $n=11$ we will be interpolation an polynomial of degree $10$ with a polynomial of degree $10$, so we can reproduce it exactly! This is why the error is $0$. When $n$ was lower, we only had an approximation. It is interesting that the error increases as we increase the number of points after $n=10$. Would it be useful to change the algorihtm used?
2. The issue is that the function is not analytic, this is because it involves the absolute value. Recall that expression for the error requires the computation of the $n$-th derivative of $f(x)$ at a $c$ point, so if the function is not differentiable, in this case the issues is at $x=0$, we could expect a bad behavior. The probles can be easily solve if we interpolate the function with a polynomial defined on $[-1,0]$ and another on $[0,1]$, ensurring the are equal at $x=0$. In this case we only need two polynomials of degree $3$ and we will be able to reproduce the function exactly! The issue is to notice this when we only have access to the computational implementation of $f(x)$ and not to the function itself.
3. 0
4. -
5. -

**We strongly suggest you to try different functions and play around!**

```python
my_functions=[lambda x: x**10,
              lambda x: np.abs(x**3),
              lambda x: np.exp(-(x**-2)),
              lambda x: 1/(1+x**2),
              lambda x: np.sin(x)**3]
labels = ["x^{10}",
          "|x^3|",
          "\exp(-x^{-2})",
          "1/(1+x^2)",
          "\sin^3(x)"]
data=zip(labels,my_functions)

radio_button_function=RadioButtons(
    options=list(data),
    description='Function:',
    disabled=False
)

radio_button_interpolation_algorithm=RadioButtons(
    options=[('Vandermonde',Vandermonde),('Lagrange',Lagrange),('Barycentric',Barycentric),('Newton',Newton)],
    value=Newton,
    description='Algorithm:',
    disabled=False
)

def convergence_study(my_function, method):
    n=100
    shift=1
    n_points=np.arange(shift,n)
    max_error=np.zeros(n-shift)
    max_error_es=np.zeros(n-shift)
    for i in n_points:
        max_error[i-shift] = Error(my_function, i, -1, 1, method, Chebyshev_points, plot_flag=False)
        max_error_es[i-shift] = Error(my_function, i, -1, 1, method, points=np.linspace, plot_flag=False)
    axis=plt.figure(figsize=(8,8))
    plt.semilogy(n_points,max_error,'bd',label='Chebyshev points',markersize=10)
    plt.semilogy(n_points,max_error_es,'r.',label='Equalspaced poins',markersize=16)
    plt.ylim(10**-16,10**4)
    plt.grid(True)
    plt.title('Interpolation Error')
    plt.xlabel('Number of points used in the interpolation')
    plt.ylabel('Max error on domain')
    plt.legend(loc='best')
    plt.show()
interact(convergence_study,my_function=radio_button_function, method=radio_button_interpolation_algorithm)
```

    interactive(children=(RadioButtons(description='Function:', options=(('x^{10}', <function <lambda> at 0x7ff830…

    <function __main__.convergence_study(my_function, method)>

## Python Modules and Functions
[Back to TOC](#toc)

Interpolation:

https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html

Vandermonde Matrix:

https://numpy.org/doc/stable/reference/generated/numpy.vander.html

Lagrange Interpolator:

https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.lagrange.html

Barycentric Interpolator:

https://people.maths.ox.ac.uk/trefethen/barycentric.pdf

https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.BarycentricInterpolator.html

Chebyshev Points for the First Kind:

https://numpy.org/doc/stable/reference/generated/numpy.polynomial.chebyshev.chebroots.html

# Acknowledgements
[Back to TOC](#toc)

* _Material created by professor Claudio Torres_ (`ctorres@inf.utfsm.cl`) _and assistants: Laura Bermeo, Alvaro Salinas, Axel Simonsen and Martín Villanueva. DI UTFSM. April 2016._
* _Material modified by Cristopher Arenas. May 2017._
* _Material modified by Claudio Torres. May 2017._
* _Bug fixed by Cristobal Carmona. April 2018._
* _Update June 2020 - v1.25 - C.Torres_ : Fixing formatting issues.
* _Update June 2020 - v1.26 - C.Torres_ : Adding "ylim" argumento to Interpolation_Plot(D,ylim=None) and addint "show_time_Li".
* _Update June 2020 - v1.27 - C.Torres_ : Adding comment that the Chebyshev nodes used are of the first kind and "Chebyshev_points_histogram".
* _Update May 2021 - v1.28 - C.Torres_ : Improving format, adding cumulative histogram in "Chebyshev_points_histogram", updating Python modules links, implementing Barycentric interpolation, changing functions names: Interpolation_Plot->PlotInterpolant, Y->EvaluateInterpolant; adding field 'n' to data structure 'D' used for building interpolant, several changes everywhere adding more explanations.
* _Update May 2021 - v1.29 - C.Torres_ : Adding "show_time_all_Li" thanks to Felipe Samur question during class. Thanks Felipe! Changing reference tu sympy from 'sp' to 'sym'.
* _Update May 2021 - v1.30 - C.Torres_ : Adding radio buttons to last examples.

```python

```


# Background

My washing machine broke. I discovered this as I had a pile of sopping wet clothes at the bottom.

And I had an idea.

My electric dryer still functions, and there was a spare bin in the corner.

# Experimental Protocol

I got the bin. As I pulled articles of clothing out of the washer, I wrung them out, collecting all the water runoff in the spare bin. I wrung each article as much as I thought reasonable, and the whole thing took about ten minutes.

After I wrung all the items and transferred them into the dryer, I weighed the bin (with water in it). Then, I dumped out as much water as I could and weighed the bin again.

# Research Questions

1. What's the approximate mass of water I wrung out?
2. If I wanted my clothes reasonably dry, how much energy did I save the dryer, given that I shunted off a bunch of water that it would no longer have to try to vaporize?
3. Given my electricity rate and the energy "savings" from wringing, how much money did I save?
4. What was my average power output for the time wringing?
5. Given the energy savings per time unit, would my time have been more profitable spent doing something else? That is, how much would I have to earn per unit time in order for the water wringing out to be a less profitable activity, and thus, have greater-than-zero opportunity cost?

# Assumptions

1. Because I lack a baseline, I'm assuming the water in the washer was about 60 degrees Fahrenheit, or about 16 degrees Centigrade

# Analysis

## What's the approximate mass of water I wrang out?

$$
\begin{equation}
\begin{split}
\Delta m & = W_{final} - W_{initial} \\
 & = 1730\textrm{g} - 1130\textrm{g} \\
 & = 600\textrm{g}
\end{split}
\end{equation}
$$

## How much energy did I save?

$$
\begin{equation}
\begin{split}
E_{saved} & = (\textrm{Energy to raise the water to 100 C}) + (\textrm{Energy to vaporize the water}) \\
    & = m \times
        \left(
           \left(
               \textrm{Specific heat of liquid water} \times \Delta T
           \right)
            +
            \left(
                \textrm{Specific heat of vaporization of water}
            \right)
        \right) \\
    & = \left(
            600 \mathrm{g}
        \right)
        \times
        \left(
            \left(
                \frac{4.181 \mathrm{J}}{\mathrm{g} \times \mathrm{^\circ K}}
                \times
                \left(
                        100^{\circ} \mathrm{C} - 16^\circ \mathrm{C}
                \right)
            \right)
        +
            \left(
                \frac{2230 \mathrm{J}}{\mathrm{g} \times \mathrm{^\circ K}}
            \right)
        \right)
\end{split}
\end{equation}
$$

```python
energy_savings = 600 * (
    (4.181 * (100 - 16))
    +
    2230
)
print(energy_savings)
```

    1548722.4000000001

## How much money did I save by wringing out my clothes

?? (you can start to figure it out now)

## What was my average power output?

What would you need to start figuring that out?

## What was the opportunity cost?

Or, how much would I have to earn at another equivalent activity to make wringing out clothes have a greater-than-zero opportunity cost in terms of electricity bill savings?

# Extra credit

What are some simplifications you can think of that the opportunity cost calculation is making? Here's one to get you started: running my dryer entails at least a little bit of wear and tear on the dryer, right? I mean, the thing can't run forever. So is the money in the opportunity cost *only* a function of how much I pay in electricity? What do you think? Argue your case.

```python

```