## Subsurface Data Analytics

### Naive Bayes Classification for Subsurface Data Analytics in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### PGE 383 Exercise: Naive Bayes Classification for Subsurface Data Analytics in Python

Here's a simple workflow, demonstration of naive Bayes classification for subsurface modeling workflows. This should help you get started with building subsurface models that with predictions based on multiple sources of information.

This method is great as it builds directly on our knowledge Bayesian statistics to provide a simple, but flexible classification method.

#### Bayesian Updating

The naive Bayes classifier is based on the conditional probability of a category, $k$, given $n$ features, $x_1, \dots , x_n$.

\begin{equation}
p(C_k | x_1, \dots , x_n)
\end{equation}

we can solve this with Bayesian updating:

\begin{equation}
p(C_k | x_1, \dots , x_n) = \frac{p(x_1, \dots , x_n | C_k) p(C_k)}{p(x_1, \dots , x_n)}
\end{equation}

let's combine the likelihood and prior for the momment:

\begin{equation}
p(x_1, \dots , x_n | C_k) p(C_k) = p(x_1, \dots , x_n, C_k)
\end{equation}

we can exand the full joint distribution recursively as follows:

\begin{equation}
p(x_1, \dots , x_n, C_k)
\end{equation}

expansion of the joint with the conditional and prior

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2, \dots , x_n, C_k)
\end{equation}

continue recursively expanding

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2 | x_3, \dots , x_n, C_k) p(x_3, \dots , x_n, C_k)
\end{equation}

we can generalize as

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2 | x_3, \dots , x_n, C_k) p(x_3 | x_4, \dots , x_n, C_k) \ldots (x_{n-1} | x_n, C_k)  (x_{n} | C_k) p(C_k)
\end{equation}

#### Naive Bayes Approach

The likelihood, conditional probability with the joint conditional is difficult to calculate.  It requires information about the joint relationship between $x_1, \dots , x_n$ features. As $n$ increases this requires a lot of data to inform the joint distribution.

With the naive bayes approach we make the 'naive' assumption that the features are all **conditionally independent**.  This entails:

\begin{equation}
p(x_i | x_{i+1}, \ldots , x_n, C_k) = p(x_i | C_k)
\end{equation}

for all $i = 1, \ldots, n$ features.

We can now solve for the needed conditional probability as:

\begin{equation}
p(C_k | x_1, \dots , x_n) = \frac{p(C_k) \prod_{i=1}^{n} p(x_i | C_k)}{p(x_1, \dots , x_n)}
\end{equation}

We only need the prior, $p(C_k)$, and a set of conditionals, $p(x_i | C_k)$, for all predictor features, $i = 1,\ldots,n$ and all categories, $k = 1,\ldots,K$.

The evidence term, $p(x_1, \dots , x_n)$, is only based on the features $x_1, \dots , x_n$; therefore, is a constant over the categories $k = 1,\ldots,n$.

* it ensures closure - probabilities over all categories sum to one

* we simply standardize the numerators to sum to one over the categories.

The naive Bayes approach is:

* simple to understand, builds on fundamental Bayesian statistics

* pratical even with small datasets since with the conditional independence we only need to estimate simple conditional distributions

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - [unconv_MV_v4.csv](https://git.io/fhHLT).

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Import Required Packages

Let's import the GeostatsPy package.  I actually don't use it in this workflow, but just incase.

```python
import geostatspy.GSLIB as GSLIB                            # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                      # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                                          # ndarrys for gridded data
import pandas as pd                                         # DataFrames for tabular data
import os                                                   # set working directory, run executables
import matplotlib.pyplot as plt                             # for plotting
from scipy import stats                                     # summary statistics
import math                                                 # trig etc.
from sklearn.model_selection import train_test_split        # train and test split
from sklearn.naive_bayes import GaussianNB                  # naive Bayes model and prediction
from sklearn import metrics                                 # measures to check our models
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Declare functions

Let's define a couple of functions to streamline plotting correlation matrices and visualization of a decision tree regression model.

```python
def plot_corr(dataframe,size=10):                               # plots a graphical correlation matrix
    corr = dataframe.corr()
    fig, ax = plt.subplots(figsize=(size, size))
    im = ax.matshow(corr,vmin = -1.0, vmax = 1.0)
    plt.xticks(range(len(corr.columns)), corr.columns);
    plt.yticks(range(len(corr.columns)), corr.columns);
    plt.colorbar(im, orientation = 'vertical')
    plt.title('Correlation Matrix')

def visualize_model(model,xfeature,yfeature,response,title,):   # plots the data points and the prediction MAPS
    n_classes = 10
    cmap = plt.cm.plasma
    plot_step = 0.02
    plt.figure(figsize=(8,6))
    x_min, x_max = min(xfeature) - 1, max(xfeature) + 1
    y_min, y_max = min(yfeature) - 1, max(yfeature) + 1
    resp_min = round(min(response)); resp_max = round(max(response));
    xx, yy = np.meshgrid(np.arange(x_min, x_max, plot_step),
                     np.arange(y_min, y_max, plot_step))
    z_min = round(min(response)); z_max = round(max(response))
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    cs = plt.contourf(xx, yy, Z, cmap=cmap,vmin=z_min, vmax=z_max)

    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=plt.cm.Greys, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title)
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar.set_label(response.name, rotation=270, labelpad=20)
    plt.subplots_adjust(left=0.0, bottom=0.0, right=0.6, top=0.8, wspace=0.2, hspace=0.2)
    plt.show()

def visualize_model_prob(model,xfeature,yfeature,response,title,):# plots the data points and the prediction probabilities
    n_classes = 10
    cmap = plt.cm.plasma
    plot_step = 0.02
    plt.figure(figsize=(8,6))
    x_min, x_max = min(xfeature) - 1, max(xfeature) + 1
    y_min, y_max = min(yfeature) - 1, max(yfeature) + 1
    resp_min = round(min(response)); resp_max = round(max(response));
    xx, yy = np.meshgrid(np.arange(x_min, x_max, plot_step),
                     np.arange(y_min, y_max, plot_step))
    z_min = 0.0; z_max = 1.0
    Z = model.predict_proba(np.c_[xx.ravel(), yy.ravel()])
    Z1 = Z[:,0].reshape(xx.shape); Z2 = Z[:,1].reshape(xx.shape)

    plt.subplot(121)
    cs1 = plt.contourf(xx, yy, Z1, cmap=cmap,vmin=z_min, vmax=z_max, levels=np.linspace(z_min, z_max, 100))
    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=plt.cm.Greys, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title + ' Probability of Low Production')
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar = plt.colorbar(cs1, orientation = 'vertical')
    cbar.set_label('Probability', rotation=270, labelpad=20)

    plt.subplot(122)
    cs2 = plt.contourf(xx, yy, Z2, cmap=cmap,vmin=z_min, vmax=z_max, levels=np.linspace(z_min, z_max, 100))
    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=plt.cm.Greys, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title + ' Probability of High Production')
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar = plt.colorbar(cs2, orientation = 'vertical')
    cbar.set_label('Probability', rotation=270, labelpad=20)
    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.0, wspace=0.2, hspace=0.2)
    plt.show()

```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")                                       # set the working directory
```

You will have to update the part in quotes with your own working directory and the format is different on a Mac (e.g. "~/PGE").

#### Read the data table

First copy the "unconv_MV.csv" comma delimited file from https://github.com/GeostatsGuy/GeoDataSets to your working directory, then run this command to read the file into a DataFrame object (part of Pandas package).

```python
my_data = pd.read_csv("unconv_MV_v4.csv")                   # load the comma delimited data file
```

Let's visualize the first several rows of our data stored in a DataFrame so we can make sure we successfully loaded the data file.

```python
my_data.head(n=13)                                          # preview the first n rows of the DataFrame
```

      Well
      Por
      Perm
      AI
      Brittle
      TOC
      VR
      Prod

      0
      1
      12.08
      2.92
      2.80
      81.40
      1.16
      2.31
      1695.360819

      1
      2
      12.38
      3.53
      3.22
      46.17
      0.89
      1.88
      3007.096063

      2
      3
      14.02
      2.59
      4.01
      72.80
      0.89
      2.72
      2531.938259

      3
      4
      17.67
      6.75
      2.63
      39.81
      1.08
      1.88
      5288.514854

      4
      5
      17.52
      4.57
      3.18
      10.94
      1.51
      1.90
      2859.469624

      5
      6
      14.53
      4.81
      2.69
      53.60
      0.94
      1.67
      4017.374438

      6
      7
      13.49
      3.60
      2.93
      63.71
      0.80
      1.85
      2952.812773

      7
      8
      11.58
      3.03
      3.25
      53.00
      0.69
      1.93
      2670.933846

      8
      9
      12.52
      2.72
      2.43
      65.77
      0.95
      1.98
      2474.048178

      9
      10
      13.25
      3.94
      3.71
      66.20
      1.14
      2.65
      2722.893266

      10
      11
      15.04
      4.39
      2.22
      61.11
      1.08
      1.77
      3828.247174

      11
      12
      16.19
      6.30
      2.29
      49.10
      1.53
      1.86
      5095.810104

      12
      13
      16.82
      5.42
      2.80
      66.65
      1.17
      1.98
      4091.637316

Let's remove the well index and check the summary summary statistics.

```python
my_data = my_data.iloc[:,1:]                                # remove the well index
my_data.describe().transpose()                              # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      200.0
      14.991150
      2.971176
      6.550000
      12.912500
      15.070000
      17.402500
      23.550000

      Perm
      200.0
      4.330750
      1.731014
      1.130000
      3.122500
      4.035000
      5.287500
      9.870000

      AI
      200.0
      2.968850
      0.566885
      1.280000
      2.547500
      2.955000
      3.345000
      4.630000

      Brittle
      200.0
      48.161950
      14.129455
      10.940000
      37.755000
      49.510000
      58.262500
      84.330000

      TOC
      200.0
      0.990450
      0.481588
      -0.190000
      0.617500
      1.030000
      1.350000
      2.180000

      VR
      200.0
      1.964300
      0.300827
      0.930000
      1.770000
      1.960000
      2.142500
      2.870000

      Prod
      200.0
      3864.407081
      1553.277558
      839.822063
      2686.227611
      3604.303507
      4752.637556
      8590.384044

It is good that we checked the summary statistics, because we have some negative values for brittleness and total organic carbon. The is physically imposible.  The values must be in error. We know the lowest possible values are 0.0, so we will truncate on 0.0.  We use the *get_numerical_data()* DataFrame member function to get a shallow copy of the data from the DataFrame.  Since it is a shallow copy, any changes we make to the copy are made to the data in the original DataFrame.  This allows us to apply this simple conditional statement to all the data values in the DataFrame all at once.

Let's also make a categorical variable for production, based on a threshold of 4,000 MCFPD.

* high production > 4,000 MCFPD, cprod = 1
* low production <= 4,000 MCFPD, cprod = 0

```python
num = my_data._get_numeric_data()                           # get shallow copy of the numerical values from the DataFrame
num[num < 0] = 0                                            # truncate negative values to 0.0
my_data['cProd'] = np.where(my_data['Prod']>=4000, 1, 0)    # conditional statement assign a new feature
my_data.describe().transpose()                              # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      200.0
      14.991150
      2.971176
      6.550000
      12.912500
      15.070000
      17.402500
      23.550000

      Perm
      200.0
      4.330750
      1.731014
      1.130000
      3.122500
      4.035000
      5.287500
      9.870000

      AI
      200.0
      2.968850
      0.566885
      1.280000
      2.547500
      2.955000
      3.345000
      4.630000

      Brittle
      200.0
      48.161950
      14.129455
      10.940000
      37.755000
      49.510000
      58.262500
      84.330000

      TOC
      200.0
      0.991950
      0.478264
      0.000000
      0.617500
      1.030000
      1.350000
      2.180000

      VR
      200.0
      1.964300
      0.300827
      0.930000
      1.770000
      1.960000
      2.142500
      2.870000

      Prod
      200.0
      3864.407081
      1553.277558
      839.822063
      2686.227611
      3604.303507
      4752.637556
      8590.384044

      cProd
      200.0
      0.435000
      0.497001
      0.000000
      0.000000
      0.000000
      1.000000
      1.000000

Let's make sure that we have the new categorical feature for production.

```python
my_data.head()                                              # preview the first n rows of the updated DataFrame
```

      Por
      Perm
      AI
      Brittle
      TOC
      VR
      Prod
      cProd

      0
      12.08
      2.92
      2.80
      81.40
      1.16
      2.31
      1695.360819
      0

      1
      12.38
      3.53
      3.22
      46.17
      0.89
      1.88
      3007.096063
      0

      2
      14.02
      2.59
      4.01
      72.80
      0.89
      2.72
      2531.938259
      0

      3
      17.67
      6.75
      2.63
      39.81
      1.08
      1.88
      5288.514854
      1

      4
      17.52
      4.57
      3.18
      10.94
      1.51
      1.90
      2859.469624
      0

This dataset has variables from 200 unconventional wells including well average porosity, log transform of permeability (to linearize the relationships with other variables), accoustic impedance (kg/m2s*10^6), brittness ratio (%), total organic carbon (%), vitrinite reflectance (%), and initial production 90 day average (MCFPD).  Note, the dataset is synthetic.

#### Calculate the correlation matrix

For multivariate analysis it is a good idea to check the correlation matrix.  We can calculate it and view it in the console with these commands.

```python
corr_matrix = np.corrcoef(my_data.iloc[:,:7], rowvar = False)  # correlation matrix without the categorical value
print(np.around(corr_matrix,2))                           # print the correlation matrix to 2 decimals
```

    [[ 1.    0.76 -0.46 -0.22  0.71  0.11  0.88]
     [ 0.76  1.   -0.24 -0.12  0.47  0.05  0.71]
     [-0.46 -0.24  1.    0.13 -0.53  0.5  -0.37]
     [-0.22 -0.12  0.13  1.   -0.21  0.32 -0.02]
     [ 0.71  0.47 -0.53 -0.21  1.    0.3   0.64]
     [ 0.11  0.05  0.5   0.32  0.3   1.    0.23]
     [ 0.88  0.71 -0.37 -0.02  0.64  0.23  1.  ]]

Note the 1.0 diagonal resulting from the correlation of each variable with themselves.

Let's use our function declared above to make a graphical correlation matrix visualization.  This may inprove our ability to spot features.  It relies on the built in correlation matrix method with Numpy DataFrames and MatPlotLib for plotting.

```python
plot_corr(my_data.iloc[:,:7],10)                            # using our correlation matrix visualization function
plt.show()
```

#### Working with Only Two Features

Let's simplify the problem to 2 feature), Porosity and Brittleness to predict Production rate.  By working with only 2 features, it is very easy to visualize the segmentation of the feature space (it is only 2D and can be shown compleltely on a single plot).

```python
my_data_subset = my_data.iloc[:,[0,3,7]]                # extract just por, brittle and prod with 100 samples
X_train, X_test, y_train, y_test = train_test_split(my_data_subset.iloc[:,[0,1]], my_data_subset.iloc[:,2], test_size=0.25, random_state=73073)
y_train = pd.DataFrame({'cprod':y_train.values})
y_test = pd.DataFrame({'cprod':y_test.values})
```

Let's first check the univariate statistics of Porosity, Brittleness and Producton.

```python
X_train.describe().transpose()                         # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      150.0
      15.005267
      2.971274
      6.55
      12.8975
      15.055
      17.500
      23.55

      Brittle
      150.0
      47.857067
      13.886701
      10.94
      37.8400
      49.150
      57.985
      81.40

```python
X_test.describe().transpose()                         # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      50.0
      14.9488
      3.00064
      7.38
      13.3525
      15.195
      16.9325
      20.96

      Brittle
      50.0
      49.0766
      14.94183
      15.68
      37.3550
      52.545
      60.0575
      84.33

```python
y_train.describe()[:2]                                    # calculate summary statistics for the data
```

      cprod

      count
      150.000000

      mean
      0.433333

```python
y_test.describe()[:2]                                    # calculate summary statistics for the data
```

      cprod

      count
      50.00

      mean
      0.44

Let's first check the univariate distributions of Porosity, Brittleness and Producton.

```python
plt.subplot(231)
plt.hist(X_train["Por"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Porosity Training Data (%)')

plt.subplot(232)
plt.hist(X_train["Brittle"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Britteness Training Data (%)')

plt.subplot(233)
plt.hist(y_train['cprod'], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Production Training Data (MCFPD)')

plt.subplot(234)
plt.hist(X_test["Por"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Porosity Testing Data (%)')

plt.subplot(235)
plt.hist(X_test["Brittle"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Britteness Testing Data (%)')

plt.subplot(236)
plt.hist(y_test['cprod'], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Production Testing Data (MCFPD)')

plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=3.2, wspace=0.2, hspace=0.2)
plt.show()
```

The distributions are well behaved, we cannot observe obvious gaps nor truncations.  Let's look at a scatter plot of Porosity vs. Brittleness with points colored by Production.

```python
plt.subplot(121)
im = plt.scatter(X_train["Por"],X_train["Brittle"],s=None, c=y_train['cprod'], marker=None, cmap=None, norm=None, vmin=None, vmax=None, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
plt.title('Training Production vs. Brittleness and Porosity'); plt.xlabel('Porosity (%)'); plt.ylabel('Brittleness (%)')
cbar = plt.colorbar(im, orientation = 'vertical')
cbar.set_label("Production", rotation=270, labelpad=20)

plt.subplot(122)
im = plt.scatter(X_test["Por"],X_test["Brittle"],s=None, c=y_test['cprod'], marker=None, cmap=None, norm=None, vmin=None, vmax=None, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
plt.title('Testing Production vs. Brittleness and Porosity'); plt.xlabel('Porosity (%)'); plt.ylabel('Brittleness (%)')
cbar = plt.colorbar(im, orientation = 'vertical')
cbar.set_label("Production", rotation=270, labelpad=20)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)

plt.show()
```

#### Instantiate, Fit and Predict with Gausian Naive Bayes

Let's build a Gaussian naive Bayes model.

We select the Gaussian model as it simplifies the inference problem to just a set of conditional means and variances given each feature.

Recall we can set a prior probability of each response category

* We will use the proportions from the training dataset.

* 0.43 for high production (the mean of the binary dataset is the proportion of 1's)

* 0.57 for low production (1 - proportion of high production)

Another option would be to assume a naive, uniform prior, substitute the following:

```python
priors = (0.5,0.5)   # naive prior
```

```python
priors = (0.57,0.43)                                        # set the prior probabilities of low and high production
```

Let's build our Gaussian naive Bayes model.

* instantiate it with the priors

* train with the training data, we use the standard fit function

```python
gnb = GaussianNB(priors = priors)                           # instantiate the Gaussian naive Bayes model
GaussianNB_fit = gnb.fit(X_train,y_train['cprod'].values)   # train with the training data
```

Let's predict with our new model over the testing dataset.

* test by predicting with the testing data, we use the standard prediction function

```python
y_pred = GaussianNB_fit.predict(np.c_[X_test['Por'].values,X_test['Brittle'].values]) # predict over the testing data
```

#### Model Checking

Let's check our model. With scikit learn we have great built in tools to evaluate our classification model. Let's try the classification report first.

```python
classification_report(truth, predicted)                # build a classification report to check our classification model
```

We get a table with summary metrics for model performance.

```python
from sklearn.metrics import classification_report
print(classification_report(y_test['cprod'].values, y_pred, labels=[0,1]))
```

                 precision    recall  f1-score   support

              0       0.96      0.93      0.95        28
              1       0.91      0.95      0.93        22

    avg / total       0.94      0.94      0.94        50

The metrics include:

    * recall - the ratio of true positives divided by all cases of the category in the testing dataset

    * precision - the ratio of true positives divided by all positives (true positives + false positives)

    * f1-score - the harmonic mean of recall and precision

    * support - the number of samples of each category in the testing data

I also like to look at the confusion matrix.

* the x axis is the prediction - category 0 or 1

* the y axis is the truth - category 0 or 1

```python
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test['cprod'].values, y_pred))
```

    [[26  2]
     [ 1 21]]

From above we can observe:

* 26 low production wells classified correctly as low production

* 1 high production well misclassified as low production

* 2 low production wells misclassified as high production

* 21 high production wells classified correctly as high production

#### Visualizing the Classification Model

Let's visualize the model over the entire feature space.

* here's the training data with the classification over the full range of predictor features.

* blue for low production and yellow for high production

Note: naive Bayes provides the posterior probability of high and low production

* the classifications below are based on maximum apriori selection (MAPS), selecting the category with the highest probability

Let's visualize the classification model (blue - low production, yellow - high production) over the predictor feature space with the training data plotted (white - low production, black - high production).

```python
visualize_model(GaussianNB_fit,X_train["Por"],X_train["Brittle"],y_train['cprod'],'Training Data and Naive Bayes Model')
```

We could also visualize the posterior probabilities of low and high production.

* here's the posterior probability of low and high production over the predictor feature space

```python
visualize_model_prob(GaussianNB_fit,X_train["Por"],X_train["Brittle"],y_train['cprod'],'Training Data and Naive Bayes Model')
```

Finally, let's look at the classification model over the predictor feature space (blue - low production, yellow - high production) with the testing data plotted (white - low production, black - high production).

```python
visualize_model(GaussianNB_fit,X_test["Por"],X_test["Brittle"],y_test['cprod'],'Testing Data and Naive Bayes Model')
```

We have a reasonable model to predict well production from porosity and brittleness for an unconventional reservoir.

#### Comments

This was a basic demonstration of naive Bayes for prediction. A lot more could be done, for example, we could have applied variants such as:

* multinomial naive Bayes

* compliment naive Bayes

* Bernoulli naive Bayes

We could have worked with more predictor features, but for learning the method, it is nice to be able to visualize the entire classification in one plot!

If you struggled with the basic Python used here check out my other basic demonstrations for DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


```python
import schemdraw
import schemdraw.elements as elm
d = schemdraw.Drawing()
d += elm.Resistor().label('100KΩ')
d += elm.Capacitor().down().label('0.1μF', loc='bottom')
d += elm.Line().left()
d += elm.Ground()
d += elm.SourceV().up().label('10V')
d.draw()
d.save('schematic.svg')
# d.save('schematic.png')
```

```python
from IPython.core.display import SVG
SVG(filename='schematic.svg')
```

```python
import schemdraw as schem
import schemdraw.elements as e

d = schem.Drawing(unit=2.5)
R7 = d.add(e.RES, d='right', botlabel='$R_7$')
R6 = d.add(e.RES, d='right', botlabel='$R_6$')
d.add(e.LINE, d='right', l=2)
d.add(e.LINE, d='right', l=2)
R5 = d.add(e.RES, d='up' , botlabel='$R_5$')
R4 = d.add(e.RES, d='up', botlabel='$R_4$')
d.add(e.LINE, d='left', l=2)
d.push()
R3 = d.add(e.RES, d='down', toy=R6.end, botlabel='$R_3$')
d.pop()
d.add(e.LINE, d='left', l=2)
d.push()
R2 = d.add(e.RES, d='down', toy=R6.end, botlabel='$R_2$')
d.pop()
R1 = d.add(e.RES, d='left', tox=R7.start, label='$R_1$')
Vt = d.add(e.BATTERY, d='up', xy=R7.start, toy=R1.end, label='$V_t$', lblofst=0.3)
d.labelI(Vt, arrowlen=1.5, arrowofst=0.5)
d.draw()
d.save('7_resistors_3_loops.svg')
#d.save('7_resistors_3_loops.pdf')
```

```python
from IPython.core.display import SVG
SVG(filename='7_resistors_3_loops.svg')
```

```python
Vt = 5.2

R1 = 0.0132
R2 = 0.021
R3 = 0.00360
R4 = 0.0152
R5 = 0.0119
R6 = 0.0022
R7 = 0.00740

```

```python
R45 = R4 + R5
R67 = R6 + R7

print(f'R45 = {round(R45,7)} Ohm, R67 = {round(R67,5)} Ohm')
```

    R45 = 0.0271 Ohm, R67 = 0.0096 Ohm

```python
d = schem.Drawing(unit=2.5)
R67 = d.add(e.RES, d='right', botlabel='$R_{67}$')
d.add(e.LINE, d='right', l=2)
d.add(e.LINE, d='right', l=2)
R45 = d.add(e.RES, d='up', botlabel='$R_{45}$')
d.add(e.LINE, d='left', l=2)
d.push()
R3 = d.add(e.RES, d='down', toy=R67.end, botlabel='$R_3$')
d.pop()
d.add(e.LINE, d='left', l=2)
d.push()
R2 = d.add(e.RES, d='down', toy=R67.end, botlabel='$R_2$')
d.pop()
R1 = d.add(e.RES, d='left', tox=R67.start, label='$R_1$')
Vt = d.add(e.BATTERY, d='up', xy=R67.start, toy=R1.end, label='$V_t$', lblofst=0.3)
d.labelI(Vt, arrowlen=1.5, arrowofst=0.5)
d.draw()
d.save('5_resistors_3_loops.svg')
#d.save('5_resistors_3_loops.pdf')
```

```python
from IPython.core.display import SVG
SVG(filename='5_resistors_3_loops.svg')
```

```python
Vt = 5.2

R1 = 0.0132
R2 = 0.021
R3 = 0.00360
R4 = 0.0152
R5 = 0.0119
R6 = 0.0022
R7 = 0.00740

R45 = R4 + R5
R67 = R6 + R7

R2345 = ((1/R2)+(1/R3)+(1/R45))**(-1)
print(f'R2345 = {round(R2345,7)} Ohm')
```

    R2345 = 0.0027602 Ohm

```python
d = schem.Drawing(unit=2.5)
R67 = d.add(e.RES, d='right', botlabel='$R_{67}$')
R345 = d.add(e.RES, d='up' , botlabel='$R_{2345}$')
R1 = d.add(e.RES, d='left', tox=R67.start, label='$R_1$')
Vt = d.add(e.BATTERY, d='up', xy=R67.start, toy=R1.end, label='$V_t$', lblofst=0.3)
d.labelI(Vt, arrowlen=1.5, arrowofst=0.5)
d.draw()
d.save('3_resistors_1_loop.svg')
#d.save('3_resistors_1_loop.pdf')
```

```python
from IPython.core.display import SVG
SVG(filename='3_resistors_1_loop.svg')
```

```python
Vt = 5.2

R1 = 0.0132
R2 = 0.021
R3 = 0.00360
R4 = 0.0152
R5 = 0.0119
R6 = 0.0022
R7 = 0.00740

R45 = R4 + R5
R67 = R6 + R7

R2345 = ((1/R2)+(1/R3)+(1/R45))**(-1)

Rt = R1 + R2345 + R67
print(f'Rt = {round(Rt,7)} Ohm')
```

    Rt = 0.0255602 Ohm

```python
d = schem.Drawing(unit=2.5)
L2 = d.add(e.LINE, d='right')
Rt = d.add(e.RES, d='up' , botlabel='$R_{t}$')
L1 = d.add(e.LINE, d='left', tox=L2.start)
Vt = d.add(e.BATTERY, d='up', xy=L2.start, toy=L1.end, label='$V_t$', lblofst=0.3)
d.labelI(Vt, arrowlen=1.5, arrowofst=0.5)
d.draw()
d.save('1_resistor_no_loops.svg')
#d.save('1_resistor_no_loops.pdf')
```

```python
from IPython.core.display import SVG
SVG(filename='1_resistor_no_loops.svg')
```

```python
Vt = 5.2

R1 = 0.0132
R2 = 0.021
R3 = 0.00360
R4 = 0.0152
R5 = 0.0119
R6 = 0.0022
R7 = 0.00740

R45 = R4 + R5
R67 = R6 + R7

R2345 = ((1/R2)+(1/R3)+(1/R45))**(-1)
Rt = R1 + R2345 + R67

It = Vt/Rt
print(f'It = {round(It,2)} A')
```

    It = 203.44 A

```python
I6 = It
I7 = It
V6 = I6 * R6
V7 = I7 * R7
print(f'V6 = {round(V6,5)} V, V7 = {round(V7,5)} V')

```

    V6 = 0.44757 V, V7 = 1.50547 V

```python
I2345 = It
V2345 = I2345 * R2345
print(f'V2345 = {round(V2345,5)} V')
```

    V2345 = 0.56153 V

```python

V3 = V2345
I3 = V3 / R3

I6 = It

print(f'I3 = {round(I3,2)} A, I6 = {round(I6,2)} A')
```

    I3 = 155.98 A, I6 = 203.44 A

```python
I7 = It
P7 = R7 * I7**2
print(f'P7 = {round(P7,2)} W')
```

    P7 = 306.27 W

```python

V45 = V2345
I45 = V45/R45
print(f'I45 = {round(I45,3)} A')

```

    I45 = 20.721 A

```python
I4 = I45
P4 = R4 * I4**2
print(f'P4 = {round(P4,4)} W')
```

    P4 = 6.5261 W

```python
print(f'V6 = {round(V6,3)} V')
print(f'V7 = {round(V7,2)} V')
print(f'I3 = {round(I3,0)} A')
print(f'I6 = {round(I6,0)} A')
print(f'P4 = {round(P4,2)} W')
print(f'P7 = {round(P7,0)} W')
```

    V6 = 0.448 V
    V7 = 1.51 V
    I3 = 156.0 A
    I6 = 203.0 A
    P4 = 6.53 W
    P7 = 306.0 W

```python
from sympy import *
init_printing(use_unicode=True)
```

[Reference](https://pythonforundergradengineers.com/circuit-diagram-problem-with-schemdraw.html?fbclid=IwAR1T560fwQeX8f6DIT86LY63gTGJS1MtjjuaVZDIgPDv-K8E3HfGOqFOt6M)

```python

```


# Introduction:

Une sonde spatiale est un véhicule spatial sans équipage lancé dans l'espace pour étudier à plus ou moins grande distance différents objets célestes et elle est amenée à franchir de grandes distances et à fonctionner loin de la Terre et du Soleil. Le facteur principale qui doit être mis en jeu afin de réussir la mission de la sonde est la précision et la complexité de la navigation.

Dans cette optique, on essayera dans ce projet de simuler la trajectoire d'une sonde spatiale dans le système solaire en intégrant les équations de mouvement de Newton, et pour ce faire on commencera d'abord dans la section 1  et 2 par simuler le système solaire et comprendre sa dynamique, et dans la section 3, on va essayer de simuler la trajectoire de la sonde spatiale New Horizons.

Dans cette première partie, on se fixe comme objectif de simuler la dynamique du système solaire en 2 dimensions.

**Remarques :**

Tout les codes de cette partie vont être dans le dossier **In_2dim**.

# 1. Simulation de système solaire:

## 1.1 Classe des objets.

Pour simplifier la tâche de manipulation des planètes et des sondes, il vaut mieux assigner à chaque objet une classe qui le caractérise avec ses propres attributs:

1. Attributs
    1. Masse
    2. Position initiale $(x_0,y_0)$
    3. Vitesse initiale $(vx_0, vy_0)$
    4. Nom de l'objet
    5. Liste des positions $(x, y)$
    7. Liste des vitesses $(vx, vy)$

On peut alors définir cette classe avec le code en bas.

(La classe est créée dans le fichier objet.py)

```python
class objet:
	""" Classe représentant les objets qui influence par la gravitation
		Attributs:
		nom
		masse: Kg
		position (x, y): au (astronomical unit)
		vitesse (v_x, v_y) : au/day
	"""

	nom = "objet"
	masse = None
	x0 = 0
	y0 = 0
	vx0 = 0
	vy0 = 0
	#Listes des positions et vitesse
	x = None
	y = None
	vx = None
	vy = None

    #Definition de constructeur
	def __init__(self, nom = "objet", masse = None, x0 = 0, y0 = 0, vx0 = 0, vy0 = 0):
		"""Constructeur de notre classe"""
		self.nom = nom
		self.masse = masse
		self.x0 = x0
		self.y0 = y0
		self.vx0 = vx0
		self.vy0 = vy0
```

**Variables Globales:**

Dans toute cette partie, on va utiliser ces variables globales qui vont servir dans les calculs prochains.

```python
#Definitions de parametres
au = 1.49597870e11 #Unité astronomique
jour = 24*3600 #Un jour

G = 6.67408e-11 #Constante gravitationelle
```

## 1.2 Equations de Newtons:

Les équations qu'on va utiliser pour décrire le mouvement des planètes et les sondes spatiales sont les équations de Newton, donc, pour décrire le mouvement d'un objet dans le système solaire qui subit une force gravitationnelle de la part du soleil, il suffit d'intégrer les équations de Newton de secondes ordres en unités internationales.

\begin{equation}
\frac{d²x}{dt²} = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} x \\
\frac{d²y}{dt²} = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} y
\end{equation}

Mais, pour des résultats plus signicatifs, il vaut mieux travailler avec des distance en $au$ (unité astronomique $:=$  distance soleil-terre) et pour le temps en $jour$, d'où les équations suivantes en unités pratiques:

\begin{equation}
\frac{d²x}{dt²} = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} x \ \frac{(day)²}{(au)³} \\
\frac{d²y}{dt²} = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} y \ \frac{(day)²}{(au)³}
\end{equation}

**Implémentation :** (dans le fichier objet.py) -> NB: Toutes les fonctions seront stockés dans le fichier "objet.py"

Pour implémenter les équations de Newton, il est préférable de définir des fonctions $fx$ et $fy$ qui prennent en argument la masse de l'objet qu'on gravite autour et les coordonnées de l'objet gravitant et qui donnent comme output l'acceleration gravitationnelle subit par l'objet suivant $\vec{x}$ et $\vec{y}$.

```python
#Definition de fonction fx(M,x,y) et fy(M,x,y)
def fx(M,x,y):
	"""
	Retourne l'acceleration gravitationnelle suivant x dû à un objet de masse M distants de l'objet étudié de x**2+y**2
	"""
	return -((G*M)/(x**2+y**2)**(3/2))*x*(jour**2/au**3)

def fy(M,x,y):
	"""
	Retourne l'acceleration gravitationnelle suivant y dû à un objet de masse M distants de l'objet étudié de x**2+y**2
	"""
	return -((G*M)/(x**2+y**2)**(3/2))*y*(jour**2/au**3)
```

## 1.3 Simulation d'interaction entre le soleil et une autre planète.

Puisque la masse de Soleil $ M_{soleil} >> M_{planète} $, on peut consider que le soleil reste fixe au cours de mouvement, donc on peut se servir des fonctions de la partie **1.2** pour intérgrer les équations de Newton, il nous faut juste une certaine condition initiale sur la position et la vitesse qu'on a pris d'ici: http://vo.imcce.fr/webservices/miriade.

Dans un premier temps, on peut faire cette première simulation pour le soleil et la terre.

Dans le code ci-dessous, on définit les conditions initiales de la terre, le pas de temps $dt$ et la période d'intégration $T$ et on rénitialise les attributs $terre.x/y$ et $terre.vx/vy$ qui vont contenir les positions et les vitesses de la terre durant toute la période d'intégration.

```python
#Pour faire des plots
import numpy as np
import matplotlib.pyplot as plt

# Definition des objets
soleil = objet("Soleil", 1.989*1e30, 0, 0, 0, 0) #(nom, masse, x, y, vx, vy)

#Données prises de http://vo.imcce.fr/webservices/miriade
terre = objet ("Terre", 5.972*1e24, -0.7528373239252,  0.6375222355089,  -0.0113914294224, -0.0131912591762)

dt = 1 #step, un jour
T = int(365/dt)*20 # Periode d'integration (Nombre de steps) -> une année * ...

#Definition des tableau de coordonnés et initiation
terre.x = np.zeros(T) ; terre.x[0] = terre.x0
terre.y = np.zeros(T) ; terre.y[0] = terre.y0

terre.vx = np.zeros(T) ; terre.vx[0] = terre.vx0
terre.vy = np.zeros(T) ; terre.vy[0] = terre.vy0
```

### 1.3.1 Comment estimer la précision des algorithmes d'intégration?

**Conservation d'énergie totale de système:**

Au cours de l'intégration des équations de mouvement, l'énergie de système doit rester conservée, donc on va utiliser comme critère de précision, la variation de l'énergie mécanique (massique) de l'objet étudié, si l'énergie mécanique reste la plus proche possible de la valeur initiale, alors la méthode d'intégration choisie est plus précise. On va calculer alors l'énergie mécanique (massique) à l'aide de la fonction $E$.

```python
def E(M, x, y, vx, vy):
	return 0.5*(vx**2+vy**2)*(au**2/jour**2)-(G*M)/(np.sqrt(x**2+y**2)*au)
E = np.vectorize(E)
```

N.B:

 * La fonction $E$ calcule l'énergie (massique) d'un objet sous effet d'un autre objet (seul) de masse M. On verra après une fonction qui permettra de calculer l'énergie d'un objet qui subit l'effet de gravitations de plusieurs autres objets.

**Comparaison des trajectoires:**

Aussi pour s'assurer des validités des calculs, on va comparer les trajectoires simulées aux données prises par les observations astrométriques.

### 1.3.2 Intégration par la méthode d'Euler:

**Méthode:**

A titre d'initiation, on va commencer par intégrer les équations de Newton à l'aide de méthode d'Euler qui consiste à faire les étapes suivantes:

$$\vec{X}_{i+1} = \vec{X_{i}} + \frac{h}{2}.\vec{V}_{i} $$

$$\vec{V}_{i+1} = \vec{V}_{i} + \frac{h}{2}.\vec{F}(\vec{X}_{i}) $$

Avec $\vec{F}$ l'accelération gravitationnelle, pour notre cas particulier de Système Soleil-Terre on a:

\begin{equation}
F_x = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} x \ \frac{(day)²}{(au)³} \\
F_y = -G \frac{M_{soleil}}{(x²+y²)^{3/2}} y \ \frac{(day)²}{(au)³}
\end{equation}

Et $h$ le pas d'intégration qui correspond à la variable $dt$.

**Implémentation:** (dans le fichier Interaction_Soleil_Planete_Euler.py)

```python
#----------------------------------------------------------------------------------------------------------

# Integration des equations de newton par methode d'euler

#-------------------------

for i in range(T-1):

    #Affectation des vitesses a l'instant i+1
	terre.vx[i+1] = terre.vx[i] + dt*fx(soleil.masse, terre.x[i], terre.y[i])
	terre.vy[i+1] = terre.vy[i] + dt*fy(soleil.masse, terre.x[i], terre.y[i])

    #Affectation des positions a l'instant i+1
	terre.x[i+1] = terre.x[i] + dt*terre.vx[i]
	terre.y[i+1] = terre.y[i] + dt*terre.vy[i]

#----------------------------------------------------------------------------------------------------
```

**Plot de trajectoire:**

```python
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot de la trajectoire simulee:
ax.plot(terre.x, terre.y) #Plot de la trajectoire simulee de la terre

plt.xlabel("y (Au)")
plt.ylabel("x (Au)")
plt.gca().set_aspect('equal', adjustable='box') #equal ratios of x and y

plt.show()
```

**Estimation précision en regardant l'énergie mécanique:**

```python
Nrg = E(soleil.masse, terre.x, terre.y, terre.vx, terre.vy) #Calcul d'energie mecanique
Nrg /= np.abs(Nrg[0])  #Pour Normaliser l'energie et pour faire un plot plus significatif

#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot d'energie en fonction de temps
t = np.linspace(1,T,T)*dt
ax.plot(t, Nrg)

ax.set_xlabel("t (jour)")
ax.set_ylabel("E/$|E_0|$")

ax.get_yaxis().get_major_formatter().set_useOffset(False) #Disable scaling of values in plot wrt y-axis

#Affichage de l'energie moyenne
print("Résultats : ")
print("Energie moyenne = " + str(np.mean(Nrg)) + ", Ecart_Type = " + str(np.std(Nrg)))

plt.show()
```

On remarque si on augmente le pas d'intégration $dt$ la précision diminue, en plus, la trajectoire simulée n'est pas fermée, par conséquent, il faut utiliser une méthode d'intégration plus précise.

### 1.3.3 Intégration par la méthode de Runge-Kutta d'ordre 2

La méthode d'Euler est une méthode dite de premier d'ordre où l'erreur d'intergration est de l'ordre $h$, donc pour encore avoir plus de précision, il faut utiliser une méthode plus précise d'où la méthode de Runge-Kutta d'ordre 2.

**Méthode:**

Soit $h$ le pas d'intégration, le méthode de Runge-Kutta d'ordre 2 consiste à faire les étapes suivantes:

$$\vec{X}_{i+1} = \vec{X_{i}} + \frac{h}{2}.\vec{V}_{i+1/2} $$

$$\vec{V}_{i+1} = \vec{V}_{i} + \frac{h}{2}.\vec{F}(\vec{X}_{i+1/2}) $$

telles que :

$$ \vec{X}_{i+1/2} = \vec{X}_{i} + \frac{h}{2}.\vec{V}_{i} $$

$$ \vec{V}_{i+1/2} = \vec{V}_{i} + \frac{h}{2}.\vec{F}(\vec{X}_{i}) $$

Avec $\vec{F}$ déjà définie dans la méthode d'Euler (1.3.2)

La particularité de cette méthode consiste à définir des variables au milieu $\vec{X}_{i+1/2}$ et $\vec{V}_{i+1/2}$ qui servent comme intermédiaires dans le calcul.

**Implémentation:** (dans le fichier Interaction_Soleil_Planete_Runge-Kutta2.py)

```python
#----------------------------------------------------------------------------------------------------------

# Integration des equations de newton par Runge Kutta2

#-------------------------

for i in range(T-1):

#Definition des variables de milieux

	vx_demi = terre.vx[i] + (dt/2)*fx(soleil.masse, terre.x[i], terre.y[i])
	vy_demi = terre.vy[i] + (dt/2)*fy(soleil.masse, terre.x[i], terre.y[i])

	x_demi = terre.x[i] + (dt/2)*terre.vx[i]
	y_demi = terre.y[i] + (dt/2)*terre.vy[i]

# Affectation des positions et vitesses à l'indice i+1

	terre.vx[i+1] = terre.vx[i] + dt*fx(soleil.masse, x_demi, y_demi)
	terre.vy[i+1] = terre.vy[i] + dt*fy(soleil.masse, x_demi, y_demi)

	terre.x[i+1] = terre.x[i] + dt*vx_demi
	terre.y[i+1] = terre.y[i] + dt*vy_demi
#----------------------------------------------------------------------------------------------------
```

**Plot de trajectoire:**

```python
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot de la trajectoire simulee:
ax.plot(terre.x, terre.y) #Plot de la trajectoire simulee de la terre

plt.xlabel("y (Au)")
plt.ylabel("x (Au)")
plt.gca().set_aspect('equal', adjustable='box') #equal ratios of x and y

plt.show()
```

**Estimation précision en regardant l'énergie mécanique:**

```python
Nrg = E(soleil.masse, terre.x, terre.y, terre.vx, terre.vy) #Calcul d'energie mecanique
Nrg /= np.abs(Nrg[0])  #Pour Normaliser l'energie et pour faire un plot plus significatif

#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot d'energie en fonction de temps
t = np.linspace(1,T,T)*dt
ax.plot(t, Nrg)

ax.set_xlabel("t (jour)")
ax.set_ylabel("E/$|E_0|$")

ax.get_yaxis().get_major_formatter().set_useOffset(False) #Disable scaling of values in plot wrt y-axis

#Affichage de l'energie moyenne
print("Résultats : ")
print("Energie moyenne = " + str(np.mean(Nrg)) + ", Ecart_Type = " + str(np.std(Nrg)))

plt.show()
```

On voit bien que le Runge-Kutta est plus précis que Euler compte tenu du fait que avec le même pas d'intégration $dt=1 \ jour$, on a une trajectoire plus précise et un écart-type de l'énergie plus petit dans le cas d'intégration par Runge-Kutta.

### 1.3.4 Intégration par Leapfrog:

Puisque, les méthodes précédentes ne conservent pas l'énérgie mécanique, il faut envisager des méthodes d'intégrations qui conservent l'énergie, ces méthodes sont dites symplectiques.

**Méthode:**

Cette méthode d'intégration comme son nom l'indique, consiste à calculer les positions et les vitesses dans des endroits différents de la manière suivante:

$$\vec{X}_{i+1} = \vec{X_{i}} + h.\vec{V}_{i+1/2} $$

$$\vec{V}_{i+3/2} = \vec{V}_{i+1/2} + h.\vec{F}(\vec{X}_{i+1}) $$

Ici, on aura besoin de $\vec{V}_{1/2}$ pour initier l'algorithme, on fait alors l'approximation suivante:

$$ \vec{V}_{1/2} = \vec{V}_0 + \frac{h}{2}.\vec{F}(\vec{X}_{0}) $$

Cette étape nous coûte un erreur de l'ordre de $h²$, ce qui est tolérable parce qu'il s'agit d'une méthode de second ordre, donc cette étape n'influe pas sur la précision globale de l'intégration.

Pour plus d'informations sur ce schéma voir les liens ci-dessous:

http://physics.ucsc.edu/~peter/242/leapfrog.pdf

https://en.wikipedia.org/wiki/Leapfrog_integration

**Implémentation:** (dans le fichier Interaction_Soleil_Planete_LeapFrog.py)

```python
#----------------------------------------------------------------------------------------------------------

# Integration des equations de newton par LeapFrog

#-------------------------

#Definition des vitesses au milieux
vx_demi = np.zeros(T); vx_demi[0] = terre.vx0 + (dt/2)*fx(soleil.masse, terre.x0, terre.y0)
vy_demi = np.zeros(T); vy_demi[0] = terre.vx0 + (dt/2)*fy(soleil.masse, terre.x0, terre.y0)

for i in range(T-1):

# Affectation des positions à l'indice i+1
	terre.x[i+1] = terre.x[i] + dt*vx_demi[i]
	terre.y[i+1] = terre.y[i] + dt*vy_demi[i]

#Affectation des vitesses:
	vx_demi[i+1] = vx_demi[i] + dt*fx(soleil.masse, terre.x[i+1], terre.y[i+1])
	vy_demi[i+1] = vy_demi[i] + dt*fy(soleil.masse, terre.x[i+1], terre.y[i+1])

    #Affecter les vitesses de la terre par celles de milieu
	terre.vx = vx_demi; terre.vy = vy_demi

```

**Plot de trajectoire:**

```python
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot de la trajectoire simulee:
ax.plot(terre.x, terre.y) #Plot de la trajectoire simulee de la terre

plt.xlabel("y (Au)")
plt.ylabel("x (Au)")
plt.gca().set_aspect('equal', adjustable='box') #equal ratios of x and y

plt.show()
```

**Estimation précision en regardant l'énergie mécanique:**

```python
Nrg = E(soleil.masse, terre.x, terre.y, terre.vx, terre.vy) #Calcul d'energie mecanique
Nrg /= np.abs(Nrg[0])  #Pour Normaliser l'energie et pour faire un plot plus significatif

#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot d'energie en fonction de temps
t = np.linspace(1,T,T)*dt
ax.plot(t, Nrg)

ax.set_xlabel("t (jour)")
ax.set_ylabel("E/$|E_0|$")

ax.get_yaxis().get_major_formatter().set_useOffset(False) #Disable scaling of values in plot wrt y-axis

#Affichage de l'energie moyenne
print("Résultats : ")
print("Energie moyenne = " + str(np.mean(Nrg)) + ", Ecart_Type = " + str(np.std(Nrg)))

plt.show()
```

Vu les résultats ci-dessus, on voit bien que Leap-Frog est un schéma d'intégration symplectique qui conserve l'énergie, mais l'incovénient de cette méthode se manifeste dans le faite que la position et la vitesse ne sont pas calculées au même instant d'où la méthode suivante:

### 1.3.5 Intégration par Verlet:

**Méthode:**

Cette méthode d'intégration est similaire à la méthode de leapfrog, mais elle permet de calculer les positions et les vitesses aux mêmes endroits, ce qui permet par exemple de tracer un portrait de phase, on pourra implémenter cette méthode de la manière suivante:

$$\vec{X}_{i+1} = \vec{X_{i}} + h.\vec{V}_{i+1/2} $$

$$\vec{V}_{i+1} = \vec{V}_{i+1/2} + \frac{h}{2}.\vec{F}(\vec{X}_{i+1}) $$

telle que :

$$ \vec{V}_{i+1/2} = \vec{V}_{i} + \frac{h}{2}.\vec{F}(\vec{X}_{i}) $$

Pour plus d'informations sur ce schéma voir ces liens:

https://en.wikipedia.org/wiki/Verlet_integration

http://www.fisica.uniud.it/~ercolessi/md/md/node21.html

**Implémentation:** (dans le fichier Interaction_Soleil_Planete_Verlet.py)

```python
#----------------------------------------------------------------------------------------------------------

# Integration des equations de newton par l'integrateur de Verlet

#-------------------------

#Il faut re-initier les vitesses à cause de la modification introduite par leapfrog
terre.vx[0] = terre.vx0; terre.vy[0] = terre.vy0

for i in range(T-1):

#Definition des variables de milieux

	vx_demi = terre.vx[i] + (dt/2)*fx(soleil.masse, terre.x[i], terre.y[i])
	vy_demi = terre.vy[i] + (dt/2)*fy(soleil.masse, terre.x[i], terre.y[i])

# Affectation des positions à l'indice i+1
	terre.x[i+1] = terre.x[i] + dt*vx_demi
	terre.y[i+1] = terre.y[i] + dt*vy_demi

	terre.vx[i+1] = vx_demi + (dt/2)*fx(soleil.masse, terre.x[i+1], terre.y[i+1])
	terre.vy[i+1] = vy_demi + (dt/2)*fy(soleil.masse, terre.x[i+1], terre.y[i+1])
#----------------------------------------------------------------------------------------------------
```

** Plot de trajectoire: **

```python
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot de la trajectoire simulee:
ax.plot(terre.x, terre.y) #Plot de la trajectoire simulee de la terre

plt.xlabel("y (Au)")
plt.ylabel("x (Au)")
plt.gca().set_aspect('equal', adjustable='box') #equal ratios of x and y

plt.show()
```

**Estimation précision en regardant l'énergie mécanique:**

```python
Nrg = E(soleil.masse, terre.x, terre.y, terre.vx, terre.vy) #Calcul d'energie mecanique
Nrg /= np.abs(Nrg[0])  #Pour Normaliser l'energie et pour faire un plot plus significatif

#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Plot d'energie en fonction de temps
t = np.linspace(1,T,T)*dt
ax.plot(t, Nrg)

ax.set_xlabel("t (jour)")
ax.set_ylabel("E/$|E_0|$")

ax.get_yaxis().get_major_formatter().set_useOffset(False) #Disable scaling of values in plot wrt y-axis

#Affichage de l'energie moyenne
print("Résultats : ")
print("Energie moyenne = " + str(np.mean(Nrg)) + ", Ecart_Type = " + str(np.std(Nrg)))

plt.show()
```

On peut conclure que la méthode de Verlet est la plus précise, vu les valeurs de l'écart-type de l'énergie, donc d'ici jusqu'à la fin de ce rapport on utilisera le schéma de Verlet comme méthode d'intégration des équations de Newton.

## 1.4 Implémentation de système solaire:

Maintenant, puisque on a compris comment évolue une seule planète autour de soleil, on va simuler la dynamique des planètes de système solaire autour de soleil.

Dans cette partie, on doit faire appel à une approche similaire mais un peu différente, parce que pour chaque planète de système solaire on doit tenir compte de la force gravitationnelle des autres planètes, donc on doit intégrer des nouvelles équations qui tiennent en compte du couplage entre les planètes.

### 1.4.1 Equations de Newton:

On va faire l'hypothèse que le soleil reste fixe à cause de sa grande masse, alors on aura:

$$
(\forall \ i \in \ [| 1,8 |] )\ ; \ \frac{d²\vec{r}_i}{dt²} = -G\sum_{j = 0 ; \ j \neq i}^{9} \frac{M_j}{||\vec{r_i}-\vec{r_j}||^{3}} (\vec{r_i}-\vec{r_j})$$

Avec:

* Objet 0: Soleil
* Objet $i$ tel que $i \ \in \ [|1,9|]$: les planètes de système solaire de Mercure à Neptune et la planète naine Pluto.

### 1.4.2 Conditions Initiales et définition de système solaire:

Une façon de stocker les positions initiales des objets consiste à les stocker dans un fichier texte qu'on peut nommer "initial_conditions_solarsystem.txt", pour des raisons techniques de Python, on va stocker les noms des objets dans un autre fichier "names_solarsystem.txt". (A cause des problèmes d'encodage on ne peut qu'importer un seul type de données avec la méthode de **numpy** : $np.genfromtxt$).

On choisit comme date de début le "2017-02-28" à "00:00 GMT".

Maintenant on a tout ce qu'il faut pour définir les objets de notre système solaire. Tout d'abord, on va créer les objets à partir de la classe **objet**.

```python
bodies = np.array([objet() for i in range(10)])  #Creation d'une liste des objets (on a au total 10 objets: soleil et 8 planetes et Pluto)
```

Après, il faut charger les données relatives aux paramètres des objets de système solaire afin d'initialiser leurs attributs.

```python
import os
os.chdir("/home/mh541/Desktop/Projet_Numerique/In_2dim") #Please change to the to the directory where 'initial_conditions_solarsystem.txt' is saved

data = np.genfromtxt("initial_conditions_solarsystem.txt", usecols=(1,2,3,4,5), skip_header=1) #On ne peut pas importer du texte avec genfromtxt
names = np.loadtxt("names_solarsystem.txt", dtype = str, skiprows=1, usecols=(1,))
```

Ici, il ne reste qu'à affecter les valeurs chargées aux attributs des objets. On définit aussi "Nbr_obj" la variable qui contient le nombre total des objets dans notre système.

```python
#Definition des parametres de chaque objet

Nbr_obj = len(bodies) #Definition de Nbr d'objets

for i in range(Nbr_obj):
	bodies[i].nom = names[i][2:-1] # [2:-1] pour supprimer les caracteres indesires
	bodies[i].masse = data[i][0]
	bodies[i].x0 = data[i][1]
	bodies[i].y0 = data[i][2]
	bodies[i].vx0 = data[i][3]
	bodies[i].vy0 = data[i][4]
```

### 1.4.3 Calcul d'accélération et d'énergie totale:

Comme approche naîve, on va tenir compte des couplages entre les autres objets autre que le soleil.
Pour simplifier la tâche de calcul d'accelération dû à la gravitation subit par un objet, ça serait mieux de définir la fonction **acceleration** qui permet de calculer cette accelération à un instant donné pour un objet donné.

(Cette fonction est dans objet.py)

```python
def acceleration(bodies, i, j):
	"""
	Calculer l'acceleration relative à un objet bodies[i]
	bodies: tous les objets
	i: index of concerned body which undergoes the gravitation of other objects.
	j: index of the step
	"""
	N = len(bodies)

	ax = 0; ay = 0 #L'acceleration

	for jp in range(N):
		#Chaque objet bodies[jp] applique une force de gravitation sur l'objet bodies[i]
		if jp == i: #On ne veut pas avoir le même objet bodies[jp]
			continue
		ax += fx(bodies[jp].masse, bodies[i].x[j]-bodies[jp].x[j], bodies[i].y[j]-bodies[jp].y[j]) #Effet du à l'objet bodies[jp]
		ay += fy(bodies[jp].masse, bodies[i].x[j]-bodies[jp].x[j], bodies[i].y[j]-bodies[jp].y[j]) #---

	return ax, ay
```

Cette fonction permet de retourner les accélérations suivant les axes x et y, en prenant comme paramètres la liste des objets $bodies$, et l'indice de l'objet concerné en plus de l'indice de pas voulu.

Pour évaluer la conservation d'énergie pendant l'intergration des équation de mouvement, c'est préférable de créer la fonction **Energy** qui permet de calculer l'énergie mécanique de chaque objet.

(Ces fonctions seront dans objet.py)

```python
#On calcule d'abord l'energie potentielle
def pot(M, x, y):
	"""
	Retourne le potentiel massique d'un objet par rapport à un autre objet de masse M et distants de x**2+y**2
	"""
	return -(G*M)/(np.sqrt(x**2+y**2)*au)

def Energy(bodies, i):
	"""
	L'Energie massique d'un objet sous l'effet d'autres objet qui lui entoure.
	"""

	N = len(bodies)

	potential = 0

	for jp in range(N):
		if jp == i:
			continue

		potential += pot(bodies[jp].masse, bodies[i].x-bodies[jp].x, bodies[i].y-bodies[jp].y)

	return 0.5*(au**2/jour**2)*(bodies[i].vx**2+bodies[i].vy**2)+potential
```

### 1.4.4 Intégrations des équations de mouvement:

Les objets maintement sont bien défini, il suffit maintenant d'intérger les équations de mouvement définit dans **1.4.1** pour avoir les trajectoires.

Puisque dans la partie précédente **1.3**, on a vu que le schéma d'intergration de Verlet est le plus précis et permet aussi de conserver l'énergie mécanique, alors on va s'en servir pour déduire les trajectoires des planètes.

**Implémentation:**

```python
#Redefinition des steps
dt = 2 #step
T = int(365/dt)*165 # (Nombre de steps)<-> Periode d'integration

#Definition des vitesses au milieu
vx_demi = np.zeros(Nbr_obj)
vy_demi = np.zeros(Nbr_obj)

#Intialisation des attributs x,y,vx,vy de chaque objet bodies[i]
for i in range(Nbr_obj):
	bodies[i].x = np.zeros(T); bodies[i].x[0] = bodies[i].x0
	bodies[i].y = np.zeros(T); bodies[i].y[0] = bodies[i].y0

	bodies[i].vx = np.zeros(T); bodies[i].vx[0] = bodies[i].vx0
	bodies[i].vy = np.zeros(T); bodies[i].vy[0] = bodies[i].vy0

#Integration a l'aide de schema de Verlet
for j in range(T-1):#A chaque pas de temps j

    #Phase 1: Calcul de vitesses milieu et affectation des position a l'intant j+1
	for i in range(1,Nbr_obj): #Modification des parametres pour chaque objet a un instant donne j

		fx_j, fy_j = acceleration(bodies, i, j) #Calcul de l'acceleration au pas j relative à l'objet i

		#Affectation des vitesses de milieu
		vx_demi[i] = bodies[i].vx[j] + (dt/2)*fx_j
		vy_demi[i] = bodies[i].vy[j] + (dt/2)*fy_j

		# Affectation des positions à l'indice j+1
		bodies[i].x[j+1] = bodies[i].x[j] + dt*vx_demi[i]
		bodies[i].y[j+1] = bodies[i].y[j] + dt*vy_demi[i]

    #Phase 2: Affectation des vitesse a l'instant j+1
	for i in range(1,Nbr_obj):

		#L'acceleration au pas j+1 relative à l'objet j
		fx_jplus1, fy_jplus1 = acceleration(bodies, i, j+1) #Il faut faire cette étape après le calcul de postion à l'indice i+1

        # Affectation des vitesses à l'indice j+1
		bodies[i].vx[j+1] = vx_demi[i] + (dt/2)*fx_jplus1
		bodies[i].vy[j+1] = vy_demi[i] + (dt/2)*fy_jplus1
```

**Plot des trajectoires:**

```python
#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

#Pour chaque objet faire un plot (Soleil non inclus)
for i in range(1,Nbr_obj):
	ax.plot(bodies[i].x, bodies[i].y, label= bodies[i].nom)

plt.xlabel("x (Au)")
plt.ylabel("y (Au)")

plt.gca().set_aspect('equal', adjustable='box') #equal ratios of x and y

plt.legend()
plt.show()
```

**Estimation précision en regardant l'énergie mécanique:**

```python
#Definition de figure
fig=plt.figure(figsize=(9, 6), dpi= 100, facecolor='w', edgecolor='k') #To modify the size of the figure
ax = fig.add_subplot(111) #definition de l'axe

Nrg = Energy(bodies, 1) #Cacul de l'energie d'un objet -> Changez le numero pour voir l'energie de chaque objet;
Nrg /= np.abs(Nrg[0])  #Pour Normaliser

#Plot de l'energie
t = np.linspace(1,T,T)*dt
ax.plot(t, Nrg)
# ax.plot(t[:365], Nrg[:365])

ax.set_xlabel("t (jour)")
ax.set_ylabel("E/$|E_0|$")

ax.get_yaxis().get_major_formatter().set_useOffset(False) #Disable scaling of values in plot wrt y-axis

#Affichage des résulats
print("Résultats : ")
print("Energie moyenne = " + str(np.mean(Nrg)) + ", Ecart_Type = " + str(np.std(Nrg)))

plt.show()
```

Pour une période d'intégration de 165 ans avec un pas de 2 jours, on observe que les planètes décrivent des trajectoires fermées avec une très bonne précision.

On remarque aussi que l'écart-type de l'énergie dimunie si on raffine le pas d'intégration $dt$, ce qui montre que l'implémentation du schéma d'intégration marche bien.

Dans la figure ci-dessus, on voit des oscillations rapides en energie de Mercure par rapport à la période totale d'intégration, pour bien voir les oscillations d'énergie mécanique (massique) de chaque planète, remplacez "ax.plot(t, Nrg)" par "ax.plot(t[:365], Nrg[:365])" dans le code au-dessus.

Pour conclure cette partie, on peut dire que notre schéma d'intégration de Verlet permet de simuler les trajectoires des planètes en **2D** avec une très bonne précision.


# Full-Waveform Inversion (FWI)

To complete the morning session we aim to highlight the core ideas behind the implementation of seismic inversion problems (where we create an image of the subsurface from field recorded data) in Devito.

## Inversion requirements

Seismic inversion relies on two known parameters:

- **Field data** - or also called **recorded data**. This is a shot record corresponding to the true velocity model. In practice this data is acquired as described in the first tutorial. In order to simplify this tutorial we will fake field data by modelling it with the true velocity model.

- **Initial velocity model**. This is a velocity model that has been obtained by processing the field data. This model is a rough and very smooth estimate of the velocity as an initial estimate for the inversion. This is a necessary requirement for any optimization (method).

## Inversion computational setup

We will introduce the gradient operator. This operator corresponds to the imaging condition introduced in the previous tutorial with some minor modifications that are defined by the objective function (also referred to in the tutorial series as the *functional*, *f*) and its gradient, *g*. We will define these two terms in the tutorial too.

## Notes on the operators

As we have already described the creation of a forward modelling operator, we will only call a wrapper function here. This wrapper already contains all the necessary operators for seismic modeling, imaging and inversion. Operators introduced for the first time in this tutorial will be properly described.

```python
import numpy as np
%matplotlib inline

from devito import configuration
configuration['log-level'] = 'WARNING'
```

## Computational considerations

As we will see, FWI is computationally extremely demanding, even more than RTM. To keep this tutorial as lightwight as possible we therefore again use a very small demonstration model. We also define here a few parameters for the final example runs that can be changed to modify the overall runtime of the tutorial.

```python
nshots = 9  # Number of shots to used to generate the gradient
nreceivers = 101  # Number of receiver locations per shot
fwi_iterations = 5  # Number of outer FWI iterations
```

# True and smooth velocity models

We will use a very simple model domain, consisting of a circle within a 2D domain. We will again use the "true" model to generate our synthetic shot data and use a "smooth" model as our initial guess. In this case the smooth model is very smooth indeed - it is simply a constant background velocity without any features.

```python
#NBVAL_IGNORE_OUTPUT
from examples.seismic import demo_model, plot_velocity, plot_perturbation

# Define true and initial model
shape = (101, 101)  # Number of grid point (nx, nz)
spacing = (10., 10.)  # Grid spacing in m. The domain size is now 1km by 1km
origin = (0., 0.)  # Need origin to define relative source and receiver locations

model = demo_model('circle-isotropic', vp_circle=3.0, vp_background=2.5,
                    origin=origin, shape=shape, spacing=spacing, nbl=40)

model0 = demo_model('circle-isotropic', vp_circle=2.5, vp_background=2.5,
                     origin=origin, shape=shape, spacing=spacing, nbl=40,
                    grid = model.grid)

plot_velocity(model)
plot_velocity(model0)
plot_perturbation(model0, model)
```

## Acquisition geometry

In this tutorial, we will use the simplest case for inversion, namely a transmission experiment. The source(s) will be located on one side of the model and the receivers on the other side. This allows most of the information necessary for inversion to be recorded, since reflections usually lead to poor inversion results.

```python
#NBVAL_IGNORE_OUTPUT
# Define acquisition geometry: source
from examples.seismic import AcquisitionGeometry

t0 = 0.
tn = 1000.
f0 = 0.010
# Position the source:
src_coordinates = np.empty((1, 2))
src_coordinates[0, 1] = np.array(model.domain_size[1]) * .5
src_coordinates[0, 0] = 20.

# Define acquisition geometry: receivers

# Initialize receivers for synthetic and imaging data
rec_coordinates = np.empty((nreceivers, 2))
rec_coordinates[:, 1] = np.linspace(0, model.domain_size[0], num=nreceivers)
rec_coordinates[:, 0] = 980.

# Geometry

geometry = AcquisitionGeometry(model, rec_coordinates, src_coordinates, t0, tn, f0=f0, src_type='Ricker')
# We can plot the time signature to see the wavelet
geometry.src.show()
```

```python
# Plot acquisition geometry
plot_velocity(model, source=geometry.src_positions,
              receiver=geometry.rec_positions[::4, :])
```

## True and smooth data

We can generate shot records for the true and smoothed initial velocity models, since the difference between them will again form the basis of our imaging procedure.

```python
# Compute synthetic data with forward operator
from examples.seismic.acoustic import AcousticWaveSolver

solver = AcousticWaveSolver(model, geometry, space_order=4)
true_d, _, _ = solver.forward(vp=model.vp)
```

```python
# Compute initial data with forward operator
smooth_d, _, _ = solver.forward(vp=model0.vp)
```

```python
#NBVAL_IGNORE_OUTPUT
from examples.seismic import plot_shotrecord

# Plot shot record for true and smooth velocity model and the difference
plot_shotrecord(true_d.data, model, t0, tn)
plot_shotrecord(smooth_d.data, model, t0, tn)
plot_shotrecord(smooth_d.data - true_d.data, model, t0, tn)
```

# Full-Waveform Inversion

## Formulation

Full-waveform inversion (FWI) aims to invert an accurate model of the discrete wave velocity, $\mathbf{c}$, or equivalently the squared slowness of the wave, $\mathbf{m} = \frac{1}{\mathbf{c}^2}$, from a given set of measurements of the pressure wavefield $\mathbf{u}$. This can be expressed as the following optimization problem [1, 2]:

\begin{aligned}
	\mathop{\hbox{minimize}}_{\mathbf{m}} \Phi_s(\mathbf{m})&=\frac{1}{2}\left\lVert\mathbf{P}_r
	\mathbf{u} - \mathbf{d}\right\rVert_2^2 \\
	\mathbf{u} &= \mathbf{A}(\mathbf{m})^{-1} \mathbf{P}_s^T \mathbf{q}_s,
\end{aligned}

where $\mathbf{P}_r$ is the sampling operator at the receiver locations, $\mathbf{P}_s^T$ is the injection operator at the source locations, $\mathbf{A}(\mathbf{m})$ is the operator representing the discretized wave equation matrix, $\mathbf{u}$ is the discrete synthetic pressure wavefield, $\mathbf{q}_s$ is the corresponding pressure source and $\mathbf{d}$ is the measured data. It is worth noting that $\mathbf{m}$ is the unknown in this formulation and that multiple implementations of the wave equation operator $\mathbf{A}(\mathbf{m})$ are possible.

We have already defined a concrete solver scheme for $\mathbf{A}(\mathbf{m})$ in the first tutorial, including appropriate implementations of the sampling operator $\mathbf{P}_r$ and source term $\mathbf{q}_s$.

To solve this optimization problem using a gradient-based method, we use the
adjoint-state method to evaluate the gradient $\nabla\Phi_s(\mathbf{m})$:

\begin{align}
 \nabla\Phi_s(\mathbf{m})=\sum_{\mathbf{t} =1}^{n_t}\mathbf{u}[\mathbf{t}] \mathbf{v}_{tt}[\mathbf{t}] =\mathbf{J}^T\delta\mathbf{d}_s,
\end{align}

where $n_t$ is the number of computational time steps, $\delta\mathbf{d}_s = \left(\mathbf{P}_r \mathbf{u} - \mathbf{d} \right)$ is the data residual (difference between the measured data and the modelled data), $\mathbf{J}$ is the Jacobian operator and $\mathbf{v}_{tt}$ is the second-order time derivative of the adjoint wavefield solving:

\begin{align}
 \mathbf{A}^T(\mathbf{m}) \mathbf{v} = \mathbf{P}_r^T \delta\mathbf{d}.
\end{align}

We see that the gradient of the FWI function is the previously defined imaging condition with an extra second-order time derivative. We will therefore reuse the operators defined previously inside a Devito wrapper.

## FWI gradient operator

To compute a single gradient $\nabla\Phi_s(\mathbf{m})$ in our optimization workflow we again use `solver.forward` to compute the entire forward wavefield $\mathbf{u}$ and a similar pre-defined gradient operator to compute the adjoint wavefield `v`. The gradient operator provided by our `solver` utility also computes the correlation between the wavefields, allowing us to encode a similar procedure to the previous imaging tutorial as our gradient calculation:

- Simulate the forward wavefield with the background velocity model to get the synthetic data and save the full wavefield $\mathbf{u}$
- Compute the data residual
- Back-propagate the data residual and compute on the fly the gradient contribution at each time step.

This procedure is applied to multiple source positions and summed to obtain a gradient image of the subsurface. We again prepare the source locations for each shot and visualize them, before defining a single gradient computation over a number of shots as a single function.

```python
#NBVAL_IGNORE_OUTPUT

# Prepare the varying source locations
source_locations = np.empty((nshots, 2), dtype=np.float32)
source_locations[:, 0] = 30.
source_locations[:, 1] = np.linspace(0., 1000, num=nshots)

plot_velocity(model, source=source_locations)
```

```python
# Create FWI gradient kernel
from devito import Function, TimeFunction
from examples.seismic import Receiver

import scipy
def fwi_gradient(vp_in):
    # Create symbols to hold the gradient and residual
    grad = Function(name="grad", grid=model.grid)
    residual = Receiver(name='rec', grid=model.grid,
                        time_range=geometry.time_axis,
                        coordinates=geometry.rec_positions)
    objective = 0.
    for i in range(nshots):
        # Update source location
        geometry.src_positions[0, :] = source_locations[i, :]

        # Generate synthetic data from true model
        true_d, _, _ = solver.forward(vp=model.vp)

        # Compute smooth data and full forward wavefield u0
        smooth_d, u0, _ = solver.forward(vp=vp_in, save=True)

        # Compute gradient from data residual and update objective function
        residual.data[:] = smooth_d.data[:] - true_d.data[:]

        objective += .5*np.linalg.norm(residual.data.flatten())**2
        solver.gradient(rec=residual, u=u0, vp=vp_in, grad=grad)

    return objective, -grad.data
```

Having defined our FWI gradient procedure we can compute the initial iteration from our starting model. This allows us to visualize the gradient alongside the model perturbation and the effect of the gradient update on the model.

```python
# Compute gradient of initial model
ff, update = fwi_gradient(model0.vp)
assert np.isclose(ff, 57283, rtol=1e0)
```

```python
#NBVAL_IGNORE_OUTPUT
from examples.seismic import plot_image

# Plot the FWI gradient
plot_image(update, vmin=-1e4, vmax=1e4, cmap="jet")

# Plot the difference between the true and initial model.
# This is not known in practice as only the initial model is provided.
plot_image(model0.vp.data - model.vp.data, vmin=-1e-1, vmax=1e-1, cmap="jet")

# Show what the update does to the model
alpha = .5 / np.abs(update).max()
plot_image(model0.vp.data - alpha*update, vmin=2.5, vmax=3.0, cmap="jet")
```

We see that the gradient and the true perturbation have the same sign, therefore, with an appropriate scaling factor, we will update the model in the correct direction.

```python
# Define bounding box constraints on the solution.
def apply_box_constraint(vp):
    # Maximum possible 'realistic' velocity is 3.5 km/sec
    # Minimum possible 'realistic' velocity is 2 km/sec
    return np.clip(vp, 2.0, 3.5)
```

```python
#NBVAL_SKIP

# Run FWI with gradient descent
history = np.zeros((fwi_iterations, 1))
for i in range(0, fwi_iterations):
    # Compute the functional value and gradient for the current
    # model estimate
    phi, direction = fwi_gradient(model0.vp)

    # Store the history of the functional values
    history[i] = phi

    # Artificial Step length for gradient descent
    # In practice this would be replaced by a Linesearch (Wolfe, ...)
    # that would guarantee functional decrease Phi(m-alpha g) <= epsilon Phi(m)
    # where epsilon is a minimum decrease constant
    alpha = .05 / np.abs(direction).max()

    # Update the model estimate and enforce minimum/maximum values
    model0.vp = apply_box_constraint(model0.vp.data - alpha * direction)

    # Log the progress made
    print('Objective value is %f at iteration %d' % (phi, i+1))
```

    Objective value is 57285.641562 at iteration 1
    Objective value is 35766.263595 at iteration 2
    Objective value is 21002.306347 at iteration 3
    Objective value is 11547.957059 at iteration 4
    Objective value is 5785.962308 at iteration 5

```python
#NBVAL_IGNORE_OUTPUT

# Plot inverted velocity model
plot_velocity(model0)
```

```python
#NBVAL_SKIP
import matplotlib.pyplot as plt

# Plot objective function decrease
plt.figure()
plt.loglog(history)
plt.xlabel('Iteration number')
plt.ylabel('Misift value Phi')
plt.title('Convergence')
plt.show()
```

## References

[1] _Virieux, J. and Operto, S.: An overview of full-waveform inversion in exploration geophysics, GEOPHYSICS, 74, WCC1–WCC26, doi:10.1190/1.3238367, http://library.seg.org/doi/abs/10.1190/1.3238367, 2009._

[2] _Haber, E., Chung, M., and Herrmann, F. J.: An effective method for parameter estimation with PDE constraints with multiple right hand sides, SIAM Journal on Optimization, 22, http://dx.doi.org/10.1137/11081126X, 2012._