# Re-creating [Capillary Hysteresis in Neutrally Wettable Fibrous Media: A Pore Network Study of a Fuel Cell Electrode](http://link.springer.com/10.1007/s11242-017-0973-2)
# Part C: Purcell Meniscus Model

## Introduction

In the final part of this series we take a deeper look at the Purcell meniscus model, which is central to the whole study. The fundamental concept is that the fibers form throats with properties similar to that of a torus:

As the fluid invades throaugh the center of the torus the meniscus is pinned to the surface and the "effective" contact angle becomes influenced by the converging diverging geometry and is a function of the filling angle $\alpha$. The shape of the meniscus as the invading phase moves upwards through the torus with key model parameters is shown below.

Different intrinsic contact angles through invading phase: (a) 60$^\circ$, (b) 90$^\circ$ and (c) 120$^\circ$. All scenarios clearly show an inflection of the meniscus curvature signifying a switch in the sign of the capillary pressure from negative to positive. This inflection is predicted to occur for all contact angles by the model with varying filling angle. The capillary pressure can be shown to be:

$P_C = -2\sigma cos(\theta-\alpha))/(r+R(1-cos(\alpha))$

## Set up
We will set up a trivially small network with one throat to demonstrate the use of the meniscus model. Here we do the imports and define a few functions for plotting.

```python
import numpy as np
import sympy as syp
import matplotlib.pyplot as plt
from ipywidgets import interact
%matplotlib inline
%pylab inline
pylab.rcParams['figure.figsize'] = (5, 5)

theta = 90
fiberRad = 5e-6
throatRad = 2e-5
max_bulge = 1e-5

# Parameters for plotting fibers
x, R, rt, s, t = syp.symbols('x, R, rt, s, t')
points = np.linspace(-fiberRad, fiberRad, 1001)
y = R*syp.sqrt(1- (x/R)**2)
r = rt + (R-y)
rx = syp.lambdify((x, R, rt), r, 'numpy')

def plot_arc(ax, arc_cen, arc_rad, arc_angle, num_points=100):
    r'''
    plot the meniscus arc from neg to pos angle
    '''
    angles = np.linspace(-arc_angle, arc_angle, num_points)
    x = arc_cen - arc_rad*np.cos(angles)
    y = arc_rad*np.sin(angles)
    print(x)
    print(y)
    ax.plot(x, y, 'b-')
```

    Populating the interactive namespace from numpy and matplotlib

Now we define our two pore network and add the meniscus model in several modes: 'max' returns the maximum pressure experienced by the meniscus as it transitions through the throat, i.e. the burst entry pressure. 'touch' is the pressure at which the meniscus has protruded past the throat center a distance defined by the 'touch_length' dictionary key. In network simulations this could be set to the pore_diameter. Finally the 'men' mode accepts a target_Pc parameter and returns all the mensicus information required for assessing cooperative filling or plotting.

```python
import openpnm as op
%config InlineBackend.figure_formats = ['svg']
import openpnm.models.physics as pm
net = op.network.Cubic(shape=[2, 1, 1], spacing=5e-5)
geo = op.geometry.SpheresAndCylinders(network=net,
                               pores=net.pores(),
                               throats=net.throats())
phase = op.phases.Water(network=net)
phase['pore.contact_angle'] = theta
phys = op.physics.Standard(network=net,
                           phase=phase,
                           geometry=geo)
geo['throat.diameter'] = throatRad*2
geo['throat.touch_length'] = max_bulge
phys.add_model(propname='throat.max',
               model=pm.meniscus.purcell,
               mode='max',
               r_toroid=fiberRad)
phys.add_model(propname='throat.touch',
               model=pm.meniscus.purcell,
               mode='touch',
               r_toroid=fiberRad)
phys.add_model(propname='throat.meniscus',
               model=pm.meniscus.purcell,
               mode='men',
               r_toroid=fiberRad,
               target_Pc=1000)
touch_Pc = phys['throat.touch'][0]
print(touch_Pc)
max_Pc = phys['throat.max'][0]
print(max_Pc)
```

    4114.018827899374
    5878.390833049412

We define a plotting function that uses the meniscus data:
$\alpha$ is filling angle as defined above,
$radius$ is the radius of curvature of the mensicus,
$center$ is the position of the centre of curvature relative to the throat center along the axis of the throat,
$\gamma$ is the angle between the throat axis and the line joining the meniscus center and meniscus contact point.

```python
def plot_phys(target_Pc, print_data=False):
    phys.add_model(propname='throat.meniscus',
               model=pm.meniscus.purcell,
               mode='men',
               r_toroid=fiberRad,
               target_Pc=target_Pc)
    throatRad = geo['throat.diameter'][0]/2
    theta = np.deg2rad(phys['pore.contact_angle'][0])
    men_data = {}
    men_data['alpha'] = phys['throat.meniscus.alpha']
    men_data['gamma'] = phys['throat.meniscus.gamma']
    men_data['radius'] = phys['throat.meniscus.radius']
    men_data['center'] = phys['throat.meniscus.center']
    fig, ax = plt.subplots()
    ax.plot(points, rx(points, fiberRad, throatRad), 'k-')
    ax.plot(points, -rx(points, fiberRad, throatRad), 'k-')
    arc_cen = men_data['center']
    arc_rad = men_data['radius']
    arc_angle = men_data['gamma']
    angles = np.linspace(-arc_angle, arc_angle, 100)
    arcx = arc_cen - arc_rad*np.cos(angles)
    arcy = arc_rad*np.sin(angles)
    ax.plot(arcx, arcy, 'b-')
    ax.axis('equal')
    ax.ticklabel_format(style='sci', axis='both', scilimits=(-6,-6))
    if print_data:
        print(men_data)
    return ax
```

We can see that the touch_Pc calculated earlier, corresponds with the tip of the meniscus exceeding the max_bulge parameter. Try changing this and re-running to see what happens.

```python
ax = plot_phys(target_Pc=touch_Pc, print_data=True);
ax.plot([max_bulge, max_bulge], [-throatRad, throatRad], 'r--');
```

    {'alpha': array([0.64350111]), 'gamma': array([0.64350111]), 'radius': array([3.5e-05]), 'center': array([-2.5e-05])}

    [<matplotlib.lines.Line2D at 0x7f79b98895b0>]

We can interact with the mensicus model by changing the target_Pc parameter.

```python
interact(plot_phys, target_Pc=(-max_Pc, max_Pc, 10));
```

    interactive(children=(FloatSlider(value=-8.390833049412322, description='target_Pc', max=5878.390833049412, mi…

    <function __main__.plot_phys(target_Pc, print_data=False)>

examples/paper_recreations/Tranter et al. (2017)/Tranter et al. (2017) - Part C.ipynb


```python
from ipywidgets import *
from sympy import *
init_printing()
from sympy.abc import a, b, c, x, y, z
from random import *
import functools
import ipywidgets as widgets

# Randomly choose between one an three elements of a list
def choose (ListOfLetters):
        picks = []
        while len(picks)<1 or len(picks)>3:
            picks = sample(range(0, len(ListOfLetters)), randint(1, len(ListOfLetters)))
        return [ListOfLetters[p] for p in picks]
# Randomly assign exponents to each value in the input list.
def assignExponents (InputList):
    exponents = sample(range(1, 5), len(InputList))
    return [l**e for (l, e) in zip(InputList, exponents)]
# Convert input expression to Latex code
def tex(expression):
    return latex(eval(str(expression)))
A = [a, b, c, x, y, z]
variables = choose(A)
varlist = assignExponents(variables)
constants = choose(list(set(set(A).difference(set(variables)))))
coefflist = assignExponents(constants)
variable = prod(varlist)
if randint(1, 6) % 3 == 0:
    coefficient = prod(coefflist) * Rational(randint(1, 6), randint(1, 6))
else:
    coefficient = prod(coefflist)
expr = coefficient * variable
var_deglist= {x:int(degree_list(x)[0]) for x in varlist}
degree = str(sum(var_deglist.values()))
question = widgets.HTMLMath(
    value=r"Treating \(" + tex(variables) + "\) as the variable in the monomial \(" + tex(expr) + "\), determine the degree and coefficient."
)
text1 = widgets.Text(
       value='',
       description='degree', )
text2 = widgets.Text(
       value='',
       description='coeffiecient', )
button = widgets.Button(description='Check Answer')
out = widgets.Output()
def on_button_clicked(_):
      # "linking function with output"
      with out:
          # what happens when we press the button
          out.clear_output()
          if text1.value== degree and eval(text2.value) == coefficient:
            print("Correct!")
          else:
            print("Incorrect")
          display(widgets.HTMLMath(
          value = r"The exponents of the terms in the variable: \(" + tex(var_deglist) +"\) sum to "+ degree +", which is the degree. Treating all other objects as constants, the coefficient is \(" + tex(str(coefficient))+"\)."
))
# linking button and function together using a button's method
button.on_click(on_button_clicked)
# displaying button and its output together
widgets.VBox([question, text1, text2, button, out])
```

    VBox(children=(HTMLMath(value='Treating \\(\\left[ z\\right]\\) as the variable in the monomial \\(a^{2} z^{2}…

```python

```

```python

```

```python

```


# Angular Acceleration and moment of inertia

- Student A
- Student B

```python
import datetime
datetime.date.today()
```

    datetime.date(2020, 11, 26)

## Abstract

The angular acceleration of an rigid body is dependent upon four factors:

- magnitude of the force
- direction of the force
- point at which the force is applied to the object
- rotational inertia of the rigid body.

This lab experiment enabled us to exampne the application of a force applied to a rigid body that is applied to a fixed axis. The magnitude of the initia was changed btween three values to observe how the magnitude of the force (weight) impacted the angular acceleration of the object.

# Introduction

During the experiment, the group hopes to examine a force that is confied to move around a particular axis. This force can be found by finding the angular acceleration through the experiment.

We can start to find the angular acceleration by deriving an equation from the large plate. After substituting values and considering the effect of friction, we can calculate the acceleration.

Fig 1 below gives an example to the basic setup of the lab. It shows a mass `m` attached to a string that goes around a rotary motion detector and is attached to a large disk. This disk is then attached to a low friction pole, which will allow the disk to rotate to give the group better results. The mass can hold different amounts of weight which produce different force which will give different torques.

Figure 2, below, gives a basic digram to the experiment. Notice how there are different diagrams for the step pulley (disk) and the motion detector.

After rotating the plates, $T_1$, or tension 1 applies torque to the plate by the step pulley. $I_M$ is the moment of inertia of the plate and $\alpha$, is the angular acceleration of the plate and $r$ is the radius of the disk.

$$\tau=T_1 r=I_M\alpha$$

At the rotary motion detector, different tensions exist on either side of the pulley. In the following equation, $I_p$ is the moment of inertia of the motion sensor, $\alpha_2$ is the angular acceleration of the motion sensor pulley, $T_1$ and $T_2$ are the tensions of the string and $R$ is the radius of the motion sensor.

$$\tau_1-\tau_2 = T_2 R - T_1 R = I_p \alpha_2$$

Where mass $m$ is suspended we can use Newton's second law where $m$ is the mass, $T_2$ is the tension in the string and $a2$ is the acceleration of the object downward.

$$mg - T_2=ma_2$$

After solving for $T_1$ and $T_2$

```python
from sympy import *
T1, r, Im, alpha1, T2, R, Ip, alpha2 = symbols("T1 r Im alpha1 T2 R Ip alpha2")
m, g, a2 = symbols("m g a2")
```

```python
T2_ = solve(Eq(m*g-T2, m*a2), T2)[0]
```

```python
T2_
```

$\displaystyle m \left(- a_{2} + g\right)$

# Procedure

yada, yada, yada,

<this is a proof of concept>

(Stuff like this would be calculated generated in python)

# Future Procedure

```python
from rhit.phys import lab
```

```python
lab.Lab1()
```


## Subsurface Data Analytics

### Naive Bayes Classification for Subsurface Data Analytics in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

### PGE 383 Exercise: Naive Bayes Classification for Subsurface Data Analytics in Python

Here's a simple workflow, demonstration of naive Bayes classification for subsurface modeling workflows. This should help you get started with building subsurface models that with predictions based on multiple sources of information.

This method is great as it builds directly on our knowledge Bayesian statistics to provide a simple, but flexible classification method.

#### Bayesian Updating

The naive Bayes classifier is based on the conditional probability of a category, $k$, given $n$ features, $x_1, \dots , x_n$.

\begin{equation}
p(C_k | x_1, \dots , x_n)
\end{equation}

we can solve this with Bayesian updating:

\begin{equation}
p(C_k | x_1, \dots , x_n) = \frac{p(x_1, \dots , x_n | C_k) p(C_k)}{p(x_1, \dots , x_n)}
\end{equation}

let's combine the likelihood and prior for the momment:

\begin{equation}
p(x_1, \dots , x_n | C_k) p(C_k) = p(x_1, \dots , x_n, C_k)
\end{equation}

we can exand the full joint distribution recursively as follows:

\begin{equation}
p(x_1, \dots , x_n, C_k)
\end{equation}

expansion of the joint with the conditional and prior

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2, \dots , x_n, C_k)
\end{equation}

continue recursively expanding

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2 | x_3, \dots , x_n, C_k) p(x_3, \dots , x_n, C_k)
\end{equation}

we can generalize as

\begin{equation}
p(x_1 | x_2, \dots , x_n, C_k) p(x_2 | x_3, \dots , x_n, C_k) p(x_3 | x_4, \dots , x_n, C_k) \ldots (x_{n-1} | x_n, C_k)  (x_{n} | C_k) p(C_k)
\end{equation}

#### Naive Bayes Approach

The likelihood, conditional probability with the joint conditional is difficult to calculate.  It requires information about the joint relationship between $x_1, \dots , x_n$ features. As $n$ increases this requires a lot of data to inform the joint distribution.

With the naive bayes approach we make the 'naive' assumption that the features are all **conditionally independent**.  This entails:

\begin{equation}
p(x_i | x_{i+1}, \ldots , x_n, C_k) = p(x_i | C_k)
\end{equation}

for all $i = 1, \ldots, n$ features.

We can now solve for the needed conditional probability as:

\begin{equation}
p(C_k | x_1, \dots , x_n) = \frac{p(C_k) \prod_{i=1}^{n} p(x_i | C_k)}{p(x_1, \dots , x_n)}
\end{equation}

We only need the prior, $p(C_k)$, and a set of conditionals, $p(x_i | C_k)$, for all predictor features, $i = 1,\ldots,n$ and all categories, $k = 1,\ldots,K$.

The evidence term, $p(x_1, \dots , x_n)$, is only based on the features $x_1, \dots , x_n$; therefore, is a constant over the categories $k = 1,\ldots,n$.

* it ensures closure - probabilities over all categories sum to one

* we simply standardize the numerators to sum to one over the categories.

The naive Bayes approach is:

* simple to understand, builds on fundamental Bayesian statistics

* pratical even with small datasets since with the conditional independence we only need to estimate simple conditional distributions

#### Objective

In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - [unconv_MV_v4.csv](https://git.io/fhHLT).

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Import Required Packages

Let's import the GeostatsPy package.  I actually don't use it in this workflow, but just incase.

```python
import geostatspy.GSLIB as GSLIB                            # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                      # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import numpy as np                                          # ndarrys for gridded data
import pandas as pd                                         # DataFrames for tabular data
import os                                                   # set working directory, run executables
import matplotlib.pyplot as plt                             # for plotting
from scipy import stats                                     # summary statistics
import math                                                 # trig etc.
from sklearn.model_selection import train_test_split        # train and test split
from sklearn.naive_bayes import GaussianNB                  # naive Bayes model and prediction
from sklearn import metrics                                 # measures to check our models
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Declare functions

Let's define a couple of functions to streamline plotting correlation matrices and visualization of a decision tree regression model.

```python
def plot_corr(dataframe,size=10):                               # plots a graphical correlation matrix
    corr = dataframe.corr()
    fig, ax = plt.subplots(figsize=(size, size))
    im = ax.matshow(corr,vmin = -1.0, vmax = 1.0)
    plt.xticks(range(len(corr.columns)), corr.columns);
    plt.yticks(range(len(corr.columns)), corr.columns);
    plt.colorbar(im, orientation = 'vertical')
    plt.title('Correlation Matrix')

def visualize_model(model,xfeature,x_min,x_max,yfeature,y_min,y_max,response,z_min,z_max,title,):# plots the data points and the decision tree prediction
    cmap = plt.cm.plasma
    xplot_step = (x_max - x_min)/300.0; yplot_step = (y_max - y_min)/300.0 # resolution of the model visualization
    xx, yy = np.meshgrid(np.arange(x_min, x_max, xplot_step), # set up the mesh
                     np.arange(y_min, y_max, yplot_step))
    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])      # predict with our trained model over the mesh
    Z = Z.reshape(xx.shape)
    cs = plt.contourf(xx, yy, Z, cmap=cmap,vmin=z_min, vmax=z_max, levels = 50) # plot the predictions
    # add the data values as a colored by response feature scatter plot
    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=cmap, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title)                                       # add the labels
    plt.xlabel(xfeature.name); plt.ylabel(yfeature.name)
    plt.xlim([x_min,x_max]); plt.ylim([y_min,y_max])
    cbar = plt.colorbar(im, orientation = 'vertical')      # add the color bar
    cbar.set_label(response.name, rotation=270, labelpad=20)
    plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.2, hspace=0.2)
    return(plt)

def visualize_model_prob(model,xfeature,x_min,x_max,yfeature,y_min,y_max,response,title,):# plots the data points and the prediction probabilities
    n_classes = 10
    cmap = plt.cm.plasma
    xplot_step = (x_max - x_min)/300.0; yplot_step = (y_max - y_min)/300.0 # resolution of the model visualization
    xx, yy = np.meshgrid(np.arange(x_min, x_max, xplot_step), # set up the mesh
                     np.arange(y_min, y_max, yplot_step))

    z_min = 0.0; z_max = 1.0
    Z = model.predict_proba(np.c_[xx.ravel(), yy.ravel()])
    Z1 = Z[:,0].reshape(xx.shape); Z2 = Z[:,1].reshape(xx.shape)

    plt.subplot(121)
    cs1 = plt.contourf(xx, yy, Z1, cmap=cmap,vmin=z_min, vmax=z_max, levels=np.linspace(z_min, z_max, 100))
    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=plt.cm.Greys, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title + ' Probability of Low Production')
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar = plt.colorbar(cs1, orientation = 'vertical')
    cbar.set_label('Probability', rotation=270, labelpad=20)

    plt.subplot(122)
    cs2 = plt.contourf(xx, yy, Z2, cmap=cmap,vmin=z_min, vmax=z_max, levels=np.linspace(z_min, z_max, 100))
    im = plt.scatter(xfeature,yfeature,s=None, c=response, marker=None, cmap=plt.cm.Greys, norm=None, vmin=z_min, vmax=z_max, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
    plt.title(title + ' Probability of High Production')
    plt.xlabel(xfeature.name)
    plt.ylabel(yfeature.name)
    cbar = plt.colorbar(cs2, orientation = 'vertical')
    cbar.set_label('Probability', rotation=270, labelpad=20)
    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.0, wspace=0.2, hspace=0.2)
    plt.show()

```

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).

```python
os.chdir("c:/PGE383")                                       # set the working directory
```

You will have to update the part in quotes with your own working directory and the format is different on a Mac (e.g. "~/PGE").

#### Read the data table

First copy the "unconv_MV.csv" comma delimited file from https://github.com/GeostatsGuy/GeoDataSets to your working directory, then run this command to read the file into a DataFrame object (part of Pandas package).

```python
my_data = pd.read_csv("unconv_MV_v4.csv")                   # load the comma delimited data file
```

Let's visualize the first several rows of our data stored in a DataFrame so we can make sure we successfully loaded the data file.

```python
my_data.head(n=13)                                          # preview the first n rows of the DataFrame
```

      Well
      Por
      Perm
      AI
      Brittle
      TOC
      VR
      Prod

      0
      1
      12.08
      2.92
      2.80
      81.40
      1.16
      2.31
      1695.360819

      1
      2
      12.38
      3.53
      3.22
      46.17
      0.89
      1.88
      3007.096063

      2
      3
      14.02
      2.59
      4.01
      72.80
      0.89
      2.72
      2531.938259

      3
      4
      17.67
      6.75
      2.63
      39.81
      1.08
      1.88
      5288.514854

      4
      5
      17.52
      4.57
      3.18
      10.94
      1.51
      1.90
      2859.469624

      5
      6
      14.53
      4.81
      2.69
      53.60
      0.94
      1.67
      4017.374438

      6
      7
      13.49
      3.60
      2.93
      63.71
      0.80
      1.85
      2952.812773

      7
      8
      11.58
      3.03
      3.25
      53.00
      0.69
      1.93
      2670.933846

      8
      9
      12.52
      2.72
      2.43
      65.77
      0.95
      1.98
      2474.048178

      9
      10
      13.25
      3.94
      3.71
      66.20
      1.14
      2.65
      2722.893266

      10
      11
      15.04
      4.39
      2.22
      61.11
      1.08
      1.77
      3828.247174

      11
      12
      16.19
      6.30
      2.29
      49.10
      1.53
      1.86
      5095.810104

      12
      13
      16.82
      5.42
      2.80
      66.65
      1.17
      1.98
      4091.637316

Let's remove the well index and check the summary summary statistics.

```python
my_data = my_data.iloc[:,1:]                                # remove the well index
my_data.describe().transpose()                              # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      200.0
      14.991150
      2.971176
      6.550000
      12.912500
      15.070000
      17.402500
      23.550000

      Perm
      200.0
      4.330750
      1.731014
      1.130000
      3.122500
      4.035000
      5.287500
      9.870000

      AI
      200.0
      2.968850
      0.566885
      1.280000
      2.547500
      2.955000
      3.345000
      4.630000

      Brittle
      200.0
      48.161950
      14.129455
      10.940000
      37.755000
      49.510000
      58.262500
      84.330000

      TOC
      200.0
      0.990450
      0.481588
      -0.190000
      0.617500
      1.030000
      1.350000
      2.180000

      VR
      200.0
      1.964300
      0.300827
      0.930000
      1.770000
      1.960000
      2.142500
      2.870000

      Prod
      200.0
      3864.407081
      1553.277558
      839.822063
      2686.227611
      3604.303507
      4752.637556
      8590.384044

It is good that we checked the summary statistics, because we have some negative values for brittleness and total organic carbon. The is physically imposible.  The values must be in error. We know the lowest possible values are 0.0, so we will truncate on 0.0.  We use the *get_numerical_data()* DataFrame member function to get a shallow copy of the data from the DataFrame.  Since it is a shallow copy, any changes we make to the copy are made to the data in the original DataFrame.  This allows us to apply this simple conditional statement to all the data values in the DataFrame all at once.

Let's also make a categorical variable for production, based on a threshold of 4,000 MCFPD.

* high production > 4,000 MCFPD, cprod = 1
* low production <= 4,000 MCFPD, cprod = 0

```python
num = my_data._get_numeric_data()                           # get shallow copy of the numerical values from the DataFrame
num[num < 0] = 0                                            # truncate negative values to 0.0
my_data['cProd'] = np.where(my_data['Prod']>=4000, 1, 0)    # conditional statement assign a new feature
my_data.describe().transpose()                              # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      200.0
      14.991150
      2.971176
      6.550000
      12.912500
      15.070000
      17.402500
      23.550000

      Perm
      200.0
      4.330750
      1.731014
      1.130000
      3.122500
      4.035000
      5.287500
      9.870000

      AI
      200.0
      2.968850
      0.566885
      1.280000
      2.547500
      2.955000
      3.345000
      4.630000

      Brittle
      200.0
      48.161950
      14.129455
      10.940000
      37.755000
      49.510000
      58.262500
      84.330000

      TOC
      200.0
      0.991950
      0.478264
      0.000000
      0.617500
      1.030000
      1.350000
      2.180000

      VR
      200.0
      1.964300
      0.300827
      0.930000
      1.770000
      1.960000
      2.142500
      2.870000

      Prod
      200.0
      3864.407081
      1553.277558
      839.822063
      2686.227611
      3604.303507
      4752.637556
      8590.384044

      cProd
      200.0
      0.435000
      0.497001
      0.000000
      0.000000
      0.000000
      1.000000
      1.000000

Let's make sure that we have the new categorical feature for production.

```python
my_data.head()                                              # preview the first n rows of the updated DataFrame
```

      Por
      Perm
      AI
      Brittle
      TOC
      VR
      Prod
      cProd

      0
      12.08
      2.92
      2.80
      81.40
      1.16
      2.31
      1695.360819
      0

      1
      12.38
      3.53
      3.22
      46.17
      0.89
      1.88
      3007.096063
      0

      2
      14.02
      2.59
      4.01
      72.80
      0.89
      2.72
      2531.938259
      0

      3
      17.67
      6.75
      2.63
      39.81
      1.08
      1.88
      5288.514854
      1

      4
      17.52
      4.57
      3.18
      10.94
      1.51
      1.90
      2859.469624
      0

This dataset has variables from 200 unconventional wells including well average porosity, log transform of permeability (to linearize the relationships with other variables), accoustic impedance (kg/m2s*10^6), brittness ratio (%), total organic carbon (%), vitrinite reflectance (%), and initial production 90 day average (MCFPD).  Note, the dataset is synthetic.

#### Calculate the correlation matrix

For multivariate analysis it is a good idea to check the correlation matrix.  We can calculate it and view it in the console with these commands.

```python
corr_matrix = np.corrcoef(my_data.iloc[:,:7], rowvar = False)  # correlation matrix without the categorical value
print(np.around(corr_matrix,2))                           # print the correlation matrix to 2 decimals
```

    [[ 1.    0.76 -0.46 -0.22  0.71  0.11  0.88]
     [ 0.76  1.   -0.24 -0.12  0.47  0.05  0.71]
     [-0.46 -0.24  1.    0.13 -0.53  0.5  -0.37]
     [-0.22 -0.12  0.13  1.   -0.21  0.32 -0.02]
     [ 0.71  0.47 -0.53 -0.21  1.    0.3   0.64]
     [ 0.11  0.05  0.5   0.32  0.3   1.    0.23]
     [ 0.88  0.71 -0.37 -0.02  0.64  0.23  1.  ]]

Note the 1.0 diagonal resulting from the correlation of each variable with themselves.

Let's use our function declared above to make a graphical correlation matrix visualization.  This may inprove our ability to spot features.  It relies on the built in correlation matrix method with Numpy DataFrames and MatPlotLib for plotting.

```python
plot_corr(my_data.iloc[:,:7],10)                            # using our correlation matrix visualization function
plt.show()
```

#### Working with Only Two Features

Let's simplify the problem to 2 feature), Porosity and Brittleness to predict Production rate.  By working with only 2 features, it is very easy to visualize the segmentation of the feature space (it is only 2D and can be shown compleltely on a single plot).

```python
my_data_subset = my_data.iloc[:,[0,3,7]]                # extract just por, brittle and prod with 100 samples
X_train, X_test, y_train, y_test = train_test_split(my_data_subset.iloc[:,[0,1]], my_data_subset.iloc[:,2], test_size=0.25, random_state=73073)
y_train = pd.DataFrame({'cprod':y_train.values})
y_test = pd.DataFrame({'cprod':y_test.values})
```

#### Set the Min and Max for Plotting

x1 and x2 are the predictor features

* x1 on x axis and x2 on the y axis for the plots below

y is the response feature

* used for the color bar

```python
x1min = 5.0; x1max = 25.0
x2min = 0.0; x2max = 100.0
ymin = 0.0; ymax = 9000.0
```

Let's first check the univariate statistics of Porosity, Brittleness and Producton.

```python
X_train.describe().transpose()                         # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      150.0
      15.005267
      2.971274
      6.55
      12.8975
      15.055
      17.500
      23.55

      Brittle
      150.0
      47.857067
      13.886701
      10.94
      37.8400
      49.150
      57.985
      81.40

```python
X_test.describe().transpose()                         # calculate summary statistics for the data
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Por
      50.0
      14.9488
      3.00064
      7.38
      13.3525
      15.195
      16.9325
      20.96

      Brittle
      50.0
      49.0766
      14.94183
      15.68
      37.3550
      52.545
      60.0575
      84.33

```python
y_train.describe()[:2]                                    # calculate summary statistics for the data
```

      cprod

      count
      150.000000

      mean
      0.433333

```python
y_test.describe()[:2]                                    # calculate summary statistics for the data
```

      cprod

      count
      50.00

      mean
      0.44

Let's first check the univariate distributions of Porosity, Brittleness and Producton.

```python
plt.subplot(231)
plt.hist(X_train["Por"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Porosity Training Data (%)')

plt.subplot(232)
plt.hist(X_train["Brittle"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Britteness Training Data (%)')

plt.subplot(233)
plt.hist(y_train['cprod'], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Production Training Data (MCFPD)')

plt.subplot(234)
plt.hist(X_test["Por"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Porosity Testing Data (%)')

plt.subplot(235)
plt.hist(X_test["Brittle"], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Britteness Testing Data (%)')

plt.subplot(236)
plt.hist(y_test['cprod'], alpha = 0.2, color = 'red', edgecolor = 'black', bins=20)
plt.title('Production Testing Data (MCFPD)')

plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=3.2, wspace=0.2, hspace=0.2)
plt.show()
```

The distributions are well behaved, we cannot observe obvious gaps nor truncations.  Let's look at a scatter plot of Porosity vs. Brittleness with points colored by Production.

```python
plt.subplot(121)
im = plt.scatter(X_train["Por"],X_train["Brittle"],s=None, c=y_train['cprod'], marker=None, cmap=None, norm=None, vmin=None, vmax=None, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
plt.title('Training Production vs. Brittleness and Porosity'); plt.xlabel('Porosity (%)'); plt.ylabel('Brittleness (%)')
cbar = plt.colorbar(im, orientation = 'vertical')
cbar.set_label("Production", rotation=270, labelpad=20)

plt.subplot(122)
im = plt.scatter(X_test["Por"],X_test["Brittle"],s=None, c=y_test['cprod'], marker=None, cmap=None, norm=None, vmin=None, vmax=None, alpha=0.8, linewidths=0.3, verts=None, edgecolors="black")
plt.title('Testing Production vs. Brittleness and Porosity'); plt.xlabel('Porosity (%)'); plt.ylabel('Brittleness (%)')
cbar = plt.colorbar(im, orientation = 'vertical')
cbar.set_label("Production", rotation=270, labelpad=20)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.2)

plt.show()
```

#### Instantiate, Fit and Predict with Gausian Naive Bayes

Let's build a Gaussian naive Bayes model.

We select the Gaussian model as it simplifies the inference problem to just a set of conditional means and variances given each feature.

Recall we can set a prior probability of each response category

* We will use the proportions from the training dataset.

* 0.43 for high production (the mean of the binary dataset is the proportion of 1's)

* 0.57 for low production (1 - proportion of high production)

Another option would be to assume a naive, uniform prior, substitute the following:

```python
priors = (0.5,0.5)   # naive prior
```

```python
priors = (1.0,0.0)                                        # set the prior probabilities of low and high production
```

Let's build our Gaussian naive Bayes model.

* instantiate it with the priors

* train with the training data, we use the standard fit function

```python
gnb = GaussianNB(priors = priors)                           # instantiate the Gaussian naive Bayes model
GaussianNB_fit = gnb.fit(X_train,y_train['cprod'].values)   # train with the training data
```

Let's predict with our new model over the testing dataset.

* test by predicting with the testing data, we use the standard prediction function

```python
y_pred = GaussianNB_fit.predict(np.c_[X_test['Por'].values,X_test['Brittle'].values]) # predict over the testing data
```

    C:\Users\17137\anaconda3\lib\site-packages\sklearn\naive_bayes.py:449: RuntimeWarning: divide by zero encountered in log
      jointi = np.log(self.class_prior_[i])

#### Model Checking

Let's check our model. With scikit learn we have great built in tools to evaluate our classification model. Let's try the classification report first.

```python
classification_report(truth, predicted)                # build a classification report to check our classification model
```

We get a table with summary metrics for model performance.

```python
from sklearn.metrics import classification_report
print(classification_report(y_test['cprod'].values, y_pred, labels=[0,1]))
```

                  precision    recall  f1-score   support

               0       0.56      1.00      0.72        28
               1       0.00      0.00      0.00        22

        accuracy                           0.56        50
       macro avg       0.28      0.50      0.36        50
    weighted avg       0.31      0.56      0.40        50

    C:\Users\17137\anaconda3\lib\site-packages\sklearn\metrics\_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
      _warn_prf(average, modifier, msg_start, len(result))

The metrics include:

    * recall - the ratio of true positives divided by all cases of the category in the testing dataset

    * precision - the ratio of true positives divided by all positives (true positives + false positives)

    * f1-score - the harmonic mean of recall and precision

    * support - the number of samples of each category in the testing data

I also like to look at the confusion matrix.

* the x axis is the prediction - category 0 or 1

* the y axis is the truth - category 0 or 1

```python
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test['cprod'].values, y_pred))
```

    [[28  0]
     [22  0]]

From above we can observe:

* 26 low production wells classified correctly as low production

* 1 high production well misclassified as low production

* 2 low production wells misclassified as high production

* 21 high production wells classified correctly as high production

#### Visualizing the Classification Model

Let's visualize the model over the entire feature space.

* here's the training data with the classification over the full range of predictor features.

* blue for low production and yellow for high production

Note: naive Bayes provides the posterior probability of high and low production

* the classifications below are based on maximum apriori selection (MAPS), selecting the category with the highest probability

Let's visualize the classification model (blue - low production, yellow - high production) over the predictor feature space with the training data plotted (white - low production, black - high production).

```python
visualize_model(GaussianNB_fit,X_train["Por"],x1min,x1max,X_train["Brittle"],x2min,x2max,y_train['cprod'],0.0,1.0,'Training Data and Naive Bayes Model')
```

We could also visualize the posterior probabilities of low and high production.

* here's the posterior probability of low and high production over the predictor feature space

```python
visualize_model_prob(GaussianNB_fit,X_train["Por"],x1min,x1max,X_train["Brittle"],x2min,x2max,y_train['cprod'],'Training Data and Naive Bayes Model')
```

Finally, let's look at the classification model over the predictor feature space (blue - low production, yellow - high production) with the testing data plotted (white - low production, black - high production).

```python
visualize_model(GaussianNB_fit,X_test["Por"],x1min,x1max,X_test["Brittle"],x2min,x2max,y_test['cprod'],0.0,1.0,'Testing Data and Naive Bayes Model')
```

We have a reasonable model to predict well production from porosity and brittleness for an unconventional reservoir.

#### Comments

This was a basic demonstration of naive Bayes for prediction. A lot more could be done, for example, we could have applied variants such as:

* multinomial naive Bayes

* compliment naive Bayes

* Bernoulli naive Bayes

We could have worked with more predictor features, but for learning the method, it is nice to be able to visualize the entire classification in one plot!

If you struggled with the basic Python used here check out my other basic demonstrations for DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


# Introduction to Deep Learning with PyTorch

Neste caderno, você será apresentado ao [PyTorch] (http://pytorch.org/), uma estrutura para construir e treinar redes neurais. O PyTorch se comporta de várias maneiras como as matrizes que você ama do Numpy. Afinal, essas matrizes Numpy são apenas tensores. O PyTorch pega esses tensores e facilita a sua transferência para GPUs para o processamento mais rápido necessário ao treinar redes neurais. Ele também fornece um módulo que calcula automaticamente gradientes (para retropropagação!) E outro módulo especificamente para a construção de redes neurais. No conjunto, o PyTorch acaba sendo mais coerente com o Python e a pilha Numpy / Scipy em comparação com o TensorFlow e outras estruturas.

## Neural Networks

O Deep Learning é baseado em redes neurais artificiais que existem de alguma forma desde o final da década de 1950. As redes são construídas a partir de partes individuais que se aproximam dos neurônios, normalmente chamadas de unidades ou simplesmente "neurônios". Cada unidade possui algum número de entradas ponderadas. Essas entradas ponderadas são somadas (uma combinação linear) e depois passadas por uma função de ativação para obter a saída da unidade.

Matematicamente, isso se parece com:

$$
\begin{align}
y &= f(w_1 x_1 + w_2 x_2 + b) \\
y &= f\left(\sum_i w_i x_i +b \right)
\end{align}
$$

Com vetores, este é o produto interno / ponto de dois vetores:

$$
h = \begin{bmatrix}
x_1 \, x_2 \cdots  x_n
\end{bmatrix}
\cdot
\begin{bmatrix}
           w_1 \\
           w_2 \\
           \vdots \\
           w_n
\end{bmatrix}
$$

## Tensors

Acontece que os cálculos de redes neurais são apenas um monte de operações de álgebra linear em * tensores *, uma generalização de matrizes. Um vetor é um tensor unidimensional, uma matriz é um tensor bidimensional, uma matriz com três índices é um tensor tridimensional (imagens em cores RGB, por exemplo). A estrutura de dados fundamental para redes neurais são os tensores e o PyTorch (assim como praticamente qualquer outra estrutura de aprendizado profundo) é construído em torno dos tensores.

Com o básico abordado, é hora de explorar como podemos usar o PyTorch para construir uma rede neural simples.

```python
# First, import PyTorch
import torch
```

```python
def activation(x):
    """ Sigmoid activation function

        Arguments
        ---------
        x: torch.Tensor
    """
    return 1/(1+torch.exp(-x))
```

```python
### Generate some data --Gere alguns dados
torch.manual_seed(7) # Set the random seed so things are predictable -- Defina a semente aleatória para que as coisas sejam previsíveis

# Features are 5 random normal variables -- Os recursos são 5 variáveis normais aleatórias
features = torch.randn((1, 5))
# True weights for our data, random normal variables again -- Pesos reais para nossos dados, variáveis normais aleatórias novamente
weights = torch.randn_like(features)
# and a true bias term -- e um termo verdadeiro viés
bias = torch.randn((1, 1))
```

Acima, geramos dados que podemos usar para obter a saída de nossa rede simples. Por enquanto, tudo é aleatório. No futuro, começaremos a usar dados normais. Passando por cada linha relevante:

`features = torch.randn ((1, 5))` cria um tensor com a forma `(1, 5)`, uma linha e cinco colunas, que contém valores distribuídos aleatoriamente de acordo com a distribuição normal, com média de zero e padrão desvio de um.

`pesos = torch.randn_like (features)` cria outro tensor com a mesma forma que `features`, novamente contendo valores de uma distribuição normal.

Finalmente, `bias = torch.randn ((1, 1))` cria um valor único a partir de uma distribuição normal.

Os tensores PyTorch podem ser adicionados, multiplicados, subtraídos, etc., assim como os arrays Numpy. Em geral, você usará os tensores PyTorch da mesma maneira que usaria as matrizes Numpy. Eles trazem alguns benefícios interessantes, como a aceleração da GPU, que veremos mais adiante. Por enquanto, use os dados gerados para calcular a saída dessa rede simples de camada única.
> ** Exercício **: Calcule a saída da rede com recursos de entrada `recursos`, pesos` pesos` e bias `bias`. Semelhante ao Numpy, o PyTorch possui a função [`torch.sum ()`] (https://pytorch.org/docs/stable/torch.html#torch.sum), além do método `.sum ()` em tensores, para somar somas. Use a função `ativação 'definida acima como a função de ativação.

```python
## Calcule a saída desta rede usando os pesos e tensores de polarização
y = activation(torch.sum(features *weights) + bias)
```

Você pode fazer a multiplicação e somar na mesma operação usando uma multiplicação de matrizes. Em geral, convém usar multiplicações de matriz, pois elas são mais eficientes e aceleradas usando bibliotecas modernas e computação de alto desempenho em GPUs.

Aqui, queremos fazer uma multiplicação matricial dos recursos e pesos. Para isso, podemos usar [`torch.mm ()`] (https://pytorch.org/docs/stable/torch.html#torch.mm) ou [`torch.matmul ()`] (https: // pytorch.org/docs/stable/torch.html#torch.matmul), que é um pouco mais complicado e suporta transmissão. Se tentarmos fazê-lo com os recursos e pesos como eles são, obteremos um erro

```python
>> torch.mm(features, weights)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-13-15d592eb5279> in <module>()
----> 1 torch.mm(features, weights)

RuntimeError: size mismatch, m1: [1 x 5], m2: [1 x 5] at /Users/soumith/minicondabuild3/conda-bld/pytorch_1524590658547/work/aten/src/TH/generic/THTensorMath.c:2033
```

Ao criar redes neurais em qualquer estrutura, você verá isso com frequência. Realmente frequentemente. O que está acontecendo aqui é que nossos tensores não têm as formas corretas para realizar uma multiplicação de matrizes. Lembre-se de que para multiplicações de matrizes, o número de colunas no primeiro tensor deve ser igual ao número de linhas na segunda coluna. Ambos os `recursos` e` pesos` têm a mesma forma, `(1, 5)`. Isso significa que precisamos alterar a forma dos pesos para que a multiplicação da matriz funcione.

**Nota:** Para ver a forma de um tensor chamado `tensor`, use` tensor.shape`. Se você estiver construindo redes neurais, estará usando esse método frequentemente.

Existem algumas opções aqui: [`weights.reshape ()`] (https://pytorch.org/docs/stable/tensors.html#torch.Tensor.reshape), [`weights.resize _ ()`] ( https://pytorch.org/docs/stable/tensors.html#torch.Tensor.resize_) e [`weights.view ()`] (https://pytorch.org/docs/stable/tensors.html# tocha.Tensor.view).

* `weights.reshape (a, b)` retornará um novo tensor com os mesmos dados que `pesos com tamanho` `(a, b)`as vezes e às vezes um clone, pois ele copia os dados para outra parte do memória.
* `weights.resize_ (a, b)` retorna o mesmo tensor com uma forma diferente. No entanto, se a nova forma resultar em menos elementos que o tensor original, alguns elementos serão removidos do tensor (mas não da memória). Se a nova forma resultar em mais elementos que o tensor original, novos elementos serão não inicializados na memória. Aqui, devo observar que o sublinhado no final do método indica que esse método é realizado ** no local **. Aqui está um ótimo tópico do fórum para [leia mais sobre operações no local] (https://discuss.pytorch.org/t/what-is-in-place-operation/16244) no PyTorch.
* `weights.view (a, b)` retornará um novo tensor com os mesmos dados que `weights` com tamanho` (a, b) `.

Eu normalmente uso `.view ()`, mas qualquer um dos três métodos funcionará para isso. Portanto, agora podemos remodelar `pesos` 'para ter cinco linhas e uma coluna com algo como` pesos.view (5, 1) `.

> **Exercício**: Calcule a saída de nossa pequena rede usando multiplicação de matrizes.

```python
## Calculate the output of this network using matrix multiplication
print(features.shape)
print(weights.shape)

y = activation(torch.mm(features, weights.view(5,1)) + bias)
print(y)

```

    torch.Size([1, 5])
    torch.Size([1, 5])
    tensor([[0.1595]])

### Stack them up!

É assim que você pode calcular a saída para um único neurônio. O poder real desse algoritmo acontece quando você começa a empilhar essas unidades individuais em camadas e pilhas de camadas, em uma rede de neurônios. A saída de uma camada de neurônios se torna a entrada para a próxima camada. Com várias unidades de entrada e unidades de saída, agora precisamos expressar os pesos como uma matriz.

A primeira camada mostrada na parte inferior aqui são as entradas, compreensivelmente chamadas de **camada de entrada**. A camada do meio é chamada de **camada oculta** e a camada final (à direita) é a **camada de saída**. Podemos expressar essa rede matematicamente com matrizes novamente e usar a multiplicação de matrizes para obter combinações lineares para cada unidade em uma operação. Por exemplo, a camada oculta ($ h_1 $ e $ h_2 $ aqui) pode ser calculada

$$
\vec{h} = [h_1 \, h_2] =
\begin{bmatrix}
x_1 \, x_2 \cdots \, x_n
\end{bmatrix}
\cdot
\begin{bmatrix}
           w_{11} & w_{12} \\
           w_{21} &w_{22} \\
           \vdots &\vdots \\
           w_{n1} &w_{n2}
\end{bmatrix}
$$

A saída para esta pequena rede é encontrada tratando a camada oculta como entradas para a unidade de saída. A saída da rede é expressa simplesmente

$$
y =  f_2 \! \left(\, f_1 \! \left(\vec{x} \, \mathbf{W_1}\right) \mathbf{W_2} \right)
$$

```python
### Generate some data
torch.manual_seed(7) # Defina a semente aleatória para que as coisas sejam previsíveis

# Os recursos são três variáveis normais aleatórias
features = torch.randn((1, 3))

# Defina o tamanho de cada camada em nossa rede
n_input = features.shape[1]     # Número de unidades de entrada, deve corresponder ao número de recursos de entrada
print(n_input)
n_hidden = 2                    # Number of hidden units
n_output = 1                    # Number of output units

# Pesos para entradas na camada oculta
W1 = torch.randn(n_input, n_hidden)
# Pesos da camada oculta para a camada de saída
W2 = torch.randn(n_hidden, n_output)

# e termos de polarização para camadas ocultas e de saída
B1 = torch.randn((1, n_hidden))
B2 = torch.randn((1, n_output))
```

    3

> **Exercise:** Calcule a saída desta rede multicamada usando os pesos `W1` e` W2` e os desvios, `B1` e` B2`.

```python
## Your solution here
h = activation(torch.mm(features, W1) + B1)
output = activation(torch.mm(h, W2) + B2)
print(output)
```

    tensor([[0.3171]])

Se você fez isso corretamente, deverá ver a saída `tensor ([[0,3171]])`.

O número de unidades ocultas é um parâmetro da rede, geralmente chamado de **hyperparameter** para diferenciá-lo dos parâmetros de pesos e desvios. Como você verá mais adiante, quando discutirmos o treinamento de uma rede neural, quanto mais unidades ocultas uma rede tiver e mais camadas, maior será a capacidade de aprender com os dados e fazer previsões precisas.

## Numpy to Torch and back

Seção de bônus especial! O PyTorch possui um ótimo recurso para converter entre matrizes Numpy e Torch tensors. Para criar um tensor a partir de uma matriz Numpy, use `torch.from_numpy ()`. Para converter um tensor em uma matriz Numpy, use o método `.numpy ()`.

```python
import numpy as np
a = np.random.rand(4,3)
a
```

    array([[0.24067663, 0.47803715, 0.7864128 ],
           [0.42600446, 0.3888375 , 0.13350729],
           [0.74851001, 0.82425433, 0.96667231],
           [0.36474919, 0.85049331, 0.01584525]])

```python
b = torch.from_numpy(a)
b
```

    tensor([[0.2407, 0.4780, 0.7864],
            [0.4260, 0.3888, 0.1335],
            [0.7485, 0.8243, 0.9667],
            [0.3647, 0.8505, 0.0158]], dtype=torch.float64)

```python
b.numpy()
```

    array([[0.24067663, 0.47803715, 0.7864128 ],
           [0.42600446, 0.3888375 , 0.13350729],
           [0.74851001, 0.82425433, 0.96667231],
           [0.36474919, 0.85049331, 0.01584525]])

A memória é compartilhada entre a matriz Numpy e o Torch tensor, portanto, se você alterar os valores no lugar de um objeto, o outro também será alterado.

```python
# Multiply PyTorch Tensor by 2, in place -- Multiplique o PyTorch Tensor por 2, no lugar
b.mul_(2)
```

    tensor([[0.4814, 0.9561, 1.5728],
            [0.8520, 0.7777, 0.2670],
            [1.4970, 1.6485, 1.9333],
            [0.7295, 1.7010, 0.0317]], dtype=torch.float64)

```python
# Numpy array matches new values from Tensor -- A matriz numpy corresponde a novos valores do tensor
a
```

    array([[0.48135326, 0.9560743 , 1.57282561],
           [0.85200891, 0.77767499, 0.26701457],
           [1.49702002, 1.64850866, 1.93334462],
           [0.72949838, 1.70098661, 0.0316905 ]])

intro-to-pytorch/Part 1 - Tensors in PyTorch (Exercises).ipynb


```
try: #if the package is installed or placed in same directory
    import scamp as sc #import master-project package

except ImportError: #check upper directory for scamp package.
    try:
        import sys
        sys.path.append("..")
        import scamp as sc
    except: #could not find scamp package
        sys.exit("scamp is not installed or placed in the correct directory")

import numpy as np
```

# Basic Introduction

## Create a world!

#### Define dimensions for grid and time.

```
dim = sc.dim() #Define dimensions for grid and time.
```

#### Define world with desired dimensions

```
world = sc.world(dim) #Creates a grid including Ocean, Ice and Atmosphere
```

#### Print the world

```
world.print_all()
```

    T
    [[[-1. -1.  4.  1.]]]

    S
    [[[ 32.  32.  32.  35.]]]

    V
    [[ 1.]]

## Make a simulation

```
world = sc.ode_solver.forward_euler(world) #forward euler with timesteps from dim.
```

## Plot the results

```
world.plot_all()
```

# Introduction to Dim()

### Create a world with a 4x3 grid

```
dim = sc.dim(n_x = 4, n_y = 3)
```

```
world = sc.world(dim)
```

```
world.print_layer('ML')
world.print_layer('ice')
```

    T
    [[-1. -1. -1.]
     [-1. -1. -1.]
     [-1. -1. -1.]
     [-1. -1. -1.]]

    S
    [[ 32.  32.  32.]
     [ 32.  32.  32.]
     [ 32.  32.  32.]
     [ 32.  32.  32.]]

    V
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]

### Change the time scale to 10-years with a stepsize of 1/100 year

```
dim = sc.dim(years=10, dt = 1/100.)
```

*Both the timescale $years$ and stepsize $dt$ are defined in years

# Introduction to Ocean(), ice() and atmos()

## Define Ocean-, Ice- and Atmosphere-module separately with custom parameters

```
dim = sc.dim()

ocean = sc.ocean(dim, layers = 2, T_start = [1, 20], S_start = [33, 34]) #

atmos = sc.atmos(dim, Q_lw_start = 310)

ice = sc.ice(dim, I_export = 0)

world = sc.world(dim, ocean_start = ocean, atmos_start = atmos, ice_start = ice)
```

## Access variables and parameters

#### Ocean

```
world.ocean.T #temperature
```

    array([[[  1.,  20.]]])

#### Ice-layer

```
world.ice.x_ice #fraction of gridpoint covered in ice
```

    array([[ 0.5]])

#### Atmosphere

```
world.atmos.Q_lw #long-wave raditation
```

    array([[ 310.]])

## Add variables and parameters

```
dim = sc.dim()
world = sc.world(dim)
```

#### Add parameter independent of grid

```
world.ocean.fish_heat = 4 #heat capacity of a fish
```

#### Add grid dependent variable

```
world.ocean.fish = world.ocean.new_value(start = 0) #
```

#### Define differential equation for a variable

\begin{equation}
\frac{d \text{fish}}{dt} = C_{fish} X, \quad X \in U(-1,1)
\end{equation}

```
def fish_ode(world, ode):
    ode[world.ocean, 'fish'] = world.ocean.fish_heat * np.random.randint(-1,1)
    return ode

world.ocean.ode_func_extra = fish_ode
```

#### Run ode-solver

```
world = sc.ode_solver.forward_euler(world)
```

```

```


# Exercises

## Problem 1: Determine if equations are nonlinear or not

Classify each term in the following equations as linear or nonlinear.
Assume that $u$, $\u$, and $p$ are unknown functions and that
all other symbols are known quantities.

1. $mu^{\prime\prime} + \beta |u^{\prime}|u^{\prime} + cu = F(t)$

2. $u_t = \dfc u_{xx}$

3. $u_{tt} = c^2\nabla^2 u$

4. $u_t = \nabla\cdot(\dfc(u)\nabla u) + f(x,y)$

5. $u_t + f(u)_x = 0$

6. $\u_t + \u\cdot\nabla \u = -\nabla p + r\nabla^2\u$, $\nabla\cdot\u = 0$
   ($\u$ is a vector field)

7. $u^{\prime} = f(u,t)$

8. $\nabla^2 u = \lambda e^u$

**Solution.**
1. $mu^{\prime\prime}$ is linear; $\beta |u^{\prime}|u^{\prime}$ is
   nonlinear; $cu$ is linear; $F(t)$ does not contain the unknown $u$
   and is hence constant in $u$, so the term is linear.

2. $u_t$ is linear; $\dfc u_{xx}$ is linear.

3. $u_{tt}$ is linear; $c^2\nabla^2 u$ is linear.

4. $u_t$ is linear; $\nabla\cdot(\dfc(u)\nabla u)$ is nonlinear;
   $f(x,y)$ is constant in $u$ and hence linear.

5. $u_t$ is linear;  $f(u)_x$ is nonlinear if $f$ is nonlinear in $u$.

6. $\u_t$ is linear; $\u\cdot\nabla \u$ is nonlinear;
   $-\nabla p$ is linear (in $p$);  $r\nabla^2\u$ is linear;
   $\nabla\cdot\u$ is linear.

7. $u^{\prime}$ is linear;  $f(u,t)$ is nonlinear if $f$ is nonlinear in $u$.

8. $\nabla^2 u$ is linear; $\lambda e^u$ is nonlinear.

Filename: `nonlinear_vs_linear`.

## Problem 2: Derive and investigate a generalized logistic model

The logistic model for population growth is derived by assuming
a nonlinear growth rate,

$$
\begin{equation}
u^{\prime} = a(u)u,\quad u(0)=I,
\label{nonlin:exer:logistic:gen:eq} \tag{1}
\end{equation}
$$

and the logistic model arises from the simplest possible choice of
$a(u)$: $r(u)=\varrho(1 - u/M)$, where $M$ is the maximum value of $u$ that
the environment can sustain, and $\varrho$ is the growth under
unlimited access to resources (as in the beginning when $u$ is small).
The idea is that $a(u)\sim\varrho$ when $u$ is small and that
$a(t)\rightarrow 0$ as $u\rightarrow M$.

An $a(u)$ that generalizes the linear choice is the polynomial form

$$
\begin{equation}
a(u) = \varrho(1-u/M)^p,
\label{nonlin:exer:logistic:gen:r1} \tag{2}
\end{equation}
$$

where $p>0$ is some real number.

**a)**
Formulate a Forward Euler, Backward Euler, and a Crank-Nicolson
scheme for ([1](#nonlin:exer:logistic:gen:eq)).

**Hint.**
Use a geometric mean approximation in the Crank-Nicolson scheme:
$[a(u)u]^{n+1/2}\approx a(u^n)u^{n+1}$.

**Solution.**
The Forward Euler scheme reads

$$
[D^+_t u = a(u)u]^n,
$$

or written out,

$$
\frac{u^{n+1}-u^n}{\Delta t} = a(u^n)u^n\thinspace .
$$

The scheme is linear in the unknown $u^{n+1}$:

$$
u^{n+1} = u^n + {\Delta t}a(u^n)u^n\thinspace .
$$

The Backward Euler scheme,

$$
[D^-_t u = a(u)u]^n,
$$

becomes

$$
\frac{u^{n}-u^{n-1}}{\Delta t} = a(u^n)u^n,
$$

which is a nonlinear equation in the unknown $u$, here expressed as
$u^{n+1}$:

$$
u^{n+1} - \Delta t a(u^{n+1})u^{n+1} = u^n\thinspace .
$$

The standard Crank-Nicolson scheme,

$$
D_t u = \overline{a(u)u}^t]^{n+\frac{1}{2}},
$$

takes the form

$$
\frac{u^{n+1}-u^n}{\Delta t} = \frac{1}{2} a(u^n)u^n + \frac{1}{2} a(u^{n+1})u^{n+1}\thinspace .
$$

This is a nonlinear equation in the unknown $u^{n+1}$,

$$
u^{n+1} - \frac{1}{2} {\Delta t} a(u^{n+1})u^{n+1}
= u^n + \frac{1}{2} {\Delta t} a(u^n)u^n\thinspace .
$$

However, with the suggested geometric mean, the $a(u)u$ term is
linearized:

$$
\frac{u^{n+1}-u^n}{\Delta t} = a(u^n)u^{n+1},
$$

leading to a linear equation in $u^{n+1}$:

$$
(1 - \Delta t a(u^n))u^{n+1} = u^n\thinspace .
$$

**b)**
Formulate Picard and Newton iteration for the Backward Euler scheme in a).

**Solution.**
A Picard iteration for

$$
u^{n+1} - \Delta t a(u^{n+1})u^{n+1} = u^n\thinspace .
$$

applies old values in for $u^{n+1}$ in $a(u^{n+1})$. If $u^-$ is the
most recently computed approximation to $u^{n+1}$, we can write the Picard
linearization as

$$
(1 - \Delta t a(u^-))u^{n+1} = u^n\thinspace .
$$

Alternatively, with an iteration index $k$,

$$
(1 - \Delta t a(u^{n+1,k}))u^{n+1,k+1} = u^n\thinspace .
$$

Newton's method starts with identifying the nonlinear equation as
$F(u)=0$, and here

$$
F(u) = u - \Delta t a(u)u - u^n\thinspace .
$$

The Jacobian is

$$
J(u) = \frac{F(u)}{du} = 1 - \Delta t(a'(u)u + a(u))\thinspace .
$$

The key equation in Newton's method is then
[hpl 1: Check notation here with previously used notation!]

$$
J(u^-)\delta u = - F(u^-),\quad u\leftarrow u - \delta u\thinspace .
$$

**c)**
Implement the numerical solution methods from a) and b).
Use [`logistic.py`](${src_nonlin}/logistic.py) to compare the case
$p=1$ and the choice ([2](#nonlin:exer:logistic:gen:r1)).

**Solution.**
We specialize the code for $a(u)$ to ([2](#nonlin:exer:logistic:gen:r1))
since the code was developed from `logistic.py`. mathcal{I}_t is convenient to
work with a dimensionless form of the problem.
Choosing a time scale $t_c = 1\varrho$ and a scale for $u$,
$u_c=M$, leads to

$$
u^{\prime} = \varrho (1-u)^p u,\quad u(0)=\alpha,
$$

where $\alpha$ is a dimensionless number

$$
\alpha = \frac{I}{M}\thinspace .
$$

The three schemes can be implemented as follows.

```
import numpy as np

def FE_logistic(p, u0, dt, Nt):
    u = np.zeros(Nt+1)
    u[0] = u0
    for n in range(Nt):
        u[n+1] = u[n] + dt*(1 - u[n])**p*u[n]
    return u

def BE_logistic(p, u0, dt, Nt, choice='Picard',
                eps_r=1E-3, omega=1, max_iter=1000):
    # u[n] = u[n-1] + dt*(1-u[n])**p*u[n]
    # -dt*(1-u[n])**p*u[n] + u[n] = u[n-1]
    if choice == 'Picard1':
        choice = 'Picard'
        max_iter = 1

    u = np.zeros(Nt+1)
    iterations = []
    u[0] = u0
    for n in range(1, Nt+1):
        c = -u[n-1]
        if choice == 'Picard':
            def F(u):
                return -dt*(1-u)**p*u + u + c

            u_ = u[n-1]
            k = 0
            while abs(F(u_)) > eps_r and k < max_iter:
                # u*(1-dt*(1-u_)**p) + c = 0
                u_ = omega*(-c/(1-dt*(1-u_)**p)) + (1-omega)*u_
                k += 1
            u[n] = u_
            iterations.append(k)

        elif choice == 'Newton':
            def F(u):
                return -dt*(1-u)**p*u + u + c

            def dF(u):
                return dt*p*(1-u)**(p-1)*u - dt*(1-u)**p + 1

            u_ = u[n-1]
            k = 0
            while abs(F(u_)) > eps_r and k < max_iter:
                u_ = u_ - F(u_)/dF(u_)
                k += 1
            u[n] = u_
            iterations.append(k)
    return u, iterations

def CN_logistic(p, u0, dt, Nt):
    # u[n+1] = u[n] + dt*(1-u[n])**p*u[n+1]
    # (1 - dt*(1-u[n])**p)*u[n+1] = u[n]
    u = np.zeros(Nt+1)
    u[0] = u0
    for n in range(0, Nt):
        u[n+1] = u[n]/(1 - dt*(1 - u[n])**p)
    return u
```

A first verification is to choose $p=1$ and compare the results with
those from `logistic.py`. The number of iterations and the final
numerical answers should be identical.

**d)**
Implement unit tests that check the asymptotic limit of the solutions:
$u\rightarrow M$ as $t\rightarrow\infty$.

**Hint.**
You need to experiment to find what "infinite time" is
(increases substantially with $p$) and what the
appropriate tolerance is for testing the asymptotic limit.

**Solution.**
The test function may look like

```
def test_asymptotic_value():
    T = 100
    dt = 0.1
    Nt = int(round(T/float(dt)))
    u0 = 0.1
    p = 1.8

    u_CN = CN_logistic(p, u0, dt, Nt)
    u_BE_Picard, iter_Picard = BE_logistic(
        p, u0, dt, Nt, choice='Picard',
        eps_r=1E-5, omega=1, max_iter=1000)
    u_BE_Newton, iter_Newton = BE_logistic(
        p, u0, dt, Nt, choice='Newton',
        eps_r=1E-5, omega=1, max_iter=1000)
    u_FE = FE_logistic(p, u0, dt, Nt)

    for arr in u_CN, u_BE_Picard, u_BE_Newton, u_FE:
        expected = 1
        computed = arr[-1]
        tol = 0.01
        msg = 'expected=%s, computed=%s' % (expected, computed)
        print msg
        assert abs(expected - computed) < tol
```

mathcal{I}_t is important with a sufficiently small `eps_r` tolerance for the
asymptotic value to be accurate (using `eps_r=1E-3` leads to a value
0.92 at $t=T$ instead of 0.994 when `eps_r=1E-5`).

**e)**
Perform experiments with Newton and Picard iteration for
the model ([2](#nonlin:exer:logistic:gen:r1)).
See how sensitive
the number of iterations is to $\Delta t$ and $p$.

**Solution.**
Appropriate code is

```
%matplotlib inline

from scitools.std import *

def demo():
    T = 12
    p = 1.2
    try:
        dt = float(sys.argv[1])
        eps_r = float(sys.argv[2])
        omega = float(sys.argv[3])
    except:
        dt = 0.8
        eps_r = 1E-3
        omega = 1
    N = int(round(T/float(dt)))

    u_FE = FE_logistic(p, 0.1, dt, N)
    u_BE31, iter_BE31 = BE_logistic(p, 0.1, dt, N,
                                    'Picard1', eps_r, omega)
    u_BE3, iter_BE3 = BE_logistic(p, 0.1, dt, N,
                                  'Picard', eps_r, omega)
    u_BE4, iter_BE4 = BE_logistic(p, 0.1, dt, N,
                                  'Newton', eps_r, omega)
    u_CN = CN_logistic(p, 0.1, dt, N)

    print 'Picard mean no of iterations (dt=%g):' % dt, \
          int(round(mean(iter_BE3)))
    print 'Newton mean no of iterations (dt=%g):' % dt, \
          int(round(mean(iter_BE4)))

    t = np.linspace(0, dt*N, N+1)
    plot(t, u_FE, t, u_BE3, t, u_BE31, t, u_BE4, t, u_CN,
         legend=['FE', 'BE Picard', 'BE Picard1', 'BE Newton', 'CN gm'],
         title='dt=%g, eps=%.0E' % (dt, eps_r), xlabel='t', ylabel='u',
         legend_loc='lower right')
    filestem = 'logistic_N%d_eps%03d' % (N, log10(eps_r))
    savefig(filestem + '_u.png')
    savefig(filestem + '_u.pdf')
    figure()
    plot(range(1, len(iter_BE3)+1), iter_BE3, 'r-o',
         range(1, len(iter_BE4)+1), iter_BE4, 'b-o',
         legend=['Picard', 'Newton'],
         title='dt=%g, eps=%.0E' % (dt, eps_r),
         axis=[1, N+1, 0, max(iter_BE3 + iter_BE4)+1],
         xlabel='Time level', ylabel='No of iterations')
    savefig(filestem + '_iter.png')
    savefig(filestem + '_iter.pdf')
```

[hpl 2: Not finished!]

Filename: `logistic_p`.

## Problem 3: Experience the behavior of Newton's method

The program [`Newton_demo.py`](${src_nonlin}/Newton_demo.py) illustrates
graphically each step in Newton's method and is run like

        Terminal> python Newton_demo.py f dfdx x0 xmin xmax

Use this program to investigate potential
problems with Newton's method when solving $e^{-0.5x^2}\cos (\pi x)=0$.
Try a starting point $x_0=0.8$ and $x_0=0.85$ and watch the different
behavior. Just run

        Terminal>  python Newton_demo.py '0.2 + exp(-0.5*x**2)*cos(pi*x)' \
                   '-x*exp(-x**2)*cos(pi*x) - pi*exp(-x**2)*sin(pi*x)' \
                   0.85 -3 3

and repeat with 0.85 replaced by 0.8.

## Exercise 4: Compute the Jacobian of a $2\times 2$ system

Write up the system ([nonlin:ode:generic:sys:pendulum:u0](#nonlin:ode:generic:sys:pendulum:u0))-([nonlin:ode:generic:sys:pendulum:u1](#nonlin:ode:generic:sys:pendulum:u1)) in the form $F(u)=0$, $F=(F_0,F_1)$, $u=(u_0,u_1)$,
and compute the Jacobian $J_{i,j}=\partial F_i/\partial u_j$.

## Problem 5: Solve nonlinear equations arising from a vibration ODE

Consider a nonlinear vibration problem

$$
\begin{equation}
mu^{\prime\prime} + bu^{\prime}|u^{\prime}| + s(u) = F(t),
\label{_auto1} \tag{3}
\end{equation}
$$

where $m>0$ is a constant, $b\geq 0$ is a constant, $s(u)$ a possibly
nonlinear function of $u$, and $F(t)$ is a prescribed function. Such
models arise from Newton's second law of motion in mechanical
vibration problems where $s(u)$ is a spring or restoring force,
$mu^{\prime\prime}$ is mass times acceleration, and
$bu^{\prime}|u^{\prime}|$ models water or air drag.

**a)**
Rewrite the equation for $u$ as a system of two first-order ODEs, and
discretize this system by a Crank-Nicolson (centered difference)
method. With $v=u^\prime$, we get a nonlinear term
$v^{n+\frac{1}{2}}|v^{n+\frac{1}{2}}|$. Use a geometric
average for $v^{n+\frac{1}{2}}$.

**b)**
Formulate a Picard iteration method to solve the system of nonlinear
algebraic equations.

**c)**
Explain how to apply Newton's method to solve the nonlinear equations
at each time level. Derive expressions for the Jacobian and the
right-hand side in each Newton iteration.

Filename: `nonlin_vib`.

## Exercise 6: Find the truncation error of arithmetic mean of products

In the section [nonlin:pdelevel:Picard:CN](#nonlin:pdelevel:Picard:CN) we introduce alternative
arithmetic means of a product. Say the product is $P(t)Q(t)$ evaluated
at $t=t_{n+\frac{1}{2}}$. The exact value is

$$
[PQ]^{n+\frac{1}{2}} = P^{n+\frac{1}{2}}Q^{n+\frac{1}{2}}
$$

There are two obvious candidates for evaluating $[PQ]^{n+\frac{1}{2}}$
as a mean of values of $P$ and $Q$ at $t_n$ and $t_{n+1}$. Either
we can take the arithmetic mean of each factor $P$ and $Q$,

$$
\begin{equation}
[PQ]^{n+\frac{1}{2}} \approx \frac{1}{2}(P^n + P^{n+1})\frac{1}{2}(Q^n + Q^{n+1}),
\label{nonlin:exer:products:arith:mean:f} \tag{4}
\end{equation}
$$

or we can take the arithmetic mean of the product $PQ$:

$$
\begin{equation}
[PQ]^{n+\frac{1}{2}} \approx \frac{1}{2}(P^nQ^n + P^{n+1}Q^{n+1})\thinspace .
\label{nonlin:exer:products:arith:mean:p} \tag{5}
\end{equation}
$$

The arithmetic average
of $P(t_{n+\frac{1}{2}})$ is $\Oof{\Delta t^2}$:

$$
P(t_{n+\frac{1}{2}}) = \frac{1}{2}(P^n + P^{n+1}) +\Oof{\Delta t^2}\thinspace .
$$

A fundamental question is whether ([4](#nonlin:exer:products:arith:mean:f))
and ([5](#nonlin:exer:products:arith:mean:p)) have different
orders of accuracy in $\Delta t = t_{n+1}-t_n$. To investigate
this question, expand quantities at $t_{n+1}$ and $t_n$ in
Taylor series around $t_{n+\frac{1}{2}}$, and subtract the true
value $[PQ]^{n+\frac{1}{2}}$ from the approximations
([4](#nonlin:exer:products:arith:mean:f)) and
([5](#nonlin:exer:products:arith:mean:p)) to see what the
order of the error terms are.

**Hint.**
You may explore `sympy` for carrying out the tedious calculations.
A general Taylor series expansion of $P(t+\frac{1}{2}\Delta t)$ around $t$
involving just a general function $P(t)$ can be
created as follows:

```
from sympy import *
t, dt = symbols('t dt')
P = symbols('P', cls=Function)
P(t).series(t, 0, 4)
```

```
P_p = P(t).series(t, 0, 4).subs(t, dt/2)
P_p
```

The error of the arithmetic mean, $\frac{1}{2}(P(-\frac{1}{2}\Delta t)
+ P(-\frac{1}{2}\Delta t))$ for $t=0$ is then

```
P_m = P(t).series(t, 0, 4).subs(t, -dt/2)
mean = Rational(1,2)*(P_m + P_p)
error = simplify(expand(mean) - P(0))
error
```

Use these examples to investigate the error of
([4](#nonlin:exer:products:arith:mean:f)) and
([5](#nonlin:exer:products:arith:mean:p)) for $n=0$. (Choosing $n=0$
is necessary for not making the expressions too complicated for `sympy`,
but there is of course no lack of generality by using $n=0$ rather
than an arbitrary $n$ - the main point is the product and addition
of Taylor series.)

Filename: `product_arith_mean`.

## Problem 7: Newton's method for linear problems

Suppose we have a linear system $F(u) = Au- b=0$. Apply Newton's method
to this system, and show that the method converges in one iteration.
Filename: `Newton_linear`.

## Problem 8: Discretize a 1D problem with a nonlinear coefficient

We consider the problem

$$
\begin{equation}
((1 + u^2)u^{\prime})^{\prime} = 1,\quad x\in (0,1),\quad u(0)=u(1)=0\thinspace .
\label{nonlin:exer:1D:1pu2:fem:pde} \tag{6}
\end{equation}
$$

% if FEM_BOOK:

**a)**
Discretize ([6](#nonlin:exer:1D:1pu2:fem:pde)) by a centered
finite difference method on a uniform mesh.

**b)**
Discretize ([6](#nonlin:exer:1D:1pu2:fem:pde)) by a finite
element method with P1 elements of equal length.
Use the Trapezoidal method to compute all integrals.
Set up the resulting matrix system in symbolic form such that the
equations can be compared with those in a).
% else:
Discretize ([6](#nonlin:exer:1D:1pu2:fem:pde)) by a centered
finite difference method on a uniform mesh.
% endif

Filename: `nonlin_1D_coeff_discretize`.

## Problem 9: Linearize a 1D problem with a nonlinear coefficient

We have a two-point boundary value problem

$$
\begin{equation}
((1 + u^2)u^{\prime})^{\prime} = 1,\quad x\in (0,1),\quad u(0)=u(1)=0\thinspace .
\label{nonlin:exer:1D:1pu2:PicardNewton:pde} \tag{7}
\end{equation}
$$

**a)**
Construct a Picard iteration method for ([7](#nonlin:exer:1D:1pu2:PicardNewton:pde))
without discretizing in space.

**b)**
Apply Newton's method to ([7](#nonlin:exer:1D:1pu2:PicardNewton:pde))
without discretizing in space.

**c)**
Discretize ([7](#nonlin:exer:1D:1pu2:PicardNewton:pde)) by a centered finite
difference scheme. Construct a Picard method for the resulting
system of nonlinear algebraic equations.

**d)**
Discretize ([7](#nonlin:exer:1D:1pu2:PicardNewton:pde)) by a centered finite
difference scheme. Define the system of nonlinear algebraic equations,
calculate the Jacobian, and set up Newton's method for solving the system.

Filename: `nonlin_1D_coeff_linearize`.

## Problem 10: Finite differences for the 1D Bratu problem

We address the so-called Bratu problem

$$
\begin{equation}
u^{\prime\prime} + \lambda e^u=0,\quad x\in (0,1),\quad u(0)=u(1)=0,
\label{nonlin:exer:1D:fu:discretize:fd:pde} \tag{8}
\end{equation}
$$

where $\lambda$ is a given parameter and $u$ is a function of $x$.
This is a widely used model problem for studying numerical
methods for nonlinear differential equations.
The problem ([8](#nonlin:exer:1D:fu:discretize:fd:pde)) has an
exact solution

$$
\uex(x) = -2\ln\left(\frac{\cosh((x-\frac{1}{2})\theta/2)}{\cosh(\theta/4)}\right),
$$

where $\theta$ solves

$$
\theta = \sqrt{2\lambda}\cosh(\theta/4)\thinspace .
$$

There are two solutions of ([8](#nonlin:exer:1D:fu:discretize:fd:pde)) for
$0<\lambda <\lambda_c$ and no solution for $\lambda >\lambda_c$.
For $\lambda = \lambda_c$ there is one unique solution. The critical
value $\lambda_c$ solves

$$
1 = \sqrt{2\lambda_c}\frac{1}{4}\sinh(\theta(\lambda_c)/4)\thinspace .
$$

A numerical value is $\lambda_c = 3.513830719$.

**a)**
Discretize ([8](#nonlin:exer:1D:fu:discretize:fd:pde)) by a
centered finite difference method.

**b)**
Set up the nonlinear equations $F_i(u_0,u_1,\ldots,u_{N_x})=0$
from a). Calculate the associated Jacobian.

**c)**
Implement a solver that can compute $u(x)$ using Newton's method.
Plot the error as a function of $x$ in each iteration.

**d)**
Investigate whether Newton's method gives second-order convergence
by computing
$|| \uex - u||/||\uex - u^{-}||^2$
in each iteration, where $u$ is solution in the current iteration and
$u^{-}$ is the solution in the previous iteration.

% if FEM_BOOK:
Filename: `nonlin_1D_Bratu_fd`.

## Problem 11: Integrate functions of finite element expansions

We shall investigate integrals on the form

$$
\begin{equation}
\int_0^L f(\sum_ku_k\basphi_k(x))\basphi_i(x)\dx,
\label{nonlin:exer:fu:fem:int:global} \tag{9}
\end{equation}
$$

where $\basphi_i(x)$ are P1 finite element basis functions and $u_k$
are unknown coefficients, more precisely the values of the unknown
function $u$ at nodes $\xno{k}$. We introduce a node numbering that
goes from left to right and also that all cells have
the same length $h$. Given $i$, the integral
only gets contributions from $[\xno{i-1},\xno{i+1}]$. On this
interval $\basphi_k(x)=0$ for $ki+1$, so only three
basis functions will contribute:

$$
\sum_k u_k\basphi_k(x) = u_{i-1}\basphi_{i-1}(x) +
u_{i}\basphi_{i}(x) + u_{i+1}\basphi_{i+1}(x)\thinspace .
$$

The integral ([9](#nonlin:exer:fu:fem:int:global)) now takes the
simplified form

$$
\int_{\xno{i-1}}^{\xno{i+1}}
f(u_{i-1}\basphi_{i-1}(x) +
u_{i}\basphi_{i}(x) + u_{i+1}\basphi_{i+1}(x))\basphi_i(x)\dx\thinspace .
$$

Split this integral in two integrals over cell L (left),
$[\xno{i-1},\xno{i}]$, and cell R (right), $[\xno{i},\xno{i+1}]$. Over
cell L, $u$ simplifies to $u_{i-1}\basphi_{i-1} + u_{i}\basphi_{i}$
(since $\basphi_{i+1}=0$ on this cell), and over cell R, $u$
simplifies to $u_{i}\basphi_{i} + u_{i+1}\basphi_{i+1}$.  Make a
`sympy` program that can compute the integral and write it out as a
difference equation. Give the $f(u)$ formula on the command line.
Try out $f(u)=u^2, \sin u, \exp u$.

**Hint.**
Introduce symbols `u_i`, `u_im1`, and `u_ip1` for $u_i$, $u_{i-1}$,
and $u_{i+1}$, respectively, and similar symbols for $x_i$, $x_{i-1}$,
and $x_{i+1}$. Find formulas for the basis functions on each of the
two cells, make expressions for $u$ on the two cells, integrate over
each cell, expand the answer and simplify. You can ask `sympy` for
LaTeX code and render it either by creating a LaTeX document and
compiling it to a PDF document or by using
<http://latex.codecogs.com> to display LaTeX formulas in a web
page. Here are some appropriate Python statements
for the latter purpose:

```
from sympy import *
...
# expr_i holdes the integral as a sympy expression
latex_code = latex(expr_i, mode='plain')
# Replace u_im1 sympy symbol name by latex symbol u_{i-1}
latex_code = latex_code.replace('im1', '{i-1}')
# Replace u_ip1 sympy symbol name by latex symbol u_{i+1}
latex_code = latex_code.replace('ip1', '{i+1}')
# Escape (quote) latex_code so it can be sent as HTML text
import cgi
html_code = cgi.escape(latex_code)
# Make a file with HTML code for displaying the LaTeX formula
f = open('tmp.html', 'w')
# Include an image that can be clicked on to yield a new
# page with an interactive editor and display area where the
# formula can be further edited
text = """
<a href="http://www.codecogs.com/eqnedit.php?latex=%(html_code)s"
 target="_blank">

</a>
 """ % vars()
f.write(text)
f.close()
```

The formula is displayed by loading `tmp.html` into a web browser.

Filename: `fu_fem_int`.

## Problem 12: Finite elements for the 1D Bratu problem

We address the same 1D Bratu problem as described in
[Problem 10: Finite differences for the 1D Bratu problem](#nonlin:exer:1D:fu:discretize:fd).

**a)**
Discretize ([Problem 12: Finite elements for the 1D Bratu problem](#nonlin:exer:1D:fu:discretize:fe)) by a finite element
method using a uniform mesh with P1 elements. Use a group
finite element method for the $e^u$ term.

**b)**
Set up the nonlinear equations $F_i(u_0,u_1,\ldots,u_{N_x})=0$
from a). Calculate the associated Jacobian.
% endif

Filename: `nonlin_1D_Bratu_fe`.

## Problem 13: Discretize a nonlinear 1D heat conduction PDE by finite differences

We address the 1D heat conduction PDE

$$
\varrho c(T) T_t = (k(T)T_x)_x,
$$

for $x\in [0,L]$,
where $\varrho$ is the density of the solid material, $c(T)$ is
the heat capacity, $T$ is the temperature, and $k(T)$ is the
heat conduction coefficient. $T(x,0)=I(x)$, and ends are
subject to a cooling law:

$$
k(T)T_x|_{x=0} = h(T)(T-T_s),\quad -k(T)T_x|_{x=L}=h(T)(T-T_s),
$$

where $h(T)$ is a heat transfer coefficient and $T_s$ is the
given surrounding temperature.

**a)**
Discretize this PDE in time using either a
Backward Euler or Crank-Nicolson scheme.

**b)**
Formulate a Picard iteration method for the time-discrete problem
(i.e., an iteration method before discretizing in space).

**c)**
Formulate a Newton method for the time-discrete problem in b).

**d)**
Discretize the PDE by a finite difference method in space.
Derive the matrix and right-hand side of a Picard iteration method applied
to the space-time discretized PDE.

**e)**
Derive the matrix and right-hand side of a Newton method applied
to the discretized PDE in d).

% if FEM_BOOK:
Filename: `nonlin_1D_heat_FD`.

## Exercise 14: Use different symbols for different approximations of the solution

The symbol $u$ has several meanings, depending on the context, as
briefly mentioned in the section [nonlin:alglevel:dD:fe](#nonlin:alglevel:dD:fe).
Go through the derivation of the Picard iteration method in
that section and use different symbols for all the different
approximations of $u$:

 * $\uex(\x,t)$ for the exact solution of the PDE problem

 * $\uex(\x)^n$ for the exact solution after time discretization

 * $u^n(\x)$ for the spatially discrete solution $\sum_jc_j\baspsi_j$

 * $u^{n,k}$ for approximation in Picard/Newton iteration no $k$
   to $u^n(\x)$

Filename: `nonlin_heat_FE_usymbols`.

## Exercise 15: Derive Picard and Newton systems from a variational form

We study the multi-dimensional heat conduction PDE

$$
\varrho c(T) T_t = \nabla\cdot (k(T)\nabla T)
$$

in a spatial domain $\Omega$, with a nonlinear Robin boundary condition

$$
-k(T)\frac{\partial T}{\partial n} = h(T)(T-T_s(t)),
$$

at the boundary $\partial\Omega$.
The primary unknown is the temperature $T$, $\varrho$ is the density
of the solid material, $c(T)$ is the heat capacity, $k(T)$ is
the heat conduction, $h(T)$ is a heat transfer coefficient, and
$T_s(T)$ is a possibly time-dependent temperature of the surroundings.

**a)**
Use a Backward Euler or Crank-Nicolson time discretization and
derive the variational form for the spatial problem to be solved
at each time level.

**b)**
Define a Picard iteration method from the variational form at
a time level.

**c)**
Derive expressions for the matrix and the right-hand side of the
equation system that arises from applying Newton's method to
the variational form at a time level.

**d)**
Apply the Backward Euler or Crank-Nicolson scheme in time first.
Derive a Newton method at the PDE level. Make a variational
form of the resulting PDE at a time level.

Filename: `nonlin_heat_FE`.

## Exercise 16: Derive algebraic equations for nonlinear 1D heat conduction

We consider the same problem as in [Exercise 15: Derive Picard and Newton systems from a variational form](#nonlin:exer:dD:heat:nonlinear:c:a), but restricted to one space dimension: $\Omega = [0,L]$.
Simplify the boundary condition to $T_x=0$ (i.e., $h(T)=0$).
Use a uniform finite element mesh of P1 elements, the group
finite element method, and the Trapezoidal
rule for integration at the nodes to derive symbolic expressions for
the algebraic equations arising from this diffusion problem.
% endif
Filename: `nonlin_1D_heat_FE`.

## Problem 17: Differentiate a highly nonlinear term

The operator $\nabla\cdot(\dfc(u)\nabla u)$ with
$\dfc(u) = |\nabla u|^q$ appears in several physical problems,
especially flow of Non-Newtonian fluids. The expression $|\nabla u|$
is defined as the Euclidean norm of a vector:
$|\nabla u|^2 = \nabla u \cdot \nabla u$.
In a Newton method one
has to carry out the differentiation $\partial\dfc(u)/\partial c_j$,
for $u=\sum_kc_k\baspsi_k$. Show that

$$
{\partial\over\partial u_j} |\nabla u|^q =
q|\nabla  u|^{q-2}\nabla u\cdot
\nabla\baspsi_j\thinspace .
$$

**Solution.**

$$
\begin{align*}
\frac{\partial }{\partial c_j}|\nabla u |^q &=
\frac{\partial }{\partial c_j}(\nabla u\cdot \nabla u )^{\tfrac{q}{2}}
= \frac{q}{2} (\nabla u\cdot \nabla u )^{\tfrac{q}{2}-1}
\frac{\partial }{\partial c_j}(\nabla u\cdot \nabla u )\\
&=\frac{q}{2} |\nabla u |^{q-2}(\frac{\partial }{\partial c_j}(\nabla u)
\cdot \nabla u + \nabla u \cdot\frac{\partial }{\partial c_j}(\nabla u))\\
&=q|\nabla u |^{q-2}(\nabla u \cdot \nabla \frac{\partial u}{\partial c_j})
=q|\nabla u |^{q-2}(\nabla u \cdot \nabla\psi_j)
\end{align*}
$$

Filename: `nonlin_differentiate`.

## Exercise 18: Crank-Nicolson for a nonlinear 3D diffusion equation

Redo the section [nonlin:alglevel:dD:fd](#nonlin:alglevel:dD:fd) when a Crank-Nicolson scheme
is used to discretize the equations in time and the problem is
formulated for three spatial dimensions.

**Hint.**
Express the Jacobian as $J_{i,j,k,r,s,t} = \partial F_{i,j,k}/\partial u_{r,s,t}$ and observe, as in the 2D case, that $J_{i,j,k,r,s,t}$ is very sparse:
$J_{i,j,k,r,s,t}\neq 0$ only for $r=i\pm i$, $s=j\pm 1$, and $t=k\pm 1$
as well as $r=i$, $s=j$, and $t=k$.

Filename: `nonlin_heat_FD_CN_2D`.

## Problem 19: Find the sparsity of the Jacobian

Consider a typical nonlinear Laplace term like
$\nabla\cdot\dfc(u)\nabla u$ discretized by centered finite differences.
Explain why the Jacobian corresponding to this term has the same
sparsity pattern as the matrix associated with the corresponding linear
term $\dfc\nabla^2 u$.

**Hint.**
Set up the unknowns that enter the difference equation at a
point $(i,j)$ in 2D or $(i,j,k)$ in 3D, and identify the
nonzero entries of the Jacobian that can arise from such a type
of difference equation.

Filename: `nonlin_sparsity_Jacobian`.

## Problem 20: Investigate a 1D problem with a continuation method

Flow of a pseudo-plastic power-law fluid between two flat plates can be
modeled by

$$
\frac{d}{dx}\left(\mu_0\left\vert\frac{du}{dx}\right\vert^{n-1}
\frac{du}{dx}\right) = -\beta,\quad u^{\prime}(0)=0,\ u(H) = 0,
$$

where $\beta>0$ and $\mu_0>0$ are constants.
A target value of $n$ may be $n=0.2$.

**a)**
Formulate a Picard iteration method directly for the differential
equation problem.

**b)**
Perform a finite difference discretization of the problem in
each Picard iteration. Implement a solver that can compute $u$
on a mesh. Verify that the solver gives an exact solution for $n=1$
on a uniform mesh regardless of the cell size.

**c)**
Given a sequence of decreasing $n$ values, solve the problem for each
$n$ using the solution for the previous $n$ as initial guess for
the Picard iteration. This is called a continuation method.
Experiment with $n=(1,0.6,0.2)$ and $n=(1,0.9,0.8,\ldots,0.2)$
and make a table of the number of Picard iterations versus $n$.

**d)**
Derive a Newton method at the differential equation level and
discretize the resulting linear equations in each Newton iteration
with the finite difference method.

**e)**
Investigate if Newton's method has better convergence properties than
Picard iteration, both in combination with a continuation method.