# Start-to-Finish Example: Unit Testing `GiRaFFE_NRPy`: Induction Equation

## Author: Patrick Nelson

## This module validates the routine to compute the flux of $\epsilon_{ijk} v^j B^k$ for `GiRaFFE`.

**Notebook Status:** In-Progress

**Validation Notes:** This module will validate the routines in [Tutorial-GiRaFFE_NRPy-Induction_Equation](Tutorial-GiRaFFE_NRPy-Induction_Equation.ipynb).

### NRPy+ Source Code for this module:
* [GiRaFFE_NRPy/Afield_flux.py](../../edit/in_progress/GiRaFFE_NRPy/Afield_flux.py) [\[**tutorial**\]](Tutorial-GiRaFFE_NRPy-Afield_flux.ipynb) Generates the symbolic expressions needed to compute the flux term of the induction equation right-hand side.

## Introduction:

This notebook validates our algorithm to compute the flux of $\epsilon_{ijk} v^j B^k$ through cell faces, which contributes to the right-hand side of the evolution equation for $A_i$, for use in `GiRaFFE_NRPy`. Because the original `GiRaFFE` used staggered grids and we do not, we can not trivially do a direct comparison to the old code. Instead, we will compare the numerical results with the expected analytic results.

It is, in general, good coding practice to unit test functions individually to verify that they produce the expected and intended output. Here, we expect our functions to produce the correct cross product between the velocity and magnetic field in an arbitrary spacetime. To that end, we will choose functions with relatively simple analytic forms.

In this test, we will generate analytic forms for the magnetic field and three-velocity. We will need to do this in a $7 \times 7 \times 7$ cube in order to run the PPM routine on the data to generate the inputs compute the non-gauge terms of the RHS of the induction equation, unless there's a way to usefully spoof the left- and right-face values for the HLLE solver. We care about this here, because we are comparing against an analytic expression and not the old code.

When this notebook is run, the difference between the approximate and exact right-hand sides will be output to text files that can be found in the same directory as this notebook. These will be read in in [Step 3](#convergence), and used there to confirm second order convergence of the algorithm.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#setup): Set up core functions and parameters for unit testing the A2B algorithm
    1. [Step 1.a](#magnetic) Set analytic magnetic field
    1. [Step 1.b](#velocity) Set analytic Valencia three-velocity
    1. [Step 1.c](#free_parameters) Set free parameters in the code
1. [Step 2](#mainc): `Induction_Equation_unit_test.c`: The Main C Code
    1. [Step 2.a](#compile_run): Compile and run the code
1. [Step 3](#convergence): Code validation: Verify that relative error in numerical solution converges to zero at the expected order
1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Set up core functions and parameters for unit testing the A2B algorithm \[Back to [top](#toc)\]

$$\label{setup}$$

We'll start by appending the relevant paths to `sys.path` so that we can access sympy modules in other places. Then, we'll import NRPy+ core functionality and set up a directory in which to carry out our test. We must also set the desired finite differencing order.

```python
import os, sys           # Standard Python modules for multiplatform OS-level functions
# First, we'll add the parent directory to the list of directories Python will check for modules.
nrpy_dir_path = os.path.join("..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)
nrpy_dir_path = os.path.join("..","..")
if nrpy_dir_path not in sys.path:
    sys.path.append(nrpy_dir_path)

from outputC import outCfunction, outputC # NRPy+: Core C code output module
import finite_difference as fin  # NRPy+: Finite difference C code generation module
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import grid as gri               # NRPy+: Functions having to do with numerical grids
import indexedexp as ixp         # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import reference_metric as rfm   # NRPy+: Reference metric support
import cmdline_helper as cmd     # NRPy+: Multi-platform Python command-line interface

out_dir = "Validation/"
cmd.mkdir(out_dir)
subdir = "Afield_flux"
cmd.mkdir(os.path.join(out_dir,subdir))

thismodule = "Start_to_Finish_UnitTest-GiRaFFE_NRPy-Afield_flux"
Use_Shock_Data = True

```

## Step 1.a: Valencia three-velocity \[Back to [top](#toc)\]
$$\label{velocity}$$

Here, we'll generate some functions for the velocity. Let's choose arctangents, since those have asymptotes that can be easily manipulated to prevent accidentally setting superluminal speeds.
\begin{align}
\bar{v}^x &= \frac{1}{5\pi} \arctan(ax + by + cz) \\
\bar{v}^y &= \frac{1}{5\pi} \arctan(bx + cy + az) \\
\bar{v}^z &= \frac{1}{5\pi} \arctan(cx + ay + bz) \\
\end{align}
If we want to add a jump at the origin, we can simply add $\max(0,x)$ to the argument of the arctangent. This will add a shock in the $x$-direction. The maximum will be described without the use of if statements using the `Min_Max_and_Piecewise_Expressions` module.

```python
import Min_Max_and_Piecewise_Expressions as noif

a,b,c = par.Cparameters("REAL",thismodule,["a","b","c"],1e300) # Note that this default value allows us to set
                                                               # these directly in the C code
M_PI  = par.Cparameters("#define",thismodule,["M_PI"], "")

par.set_parval_from_str("reference_metric::CoordSystem","Cartesian")
rfm.reference_metric()
x = rfm.xxCart[0]
y = rfm.xxCart[1]
z = rfm.xxCart[2]

offset = sp.sympify(5)
args = ixp.zerorank1()
args[0] = a*(x-offset) + b*y + c*z
args[1] = b*x + c*(y-offset) + a*z
args[2] = c*x + a*y + b*(z-offset)
if Use_Shock_Data:
    for i in range(3):
        args[i] += noif.max_noif(0,5*x)

ValenciavU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","ValenciavU")
for i in range(3):
    ValenciavU[i] = (sp.Rational(1,5)/M_PI)*sp.atan(args[i])
```

## Step 1.b: Magnetic field \[Back to [top](#toc)\]
$$\label{magnetic}$$

We'll also need some functions for the magnetic field. Exponentials sound fun.
\begin{align}
B^x &= \exp(ey+fz) \\
B^y &= \exp(fz+dx) \\
B^z &= \exp(dx+ey) \\
\end{align}
In this case, we'll add $\max{0,x}$ to the field to add the jump.

```python
d,e,f = par.Cparameters("REAL",thismodule,["d","e","f"],1e300) # Note that this default value allows us to set
                                                               # these directly in the C code
BU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","BU")
BU[0] = sp.exp(sp.Rational(1,10)*(e*y+f*z))
BU[1] = sp.exp(sp.Rational(1,10)*(f*z+d*x))
BU[2] = sp.exp(sp.Rational(1,10)*(d*x+e*y))
if Use_Shock_Data:
    for i in range(3):
        BU[i] += noif.max_noif(0,5*x)

```

## Step 1.c: Generate C functions to write the test data \[Back to [top](#toc)\]
$$\label{functions}$$

```python
BU_to_print = [\
                lhrh(lhs=gri.gfaccess("out_gfs","BU0"),rhs=BU[0]),\
                lhrh(lhs=gri.gfaccess("out_gfs","BU1"),rhs=BU[1]),\
                lhrh(lhs=gri.gfaccess("out_gfs","BU2"),rhs=BU[2]),\
               ]

desc = "Calculate sample magnetic field data"
name = "calculate_BU"
outCfunction(
    outfile  = os.path.join(out_dir,name+".h"), desc=desc, name=name,
    params   ="const paramstruct *params,REAL *xx[3],REAL *auxevol_gfs",
    body     = fin.FD_outputC("returnstring",BU_to_print,params="outCverbose=False").replace("IDX4","IDX4S"),
    loopopts="AllPoints,Read_xxs")

ValenciavU_to_print = [\
                       lhrh(lhs=gri.gfaccess("out_gfs","ValenciavU0"),rhs=ValenciavU[0]),\
                       lhrh(lhs=gri.gfaccess("out_gfs","ValenciavU1"),rhs=ValenciavU[1]),\
                       lhrh(lhs=gri.gfaccess("out_gfs","ValenciavU2"),rhs=ValenciavU[2]),\
                      ]

desc = "Calculate sample velocity data"
name = "calculate_ValenciavU"
outCfunction(
    outfile  = os.path.join(out_dir,name+".h"), desc=desc, name=name,
    params   ="const paramstruct *params,REAL *xx[3],REAL *auxevol_gfs",
    body     = fin.FD_outputC("returnstring",ValenciavU_to_print,params="outCverbose=False").replace("IDX4","IDX4S"),
    loopopts="AllPoints,Read_xxs")

```

    Output C function calculate_BU() to file Validation/calculate_BU.h
    Output C function calculate_ValenciavU() to file Validation/calculate_ValenciavU.h

## Step 1.d: Set free parameters in the code \[Back to [top](#toc)\]
$$\label{free_parameters}$$

We also need to create the files that interact with NRPy's C parameter interface.

```python
# Step 3.d.i: Generate declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
NGHOSTS  = par.Cparameters("int",thismodule,["NGHOSTS"], 3)
# Step 3.d.ii: Set free_parameters.h
with open(os.path.join(out_dir,"free_parameters.h"),"w") as file:
    file.write("""
// Override parameter defaults with values based on command line arguments and NGHOSTS.
// We'll use this grid. It has one point and one ghost zone.
params.NGHOSTS = 3;
params.Nxx0 = atoi(argv[1]);
params.Nxx1 = atoi(argv[2]);
params.Nxx2 = atoi(argv[3]);
params.Nxx_plus_2NGHOSTS0 = params.Nxx0 + 2*params.NGHOSTS;
params.Nxx_plus_2NGHOSTS1 = params.Nxx1 + 2*params.NGHOSTS;
params.Nxx_plus_2NGHOSTS2 = params.Nxx2 + 2*params.NGHOSTS;
// Step 0d: Set up space and time coordinates
// Step 0d.i: Declare \Delta x^i=dxx{0,1,2} and invdxx{0,1,2}, as well as xxmin[3] and xxmax[3]:
const REAL xxmin[3] = {-0.1,-0.1,-0.1};
const REAL xxmax[3] = { 0.1, 0.1, 0.1};

params.dxx0 = (xxmax[0] - xxmin[0]) / ((REAL)params.Nxx0);
params.dxx1 = (xxmax[1] - xxmin[1]) / ((REAL)params.Nxx1);
params.dxx2 = (xxmax[2] - xxmin[2]) / ((REAL)params.Nxx2);
//printf("dxx0,dxx1,dxx2 = %.5e,%.5e,%.5e\\n",params.dxx0,params.dxx1,params.dxx2);
params.invdx0 = 1.0 / params.dxx0;
params.invdx1 = 1.0 / params.dxx1;
params.invdx2 = 1.0 / params.dxx2;
\n""")

# Generates declare_Cparameters_struct.h, set_Cparameters_default.h, and set_Cparameters[-SIMD].h
par.generate_Cparameters_Ccodes(os.path.join(out_dir))
```

## Step 1.e: Generate `GiRaFFE_NRPy` Files \[Back to [top](#toc)\]
$$\label{module}$$

Here, we generate the functions we want to test by calling the function found [here](../../edit/in_progress/GiRaFFE_NRPy/GiRaFFE_NRPy_PPM.py) and documented in [this tutorial](Tutorial-Start_to_Finish-GiRaFFE_NRPy-PPM.ipynb).

```python
import GiRaFFE_NRPy.GiRaFFE_NRPy_PPM as PPM
PPM.GiRaFFE_NRPy_PPM(os.path.join(out_dir,subdir))
```

## Step 1.f: Calculate the $E_i$ field contribution to $\partial_t A_i$ \[Back to [top](#toc)\]
$$\label{electric_field}$$

Here, we generate the functions necessary to calculate the electric flux on the cell faces, which is the algorithm we are specifically trying to test.

```python
import GiRaFFE_NRPy.Afield_flux as Af

# We will pass values of the gridfunction on the cell faces into the function. This requires us
# to declare them as C parameters in NRPy+. We will denote this with the _face infix/suffix.
alpha_face = gri.register_gridfunctions("AUXEVOL","alpha_face")
gamma_faceDD = ixp.register_gridfunctions_for_single_rank2("AUXEVOL","gamma_faceDD","sym01")
beta_faceU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","beta_faceU")

# We'll need some more gridfunctions, now, to represent the reconstructions of BU and ValenciavU
# on the right and left faces
Valenciav_rU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","Valenciav_rU",DIM=3)
B_rU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","B_rU",DIM=3)
Valenciav_lU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","Valenciav_lU",DIM=3)
B_lU = ixp.register_gridfunctions_for_single_rank1("AUXEVOL","B_lU",DIM=3)

AD = ixp.register_gridfunctions_for_single_rank1("EVOL","AD",DIM=3)

Memory_Read = """const double alpha_face = auxevol_gfs[IDX4S(ALPHA_FACEGF, i0,i1,i2)];
const double gamma_faceDD00 = auxevol_gfs[IDX4S(GAMMA_FACEDD00GF, i0,i1,i2)];
const double gamma_faceDD01 = auxevol_gfs[IDX4S(GAMMA_FACEDD01GF, i0,i1,i2)];
const double gamma_faceDD02 = auxevol_gfs[IDX4S(GAMMA_FACEDD02GF, i0,i1,i2)];
const double gamma_faceDD11 = auxevol_gfs[IDX4S(GAMMA_FACEDD11GF, i0,i1,i2)];
const double gamma_faceDD12 = auxevol_gfs[IDX4S(GAMMA_FACEDD12GF, i0,i1,i2)];
const double gamma_faceDD22 = auxevol_gfs[IDX4S(GAMMA_FACEDD22GF, i0,i1,i2)];
const double beta_faceU0 = auxevol_gfs[IDX4S(BETA_FACEU0GF, i0,i1,i2)];
const double beta_faceU1 = auxevol_gfs[IDX4S(BETA_FACEU1GF, i0,i1,i2)];
const double beta_faceU2 = auxevol_gfs[IDX4S(BETA_FACEU2GF, i0,i1,i2)];
const double Valenciav_rU0 = auxevol_gfs[IDX4S(VALENCIAV_RU0GF, i0,i1,i2)];
const double Valenciav_rU1 = auxevol_gfs[IDX4S(VALENCIAV_RU1GF, i0,i1,i2)];
const double Valenciav_rU2 = auxevol_gfs[IDX4S(VALENCIAV_RU2GF, i0,i1,i2)];
const double B_rU0 = auxevol_gfs[IDX4S(B_RU0GF, i0,i1,i2)];
const double B_rU1 = auxevol_gfs[IDX4S(B_RU1GF, i0,i1,i2)];
const double B_rU2 = auxevol_gfs[IDX4S(B_RU2GF, i0,i1,i2)];
const double Valenciav_lU0 = auxevol_gfs[IDX4S(VALENCIAV_LU0GF, i0,i1,i2)];
const double Valenciav_lU1 = auxevol_gfs[IDX4S(VALENCIAV_LU1GF, i0,i1,i2)];
const double Valenciav_lU2 = auxevol_gfs[IDX4S(VALENCIAV_LU2GF, i0,i1,i2)];
const double B_lU0 = auxevol_gfs[IDX4S(B_LU0GF, i0,i1,i2)];
const double B_lU1 = auxevol_gfs[IDX4S(B_LU1GF, i0,i1,i2)];
const double B_lU2 = auxevol_gfs[IDX4S(B_LU2GF, i0,i1,i2)];
REAL A_rhsD0 = 0; REAL A_rhsD1 = 0; REAL A_rhsD2 = 0;
"""
Memory_Write = """rhs_gfs[IDX4S(AD0GF,i0,i1,i2)] += A_rhsD0;
rhs_gfs[IDX4S(AD1GF,i0,i1,i2)] += A_rhsD1;
rhs_gfs[IDX4S(AD2GF,i0,i1,i2)] += A_rhsD2;
"""

indices = ["i0","i1","i2"]
indicesp1 = ["i0+1","i1+1","i2+1"]

for flux_dirn in range(3):
    Af.calculate_E_i_flux(flux_dirn,True,alpha_face,gamma_faceDD,beta_faceU,\
                          Valenciav_rU,B_rU,Valenciav_lU,B_lU)

    E_field_to_print = [\
                        -sp.Rational(1,4)*Af.E_fluxD[(flux_dirn+1)%3],\
                        -sp.Rational(1,4)*Af.E_fluxD[(flux_dirn+2)%3],\
                       ]
    E_field_names = [\
                     "A_rhsD"+str((flux_dirn+1)%3),\
                     "A_rhsD"+str((flux_dirn+2)%3),\
                    ]

    desc = "Calculate the electric flux on the left face in direction " + str(flux_dirn) + "."
    name = "calculate_E_field_D" + str(flux_dirn) + "_right"
    outCfunction(
        outfile  = os.path.join(out_dir,subdir,name+".h"), desc=desc, name=name,
        params   ="const paramstruct *params,REAL *xx[3],const REAL *auxevol_gfs,REAL *rhs_gfs",
        body     =  Memory_Read \
                   +outputC(E_field_to_print,E_field_names,"returnstring",params="outCverbose=False").replace("IDX4","IDX4S")\
                   +Memory_Write,
        loopopts ="InteriorPoints",
        rel_path_for_Cparams=os.path.join("../"))

    desc = "Calculate the electric flux on the left face in direction " + str(flux_dirn) + "."
    name = "calculate_E_field_D" + str(flux_dirn) + "_left"
    outCfunction(
        outfile  = os.path.join(out_dir,subdir,name+".h"), desc=desc, name=name,
        params   ="const paramstruct *params,REAL *xx[3],const REAL *auxevol_gfs,REAL *rhs_gfs",
        body     =  Memory_Read.replace(indices[flux_dirn],indicesp1[flux_dirn]) \
                   +outputC(E_field_to_print,E_field_names,"returnstring",params="outCverbose=False").replace("IDX4","IDX4S")\
                   +Memory_Write,
        loopopts ="InteriorPoints",
        rel_path_for_Cparams=os.path.join("../"))

```

    Output C function calculate_E_field_D0_right() to file Validation/Afield_flux/calculate_E_field_D0_right.h
    Output C function calculate_E_field_D0_left() to file Validation/Afield_flux/calculate_E_field_D0_left.h
    Output C function calculate_E_field_D1_right() to file Validation/Afield_flux/calculate_E_field_D1_right.h
    Output C function calculate_E_field_D1_left() to file Validation/Afield_flux/calculate_E_field_D1_left.h
    Output C function calculate_E_field_D2_right() to file Validation/Afield_flux/calculate_E_field_D2_right.h
    Output C function calculate_E_field_D2_left() to file Validation/Afield_flux/calculate_E_field_D2_left.h

## Step 1.g: Calculate the *exact* flux of $\epsilon_{ijk} v^j B^k$ \[Back to [top](#toc)\]
$$\label{exact_flux}$$

Here, we generate a function to analytically calculate the electric flux on the cell faces for comparison. We'll need to import the Levi-Civita tensor for this.

```python
import WeylScal4NRPy.WeylScalars_Cartesian as weyl
# import GRHD.equations as GH
# GH.compute_sqrtgammaDET(gamma_faceDD)
LeviCivitaDDD = ixp.LeviCivitaSymbol_dim3_rank3()

driftvU = ixp.zerorank1()
for i in range(3):
    # TODO: don't use _face metric gridfunctions once we're in curved space!
    driftvU[i] = alpha_face*ValenciavU[i]-beta_faceU[i]

A_rhsD = ixp.zerorank1()
for i in range(3):
    for j in range(3):
        for k in range(3):
            A_rhsD[i] += -LeviCivitaDDD[i][j][k]*driftvU[j]*BU[k]

A_rhsD_to_print = [\
                   lhrh(lhs=gri.gfaccess("rhs_gfs","AD0"),rhs=A_rhsD[0]),\
                   lhrh(lhs=gri.gfaccess("rhs_gfs","AD1"),rhs=A_rhsD[1]),\
                   lhrh(lhs=gri.gfaccess("rhs_gfs","AD2"),rhs=A_rhsD[2]),\
                  ]

desc = "Calculate analytic electric field, part of the right-hand side of AD"
name = "calculate_E_exactD"
outCfunction(
    outfile  = os.path.join(out_dir,name+".h"), desc=desc, name=name,
    params   ="const paramstruct *params,REAL *xx[3],const REAL *auxevol_gfs,REAL *rhs_gfs",
    body     = fin.FD_outputC("returnstring",A_rhsD_to_print,params="outCverbose=False").replace("IDX4","IDX4S"),
    loopopts ="AllPoints,Read_xxs")

```

    Output C function calculate_E_exactD() to file Validation/calculate_E_exactD.h

# Step 2: `Induction_Equation_unit_test.c`: The Main C Code \[Back to [top](#toc)\]
$$\label{mainc}$$

Now that we have our vector potential and analytic magnetic field to compare against, we will start writing our unit test. We'll also import common C functionality, define `REAL`, the number of ghost zones, and the faces, and set the standard macros for NRPy+ style memory access.

```python
%%writefile $out_dir/Afield_flux_unit_test.c
// These are common packages that we are likely to need.
#include "stdio.h"
#include "stdlib.h"
#include "math.h"
#include "string.h" // Needed for strncmp, etc.
#include "stdint.h" // Needed for Windows GCC 6.x compatibility
#include <time.h>   // Needed to set a random seed.

#define REAL double
#include "declare_Cparameters_struct.h"

REAL a,b,c,d,e,f;

// Standard NRPy+ memory access:
#define IDX3S(i,j,k) ( (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * (k) ) )
#define IDX4S(g,i,j,k) \
( (i) + Nxx_plus_2NGHOSTS0 * ( (j) + Nxx_plus_2NGHOSTS1 * ( (k) + Nxx_plus_2NGHOSTS2 * (g) ) ) )

#define LOOP_REGION(i0min,i0max, i1min,i1max, i2min,i2max) \
  for(int i2=i2min;i2<i2max;i2++) for(int i1=i1min;i1<i1max;i1++) for(int i0=i0min;i0<i0max;i0++)

```

    Overwriting Validation//Afield_flux_unit_test.c

We'll now define the gridfunction names.

```python
%%writefile -a $out_dir/Afield_flux_unit_test.c
// Let's also #define the NRPy+ gridfunctions
#define VALENCIAVU0GF 0
#define VALENCIAVU1GF 1
#define VALENCIAVU2GF 2
#define BU0GF 3
#define BU1GF 4
#define BU2GF 5
#define VALENCIAV_RU0GF 6
#define VALENCIAV_RU1GF 7
#define VALENCIAV_RU2GF 8
#define B_RU0GF 9
#define B_RU1GF 10
#define B_RU2GF 11
#define VALENCIAV_LU0GF 12
#define VALENCIAV_LU1GF 13
#define VALENCIAV_LU2GF 14
#define B_LU0GF 15
#define B_LU1GF 16
#define B_LU2GF 17
#define GAMMA_FACEDD00GF 18
#define GAMMA_FACEDD01GF 19
#define GAMMA_FACEDD02GF 20
#define GAMMA_FACEDD11GF 21
#define GAMMA_FACEDD12GF 22
#define GAMMA_FACEDD22GF 23
#define BETA_FACEU0GF 24
#define BETA_FACEU1GF 25
#define BETA_FACEU2GF 26
#define ALPHA_FACEGF 27
#define FLUXD0GF 28
#define FLUXD1GF 29
#define FLUXD2GF 30
#define NUM_AUXEVOL_GFS 31

#define AD0GF 0
#define AD1GF 1
#define AD2GF 2
#define NUM_EVOL_GFS 3
```

    Appending to Validation//Afield_flux_unit_test.c

Now, we'll handle the different A2B codes. There are several things to do here. First, we'll add `#include`s to the C code so that we have access to the functions we want to test. We must also create a directory and copy the files to that directory. We will choose to do this in the subfolder `A2B` relative to this tutorial.

```python
%%writefile -a $out_dir/Afield_flux_unit_test.c
// Some specific definitions needed for this file
typedef struct __gf_and_gz_struct__ {
  REAL *gf;
  int gz_lo[4],gz_hi[4];
} gf_and_gz_struct;

const int VX=0,VY=1,VZ=2,BX=3,BY=4,BZ=5;
const int NUM_RECONSTRUCT_GFS = 6;

#include "Afield_flux/reconstruct_set_of_prims_PPM_GRFFE_NRPy.c"
#include "Afield_flux/loop_defines_reconstruction_NRPy.h"

#include "calculate_BU.h"
#include "calculate_ValenciavU.h"
#include "calculate_E_exactD.h"

// These are the functions we want to test.
#include "Afield_flux/calculate_E_field_D0_right.h"
#include "Afield_flux/calculate_E_field_D0_left.h"
#include "Afield_flux/calculate_E_field_D1_right.h"
#include "Afield_flux/calculate_E_field_D1_left.h"
#include "Afield_flux/calculate_E_field_D2_right.h"
#include "Afield_flux/calculate_E_field_D2_left.h"

```

    Appending to Validation//Afield_flux_unit_test.c

Now, we'll write the main method. First, we'll set up the grid. In this test, we cannot use only one point. As we are testing a three-point stencil, we can get away with a minimal $3 \times 3 \times 3$ grid. Then, we'll write the A fields. After that, we'll calculate the magnetic field two ways.

```python
%%writefile -a $out_dir/Afield_flux_unit_test.c
int main(int argc, const char *argv[]) {
    paramstruct params;
#include "set_Cparameters_default.h"

    // Step 0c: Set free parameters, overwriting Cparameters defaults
    //          by hand or with command-line input, as desired.
#include "free_parameters.h"
#include "set_Cparameters-nopointer.h"

    // We'll define our grid slightly different from how we normally would. We let our outermost
    // ghostzones coincide with xxmin and xxmax instead of the interior of the grid. This means
    // that the ghostzone points will have identical positions so we can do convergence tests of them.    // Step 0d.ii: Set up uniform coordinate grids
    REAL *xx[3];
    xx[0] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS0);
    xx[1] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS1);
    xx[2] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS2);
    for(int j=0;j<Nxx_plus_2NGHOSTS0;j++) xx[0][j] = xxmin[0] + (j-NGHOSTS)*dxx0;
    for(int j=0;j<Nxx_plus_2NGHOSTS1;j++) xx[1][j] = xxmin[1] + (j-NGHOSTS)*dxx1;
    for(int j=0;j<Nxx_plus_2NGHOSTS2;j++) xx[2][j] = xxmin[2] + (j-NGHOSTS)*dxx2;

    //for(int j=0;j<Nxx_plus_2NGHOSTS0;j++) printf("x[%d] = %.5e\n",j,xx[0][j]);

    // This is the array to which we'll write the NRPy+ variables.
    REAL *auxevol_gfs  = (REAL *)malloc(sizeof(REAL) * NUM_AUXEVOL_GFS * Nxx_plus_2NGHOSTS2 * Nxx_plus_2NGHOSTS1 * Nxx_plus_2NGHOSTS0);
    REAL *rhs_gfs  = (REAL *)malloc(sizeof(REAL) * NUM_EVOL_GFS * Nxx_plus_2NGHOSTS2 * Nxx_plus_2NGHOSTS1 * Nxx_plus_2NGHOSTS0);
    REAL *rhs_exact_gfs  = (REAL *)malloc(sizeof(REAL) * NUM_EVOL_GFS * Nxx_plus_2NGHOSTS2 * Nxx_plus_2NGHOSTS1 * Nxx_plus_2NGHOSTS0);

    LOOP_REGION(0,Nxx_plus_2NGHOSTS2,0,Nxx_plus_2NGHOSTS1,0,Nxx_plus_2NGHOSTS0) {
        auxevol_gfs[IDX4S(GAMMA_FACEDD00GF,i0,i1,i2)] = 1.0; // Flat Space
        auxevol_gfs[IDX4S(GAMMA_FACEDD01GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(GAMMA_FACEDD02GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(GAMMA_FACEDD11GF,i0,i1,i2)] = 1.0; // Flat Space
        auxevol_gfs[IDX4S(GAMMA_FACEDD12GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(GAMMA_FACEDD22GF,i0,i1,i2)] = 1.0; // Flat Space
        auxevol_gfs[IDX4S(BETA_FACEU0GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(BETA_FACEU1GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(BETA_FACEU2GF,i0,i1,i2)] = 0.0; // Flat Space
        auxevol_gfs[IDX4S(ALPHA_FACEGF,i0,i1,i2)] = 1.0; // Flat Space
    }

    a = (double)(rand()%20-10);
    b = (double)(rand()%20-10);
    c = (double)(rand()%20-10);
    d = (double)(rand()%20-10);
    e = (double)(rand()%20-10);
    f = (double)(rand()%20-10);
    //printf("a,b,c,d,e,f = %f,%f,%f,%f,%f,%f\n",a,b,c,d,e,f);

    // Calculate the initial data and the exact solution.
    calculate_BU(&params,xx,auxevol_gfs);
    calculate_ValenciavU(&params,xx,auxevol_gfs);
    //for(int j=0;j<Nxx_plus_2NGHOSTS0;j++) xx[0][j] -= 0.5*dxx0;
    calculate_E_exactD(&params,xx,auxevol_gfs,rhs_exact_gfs);

    // Set up the pointers and constants for the reconstruction procedure.
    gf_and_gz_struct in_prims[NUM_RECONSTRUCT_GFS], out_prims_r[NUM_RECONSTRUCT_GFS], out_prims_l[NUM_RECONSTRUCT_GFS];
    int which_prims_to_reconstruct[NUM_RECONSTRUCT_GFS],num_prims_to_reconstruct;

    const int Nxxp2NG012 = Nxx_plus_2NGHOSTS0*Nxx_plus_2NGHOSTS1*Nxx_plus_2NGHOSTS2;
    REAL temporary[Nxxp2NG012];

    int ww=0;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*VALENCIAVU0GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_RU0GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_LU0GF;
    ww++;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*VALENCIAVU1GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_RU1GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_LU1GF;
    ww++;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*VALENCIAVU2GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_RU2GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*VALENCIAV_LU2GF;
    ww++;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*BU0GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*B_RU0GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*B_LU0GF;
    ww++;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*BU1GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*B_RU1GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*B_LU1GF;
    ww++;
    in_prims[ww].gf      = auxevol_gfs + Nxxp2NG012*BU2GF;
      out_prims_r[ww].gf = auxevol_gfs + Nxxp2NG012*B_RU2GF;
      out_prims_l[ww].gf = auxevol_gfs + Nxxp2NG012*B_LU2GF;
    ww++;

    // Prims are defined AT ALL GRIDPOINTS, so we set the # of ghostzones to zero:
    for(int i=0;i<NUM_RECONSTRUCT_GFS;i++) for(int j=1;j<=3;j++) { in_prims[i].gz_lo[j]=0; in_prims[i].gz_hi[j]=0; }
    // Left/right variables are not yet defined, yet we set the # of gz's to zero by default:
    for(int i=0;i<NUM_RECONSTRUCT_GFS;i++) for(int j=1;j<=3;j++) { out_prims_r[i].gz_lo[j]=0; out_prims_r[i].gz_hi[j]=0; }
    for(int i=0;i<NUM_RECONSTRUCT_GFS;i++) for(int j=1;j<=3;j++) { out_prims_l[i].gz_lo[j]=0; out_prims_l[i].gz_hi[j]=0; }

    ww=0;
    which_prims_to_reconstruct[ww]=VX; ww++;
    which_prims_to_reconstruct[ww]=VY; ww++;
    which_prims_to_reconstruct[ww]=VZ; ww++;
    which_prims_to_reconstruct[ww]=BX; ww++;
    which_prims_to_reconstruct[ww]=BY; ww++;
    which_prims_to_reconstruct[ww]=BZ; ww++;
    num_prims_to_reconstruct=ww;

    // In each direction, perform the PPM reconstruction procedure.
    // Then, add the fluxes to the RHS as appropriate.
    for(int flux_dirn=0;flux_dirn<3;flux_dirn++) {
        // This function is housed in the file: "reconstruct_set_of_prims_PPM_GRFFE_NRPy.c"
        reconstruct_set_of_prims_PPM_GRFFE_NRPy(&params, auxevol_gfs, flux_dirn+1, num_prims_to_reconstruct,
                                                which_prims_to_reconstruct, in_prims, out_prims_r, out_prims_l, temporary);
        if(flux_dirn==0) {
            calculate_E_field_D0_right(&params,xx,auxevol_gfs,rhs_gfs);
            calculate_E_field_D0_left(&params,xx,auxevol_gfs,rhs_gfs);
        }
        else if(flux_dirn==1) {
            calculate_E_field_D1_right(&params,xx,auxevol_gfs,rhs_gfs);
            calculate_E_field_D1_left(&params,xx,auxevol_gfs,rhs_gfs);
        }
        else {
            calculate_E_field_D2_right(&params,xx,auxevol_gfs,rhs_gfs);
            calculate_E_field_D2_left(&params,xx,auxevol_gfs,rhs_gfs);
        }
    }

    char filename[100];
    sprintf(filename,"out%d-numer.txt",Nxx0);
    FILE *out2D = fopen(filename, "w");
    // We print the difference between approximate and exact numbers.
    int i0 = Nxx_plus_2NGHOSTS0/2;
    int i1 = Nxx_plus_2NGHOSTS1/2;
    int i2 = Nxx_plus_2NGHOSTS2/2;

    printf("Numerical: %.15e\n",rhs_gfs[IDX4S(AD0GF,i0,i1,i2)]);
    printf("Analytic:  %.15e\n",rhs_exact_gfs[IDX4S(AD0GF,i0,i1,i2)]);
    printf("Numerical: %.15e\n",rhs_gfs[IDX4S(AD1GF,i0,i1,i2)]);
    printf("Analytic:  %.15e\n",rhs_exact_gfs[IDX4S(AD1GF,i0,i1,i2)]);
    printf("Numerical: %.15e\n",rhs_gfs[IDX4S(AD2GF,i0,i1,i2)]);
    printf("Analytic:  %.15e\n\n",rhs_exact_gfs[IDX4S(AD2GF,i0,i1,i2)]);

    //printf("i0,i1,i2 = %d,%d,%d\n",i0,i1,i2);
    fprintf(out2D,"%.16e\t%.16e\t%.16e\t %e %e %e\n",
            rhs_exact_gfs[IDX4S(AD0GF,i0,i1,i2)]-rhs_gfs[IDX4S(AD0GF,i0,i1,i2)],
            rhs_exact_gfs[IDX4S(AD1GF,i0,i1,i2)]-rhs_gfs[IDX4S(AD1GF,i0,i1,i2)],
            rhs_exact_gfs[IDX4S(AD2GF,i0,i1,i2)]-rhs_gfs[IDX4S(AD2GF,i0,i1,i2)],
            xx[0][i0],xx[1][i1],xx[2][i2]
            );
    fclose(out2D);

}

```

    Appending to Validation//Afield_flux_unit_test.c

## Step 2.a: Compile and run the code

$$\label{compile_run}$$

Now that we have our file, we can compile it and run the executable.

```python
import time

print("Now compiling, should take ~2 seconds...\n")
start = time.time()
cmd.C_compile(os.path.join(out_dir,"Afield_flux_unit_test.c"), os.path.join(out_dir,"Afield_flux_unit_test"))
end = time.time()
print("Finished in "+str(end-start)+" seconds.\n\n")

print("Now running...\n")
start = time.time()
!./Validation/Afield_flux_unit_test 2 2 2
# To do a convergence test, we'll also need a second grid with twice the resolution.
!./Validation/Afield_flux_unit_test 4 4 4
end = time.time()
print("Finished in "+str(end-start)+" seconds.\n\n")

```

    Now compiling, should take ~2 seconds...

    Compiling executable...
    Executing `gcc -Ofast -fopenmp -march=native -funroll-loops Validation/Afield_flux_unit_test.c -o Validation/Afield_flux_unit_test -lm`...
    Finished executing in 1.0149364471435547 seconds.
    Finished compilation.
    Finished in 1.0259525775909424 seconds.

    Now running...

    Numerical: 1.950138053436466e-01
    Analytic:  1.950011308136350e-01
    Numerical: 1.781172160356394e-03
    Analytic:  1.362031316105586e-03
    Numerical: -2.008645577931328e-01
    Analytic:  -1.963631621297406e-01

    Numerical: 1.950043012242935e-01
    Analytic:  1.950011308136350e-01
    Numerical: 1.575001178639817e-03
    Analytic:  1.362031316105586e-03
    Numerical: -1.986137199980445e-01
    Analytic:  -1.963631621297406e-01

    Finished in 0.26262760162353516 seconds.

# Step 3: Code validation: Verify that relative error in numerical solution converges to zero at the expected order \[Back to [top](#toc)\]
$$\label{convergence}$$

For testing purposes, we have only checked these algorithms on a small grid. By construction, we have only guaranteed ourselves output from the functions we are testing at a point, so we will simply print the convergence order at that point after processing our outputs below.

```python
import numpy as np
import matplotlib.pyplot as plt

Data1 = np.loadtxt("out2-numer.txt")
Data2 = np.loadtxt("out4-numer.txt")

print("The following quantities converge at the listed order (should be ~1 for Shock data, ~2 otherwise):")
print("A_rhsD0: "+str(np.log2(np.abs(Data1[0]/Data2[0]))))
print("A_rhsD1: "+str(np.log2(np.abs(Data1[1]/Data2[1]))))
print("A_rhsD2: "+str(np.log2(np.abs(Data1[2]/Data2[2]))))

```

    The following quantities converge at the listed order (should be ~1 for Shock data, ~2 otherwise):
    A_rhsD0: 1.999190623480292
    A_rhsD1: 0.9767858269857116
    A_rhsD2: 1.0000897191830145

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]

$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-A2B.pdf](Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-A2B.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-Start_to_Finish_UnitTest-GiRaFFE_NRPy-Afield_flux")
```

    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode


## Data Analytics

### Confidence Intervals and Hypothesis Testing for Data Analytics in Python

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

##### Reporting Uncertainty and Significance

With confidence intervals and hypothesis testing we have the opportunity to report uncertainty and to report significance in our statistics. These methods are based on standard methods with their associated limitations and assumptions. For additional learning content to support learning about confidence intervals see [Confidence Intervals Lecture](https://youtu.be/oaXCcTWcU04) for more details and the [Confidence Intervals Interactive Demonstration](https://github.com/GeostatsGuy/PythonNumericalDemos/blob/master/Interactive_Confidence_Interval.ipynb) to play with and observe confidence intervals in real-time! For more information about hypothesis testing see [Hypothesis Testing Lecture](https://youtu.be/rvt9UM148tQ) and [Hypothesis Testing Interactive Demonstration](https://github.com/GeostatsGuy/PythonNumericalDemos/blob/master/Interactive_Hypothesis_Testing.ipynb).

Also, for more information about statistical bootstrap see the [Bootstrap Lecture](https://youtu.be/oaXCcTWcU04) and [Bootstrap Workflow](https://git.io/fhgUW) for a general, empirical approach to assess uncertainty in statistics and models and [Bootstrap Interactive Demonstration](https://github.com/GeostatsGuy/PythonNumericalDemos/blob/master/Interactive_Bootstrap_Simple.ipynb).

This is a tutorial / demonstration of **Confidence Intervals and Hypothesis Testing in Python** for data analytics and data science workflows.  In Python, the SciPy package, specifically the [SciPy Stats Functions](https://docs.scipy.org/doc/scipy/reference/stats.html) provides excellent statistics tools.

##### More Comfortable in Excel or R?

I have previously provided these examples worked out by-hand in [Excel](https://github.com/GeostatsGuy/LectureExercises/blob/master/Lecture7_CI_Hypoth_eg_R.xlsx) and also in [R](https://github.com/GeostatsGuy/LectureExercises/blob/master/Lecture7_CI_Hypoth_eg.R).  In all cases, I use the same [dataset](https://git.io/fxLAt) available as a comma delimited file in my [GeoDataSets](https://github.com/GeostatsGuy/GeoDataSets) repository. Feel free to compare the workflows and results as a bridge to working in Python.

##### Topics Covered

This tutorial, well-documented workflow, includes basic, typical confidence interval and hypothesis testing methods that would commonly be required for Engineers and Geoscientists including:

1. Student-t confidence interval for the mean
2. Student-t hypothesis test for difference in means (pooled variance)
3. Student-t hypothesis test for difference in means (difference variances), Welch's t Test
3. F-distribution hypothesis test for difference in variances

#### Workflow Goal

0. Introduction to Python in Jupyter including setting a working directory, loading data into a Pandas DataFrame.
1. Learn the basics for working with confidence intervals and hypothesis testing in Python.
2. Demonstrate the efficiency of using Python and SciPy package for statistical analysis.
3. Learn how to quantify uncertainty and significance in samples.

#### Objective

I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows.

The objective is to remove the hurdles of data science modeling workflow construction by providing building blocks and sufficient examples. My courses are not a coding classes per se, but we need the ability to 'script' workflows working with numerical methods.

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal.
3. In the terminal type: pip install geostatspy.
4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

You will need to copy the data file to your working directory.  They are available here:

* Tabular data - PorositySample2Units.csv at https://git.io/fhrM8

There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code.

#### Load the required libraries

The following code loads the required libraries.

```python
import geostatspy.GSLIB as GSLIB                       # GSLIB utilies, visualization and wrapper
import geostatspy.geostats as geostats                 # GSLIB methods convert to Python
```

We will also need some standard packages. These should have been installed with Anaconda 3.

```python
import os                                              # to set current working directory
import numpy as np                                     # arrays and matrix math
import scipy.stats as st                               # statistical methods
import pandas as pd                                    # DataFrames
import matplotlib.pyplot as plt                        # plotting
from matplotlib.ticker import (MultipleLocator, AutoMinorLocator) # control of axes ticks
plt.rc('axes', axisbelow=True)                         # set axes and grids in the background for all plots
```

If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.

#### Set the working directory

I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see below) data file in this directory.  When we are done with this tutorial we will write our new dataset back to this directory.

```python
#os.chdir("C:\PGE383")                                  # set the working directory
```

#### Loading Data

Let's load the provided dataset. 'PorositySamples2Units.csv' is available at https://github.com/GeostatsGuy/GeoDataSets. It is a comma delimited file with 20 porosity measures from 2 rock units from the subsurface, porosity (as a fraction). We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it by printing a slice and by utilizing the 'head' DataFrame member function (with a nice and clean format, see below).

```python
#df = pd.read_csv("PorositySample2Units.csv")           # read a .csv file in as a DataFrame
df = pd.read_csv("https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/PorositySample2Units.csv") # load data from Dr.Pyrcz's GitHub repository
#print(df.iloc[0:5,:])                                 # display first 4 samples in the table as a preview
df.head()                                              # we could also use this command for a table preview
```

      X1
      X2

      0
      0.21
      0.20

      1
      0.17
      0.26

      2
      0.15
      0.20

      3
      0.20
      0.19

      4
      0.19
      0.13

#### Checking Data Summary Statistics

It is useful to review the summary statistics of our loaded DataFrame.  That can be accomplished with the 'describe' DataFrame member function.  We transpose to switch the axes for ease of visualization.

```python
df.describe().transpose()                              # visualize summary statistics
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      X1
      20.0
      0.1645
      0.027810
      0.11
      0.1500
      0.17
      0.19
      0.21

      X2
      20.0
      0.2000
      0.045422
      0.11
      0.1675
      0.20
      0.23
      0.30

Here we extract the X1 and X2 unit porosity samples from the DataFrame into separate arrays called 'Por1' and 'Por2' for convenience.

```python
por1 = df['X1'].values                                 # extract well 1 porosity to a ndarray
por2 = df['X2'].values                                 # extract well 2 porosity to a ndarray
```

#### Visualizing the Data Distributions

We should first visualize the distributions.  It is convenient to plot the distributions on top of each other.

* we compare the histograms and cumulative distribution functions (CDFs)

```python
plt.subplot(121)
plt.hist(por1, facecolor='red',bins=np.linspace(0.0,0.4,20),alpha=0.8,density=False,edgecolor='black',linewidth=2,label='Well 1')
plt.hist(por2, facecolor='gold',bins=np.linspace(0.0,0.4,20),alpha=0.6,density=False,edgecolor='black',linewidth=2,label = 'Well 2')
plt.xlim([0.0,0.4]); plt.ylim([0,8.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Frequency'); plt.title('Porosity Well 1 and 2')
plt.legend(loc='upper left')
plt.gca().grid(True, which='major',axis='both',linewidth = 1.0); plt.gca().grid(True, which='minor',axis='x',linewidth = 0.2) # add y grids
plt.gca().tick_params(which='major',length=7); plt.gca().tick_params(which='minor', length=4)
plt.gca().xaxis.set_minor_locator(AutoMinorLocator()); plt.gca().yaxis.set_minor_locator(AutoMinorLocator()) # turn on minor ticks

plt.subplot(122)
plt.hist(por1, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=1.0,density=True,cumulative=True,edgecolor='black',linewidth=2,label='Well 1')
plt.hist(por2, facecolor='gold',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=1.0,density=True,cumulative=True,edgecolor='black',linewidth=2,label='Well 2')
plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Well 1 and 2')
plt.legend(loc='upper left')
plt.grid(True, which='major',linewidth = 1.0); plt.grid(True, which='minor',linewidth = 0.2) # add y grids
plt.gca().tick_params(which='major',length=7); plt.gca().tick_params(which='minor', length=4)
plt.gca().xaxis.set_minor_locator(AutoMinorLocator()); plt.gca().yaxis.set_minor_locator(AutoMinorLocator()) # turn on minor ticks

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

The histogram and cumulative distribution functions indicate the porosity distributions in both wells 1 and 2. Ocular inspection does indicate:

1. not a lot of samples are available for each (n=20)
2. the distributions look different

Let's use statistics to learn more about the subsurface from these two wells.

#### Confidence Intervals

Let's first demonstrate the calculation of the confidence interval for the sample mean at a 95% confidence level.  We interpret the confidence interval as the interval over which there is a 95% confidence that it contains the true population mean. We use the student's t distribution as we assume we do not know the variance and the sample size is small.

\begin{equation}
x̅ \pm t_{\frac{\alpha}{2},n-1} \times \frac {s}{\sqrt{n}}
\end{equation}

```python
ci_95_por1 = st.t.interval(0.95, len(df)-1, loc=np.mean(por1), scale=st.sem(por1))
print('The confidence interval for the Well 1 mean porosity is ' + str(round(np.mean(por1),3)) + ' +/- ' + str(round(ci_95_por1[1]-np.mean(por1),3))
      + ', with a range of ' + str(round(ci_95_por1[0],3)) + ', ' + str(round(ci_95_por1[1],3)))

ci_95_por2 = st.t.interval(0.95, len(df)-1, loc=np.mean(por2), scale=st.sem(por2))
print('The confidence interval for the Well 2 mean porosity is ' + str(round(np.mean(por2),3)) + ' +/- ' + str(round(ci_95_por2[1]-np.mean(por2),3))
      + ', with a range of ' + str(round(ci_95_por2[0],3)) + ', ' + str(round(ci_95_por2[1],3)))
```

    The confidence interval for the Well 1 mean porosity is 0.164 +/- 0.013, with a range of 0.151, 0.178
    The confidence interval for the Well 2 mean porosity is 0.2 +/- 0.021, with a range of 0.179, 0.221

#### Using Confidence Intervals to Build an Uncertainty Model

The confidence interval for the mean could be applied directly as an uncertainty model! The uncertainty in mean porosity at each well could be applied to calculate the uncertainty in the mean porosity applied to model porosity around the two wells. This method would account for the limited number of porosity samples available. If more porosity samples we available then this uncertainty would decrease (note the $\sqrt{n}$ in the denominator of the confidence interval in the mean calculation above.

One possible work flow would be to apply the affine correction from the *GeostatsPy* package to formulate a the P5 and P95 porosity distributions for porosity in each well. Note, these distribution scenarios are based only on uncertainty in the mean porosity and assumes the distribution tails are not anchored (are free to shift).

```python
stdev1 = np.std(por1); stdev2 = np.std(por2)
por1r05 = GSLIB.affine(por1,tmean=ci_95_por1[0],tstdev=stdev1)
por1r95 = GSLIB.affine(por1,tmean=ci_95_por1[1],tstdev=stdev1)
por2r05 = GSLIB.affine(por2,tmean=ci_95_por2[0],tstdev=stdev2)
por2r95 = GSLIB.affine(por2,tmean=ci_95_por2[1],tstdev=stdev2)

plt.subplot(121)
plt.hist(por1r05, facecolor='yellow',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='P05')
plt.hist(por1, facecolor='orange',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='Expected')
plt.hist(por1r95, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='P95')

plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Distribution Uncertainty - Well 1')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplot(122)
plt.hist(por2r05, facecolor='yellow',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='P05')
plt.hist(por2, facecolor='orange',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='Expected')
plt.hist(por2r95, facecolor='red',bins=np.linspace(0.0,0.4,50),histtype="stepfilled",alpha=0.8,density=True,cumulative=True,edgecolor='black',linewidth=2,label='P95')

plt.xlim([0.0,0.4]); plt.ylim([0,1.0])
plt.xlabel('Porosity (fraction)'); plt.ylabel('Cumulative Probability'); plt.title('Porosity Distribution Uncertainty - Well 2')
plt.legend(loc='upper left')
plt.grid(True)

plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.2, hspace=0.3)
plt.show()
```

We just calculated a scenario-based uncertainty model for the porosity distribution around wells 1 and 2. Of course, we could actually sample porosity means continuously from our confidence calculation as we have access to the complete distribution for uncertainty in the mean porosity of both wells.

Communicating uncertainty is powerful, but always remember to state the assumptions. For example, here we assumed:

1. The population distribution of porosity for each well is Gaussian distributed
2. That the samples are independent.
3. The distribuiton dispersion and shape don't change as we vary the (account for uncertainty in the) mean.

One can check the Excel file linked above with the confidence interval calculated by hand and confirm that this result is correct.

#### Hypothesis Testing

The confidence intervals help with uncertainty in the distributions of porosity.  Now let's try to figure out if:

1. wells 1 and 2 drilled into the same type of rock?
2. did something change between the 2 wells?
3. different units are being compared between the 2 wells (issues with stratal correlation)?

Now, let's try the t test, hypothesis test for difference in means.  This test assumes that the variances are similar along with the data being Gaussian distributed (see the course notes for more on this).  This is our test:

\begin{equation}
H_0: \mu_{X1} = \mu_{X2}
\end{equation}

\begin{equation}
H_1: \mu_{X1} \ne \mu_{X2}
\end{equation}

For the resulting t-statistic and p-value we run this command.

```python
t_pooled, p_pooled = st.ttest_ind(por1,por2)
print('The t statistic is ' + str(round(t_pooled,3)) + ' and the p-value is ' + str(round(p_pooled,3)))
```

    The t statistic is -2.981 and the p-value is 0.005

The p-value, $p$, is the symmetric interval probaiblity our outside.  In other words the $p$ reported is 2 x cumulative probaiblity of the t statistic applied to the sampling t distribution.  Another way to look at it, if one used the $\pm t_{t_{statistic},.d.f}$ statistic as thresholds, $p$ is the probability being outside this symmetric interval. So we will reject the null hypothesis if $p \lt \alpha$.  From the p-value alone it is clear that we would **reject the null hypothesis** and **accept the alternative hypothesis** that the means are not equal.

In case you want to compare the t-statistic to t-critical, we can apply the inverse of the student's t distribution at $\frac{\alpha}{2}$ and $1-\frac{\alpha}{2}$ to get the upper and lower critcal values.

```python
t_critical = st.t.ppf([0.025,0.975], df=len(por1)+len(por2)-2)
print('The t crical lower and upper values are ' + str(np.round(t_critical,2)))
```

    The t crical lower and upper values are [-2.02  2.02]

We can observe that, as expected, the t-statistic is outside the t-critcal interval.  These results are exactly what we got when we worked out the problem by hand in Excel, but so much more efficient!

Now let's try the t-test, hypothesis test for difference in means allowing for unequal variances, this is also known as the Welch's t test.  All we have to do is set the parameter 'equal_var' to false, note it defaults to true (e.g. the command above).

```python
st.ttest_ind(por1, por2, equal_var = False)
```

    Ttest_indResult(statistic=-2.9808897468855644, pvalue=0.005502572350112333)

Once again we can see by $p$ that we will clearly reject the null hypothesis.

Let's now compare the variances with the F-test for difference in variances.

\begin{equation}
H_0: \sigma^{2}_{X2} \le \sigma^{2}_{X1}
\end{equation}

\begin{equation}
H_1: \sigma^{2}_{X2} \gt \sigma^{2}_{X1}
\end{equation}

Note, by ordering the variances we eliminate the case of $\sigma^{2}_{X2} \lt \sigma^{2}_{X1}$.

Details about the test are available in the course notes (along with assumptions such as Gaussian distributed) and this example is also worked out by hand in the linked Excel workbook.  We can accomplish the F-test in with SciPy.Stats the function with one line of code if we calculate the ratio of the sample variances ensuring that the larger variance is in the numerator and get the degrees of freedom using the len() command, ensuring that we are consistent with the numerator degrees of freedom set as 'dfn' and the denominator degrees of freedom set as 'dfd'.  We take a p-value of $1-p$ since the test is configured to be a single, right tailed test.

```python
p_value = 1 - st.f.cdf(np.var(por2)/np.var(por1), dfn=len(por2)-1, dfd=len(por1)-1)
print('The p-value for the F-test for difference is variances is ' + str(round(p_value,3)))
```

    The p-value for the F-test for difference is variances is 0.019

Once again we would clearly reject the null hypothesis since $p \lt alpha$ and assume that the variances are not equal.  Note this is a single tail test so our p-critical is 0.05 for a 95% confidence level.

#### Reporting the Hypothesis Test Results

The difference in the means and variances were tested for statistical significance for porosity measured in wells 1 and 2.  In all cases we rejected the null hypotheses and adopted the alternative hypotheses that the porosity distributions' means and variances for well 1 and 2 are statistically significant in their differences. It is possible that Well 1 and 2 have drilled into different rock units or have encountered nonstationary behavoirs in a single rock unit. This result may be applied to update subsurface maps and consider a difference in the subsurface at each well for development decision making.

#### Comments

We are just scratching the surface for the application of confidence intervals and hypothesis tests to subsurface modeling.  Once again, there are a lot of details left out of the problem formulation and assumptions, see the course notes for more coverage.  By running the same confidence interval and hypothesis tests: 1) by hand in Excel and with 2) R and now here in 3) Python code, I hope this demonstration will enable and encourage more engineers and scientists to make these R and Python tools part of their common practice. I'm always happy to discuss,

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)


# Tips and Tricks

A markdown cell lets you do many _useful things_.

```python
# Add this to python2 code to make life easier -- safe to use in python3 as well.
from __future__ import absolute_import, division, print_function
```

```python
import numpy as np
# don't do:
# from numpy import *
```

```python
max('a')
```

    'a'

```python
np.max('a')
```

```python
%matplotlib inline
%config InlineBackend.figure_format='retina'
```

```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import ipywidgets
import os
import sys
import warnings
warnings.filterwarnings('ignore')
```

# Keyboard shortcuts

Practice doing these a few times and try to force yourself to use them whenever you can.

## Help (h)
For help in the Notebook, `ESC` + `h`
(doesn't work in Lab)

## Select multiple cells

j, k, and arrow keys while holding shift selects multiple cells.

```python
first = 1
```

```python
second = 2
```

```python
third = 3
```

## Split a cell with -

Well, ctrl + shift + -, but remember it by horizontal.

```python

```

```python

```

## Add a new cell (a)bove

## Add a new cell (b)elow

```python

```

## (d)(d)elete

```python

```

```python

```

## (c)opy cells

## (v)aste cells

# Pivot Tables w/ pandas

A library and example from: http://nicolas.kruchten.com/content/2015/09/jupyter_pivottablejs/

```python
df = pd.read_csv("../data/mps.csv", encoding="ISO-8859-1")
```

```python
# recommend using .head()
df.head(10)
```

      Name
      Party
      Province
      Age
      Gender

      0
      Liu, Laurin
      NDP
      Quebec
      22.0
      Female

      1
      Mourani, Maria
      Bloc Quebecois
      Quebec
      43.0
      Female

      2
      Sellah, Djaouida
      NDP
      Quebec
      NaN
      Female

      3
      St-Denis, Lise
      NDP
      Quebec
      72.0
      Female

      4
      Fry, Hedy
      Liberal
      British Columbia
      71.0
      Female

      5
      Turmel, Nycole
      NDP
      Quebec
      70.0
      Female

      6
      Sgro, Judy
      Liberal
      Ontario
      68.0
      Female

      7
      Raynault, Francine
      NDP
      Quebec
      67.0
      Female

      8
      Davidson, Patricia
      Conservative
      Ontario
      66.0
      Female

      9
      Smith, Joy
      Conservative
      Manitoba
      65.0
      Female

# Enhanced Pandas Dataframe Display

```python
# Province, Party, Average, Age, Heatmap
```

```python
from pivottablejs import pivot_ui
```

```python
pivot_ui(df)
```

# Tab

```python
import numpy as np
```

```python
np.random.chisquare(size=5)
```

```python
# pure tab right ↓ less useful
np.random.choice()
```

## shift-tab

```python
# shift-tab right ↓ more useful
np.linspace(start=, endpoint=, )
```

## shift-tab-tab

(equivalent in in Lab to shift-tab)

```python
np.linspace(50, 150, num=100, endpoint=False, )
```

## shift-tab-tab-tab-tab

(doesn't work in lab)

```python
plt.plot(np.linspace(start, stop,), )
```

## ?

```python
np.linspace?
```

```python
?np.linspace
```

    Signature: np.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None)
    Docstring:
    Return evenly spaced numbers over a specified interval.

    Returns `num` evenly spaced samples, calculated over the
    interval [`start`, `stop`].

    The endpoint of the interval can optionally be excluded.

    Parameters
    ----------
    start : scalar
        The starting value of the sequence.
    stop : scalar
        The end value of the sequence, unless `endpoint` is set to False.
        In that case, the sequence consists of all but the last of ``num + 1``
        evenly spaced samples, so that `stop` is excluded.  Note that the step
        size changes when `endpoint` is False.
    num : int, optional
        Number of samples to generate. Default is 50. Must be non-negative.
    endpoint : bool, optional
        If True, `stop` is the last sample. Otherwise, it is not included.
        Default is True.
    retstep : bool, optional
        If True, return (`samples`, `step`), where `step` is the spacing
        between samples.
    dtype : dtype, optional
        The type of the output array.  If `dtype` is not given, infer the data
        type from the other input arguments.

        .. versionadded:: 1.9.0

    Returns
    -------
    samples : ndarray
        There are `num` equally spaced samples in the closed interval
        ``[start, stop]`` or the half-open interval ``[start, stop)``
        (depending on whether `endpoint` is True or False).
    step : float, optional
        Only returned if `retstep` is True

        Size of spacing between samples.

    See Also
    --------
    arange : Similar to `linspace`, but uses a step size (instead of the
             number of samples).
    logspace : Samples uniformly distributed in log space.

    Examples
    --------
    >>> np.linspace(2.0, 3.0, num=5)
    array([ 2.  ,  2.25,  2.5 ,  2.75,  3.  ])
    >>> np.linspace(2.0, 3.0, num=5, endpoint=False)
    array([ 2. ,  2.2,  2.4,  2.6,  2.8])
    >>> np.linspace(2.0, 3.0, num=5, retstep=True)
    (array([ 2.  ,  2.25,  2.5 ,  2.75,  3.  ]), 0.25)

    Graphical illustration:

    >>> import matplotlib.pyplot as plt
    >>> N = 8
    >>> y = np.zeros(N)
    >>> x1 = np.linspace(0, 10, N, endpoint=True)
    >>> x2 = np.linspace(0, 10, N, endpoint=False)
    >>> plt.plot(x1, y, 'o')
    [<matplotlib.lines.Line2D object at 0x...>]
    >>> plt.plot(x2, y + 0.5, 'o')
    [<matplotlib.lines.Line2D object at 0x...>]
    >>> plt.ylim([-0.5, 1])
    (-0.5, 1)
    >>> plt.show()
    File:      ~/miniconda3/envs/dspy3/lib/python3.6/site-packages/numpy/core/function_base.py
    Type:      function

## ??

(Lab can scroll if you click)

```python
np.linspace??
```

```python
!subl ~/miniconda3/envs/dspy3/lib/python3.6/site-packages/numpy/core/function_base.py
```

## Inspect _everything_ and Find and Replace

```python
# But first find and replace
def silly_function(xval):
    """Takes a value and returns the value."""
    xval_sq = xval ** 2.0
    1 + 15
    xval_abs = np.sqrt(xval_sq)
    return xval_abs
```

```python
silly_function(2,)
```

```python
silly_function?
```

```python
silly_function??
```

## Line numbers (lowercase "L")

Type lowercase "L" to have line numbers; shift-L for line numbers notebook-wide.

## Move blocks of code around

### Indent/dedent

    Cmd + [
    Cmd + ]

### Comment

    Cmd + /

```python
example_dict = {}

# Indent/dedent/comment
for _ in range(5):
    example_dict["one"] = 1
    example_dict["two"] = 2
    example_dict["three"] = 3
    example_dict["four"] = 4
```

## Multicursor magic

Hold down `option`, click and drag (for big cursor).

cmd + click == wherever you click.

Shift command P -- command palette.

```python
example_dict["one_better"] = 1.0
example_dict["two_better"] = 2.0
example_dict["three_better"] = 3.0
example_dict["four_better"] = 4.0
```

## Monospace

You can also get `monospaced` fonts by indenting 4 spaces:

    mkdir toc
    cd toc

## Syntax Highlighting

Wrap with triple-backticks and language:

```bash
mkdir toc
cd toc
wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
```

```SQL
SELECT first_name,
       last_name,
       year_of_birth
FROM presidents
WHERE year_of_birth > 1800;
```

## Headings and LaTeX

With text and $\LaTeX$ support.

$$\begin{align}
        B'&=-\nabla \times E,\\
        E'&=\nabla \times B - 4\pi j
\end{align}$$

# Magics

 - `%` $\equiv$ inline magic
 - `%%` $\equiv$ cell magic

```latex
%%latex

If you want to get crazier$\ldots$

\begin{equation}
\oint_S {E_n dA = \frac{1}{{\varepsilon _0 }}} Q_\textrm{inside}
\end{equation}
```

If you want to get crazier$\ldots$

\begin{equation}
\oint_S {E_n dA = \frac{1}{{\varepsilon _0 }}} Q_\textrm{inside}
\end{equation}

```bash
%%bash
wget http://www.ast.cam.ac.uk/~rfc/vpfit10.2.tar.gz
mkdir vpfit10
cd vpfit10
tar -xvzf ../vpfit10.2.tar.gz
```

    ./
    ./00README
    ./atom.dat
    ./atomtab.f
    ./autovpin.f
    ./bintofits.f
    ./calcn.f
    ./case.f
    ./chanwav.f
    ./checkb.f
    ./chisum26.f
    ./cmax1.f
    ./cms_src.f
    ./com_cons.f
    ./dcholin.f
    ./dexpf.f
    ./dlycabscft.f
    ./docs/
    ./docs/rdgen10.2.pdf
    ./docs/vpfit10.2.pdf
    ./dotick.f
    ./dpoly.f
    ./dsepvar.f
    ./eigen.f
    ./erfcc.f
    ./externals.f
    ./fl_gen.f
    ./gen_vpin.f
    ./ginsert.f
    ./gu_tabsort.f
    ./hedit.f
    ./linfit.f
    ./lm_udchole.f
    ./lyregset.f
    ./makefile
    ./medianv.f
    ./MMatomdat_atom_noiso.dat
    ./pda_che2d.f
    ./pda_dp1vlu.f
    ./pda_dpcoef.f
    ./pda_dpolft.f
    ./pda_qsiad.f
    ./pda_src.f
    ./pda_xermsg.f
    ./pgfiles/
    ./pgfiles/Al.pg
    ./pgfiles/AlSi.pg
    ./pgfiles/CaII.pg
    ./pgfiles/CI2.pg
    ./pgfiles/CIH2.pg
    ./pgfiles/CII.pg
    ./pgfiles/CII.pg~
    ./pgfiles/CIIOI.pg
    ./pgfiles/CIIOI.pg~
    ./pgfiles/CIISi.pg
    ./pgfiles/CIV.pg
    ./pgfiles/CIVSiIV.pg
    ./pgfiles/CrZn.pg
    ./pgfiles/FeII.pg
    ./pgfiles/FeIIhi.pg
    ./pgfiles/hc.pg
    ./pgfiles/HC4.pg
    ./pgfiles/HCNlo.pg
    ./pgfiles/hcnosi.pg
    ./pgfiles/hcsi.pg
    ./pgfiles/HD.pg
    ./pgfiles/HDs.pg
    ./pgfiles/HHC4N5O6.pg
    ./pgfiles/HHC4O6.pg
    ./pgfiles/HHC4O6Si4.pg
    ./pgfiles/HHC4Si4.pg
    ./pgfiles/HHO6.pg
    ./pgfiles/hiHCNO.pg
    ./pgfiles/hiHCOSi.pg
    ./pgfiles/hiion.pg
    ./pgfiles/hiSi.pg
    ./pgfiles/HNI.pg
    ./pgfiles/HOhi.pg
    ./pgfiles/HQhi.pg
    ./pgfiles/Hvhi.pg
    ./pgfiles/LaCIV.pg
    ./pgfiles/LaN.pg
    ./pgfiles/laonly.pg
    ./pgfiles/LL.pg
    ./pgfiles/LLO.pg
    ./pgfiles/llyman.pg
    ./pgfiles/loc.pg
    ./pgfiles/local.pg
    ./pgfiles/lohi.pg
    ./pgfiles/lointhi.pg
    ./pgfiles/lointz.pg
    ./pgfiles/loion.pg
    ./pgfiles/loloz.pg
    ./pgfiles/lorare.pg
    ./pgfiles/lowz.pg
    ./pgfiles/lowzFeII.pg
    ./pgfiles/ly4.pg
    ./pgfiles/lyabg.pg
    ./pgfiles/lyman.pg
    ./pgfiles/lyman8.pg
    ./pgfiles/MgFeMn.pg
    ./pgfiles/MgICI.pg
    ./pgfiles/MgII.pg
    ./pgfiles/MgIIFeII.pg
    ./pgfiles/NaCa.pg
    ./pgfiles/NiII.pg
    ./pgfiles/nitrogen.pg
    ./pgfiles/NV.pg
    ./pgfiles/OI.pg
    ./pgfiles/OIArI.pg
    ./pgfiles/oneline.pg
    ./pgfiles/OSipair.pg
    ./pgfiles/OVI.pg
    ./pgfiles/OVIHI.pg
    ./pgfiles/range.pg
    ./pgfiles/sdlaion.pg
    ./pgfiles/SiII.pg
    ./pgfiles/SiIIfs.pg
    ./pgfiles/SiIIl.pg
    ./pgfiles/SiIV.pg
    ./pgfiles/stloion.pg
    ./pgfiles/strong.pg
    ./pgfiles/TiII.pg
    ./pgfiles/ultralow.pg
    ./pgfiles/vhiHCNO.pg
    ./pldef.f
    ./plset.f
    ./pr_mdian1.f
    ./prg_polyd.f
    ./probks.f
    ./rd_ablin.f
    ./rd_adnois.f
    ./rd_alias.f
    ./rd_comalias.f
    ./rd_compreset.f
    ./rd_cuchstr.f
    ./rd_cufilnm.f
    ./rd_cuplot.f
    ./rd_cuwvrest.f
    ./rd_cydata.f
    ./rd_daswap.f
    ./rd_dblid.f
    ./rd_dmanip.f
    ./rd_drebin.f
    ./rd_dvcont.f
    ./rd_estcolb.f
    ./rd_ewwav.f
    ./rd_extabs.f
    ./rd_fluxbb.f
    ./rd_gen.f
    ./rd_gprof.f
    ./rd_gprofe.f
    ./rd_gpsim.f
    ./rd_hdust.f
    ./rd_intvtab.f
    ./rd_markreg.f
    ./rd_mattrd.f
    ./rd_meantau.f
    ./rd_moderrs.f
    ./rd_multiset.f
    ./rd_pgannot.f
    ./rd_plcset.f
    ./rd_pmpset.f
    ./rd_prhelp.dat
    ./rd_prhelp.f
    ./rd_qdatafiles.f
    ./rd_readlinwav.f
    ./rd_sclsig.f
    ./rd_scrn.f
    ./rd_setintv.f
    ./rd_sizes.f
    ./rd_splvtick.f
    ./rd_stackz.f
    ./rd_start.dat
    ./rd_stplot.f
    ./rd_tgplots.f
    ./rd_tickvpfl.f
    ./rd_ulims.f
    ./rd_ulimse.f
    ./rd_ulimx.f
    ./rd_ulinfo.f
    ./rd_wrfits.f
    ./rd_wrwcfits.f
    ./rd_wvsc.f
    ./rd_xcdc.f
    ./rd_zcorr.f
    ./rd_ztabcf.f
    ./sepvar.f
    ./splot.f
    ./spvoigt.f
    ./svbksbd.f
    ./svdajc.f
    ./svdcmpd.f
    ./svdfit.f
    ./test.f
    ./tickpg.f
    ./tiedb.f
    ./tikset.f
    ./tstfit3d.f
    ./udchole.f
    ./varythis.f
    ./voigt_king.f
    ./vp_abfind.f
    ./vp_abundset.f
    ./vp_addline.f
    ./vp_addxlink.f
    ./vp_archwav.f
    ./vp_asciin.f
    ./vp_ationsep.f
    ./vp_atomass.f
    ./vp_blfixf.f
    ./vp_cfcopy.f
    ./vp_chanwav.f
    ./vp_charlims.f
    ./vp_chdisprof.f
    ./vp_checkn.f
    ./vp_chipconv.f
    ./vp_chsation.f
    ./vp_chspread.f
    ./vp_cmssrc.f
    ./vp_curse.f
    ./vp_cval.f
    ./vp_dattim.f
    ./vp_dattim_alt.f
    ./vp_deriv.f
    ./vp_dlinin.f
    ./vp_dlset.f
    ./vp_dotick.f
    ./vp_emlsmry.f
    ./vp_empval.f
    ./vp_emscchk.f
    ./vp_emtset.f
    ./vp_endorder.f
    ./vp_endrej.f
    ./vp_errej.f
    ./vp_ewred.f
    ./vp_f13fin.f
    ./vp_f13finx.f
    ./vp_f13read.f
    ./vp_f_sigscl.f
    ./vp_fgf13write.f
    ./vp_flchderivs.f
    ./vp_fnzbin.f
    ./vp_getargs.f
    ./vp_getres.f
    ./vp_gglwcoeff.f
    ./vp_gsclfts.f
    ./vp_gwatfits.f
    ./vp_initval.f
    ./vp_ionregcheck.f
    ./vp_ksvnorm.f
    ./vp_lineid.f
    ./vp_linlim.f
    ./vp_listfiles.f
    ./vp_lycont.f
    ./vp_lyconte.f
    ./vp_minpol.f
    ./vp_minquad.f
    ./vp_nearlin.f
    ./vp_newtfxl.f
    ./vp_pareorder.f
    ./vp_parmcest.f
    ./vp_pgbegin.f
    ./vp_pgcurs.f
    ./vp_plcset.f
    ./vp_poisson.f
    ./vp_presetsys.f
    ./vp_psfinterp.f
    ./vp_rdbwvfts.f
    ./vp_rdlines.f
    ./vp_rdspecial.f
    ./vp_rdwaveval.f
    ./vp_readfits.f
    ./vp_rfitsext.f
    ./vp_runstst.f
    ./vp_rwlims.f
    ./vp_scontf.f
    ./vp_setintv.f
    ./vp_setoptn.f
    ./vp_setup.dat
    ./vp_sizes.f
    ./vp_smry.f
    ./vp_spdatin.f
    ./vp_splot.dat
    ./vp_spread.f
    ./vp_spvoigte.f
    ./vp_srcpath.f
    ./vp_startval.f
    ./vp_stration.f
    ./vp_stripcmt.f
    ./vp_subchspread.f
    ./vp_sumlinclear.f
    ./vp_sysdel.f
    ./vp_tdcolset.f
    ./vp_tieval.f
    ./vp_trimattab.f
    ./vp_tsetxref.f
    ./vp_ucheck.f
    ./vp_ucoptv.f
    ./vp_ucprinerr.f
    ./vp_ucprinit.f
    ./vp_usdef.f
    ./vp_vreset.f
    ./vp_wrt25.f
    ./vp_wval.f
    ./vp_wvalch.f
    ./vp_wvalsubch.f
    ./vp_wverass.f
    ./vpf_bvalsp.f
    ./vpf_dvresn.f
    ./vpfit.f
    ./vpgti.f
    ./vppreset.com
    ./vpreset.awk
    ./wcor.f
    ./wval.f

    --2018-06-06 18:41:19--  http://www.ast.cam.ac.uk/~rfc/vpfit10.2.tar.gz
    Resolving www.ast.cam.ac.uk (www.ast.cam.ac.uk)... 131.111.71.71
    Connecting to www.ast.cam.ac.uk (www.ast.cam.ac.uk)|131.111.71.71|:80... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: https://www.ast.cam.ac.uk:443/~rfc/vpfit10.2.tar.gz [following]
    --2018-06-06 18:41:19--  https://www.ast.cam.ac.uk/~rfc/vpfit10.2.tar.gz
    Connecting to www.ast.cam.ac.uk (www.ast.cam.ac.uk)|131.111.71.71|:443... connected.
    HTTP request sent, awaiting response... 404 Not Found
    2018-06-06 18:41:20 ERROR 404: Not Found.

    mkdir: vpfit10: File exists

```python
ls
```

    00-Overview.ipynb                Advanced-Notebook-Tricks.ipynb
    01-Tips-and-tricks.ipynb         Basic-Plotting.ipynb
    02-NBExtensions.ipynb            Data_Cleaning.ipynb
    03-Interactive-Splines.ipynb     ex1.png
    04-Autoreload.ipynb              history.ipynb
    05-SQL-Example.ipynb             jupyterlab-example.md
    06-Advanced-Visualization.ipynb  pivottablejs.html
    07-Some_basics.ipynb             presidents.sqlite
    08-More_basics.ipynb             vpfit10/
    09-Extras.ipynb                  vpfit10.2.tar.gz
    10-R-stuff.ipynb                 vpfit10.2.tar.gz.1
    11-older-stuff.ipynb             vpfit10.2.tar.gz.2
    12-old-pandas.ipynb              vpfit10.2.tar.gz.3
    3d-example-plot.html

```python
!ls
```

    00-Overview.ipynb               3d-example-plot.html
    01-Tips-and-tricks.ipynb        Advanced-Notebook-Tricks.ipynb
    02-NBExtensions.ipynb           Basic-Plotting.ipynb
    03-Interactive-Splines.ipynb    Data_Cleaning.ipynb
    04-Autoreload.ipynb             ex1.png
    05-SQL-Example.ipynb            pivottablejs.html
    06-Advanced-Visualization.ipynb presidents.sqlite
    07-Some_basics.ipynb            vpfit10
    08-More_basics.ipynb            vpfit10.2.tar.gz
    09-Extras.ipynb                 vpfit10.2.tar.gz.1
    10-R-stuff.ipynb                vpfit10.2.tar.gz.2
    11-older-stuff.ipynb            vpfit10.2.tar.gz.3
    12-old-pandas.ipynb

```python
%ls
```

    00-Overview.ipynb                3d-example-plot.html
    01-Tips-and-tricks.ipynb         Advanced-Notebook-Tricks.ipynb
    02-NBExtensions.ipynb            Basic-Plotting.ipynb
    03-Interactive-Splines.ipynb     Data_Cleaning.ipynb
    04-Autoreload.ipynb              ex1.png
    05-SQL-Example.ipynb             pivottablejs.html
    06-Advanced-Visualization.ipynb  presidents.sqlite
    07-Some_basics.ipynb             vpfit10/
    08-More_basics.ipynb             vpfit10.2.tar.gz
    09-Extras.ipynb                  vpfit10.2.tar.gz.1
    10-R-stuff.ipynb                 vpfit10.2.tar.gz.2
    11-older-stuff.ipynb             vpfit10.2.tar.gz.3
    12-old-pandas.ipynb

```python
ls vpfit10/
```

    00README                  rd_plcset.f               vp_f_sigscl.f
    MMatomdat_atom_noiso.dat  rd_pmpset.f               vp_fgf13write.f
    atom.dat                  rd_prhelp.dat             vp_flchderivs.f
    atomtab.f                 rd_prhelp.f               vp_fnzbin.f
    autovpin.f                rd_qdatafiles.f           vp_getargs.f
    bintofits.f               rd_readlinwav.f           vp_getres.f
    calcn.f                   rd_sclsig.f               vp_gglwcoeff.f
    case.f                    rd_scrn.f                 vp_gsclfts.f
    chanwav.f                 rd_setintv.f              vp_gwatfits.f
    checkb.f                  rd_sizes.f                vp_initval.f
    chisum26.f                rd_splvtick.f             vp_ionregcheck.f
    cmax1.f                   rd_stackz.f               vp_ksvnorm.f
    cms_src.f                 rd_start.dat              vp_lineid.f
    com_cons.f                rd_stplot.f               vp_linlim.f
    dcholin.f                 rd_tgplots.f              vp_listfiles.f
    dexpf.f                   rd_tickvpfl.f             vp_lycont.f
    dlycabscft.f              rd_ulims.f                vp_lyconte.f
    docs/                     rd_ulimse.f               vp_minpol.f
    dotick.f                  rd_ulimx.f                vp_minquad.f
    dpoly.f                   rd_ulinfo.f               vp_nearlin.f
    dsepvar.f                 rd_wrfits.f               vp_newtfxl.f
    eigen.f                   rd_wrwcfits.f             vp_pareorder.f
    erfcc.f                   rd_wvsc.f                 vp_parmcest.f
    externals.f               rd_xcdc.f                 vp_pgbegin.f
    fl_gen.f                  rd_zcorr.f                vp_pgcurs.f
    gen_vpin.f                rd_ztabcf.f               vp_plcset.f
    ginsert.f                 sepvar.f                  vp_poisson.f
    gu_tabsort.f              splot.f                   vp_presetsys.f
    hedit.f                   spvoigt.f                 vp_psfinterp.f
    linfit.f                  svbksbd.f                 vp_rdbwvfts.f
    lm_udchole.f              svdajc.f                  vp_rdlines.f
    lyregset.f                svdcmpd.f                 vp_rdspecial.f
    makefile*                 svdfit.f                  vp_rdwaveval.f
    medianv.f                 test.f                    vp_readfits.f
    pda_che2d.f               tickpg.f                  vp_rfitsext.f
    pda_dp1vlu.f              tiedb.f                   vp_runstst.f
    pda_dpcoef.f              tikset.f                  vp_rwlims.f
    pda_dpolft.f              tstfit3d.f                vp_scontf.f
    pda_qsiad.f               udchole.f                 vp_setintv.f
    pda_src.f                 varythis.f                vp_setoptn.f
    pda_xermsg.f              voigt_king.f              vp_setup.dat
    pgfiles/                  vp_abfind.f               vp_sizes.f
    pldef.f                   vp_abundset.f             vp_smry.f
    plset.f                   vp_addline.f              vp_spdatin.f
    pr_mdian1.f               vp_addxlink.f             vp_splot.dat
    prg_polyd.f               vp_archwav.f              vp_spread.f
    probks.f                  vp_asciin.f               vp_spvoigte.f
    rd_ablin.f                vp_ationsep.f             vp_srcpath.f
    rd_adnois.f               vp_atomass.f              vp_startval.f
    rd_alias.f                vp_blfixf.f               vp_stration.f
    rd_comalias.f             vp_cfcopy.f               vp_stripcmt.f
    rd_compreset.f            vp_chanwav.f              vp_subchspread.f
    rd_cuchstr.f              vp_charlims.f             vp_sumlinclear.f
    rd_cufilnm.f              vp_chdisprof.f            vp_sysdel.f
    rd_cuplot.f               vp_checkn.f               vp_tdcolset.f
    rd_cuwvrest.f             vp_chipconv.f             vp_tieval.f
    rd_cydata.f               vp_chsation.f             vp_trimattab.f
    rd_daswap.f               vp_chspread.f             vp_tsetxref.f
    rd_dblid.f                vp_cmssrc.f               vp_ucheck.f
    rd_dmanip.f               vp_curse.f                vp_ucoptv.f
    rd_drebin.f               vp_cval.f                 vp_ucprinerr.f
    rd_dvcont.f               vp_dattim.f               vp_ucprinit.f
    rd_estcolb.f              vp_dattim_alt.f           vp_usdef.f
    rd_ewwav.f                vp_deriv.f                vp_vreset.f
    rd_extabs.f               vp_dlinin.f               vp_wrt25.f
    rd_fluxbb.f               vp_dlset.f                vp_wval.f
    rd_gen.f                  vp_dotick.f               vp_wvalch.f
    rd_gprof.f                vp_emlsmry.f              vp_wvalsubch.f
    rd_gprofe.f               vp_empval.f               vp_wverass.f
    rd_gpsim.f                vp_emscchk.f              vpf_bvalsp.f
    rd_hdust.f                vp_emtset.f               vpf_dvresn.f
    rd_intvtab.f              vp_endorder.f             vpfit.f
    rd_markreg.f              vp_endrej.f               vpgti.f
    rd_mattrd.f               vp_errej.f                vppreset.com
    rd_meantau.f              vp_ewred.f                vpreset.awk
    rd_moderrs.f              vp_f13fin.f               wcor.f
    rd_multiset.f             vp_f13finx.f              wval.f
    rd_pgannot.f              vp_f13read.f

```python
tailthing = "*.ipynb"
```

```python
tailthing
```

    '*.ipynb'

```python
!ls {tailthing}
```

    00-Overview.ipynb               09-Extras.ipynb
    01-Tips-and-tricks.ipynb        10-R-stuff.ipynb
    02-NBExtensions.ipynb           11-older-stuff.ipynb
    03-Interactive-Splines.ipynb    12-old-pandas.ipynb
    04-Autoreload.ipynb             Advanced-Notebook-Tricks.ipynb
    05-SQL-Example.ipynb            Basic-Plotting.ipynb
    06-Advanced-Visualization.ipynb Data_Cleaning.ipynb
    07-Some_basics.ipynb            history.ipynb
    08-More_basics.ipynb

```python
output = !ls
```

```python
output
```

    ['00-Overview.ipynb',
     '01-Tips-and-tricks.ipynb',
     '02-NBExtensions.ipynb',
     '03-Interactive-Splines.ipynb',
     '04-Autoreload.ipynb',
     '05-SQL-Example.ipynb',
     '06-Advanced-Visualization.ipynb',
     '07-Some_basics.ipynb',
     '08-More_basics.ipynb',
     '09-Extras.ipynb',
     '10-R-stuff.ipynb',
     '11-older-stuff.ipynb',
     '12-old-pandas.ipynb',
     '3d-example-plot.html',
     'Advanced-Notebook-Tricks.ipynb',
     'Basic-Plotting.ipynb',
     'Data_Cleaning.ipynb',
     'ex1.png',
     'history.ipynb',
     'jupyterlab-example.md',
     'pivottablejs.html',
     'presidents.sqlite',
     'vpfit10',
     'vpfit10.2.tar.gz',
     'vpfit10.2.tar.gz.1',
     'vpfit10.2.tar.gz.2',
     'vpfit10.2.tar.gz.3']

```python
%env
```

    {'SSH_AGENT_PID': '575',
     'TERM_PROGRAM': 'iTerm.app',
     'TERM': 'xterm-color',
     'SHELL': '/bin/bash',
     'initials': 'jbw',
     'CLICOLOR': '1',
     'TMPDIR': '/var/folders/yk/2rgr0zrj0614jntythcnlyvw0000gn/T/',
     'CONDA_SHLVL': '1',
     'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.J09pBRYHDx/Render',
     'CONDA_PROMPT_MODIFIER': '(dspy3) ',
     'BASH': '/Users/jonathan/.dotfiles',
     'TERM_PROGRAM_VERSION': '3.1.6',
     'OLDPWD': '/Users/jonathan',
     'TERM_SESSION_ID': 'w0t1p0:F8385233-10A6-4744-B11A-9F20617B1326',
     'USER': 'jonathan',
     'CONDA_EXE': '/Users/jonathan/miniconda3/bin/conda',
     'SSH_AUTH_SOCK': '/var/folders/yk/2rgr0zrj0614jntythcnlyvw0000gn/T//ssh-5oUKvrjtAsqE/agent.573',
     '__CF_USER_TEXT_ENCODING': '0x1F5:0x0:0x0',
     'PATH': '/Users/jonathan/miniconda3/envs/dspy3/bin:/Users/jonathan/miniconda3/envs/dspy3/bin:/Users/jonathan/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin',
     'CONDA_PREFIX': '/Users/jonathan/miniconda3/envs/dspy3',
     'PWD': '/Users/jonathan/github/jupyter-tips-and-tricks',
     'EDITOR': 'vim',
     'LANG': 'en_US.UTF-8',
     'ITERM_PROFILE': 'Default',
     'XPC_FLAGS': '0x0',
     'ITERM_ORIG_PS1': '(dspy3) \\n# \\w\\n# \\[\\e[0;32m\\]\\h \\[\\e[0m\\]\\[\\e[0;36m\\]${git_branch}\\[\\e[0;31m\\]$git_dirty\\[\\e[0m\\]$ ',
     'PS1': '(dspy3) \\n# \\w\\n# \\[\\e[0;32m\\]\\h \\[\\e[0m\\]\\[\\e[0;36m\\]${git_branch}\\[\\e[0;31m\\]$git_dirty\\[\\e[0m\\]$ ',
     'PS3': '#? ',
     'HISTCONTROL': 'ignoreboth',
     'PS2': '> ',
     'XPC_SERVICE_NAME': '0',
     'GITAWAREPROMPT': '/Users/jonathan/.bash/git-aware-prompt',
     'SHLVL': '1',
     'PS4': '+',
     'HOME': '/Users/jonathan',
     'COLORFGBG': '15;0',
     'ITERM_PREV_PS1': '\\[\x1b]133;D;$?\x07\x1b]133;A\x07\\](dspy3) \\n# \\w\\n# \\[\\e[0;32m\\]\\h \\[\\e[0m\\]\\[\\e[0;36m\\]${git_branch}\\[\\e[0;31m\\]$git_dirty\\[\\e[0m\\]$ \\[\x1b]133;B\x07\\]',
     'ITERM_SESSION_ID': 'w0t1p0:F8385233-10A6-4744-B11A-9F20617B1326',
     'CONDA_PYTHON_EXE': '/Users/jonathan/miniconda3/bin/python',
     'LOGNAME': 'jonathan',
     'CONDA_DEFAULT_ENV': 'dspy3',
     'PROMPT_COMMAND': '__bp_precmd_invoke_cmd; echo -ne "\\033]0;${USER}@${HOSTNAME%%.*} ${PWD}"; echo -ne "\\007"; __bp_interactive_mode',
     'DISPLAY': '/private/tmp/com.apple.launchd.a7eirdr9W4/org.macosforge.xquartz:0',
     'COLORTERM': 'truecolor',
     '_': '/Users/jonathan/miniconda3/envs/dspy3/bin/jupyter',
     'JPY_PARENT_PID': '4608',
     'PAGER': 'cat',
     'GIT_PAGER': 'cat',
     'MPLBACKEND': 'module://ipykernel.pylab.backend_inline'}

## Danger zone

    _, _N and _iN

```python
!pwd
```

    /Users/jonathan/github/jupyter-tips-and-tricks/notebooks

```python
a = 3
a
```

    3

```python
_i51
```

    'a = 3\na'

```python
_51
```

    3

```python
b = 5
```

```python
c = a * b
```

```python

```

```python

```

    4

```python

```

```python
%notebook history.ipynb
```

```python

```

```python

```

```python
!subl 01-Tips-and-tricks.ipynb
```

```python

```

```python

```

```python

```

```python

```

```python

```