# NRPy+'s Reference Metric Interface

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

### NRPy+ Source Code for this module: [reference_metric.py](../edit/reference_metric.py)

## Introduction:
### Why use a reference metric? Benefits of choosing the best coordinate system for the problem

When solving a partial differential equation on the computer, it is useful to first pick a coordinate system well-suited to the geometry of the problem. For example, if we are modeling a spherically-symmetric star, it would be hugely wasteful to model the star in 3-dimensional Cartesian coordinates ($x$,$y$,$z$). This is because in Cartesian coordinates, we would need to choose high sampling in all three Cartesian directions. If instead we chose to model the star in spherical coordinates  ($r$,$\theta$,$\phi$), so long as the star is centered at $r=0$, we would not need to model the star with more than one point in the $\theta$ and $\phi$ directions!

A similar argument holds for stars that are *nearly* spherically symmetric. Such stars may exhibit density distributions that vary slowly in $\theta$ and $\phi$ directions (e.g., isolated neutron stars or black holes). In these cases the number of points needed to sample the angular directions will still be much smaller than in the radial direction.

Thus choice of an appropriate reference metric may directly mitigate the [Curse of Dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality).

# Table of Contents
$$\label{toc}$$

This notebook is organized as follow

1. [Step 1](#define_ref_metric): Defining a reference metric, [`reference_metric.py`](../edit/reference_metric.py)
1. [Step 2](#define_geometric): Defining geometric quantities, **`ref_metric__hatted_quantities()`**
1. [Step 3](#prescribed_ref_metric): Prescribed reference metrics in [`reference_metric.py`](../edit/reference_metric.py)
    1. [Step 3.a](#sphericallike): Spherical-like coordinate systems
        1. [Step 3.a.i](#spherical): **`reference_metric::CoordSystem = "Spherical"`**
        1. [Step 3.a.ii](#sinhspherical): **`reference_metric::CoordSystem = "SinhSpherical"`**
        1. [Step 3.a.iii](#sinhsphericalv2): **`reference_metric::CoordSystem = "SinhSphericalv2"`**
    1. [Step 3.b](#cylindricallike): Cylindrical-like coordinate systems
        1. [Step 3.b.i](#cylindrical): **`reference_metric::CoordSystem = "Cylindrical"`**
        1. [Step 3.b.ii](#sinhcylindrical): **`reference_metric::CoordSystem = "SinhCylindrical"`**
        1. [Step 3.b.iii](#sinhcylindricalv2): **`reference_metric::CoordSystem = "SinhCylindricalv2"`**
    1. [Step 3.c](#cartesianlike): Cartesian-like coordinate systems
        1. [Step 3.c.i](#cartesian): **`reference_metric::CoordSystem = "Cartesian"`**
    1. [Step 3.d](#prolatespheroidal): Prolate spheroidal coordinates
        1. [Step 3.d.i](#symtp): **`reference_metric::CoordSystem = "SymTP"`**
        1. [Step 3.d.ii](#sinhsymtp): **`reference_metric::CoordSystem = "SinhSymTP"`**
1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Defining a reference metric, [`reference_metric.py`](../edit/reference_metric.py) \[Back to [top](#toc)\]
$$\label{define_ref_metric}$$

***Note that currently only orthogonal reference metrics of dimension 3 or fewer are supported. This can be extended if desired.***

NRPy+ assumes all curvilinear coordinate systems map directly from a uniform, Cartesian numerical grid with coordinates $(x,y,z)$=(`xx[0]`,`xx[1]`,`xx[2]`). Thus when defining reference metrics, all defined coordinate quantities must be in terms of the `xx[]` array. As we will see, this adds a great deal of flexibility

For example,  [**reference_metric.py**](../edit/reference_metric.py) requires that the *orthogonal coordinate scale factors* be defined. As described [here](https://en.wikipedia.org/wiki/Curvilinear_coordinates), the $i$th scale factor is the positive root of the metric element $g_{ii}$. In ordinary spherical coordinates $(r,\theta,\phi)$, with line element $ds^2 = g_{ij} dx^i dx^j = dr^2+ r^2 d \theta^2 + r^2 \sin^2\theta \ d\phi^2$, we would first define
* $r = xx_0$
* $\theta = xx_1$
* $\phi = xx_2$,

so that the scale factors are defined as
* `scalefactor_orthog[0]` = $1$
* `scalefactor_orthog[1]` = $r$
* `scalefactor_orthog[2]` = $r \sin \theta$

Here is the corresponding code:

```python
import sympy as sp
import NRPy_param_funcs as par
import reference_metric as rfm

r = rfm.xx[0]
th = rfm.xx[1]
ph = rfm.xx[2]

rfm.scalefactor_orthog[0] = 1
rfm.scalefactor_orthog[1] = r
rfm.scalefactor_orthog[2] = r*sp.sin(th)

# Notice that the scale factor will be given
#    in terms of the fundamental Cartesian
#    grid variables, and not {r,th,ph}:
print("r*sin(th) = "+str(rfm.scalefactor_orthog[2]))
```

    r*sin(th) = xx0*sin(xx1)

Next suppose we wish to modify our radial coordinate $r(xx_0)$ to be an exponentially increasing function, so that our numerical grid $(xx_0,xx_1,xx_2)$ will map to a spherical grid with radial grid spacing ($\Delta r$) that *increases* with $r$. Generally we will find it useful to define $r(xx_0)$ to be an odd function, so let's choose

$$r(xx_0) = a \sinh(xx_0/s),$$

where $a$ is an overall radial scaling factor, and $s$ denotes the scale (in units of $xx_0$) over which exponential growth will take place. In our implementation below, note that we use the relation

$$\sinh(x) = \frac{e^x - e^{-x}}{2},$$

as SymPy finds it easier to evaluate exponentials than hyperbolic trigonometric functions.

```python
a,s = sp.symbols('a s',positive=True)
xx0_rescaled = rfm.xx[0] / s
r = a*(sp.exp(xx0_rescaled) - sp.exp(-xx0_rescaled))/2

# Must redefine the scalefactors since 'r' has been updated!
rfm.scalefactor_orthog[0] = 1
rfm.scalefactor_orthog[1] = r
rfm.scalefactor_orthog[2] = r*sp.sin(th)

print(rfm.scalefactor_orthog[2])
```

    a*(exp(xx0/s) - exp(-xx0/s))*sin(xx1)/2

Often we will find it useful to also define the appropriate mappings from (`xx[0]`,`xx[1]`,`xx[2]`) to Cartesian coordinates (for plotting purposes) and ordinary spherical coordinates (e.g., in case initial data when solving a PDE are naturally written in spherical coordinates). For this purpose, reference_metric.py also declares lists **`xxCart[]`** and **`xxSph[]`**, which in this case are defined as

```python
rfm.xxSph[0] = r
rfm.xxSph[1] = th
rfm.xxSph[2] = ph

rfm.xxCart[0] = r*sp.sin(th)*sp.cos(ph)
rfm.xxCart[1] = r*sp.sin(th)*sp.sin(ph)
rfm.xxCart[2] = r*sp.cos(th)

# Here we show off SymPy's pretty_print()
#   and simplify() functions. Nice, no?
sp.pretty_print(sp.simplify(rfm.xxCart[0]))
```

                            ⎛xx₀⎞
    a⋅sin(xx₁)⋅cos(xx₂)⋅sinh⎜───⎟
                            ⎝ s ⎠

# Step 2: Define geometric quantities, `ref_metric__hatted_quantities()` \[Back to [top](#toc)\]
$$\label{define_geometric}$$

Once `scalefactor_orthog[]` has been defined, the function **`ref_metric__hatted_quantities()`** within [reference_metric.py](../edit/reference_metric.py) can be called to define a number of geometric quantities useful for solving PDEs in curvilinear coordinate systems.

Adopting the notation of [Baumgarte, Montero, Cordero-Carrión, and Müller, PRD 87, 044026 (2012)](https://arxiv.org/abs/1211.6632), geometric quantities related to the reference metric are named "hatted" quantities, . For example, the reference metric is defined as $\hat{g}_{ij}$=`ghatDD[i][j]`:

```python
rfm.ref_metric__hatted_quantities()

sp.pretty_print(sp.Matrix(sp.simplify(rfm.ghatDD)))
```

    ⎡1           0                         0              ⎤
    ⎢                                                     ⎥
    ⎢                     2                               ⎥
    ⎢      ⎛ xx₀    -xx₀ ⎞                                ⎥
    ⎢      ⎜ ───    ─────⎟                                ⎥
    ⎢    2 ⎜  s       s  ⎟                                ⎥
    ⎢   a ⋅⎝ℯ    - ℯ     ⎠                                ⎥
    ⎢0  ───────────────────                0              ⎥
    ⎢            4                                        ⎥
    ⎢                                                     ⎥
    ⎢                                          2          ⎥
    ⎢                           ⎛ xx₀    -xx₀ ⎞           ⎥
    ⎢                           ⎜ ───    ─────⎟           ⎥
    ⎢                         2 ⎜  s       s  ⎟     2     ⎥
    ⎢                        a ⋅⎝ℯ    - ℯ     ⎠ ⋅sin (xx₁)⎥
    ⎢0           0           ─────────────────────────────⎥
    ⎣                                      4              ⎦

In addition to $\hat{g}_{ij}$, **`ref_metric__hatted_quantities()`** also provides:
* The rescaling "matrix" `ReDD[i][j]`, used for separating singular (due to chosen coordinate system) pieces of smooth rank-2 tensor components from the smooth parts, so that the smooth parts can be used within temporal and spatial differential operators.
* Inverse reference metric: $\hat{g}^{ij}$=`ghatUU[i][j]`.
* Reference metric determinant: $\det\left(\hat{g}_{ij}\right)$=`detgammahat`.
* First and second derivatives of the reference metric: $\hat{g}_{ij,k}$=`ghatDD_dD[i][j][k]`; $\hat{g}_{ij,kl}$=`ghatDD_dDD[i][j][k][l]`
* Christoffel symbols associated with the reference metric, $\hat{\Gamma}^i_{jk}$ = `GammahatUDD[i][j][k]` and their first derivatives $\hat{\Gamma}^i_{jk,l}$ = `GammahatUDD_dD[i][j][k][l]`

For example, the Christoffel symbol $\hat{\Gamma}^{xx_1}_{xx_2 xx_2}=\hat{\Gamma}^1_{22}$ is given by `GammahatUDD[1][2][2]`:

```python
sp.pretty_print(sp.simplify(rfm.GammahatUDD[1][2][2]))
```

    -sin(2⋅xx₁)
    ────────────
         2

Given the trigonometric identity $2\sin(x)\cos(x) = \sin(2x)$, notice that the above expression is equivalent to Eq. 18 of [Baumgarte, Montero, Cordero-Carrión, and Müller, PRD 87, 044026 (2012)](https://arxiv.org/abs/1211.6632). This is expected since the sinh-radial spherical coordinate system is equivalent to ordinary spherical coordinates in the angular components.

# Step 3: Prescribed reference metrics in [`reference_metric.py`](../edit/reference_metric.py) \[Back to [top](#toc)\]
$$\label{prescribed_ref_metric}$$

One need not manually define scale factors or other quantities for reference metrics, as a number of prescribed reference metrics are already defined in [reference_metric.py](../edit/reference_metric.py). These can be accessed by first setting the parameter **reference_metric::CoordSystem** to one of the following, and then calling the function **`rfm.reference_metric()`**.

```python
import NRPy_param_funcs as par
import indexedexp as ixp
import grid as gri

# Step 0a: Initialize parameters
thismodule = __name__
par.initialize_param(par.glb_param("char", thismodule, "CoordSystem", "Spherical"))

# Step 0b: Declare global variables
xx = gri.xx
xxCart = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
Cart_to_xx = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
Cartx,Carty,Cartz = sp.symbols("Cartx Carty Cartz", real=True)
Cart = [Cartx,Carty,Cartz]
xxSph  = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
scalefactor_orthog = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
have_already_called_reference_metric_function = False

CoordSystem = par.parval_from_str("reference_metric::CoordSystem")
M_PI,M_SQRT1_2 = par.Cparameters("#define",thismodule,["M_PI","M_SQRT1_2"],"")

global xxmin
global xxmax

global UnitVectors
UnitVectors = ixp.zerorank2(DIM=3)
```

We will find the following plotting function useful for analyzing coordinate systems in which the radial coordinate is rescaled.

```python
def create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0):
    import matplotlib.pyplot as plt
    plt.clf()
    Nr = 20
    dxx0 = 1.0 / float(Nr)
    xx0s    = []
    rs      = []
    deltars = []
    rprimes = []
    for i in range(Nr):
        xx0 = (float(i) + 0.5)*dxx0
        xx0s.append(xx0)
        rs.append(     sp.sympify(str(r_of_xx0     ).replace("xx0",str(xx0))))
        rprimes.append(sp.sympify(str(rprime_of_xx0).replace("xx0",str(xx0))))
        if i>0:
            deltars.append(sp.log(rs[i]-rs[i-1],10))
        else:
            deltars.append(sp.log(2*rs[0],10))

    # fig, ax = plt.subplots()
    fig = plt.figure(figsize=(12,12)) # 8 in x 8 in

    ax = fig.add_subplot(221)
    ax.set_title('$r(xx_0)$ for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$r(xx_0)$',fontsize='x-large')
    ax.plot(xx0s, rs, 'k.', label='Spacing between\nadjacent gridpoints')
    # legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
    # legend.get_frame().set_facecolor('C1')

    ax = fig.add_subplot(222)
    ax.set_title('Grid spacing for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$\log_{10}(\Delta r)$',fontsize='x-large')
    ax.plot(xx0s, deltars, 'k.', label='Spacing between\nadjacent gridpoints\nin $r(xx_0)$ plot')
    legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
    legend.get_frame().set_facecolor('C1')

    ax = fig.add_subplot(223)
    ax.set_title('$r\'(xx_0)$ for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$r\'(xx_0)$',fontsize='x-large')
    ax.plot(xx0s, rprimes, 'k.', label='Nr=96')
    # legend = ax.legend(loc='upper left', shadow=True, fontsize='x-large')
    # legend.get_frame().set_facecolor('C1')

    plt.tight_layout(pad=2)
    plt.show()
```

## Step 3.a: Spherical-like coordinate systems \[Back to [top](#toc)\]
$$\label{sphericallike}$$

### Step 3.a.i: **`reference_metric::CoordSystem = "Spherical"`** \[Back to [top](#toc)\]
$$\label{spherical}$$

Standard spherical coordinates, with $(r,\theta,\phi)=(xx_0,xx_1,xx_2)$

```python
if CoordSystem == "Spherical":
    # Adding assumption real=True can help simplify expressions involving xx[0] & xx[1] below.
    xx[0] = sp.symbols("xx0", real=True)
    xx[1] = sp.symbols("xx1", real=True)

    RMAX = par.Cparameters("REAL", thismodule, ["RMAX"],10.0)
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [         RMAX,          M_PI,  M_PI]

    r  = xx[0]
    th = xx[1]
    ph = xx[2]

    Cart_to_xx[0] = sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)
    Cart_to_xx[1] = sp.acos(Cartz / Cart_to_xx[0])
    Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now let's analyze $r(xx_0)$ for **"Spherical"** coordinates.

```python
%matplotlib inline

import sympy as sp

import reference_metric as rfm
import NRPy_param_funcs as par
CoordSystem = "Spherical"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

RMAX     = 10.0
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("RMAX",str(RMAX)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("RMAX",str(RMAX)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

### Step 3.a.ii: **`reference_metric::CoordSystem = "SinhSpherical"`** \[Back to [top](#toc)\]
$$\label{sinhspherical}$$

Spherical coordinates, but with $$r(xx_0) = \text{AMPL} \frac{\sinh\left(\frac{xx_0}{\text{SINHW}}\right)}{\sinh\left(\frac{1}{\text{SINHW}}\right)}.$$

SinhSpherical uses two parameters: `AMPL` and `SINHW`. `AMPL` sets the outer boundary distance; and `SINHW` sets the focusing of the coordinate points near $r=0$, where a small `SINHW` ($\sim 0.125$) will greatly focus the points near $r=0$ and a large `SINHW` will look more like an ordinary spherical polar coordinate system.

```python
if CoordSystem == "SinhSpherical":
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [sp.sympify(1),          M_PI,  M_PI]

    AMPL, SINHW = par.Cparameters("REAL",thismodule,["AMPL","SINHW"],[10.0,0.2])
    # Set SinhSpherical radial coordinate by default; overwrite later if CoordSystem == "SinhSphericalv2".
    r = AMPL * (sp.exp(xx[0] / SINHW) - sp.exp(-xx[0] / SINHW)) / \
               (sp.exp(1 / SINHW) - sp.exp(-1 / SINHW))
    th = xx[1]
    ph = xx[2]

    Cart_to_xx[0] = SINHW*sp.asinh(sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)*sp.sinh(1/SINHW)/AMPL)
    Cart_to_xx[1] = sp.acos(Cartz / sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2))
    Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now we explore $r(xx_0)$ for `SinhSpherical` assuming `AMPL=10.0` and `SINHW=0.2`:

```python
%matplotlib inline

import sympy as sp

import reference_metric as rfm
import NRPy_param_funcs as par
CoordSystem = "SinhSpherical"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

AMPL     = 10.0
SINHW    = 0.2
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

### Step 3.a.iii: **`reference_metric::CoordSystem = "SinhSphericalv2"`** \[Back to [top](#toc)\]
$$\label{sinhsphericalv2}$$

The same as SinhSpherical coordinates, but with an additional `AMPL*const_dr*xx_0` term:
$$r(xx_0) = \text{AMPL} \left[\text{const_dr}\ xx_0 + \frac{\sinh\left(\frac{xx_0}{\text{SINHW}}\right)}{\sinh\left(\frac{1}{\text{SINHW}}\right)}\right].$$

```python
if CoordSystem == "SinhSphericalv2":
    # SinhSphericalv2 adds the parameter "const_dr", which allows for a region near xx[0]=0 to have
    # constant radial resolution of const_dr, provided the sinh() term does not dominate near xx[0]=0.
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [sp.sympify(1),          M_PI,  M_PI]

    AMPL, SINHW = par.Cparameters("REAL",thismodule,["AMPL","SINHW"],[10.0,0.2])
    const_dr = par.Cparameters("REAL",thismodule,["const_dr"],0.0625)

    r = AMPL*( const_dr*xx[0] + (sp.exp(xx[0] / SINHW) - sp.exp(-xx[0] / SINHW)) /
               (sp.exp(1 / SINHW) - sp.exp(-1 / SINHW)) )
    th = xx[1]
    ph = xx[2]

    # NO CLOSED-FORM EXPRESSION FOR RADIAL INVERSION.
    # Cart_to_xx[0] = "NewtonRaphson"
    # Cart_to_xx[1] = sp.acos(Cartz / sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2))
    # Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now we explore $r(xx_0)$ for `SinhSphericalv2` assuming `AMPL=10.0`, `SINHW=0.2`, and `const_dr=0.05`. Notice that the `const_dr` term significantly increases the grid spacing near $xx_0=0$ relative to `SinhSpherical` coordinates.

```python
%matplotlib inline

import sympy as sp

import reference_metric as rfm
import NRPy_param_funcs as par
CoordSystem = "SinhSphericalv2"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

AMPL     = 10.0
SINHW    = 0.2
const_dr = 0.05
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)).replace("const_dr",str(const_dr)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)).replace("const_dr",str(const_dr)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

## Step 3.b: Cylindrical-like coordinate systems \[Back to [top](#toc)\]
$$\label{cylindricallike}$$

### Step 3.b.i: **`reference_metric::CoordSystem = "Cylindrical"`** \[Back to [top](#toc)\]
$$\label{cylindrical}$$

Standard cylindrical coordinates, with $(\rho,\phi,z)=(xx_0,xx_1,xx_2)$

```python
if CoordSystem == "Cylindrical":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    RHOMAX,ZMIN,ZMAX = par.Cparameters("REAL",thismodule,["RHOMAX","ZMIN","ZMAX"],[10.0,-10.0,10.0])
    xxmin = [sp.sympify(0), -M_PI, ZMIN]
    xxmax = [       RHOMAX,  M_PI, ZMAX]

    RHOCYL = xx[0]
    PHICYL = xx[1]
    ZCYL   = xx[2]

    Cart_to_xx[0] = sp.sqrt(Cartx ** 2 + Carty ** 2)
    Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    Cart_to_xx[2] = Cartz

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

Next let's plot **"Cylindrical"** coordinates.

```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

R = np.linspace(0, 2, 24)
h = 2
u = np.linspace(0,  2*np.pi, 24)

x = np.outer(R, np.cos(u))
y = np.outer(R, np.sin(u))
z = h * np.outer(np.ones(np.size(u)), np.ones(np.size(u)))

r = np.arange(0,2,0.25)
theta = 2*np.pi*r*0

fig = plt.figure(figsize=(12,12)) # 8 in x 8 in
fig, (ax1, ax2) = plt.subplots(1, 2)

ax1 = plt.axes(projection='polar')

ax1.set_rmax(2)

ax1.set_rgrids(r,labels=[])

thetas = np.linspace(0,360,24, endpoint=True)
ax1.set_thetagrids(thetas,labels=[])

# ax.grid(True)
ax1.grid(True,linewidth='1.0')
ax1.set_title("Top Down View")
plt.show()

ax2 = plt.axes(projection='3d', xticklabels=[], yticklabels=[], zticklabels=[])
#ax2.plot_surface(x,y,z, alpha=.75, cmap = 'viridis') # z in case of disk which is parallel to XY plane is constant and you can directly use h

x=np.linspace(-2, 2, 100)
z=np.linspace(-2, 2, 100)
Xc, Zc=np.meshgrid(x, z)
Yc = np.sqrt(4-Xc**2)

rstride = 10
cstride = 10
ax2.plot_surface(Xc, Yc, Zc, alpha=1.0, rstride=rstride, cstride=cstride, cmap = 'viridis')
ax2.plot_surface(Xc, -Yc, Zc, alpha=1.0, rstride=rstride, cstride=cstride, cmap = 'viridis')
ax2.set_title("Standard Cylindrical Grid in 3D")
ax2.grid(False)
plt.axis('off')

plt.show()
```

### Step 3.b.ii" **`reference_metric::CoordSystem = "SinhCylindrical"`** \[Back to [top](#toc)\]
$$\label{sinhcylindrical}$$

Cylindrical coordinates, but with
$$\rho(xx_0) = \text{AMPLRHO} \frac{\sinh\left(\frac{xx_0}{\text{SINHWRHO}}\right)}{\sinh\left(\frac{1}{\text{SINHWRHO}}\right)}$$
and
$$z(xx_2) = \text{AMPLZ} \frac{\sinh\left(\frac{xx_2}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWZ}}\right)}$$

```python
if CoordSystem == "SinhCylindrical":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    xxmin = [sp.sympify(0), -M_PI, sp.sympify(-1)]
    xxmax = [sp.sympify(1),  M_PI, sp.sympify(+1)]

    AMPLRHO, SINHWRHO, AMPLZ, SINHWZ = par.Cparameters("REAL",thismodule,
                                                       ["AMPLRHO","SINHWRHO","AMPLZ","SINHWZ"],
                                                       [     10.0,       0.2,   10.0,    0.2])

    # Set SinhCylindrical radial & z coordinates by default; overwrite later if CoordSystem == "SinhCylindricalv2".
    RHOCYL = AMPLRHO * (sp.exp(xx[0] / SINHWRHO) - sp.exp(-xx[0] / SINHWRHO)) / (sp.exp(1 / SINHWRHO) - sp.exp(-1 / SINHWRHO))
    # phi coordinate remains unchanged.
    PHICYL = xx[1]
    ZCYL   = AMPLZ   * (sp.exp(xx[2] / SINHWZ)   - sp.exp(-xx[2] / SINHWZ))   / (sp.exp(1 / SINHWZ)   - sp.exp(-1 / SINHWZ))
    Cart_to_xx[0] = SINHWRHO*sp.asinh(sp.sqrt(Cartx ** 2 + Carty ** 2)*sp.sinh(1/SINHWRHO)/AMPLRHO)
    Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    Cart_to_xx[2] = SINHWZ*sp.asinh(Cartz*sp.sinh(1/SINHWZ)/AMPLZ)

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

Next let's plot **"SinhCylindrical"** coordinates.

```python
fig=plt.figure()

plt.clf()

fig = plt.figure()

ax = plt.subplot(1,1,1, projection='polar')

ax.set_rmax(2)

Nr = 20
xx0s = np.linspace(0,2,Nr, endpoint=True) + 1.0/(2.0*Nr)

rs = []
AMPLRHO = 1.0
SINHW = 0.4
for i in range(Nr):
    rs.append(AMPLRHO * (np.exp(xx0s[i] / SINHW) - np.exp(-xx0s[i] / SINHW)) / \
                        (np.exp(1.0 / SINHW) - np.exp(-1.0 / SINHW)))

ax.set_rgrids(rs,labels=[])

thetas = np.linspace(0,360,25, endpoint=True)
ax.set_thetagrids(thetas,labels=[])

# ax.grid(True)
ax.grid(True,linewidth='1.0')

plt.show()

```

### Step 3.b.iii: **`reference_metric::CoordSystem = "SinhCylindricalv2"`** \[Back to [top](#toc)\]
$$\label{sinhcylindricalv2}$$

Cylindrical coordinates, but with
$$\rho(xx_0) = \text{AMPLRHO} \left[\text{const_drho}\ xx_0 + \frac{\sinh\left(\frac{xx_0}{\text{SINHWRHO}}\right)}{\sinh\left(\frac{1}{\text{SINHWRHO}}\right)}\right]$$
and
$$z(xx_2) = \text{AMPLZ} \left[\text{const_dz}\ xx_2 + \frac{\sinh\left(\frac{xx_2}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWZ}}\right)}\right]$$

```python
if CoordSystem == "SinhCylindricalv2":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    # SinhCylindricalv2 adds the parameters "const_drho", "const_dz", which allows for regions near xx[0]=0
    # and xx[2]=0 to have constant rho and z resolution of const_drho and const_dz, provided the sinh() terms
    # do not dominate near xx[0]=0 and xx[2]=0.
    xxmin = [sp.sympify(0), -M_PI, sp.sympify(-1)]
    xxmax = [sp.sympify(1),  M_PI, sp.sympify(+1)]
    AMPLRHO, SINHWRHO, AMPLZ, SINHWZ = par.Cparameters("REAL",thismodule,
                                                       ["AMPLRHO","SINHWRHO","AMPLZ","SINHWZ"],
                                                       [     10.0,       0.2,   10.0,    0.2])
    const_drho, const_dz = par.Cparameters("REAL",thismodule,["const_drho","const_dz"],[0.0625,0.0625])

    RHOCYL = AMPLRHO * ( const_drho*xx[0] + (sp.exp(xx[0] / SINHWRHO) - sp.exp(-xx[0] / SINHWRHO)) / (sp.exp(1 / SINHWRHO) - sp.exp(-1 / SINHWRHO)) )
    PHICYL = xx[1]
    ZCYL   = AMPLZ   * ( const_dz  *xx[2] + (sp.exp(xx[2] / SINHWZ  ) - sp.exp(-xx[2] / SINHWZ  )) / (sp.exp(1 / SINHWZ  ) - sp.exp(-1 / SINHWZ  )) )

    # NO CLOSED-FORM EXPRESSION FOR RADIAL OR Z INVERSION.
    # Cart_to_xx[0] = "NewtonRaphson"
    # Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    # Cart_to_xx[2] = "NewtonRaphson"

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

For example, let's set up **`SinhCylindricalv2`** coordinates and output the Christoffel symbol $\hat{\Gamma}^{xx_2}_{xx_2 xx_2}$, or more simply $\hat{\Gamma}^2_{22}$:

```python
par.set_parval_from_str("reference_metric::CoordSystem","SinhCylindricalv2")

rfm.reference_metric()

sp.pretty_print(sp.simplify(rfm.GammahatUDD[2][2][2]))
```

                             ⎛ 2⋅xx₂     ⎞    1
                             ⎜ ──────    ⎟  ──────
                             ⎜ SINHWZ    ⎟  SINHWZ
                            -⎝ℯ       - 1⎠⋅ℯ
    ────────────────────────────────────────────────────────────────────────
           ⎛                  ⎛   2       ⎞   xx₂     ⎛ 2⋅xx₂     ⎞    1   ⎞
           ⎜                  ⎜ ──────    ⎟  ──────   ⎜ ──────    ⎟  ──────⎟
           ⎜                  ⎜ SINHWZ    ⎟  SINHWZ   ⎜ SINHWZ    ⎟  SINHWZ⎟
    SINHWZ⋅⎝- SINHWZ⋅const_dz⋅⎝ℯ       - 1⎠⋅ℯ       - ⎝ℯ       + 1⎠⋅ℯ      ⎠

As we will soon see, defining these "hatted" quantities will be quite useful when expressing hyperbolic ([wave-equation](https://en.wikipedia.org/wiki/Wave_equation)-like) PDEs in non-Cartesian coordinate systems.

## Step 3.c: Cartesian-like coordinate systems \[Back to [top](#toc)\]
$$\label{cartesianlike}$$

### Step 3.c.i: **`reference_metric::CoordSystem = "Cartesian"`** \[Back to [top](#toc)\]
$$\label{cartesian}$$

Standard Cartesian coordinates, with $(x,y,z)=$ `(xx0,xx1,xx2)`

```python
if CoordSystem == "Cartesian":
    xmin, xmax, ymin, ymax, zmin, zmax = par.Cparameters("REAL",thismodule,
                                                         ["xmin","xmax","ymin","ymax","zmin","zmax"],
                                                         [ -10.0,  10.0, -10.0,  10.0, -10.0,  10.0])
    xxmin = ["xmin", "ymin", "zmin"]
    xxmax = ["xmax", "ymax", "zmax"]

    xxCart[0] = xx[0]
    xxCart[1] = xx[1]
    xxCart[2] = xx[2]

    xxSph[0] = sp.sqrt(xx[0] ** 2 + xx[1] ** 2 + xx[2] ** 2)
    xxSph[1] = sp.acos(xx[2] / xxSph[0])
    xxSph[2] = sp.atan2(xx[1], xx[0])

    Cart_to_xx[0] = Cartx
    Cart_to_xx[1] = Carty
    Cart_to_xx[2] = Cartz

    scalefactor_orthog[0] = sp.sympify(1)
    scalefactor_orthog[1] = sp.sympify(1)
    scalefactor_orthog[2] = sp.sympify(1)

    # Set the transpose of the matrix of unit vectors
    UnitVectors = [[sp.sympify(1), sp.sympify(0), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(1), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(0), sp.sympify(1)]]
```

```python
%matplotlib inline

import numpy as np
import matplotlib.pyplot as plt

plt.clf()

fig = plt.figure()
ax = fig.gca()
Nx = 16
ax.set_xticks(np.arange(0, 1., 1./Nx))
ax.set_yticks(np.arange(0, 1., 1./Nx))
# plt.scatter(x, y)
ax.set_aspect('equal')

plt.grid()

# plt.savefig("Cartgrid.png",dpi=300)
plt.show()
# plt.close(fig)
```

## Step 3.d: [Prolate spheroidal](https://en.wikipedia.org/wiki/Prolate_spheroidal_coordinates)-like coordinate systems \[Back to [top](#toc)\]
$$\label{prolatespheroidal}$$

### Step 3.d.i: **`reference_metric::CoordSystem = "SymTP"`** \[Back to [top](#toc)\]
$$\label{symtp}$$

Symmetric TwoPuncture coordinates, with $(\rho,\phi,z)=(xx_0\sin(xx_1), xx_2, \sqrt{xx_0^2 + \text{bScale}^2}\cos(xx_1))$

```python
if CoordSystem == "SymTP":

    var1, var2= sp.symbols('var1 var2',real=True)
    bScale, AW, AMAX, RHOMAX, ZMIN, ZMAX = par.Cparameters("REAL",thismodule,
                                                           ["bScale","AW","AMAX","RHOMAX","ZMIN","ZMAX"],
                                                           [0.5,     0.2,   10.0,    10.0, -10.0,  10.0])

    # Assuming xx0, xx1, and bScale
    #   are positive makes nice simplifications of
    #   unit vectors possible.
    xx[0],xx[1] = sp.symbols("xx0 xx1", real=True)

    xxmin = [sp.sympify(0), sp.sympify(0),-M_PI]
    xxmax = [         AMAX,          M_PI, M_PI]

    AA = xx[0]

    if CoordSystem == "SinhSymTP":
        AA = (sp.exp(xx[0]/AW)-sp.exp(-xx[0]/AW))/2

    var1 = sp.sqrt(AA**2 + (bScale * sp.sin(xx[1]))**2)
    var2 = sp.sqrt(AA**2 + bScale**2)

    RHOSYMTP = AA*sp.sin(xx[1])
    PHSYMTP = xx[2]
    ZSYMTP = var2*sp.cos(xx[1])

    xxCart[0] = AA  *sp.sin(xx[1])*sp.cos(xx[2])
    xxCart[1] = AA  *sp.sin(xx[1])*sp.sin(xx[2])
    xxCart[2] = ZSYMTP

    xxSph[0] = sp.sqrt(RHOSYMTP**2 + ZSYMTP**2)
    xxSph[1] = sp.acos(ZSYMTP / xxSph[0])
    xxSph[2] = PHSYMTP

    rSph  = sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)
    thSph = sp.acos(Cartz / rSph)
    phSph = sp.atan2(Carty, Cartx)

    # Mathematica script to compute Cart_to_xx[]
    #             AA = x1;
    #             var2 = Sqrt[AA^2 + bScale^2];
    #             RHOSYMTP = AA*Sin[x2];
    #             ZSYMTP = var2*Cos[x2];
    #             Solve[{rSph == Sqrt[RHOSYMTP^2 + ZSYMTP^2],
    #                    thSph == ArcCos[ZSYMTP/Sqrt[RHOSYMTP^2 + ZSYMTP^2]],
    #                    phSph == x3},
    #                   {x1, x2, x3}]
    Cart_to_xx[0] = sp.sqrt(-bScale**2 + rSph**2 +
                            sp.sqrt(bScale**4 + 2*bScale**2*rSph**2 + rSph**4 -
                                    4*bScale**2*rSph**2*sp.cos(thSph)**2))*M_SQRT1_2 # M_SQRT1_2 = 1/sqrt(2); define this way for UnitTesting

    # The sign() function in the following expression ensures the correct root is taken.
    Cart_to_xx[1] = sp.acos(sp.sign(Cartz)*(
                              sp.sqrt(1 + rSph**2/bScale**2 -
                                      sp.sqrt(bScale**4 + 2*bScale**2*rSph**2 + rSph**4 -
                                              4*bScale**2*rSph**2*sp.cos(thSph)**2)/bScale**2)*M_SQRT1_2)) # M_SQRT1_2 = 1/sqrt(2); define this way for UnitTesting

    Cart_to_xx[2] = phSph
```

### Step 3.d.ii: **`reference_metric::CoordSystem = "SinhSymTP"`** \[Back to [top](#toc)\]
$$\label{sinhsymtp}$$

Symmetric TwoPuncture coordinates, but with $$xx_0 \to \sinh(xx_0/\text{AW})$$

```python
if CoordSystem == "SinhSymTP":

    var1, var2= sp.symbols('var1 var2',real=True)
    bScale, AW, AMAX, RHOMAX, ZMIN, ZMAX = par.Cparameters("REAL",thismodule,
                                                           ["bScale","AW","AMAX","RHOMAX","ZMIN","ZMAX"],
                                                           [0.5,     0.2,   10.0,    10.0, -10.0,  10.0])

    # Assuming xx0, xx1, and bScale
    #   are positive makes nice simplifications of
    #   unit vectors possible.
    xx[0],xx[1] = sp.symbols("xx0 xx1", real=True)

    xxmin = [sp.sympify(0), sp.sympify(0),-M_PI]
    xxmax = [         AMAX,          M_PI, M_PI]

    AA = xx[0]

    if CoordSystem == "SinhSymTP":
        # With xxmax[0] == AMAX, sinh(xx0/AMAX) will evaluate to a number between 0 and 1.
        #   Similarly, sinh(xx0/(AMAX*SINHWAA)) / sinh(1/SINHWAA) will also evaluate to a number between 0 and 1.
        #   Then AA = AMAX*sinh(xx0/(AMAX*SINHWAA)) / sinh(1/SINHWAA) will evaluate to a number between 0 and AMAX.
        AA = AMAX * (sp.exp(xx[0] / (AMAX*SINHWAA)) - sp.exp(-xx[0] / (AMAX*SINHWAA))) / (sp.exp(1 / SINHWAA) - sp.exp(-1 / AMAX))

    var1 = sp.sqrt(AA**2 + (bScale * sp.sin(xx[1]))**2)
    var2 = sp.sqrt(AA**2 + bScale**2)

    RHOSYMTP = AA*sp.sin(xx[1])
    PHSYMTP = xx[2]
    ZSYMTP = var2*sp.cos(xx[1])

    xxCart[0] = AA  *sp.sin(xx[1])*sp.cos(xx[2])
    xxCart[1] = AA  *sp.sin(xx[1])*sp.sin(xx[2])
    xxCart[2] = ZSYMTP

    xxSph[0] = sp.sqrt(RHOSYMTP**2 + ZSYMTP**2)
    xxSph[1] = sp.acos(ZSYMTP / xxSph[0])
    xxSph[2] = PHSYMTP

    scalefactor_orthog[0] = sp.diff(AA,xx[0]) * var1 / var2
    scalefactor_orthog[1] = var1
    scalefactor_orthog[2] = AA * sp.sin(xx[1])

    # Set the transpose of the matrix of unit vectors
    UnitVectors = [[sp.sin(xx[1]) * sp.cos(xx[2]) * var2 / var1,
                    sp.sin(xx[1]) * sp.sin(xx[2]) * var2 / var1,
                    AA * sp.cos(xx[1]) / var1],
                   [AA * sp.cos(xx[1]) * sp.cos(xx[2]) / var1,
                    AA * sp.cos(xx[1]) * sp.sin(xx[2]) / var1,
                        -sp.sin(xx[1]) * var2 / var1],
                   [-sp.sin(xx[2]), sp.cos(xx[2]), sp.sympify(0)]]
```

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Reference_Metric.pdf](Tutorial-Reference_Metric.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx --log-level='WARN' Tutorial-Reference_Metric.ipynb
!pdflatex -interaction=batchmode Tutorial-Reference_Metric.tex
!pdflatex -interaction=batchmode Tutorial-Reference_Metric.tex
!pdflatex -interaction=batchmode Tutorial-Reference_Metric.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    [NbConvertApp] Converting notebook Tutorial-Reference_Metric.ipynb to latex
    [NbConvertApp] Support files will be in Tutorial-Reference_Metric_files/
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Making directory Tutorial-Reference_Metric_files
    [NbConvertApp] Writing 135717 bytes to Tutorial-Reference_Metric.tex
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode


# YOUR PROJECT TITLE

> **Note the following:**
> 1. This is *not* meant to be an example of an actual **model analysis project**, just an example of how to structure such a project.
> 1. Remember the general advice on structuring and commenting your code from [lecture 5](https://numeconcopenhagen.netlify.com/lectures/Workflow_and_debugging).
> 1. Remember this [guide](https://www.markdownguide.org/basic-syntax/) on markdown and (a bit of) latex.
> 1. Turn on automatic numbering by clicking on the small icon on top of the table of contents in the left sidebar.
> 1. The `modelproject.py` file includes a function which could be used multiple times in this notebook.

Imports and set magics:

```python
#import numpy as np
#from scipy import optimize
#import sympy as sm

# autoreload modules when code is run
#%load_ext autoreload
#%autoreload 2

# local modules
#import modelproject
```

# Model description

**Write out the model in equations here.**

Make sure you explain well the purpose of the model and comment so that other students who may not have seen it before can follow.

## Analytical solution

If your model allows for an analytical solution, you should provide here.

You may use Sympy for this. Then you can characterize the solution as a function of a parameter of the model.

To characterize the solution, first derive a steady state equation as a function of a parameter using Sympy.solve and then turn it into a python function by Sympy.lambdify. See the lecture notes for details.

## Numerical solution

You can always solve a model numerically.

Define first the set of parameters you need.

Then choose one of the optimization algorithms that we have gone through in the lectures based on what you think is most fitting for your model.

Are there any problems with convergence? Does the model converge for all starting values? Make a lot of testing to figure these things out.

# Further analysis

Make detailed vizualizations of how your model changes with parameter values.

Try to make an extension of the model.

# Conclusion

Add concise conclusion.

```python

```

Writing latex using:

https://personal.math.ubc.ca/~pwalls/math-python/jupyter/latex/

Brug scipy linalg til at løse ax=b for x.
sympy kan bruges med symboler og beregne som man normalt gør. Dette skriver resultaterne som funktioner med tegn.

# Modelproject

# Importing

```python
import modelproject
import sympy as sm
import numpy as np
import ipywidgets as widgets
import matplotlib.pyplot as plt

%load_ext autoreload
%autoreload 2
```

Describe the model and the intention

**The model**

In this project, we examine the OLG-model. It is based on an idea, that agents live for two periods, and maximizes their utility in these.

Utility is given by:
$$U_t=ln(c_{1t})+\beta ln(c_{2t+1})$$
where: $$\beta>0$$

Consumption is given by:
$$c_{1t}=w_t-s_t$$
$$c_{2t+1}=R_{t+1} s_t$$

Assuming a cobb douglas production function:
$$Y_t=K_t^\alpha L_t^{1-\alpha}$$
where: $$0<\alpha<1$$

**Variables and parameters**

In the model, small letters are pr. worker terms, and capital letters are agregate.

U is utility

C is consumption. 1t is for young individuals in period t, while 2t+1 is for the old generation in periode t+1

K is capital

S is saving

W is wage

R is interest rate

beta is the time preferences

alpha is the share of capital in production

**Extension for model from PS5 OPG2:**

$$Y_t^i=A(K_t^i)^\alpha (K_t N_t^i)^{1-\alpha}$$
where: $$A>0$$

**Extension for model with PAYG og FF:**

**Pay as you go:**
$$c_{1t}=w_t(1-\tau)-s_t$$
$$c_{2t+1}=s_tR_{t+1}+{\tau}w_{t+1}$$

**Fully funded:**
$$c_{1t}=w_t(1-\tau)-s_t$$
$$c_{2t+1}=(s_t+{\tau}w_t)R_{t+1}$$

**Fully funded is always prefered, since it does not affect capital accumulation**

## Analytical solution

We start by defining the variables with sympy, in order to solve the model analytically:

```python
# Aggregate variables
U_t = sm.symbols('U_t')
R_t1 = sm.symbols('R_t1')
Y_t = sm.symbols('Y_t')
K_t = sm.symbols('K_t')
L_t = sm.symbols('L_t')
N_t = sm.symbols('N_t')
A = sm.symbols('A')
pi_t = sm.symbols('Pi_t')

# pr. worker variables
r_t = sm.symbols('r_t')
r_t_star = sm.symbols('r_t^*')
r_star = sm.symbols('r^*')
s_t = sm.symbols('s_t')
s_t_star = sm.symbols('s_t^*')
s_star = sm.symbols('s^*')
w_t = sm.symbols('w_t')
w_star = sm.symbols('w^*')
k_t = sm.symbols('k_t')
k_ss = sm.symbols('k_ss')
k_star = sm.symbols('k^*')
c_1t = sm.symbols('c_1t')
c_2t1 = sm.symbols('c_2t+1')
i = sm.symbols('i')
n = sm.symbols('n')

# Parameters
alpha = sm.symbols('alpha')
beta = sm.symbols('beta')
```

The firms profit is given by their production minus the cost of inputs:

```python
# The firm maximizes profit
profit = K_t**alpha * L_t**(1-alpha) - r_t * K_t - w_t*L_t
sm.Eq(pi_t,profit)
```

$\displaystyle \Pi_{t} = - K_{t} r_{t} + K_{t}^{\alpha} L_{t}^{1 - \alpha} - L_{t} w_{t}$

Now the profitmaximizing rent is found by finding the first order condition and then solve:

```python
# The profitmaximizing rent is found
# FOC wrt. capital
profit_diff_K = sm.diff(profit,K_t)

# Solving for rental rent
r_solution = sm.solve(profit_diff_K,r_t)[0]
#r_solution.subs(K_t**(alpha-1)*L_t**(1-alpha),k_t**(alpha-1))
sm.Eq(r_t,r_solution)
```

$\displaystyle r_{t} = K_{t}^{\alpha - 1} L_{t}^{1 - \alpha} \alpha$

```python
# The rent rewritten
r_rewritten = alpha*k_t**(alpha-1)

# Saving the rent as a python function:
r_ss = sm.lambdify((alpha,k_t),r_rewritten)
```

Now the profitmaximizing wage is found in the same way as the rent:

```python
# The profitmaximizing wage is found
# FOC wrt. labor
profit_diff_L = sm.diff(profit,L_t)

# Solving for wage
w_solution = sm.solve(profit_diff_L,w_t)[0]
sm.Eq(w_t,w_solution)
```

$\displaystyle w_{t} = K_{t}^{\alpha} L_{t}^{- \alpha} \left(1 - \alpha\right)$

```python
# The wage rewritten
w_rewritten = (1-alpha)*k_t**alpha

# Saving the wage as a python function:
w_ss = sm.lambdify((alpha,k_t),w_rewritten)
```

Given the wage and rent, the agents maximizes utility in the two periods:

```python
# The utilitymaximizing saving is found
# The budgetconstraints is given by
budgetcons_1 = w_t - s_t
budgetcons_2 = s_t*R_t1

# The budget constraints are inserted into the utility
utility = sm.ln(budgetcons_1) + beta*sm.ln(budgetcons_2)
utility

# FOC wrt. s
utility_diff_s = sm.diff(utility,s_t)

# Solving for s
s_solution = sm.solve(utility_diff_s,s_t)[0]
sm.Eq(s_t,s_solution)
```

$\displaystyle s_{t} = \frac{\beta w_{t}}{\beta + 1}$

```python
# The saving rewritten
s_rewritten = (beta/(beta+1))*(1-alpha)*k_t**alpha

# Saving the savings as a python function:
s_ss = sm.lambdify((beta,alpha,k_t),s_rewritten)
```

We use the savings to find the staedy state capital. We have that:
$$K_{t+1}=S_t$$
and
$$L_{t+1}={L_t}(1+n)$$

```python
# The steady state capital is found
# In steady state we have k_t+1=s_t:
savings = ((beta)/((1+beta)*(1+n)))*(k_ss**alpha)*(1-alpha)-k_ss

k_solution = sm.solve(savings,k_ss)[0]
k_solution
sm.Eq(k_ss,k_solution)
```

$\displaystyle k_{ss} = \left(\frac{\beta \left(1 - \alpha\right)}{\beta n + \beta + n + 1}\right)^{- \frac{1}{\alpha - 1}}$

```python
# The capital rewritten:
k_rewritten = ((beta*(1-alpha))/((1+beta)*(1+n)))**(1/(1-alpha))

# Saving the capital as a python function:
k_ss = sm.lambdify((beta, alpha, n),k_rewritten)
```

```python
# We have the following results:
display(sm.Eq(r_star,r_rewritten),
        sm.Eq(w_star,w_rewritten),
        sm.Eq(s_star,s_rewritten),
        sm.Eq(k_star,k_rewritten))
```

$\displaystyle r^{*} = \alpha k_{t}^{\alpha - 1}$

$\displaystyle w^{*} = k_{t}^{\alpha} \left(1 - \alpha\right)$

$\displaystyle s^{*} = \frac{\beta k_{t}^{\alpha} \left(1 - \alpha\right)}{\beta + 1}$

$\displaystyle k^{*} = \left(\frac{\beta \left(1 - \alpha\right)}{\left(\beta + 1\right) \left(n + 1\right)}\right)^{\frac{1}{1 - \alpha}}$

## Numerical solution

**Solving nummerically**

Now we solve the model nummerically. We assume that the agent place higher utility on consumption in the first period than on consumption in the second period. Therefore beta is below 1.

```python
# The capital
beta_estimat = 2/3
alpha_estimat = 1/3
n_estimat = 0.1
ss_capital = k_ss(beta_estimat,alpha_estimat,n_estimat)
ss_saving = s_ss(beta_estimat,alpha_estimat,ss_capital)
ss_rent = r_ss(alpha_estimat, ss_capital)
ss_wage = w_ss(alpha_estimat, ss_capital)

print("Rent in steady state is: ", ss_rent)
print("Wage in steady state is: ", ss_wage)
print("Saving in steady state is: ", ss_saving)
print("Capital in steady state is: ", ss_capital)
```

    Rent in steady state is:  1.3749999999999996
    Wage in steady state is:  0.3282439759448874
    Saving in steady state is:  0.13129759037795496
    Capital in steady state is:  0.1193614457981409

## Monte carlo simulation:

**Simulating utility**

We start by simulating the utility as a function of beta, which we assume to be uniformly destributed.

```python
print(modelproject.MonteCarlo(alpha_estimat, n_estimat))
```

    0.1200501800356351

## Interactive graph

```python
widgets.interact(modelproject.fig,
    beta = widgets.FloatSlider(description='beta', min=0, max=1, step=0.01, value=0.5),
    alpha = widgets.fixed(1/3)
);
```

    interactive(children=(FloatSlider(value=0.5, description='beta', max=1.0, step=0.01), Output()), _dom_classes=…

# Extension

**Pay as you go:**
$$c_{1t}=w_t(1-\tau)-s_t$$
$$c_{2t+1}=s_tR_{t+1}+{\tau}w_{t+1}$$

## Analytical

```python
w_t1 = sm.symbols('w_t1')
k_ss_PAYG = sm.symbols('k_ss(PAYG)')
tau = sm.symbols('tau')
```

```python
# The utilitymaximizing saving is found
# The budgetconstraints is given by
budgetcons_1_PAYG = w_t*(1-tau) - s_t
budgetcons_2_PAYG = s_t*R_t1+tau*w_t1

# The budget constraints are inserted into the utility
utility_PAYG = sm.ln(budgetcons_1_PAYG) + beta*sm.ln(budgetcons_2_PAYG)
utility_PAYG

# FOC wrt. s
utility_diff_s_PAYG = sm.diff(utility_PAYG,s_t)

# Solving for s
s_solution_PAYG = sm.solve(utility_diff_s_PAYG,s_t)[0]
sm.Eq(s_t,s_solution_PAYG)
```

$\displaystyle s_{t} = \frac{- R_{t1} \beta \tau w_{t} + R_{t1} \beta w_{t} - \tau w_{t1}}{R_{t1} \left(\beta + 1\right)}$

```python
# The saving rewritten
s_rewritten_PAYG = (((R_t1*beta*w_t)*(1-tau))-tau*w_t1)/(R_t1*(beta+1))

# Saving the savings as a python function:
s_ss_PAYG = sm.lambdify((beta,alpha,tau,R_t1,w_t,w_t1),s_rewritten_PAYG)
s_rewritten_PAYG
```

$\displaystyle \frac{R_{t1} \beta w_{t} \left(1 - \tau\right) - \tau w_{t1}}{R_{t1} \left(\beta + 1\right)}$

```python
# The steady state capital is found
# In steady state we have k_t+1=s_t:
savings_PAYG = (((R_t1*beta*(1-alpha)*(k_ss_PAYG**alpha))*(1-tau))-tau*((1-alpha)*(k_ss_PAYG**alpha)))/(R_t1*(beta+1))-k_ss_PAYG

k_solution_PAYG = sm.solve(savings_PAYG,k_ss_PAYG)[0]
k_solution_PAYG
sm.Eq(k_ss_PAYG,k_solution_PAYG)
```

$\displaystyle k_{ss(PAYG)} = \left(\frac{R_{t1} \alpha \beta \tau - R_{t1} \alpha \beta - R_{t1} \beta \tau + R_{t1} \beta + \alpha \tau - \tau}{R_{t1} \left(\beta + 1\right)}\right)^{- \frac{1}{\alpha - 1}}$

```python
# The capital rewritten:
k_rewritten_PAYG = ((R_t1*beta*(alpha-1)*(tau-1)+alpha*tau-tau)/(R_t1*(beta+1)))**(1/(1-alpha))

# Saving the capital as a python function:
k_ss_PAYG = sm.lambdify((beta, alpha, n, tau, R_t1),k_rewritten_PAYG)
```

```python
# We have the following results:
display(sm.Eq(r_star,r_rewritten),
        sm.Eq(w_star,w_rewritten),
        sm.Eq(s_star,s_rewritten_PAYG),
        sm.Eq(k_star,k_rewritten_PAYG))
```

$\displaystyle r^{*} = \alpha k_{t}^{\alpha - 1}$

$\displaystyle w^{*} = k_{t}^{\alpha} \left(1 - \alpha\right)$

$\displaystyle s^{*} = \frac{R_{t1} \beta w_{t} \left(1 - \tau\right) - \tau w_{t1}}{R_{t1} \left(\beta + 1\right)}$

$\displaystyle k^{*} = \left(\frac{R_{t1} \beta \left(\alpha - 1\right) \left(\tau - 1\right) + \alpha \tau - \tau}{R_{t1} \left(\beta + 1\right)}\right)^{\frac{1}{1 - \alpha}}$

## Numerical

```python
# The capital
beta_estimat = 2/3
alpha_estimat = 1/3
n_estimat = 0.1
tau_estimat = 0.2
R_t1_estimat = 1.2

ss_capital_PAYG = k_ss_PAYG(beta_estimat,alpha_estimat,n_estimat, tau_estimat, R_t1_estimat)
ss_rent_PAYG = r_ss(alpha_estimat, ss_capital_PAYG)
ss_wage_PAYG = w_ss(alpha_estimat, ss_capital_PAYG)
ss_saving_PAYG = s_ss_PAYG(beta_estimat,alpha_estimat,tau_estimat,R_t1_estimat,ss_wage_PAYG,ss_wage_PAYG)

print("Rent in steady state is with PAYG: ", ss_rent_PAYG)
print("Wage in steady state is with PAYG: ", ss_wage_PAYG)
print("Saving in steady state is with PAYG: ", ss_saving_PAYG)
print("Capital in steady state is with PAYG: ", ss_capital_PAYG)
```

    Rent in steady state is with PAYG:  2.2727272727272716
    Wage in steady state is with PAYG:  0.2553138954016903
    Saving in steady state is with PAYG:  0.05616905698837187
    Capital in steady state is with PAYG:  0.05616905698837188

```python
s_rewritten_PAYG = ((((1.2)*(2/3)*w_t)*(1-(0.2)))-(0.2)*w_t1)/((1.2)*((2/3)+1))
```


Just to show simple this can get and not put you off so early, here's a few more simplifications. What if we're only concerned with very short-term dynamics (e.g. seasonal)? Then we can dispense with burial and arguably mixing which may occur on longer time scales. Then we get simply,

\begin{align}
\frac {\partial C}{\partial t} = aR - \lambda C \\
\end{align}
Maybe we've even got good reason to discard one of the two terms on the right-hand side of that equation (test production and taphonomic loss), in which case it gets even simpler.

\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\   \nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}

```

```


```
figsize(12.5,4)
import scipy.stats as stats
```

Chapter 1
======
***

The Philosophy of Bayesian Inference
------

> You are a skilled programmer, but bugs still slip into your code. After a particularly difficult implementation of an algorithm, you decide to test your code on a trivial example. It passes. You test the code on a harder problem. It passes once again. And it passes the next, *even more difficult*, test too! You are starting to believe that there are may be no bugs present...

If you think this way, then congratulations, you already are a Bayesian practitioner! Bayesian inference is simply updating your beliefs after considering new evidence. A Bayesian can rarely be certain about a result, but he or she can be very confident. Just like in the example above, we can never be 100% sure that our code is bug-free unless we test it on every possible problem; something rarely possible in practice. Instead, we can test it on a large number of problems, and if it succeeds we can feel more *confident* about our code.  Bayesian inference works identically: we can only update our beliefs about an outcome, rarely can we be absolutely sure unless we rule out all other alternatives. We will see that being uncertain can have its advantages.

###The Bayesian state of mind

Bayesian inference differs from more traditional statistical analysis by preserving *uncertainty* about our beliefs. At first, this sounds like a bad statistical technique. Isn't statistics all about deriving *certainty* from randomness? The Bayesian method interprets probability as measure of *believability in an event*, that is, how confident one is in an event occuring. In fact, we will see in a moment that this is the natural interpretation of probability.

For this to be clearer, we consider an alternative interpretation of probability: *Frequentist* methods assume that probability is the long-run frequency of events (hence the bestowed title). For example, the *probability of plane accidents* under a frequentist philosophy is interpreted as the *long-term frequency of plane accidents*. This makes logical sense for many probabilities and events, but becomes more difficult to understand when events have no long-term frequency of occurences. Consider: we often assign probabilities to outcomes of presidential elections, but the election itself only happens once! Frequentists get around this by invoking alternative realities and saying across all these universes, the frequency of occurences is the probability.

Bayesians, on the other hand, have a more intuitive approach. Bayesians interpret a probability as measure of *belief*, or confidence, of an event occurring. An individual who assigns a belief of 0 to an event has no confidence that the event will occur; conversely, assigning a belief of 1 implies that the individual is absolutely certain of an event occurring. Beliefs between 0 and 1 allow for weightings of other outcomes. This definition agrees with the probability of a plane accident example, for having observed the frequency of plane accidents, an individual's belief should be equal to that frequency. Similarly, under this definition of probability being equal to beliefs, it is clear how we can speak about probabilities (beliefs) of presidential election outcomes.

Notice in the paragraph above, I assigned the belief (probability) measure to an *individual*, not to Nature. This is very interesting, as this definition leaves room for conflicting beliefs between individuals. Again, this is appropriate for what naturally occurs: different individuals have different beliefs of events occuring, because they possess different *information* about the world.

Think about how we can extend this definition of probability to events that are not *really* random. That is, we can extend this to anything that is fixed, but we are unsure about:

-  Your code either has a bug in it or not, but we do not know for certain which is true. Though we have a belief about the presence or absence of a bug.

-  A medical patient is exhibiting symptoms $x$, $y$ and $z$. There are a number of diseases that could be causing all of them, but only a single disease is present. A doctor has beliefs about which disease.

- You believe that the pretty girl in your English class doesn't have a crush you. You assign a low probability that she does. She, on the other hand, knows for certain that she *does indeed* like you. She (implicitly) assigns a probability 1.

This philosophy of treating beliefs as probability is natural to humans. We employ it constantly as we interact with the world and only see partial evidence. Alternatively, you have to be *trained* to think like a frequentist.

To align ourselves with traditional probability notation, we denote our belief about event $A$ as $P(A)$.

John Maynard Keynes, a great economist and thinker, said "When the facts change, I change my mind. What do you do, sir?" This quote reflects the way a Bayesian updates his or her beliefs after seeing evidence. Even --especially-- if the evidence is counter to what was initially believed, the evidence cannot be ignored. We denote our updated belief as $P(A |X )$, interpreted as the probability of $A$ given the evidence $X$. We call the updated belief the *posterior probability* so as to contrast it with the *prior probability*. For example, consider the posterior probabilities (read: posterior belief) of the above examples, after observing some evidence $X$.:

1\.   $P(A): \;\;$  This big, complex code likely has a bug in it. $P(A | X): \;\;$ The code passed all $X$ tests; there still might be a bug, but its presence is less likely now.

2\.  $P(A):\;\;$ The patient could have any number of diseases. $P(A | X):\;\;$ Performing a blood test generated evidence $X$, ruling out some of the possible diseases from consideration.

3\.  $P(A):\;\;$ That girl in your class probably doesn't have a crush on you.  $P(A | X): \;\;$           She sent you an SMS message about some statistics homework. Maybe she does like me...

It's clear that in each example we did not completely discard the prior belief after seeing new evidence, but we *re-weighted the prior* to incorporate the new evidence (i.e. we put more weight, or confidence, on some beliefs versus others).

By introducing prior uncertainity about events, we are already admitting that any guess we make is potentially very wrong. After observing data, evidence, or other information, and we update our beliefs, our guess becomes *less wrong*. This is the alternative side of the prediction coin, where typically we try to be *more right*.

###Bayesian Inference in Practice

 If frequentist and Bayesian inference were computer programming functions, with inputs being statistical problems, then the two would be different in what they return to the user. The frequentist inference function would return a number, whereas the Bayesian function would return a *distribution*.

For example, in our debugging problem above, calling the frequentist function with the argument "My code passed all $X$ tests; is my code bug-free?" would return a *YES*. On the other hand, asking our Bayesian function "Often my code has bugs. My code passed all $X$ tests; is my code bug-free?" would return something very different: a distribution over *YES* and *NO*. The function might return

>    *YES*, with probability 0.8; *NO*, with probability 0.2

This is very different from the answer the frequentist function returned. Notice that the Bayesian function accepted an additional argument:  *"Often my code has bugs"*. This parameter is the *prior*. By including the prior parameter, we are telling the Bayesian function to include our personal belief about the situation. Technically this parameter in the Bayesian function is optional, but we will see excluding it has its own consequences.

As we acquire more and more instances of evidence, our prior belief is *washed out* by the new evidence. This is to be expected. For example, if your prior belief is something ridiculous, like "I expect the sun to explode today", and each day you are proved wrong, you would hope that any inference would correct you, or at least align your beliefs better.

Denote $N$ as the number of instances of evidence we possess. As we gather an *infinite* amount of evidence, say as $N \rightarrow \infty$, our Bayesian results align with frequentist results. Hence for large $N$, statistical inference is more or less objective. On the other hand, for small $N$, inference is much more *unstable*: frequentist estimates have more variance and larger confidence intervals. This is where Bayesian analysis excels. By introducing a prior, and returning a distribution (instead of a scalar estimate), we *preserve the uncertainity* to reflect the instability of stasticial inference of a small $N$ dataset.

One may think that for large $N$, one can be indifferent between the two techniques, and might lean towards the computational-simpler, frequentist methods. An analysist in this position should consider the following quote by Andrew Gelman (2005)[1], before making such a decision:

> Sample sizes are never large. If $N$ is too small to get a sufficiently-precise estimate, you need to get more data (or make more assumptions). But once $N$ is "large enough," you can start subdividing the data to learn more (for example, in a public opinion poll, once you have a good estimate for the entire country, you can estimate among men and women, northerners and southerners, different age groups, etc etc). $N$ is never enough because if it were "enough" you'd already be on to the next problem for which you need more data.

#### A note on *Big Data*
Paradoxically, big data's predictive analytic problems are actually solved by relatively simple models [2]. Thus we can argue that big data's prediction difficulty does not lie in the algorithm used, but instead on the computational difficulties of storage and execution on big data. (One should also consider Gelman's qoute from above and ask "Do I really have a big data prediction problem?" )

The much more difficult analytic problems involve *medium data* and, especially troublesome, *really small data*. Using a similar argument as  Gelman's above, if big data problems are *big enough* to be readily solved, then we should be more interested in the *not-quite-big enough* datasets.

### Our Bayesian framework

We are interested in beliefs, which can be interpreted as probabilities by thinking Bayesian. We have a *prior* belief in event $A$: it is what you believe before looking at any evidence, e.g., our prior belief about bugs being in our code before performing tests.

Secondly, we observe our evidence. To continue our example, if our code passes $X$ tests, we want to update our belief to incorporate this. We call this new belief the *posterior* probability. Updating our belief is done via the following equation, known as Bayes' Theorem, after Thomas Bayes:

\begin{align}
 P( A | X ) = & \frac{ P(X | A) P(A) } {P(X) } \\\\[5pt]
& \propto P(X | A) P(A)\;\; (\propto \text{is proportional to } )
\end{align}

The above formula is not unique to Bayesian inference: it is a mathematical fact with uses outside Bayesian inference. Bayesian inference merely uses it to connect $P(A)$ with an updated $P(A | X )$.

#####Example: Coding Horror
______

Let $A$ denote the event that our code has **no bugs** in it. Let $X$ denote the event that the code passes all debugging tests. For now, we will leave the prior probability of no bugs as a variable, i.e. $P(A) = p$.

We are interested in $P(A|X)$, i.e. the probability of no bugs, given our debugging tests $X$. To use the formula above, we need to compute some quantities from the formula above.

What is $P(X | A)$, i.e., the probability that the code passes $X$ tests *given* there are no bugs? Well, it is equal to 1, for a code with no bugs will pass all tests.

$P(X)$ is a little bit trickier: The event $X$ can be divided into two possibilities, event $X$ ocurring even though our code *indeed has* bugs (denoted $\sim A\;$, spoken *not $A$*), or event $X$ without bugs ($A$). $P(X)$ can be represented as:

\begin{align}
P(X ) & = P(X \text{ and } A) + P(X \text{ and } \sim A) \\\\[5pt]
 & = P(X|A)P(A) + P(X | \sim A)P(\sim A)\\\\[5pt]
& = P(X|A)p + P(X | \sim A)(1-p)
\end{align}

We have already computed $P(X|A)$ above. On the other hand, $P(X | \sim A)$ is subjective: our code can pass tests but still have a bug in it, though the probability there is a bug present is less. Note this is dependent on the number of tests performed, the degree of complication in the tests, etc. Let's be conservative and assign $P(X|\sim A) = 0.5$. Then

\begin{align}
P(A | X) & = \frac{1\cdot p}{ 1\cdot p +0.5 (1-p) } \\\\
& = \frac{ 2 p}{1+p}
\end{align}
This is the posterior probability distribution. What does it look like as a function of our prior, $p \in [0,1]$?

```
figsize( 8, 4 )
p = np.linspace( 0,1, 50)
plt.plot( p, 2*p/(1+p), color = "#348ABD" )
plt.fill_between( p, 2*p/(1+p), alpha = .4, facecolor = ["#348ABD"])
plt.scatter( 0.2, 2*(0.2)/1.2, s = 140, c ="#348ABD"  )
plt.xlim( 0, 1)
plt.ylim( 0, 1)
plt.xlabel( "Prior, $P(A) = p$")
plt.ylabel("Posterior, $P(A|X)$, with $P(A) = p$")
plt.title( "Are there bugs in my code?")
```

We can see the biggest gains if we observe the $X$ tests passed are when the prior probability, $p$, is low. Let's settle on a specific value for the prior. I'm a (I think) strong programmer, so I'm going to give myself a realistic prior of 0.20, that is, there is a 20% chance that I write code bug-free. To be more realistic, this prior should be a function of how complicated and large the code is, but let's pin it at 0.20. Then my updated belief that my code is bug-free is 0.33.

Let's not forget from the idea that the prior is a probability distribution: $p$ is the prior probability that there *are no bugs*, so $1-p$ is the prior probability that there *are bugs*. What does our prior probability distribution look like?

Similarly, our posterior is also a probability distribution, with $P(A | X)$ the probability there is no bug *given we saw all tests pass*, hence $1-P(A|X)$ is the probability there is a bug *given all tests passed*. What does our posterior probability distribution look like? Below is a graph of both the prior and the posterior distributions.

```
figsize( 8, 4 )
prior = [0.20, 0.80]
postierior =  [1./3, 2./3]
plt.bar( [0,.7], prior ,alpha = 0.80, width = 0.25, \
            color = "#348ABD", label = "prior distribution" )
plt.bar( [0+0.25,.7+0.25], postierior ,alpha = 0.7, \
          width = 0.25, color =  "#A60628", label = "posterior distribution" )

plt.xticks( [0.20,.95], ["Bugs Absent", "Bugs Present"] )
plt.title("Prior and Posterior probability of bugs present, prior = 0.2")
plt.ylabel("Probability")
plt.legend(loc="upper left")
```

Notice that after we observed $X$ occur, the probability of no bugs present increased. By increasing the number of tests, we can approach confidence (probability 1) that there are not bugs.

This was a very simple example, but the mathematics from here only becomes difficult except for artifically constructed instances. We will see that this math is actually unnecessary. First we must broaden our modeling tools.

_______

##Probability Distributions

**Let's quickly recall what a probability distribution is:** Let $Z$ be some random variable. Then associated with $Z$ is a *probability distribution function* that assigns probabilities to the different outcomes $Z$ can take. There are three cases:

-   **$Z$ is discrete**: Discrete random variables may only assume values on a specified list. Things like populations, movie ratings, and number of votes are all discrete random variables. It's more clear when we contrast it with...

-   **$Z$ is continuous**: Continuous random variable can take on arbitrarily exact values. For example, temperature, speed, time, color are all modeled as continuous variables because you can constantly make the values more and more precise.

- **$Z$ is mixed**: Mixed random variables assign probabilities to both discrete and continuous random variables, i.e. is is a combination of the above two categories.

###Discrete Case
If $Z$ is discrete, then its distribution is called a *probability mass function*, which measures the probability $Z$ takes on the value $k$, denoted $P(Z=k)$. Note that the probability mass function completely describes the random variable $Z$, that is, if we know the mass function, we know how $Z$ should behave. There are popular probability mass functions that consistently appear: we will introduce them as needed, but let's introduce the first very useful probability mass function. We say $Z$ is *Poisson*-distributed if:

$$P(Z = k) =\frac{ \lambda^k e^{-\lambda} }{k!}, \; \; k=0,1,2, \dots $$

What is $\lambda$? It is called the parameter, and it describes the shape of the distribution. For the Poisson random variable, $\lambda$ can be any positive number. By increasing $\lambda$, we add more probability to larger values, and conversely by decreasing $\lambda$ we add more probability to smaller values. Unlike $\lambda$, which can be any positive number, $k$ must be a non-negative integer, i.e., $k$ must take on values 0,1,2, and so on. This is very important, because if you wanted to model a population you could not make sense of populations with 4.25 or 5.612 members.

If a random variable $Z$ has a Poisson mass distribution, we denote this by writing

$$Z \sim \text{Poi}(\lambda) $$

One very useful property of the Poisson random variable, given we know $\lambda$, is that its expected value is equal to the parameter, ie.:

$$E\large[ \;Z\; | \; \lambda \;\large] = \lambda $$

We will use this property often, so it's something useful to remember. Below we plot the probablity mass distribution for different $\lambda$ values. The first thing to notice is that by increasing $\lambda$ we add more probability to larger values occuring. The second notice is although the graph ends at 15, the distributions do not. They assign positive probability to every integer.

```
import scipy.stats as stats
a = np.arange( 16 )
poi = stats.poisson
lambda_ = [1.5, 4.25 ]
colours = ["#348ABD", "#A60628"]

plt.bar( a, poi.pmf( a, lambda_[0]), color=colours[0],
        label = "$\lambda = %.1f$"%lambda_[0], alpha = 0.95)
plt.bar( a, poi.pmf( a, lambda_[1]), color=colours[1],
         label = "$\lambda = %.1f$"%lambda_[1], alpha = 0.60)

plt.xticks( a + 0.4, a )
plt.legend()
plt.ylabel("probability of $k$")
plt.xlabel("$k$")
plt.title("Probability mass function of a Poisson random variable; differing \
$\lambda$ values")
```

###Continuous Case
Instead of a probability mass function, a continuous random variable has a *probability density function*. This might seem like unnecessary nomenclature, but the density function and the mass function are very different creatures. An example of continuous random variable is a random variable with a *exponential density*. The density function for an exponential random variable looks like:

$$f_Z(z | \lambda) = \lambda e^{-\lambda z }, \;\; z\ge 0$$

Like the Poisson random variable, an exponential random variable can only take on non-negative values. But unlike a Poisson random variable, the exponential can take on *any* non-negative values, like 4.25 or 5.612401. This makes it a poor choice for count data, which must be integers, but a great choice for time data, or temperature data (measured in Kelvins, of course), or any other precise *and positive* variable. Below are two probability density functions with different $\lambda$ value.

When a random variable $Z$ has an exponential distribution with parameter $\lambda$, we say *$Z$ is exponential* and write

$$Z \sim \text{Exp}(\lambda)$$

Given a specific $\lambda$, the expected value of an exponential random variable is equal to the inverse of $\lambda$, that is:

$$E[\; Z \;|\; \lambda \;] = \frac{1}{\lambda}$$

```
a = np.linspace(0,4, 100)
expo = stats.expon
lambda_ = [0.5, 1]
colours = [ "#A60628", "#348ABD"]
for l,c in zip(lambda_,colours):
    plt.plot( a, expo.pdf( a, scale=1./l), lw=3,
                color=c, label = "$\lambda = %.1f$"%l)
    plt.fill_between( a, expo.pdf( a, scale=1./l), color=c, alpha = .33)

plt.legend()
plt.ylabel("PDF at $z$")
plt.xlabel("$z$")
plt.title("Probability density function of an Exponential random variable;\
 differing $\lambda$")
```

###But what is $\lambda \;\;$?

**This question is what motivates statistics**. In the real world, $\lambda$ is hidden from us. We only see $Z$, and must go backwards to try and determine $\lambda$. The problem is so difficult because there is not a one-to-one mapping from $Z$ to $\lambda$. Many different methods have been created to solve the problem of estimating $\lambda$, but since $\lambda$ is never actually observed, no one can say for certain which method is better!

Bayesian inference is concerned with *beliefs* about what $\lambda$ is. Rather than try to guess $\lambda$ exactly, we can only talk about what $\lambda$ is likely to be by assigning a probability distribution to $\lambda$.

This might seem odd at first: after all, $\lambda$ is fixed, it is not (necessarily) random! How can we assign probabilities to a non-random event. Ah, we have fallen for the frequentist interpretation. Recall, under our Bayesian philosophy, we *can* assign probabilties if we interpret them as beliefs. And it is entirely acceptable to have *beliefs* about the parameter $\lambda$.

##### Example: Inferring behaviour from text-message data
_____
Let's try to model a more interesting example, concerning text-message rates:

>  You are given a series of text-message counts from a user of your system. The data, plotted over time, appears in the graph below. You are curious if the user's text-messaging habits changed over time, either gradually or suddenly. How can you model this? (This is in fact my own text-message data. Judge my popularity as you wish.)

```
figsize( 12, 3.5 )
count_data = np.loadtxt("chp1data/txtdata.csv")
n_count_data = len(count_data)
plt.bar( np.arange( n_count_data ), count_data, color ="#348ABD"  )
plt.xlabel( "Time (days)")
plt.ylabel("# Text-msg recieved")
plt.title("Did the user's texting habits change over time?")
plt.xlim( 0, n_count_data )
print
```

Before we begin, with resepect to the plt above, do you say think there is a change in behaviour?

How can we start to model this? Well, as I conveniently already introduced, a Poisson random variable would be a very appropriate model for this *count* data. Denoting a day $i$'s text-message count $C_i$,

$$ C_i \sim \text{Poisson}(\lambda)  $$

We are not sure about what the $\lambda$ parameter is though. Looking at the chart above, it appears that the rate might become higher at some later date, which is equivalently saying the parameter $\lambda$ increases at some later date (recall a higher $\lambda$ means more probability on larger outcomes).

How can we mathematically represent this? We can think, that at some later date (call it $\tau$), the parameter $\lambda$ suddenly jumps to a higher value. So we create two $\lambda$ parameters, one for before the day $\tau$, and one for after. In literature, a sudden transition like this would be called a *switchpoint*:

$$
\lambda =
\begin{cases}
\lambda_1  & \text{if } t \lt \tau \cr
\lambda_2 & \text{if } t \ge \tau
\end{cases}
$$

 If, in reality, no sudden change occurred, the $\lambda$'s should look about equal. What would be a good prior distribution on $\lambda_{1}$ and $\lambda_2$?

Recall that $\lambda_i, \; i=1,2,$ can be any positive number. The *exponential* random variable has a density function for any positive number. This would be a good choice to model $\lambda_i$. But again, we need a parameter for this exponential distribution: call it $\alpha$.

\begin{align}
&\lambda_1 \sim \text{Exp}( \alpha ) \\\
&\lambda_2 \sim \text{Exp}( \alpha )
\end{align}

$\alpha$ is called a *hyper-parameter*, literally a parameter that influences other parameters. The influence is not too strong, so we can choose $\alpha$ liberally.  A good rule of thumb is to set the exponential parameter equal to the inverse of the average of the count data, since

$$\frac{1}{N}\sum_{i=0}^N \;C_i \approx E[\; \lambda \; |\; \alpha ] = \frac{1}{\alpha}$$

Alternatively, and something I encourage the reader to try, is to have two priors: one for each $\lambda$; creating two exponential distributions with different $\alpha$ values reflects our belief was that the rate changed (increased) after some period.

What about $\tau$? Well, due to the randomness, it is too difficult to pick out when $\tau$ might have occurred. Instead, we can assign an *uniform prior belief* to every possible day. This is equivalent to saying

\begin{align}
& \tau \sim \text{DiscreteUniform(1,70) }\\\\
& \Rightarrow P( \tau = k ) = \frac{1}{70}
\end{align}

So after all this, what does our prior distribution look like? Frankly, *it doesn't matter*. What we should understand is that it would be an ugly, complicated, mess involving symbols only a mathematician would love. And things would only get uglier the more complicated our models become. Regardless, all we really care about is the posterior distribution anyways. We next turn to PyMC, a Python library for performing Bayesian analysis, and is agnostic to the mathematical monster we have created.
____

Introducing our first hammer: PyMC
-----

PyMC is a Python library for programming Bayesian analysis  [3]. It is a fast, well-maintained library. The only unfortunate part is that documentation can be lacking in areas, especially the bridge between problem to solution. One this book's main goals is to solve that problem, and also to demonstrate why PyMC is so cool.

We will model the above problem using the PyMC library. This type of programming is called *probabilistic programming*, an unfortunate misnomer that invokes ideas of randomly-generated code and has likely confused and frightened users from this field. The code is not random. The title is given because we create probability models using pogramming variables as the model's components. This will be the last time I use the term *probablistic programming*. Instead, I'll simply use *programming*, as that is what it really is.

The PyMC code is easy to follow along: the only novel thing should be the syntax, and I will interrupt the code to explain sections. Simply remember we are representing the model's components ($\tau, \lambda_1, \lambda_2$ ) as variables:

```
import pymc as mc

n = count_data.shape[0]

alpha = 1.0/count_data.mean() #recall count_data is
                              #the variable that holds our txt counts

lambda_1 = mc.Exponential( "lambda_1",  alpha )
lambda_2 = mc.Exponential( "lambda_2", alpha )

tau = mc.DiscreteUniform( "tau", lower = 0, upper = n )
```

In the above code, we create the PyMC variables corresponding to $\lambda_1, \; \lambda_2$ in lines `8,9`. We assign them to PyMC's *stochastic variables*, called stochastic variables because they are treated by the backend as random number generators. We can test this by calling their built-in `random()` method.

```
print "Random output:", tau.random(),tau.random(), tau.random()
```

    Random output: 30 57 19

```
@mc.deterministic
def lambda_( tau = tau, lambda_1 = lambda_1, lambda_2 = lambda_2 ):
    out = np.zeros( n )
    out[:tau] = lambda_1 #lambda before tau is lambda1
    out[tau:] = lambda_2 #lambda after tau is lambda1
    return out
```

This code is creating a new function `lambda_`, but really we think of it as a random variable: the random variable $\lambda$ from above. Note that because `lambda_1`, `lambda_2` and `alpha` are random, `lambda_` will be random. We are **not** fixing any variables yet. The `@mc.deterministic` is a decorator to tell PyMC that this is a deterministic function, i.e., if the arguments were deterministic (which they are not), the output would be deterministic as well.

```
observation = mc.Poisson( "obs", lambda_, value = count_data, observed = True)

model = mc.Model( [observation, lambda_1, lambda_2, tau] )
```

The variable `observations` combines our data, `count_data`, with our proposed data-generation scheme, given by the variable `lambda_`, through the `value` keyword. We also set `observed = True` to tell PyMC that this should stay fixed in our analysis. Finally, PyMC wants us to collect all the variables of interest and create a `Model` instance out of them. This makes our life easier when we try to retrieve the results.

```
mcmc = mc.MCMC(model)
mcmc.sample( 100000, 50000, 1 )
```

    [****************100%******************]  100000 of 100000 complete

The above code will be explained in the Chapter 2, but this is where our results come from. The machinery being employed is called *Monte Carlo Markov Chains*. It returns random variables from the posterior distributions of $\lambda_1, \lambda_2$ and $\tau$. Be can plot a histogram of the random variables to see what the posterior distribution looks like. Below, we collect the samples (called *traces* in MCMC literature).

```
lambda_1_samples = mcmc.trace( 'lambda_1' )[:]
lambda_2_samples = mcmc.trace( 'lambda_2' )[:]
tau_samples = mcmc.trace( 'tau' )[:]
```

```
figsize(12.5, 10)
#histogram of the samples:

plt.subplot(311)

plt.hist( lambda_1_samples, histtype='stepfilled', bins = 70, alpha = 0.85,
        label = "positerior of $\lambda_1$", color = "#A60628",normed = True )
plt.legend()
plt.title(r"Posterior distributions of the variables $\lambda_1,\;\lambda_2,\;\tau$")

plt.subplot(312)

plt.hist( lambda_2_samples,histtype='stepfilled', bins = 70, alpha = 0.85,
          label = "positerior of $\lambda_2$",color="#7A68A6", normed = True )
plt.legend(loc = "upper left")

plt.subplot(313)

plt.hist( tau_samples, bins = n_count_data, alpha = 0.85,
         label = r"positerior of $\tau$",
         color="#467821", normed = True, histtype='stepfilled' )
plt.legend(loc = "upper left")

```

### Interpretation

Recall that the Bayesian methodology returns a *distribution*, hence we now have distributions to describe the unknown $\lambda$'s and $\tau$. What have we gained? Immediately we can see the uncertainty in our estimates: the more variance in the distribution, the less certain our posterior belief should be. We can also say what a plausible value for the parameters might be. What other observations can you make? Look at the data again, do they seem reasonable? The distributions of the two $\lambda$s look very different, indicating that it's likely there was a change in the user's text-message behavior.

Also notice that posteriors' distributions do not look like any Poisson distributions. They are really not anything we recognize. But this is OK. This is one of the benefits of taking a computational point-of-view. If we had instead done this mathematically, we would have been stuck with a very intractable (and messy) distribution. Via computations, we are agnostic to the tractability.

Our analysis also returned a distribution for what $\tau$ might be. Had no change occurred, or the change been gradual, the posterior distribution of $\tau$ would have been more spread out. On the contrary, it is very peaked. It appears that near day 50, the individual's text-message behavior suddenly changed.

###Why would I want samples from the posterior, anyways?

We will deal with this question for the remainder of the book, and it is an understatement to say we can perform amazingly useful things. For now, let's finishing with using posterior samples to answer the following question: what is the expected number of texts at day $t, \; 0 \le t \le70$? Recall that the expected value of a Poisson is equal to its parameter $\lambda$, then the question is equivalent to *what is the expected value of $\lambda$ at time $t$*?

In the code below, we are calculating the following: Let $i$ index a particular sample from the posterior distributions. Given a day $t$, we average over all $\lambda_i$ on that day $t$, using $\lambda_{1,i}$ if $t \lt \tau_i$ else we use $\lambda_{2,i}$.

___________________

```
figsize( 12.5, 4)
N = tau_samples.shape[0]
expected_texts_per_day = np.zeros(n_count_data)
for  day in range(0, n_count_data):
    ix = day < tau_samples
    expected_texts_per_day[day] = (lambda_1_samples[ix].sum()
                                    + lambda_2_samples[~ix].sum() ) /N

plt.plot( range( n_count_data), expected_texts_per_day, lw =4, color = "#E24A33" )
plt.xlim( 0, n_count_data )
plt.xlabel( "Day" )
plt.ylabel( "Expected # text-messages" )
plt.title( "Expected number of text-messages recieved")
#plt.ylim( 0, 35 )
plt.bar( np.arange( len(count_data) ), count_data, color ="#348ABD", alpha = 0.4,
            label="observed texts per day")

plt.legend(loc="upper left")
```

Our analysis shows strong support that the user's behavior suddenly changed, versus no change ($\lambda_1$ would appear like $\lambda_2$ had this been true), versus a gradual change (more variation in the posterior of $\tau$ had this been true). We can speculate what might have caused this: a cheaper text-message rate, a recent weather-2-text subscription, or a new relationship. (The 45th day corresponds to Christmas, and I moved away to Toronto the next month leaving a lovely girlfriend behind ;)

##### Exercises

1\.  Using `lambda_1_samples` and `lambda_2_samples`, what is the mean of the positior distributions of $\lambda_1$ and $\lambda_2$?

```
#type your code here.
```

2\.  What is the expected percent increase text-message rates? `hint:` compute the mean of `lambda_1_samples/lambda_2_samples`. Note that quanitity is very different from `lambda_1_samples.mean()/lambda_2_samples.mean()`.

```
#type your code here.
```

3\. Looking at the posterior distribution graph of $\tau$, why do you think there is a small number of posterior $\tau$ samples near 0? `hint:` Look at the data again.

4\. What is the mean of $\lambda_1$ **given** $\tau$ is less than 45.

```
#type your code here.
```

### References

-  [1] Gelman, Andrew. N.p.. Web. 22 Jan 2013. <http://andrewgelman.com/2005/07/n_is_never_larg/>.
-  [2] Norvig, Peter. 2009. [*The Unreasonable Effectivness of Data*](http://www.csee.wvu.edu/~gidoretto/courses/2011-fall-cp/reading/TheUnreasonable EffectivenessofData_IEEE_IS2009.pdf).
- [3] Patil, A., D. Huard and C.J. Fonnesbeck. 2010.
PyMC: Bayesian Stochastic Modelling in Python. Journal of Statistical
Software, 35(4), pp. 1-81.

```
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    h1 {
        font-family: "Charis SIL", Palatino, serif;
    }
    div.text_cell_render{
        font-family: Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 145%;
        font-size: 120%;
        width:800px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro", source-code-pro,Consolas, monospace;
    }
    .prompt{
        display: None;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #4057A1;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>

```

```


```python
import tensorflow as tf
from tensorflow.contrib.compiler import xla
tfe = tf.contrib.eager # Shorthand for some symbols
#tf.enable_eager_execution()

# Having trouble getting GPU to be detected properly
config = tf.ConfigProto(
        device_count = {'GPU': 0}
    )
config.graph_options.optimizer_options.global_jit_level = tf.OptimizerOptions.ON_1
sess = tf.Session(config=config)
```

```python
from pycalphad import Database, Model, variables as v
from pycalphad.codegen.sympydiff_utils import build_functions
from sympy import lambdify
import numpy as np

dbf = Database('Al-Cu-Zr_Zhou.tdb')
mod = Model(dbf, ['AL', 'CU', 'ZR'], 'LIQUID')
```

```python
mod.variables
```

    [T, LIQUID0AL, LIQUID0CU, LIQUID0ZR]

```python
cy_func, cy_grad = build_functions(mod.GM, mod.variables, include_grad=True)
%time cy_func.kernel # trigger JIT
%time cy_grad.kernel # trigger JIT
```

    CPU times: user 208 ms, sys: 6.93 ms, total: 215 ms
    Wall time: 1.9 s
    CPU times: user 412 ms, sys: 8.72 ms, total: 421 ms
    Wall time: 2.3 s

    <function wrapper_module_5e8b7be6_0e88_41e7_9740_bbb826dd132a.autofunc_c>

```python
%timeit cy_func.kernel(np.array([[300., 0.3, 0.3, 0.4]]), np.array([0.]))
```

    The slowest run took 15.18 times longer than the fastest. This could mean that an intermediate result is being cached.
    100000 loops, best of 3: 6.7 µs per loop

```python
%timeit cy_grad.kernel(np.array([[300., 0.3, 0.3, 0.4]]), np.array([0.]))
```

    The slowest run took 5.42 times longer than the fastest. This could mean that an intermediate result is being cached.
    100000 loops, best of 3: 9.09 µs per loop

```python
%%timeit
tf_func = lambdify(mod.variables, mod.GM, modules='tensorflow')
```

    10 loops, best of 3: 107 ms per loop

```python
tf_func = lambdify(mod.variables, mod.GM, modules='tensorflow')
```

```python
%%timeit
x1 = tf.placeholder("float", None)
x2 = tf.placeholder("float", None)
x3 = tf.placeholder("float", None)
x4 =  tf.placeholder("float", None)
result = tf_func(x1, x2, x3, x4)
```

    1 loop, best of 3: 375 ms per loop

```python
#x2_vals = np.random.uniform(size=1000)
#x3_vals = 1-x2_vals
x1 = tf.placeholder("float", None)
x2 = tf.placeholder("float", None)
x3 = tf.placeholder("float", None)
x4 = tf.placeholder("float", None)
result = tf_func(x1, x2, x3, x4)
```

```python
grad = tf.gradients(result, [x1, x2, x3, x4], stop_gradients=[x1, x2, x3, x4])
```

```python
%timeit sess.run(result, {x1: 300., x2: 0.3, x3: 0.3, x4: 0.4})
```

    The slowest run took 258.49 times longer than the fastest. This could mean that an intermediate result is being cached.
    1 loop, best of 3: 865 µs per loop

```python
%timeit sess.run(grad, {x1: 300., x2: 0.3, x3: 0.3, x4: 0.4})
```

    The slowest run took 229.71 times longer than the fastest. This could mean that an intermediate result is being cached.
    1 loop, best of 3: 3.28 ms per loop

```python
[tf_xla_func] = xla.compile(tf_func, [x1, x2, x3, x4])
```

```python
%timeit sess.run(tf_xla_func, {x1: 300., x2: 0.3, x3: 0.3, x4: 0.4})
```

    The slowest run took 996.98 times longer than the fastest. This could mean that an intermediate result is being cached.
    1 loop, best of 3: 807 µs per loop

```python

```