# GR HD Equations

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

$\newcommand{\be}{\begin{equation}}$
$\newcommand{\ee}{\end{equation}}$
$\newcommand{\grad}{{\boldsymbol{\nabla}}}$
$\newcommand{\vel}{{\boldsymbol{v}}}$
$\newcommand{\mom}{{\boldsymbol{p}}}$
$\newcommand{\ddt}[1]{{\frac{\partial #1}{\partial t}}}$
$\newcommand{\ddx}[1]{{\frac{\partial #1}{\partial x}}}$
$\newcommand{\state}{{\boldsymbol{\mathcal{U}}}}$
$\newcommand{\charge}{{\boldsymbol{U}}}$
$\newcommand{\psicharge}{{\boldsymbol{\psi}}}$
$\newcommand{\lapse}{\alpha}$
$\newcommand{\shift}{\boldsymbol{\beta}}$
$\newcommand{\rhostar}{{\rho_*}}$
$\newcommand{\tautilde}{{\tilde{\tau}}}$
$\newcommand{\Svectilde}{{\tilde{\boldsymbol{S}}}}$
$\newcommand{\rtgamma}{{\sqrt{\gamma}}}$
$\newcommand{\T}[2]{{T^{#1 #2}}}$
$\newcommand{\uvec}{{\boldsymbol{u}}}$
$\newcommand{\Vvec}{{\boldsymbol{\mathcal{V}}}}$
$\newcommand{\vfluid}{{\boldsymbol{v}_{\rm f}}}$
$\newcommand{\vVal}{{\tilde{\boldsymbol{v}}}}$

$\newcommand{\flux}{{\boldsymbol{\mathcal{F}}}}$
$\newcommand{\fluxV}{{\boldsymbol{F}}}$
$\newcommand{\source}{{\boldsymbol{\mathcal{S}}}}$
$\newcommand{\sourceV}{{\boldsymbol{S}}}$

$\newcommand{\area}{{\boldsymbol{A}}}$
$\newcommand{\normal}{{\hat{\boldsymbol{n}}}}$
$\newcommand{\pt}{{\boldsymbol{p}}}$
$\newcommand{\nb}{{\boldsymbol{n}}}$
$\newcommand{\meshv}{{\boldsymbol{w}}}$
$\newcommand{\facev}{{\boldsymbol{\tilde{w}}_{ij}}}$
$\newcommand{\facer}{{\boldsymbol{\tilde{r}}_{ij}}}$
$\newcommand{\meshr}{{\boldsymbol{r}}}$
$\newcommand{\cmr}{{\boldsymbol{c}}}$

## Introduction:
We start out with the ** GRHD ** equations in conservative form with the state vector $\state=(\rhostar, \Svectilde, \tautilde)$:
\begin{equation}
\ddt{\state} + \grad\cdot\flux = \source,
\end{equation}
where $\rhostar = \lapse\rho\rtgamma u^0$, $\Svectilde = \rhostar h \uvec$, $\tautilde = \lapse^2\rtgamma \T00 - \rhostar$. The associated set of primitive variables are $(\rho, \vel, \epsilon)$, which are the rest mass density, fluid 3-velocity, and internal energy (measured in the rest frame).

The flux, $\flux$ is given by
\begin{equation}
 \flux=(\rhostar \vel, \lapse\rtgamma\T{j}{\beta}g_{\beta i}, \lapse^2\rtgamma\T0j - \rhostar\vel
\end{equation}
where $\vel$ is the 3-velocity, and $\source = (0, \frac 1 2 \lapse\rtgamma \T{\lapse}{\beta}g_{\lapse\beta,i}, s)$ is the source function, and
\begin{equation}
s = \lapse\rtgamma\left[\left(\T00\beta^i\beta^j + 2\T0i\beta^j\right)K_{ij} - \left(\T00\beta^i + \T0i\right)\partial_i\lapse\right]
\end{equation}
The stress energy tensor for a perfect fluid is written as
\begin{equation}
\T{\mu}{\nu} = \rho h u^{\mu} u^{\nu} + P g^{\mu\nu},
\end{equation}
where $h = 1 + \epsilon + P/\rho$ is the specific enthalpy and $u^{\mu}$ are the respective components of the four velocity.

Noting that the mass $\flux$ is defined in terms of $\rhostar$ and $\vel$, we need to first find a mapping between $\vel$ and $u$.

### Alternative formulation

The Athena++ folks have an alternative formulations that might be superior.
Begin with the continuity equation
\begin{equation}
\grad_{\mu}\rho u^{\mu} = 0,
\end{equation}
where $\grad$ is the covariant derivative.  This can be mapped directly to
\begin{equation}
\partial_{0} \sqrt{-g}\rho u^0 + \partial_i\sqrt{-g} \rho u^0 v^i = 0
\end{equation}
which we can identify with $\rhostar = \alpha\rtgamma \rho u^0$ because $\sqrt{-g} = \alpha\rtgamma$.

Now the second equation is conservation of energy-momentum which we write as
\begin{equation}
\grad_{\nu}T^{\nu}_{\mu} = 0
\end{equation}
writing this out we have
\begin{equation}
\partial_0 g_{\mu\alpha}T^{\alpha 0} + \partial_i g_{\mu\alpha}T^{\alpha i} - \Gamma_{\mu\alpha}^{\gamma} g_{\gamma\beta}T^{\alpha\beta} = 0
\end{equation}
Noting that
\begin{equation}
\Gamma^{\alpha}_{\beta\gamma} = \frac 1 2 g^{\alpha\delta}\left(\partial_{\gamma}g_{\beta\delta} + \partial_{\beta}g_{\gamma\delta} - \partial_{\delta}g_{\beta\gamma}\right)
\end{equation}
Writing this all out, we note the last term is
\begin{equation}
\Gamma_{\mu\alpha}^{\gamma} g_{\gamma\beta}T^{\alpha\beta} =
\frac 1 2 g^{\gamma\delta}\left(\partial_{\alpha}g_{\mu\delta} + \partial_{\mu}g_{\alpha \delta} - \partial_{\delta}g_{\mu \alpha}\right) T_{\gamma}^{\alpha} =
\frac 1 2 \left(\partial_{\alpha}g_{\mu\delta} + \partial_{\mu}g_{\alpha \delta} - \partial_{\delta}g_{\mu \alpha}\right)
T^{\alpha\delta}
\end{equation}
We sum over $\alpha$ and $\delta$, but noting that we are antisymmetric in first and last terms in $\alpha$ and $\delta$ in the () but symmetric in $T_{\alpha\delta}$ so we have
\begin{equation}
\Gamma_{\mu\alpha}^{\gamma} g_{\gamma\beta}T^{\alpha\beta} = \frac 1 2 \partial_{\mu}g_{\alpha \delta} T^{\alpha\delta}
\end{equation}

Thus we have
\begin{equation}
\partial_0 T^{0}_{\mu} + \partial_i T^{i}_{\mu} = \frac 1 2 \partial_{\mu}g_{\alpha \delta} T^{\alpha\delta}
\end{equation}
The $\mu = (1,2,3)$, we almost get back the equations in the standard formulation
\begin{equation}
\partial_0 \rho h u^0 u_i + \partial_j T^j_i = \frac 1 2 \partial_{i}g_{\alpha \delta} T^{\alpha\delta},
\end{equation}
which modulo a factors of $\lapse\rtgamma$ in front is the same as the "standard" equations.

The $T^0_0$ term is more interesting. Here we have
\begin{equation}
\partial_0 (\rho h u^0 u_0 + + \partial_j T^j_i = \frac 1 2 \partial_{0}g_{\alpha \delta} T^{\alpha\delta},
\end{equation}

However the disadvantage is that we need the time derivative of the metric.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#mapping): Primitive to Conservative Mapping
1. [Step 2](#zach): Compute $u^0$ from the Valencia 3-velocity (Zach step)
1. [Step 3](#flux): Compute the flux
1. [Step 4](#source): Source Terms
1. [Step 5](#rotation): Rotation
1. [Step 6](#solver): Conservative to Primitive Solver
1. [Step 7](#lorentz): Lorentz Boosts
1. [Step 8](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Primitive to Conservative Mapping
$$\label{mapping}$$

We want to make a mapping from the primitives to conserved variables:
\begin{equation}
(\rho, \vel, \epsilon) \rightarrow (\rhostar = \lapse\rho\rtgamma u^0, \Svectilde = \rhostar h \uvec, \tautilde = \lapse^2\rtgamma \T00 - \rhostar).
\end{equation}
To do so, we first need to determine $u^0$ and $\uvec$. Noting that $g_{\mu\nu} u^{\mu} u^{\nu} = -1$, we find
\begin{equation}
u^0 = \left(-g_{00} - 2g_{i0} v^i - g_{ij}v^iv^j\right)^{-1/2},
\end{equation}
where we have used $\vel = \uvec/u^0$.  This gives me $\rhostar$ and $\uvec$.  We note that the metric is (B&S 2.122)
\begin{equation}
g_{\mu\nu} = \begin{pmatrix}
-\lapse^2 + \shift\cdot\shift & \beta_i \\
\beta_j & \gamma_{ij}
\end{pmatrix},
\end{equation}
Lets write some code to define metric contraction on four vectors in this context:

```python
import NRPy_param_funcs as par
import indexedexp as ixp
import sympy as sp

from outputC import *
import NRPy_param_funcs as par

DIM = 3
# Declare rank-2 covariant gmunu

gmunuDD = ixp.declarerank2("gmunuDD","sym01",DIM=4)

gammaDD = ixp.declarerank2("gammaDD","sym01")
components = ["xx", "xy", "xz", "yy", "yz", "zz"]
names = ""
for comp in components :
    names = names + "mi.gamDD{0} ".format(comp)

gxx, gxy, gxz, gyy, gyz, gzz = sp.symbols( names)

gammaDD[0][0] = gxx
gammaDD[0][1] = gxy
gammaDD[0][2] = gxz
gammaDD[1][0] = gxy
gammaDD[1][1] = gyy
gammaDD[1][2] = gyz
gammaDD[2][0] = gxz
gammaDD[2][1] = gyz
gammaDD[2][2] = gzz

lapse, rtgamma, beta_x, beta_y, beta_z = sp.symbols( "mi.alpha mi.rtDetGamma mi.betaX mi.betaY mi.betaZ")
u10, u1x, u1y, u1z = sp.symbols("u1[0] u1[1] u1[2] u1[3]")
u20, u2x, u2y, u2z = sp.symbols("u2[0] u2[1] u2[2] u2[3]")

u1U = ixp.declarerank1("u1Vector", DIM=4)
u2U = ixp.declarerank1("u2Vector", DIM=4)

u1U[0] = u10
u1U[1] = u1x
u1U[2] = u1y
u1U[3] = u1z

u2U[0] = u20
u2U[1] = u2x
u2U[2] = u2y
u2U[3] = u2z

shiftU = ixp.declarerank1("shiftU")
shiftU[0] = beta_x
shiftU[1] = beta_y
shiftU[2] = beta_z

beta2 = 0
for i in range(DIM) :
    for j in range(DIM) :
        beta2 += gammaDD[i][j] * shiftU[i]*shiftU[j]

gmunuDD[0][0] = -lapse*lapse + beta2
for i in range(DIM) :
    gmunuDD[i+1][0] = shiftU[i]
    gmunuDD[0][i+1] = shiftU[i]
    for j in range(DIM) :
        gmunuDD[i+1][j+1] = gammaDD[i][j]

dot4Product = 0
for i in range(4):
    for j in range(4):
        dot4Product += gmunuDD[i][j]*u1U[i]*u2U[j]

str = outputC( dot4Product, "dotProduct", filename="returnstring")
print(str)
```

    /*
     *  Original SymPy expression:
     *  "dotProduct = mi.betaX*u1[0]*u2[1] + mi.betaX*u1[1]*u2[0] + mi.betaY*u1[0]*u2[2] + mi.betaY*u1[2]*u2[0] + mi.betaZ*u1[0]*u2[3] + mi.betaZ*u1[3]*u2[0] + mi.gamDDxx*u1[1]*u2[1] + mi.gamDDxy*u1[1]*u2[2] + mi.gamDDxy*u1[2]*u2[1] + mi.gamDDxz*u1[1]*u2[3] + mi.gamDDxz*u1[3]*u2[1] + mi.gamDDyy*u1[2]*u2[2] + mi.gamDDyz*u1[2]*u2[3] + mi.gamDDyz*u1[3]*u2[2] + mi.gamDDzz*u1[3]*u2[3] + u1[0]*u2[0]*(-mi.alpha**2 + mi.betaX**2*mi.gamDDxx + 2*mi.betaX*mi.betaY*mi.gamDDxy + 2*mi.betaX*mi.betaZ*mi.gamDDxz + mi.betaY**2*mi.gamDDyy + 2*mi.betaY*mi.betaZ*mi.gamDDyz + mi.betaZ**2*mi.gamDDzz)"
     */
    {
       const double tmp0 = 2*mi.betaX;
       dotProduct = mi.betaX*u1[0]*u2[1] + mi.betaX*u1[1]*u2[0] + mi.betaY*u1[0]*u2[2] + mi.betaY*u1[2]*u2[0] + mi.betaZ*u1[0]*u2[3] + mi.betaZ*u1[3]*u2[0] + mi.gamDDxx*u1[1]*u2[1] + mi.gamDDxy*u1[1]*u2[2] + mi.gamDDxy*u1[2]*u2[1] + mi.gamDDxz*u1[1]*u2[3] + mi.gamDDxz*u1[3]*u2[1] + mi.gamDDyy*u1[2]*u2[2] + mi.gamDDyz*u1[2]*u2[3] + mi.gamDDyz*u1[3]*u2[2] + mi.gamDDzz*u1[3]*u2[3] + u1[0]*u2[0]*(-pow(mi.alpha, 2) + pow(mi.betaX, 2)*mi.gamDDxx + pow(mi.betaY, 2)*mi.gamDDyy + 2*mi.betaY*mi.betaZ*mi.gamDDyz + mi.betaY*mi.gamDDxy*tmp0 + pow(mi.betaZ, 2)*mi.gamDDzz + mi.betaZ*mi.gamDDxz*tmp0);
    }

which then gives
\begin{equation}
u^0 = \left(\lapse^2 - \shift\cdot\shift - 2\shift\cdot\vel - \gamma_{ij}v^iv^j\right)^{-1/2},
\end{equation}

The other thing is $\uvec = u^0\vel$.  So then we can proceed and spit out the conservative variables: $\rhostar, \Svectilde$.

To get $\tau$, we note that we have defined the metric as the covariant form, e.g., lower indices.  The upper form of $g^{\mu\nu}$ is found in [B&S]() 2.119 and is given by
\begin{equation}
g^{\mu\nu} =
\begin{pmatrix}
-\lapse^{-2} & \lapse^{-2}\beta^i \\
\lapse^{-2}\beta^j & \gamma^{ij} - \lapse^{-2} \beta^i\beta^j
\end{pmatrix}
\end{equation}
Lets get the form of this in code:

The main challenge is the calculation of the inverse of the 3x3 matrix $\gamma_{ij}$.  To do so we note:

```python
import indexedexp as ixp
gammaUU, gammabarDet = ixp.symm_matrix_inverter3x3(gammaDD)
gUUxx = gammaUU[0][0]
gUUxy = gammaUU[0][1]
gUUxz = gammaUU[0][2]
gUUyy = gammaUU[1][1]
gUUyz = gammaUU[1][2]
gUUzz = gammaUU[2][2]
rtDetGamma = sp.sqrt(gammabarDet)

outputC( [gUUxx,gUUxy,gUUxz,gUUyy,gUUyz,gUUzz, rtDetGamma], ["mi.gamUUxx", "mi.gamUUxy","mi.gamUUxz","mi.gamUUyy","mi.gamUUyz","mi.gamUUzz","mi.rtDetGamma"], filename="NRPY+gmunuUU_and_det.h")
#print str

```

    Wrote to file "NRPY+gmunuUU_and_det.h"

# Step 2: Compute $u^0$ from the Valencia 3-velocity (Zach step)
$$\label{zach}$$

According to Eqs. 9-11 of [the IllinoisGRMHD paper](https://arxiv.org/pdf/1501.07276.pdf), the Valencia 3-velocity $v^i_{(n)}$ is related to the 4-velocity $u^\mu$ via

\begin{align}
\alpha v^i_{(n)} &= \frac{u^i}{u^0} + \beta^i \\
\implies u^i &= u^0 \left(\alpha v^i_{(n)} - \beta^i\right)
\end{align}

Defining $v^i = \frac{u^i}{u^0}$, we get

$$v^i = \alpha v^i_{(n)} - \beta^i,$$

Or in other words in terms of the 3 velocity

$$v^i_{(n)} = \alpha^{-1}\left(v^i + \beta^i\right)$$

and in terms of this variable we get

\begin{align}
g_{00} \left(u^0\right)^2 + 2 g_{0i} u^0 u^i + g_{ij} u^i u^j &= \left(u^0\right)^2 \left(g_{00} + 2 g_{0i} v^i + g_{ij} v^i v^j\right)\\
\implies u^0 &= \pm \sqrt{\frac{-1}{g_{00} + 2 g_{0i} v^i + g_{ij} v^i v^j}} \\
&= \pm \sqrt{\frac{-1}{(-\alpha^2 + \beta^2) + 2 \beta_i v^i + \gamma_{ij} v^i v^j}} \\
&= \pm \sqrt{\frac{1}{\alpha^2 - \gamma_{ij}\left(\beta^i + v^i\right)\left(\beta^j + v^j\right)}}\\
&= \pm \sqrt{\frac{1}{\alpha^2 - \alpha^2 \gamma_{ij}v^i_{(n)}v^j_{(n)}}}\\
&= \pm \frac{1}{\alpha}\sqrt{\frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}}}
\end{align}

Generally speaking, numerical errors will occasionally drive expressions under the radical to either negative values or potentially enormous values (corresponding to enormous Lorentz factors). Thus a reliable approach for computing $u^0$ requires that we first rewrite the above expression in terms of the Lorentz factor squared: $\Gamma^2=\left(\alpha u^0\right)^2$:
\begin{align}
u^0 &= \pm \frac{1}{\alpha}\sqrt{\frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}}}\\
\implies \left(\alpha u^0\right)^2 &= \frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}} \\
\implies \gamma_{ij}v^i_{(n)}v^j_{(n)} &= 1 - \frac{1}{\left(\alpha u^0\right)^2}
\end{align}
In order for the bottom expression to hold true, the left-hand side must be between 0 and 1. Again, this is not guaranteed due to the appearance of numerical errors. In fact, a robust algorithm will not allow $\Gamma^2$ to become too large (which might contribute greatly to the stress-energy of a given gridpoint), so let's define $\Gamma_{\rm max}$, the largest allowed Lorentz factor.

Then our algorithm for computing $u^0$ is as follows:

If
$$R=\gamma_{ij}v^i_{(n)}v^j_{(n)}>1 - \frac{1}{\Gamma_{\rm max}},$$
then adjust the 3-velocity $v^i$ as follows:

$$v^i_{(n)} = \sqrt{\frac{1 - \frac{1}{\Gamma_{\rm max}}}{R}}v^i_{(n)}.$$

After this rescaling, we are then guaranteed that if $R$ is recomputed, it will be set to its ceiling value $R=1 - \frac{1}{\Gamma_{\rm max}}$.

Then $u^0$ can be safely computed via

$$
u^0 = \frac{1}{\alpha \sqrt{1-R}}.
$$

```python
import sympy as sp
import NRPy_param_funcs as par
import grid as gri
import indexedexp as ixp
import reference_metric as rfm
from outputC import *

vx, vy, vz = sp.symbols( "vx vy vz")
vU = ixp.declarerank1("vU")
vU[0] = vx
vU[1] = vy
vU[2] = vz

ValenciavU = ixp.register_gridfunctions_for_single_rank1("AUX","ValenciavU",DIM=3)
ValenciavU[0] = (vx + beta_x)/lapse
ValenciavU[1] = (vy + beta_y)/lapse
ValenciavU[2] = (vz + beta_z)/lapse

# Step 1: Compute R = 1 - 1/max(Gamma)
R = sp.sympify(0)
for i in range(DIM):
    for j in range(DIM):
        R += gammaDD[i][j]*ValenciavU[i]*ValenciavU[j]

GAMMA_SPEED_LIMIT = par.Cparameters("REAL","GRMHD_equations","GAMMA_SPEED_LIMIT", 10.0) # 10.0 is default for IllinoisGRMHD
Rmax = 1 - 1/(GAMMA_SPEED_LIMIT*GAMMA_SPEED_LIMIT)

rescaledValenciavU = ixp.zerorank1()
for i in range(DIM):
    rescaledValenciavU[i] = ValenciavU[i]*sp.sqrt(Rmax/R)

rescaledu0 = 1/(lapse*sp.sqrt(1-Rmax))
regularu0 =  1/(lapse*sp.sqrt(1-R))

computeu0_Cfunction  = "/* Function for computing u^0 from Valencia 3-velocity. */\n"
computeu0_Cfunction += "/* Inputs: vx, vy, vz, lapse, MetricInformation, GAMMA_SPEED_LIMIT (C parameter) */\n"
computeu0_Cfunction += "/* Output: u0=u^0  */\n\n"

computeu0_Cfunction += outputC([R,Rmax],["const double R","const double Rmax"],"returnstring",
                               params="includebraces=False,CSE_varprefix=tmpR,outCverbose=False")

computeu0_Cfunction += "if(R <= Rmax) "
computeu0_Cfunction += outputC(regularu0,"u0","returnstring",
                               params="includebraces=True,CSE_varprefix=tmpnorescale,outCverbose=False")
computeu0_Cfunction += " else "
computeu0_Cfunction += outputC([rescaledu0],
                               ["u0"],"returnstring",
                               params="includebraces=True,CSE_varprefix=tmprescale,outCverbose=False")

print(computeu0_Cfunction)
```

    /* Function for computing u^0 from Valencia 3-velocity. */
    /* Inputs: vx, vy, vz, lapse, MetricInformation, GAMMA_SPEED_LIMIT (C parameter) */
    /* Output: u0=u^0  */

    const double tmpR0 = pow(mi.alpha, -2);
    const double tmpR1 = mi.betaX + vx;
    const double tmpR2 = mi.betaY + vy;
    const double tmpR3 = mi.betaZ + vz;
    const double tmpR4 = 2*tmpR0*tmpR1;
    const double R = mi.gamDDxx*tmpR0*pow(tmpR1, 2) + mi.gamDDxy*tmpR2*tmpR4 + mi.gamDDxz*tmpR3*tmpR4 + mi.gamDDyy*tmpR0*pow(tmpR2, 2) + 2*mi.gamDDyz*tmpR0*tmpR2*tmpR3 + mi.gamDDzz*tmpR0*pow(tmpR3, 2);
    const double Rmax = 1 - 1/pow(GAMMA_SPEED_LIMIT, 2);
    if(R <= Rmax) {
       const double tmpnorescale0 = pow(mi.alpha, -2);
       const double tmpnorescale1 = mi.betaX + vx;
       const double tmpnorescale2 = mi.betaY + vy;
       const double tmpnorescale3 = mi.betaZ + vz;
       const double tmpnorescale4 = 2*tmpnorescale0*tmpnorescale1;
       u0 = 1/(mi.alpha*sqrt(-mi.gamDDxx*tmpnorescale0*pow(tmpnorescale1, 2) - mi.gamDDxy*tmpnorescale2*tmpnorescale4 - mi.gamDDxz*tmpnorescale3*tmpnorescale4 - mi.gamDDyy*tmpnorescale0*pow(tmpnorescale2, 2) - 2*mi.gamDDyz*tmpnorescale0*tmpnorescale2*tmpnorescale3 - mi.gamDDzz*tmpnorescale0*pow(tmpnorescale3, 2) + 1));
    }
     else {
       u0 = fabs(GAMMA_SPEED_LIMIT)/mi.alpha;
    }

We now note that $\tau = \lapse^2\rtgamma T^{00} - \rhostar$, which gives
\begin{equation}
\tau = \lapse\rhostar h u^0 - P\rtgamma - \rhostar
\end{equation}

The code for this is

```python
rho, epsilon, gamma1, p = sp.symbols("rho ie gamma p")

betaDotV = 0
for i in range(DIM) :
    for j in range(DIM) :
        betaDotV += gammaDD[i][j] * shiftU[i]*vU[j]

v2 = 0
for i in range(DIM) :
    for j in range(DIM) :
         v2 += gammaDD[i][j] * vU[i]*vU[j]

u0 = sp.symbols("u0")

uvec4U = ixp.zerorank1(DIM=4)
uvec4D = ixp.zerorank1(DIM=4)
#StildeU = ixp.declarerank1("StildeU")
StildeD = ixp.zerorank1()
rhostar = lapse*rtgamma*rho*u0
h = 1. + epsilon + p/rho

for i in range(1,4):
    uvec4U[i] = vU[i-1]*u0
uvec4U[0] = u0

for mu in range(4) :
    for nu in range(4) :
        uvec4D[mu] += gmunuDD[mu][nu]*uvec4U[nu]

for i in range(DIM):
    StildeD[i] = uvec4D[i+1]*rhostar*h

tau = lapse*rhostar*h*u0 - rtgamma*p - rhostar

cFunction = "double u0 = 0.;\n" + computeu0_Cfunction
str = outputC([rhostar, StildeD[0], StildeD[1], StildeD[2], tau], ["con[iRhoStar]", "con[iSx]", "con[iSy]", "con[iSz]", "con[iTau]"], filename="returnstring")
print(str)
f = open("NRPY+prim2Con.h", "w")
f.write( cFunction + str)
f.close()
```

    /*
     *  Original SymPy expressions:
     *  "[con[iRhoStar] = mi.alpha*mi.rtDetGamma*rho*u0,
     *    con[iSx] = mi.alpha*mi.rtDetGamma*rho*u0*(ie + p/rho + 1.0)*(mi.betaX*u0 + mi.gamDDxx*u0*vx + mi.gamDDxy*u0*vy + mi.gamDDxz*u0*vz),
     *    con[iSy] = mi.alpha*mi.rtDetGamma*rho*u0*(ie + p/rho + 1.0)*(mi.betaY*u0 + mi.gamDDxy*u0*vx + mi.gamDDyy*u0*vy + mi.gamDDyz*u0*vz),
     *    con[iSz] = mi.alpha*mi.rtDetGamma*rho*u0*(ie + p/rho + 1.0)*(mi.betaZ*u0 + mi.gamDDxz*u0*vx + mi.gamDDyz*u0*vy + mi.gamDDzz*u0*vz),
     *    con[iTau] = mi.alpha**2*mi.rtDetGamma*rho*u0**2*(ie + p/rho + 1.0) - mi.alpha*mi.rtDetGamma*rho*u0 - mi.rtDetGamma*p]"
     */
    {
       const double tmp0 = mi.rtDetGamma*rho;
       const double tmp1 = mi.alpha*tmp0*u0;
       const double tmp2 = u0*vx;
       const double tmp3 = u0*vy;
       const double tmp4 = u0*vz;
       const double tmp5 = ie + p/rho + 1.0;
       const double tmp6 = tmp1*tmp5;
       con[iRhoStar] = tmp1;
       con[iSx] = tmp6*(mi.betaX*u0 + mi.gamDDxx*tmp2 + mi.gamDDxy*tmp3 + mi.gamDDxz*tmp4);
       con[iSy] = tmp6*(mi.betaY*u0 + mi.gamDDxy*tmp2 + mi.gamDDyy*tmp3 + mi.gamDDyz*tmp4);
       con[iSz] = tmp6*(mi.betaZ*u0 + mi.gamDDxz*tmp2 + mi.gamDDyz*tmp3 + mi.gamDDzz*tmp4);
       con[iTau] = pow(mi.alpha, 2)*tmp0*tmp5*pow(u0, 2) - mi.rtDetGamma*p - tmp1;
    }

# Step 3: Compute the flux
$$\label{flux}$$

The fluxes are as follows
\begin{equation}
\frac{\partial}{\partial t}
\begin{pmatrix}
\rhostar\\
\Svectilde\\
\tautilde
\end{pmatrix} + \frac{\partial}{\partial x^j}\begin{pmatrix} \rhostar v^j\\
\lapse\rtgamma T^j_i\\ \lapse^2\rtgamma T^{0j} - \rhostar v^j
\end{pmatrix}  = \begin{pmatrix} 0 \\ \frac 1 2 \lapse\rtgamma T^{\alpha\beta}g_{\alpha\beta,i} \\ s \end{pmatrix}
\end{equation}
so the flux is
\begin{equation}
\mathcal{F} = \begin{pmatrix} \rhostar v^i \\ \lapse\rtgamma T^i_k \\ \lapse^2\rtgamma T^{0i} - \rhostar v^i
\end{pmatrix}
\end{equation}
In the moving-mesh formalism, the flux is just taken along the x directions so we have
\begin{equation}
\mathcal{F} = \begin{pmatrix} \rhostar v^1 \\ \lapse\rtgamma T^1_k \\ \lapse^2\rtgamma T^{01} - \rhostar v^1
\end{pmatrix}
\end{equation}
Note that we will need to rotate $T^{\mu\nu}$ and $g_{\mu\nu}$ to get the right orientation.
In order to do this, we must first compute the stress energy tensor:
\begin{equation}
T^{\mu\nu} = \rho h u^{\mu}u^{\nu} + Pg^{\mu\nu} = \rho h (u^0)^2v^iv^j + P g^{\mu\nu}
\end{equation}

```python
TmunuUU = ixp.declarerank2("TmunuUU","sym01",DIM=4)
uvecU = ixp.zerorank1()
for i in range(3) :
    uvecU[i] = uvec4U[i+1]

TmunuUU[0][0] = rho*h*u0*u0 - p/(lapse*lapse) #is this \pm?
for i in range(3):
    TmunuUU[0][i+1] = rho*h*u0*uvecU[i] + p/(lapse*lapse)*shiftU[i]
    TmunuUU[i+1][0] = rho*h*u0*uvecU[i] + p/(lapse*lapse)*shiftU[i]

for i in range(3):
    for j in range(3):
        TmunuUU[i+1][j+1] = rho*h*uvecU[i]*uvecU[j] + p*(gammaUU[i][j] - 1./(lapse*lapse)*shiftU[i]*shiftU[j])

#str = outputC([TmunuUU[1][0], TmunuUU[1][1], TmunuUU[1][2], TmunuUU[1][3]], ["Tmunu10", "Tmunu11", "Tmunu12", "Tmunu13"], filename="returnstring")

#print(str)
#str = outputC([gmunuDD[1][0], gmunuDD[1][1], gmunuDD[1][2], gmunuDD[1][3]], ["gmunu10", "gmunu11", "gmunu12", "gmunu13"], filename="returnstring")

#print(str)

#calculate Tmunu^1_i
Tmunu1D = ixp.zerorank1()

for i in range(3):
    for j in range(0,4) :
        Tmunu1D[i] += gmunuDD[i+1][j] * TmunuUU[1][j]

#str = outputC([Tmunu1D[0], Tmunu1D[1], Tmunu1D[2]], ["Tmunu1Dx", "Tmunu1Dy", "Tmunu1Dz"], filename="returnstring")
#print str

# now get the flux
fluxRho, fluxMomX, fluxMomY, fluxMomZ, fluxEnergy = sp.symbols("flux[iRhoStar] flux[iSx] flux[iSy] flux[iSz] flux[iTau]")
fluxRho = rhostar * vU[0]
fluxMomX = lapse*rtgamma*Tmunu1D[0]
fluxMomY = lapse*rtgamma*Tmunu1D[1]
fluxMomZ = lapse*rtgamma*Tmunu1D[2]
fluxEnergy = lapse*lapse*rtgamma*TmunuUU[0][1] - rhostar*vU[0]

cFunction = "double u0 = 0.;\n" + computeu0_Cfunction

str = outputC([fluxRho, fluxMomX, fluxMomY, fluxMomZ, fluxEnergy], ["flux[iRhoStar]", "flux[iSx]", "flux[iSy]", "flux[iSz]", "flux[iTau]"], filename="returnstring")
print(str)

f = open("NRPY+calFlux.h", "w")
f.write( cFunction + str)
f.close()

```

    /*
     *  Original SymPy expressions:
     *  "[flux[iRhoStar] = mi.alpha*mi.rtDetGamma*rho*u0*vx,
     *    flux[iSx] = mi.alpha*mi.rtDetGamma*(mi.betaX*(rho*u0**2*vx*(ie + p/rho + 1.0) + mi.betaX*p/mi.alpha**2) + mi.gamDDxx*(p*((mi.gamDDyy*mi.gamDDzz - mi.gamDDyz**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX**2/mi.alpha**2) + rho*u0**2*vx**2*(ie + p/rho + 1.0)) + mi.gamDDxy*(p*((-mi.gamDDxy*mi.gamDDzz + mi.gamDDxz*mi.gamDDyz)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaY/mi.alpha**2) + rho*u0**2*vx*vy*(ie + p/rho + 1.0)) + mi.gamDDxz*(p*((mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaZ/mi.alpha**2) + rho*u0**2*vx*vz*(ie + p/rho + 1.0))),
     *    flux[iSy] = mi.alpha*mi.rtDetGamma*(mi.betaY*(rho*u0**2*vx*(ie + p/rho + 1.0) + mi.betaX*p/mi.alpha**2) + mi.gamDDxy*(p*((mi.gamDDyy*mi.gamDDzz - mi.gamDDyz**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX**2/mi.alpha**2) + rho*u0**2*vx**2*(ie + p/rho + 1.0)) + mi.gamDDyy*(p*((-mi.gamDDxy*mi.gamDDzz + mi.gamDDxz*mi.gamDDyz)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaY/mi.alpha**2) + rho*u0**2*vx*vy*(ie + p/rho + 1.0)) + mi.gamDDyz*(p*((mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaZ/mi.alpha**2) + rho*u0**2*vx*vz*(ie + p/rho + 1.0))),
     *    flux[iSz] = mi.alpha*mi.rtDetGamma*(mi.betaZ*(rho*u0**2*vx*(ie + p/rho + 1.0) + mi.betaX*p/mi.alpha**2) + mi.gamDDxz*(p*((mi.gamDDyy*mi.gamDDzz - mi.gamDDyz**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX**2/mi.alpha**2) + rho*u0**2*vx**2*(ie + p/rho + 1.0)) + mi.gamDDyz*(p*((-mi.gamDDxy*mi.gamDDzz + mi.gamDDxz*mi.gamDDyz)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaY/mi.alpha**2) + rho*u0**2*vx*vy*(ie + p/rho + 1.0)) + mi.gamDDzz*(p*((mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaZ/mi.alpha**2) + rho*u0**2*vx*vz*(ie + p/rho + 1.0))),
     *    flux[iTau] = mi.alpha**2*mi.rtDetGamma*(rho*u0**2*vx*(ie + p/rho + 1.0) + mi.betaX*p/mi.alpha**2) - mi.alpha*mi.rtDetGamma*rho*u0*vx]"
     */
    {
       const double tmp0 = mi.alpha*mi.rtDetGamma;
       const double tmp1 = rho*vx;
       const double tmp2 = tmp0*tmp1*u0;
       const double tmp3 = pow(mi.alpha, 2);
       const double tmp4 = 1.0/tmp3;
       const double tmp5 = mi.betaX*tmp4;
       const double tmp6 = pow(u0, 2)*(ie + p/rho + 1.0);
       const double tmp7 = tmp1*tmp6;
       const double tmp8 = p*tmp5 + tmp7;
       const double tmp9 = 1.0*tmp5;
       const double tmp10 = mi.gamDDxz*mi.gamDDyz;
       const double tmp11 = mi.gamDDyy*mi.gamDDzz;
       const double tmp12 = pow(mi.gamDDyz, 2);
       const double tmp13 = 1.0/(mi.gamDDxx*tmp11 - mi.gamDDxx*tmp12 - pow(mi.gamDDxy, 2)*mi.gamDDzz + 2*mi.gamDDxy*tmp10 - pow(mi.gamDDxz, 2)*mi.gamDDyy);
       const double tmp14 = p*(-mi.betaY*tmp9 + tmp13*(-mi.gamDDxy*mi.gamDDzz + tmp10)) + tmp7*vy;
       const double tmp15 = p*(-mi.betaZ*tmp9 + tmp13*(mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)) + tmp7*vz;
       const double tmp16 = p*(-1.0*pow(mi.betaX, 2)*tmp4 + tmp13*(tmp11 - tmp12)) + rho*tmp6*pow(vx, 2);
       flux[iRhoStar] = tmp2;
       flux[iSx] = tmp0*(mi.betaX*tmp8 + mi.gamDDxx*tmp16 + mi.gamDDxy*tmp14 + mi.gamDDxz*tmp15);
       flux[iSy] = tmp0*(mi.betaY*tmp8 + mi.gamDDxy*tmp16 + mi.gamDDyy*tmp14 + mi.gamDDyz*tmp15);
       flux[iSz] = tmp0*(mi.betaZ*tmp8 + mi.gamDDxz*tmp16 + mi.gamDDyz*tmp14 + mi.gamDDzz*tmp15);
       flux[iTau] = mi.rtDetGamma*tmp3*tmp8 - tmp2;
    }

# Step 4: Source Terms
$$\label{source}$$

The sources terms are for mass, momentum and energy are:
\begin{equation}
\source = (0, \frac 1 2 \lapse\rtgamma \T{\alpha}{\beta}g_{\alpha\beta,i}, s),
\end{equation}
For a time stationary metric $s\neq 0$, so we will ignore this until the next section.  As for the rest, we need to define derivatives of the metric.  Suppose I have done this already.  Then the code for the source terms is:

```python
gmunuDDind = [0,0]
gammaDDind = [0,0]
lapseind = [0,0]
alpha = [0.,0.]
h = sp.symbols( "h")

for ind in range(2) :
    gmunuDDind[ind] = ixp.zerorank2(DIM=4) #derivative of gmunu in some direction
    gammaDDind[ind] = ixp.zerorank2()

    components = ["xx", "xy", "xz", "yy", "yz", "zz"]
    names = ""
    for comp in components :
        names = names + "mi{1}.gamDD{0} ".format(comp, ind+1)

    gxx, gxy, gxz, gyy, gyz, gzz = sp.symbols( names)

    gammaDDind[ind][0][0] = gxx
    gammaDDind[ind][0][1] = gxy
    gammaDDind[ind][0][2] = gxz
    gammaDDind[ind][1][0] = gxy
    gammaDDind[ind][1][1] = gyy
    gammaDDind[ind][1][2] = gyz
    gammaDDind[ind][2][0] = gxz
    gammaDDind[ind][2][1] = gyz
    gammaDDind[ind][2][2] = gzz

    lapse, rtgamma, beta_x, beta_y, beta_z = sp.symbols( "mi{0}.alpha mi{0}.rtDetGamma mi{0}.betaX mi{0}.betaY mi{0}.betaZ".format(ind+1))
    u10, u1x, u1y, u1z = sp.symbols("u1[0] u1[1] u1[2] u1[3]")
    u20, u2x, u2y, u2z = sp.symbols("u2[0] u2[1] u2[2] u2[3]")

    lapseind[ind] = lapse

    shiftU = ixp.zerorank1()
    shiftU[0] = beta_x
    shiftU[1] = beta_y
    shiftU[2] = beta_z

    beta2 = 0
    for i in range(DIM) :
        for j in range(DIM) :
            beta2 += gammaDDind[ind][i][j] * shiftU[i]*shiftU[j]

    gmunuDDind[ind][0][0] = -lapse*lapse + beta2
    for i in range(DIM) :
        gmunuDDind[ind][i+1][0] = shiftU[i]
        gmunuDDind[ind][0][i+1] = shiftU[i]
        for j in range(DIM) :
            gmunuDDind[ind][i+1][j+1] = gammaDDind[ind][i][j]

dgmunuDD = ixp.zerorank2(DIM=4)
source = 0
for mu in range(4) :
    for nu in range(4) :
        dgmunuDD[mu][nu] = (gmunuDDind[1][mu][nu] - gmunuDDind[0][mu][nu])/(2*h) # /(2*h) = 1.0/(2.0*h); for the finite differene
        source = source + TmunuUU[mu][nu]*dgmunuDD[mu][nu]

# precalculate dalpha/di for next calculation
dalpha = (lapseind[1] - lapseind[0])/(2*h)
#print TmunuUU[2][1]
cFunction = "double u0 = 0.;\n" + computeu0_Cfunction
str = outputC( [source, dalpha, gmunuDDind[1][0][0]], ["source", "dlapse", "dgmunu"], filename="returnstring")
print(str)

f = open("NRPY+calMomSources.h", "w")
f.write( cFunction + str)
f.close()
```

    /*
     *  Original SymPy expressions:
     *  "[source = (-mi1.betaX + mi2.betaX)*(rho*u0**2*vx*(ie + p/rho + 1.0) + mi.betaX*p/mi.alpha**2)/h + (-mi1.betaY + mi2.betaY)*(rho*u0**2*vy*(ie + p/rho + 1.0) + mi.betaY*p/mi.alpha**2)/h + (-mi1.betaZ + mi2.betaZ)*(rho*u0**2*vz*(ie + p/rho + 1.0) + mi.betaZ*p/mi.alpha**2)/h + (-mi1.gamDDxx + mi2.gamDDxx)*(p*((mi.gamDDyy*mi.gamDDzz - mi.gamDDyz**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX**2/mi.alpha**2) + rho*u0**2*vx**2*(ie + p/rho + 1.0))/(2*h) + (-mi1.gamDDxy + mi2.gamDDxy)*(p*((-mi.gamDDxy*mi.gamDDzz + mi.gamDDxz*mi.gamDDyz)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaY/mi.alpha**2) + rho*u0**2*vx*vy*(ie + p/rho + 1.0))/h + (-mi1.gamDDxz + mi2.gamDDxz)*(p*((mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaX*mi.betaZ/mi.alpha**2) + rho*u0**2*vx*vz*(ie + p/rho + 1.0))/h + (-mi1.gamDDyy + mi2.gamDDyy)*(p*((mi.gamDDxx*mi.gamDDzz - mi.gamDDxz**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaY**2/mi.alpha**2) + rho*u0**2*vy**2*(ie + p/rho + 1.0))/(2*h) + (-mi1.gamDDyz + mi2.gamDDyz)*(p*((-mi.gamDDxx*mi.gamDDyz + mi.gamDDxy*mi.gamDDxz)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaY*mi.betaZ/mi.alpha**2) + rho*u0**2*vy*vz*(ie + p/rho + 1.0))/h + (-mi1.gamDDzz + mi2.gamDDzz)*(p*((mi.gamDDxx*mi.gamDDyy - mi.gamDDxy**2)/(mi.gamDDxx*mi.gamDDyy*mi.gamDDzz - mi.gamDDxx*mi.gamDDyz**2 - mi.gamDDxy**2*mi.gamDDzz + 2*mi.gamDDxy*mi.gamDDxz*mi.gamDDyz - mi.gamDDxz**2*mi.gamDDyy) - 1.0*mi.betaZ**2/mi.alpha**2) + rho*u0**2*vz**2*(ie + p/rho + 1.0))/(2*h) + (rho*u0**2*(ie + p/rho + 1.0) - p/mi.alpha**2)*(mi1.alpha**2 - mi1.betaX**2*mi1.gamDDxx - 2*mi1.betaX*mi1.betaY*mi1.gamDDxy - 2*mi1.betaX*mi1.betaZ*mi1.gamDDxz - mi1.betaY**2*mi1.gamDDyy - 2*mi1.betaY*mi1.betaZ*mi1.gamDDyz - mi1.betaZ**2*mi1.gamDDzz - mi2.alpha**2 + mi2.betaX**2*mi2.gamDDxx + 2*mi2.betaX*mi2.betaY*mi2.gamDDxy + 2*mi2.betaX*mi2.betaZ*mi2.gamDDxz + mi2.betaY**2*mi2.gamDDyy + 2*mi2.betaY*mi2.betaZ*mi2.gamDDyz + mi2.betaZ**2*mi2.gamDDzz)/(2*h),
     *    dlapse = (-mi1.alpha + mi2.alpha)/(2*h),
     *    dgmunu = -mi2.alpha**2 + mi2.betaX**2*mi2.gamDDxx + 2*mi2.betaX*mi2.betaY*mi2.gamDDxy + 2*mi2.betaX*mi2.betaZ*mi2.gamDDxz + mi2.betaY**2*mi2.gamDDyy + 2*mi2.betaY*mi2.betaZ*mi2.gamDDyz + mi2.betaZ**2*mi2.gamDDzz]"
     */
    {
       const double tmp0 = 1.0/h;
       const double tmp1 = pow(mi.alpha, -2);
       const double tmp2 = p*tmp1;
       const double tmp3 = rho*pow(u0, 2)*(ie + p/rho + 1.0);
       const double tmp4 = tmp3*vx;
       const double tmp5 = tmp3*vy;
       const double tmp6 = 1.0*tmp1;
       const double tmp7 = mi.betaX*tmp6;
       const double tmp8 = mi.gamDDxz*mi.gamDDyz;
       const double tmp9 = mi.gamDDyy*mi.gamDDzz;
       const double tmp10 = pow(mi.gamDDyz, 2);
       const double tmp11 = pow(mi.gamDDxy, 2);
       const double tmp12 = pow(mi.gamDDxz, 2);
       const double tmp13 = 1.0/(-mi.gamDDxx*tmp10 + mi.gamDDxx*tmp9 + 2*mi.gamDDxy*tmp8 - mi.gamDDyy*tmp12 - mi.gamDDzz*tmp11);
       const double tmp14 = (1.0/2.0)*tmp0;
       const double tmp15 = 2*mi1.betaX;
       const double tmp16 = 2*mi2.betaX;
       const double tmp17 = -pow(mi2.alpha, 2) + pow(mi2.betaX, 2)*mi2.gamDDxx + pow(mi2.betaY, 2)*mi2.gamDDyy + 2*mi2.betaY*mi2.betaZ*mi2.gamDDyz + mi2.betaY*mi2.gamDDxy*tmp16 + pow(mi2.betaZ, 2)*mi2.gamDDzz + mi2.betaZ*mi2.gamDDxz*tmp16;
       source = tmp0*(-mi1.betaX + mi2.betaX)*(mi.betaX*tmp2 + tmp4) + tmp0*(-mi1.betaY + mi2.betaY)*(mi.betaY*tmp2 + tmp5) + tmp0*(-mi1.betaZ + mi2.betaZ)*(mi.betaZ*tmp2 + tmp3*vz) + tmp0*(-mi1.gamDDxy + mi2.gamDDxy)*(p*(-mi.betaY*tmp7 + tmp13*(-mi.gamDDxy*mi.gamDDzz + tmp8)) + tmp4*vy) + tmp0*(-mi1.gamDDxz + mi2.gamDDxz)*(p*(-mi.betaZ*tmp7 + tmp13*(mi.gamDDxy*mi.gamDDyz - mi.gamDDxz*mi.gamDDyy)) + tmp4*vz) + tmp0*(-mi1.gamDDyz + mi2.gamDDyz)*(p*(-mi.betaY*mi.betaZ*tmp6 + tmp13*(-mi.gamDDxx*mi.gamDDyz + mi.gamDDxy*mi.gamDDxz)) + tmp5*vz) + tmp14*(-mi1.gamDDxx + mi2.gamDDxx)*(p*(-pow(mi.betaX, 2)*tmp6 + tmp13*(-tmp10 + tmp9)) + tmp3*pow(vx, 2)) + tmp14*(-mi1.gamDDyy + mi2.gamDDyy)*(p*(-pow(mi.betaY, 2)*tmp6 + tmp13*(mi.gamDDxx*mi.gamDDzz - tmp12)) + tmp3*pow(vy, 2)) + tmp14*(-mi1.gamDDzz + mi2.gamDDzz)*(p*(-pow(mi.betaZ, 2)*tmp6 + tmp13*(mi.gamDDxx*mi.gamDDyy - tmp11)) + tmp3*pow(vz, 2)) + tmp14*(-tmp2 + tmp3)*(pow(mi1.alpha, 2) - pow(mi1.betaX, 2)*mi1.gamDDxx - pow(mi1.betaY, 2)*mi1.gamDDyy - 2*mi1.betaY*mi1.betaZ*mi1.gamDDyz - mi1.betaY*mi1.gamDDxy*tmp15 - pow(mi1.betaZ, 2)*mi1.gamDDzz - mi1.betaZ*mi1.gamDDxz*tmp15 + tmp17);
       dlapse = tmp14*(-mi1.alpha + mi2.alpha);
       dgmunu = tmp17;
    }

So now we need to include source term for the energy.  It is
\begin{equation}
s = \lapse\rtgamma\left[\left(\T00\beta^i\beta^j + 2\T0i\beta^j\right)K_{ij} - \left(\T00\beta^i + \T0i\right)\partial_i\lapse\right]
\end{equation}
For a stationary metric.  $\beta^i = 0$, so all the terms go away except for the final term. So we have
\begin{equation}
s = -\lapse\rtgamma\T0i\partial_i\lapse
\end{equation}

```python
dalpha = ixp.zerorank1()

for i in range(3) :
    dalpha[i] = sp.symbols("dalpha[{0}]".format(i))

esource = 0
for j in range(3) :
    esource += TmunuUU[0][j+1]*dalpha[j]

cFunction = "double u0 = 0.;\n" + computeu0_Cfunction

str = outputC( [esource], ["eSource"], filename="returnstring")
f = open("NRPY+calTauSource.h", "w")
f.write( cFunction + str)
f.close()
```

# Step 5: Rotation
$$\label{rotation}$$

One of the key ideas behind the moving-mesh idea is that we must rotate the vector to the appropriate direction such that the Riemann solve is along the x-direction.  This is done by computing the normal vector also the "x-direction" and two normal vectors orthogonal to it.  In MANGA they are labeled $n_0$, $n_1$, and $n_2$.  And the rotation matrix looks like
\begin{equation}
R = \begin{pmatrix}
n_{0,x} & n_{0,y} & n_{0,z} \\
n_{1,x} & n_{1,y} & n_{1,z} \\
n_{2,x} & n_{2,y} & n_{2,z}
\end{pmatrix}
\end{equation}
Likewise, I also define an inverse Rotation as
\begin{equation}
R^{-1} = \begin{pmatrix}
n'_{0,x} & n'_{0,y} & n'_{0,z} \\
n'_{1,x} & n'_{1,y} & n'_{1,z} \\
n'_{2,x} & n'_{2,y} & n'_{2,z}
\end{pmatrix}
\end{equation}
Base on this how does $g_{\mu\nu}$ and $T_{\mu\nu}$ transform under rotation.  We begin by defining an extended rotation matrix:
\begin{equation}
\mathcal{R} = \begin{pmatrix}
1 &0 &0 &0 \\
0 &n_{0,x} & n_{0,y} & n_{0,z} \\
0 &n_{1,x} & n_{1,y} & n_{1,z} \\
0 & n_{2,x} & n_{2,y} & n_{2,z}
\end{pmatrix}
\end{equation}
Now we note that the term $g_{\mu\nu} x^{\mu} x^{\nu}$ is invariant under rotation.  Defining $x' = R x$, we then note
\begin{equation}
g_{\mu\nu} x^{\mu} x^{\nu} = g_{\mu\nu} \mathcal{R}^{-1\mu}_{\alpha}x'^{\alpha} \mathcal{R}^{-1\nu}_{\beta}x'^{\beta} = g_{\alpha\beta} \mathcal{R}^{-1\alpha}_{\mu}\mathcal{R}^{-1\beta}_{\nu}x'^{\mu} x'^{\nu} \rightarrow g'_{\mu\nu} = g_{\alpha\beta} \mathcal{R}^{-1\alpha}_{\mu}\mathcal{R}^{-1\beta}_{\nu},
\end{equation}
which gives us the appropriate rotated metric.
To get the similar transformation for $T^{\mu\nu}$, we note the $T = T^{\mu}_{\mu} = g_{\mu\nu}T^{\mu\nu}$ tranforms as a scalar.  So we have
\begin{equation}
T = g_{\mu\nu}T^{\mu\nu} = g_{\alpha\beta}\mathcal{R}^{-1,\alpha}_{\gamma}\mathcal{R}^{-1,\beta}_{\delta}\mathcal{R}^{\gamma}_{\mu}\mathcal{R}^{\delta}_{\nu} T^{\mu\nu} = \rightarrow g'_{\mu\nu} T'^{\mu\nu}
\end{equation}
which provide the identity
\begin{equation}
T'^{\mu\nu} = \mathcal{R}^{\mu}_{\alpha}\mathcal{R}^{\nu}_{\beta} T^{\alpha\beta}
\end{equation}

```python
RotUD = ixp.declarerank2("RotUD", "nosym", DIM=4)
RotInvUD = ixp.declarerank2("RotInvUD", "nosym", DIM=4)

#declare normal primed vectors for rotation
n00, n01, n02 = sp.symbols("n0[0] n0[1] n0[2]")
n10, n11, n12 = sp.symbols("n1[0] n1[1] n1[2]")
n20, n21, n22 = sp.symbols("n2[0] n2[1] n2[2]")

#declare normal primed vectors for inverse rotation
n0p0, n0p1, n0p2 = sp.symbols("n0p[0] n0p[1] n0p[2]")
n1p0, n1p1, n1p2 = sp.symbols("n1p[0] n1p[1] n1p[2]")
n2p0, n2p1, n2p2 = sp.symbols("n2p[0] n2p[1] n2p[2]")

for i in range(4):
    RotUD[0][i] = 0.
    RotUD[i][0] = 0.
    RotInvUD[0][i] = 0.
    RotInvUD[i][0] = 0.

RotUD[0][0] = 1.
RotInvUD[0][0] = 1.

RotUD[1][1] = n00
RotUD[1][2] = n01
RotUD[1][3] = n02
RotUD[2][1] = n10
RotUD[2][2] = n11
RotUD[2][3] = n12
RotUD[3][1] = n20
RotUD[3][2] = n21
RotUD[3][3] = n22

RotInvUD[1][1] = n0p0
RotInvUD[1][2] = n0p1
RotInvUD[1][3] = n0p2
RotInvUD[2][1] = n1p0
RotInvUD[2][2] = n1p1
RotInvUD[2][3] = n1p2
RotInvUD[3][1] = n2p0
RotInvUD[3][2] = n2p1
RotInvUD[3][3] = n2p2

gmunuRotDD = ixp.declarerank2("gmunuRotDD", "sym01", DIM=4)

for i in range(4) :
    for j in range(4) :
        gmunuRotDD[i][j] = 0.
        for k in range(4) :
            for l in range(4) :
                gmunuRotDD[i][j] += gmunuDD[l][k]*RotInvUD[l][i]*RotInvUD[k][j]

outputC([gmunuRotDD[1][1], gmunuRotDD[1][2], gmunuRotDD[1][3],gmunuRotDD[2][2], gmunuRotDD[2][3], gmunuRotDD[3][3]], ["metricInfo.gamDDxx", "metricInfo.gamDDxy","metricInfo.gamDDxz","metricInfo.gamDDyy","metricInfo.gamDDyz","metricInfo.gamDDzz"], filename="NRPY+rotateMetric.h")

# provie

TmunuRotUU = ixp.declarerank2("TmunuRotUU", "sym01", DIM=4)

for i in range(4) :
    for j in range(4) :
        TmunuRotUU[i][j] = 0.
        for k in range(4) :
            for l in range(4) :
                TmunuRotUU[i][j] += TmunuUU[l][k]*RotUD[i][l]*RotUD[j][k]

str = outputC([TmunuRotUU[0][0], TmunuRotUU[0][1], TmunuRotUU[1][0]], ["Tmunu00", "Tmunu12", "Tmunu21"], filename="returnstring")

print(str)

```

# Step 6: Conservative to Primitive Solver
$$\label{solver}$$

We now discuss the reverse mapping from conservative to primitive variables.
Given the lapse, shift vector and $\rtgamma$, the mapping between primitive and conserved variable is straightforward.  However, the reverse is not as simple.  In GRMHD, the conservative to primitive solver is amplified by the inclusion of the magnetic field, leading to rather sophisticated root finding strategies.  The failure rates of these algorithms are low (??), but since this algorithm may be executed several times per timestep for every gridpoint, even a low failure can give unacceptable collective failure rates.  However, for purely polytropic equations of state, e.g., $P\propto\rho^{\Gamma_1}$, the convervative to primitive variable solver is greatly simplified.

To construct the conservative-to-primitive variable solver, we restrict ourselves to polytropic equations of states
\begin{equation}
P = P_0\left(\frac{\rho}{\rho_0}\right)^{\Gamma_1} \quad\textrm{and}\quad \epsilon = \epsilon_0\left(\frac{\rho}{\rho_0}\right)^{\Gamma_1-1},
\end{equation}
where $P_0$, $\rho_0$, and $\epsilon_0$ are the fiducial pressure, density, and internal energy, and we have used the relation $P = (\Gamma_1 - 1)\rho\epsilon$.

For such a polytropic equation of state, the energy equation is redundant and effectively we are only concerned with the continuity and momentum equations. The conservative variables of concern are $\rhostar$ and $\Svectilde$.  Noting that the shift, $\alpha$, and $\rtgamma$ are provided by the Einsteins field equation solver, we can write
\begin{equation}
u^0 = \frac{\rhostar}{\alpha\rtgamma\rho} = u^0(\rho)  \quad\textrm{and}\quad \uvec = \frac{\Svectilde}{\alpha\rtgamma\rho h} = \uvec(\rho).
\end{equation}
Noting that the four velocity $u^2 = g_{\mu\nu}u^{\mu}u^{\nu} = g^{00}u^0u^0 + 2g^{0i}u^0\uvec^i + g_{ij}\uvec^i\uvec^j = -1$, we have
\begin{equation}
 0 = f(\rho)\equiv \alpha^2\gamma\rho^2h^2 + \left(-\lapse^2 + \shift\cdot\shift\right)\rhostar^2h^2 + 2h\rhostar\shift\cdot\Svectilde + \Svectilde\cdot\Svectilde,
\end{equation}
which is an implicit equation of either $\rho$ or $u^0$, where $h(\rho = \rhostar/(\alpha\rtgamma u^0)) = 1 + \gamma_1 \epsilon$ which can be inverted by standard nonlinear root finding algorithms, e.g., Newton-raphson.

We put this all together to define a function, $f(\rho)$, whose root is zero that we will find via Newton-raphson.

Several checks must be performed:

1. $\rhostar > 0$ : This check is performed at the very beginning

2. $\rho > \rho_{\rm min}$ : This check is performed after the fact

3. $u_0 < \alpha^{-1}\Gamma_{\rm max}$ : This check is performed after the fact as well

```python
DIM = 3
# Declare rank-1 contravariant ("v") vector
vU = ixp.declarerank1("vU")
shiftU = ixp.zerorank1()
rho, gamma1 = sp.symbols("rho gamma")
Sx, Sy, Sz = sp.symbols("con[iSx] con[iSy] con[iSz]")
p0, rho0, rhostar = sp.symbols("p_0 rho_0 rhostar")
# Declare rank-2 covariant gmunu
#gammaDD = ixp.declarerank2("gammaDD","sym01")
StildeD = ixp.declarerank1("StildeD")
lapse, rtgamma, beta_x, beta_y, beta_z = sp.symbols( "mi.alpha mi.rtDetGamma mi.betaX mi.betaY mi.betaZ")

shiftU[0] = beta_x
shiftU[1] = beta_y
shiftU[2] = beta_z

StildeD[0] = Sx
StildeD[1] = Sy
StildeD[2] = Sz

gamma = rtgamma*rtgamma
lapse2 = lapse*lapse
uU0 = rhostar/(lapse*rtgamma*rho)
epsilon = p0/rho0*(rho/rho0)**(gamma1 - 1)/(gamma1 - 1)
h = 1. + gamma1*epsilon

beta2 = 0.

for i in range(DIM) :
    for j in range(DIM) :
        beta2 += gammaDD[i][j] * shiftU[i]*shiftU[j]

betaDotStilde = 0
for i in range(DIM) :
    betaDotStilde += shiftU[i]*StildeD[i]

Stilde2 = 0
for i in range(DIM) :
    for j in range(DIM) :
        Stilde2 += gammaUU[i][j] * StildeD[i]*StildeD[j]

f = rhostar**2*h**2 + (-lapse2 + beta2)*rhostar**2.*h**2.*uU0**2 + 2.*h*rhostar*betaDotStilde*uU0 + Stilde2

outputC(f,"rootRho",filename="NRPY+rhoRoot.h")
outputC(Stilde2, "Stilde2", filename="NRPY+Stilde2.h")
```

The root solve above finds $\rho$, which then allows us to get
\begin{equation}
u^0 = \frac{\rhostar}{\alpha\rtgamma\rho}\quad\textrm{and}\quad \vel = \frac{\uvec}{u^0} = \frac{\Svectilde}{\rhostar h(\rho)}.
\end{equation}
and thus we can find the rest of the primitives.

```python
#rhostar = sp.symbols("rhostar")
#StildeU = ixp.declarerank1("StildeU")
velU = ixp.zerorank1()

#lapse, rtgamma, rho, gamma1, c = sp.symbols("lapse rtgamma rho gamma1 c")
rho, rhostar = sp.symbols("testPrim[iRho] con[iRhoStar]")

u0 = rhostar/(lapse*rtgamma*rho)
epsilon = p0/rho0*(rho/rho0)**(gamma1 - 1)/(gamma1 - 1)

h = 1. + gamma1*epsilon

for i in range(DIM) :
    for j in range(DIM) :
        velU[i] += gammaUU[i][j]*StildeD[j]/(rhostar * h)/u0

outputC([h,u0,velU[0],velU[1],velU[2]], ["h", "u0","testPrim[ivx]", "testPrim[ivy]", "testPrim[ivz]"],filename="NRPY+getv.h")
```

# Step 7: Lorentz Boosts
$$\label{lorentz}$$

We need to boost to the frame of the moving face.  The boost is
\begin{equation}
B(\beta) =\begin{pmatrix}
\gamma & -\beta\gamma n_x & -\beta\gamma n_y & -\beta\gamma n_z  \\
-\beta\gamma n_x  & 1 + (\gamma-1)n_x^2 & (\gamma-1)n_x n_y & (\gamma-1)n_x n_z\\
-\beta\gamma n_x  & (\gamma-1)n_y n_x & 1 + (\gamma-1)n_y^2 & (\gamma-1)n_y n_z\\
-\beta\gamma n_x & (\gamma-1) n_z n_x & (\gamma-1)n_z n_x & 1 + (\gamma-1)n_z^2
\end{pmatrix}
\end{equation}
And the boost is $X' = B(\beta) X$, where $X'$ and $X$ are four vectors.

So the rest of this is straightforward.

# Step 8: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-GRMHD_Equations-Cartesian.pdf](Tutorial-GRMHD_Equations-Cartesian.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx Tutorial-GRMHD_Equations-Cartesian.ipynb
!pdflatex -interaction=batchmode Tutorial-GRMHD_Equations-Cartesian.tex
!pdflatex -interaction=batchmode Tutorial-GRMHD_Equations-Cartesian.tex
!pdflatex -interaction=batchmode Tutorial-GRMHD_Equations-Cartesian.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```


Table of Contents

1  Tree based methods

1.1  Regression Example

1.1.1  Load the data and split the dataset

1.1.2  Fit a simple regression tree (no CV)

1.1.3  Display the tree

1.1.4  Variables Importance

1.1.5  Understanding the tree structure (partially from here)

1.1.6  Evaluating the tree

1.2  Classification Example

1.2.1  Display the tree

1.2.2  Variables importance

1.2.3  CV Evaluation of the classification tree

1.2.4  Manually finding the optimal tree depth

1.2.5  Cost Complexity Pruning

1.2.6  Grid search

1.2.7  Working with the probabilities of the estimations

1.2.8  Laplacian Correction (Smoothing)

1.3  Optimization techniques

1.3.1  Bagging

1.3.2  Random Forests

1.3.3  Boosting

1.4  Hyperparameter tuning

# Tree based methods

J. Renero

Trees construct prediction models from data. They recursively partition the data space to fit a boolean prediction model within each partition. The result is represented graphically as a decision tree. Trees can be used for regression of classification.

When the number of features, $p$, is large interpretability is easier than in distance based methods or probabilistic classification, where boundaries are much more difficult to be explained in higer dimensions.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import pydotplus
import graphviz
import warnings

from IPython.display import Image
from sklearn.model_selection import cross_val_score, train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeRegressor, DecisionTreeClassifier, export_graphviz
from sklearn.externals.six import StringIO
from sklearn.metrics import f1_score, mean_squared_error, r2_score, mean_absolute_error
from sklearn.tree import export_graphviz
from sklearn import tree
from sklearn.exceptions import DataConversionWarning
from sklearn.ensemble import BaggingClassifier, RandomForestClassifier, GradientBoostingClassifier
from sklearn.utils.validation import column_or_1d
from tqdm import tqdm_notebook

from lime.lime_tabular import LimeTabularExplainer

from dataset import Dataset

warnings.filterwarnings("ignore", category=DeprecationWarning)
warnings.filterwarnings("ignore", category=DataConversionWarning)

%matplotlib inline
```

```python
# These are methods I use along the notebook
def plot_compare(d, label, new_training_score, new_test_score, ymin=0.91, ymax=0.97):
    d.update({label: [np.median(new_training_score),
                      np.median(new_test_score)]})
    plt.figure(figsize=(9, 7))
    pd.DataFrame(d).set_index([pd.Index(['training', 'test'])]).plot.bar()
    plt.ylim(ymin, ymax)
    plt.show()

def features_importance(tree, X):
    """
    Plot the variables importance from the fitted tree and the
    training dataset used.
    """
    importances = tree.feature_importances_
    indices = np.argsort(importances)

    # Plot the feature importances of the forest
    plt.figure(figsize=(10, 4))
    plt.title("Feature importances")
    plt.barh(range(X.train.shape[1]), importances[indices],
             color="r", align="center")
    index_names = [list(X.train)[i] for i in indices]
    plt.yticks(range(X.train.shape[1]), index_names)
    plt.ylim([-1, X.train.shape[1]])
    plt.xlim([0.0, 1.0])
    plt.show()
```

## Regression Example

We will start by building a simple regression example with the [wholesale customer dataset in UCI](https://archive.ics.uci.edu/ml/datasets/wholesale+customers). It contains 440 samples of 8 variables, and we will try to build a model that will help us to predict the annual spending on fresh products (first variable) with a regression tree.

### Load the data and split the dataset

```python
data_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00292/Wholesale%20customers%20data.csv'
wholesale = Dataset(data_url)
wholesale.set_target('Fresh')
X, Y = wholesale.split()
regression_results = {}
wholesale.all.head()
```

      Channel
      Region
      Milk
      Grocery
      Frozen
      Detergents_Paper
      Delicassen
      Fresh

      0
      2.0
      3.0
      9656.0
      7561.0
      214.0
      2674.0
      1338.0
      12669.0

      1
      2.0
      3.0
      9810.0
      9568.0
      1762.0
      3293.0
      1776.0
      7057.0

      2
      2.0
      3.0
      8808.0
      7684.0
      2405.0
      3516.0
      7844.0
      6353.0

      3
      1.0
      3.0
      1196.0
      4221.0
      6404.0
      507.0
      1788.0
      13265.0

      4
      2.0
      3.0
      5410.0
      7198.0
      3915.0
      1777.0
      5185.0
      22615.0

### Fit a simple regression tree (no CV)

```python
regressor = DecisionTreeRegressor(random_state=0, max_depth=3)
regressor.fit(X.train, Y.train)
```

    DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=3,
                          max_features=None, max_leaf_nodes=None,
                          min_impurity_decrease=0.0, min_impurity_split=None,
                          min_samples_leaf=1, min_samples_split=2,
                          min_weight_fraction_leaf=0.0, presort='deprecated',
                          random_state=0, splitter='best')

### Display the tree

Things get a bit uglier in Python when it comes to plot a tree (this time, R is better than Python...). We must provide the fitted tree to the `plot_tree()` method and adjust `figsize` and `fontsize`.

```python
plt.figure(figsize=(16, 6))
expl = tree.plot_tree(regressor.fit(X.train, Y.train),
               filled=True, rounded=True, fontsize=9)
```

### Variables Importance

Whenever a tree is grown (fit) we can examine the results by accessing the properties of the object created. In this case, the object is what we've called `regressor`. To check variables importance, we must access the property called `feature_importances_`

    >>> my_tree.feature_importances_

You can easily construct a function that plots the features sorted by importance (as computed by the tree).

```python
features_importance(regressor, X)
```

### Understanding the tree structure (partially from [here](https://scikit-learn.org/stable/auto_examples/tree/plot_unveil_tree_structure.html#sphx-glr-auto-examples-tree-plot-unveil-tree-structure-py))

From an example in SciKit Learn pages. Do not try this at home!

```python
def explain_tree(regressor):
    n_nodes = regressor.tree_.node_count
    children_left = regressor.tree_.children_left
    children_right = regressor.tree_.children_right
    feature = regressor.tree_.feature
    threshold = regressor.tree_.threshold

    # The tree structure can be traversed to compute various properties such
    # as the depth of each node and whether or not it is a leaf.
    node_depth = np.zeros(shape=n_nodes, dtype=np.int64)
    is_leaves = np.zeros(shape=n_nodes, dtype=bool)
    stack = [(0, -1)]  # seed is the root node id and its parent depth
    while len(stack) > 0:
        node_id, parent_depth = stack.pop()
        node_depth[node_id] = parent_depth + 1

        # If we have a test node
        if (children_left[node_id] != children_right[node_id]):
            stack.append((children_left[node_id], parent_depth + 1))
            stack.append((children_right[node_id], parent_depth + 1))
        else:
            is_leaves[node_id] = True

    print("The binary tree structure has %s nodes and has "
          "the following tree structure:"
          % n_nodes)
    for i in range(n_nodes):
        if is_leaves[i]:
            print("%snode=%s (leaf node)" % (node_depth[i] * "  ", i))
        else:
            print("%snode=%s: go to node %s if X[:, %s] <= %s else to "
                  "node %s."
                  % (node_depth[i] * "  ",
                     i,
                     children_left[i],
                     feature[i],
                     threshold[i],
                     children_right[i],
                     ))
```

```python
explain_tree(regressor)
```

    The binary tree structure has 15 nodes and has the following tree structure:
    node=0: go to node 1 if X[:, 4] <= 15943.0 else to node 8.
      node=1: go to node 2 if X[:, 4] <= 2905.5 else to node 5.
        node=2: go to node 3 if X[:, 5] <= 2803.5 else to node 4.
          node=3 (leaf node)
          node=4 (leaf node)
        node=5: go to node 6 if X[:, 2] <= 2547.0 else to node 7.
          node=6 (leaf node)
          node=7 (leaf node)
      node=8: go to node 9 if X[:, 4] <= 17386.5 else to node 12.
        node=9: go to node 10 if X[:, 4] <= 16641.5 else to node 11.
          node=10 (leaf node)
          node=11 (leaf node)
        node=12: go to node 13 if X[:, 5] <= 155.0 else to node 14.
          node=13 (leaf node)
          node=14 (leaf node)

### Evaluating the tree

What's the **$\text{F1}$** of this tree when making predictions? We **never** evaluate a predictor using a single split or a single measure. Instead, we apply CV to evaluate its performance, on average, over different splits from the training set, to foresee what is its expected performance in training, no matter what portion of the data is used to train the algorithm.

To finally evaluate the tree, we must compute the same metric over the test set, without CV (no extra levels of paranoia, yet).

To be sure that our metrics are OK, we will also repeat the above process to guarantee the our initial splitting is not conditioning very much the final evaluation metric. It is not enough to split once and call `cross_val_score()` once. We we still need to be sure that our initial split (line 7) is not biassed.

    X, y = my_dataset.split(seed=23*i ^ 2)

To solve that, we repeat the CV process to ensure that we repeatedly evaluate (line 6) with a different split each time:

    for i in range(num_iterations):

```python
def cv_regression(my_dataset, depth=3, num_iterations=20):
    # I will store results here
    train_scores = []
    test_scores = []

    for i in range(num_iterations):
        X, y = my_dataset.split(seed=23*i ^ 2)
        estimator = DecisionTreeRegressor(random_state=0, max_depth=depth)
        train_metric = cross_val_score(estimator, X.train, y.train,
                                       scoring='neg_root_mean_squared_error')
        train_scores.append(np.median(train_metric))

        # Now I evaluate the test set, fitted with the training split.
        estimator.fit(X.train, y.train)
        test_score = mean_squared_error(y.test, estimator.predict(X.test),
                                        squared=False)
        test_scores.append(test_score)

    return train_scores, test_scores
```

```python
cv_scores, test_scores = cv_regression(wholesale)

print('Training median RMSE: {:.4f} +/- {:.2f}%'.format(
    -np.median(cv_scores), np.std(cv_scores)*100.))
print('Test RMSE: {:.4f} +/- {:.2f}%'.format(
    np.median(test_scores), np.std(test_scores)*100.))

# This is simply to keep track of my results
regression_results.update({'regression': [-np.median(cv_scores), np.median(test_scores)]})
```

    Training median RMSE: 12944.2812 +/- 148014.46%
    Test RMSE: 12542.5902 +/- 199084.45%

And now, let's apply our (paranoid) `cv_regression()` evaluator to our problem, trying with different depths for the tree, so we can have a safe estimate of what is the effect of the depth of the tree to the final performance of the model.

```python
min_depth = 1
max_depth = 15
metric_train = []
metric_test = []
for i in range(min_depth, max_depth):
    train, test = cv_regression(wholesale, depth=i)
    metric_train.append(-np.median(train))
    metric_test.append(np.median(test))
```

```python
# plot the results
pd.DataFrame(
    {'training': metric_train, 'testing': metric_test}
).plot(legend=True, style='o-', figsize=(8, 5), title="RMSE vs. depth")
```

What is the best depth that we can obtain? It seems that increasing the depth is nor improving performance but increasing the error. Trees are not very good regressors... unless you play a little bit with parameters.

```python
opt_training_scores, opt_test_scores = cv_regression(wholesale, depth=4)

plot_compare(regression_results, 'pruned', np.negative(opt_training_scores),
             opt_test_scores, 10000, 15000)
```

## Classification Example

Let's build, using the same dataset a classification case. To do so the goal of this new model will be predicting the channel used by the customer (1 or 2), so we have a **binomial** classification problem.

```python
wholesale = Dataset(data_url)
wholesale.set_target('Channel')

X, Y = wholesale.split()
classification_results = {}

classifier = DecisionTreeClassifier(random_state=0, max_depth=3)
classifier.fit(X.train, Y.train)
```

    DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='gini',
                           max_depth=3, max_features=None, max_leaf_nodes=None,
                           min_impurity_decrease=0.0, min_impurity_split=None,
                           min_samples_leaf=1, min_samples_split=2,
                           min_weight_fraction_leaf=0.0, presort='deprecated',
                           random_state=0, splitter='best')

### Display the tree

Colors correspond to the different classes, and color intensity corresponds to Gini index (i.e.: impurity: the purer the darker).

```python
plt.figure(figsize=(16, 7))
tree.plot_tree(classifier, feature_names=wholesale.feature_names,
               node_ids=True, filled=True);
```

### Variables importance

Feature importance is calculated as the decrease in **node impurity weighted by the probability of reaching that node**. The node probability can be calculated by the number of samples that reach the node, divided by the total number of samples. The higher the value the more important the feature.

Scikit learns computes the node importance using the following expression to compute the node importance:

\begin{equation}
    ni_j = w_j C_j - w_{left(j)} C_{left(j)} - w_{right(j)} C_{right(j)}
\end{equation}

where:

- $ni_j$: the importance of node $j$
- $w_j$: weighted nr. of samples reaching node $j$
- $C_j$: Impurity of node $j$
- $left(j)$ / $right(j)$: child node from left/right split on node $j$

and then, **features importance** is computed as follows:

\begin{equation}
    fi_i = \frac{ \sum_{j:node\ j\ splits\ on\ feature\ i} ni_j }{ \sum_{k\ \in \ all\ nodes} ni_k }
\end{equation}

Feature importance of a feature is a sum of importances of the nodes where that feature is used to take a split decision.

```python
features_importance(classifier, X)
```

### CV Evaluation of the classification tree

Evaluate the tree in training and test (hold out split):

1. Repeat `num_iterations` times
  1. Split between training and test
  2. Obtain a list of training metrics obtained along the CV process, with the training subset
  3. Store the median of the training metrics.
  4. Obtain a test metric obtained with the held out subset resulting from the split at step A.
  5. Store the median of the test metric.

```python
def cv_classification(my_dataset, estimator, num_iterations=20):
    train_scores=[]
    test_scores=[]
    for i in range(num_iterations):
        X, y = my_dataset.split(seed=23*i ^ 2)

        # We want to call a method whose name is the argument to this func.
        estimator.fit(X.train, y.train)
        train_metric = cross_val_score(estimator, X.train, y.train, scoring='f1')
        train_scores.append(np.median(train_metric))

        test_score = f1_score(y.test, estimator.predict(X.test))
        test_scores.append(test_score)
    return train_scores, test_scores
```

And now, simply apply it with a fixed tree depth.

```python
simple_tree = DecisionTreeClassifier(random_state=0, max_depth=3)
cv_scores, test_scores = cv_classification(wholesale, simple_tree)

print('Training median F1: {:.4f} +/- {:.2f}%'.format(
    np.median(cv_scores), np.std(cv_scores)*100.))
print('Test F1: {:.4f} +/- {:.2f}%'.format(
    np.median(test_scores), np.std(test_scores)*100.))

classification_results.update({'basic': [np.median(cv_scores), np.median(test_scores)]})
```

    Training median F1: 0.9355 +/- 1.29%
    Test F1: 0.9323 +/- 2.48%

### Manually finding the optimal tree depth

Let's try now to perform our evaluation in an scenario where we're looking for the optimum tree depth, precisely, to avoid overfittin gin our selection.

To do so, we must repeat the sequence described in the previous section with all possible __depths__ we want to try: we will try with values between 1 and 19, but you will quickly realize that trees deeper than 11 show no change in F1 metric.

```python
min_depth = 1
max_depth = 15
metric_train = []
metric_test = []
for i in range(min_depth, max_depth):
    estimator = DecisionTreeClassifier(random_state=0, max_depth=i)
    train, test = cv_classification(wholesale, estimator)
    metric_train.append(np.median(train))
    metric_test.append(np.median(test))
```

```python
# plot the results
pd.DataFrame(
    {'training': metric_train, 'testing': metric_test}
).plot(legend=True, style='o-', figsize=(8, 5),  title='F1 vs. depth');
```

From depth=4 our test F1 starts to decrease very fast. So, my take is that depth=4 is the optimal value

```python
opt_tree = DecisionTreeClassifier(
    random_state=1, max_depth=4).fit(X.train, Y.train)

f1_values = cross_val_score(opt_tree, X.train, Y.train, scoring="f1")
opt_training_score = np.median(f1_values)
print('F1 (training average): {:.4f}'.format(opt_training_score))

opt_test_score = f1_score(Y.test, opt_tree.predict(X.test))
print('F1 (test): {:.4f}'.format(opt_test_score))
```

    F1 (training average): 0.9451
    F1 (test): 0.9310

**Compare** the results with the optimal tree, against the base tree grown using suboptimal max_depth. Your test metric with optimal parameters must always be **greater or equal** than the default one.

```python
plot_compare(classification_results, 'optimal(depth=4)',
             opt_training_score, opt_test_score);
```

### Cost Complexity Pruning

This pruning technique is parameterized by the cost complexity parameter, `ccp_alpha`. Greater values of ccp_alpha increase the number of nodes pruned.

**Our problem is to find the optimal value for $\alpha$**.

Minimal cost complexity pruning recursively finds the node with the “weakest link”. The weakest link is characterized by an effective alpha, where the nodes with the smallest effective alpha are pruned first.

```python
clf = DecisionTreeClassifier(random_state=0)
path = clf.cost_complexity_pruning_path(X.train, Y.train)
ccp_alphas, impurities = path.ccp_alphas, path.impurities
```

```python
def plot_effective_alphas():
    fig, ax = plt.subplots()
    fig.set_figwidth(8)
    fig.set_figheight(6)
    ax.plot(ccp_alphas[:-1], impurities[:-1], marker='.', drawstyle="steps-post")
    ax.set_xlabel("effective alpha")
    ax.set_ylabel("total impurity of leaves")
    ax.set_title("Total Impurity vs effective alpha for training set");
```

```python
plot_effective_alphas()
```

Next, we train a collection of decision trees using different values for alphas taken from the list of effective alphas. This is just a way of avoiding to blindly search for optimal alpha values among a long list.

The last value in ccp_alphas is the alpha value that prunes the whole tree, leaving the tree, `clfs[-1]`, with one node.

```python
clfs = []
for ccp_alpha in ccp_alphas:
    clf = DecisionTreeClassifier(random_state=0, ccp_alpha=ccp_alpha)
    clf.fit(X.train, Y.train)
    clfs.append(clf)
print("Number of nodes in the last tree is: {} with ccp_alpha: {}".format(
      clfs[-1].tree_.node_count, ccp_alphas[-1]))
```

    Number of nodes in the last tree is: 1 with ccp_alpha: 0.2817756667589173

Here we show that the number of nodes and tree depth decreases as alpha increases.

```python
def plot_alphas():
    global clfs, ccp_alphas
    clfs = clfs[:-1]
    ccp_alphas = ccp_alphas[:-1]

    node_counts = [clf.tree_.node_count for clf in clfs]
    depth = [clf.tree_.max_depth for clf in clfs]
    fig, ax = plt.subplots(2, 1)
    fig.set_figwidth(6)
    fig.set_figheight(8)

    ax[0].plot(ccp_alphas, node_counts, marker='o', drawstyle="steps-post")
    ax[0].set_xlabel("alpha")
    ax[0].set_ylabel("number of nodes")
    ax[0].set_title("Number of nodes vs alpha")

    ax[1].plot(ccp_alphas, depth, marker='o', drawstyle="steps-post")
    ax[1].set_xlabel("alpha")
    ax[1].set_ylabel("depth of tree")
    ax[1].set_title("Depth vs alpha")
    fig.tight_layout()
```

```python
plot_alphas()
```

When `ccp_alpha` is set to zero and keeping the other default parameters of DecisionTreeClassifier, the tree overfits, leading to a 100% training accuracy and 88% testing accuracy. As alpha increases, more of the tree is pruned, thus creating a decision tree that generalizes better. In this example, setting ccp_alpha=0.015 maximizes the testing accuracy.

```python
def plot_alpha_scores():
    global ccp_alphas, train_scores, test_scores

    fig, ax = plt.subplots()
    fig.set_figwidth(10)
    fig.set_figheight(6)
    ax.set_xlabel("alpha")
    ax.set_ylabel("F1")
    ax.set_title("F1 vs alpha for training and testing sets")
    ax.plot(ccp_alphas, train_scores, marker='o', label="train",
            drawstyle="steps-post")
    ax.plot(ccp_alphas, test_scores, marker='o', label="test",
            drawstyle="steps-post")
    ax.legend()
    plt.show()
```

```python
train_scores = [f1_score(clf.predict(X.train), Y.train) for clf in clfs]
test_scores = [f1_score(clf.predict(X.test), Y.test) for clf in clfs]

plot_alpha_scores()
```

In our example it seems that in order to maximize the value of F1 in testing; we should choose a value of `alpha` between 0.005 and 0.007 (if you take a look to the list of `ccp_alphas` you can check that it is: 0.00550964). That value maximizes F1 in test, while keeping a good F1 value in training.

```python
ccp = DecisionTreeClassifier(random_state=0, ccp_alpha=0.0055)
ccp_train, ccp_test = cv_classification(wholesale, ccp)

plot_compare(classification_results, 'ccp', ccp_train, ccp_test);
```

### Grid search

What is the optimal depth for this tree given the results obtained with CV? Instead of looking at a plot, we can use the grid search over the parameters we want to try, and for the range of values we decide. The result is a much simpler approach to the problem where you don't have to write the loop, and a single call does all the job for you.

```python
parameters = {'max_depth':range(min_depth, max_depth)}
grid_tree = GridSearchCV(DecisionTreeClassifier(), scoring='f1',
                         param_grid=parameters)
grid_tree.fit(X.train, Y.train)
best_tree = grid_tree.best_estimator_

print('Optimal F1: {:.4f}, for max_depth={}'.format(
    grid_tree.best_score_, grid_tree.best_params_['max_depth']))
```

    Optimal F1: 0.9457, for max_depth=4

### Working with the probabilities of the estimations

The predicted class probability is the fraction of samples of the same class in a leaf. We can apply __laplace smoothing__ or play with the __threshold__ to be applied to consider when to assign a leaf to each class to find better results than those obtained with the default 0.5 probability.

In the code below, we obtain the prediction probabilities for the 23rd sample in the test set.

```python
sample = X.test.iloc[23, :].values.reshape(1, -1)
leaf_id = classifier.apply(sample)
raw_probs = classifier.predict_proba(sample)[0]

print('Leaf node ID................:', leaf_id[0])
print('Decision path:\n', classifier.decision_path(sample), sep='')
print('Raw Probabilities...........:', ', '.join(['{:.2f}'.format(prob) for prob in raw_probs]))
print('Nr. of samples of leaf node.:', classifier.tree_.n_node_samples[leaf_id][0])
print('Impurity (Gini) of leaf node:', classifier.tree_.impurity[leaf_id][0])
print('Samples from each class.....:', classifier.tree_.value[leaf_id].flatten())
```

    Leaf node ID................: 9
    Decision path:
      (0, 0)	1
      (0, 6)	1
      (0, 8)	1
      (0, 9)	1
    Raw Probabilities...........: 0.41, 0.59
    Nr. of samples of leaf node.: 29
    Impurity (Gini) of leaf node: 0.4851367419738407
    Samples from each class.....: [12. 17.]

### Laplacian Correction (Smoothing)

Let's implement a simple function that will compute the smoothed probabilities for a leaf node, given the proportions it presents. Remember that this method smooths probability estimates that are too extreme because of the small size of the sample that reaches leaf node.

$$ P(k) = \frac{k+1}{N+C} $$

```python
def laplacian_correction(K, C=2):
    """
    K = array with the number of samples from each class
    C = numer of classes (default=2)
    """
    N = K.sum()       # total nr of samples in this node
    prob = []
    for i, k in enumerate(K):
        prob.append((k+1)/(N+C))
    return prob

# the list with the nr of elements from each class (flattened)
K = classifier.tree_.value[leaf_id].flatten()
raw_probs = classifier.predict_proba(sample)[0]
laplace_probs = laplacian_correction(K)

print('Raw Probabilities:',
      ', '.join(['{:.2f}'.format(prob) for prob in raw_probs]))
print('Laplace Probs....:',
      ', '.join(['{:.2f}'.format(prob) for prob in laplace_probs]))
```

    Raw Probabilities: 0.41, 0.59
    Laplace Probs....: 0.42, 0.58

## Optimization techniques

### Bagging

A Bagging classifier is an ensemble meta-estimator that fits base classifiers each on random subsets of the original dataset and then aggregate their individual predictions (either by voting or by averaging) to form a final prediction. Such a meta-estimator can typically be used as a way to reduce the variance of a black-box estimator (e.g., a decision tree), by introducing randomization into its construction procedure and then making an ensemble out of it.

We're producing `cv` splits, and for each of them we're growing 100 trees, each with a bootstrapped sample of the training set, to produce a classification, based on voting (hard or soft).

```python
bagging = BaggingClassifier(
    DecisionTreeClassifier(max_depth=4),
    n_estimators=100,
    max_features=wholesale.num_features,
    max_samples=0.5,
    oob_score=True,
    random_state=0, n_jobs=-1)
bagging_train, bagging_test = cv_classification(wholesale, bagging)

plot_compare(classification_results, 'bagging', bagging_train, bagging_test);
```

The result is better than the one obtained by simply growing a single tree.

### Random Forests

Now, we're not only going to pick subsets of the training set to train a number of trees, but also to select subset of features. In the end, the classification will be performed by the avearege prediction made by the grown trees.

```python
rf = RandomForestClassifier(n_estimators=100,
                            max_features=3,     # <---- difference w Bagging
                            max_depth=4,
                            oob_score=True,
                            random_state=0, n_jobs=-1)

rf_train, rf_test = cv_classification(wholesale, rf)

plot_compare(classification_results, 'r.forests', rf_train, rf_test)
```

The result is much better than the one obtained by simply applying grid search over the max_depth parameters with cross-validation, but slightly worse than in the bagging case.

### Boosting

Gradient Tree Boosting or Gradient Boosted Trees (GBT) is a generalization of boosting to arbitrary differentiable loss functions. GBRT is an accurate and effective off-the-shelf procedure that can be used for both regression and classification problems. Gradient Tree Boosting models are used in a variety of areas including Web search ranking and ecology.

```python
gb = GradientBoostingClassifier(n_estimators=100,
                                learning_rate=1.0,
                                max_features=3,
                                max_depth=4,
                                random_state=0)

gb_train, gb_test = cv_classification(wholesale, gb)

plot_compare(classification_results, 'boosting', gb_train, gb_test)
```

## Hyperparameter tuning

One of the most important phases in trees is deciding on what is the best combination of parameters that control the optimal values for

- depth of the tree,
- min number of samples per leaf,
- min number of samples to perform a split,
- max number of features to consider when looking for the best split,
- ...

All these parameters control the growth of the tree, and given the lack of other pruning mechanisms, this is the best way of considering that your tree model gets "**pruned**" to avoid overfitting.

How to perform gridsearch or hyper parameter tuning over Gradient Boosting trees?

```python
parameters = {
    "learning_rate": [0.01, 0.1, 0.5, 1.0],
    "min_samples_split": [0.05, 0.1, 0.5],
    "min_samples_leaf": [0.05, 0.1, 0.5],
    "max_depth":[2, 3, 4],
    "max_features":["log2","sqrt"],
    "criterion": ["friedman_mse",  "mae"],
    "subsample":[0.1, 0.5, 0.8, 1.0],
    "n_estimators":[10, 100, 1000]
    }
```

```python
# Grid search takes 30 or more minutes, so I disable it.
# grid_gb = GridSearchCV(GradientBoostingClassifier(), parameters, cv=10, n_jobs=-1)
# grid_gb.fit(X.train, column_or_1d(Y.train))
# score = grid_gb.score(X.train, column_or_1d(Y.train))
# best_params = grid_gb.best_params_

# I include the result here to avoid having to run this cell again
score = 0.9773
best_params = {'criterion': 'mae', 'learning_rate': 0.1, 'max_depth': 4, 'max_features': 'log2',
               'min_samples_leaf': 0.1, 'min_samples_split': 0.05, 'n_estimators': 1000, 'subsample': 0.5}

print('Score: {:.4f}'.format(score))
print('Best Params:\n{}'.format(best_params))
```

    Score: 0.9773
    Best Params:
    {'criterion': 'mae', 'learning_rate': 0.1, 'max_depth': 4, 'max_features': 'log2', 'min_samples_leaf': 0.1, 'min_samples_split': 0.05, 'n_estimators': 1000, 'subsample': 0.5}

After a very long run, we find that optimal setup is:

- criterion': 'mae',
- learning_rate': 0.1,
- max_depth': 4,
- max_features': 'log2',
- min_samples_leaf': 0.1,
- min_samples_split': 0.05,
- n_estimators': 1000,
- subsample': 0.5

```python
optimal_gb = GradientBoostingClassifier(n_estimators=1000,
                                        loss='deviance',
                                        min_samples_leaf=0.1,
                                        min_samples_split=0.05,
                                        subsample=0.5,
                                        learning_rate=0.1,
                                        max_features='log2',
                                        max_depth=4,
                                        random_state=0)

opt_gb_train, opt_gb_test = cv_classification(wholesale, optimal_gb)
plot_compare(classification_results, 'opt.boost', opt_gb_train, opt_gb_test);
```


# Table of Contents

1. [Building Causal Models](#Building-Causal-Models)
2. [Model Analysis](#Model-Analysis)
    1. [Causal Effect of Changing Neighborhoods](#Causal-Effect-on-ROI-of-Changing-Neighborhoods---Should-we-choose-North-or-South?)
    2. [Causal Effect of Different BPBs](#Causal-Effect-on-ROI-of-Different-BPBs)
    3. [Given we want a certain Bed and Baths, where should we buy?](#Given-we-want-a-certain-Bed-and-Baths,-where-should-we-buy?)
    4. [Given we want to buy in a certain area, what beds and baths should we buy?](#Given-we-want-to-buy-in-a-certain-area,-what-beds-and-baths-should-we-buy?)
3. [Property Examples](#Property-Examples)
4. [Next Steps](#Next-Steps)

```python
import pyro
import pyro.distributions as dist
from pyro.infer import Importance, EmpiricalMarginal
from statistics import mean
import torch
import pandas as pd
import numpy as np
import statsmodels.api as sm
from statsmodels.formula.api import ols
import statsmodels.stats.multicomp
pyro.set_rng_seed(101)
```

```python
df = pd.read_csv("data/cleansed_data.csv")
```

# Building Causal Models

### Fitting Factor Effects Models for Rent and Zestimate
- In order to get the parameters for the Rent and Zestimate calculations we use factor effects models(aka ANOVA models)

Taken from [this](https://pythonfordatascience.org/anova-2-way-n-way/) tutorial

```python
rent_model = ols('Rent ~ C(BPB)', df).fit()

# Seeing if the overall model is significant
print(f"Overall model F({rent_model.df_model: .0f},{rent_model.df_resid: .0f}) = {rent_model.fvalue: .3f}, p = {rent_model.f_pvalue: .4f}")
```

    Overall model F( 4, 200) =  13.359, p =  0.0000

Model is signficant

```python
res = sm.stats.anova_lm(rent_model, typ=2)
rent_model_se = (res.sum_sq.Residual / res.df.Residual)**0.5
res
```

      sum_sq
      df
      F
      PR(>F)

      C(BPB)
      242221.823818
      4.0
      13.359283
      1.148343e-09

      Residual
      906567.424962
      200.0
      NaN
      NaN

- Do OLS model for Zestimates
- Get Neighborhood Marginal
- Get BPB marginal
- Get ROI function in the code

```python
zest_model = ols('Zest ~ C(BPB) + C(Neigh)', df).fit()

# Seeing if the overall model is significant
print(f"Overall model F({zest_model.df_model: .0f},{zest_model.df_resid: .0f}) = {zest_model.fvalue: .3f}, p = {zest_model.f_pvalue: .4f}")
```

    Overall model F( 5, 199) =  8.702, p =  0.0000

```python
res = sm.stats.anova_lm(zest_model, typ=2)
zest_model_se = (res.sum_sq.Residual / res.df.Residual)**0.5
res
```

      sum_sq
      df
      F
      PR(>F)

      C(BPB)
      9.832538e+11
      4.0
      0.417948
      7.955891e-01

      C(Neigh)
      2.399361e+13
      1.0
      40.795532
      1.170317e-09

      Residual
      1.170405e+14
      199.0
      NaN
      NaN

### Functions for Calculating ROI
- below are the functions and hyperparameters for calculating ROI

```python
# ROI Consts
INFLATION_RATE = 0.028
MORTGAGE_RATE = 0.036
NUM_YEARS = 15
DOWN_PAYMENT = 0.20

def calculate_monthly(P, mortgage_rate, num_years):
    n = num_years * 12
    monthly_i = mortgage_rate / 12
    numerator = monthly_i * (1 + monthly_i) ** n
    denominator = ((1 + monthly_i) ** n) - 1
    return P * numerator / denominator

def airbnb_income(price, inflation_rate, num_years):
    total = 0
    for year_number in range(num_years):
        curr_inflation = (1 + inflation_rate) ** year_number
        total += (price * curr_inflation) * 12
    return total

def roi(zestimate, inflation_rate, mortgage_rate, num_years, rental_price, down_payment_percent):
    down_payment = zestimate * down_payment_percent
    P = zestimate * (1 - down_payment_percent)

    incurred_cost = calculate_monthly(
        P, mortgage_rate, num_years) * 12 * num_years + down_payment

    income = airbnb_income(price=rental_price,
                           inflation_rate=inflation_rate,
                           num_years=num_years)

    return (income - incurred_cost) / incurred_cost
```

### Model 1

- Simpler causal model without exogenous variables

```python
Neigh_alias = ['Neigh_North', 'Neigh_South']
BPB_alias = [2, 3, 4, 5, 6]

north_prob = float(len(df[df['Neigh'] == 'Neigh_North']))/len(df)
bpb_prob = [(df.BPB == a).count()/len(df) for a in BPB_alias]

Neigh_prob = torch.tensor([north_prob, 1 - north_prob])
BPB_prob = torch.tensor(bpb_prob)

def model():
    Neigh = pyro.sample("Neigh", dist.Categorical(probs=Neigh_prob))
    BPB = pyro.sample("BPB", dist.Categorical(probs=BPB_prob))

    bpb_converted = BPB_alias[BPB]
    neigh_converted = Neigh_alias[Neigh]
    rent_pred = rent_model.predict(pd.DataFrame({'BPB': [bpb_converted]}))[0]
    zest_pred = zest_model.predict(pd.DataFrame(
        {'BPB': [bpb_converted], 'Neigh': [neigh_converted]}))[0]

    Rent = pyro.sample("Rent", dist.Delta(torch.tensor(rent_pred)))
    Zest = pyro.sample("Zest", dist.Delta(torch.tensor(zest_pred)))

    ROI = pyro.sample("ROI", dist.Delta(roi(zestimate=Zest,
                                            rental_price=Rent,
                                            inflation_rate=INFLATION_RATE,
                                            mortgage_rate=MORTGAGE_RATE,
                                            num_years=NUM_YEARS,
                                            down_payment_percent=DOWN_PAYMENT)))

    return {'Neigh': Neigh, 'BPB': BPB, 'Rent': Rent, 'Zest': Zest, 'ROI': ROI}
```

### Model 2 - SCM

\begin{align}
Nb &= p(BPB=2)=p_2, p(BPB=3)=p_3, p(BPB=4)=p_4,p(BPB=5)=p_5, p(BPB=6)=p_6 \\
Nr &= \mathcal{N}(0, \text{rent model standard error}) \\
Nz &= \mathcal{N}(0, \text{zest model standard error}) \\
Nn &= Bernoulli(p_{south})\\\\
Neigh &= Nn \\
BPB &= Nb \\
Rent &= bpb_3 \cdot a + bpb_4 \cdot b + bpb_5 \cdot d + bpb_6 \cdot d + e +Nrent\\
Zest &= bpb_3 \cdot a + bpb_4 \cdot b + bpb_5 \cdot d + bpb_6 \cdot d + NeighSouth \cdot e + f + Nz \\
ROI &= \text{ROIfunc}(Rent, Zest)
\end{align}

```python
Neigh_alias = ['Neigh_North', 'Neigh_South']
BPB_alias = [2, 3, 4, 5, 6]

south_prob = float(len(df[df['Neigh'] == 'Neigh_South']))/len(df)
bpb_prob = [(df.BPB == a).count()/len(df) for a in BPB_alias]

# Neigh_prob = torch.tensor([north_prob, 1- north_prob])
BPB_prob = torch.tensor(bpb_prob)

exogenous_dists = {
    'Nn': dist.Bernoulli(torch.tensor(south_prob)),
    'Nb': dist.Categorical(BPB_prob),
    'Nr': dist.Normal(torch.tensor(0.), torch.tensor(rent_model_se)),
    'Nz': dist.Normal(torch.tensor(0.), torch.tensor(zest_model_se))
}

def scm(exogenous_dists):
    Nn = pyro.sample("Nn", exogenous_dists['Nn'])
    Nb = pyro.sample("Nb", exogenous_dists['Nb'])
    Nr = pyro.sample("Nr", exogenous_dists['Nr'])
    Nz = pyro.sample("Nz", exogenous_dists['Nz'])

    z_bounds = torch.tensor(zest_model_se)
    r_bounds = torch.tensor(rent_model_se)

    # Stay within 1 SD of 0
    Nz = min(z_bounds, Nz)
    Nz = max(-2 * z_bounds, Nz)

    Nr = min(r_bounds, Nr)
    Nr = max(-2 * r_bounds, Nr)

    Neigh = pyro.sample("Neigh", dist.Delta(Nn))
    BPB = pyro.sample("BPB", dist.Delta(Nb))

    bpb_converted = BPB_alias[BPB.int()]
    neigh_converted = Neigh_alias[Neigh.int()]

    rent_pred = rent_model.predict(pd.DataFrame({'BPB': [bpb_converted]}))[0]
    zest_pred = zest_model.predict(pd.DataFrame(
        {'BPB': [bpb_converted], 'Neigh': [neigh_converted]}))[0]

    Rent = pyro.sample("Rent", dist.Delta(torch.tensor(rent_pred) + Nr))
    Zest = pyro.sample("Zest", dist.Delta(torch.tensor(zest_pred) + Nz))

    ROI = pyro.sample("ROI", dist.Delta(roi(zestimate=Zest,
                                            rental_price=Rent,
                                            inflation_rate=INFLATION_RATE,
                                            mortgage_rate=MORTGAGE_RATE,
                                            num_years=NUM_YEARS,
                                            down_payment_percent=DOWN_PAYMENT)))

    return {'Neigh': Neigh, 'BPB': BPB, 'Rent': Rent, 'Zest': Zest, 'ROI': ROI}
```

# Model Analysis

### Causal Effect on ROI of Changing Neighborhoods - Should we choose North or South?

$$
P(ROI | do(Neigh=South)) - P(ROI|do(Neigh=North))
$$

```python
south_model = pyro.do(
    model, data={'Neigh': torch.tensor(Neigh_alias.index('Neigh_South'))})
north_model = pyro.do(
    model, data={'Neigh': torch.tensor(Neigh_alias.index('Neigh_North'))})

south_scm = pyro.do(
    scm, data={'Neigh': torch.tensor(Neigh_alias.index('Neigh_South'))})
north_scm = pyro.do(
    scm, data={'Neigh': torch.tensor(Neigh_alias.index('Neigh_North'))})
```

Causal Effect for Model 2(SCM)

```python
scm_south_roi_samples = [south_scm(exogenous_dists)[
    'ROI'].item() for _ in range(2000)]
scm_north_roi_samples = [north_scm(exogenous_dists)[
    'ROI'].item() for _ in range(2000)]

scm_causal_effect_neigh = mean(
    scm_south_roi_samples) - mean(scm_north_roi_samples)
scm_causal_effect_neigh
```

    0.0061561012649907765

Showing the problematic large variance in the samples

```python
mean(scm_south_roi_samples), np.std(scm_south_roi_samples), min(
    scm_south_roi_samples), max(scm_south_roi_samples)
```

    (-0.9635449091708288, 1.032904101041279, -36.795448303222656, 21.0173282623291)

Causal Effect for Model 1

```python
south_roi_samples = [south_model()['ROI'].item() for _ in range(2000)]
north_roi_samples = [north_model()['ROI'].item() for _ in range(2000)]

causal_effect_neigh = mean(south_roi_samples) - mean(north_roi_samples)
causal_effect_neigh
```

    0.013694935142993914

### Results:
- SCM Model has too much variance and so it makes it difficult to get good samples
- Model 1 shows a 1.3% Increase In ROI from Investing in property in the South as opposed to the North
    - Why?
        - South includes: San Jose, Cupertino and other properties a bit further from SF
        - North includes: Mountain View, Palo Alto - properties in the heart of Silicon Valley and closer to SF

### Causal Effect on ROI of Different BPBs

```python
bpb_roi_samples = []
for i in range(len(BPB_alias)):
    bpb = BPB_alias[i]
    bpb_model = pyro.do(model, data={'BPB': torch.tensor(i)})
    bpb_roi_samples.append([bpb_model()['ROI'].item() for _ in range(2000)])
```

```python
for i in range(len(BPB_alias)):
    bpb = BPB_alias[i]
    print(bpb, mean(bpb_roi_samples[i]))
```

    2 -0.9815098011493683
    3 -0.977315743714571
    4 -0.9739876031577587
    5 -0.9663166411519051
    6 -0.9708639974594117

### What about BPB=5 vs BPB=2 ?
$P(ROI | do(BPB=5)) - P(ROI | do(BPB=2))$

```python
mean(bpb_roi_samples[3]) - mean(bpb_roi_samples[0])
```

    0.01519315999746318

### Results
- 3 Beds 2 baths is the best configuration for maximizing ROI
- You can increase ROI by 1.5% by doing 3 Beds 2 Baths over a 1 Bed 1 Bath

### Given we want a certain Bed and Baths, where should we buy?

Example for 2 bed 1 bath:
$$
P(ROI | do(Neigh= North), BPB=3) - P(ROI | do(Neigh=South), BPB=3)
$$

```python
for bpb_index in range(len(BPB_alias)):
    bpb = BPB_alias[bpb_index]
    roi_means = []
    cond_model = pyro.condition(model, {"BPB": bpb_index})

    for neigh_index in range(len(Neigh_prob)):
        curr_model = pyro.do(cond_model, {'Neigh': neigh_index})
        roi_mean = curr_model()['ROI'].item()
        roi_means.append(roi_mean)

    print("𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=%d)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=%d) = %.3f - %.3f = %.3f" % (bpb, bpb,
                                                                                roi_means[0],
                                                                                roi_means[1],
                                                                                roi_means[0] -
                                                                                roi_means[1]
                                                                                ))
```

    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=2)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=2) = -0.985 - -0.976 = -0.009
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=3)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=3) = -0.983 - -0.970 = -0.013
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=4)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=4) = -0.980 - -0.966 = -0.014
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=5)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=5) = -0.974 - -0.956 = -0.019
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑁𝑜𝑟𝑡ℎ),𝐵𝑃𝐵=6)−𝑃(𝑅𝑂𝐼|𝑑𝑜(𝑁𝑒𝑖𝑔ℎ=𝑆𝑜𝑢𝑡ℎ),𝐵𝑃𝐵=6) = -0.977 - -0.964 = -0.013

### Results
- No matter which BPB you are looking at, it is always better to pick a property in the South rather than the North

### Given we want to buy in a certain area, what beds and baths should we buy?

Example for the North Neighborhood
$$
P(ROI | do(BPB= x), Neigh= North) - P(ROI | do(BPB=y), Neigh= North)
$$

```python
for neigh_index in range(len(Neigh_prob)):
    roi_means = []
    cond_model = pyro.condition(model, {"Neigh": neigh_index})
    neigh = Neigh_alias[neigh_index]
    print(neigh)

    for bpb_index in range(len(BPB_prob)):
        bpb = BPB_alias[bpb_index]
        curr_model = pyro.do(cond_model, {'BPB': bpb_index})
        roi_mean = curr_model()['ROI'].item()
        print("𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=%s),𝑁𝑒𝑖𝑔ℎ=%s) = %.3f" % (bpb, neigh, roi_mean))
```

    Neigh_North
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=2),𝑁𝑒𝑖𝑔ℎ=Neigh_North) = -0.985
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=3),𝑁𝑒𝑖𝑔ℎ=Neigh_North) = -0.983
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=4),𝑁𝑒𝑖𝑔ℎ=Neigh_North) = -0.980
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=5),𝑁𝑒𝑖𝑔ℎ=Neigh_North) = -0.974
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=6),𝑁𝑒𝑖𝑔ℎ=Neigh_North) = -0.977
    Neigh_South
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=2),𝑁𝑒𝑖𝑔ℎ=Neigh_South) = -0.976
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=3),𝑁𝑒𝑖𝑔ℎ=Neigh_South) = -0.970
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=4),𝑁𝑒𝑖𝑔ℎ=Neigh_South) = -0.966
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=5),𝑁𝑒𝑖𝑔ℎ=Neigh_South) = -0.956
    𝑃(𝑅𝑂𝐼|𝑑𝑜(𝐵𝑃𝐵=6),𝑁𝑒𝑖𝑔ℎ=Neigh_South) = -0.964

### Results
- If you are interested in purchasing a property in the North, it should be a 3 bed 2 bath. If you are interested in purchasing a property in the South it should also be a 3 bed 2 bath.

# Property Examples
- here are the properties with the highest ROIs
- We can see that these properties follow the effects we have seen in our model
    - 5 BPB, all are South Neighborhood

```python
df.sort_values(["ROI"], ascending=False)[:5]
```

      Beds
      Baths
      Rent
      neighbourhood
      latitude
      longitude
      Type
      Zest
      ROI
      Neigh
      BPB
      latlng

      27
      2
      2
      140.0
      San Jose
      37.40075
      -121.92194
      Townhouse
      214665.0
      -0.883995
      Neigh_South
      4
      -4559.971997

      110
      3
      2
      550.0
      San Jose
      37.31737
      -121.78764
      Townhouse
      998982.0
      -0.902070
      Neigh_South
      5
      -4544.794423

      115
      2
      2
      275.0
      San Jose
      37.31392
      -121.93997
      Condominium
      513454.0
      -0.904733
      Neigh_South
      4
      -4550.058285

      16
      2
      2
      499.0
      San Jose
      37.31996
      -121.94828
      Condominium
      1210000.0
      -0.926646
      Neigh_South
      4
      -4551.104932

      17
      3
      2
      375.0
      San Jose
      37.39220
      -121.86552
      Condominium
      930000.0
      -0.928277
      Neigh_South
      5
      -4556.819897

- [Property 1](https://www.zillow.com/homedetails/435-Milan-Dr-San-Jose-CA-95134/2143393753_zpid/)
- [Property 2](https://www.zillow.com/homedetails/3215-Lamond-Ct-San-Jose-CA-95148/19793643_zpid/)
- [Property 5](https://www.zillow.com/homedetails/2409-Venturi-Pl-UNIT-2-San-Jose-CA-95132/124743156_zpid/)

# Next Steps

1. **Get more data**
2. Consider adding new nodes to the DAG
    1. Individual Neighborhoods as opposed to North and South
    2. Host and ratings
    3. Distance to transit
3. Reconsider new edges with our new data
3. Test more complex functions to fit Rent and Zestimate
    1. Currently using factor effects models
4. Attempt to use SCM again
    1. Our models have high Std Error so using SCM is difficult
    2. Increasing the amount of data would also decrease Std. Error and help give better results from SCM