# Numpy, Estadística, Probabilidades

## Estadística

En esta parte vamos a revisar conceptos de estadística descriptiva.

La estadística descriptiva busca describir, sumarizar y comprender los datos.

Para ello empleamos Medidas de Tendencia Central, y medidas de Variabilidad.

La función de las Medidas de Tendencia Central es proveer información descriptiva sobre el valor numérico que es considerado el más usual para una variable cuantitativa.

Las medidas de tendencia central son
* **Media**

Dados los n números ${\{x_{1},x_{2},\ldots ,x_{n}\}}$ la media aritmética se define como:

\begin{equation}
    \bar {x} = {\frac {1}{n}} \sum _{i=1}^{n}x_{i} ={\frac {x_{1}+x_{2}+\cdots +x_{n}}{n}}
\end{equation}

* **Mediana**

Sean ${ x_{1},x_{2},x_{3},\ldots ,x_{n} }$ los datos de una muestra ordenada en orden creciente y designando la mediana como $M_{e}$, distinguimos dos casos:

a) Si n es impar, la mediana es el valor que ocupa la posición $(n+1)/2$ una vez que los datos han sido ordenados (en orden creciente o decreciente), porque este es el valor central. Es decir: $M_{e}=x_{{(n+1)/2}}$.

b) Si n es par, la mediana es la media aritmética de los dos valores centrales. Cuando $n$ es par, los dos datos que están en el centro de la muestra ocupan las posiciones $n/2$ y $(n/2)+1$. Es decir: $M_{e}=(x_{{{\frac{n}{2}}}}+x_{{{{\frac{n}{2}}}+1}})/2$.

* **Moda**

Es el valor del dato con mayor frecuencia en un conjunto.

Las medidas de variabilidad o dispersión nos indican si los valores de los datos están próximos entre sí o si
por el contrario están muy dispersos. Estas medidas se determinan en función de la distancia entre los datos y algún estadístico de tendencia central.

Las medidas de variabilidad son
* **Rango**

Sean ${ x_{1},x_{2},x_{3},\ldots ,x_{n} }$ los datos de una muestra ordenada en orden creciente, el rango es $x_{n} - x_{1}$

* **Varianza**

Si tenemos un conjunto de valores de una variable, la varianza se calcula de la siguiente forma:

\begin{equation}
\sigma_{n}^{2} = \frac{1}{n} \sum _{i=1}^{n} (x_i - \bar{X})^{2}
\end{equation}

Siendo:

$x_{i}$: cada dato

$\bar{X}$: media de los datos

$n$: número de datos

* **Desvío Estándar**

Es la raíz cuadrada de la varianza

\begin{equation}
\sigma_{n} = \sqrt{\frac{1}{n} \sum _{i=1}^{n} (x_i - \bar{X})^{2}}
\end{equation}

* **Coeficiente de Variación**

Es el desvío estándar dividido por la media

\begin{equation}
CV = \frac{\sigma_{n}}{\bar{X}} . 100
\end{equation}

NumPy tiene funciones para calcular todas estas medidas.

---

Vamos a ver cómo calcular algunas medidas de estadística descriptiva usando Numpy.

Para eso, usaremos datos que tenemos en un csv con tres poblaciones:
* conejos
* linces
* zanahorias

Leeremos los datos del csv usando `genfromtxt` de Numpy

La primera columna de la matriz corresponde al año, la segunda a la población de conejos, la tercera corresponde a la población de linces, y la cuarta a la población de zanahorias.

Y luego vamos a convertir en arrays, que aprendimos a usar recientemente, cada una de sus columnas. ¿Cómo hacemos esto?

```python
import numpy as np

data_location = '../Data/populations.txt'
data = np.genfromtxt(data_location, skip_header=1, delimiter='\t')
data
```

Definimos variables con el índice que corresponde a cada población:

```python
anno_col_index  = 0
conejos_col_index = 1
linces_col_index = 2
zanahorias_col_index = 3
```

Vamos a crear un array de Numpy para cada población y para el año, usando slicing (si no lo recuerdan, revisen la clase  de Numpy):

```python
anno = data[:, anno_col_index]
#print(anno)
conejos = data[:, conejos_col_index]
#print(conejos)
linces = data[:, linces_col_index]
#print(linces)
zanahorias = data[:, zanahorias_col_index]
#print(zanahorias)
```

Y por último vamos crear una variable poblaciones, que tenga todos los datos de la matriz data menos la columna "# year"

```python
poblaciones = data[:, 1:]
poblaciones
```

### Estadística descriptiva
[volver a TOC](#section_toc)

Para cada variable de la serie histórica, calculamos la media y el desvío. Redondeamos los valores para que tengan sólo 2 decimales.

Vamos a usar los métodos de Numpy
* **mean**: calcula la media de los valores pasados en el primer parámetro https://docs.scipy.org/doc/numpy/reference/generated/numpy.mean.html
* **std**: calcula el desvio standard de los valores pasados en el primer parámetro  https://docs.scipy.org/doc/numpy/reference/generated/numpy.std.html
* **around**: redondea los valores a la cantidad de decimales pasada como parámetro, el default es 0 https://docs.scipy.org/doc/numpy/reference/generated/numpy.around.html

Ahora vamos a calcular la media de cada una de las poblaciones usando la matriz `poblaciones` y el método `mean` de Numpy, redondenado a dos decimales.

¿Qué valor de `axis` debemos usar? ¿Por qué?

Respuesta: Cada población está representada por una columna en la matriz poblaciones, por lo tanto para calcular la media por población debemos colapsar las filas, y para eso el valor de axis debe ser 0

```python
print ("        Conejos, Linces, Zanahorias")
print ("Mean:", np.around(poblaciones.mean(axis=0), decimals=2))
print ("Std:", np.around(poblaciones.std(axis=0), decimals=2))
```

A continuación, calculamos para cada especie el año en el que tuvieron máxima población.

Para eso vamos a usar el método `argmax` de Numpy que devuelve el índice en el array del valor máximo https://docs.scipy.org/doc/numpy/reference/generated/numpy.argmax.html

Igual que en el punto anterior, las poblaciones están representadas por columnas, entonces debemos colapsar filas y así axis debe tener valor 0.

`indice_max_poblacion` tendrá tantos elementos como poblaciones considero (3 en este caso: conejos, linces, zanahorias) y cada elemento será **el índice de la fila** donde esa población tuvo la máxima cantidad de individuos.

```python
indice_max_poblacion = np.argmax(poblaciones, axis=0)
indice_max_poblacion
```

Por lo tanto, la población de conejos fue máxima en la cuarta fila, la de linces en la quinta fila, la de zanahorias en la primer fila.

Veamos a que años corresponden esas filas, usando Fancy indexing (si no lo recuerdan, por favor revisen la guía de numpy1!)

```python
annos_con_maximos = anno[indice_max_poblacion]
annos_con_maximos
```

Entonces la máxima población de conejos fue en 1903, la de linces fue en 1904, y la de zanahorias fue en 1900

```python
# Fancy Indexing
print ("                          Conejos, Linces, Zanahorias")
print ("Años de máxima población:", anno[indice_max_poblacion])
```

Como adicional, vamos a graficar estas tres poblaciones para ver si los resultados que obtuvimos se corresponden con los graficados.

En unos días vamos a tener una clase de visualización y veremos en detalle bibliotecas como `matplotlib` que usamos ahora de forma muy básica.

```python
%matplotlib inline
import matplotlib.pyplot as plt
plt.plot(anno, conejos, anno, linces, anno, zanahorias)
plt.legend(('Conejos (hares)', 'Linces (lynxes)', 'Zanahorias (carrots)'), loc=(1.05, 0.5))
plt.show()
```

Ejercicio:

¿En qué años alguna de las poblaciones se encuentra por encima de 50000?

Ayuda: boolean indexing

Ayuda 2: podemos resolverlo usando la matriz poblaciones o usando los array de poblaciones por separado

Podemos autocorregir los ejercicios mirando el gráfico

Ejercicio:

¿En qué dos años, cada especie tuvo sus niveles más bajos de población?

Ayuda: pueden buscar la documentación de `argsort` de Numpy

Ayuda 2: podemos resolverlo usando la matriz poblaciones o usando los array de poblaciones por separado

Podemos autocorregir los ejercicios mirando el gráfico

#### Covarianza

La covarianza es un valor que indica el grado de variación conjunta de dos variables aleatorias respecto a sus medias.

Es el dato básico para determinar si existe una dependencia entre ambas variables

El método que calcula la covarianza en Numpy es `cov`
https://docs.scipy.org/doc/numpy/reference/generated/numpy.cov.html

```python
np.cov([conejos, linces, zanahorias])
```

Ahora intentemos responder estas preguntas:

* **¿Puedo comparar las distintas varianzas? ¿Por qué?**

Si las variables se miden en las misma unidades y sus valores son de magnitudes parecidas, puedo comparar sus varianzas. Pero sólo en ese caso y no en general.

* **¿Qué variable tiene la mayor varianza? ¿Cómo se ve esto en el gráfico?**

Podemos mirar los valores de la diagonal de la matriz de covarianzas para obtener los valores de las varianzas de esas variables.

La variable de mayor varianza es el conejo, su valor está en el elemento (0,0) de la matriz de varianzas y covarianzas.

En el gráfico podemos ver que la dispersión de la serie de los conejos es parecida a la serie de los linces, pero la serie de conejos tiene mayor rango (diferencia entre los valores máximo y mínimo) por lo tanto aporta más en la suma de distancias a la media.

* **¿Qué significa una covarianza positiva? ¿Y una negativa?**

Dos variables X e Y, tienen covarianza positiva cuando tienden a encontrarse por encima de su media al mismo tiempo y tienen covarianza negativa cuando al mismo tiempo, tienden a encontrarse una por debajo y otra por encima.

En cambio X e Y tienen covarianza cercana a cero cuando las variables pueden encontrarse por encima o por debajo de su media independientemente de lo que haga la otra.

La covarianza mide la relación lineal entre ambas variables, es decir, qué tanto se asemeja la relación con una función lineal.

#### Correlación

El coeficiente de correlación de Pearson es una medida lineal entre dos variables aleatorias cuantitativas.

A diferencia de la covarianza, la correlación de Pearson es independiente de la escala de medida de las variables.

Podemos definir el coeficiente de correlación de Pearson como un índice que puede utilizarse para medir el grado de relación de dos variables siempre y cuando ambas sean cuantitativas.

El método que calcula la correlación de Pearson en Numpy es `corrcoef`. Los valores de los elementos de la matriz de correlación están entre -1 y 1 inclusive. Es equivalente a la matriz de covarianzas normalizada.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.corrcoef.html

```python
np.corrcoef([conejos, linces, zanahorias])
```


# Problem Sheet 1 Question 2a

The general form of the population growth differential equation
\begin{equation} y^{'}=t-y, \ \ (0 \leq t \leq 4), \end{equation}
with the initial condition
\begin{equation}y(0)=1.\end{equation}
For N=4
with the analytic (exact) solution
\begin{equation} y= 2e^{-t}+t+1.\end{equation}

Apply __Euler’s Method__ to approximate the solution of the given initial value problems using the indicated number of time steps. Compare the approximate solution with the given exact solution, and compare the actual error with the theoretical error.

```python
## Library
import numpy as np
import math

%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import matplotlib.gridspec as gridspec # subplots
import warnings
import pandas as pd

warnings.filterwarnings("ignore")
```

## General Discrete Interval
The continuous time $a\leq t \leq b $ is discretised into $N$ points seperated by a constant stepsize
\begin{equation} h=\frac{b-a}{N}.\end{equation}
## Specific Discrete Interval
Here the interval is $0\leq t \leq 4$ with $N=4$
\begin{equation} h=\frac{4-0}{4}=1.\end{equation}
This gives the 5 discrete points with stepsize h=1:
\begin{equation} t_0=0, \ t_1=1, \ ... t_{4}=4. \end{equation}
This is generalised to
\begin{equation} t_i=0+i1, \ \ \ i=0,1,...,4.\end{equation}
The plot below illustrates the discrete time steps from 0 to 4.

```python
### Setting up time
t_end=4
t_start=0
N=4
h=(t_end-t_start)/(N)
t=np.arange(t_start,t_end+0.01,h)
fig = plt.figure(figsize=(10,4))
plt.plot(t,0*t,'o:',color='red')
plt.plot(t[0],0*t[0],'o',color='green')

plt.title('Illustration of discrete time points for h=%s'%(h))
plt.plot();
```

## Euler Solution

The Euler difference equation is given by
\begin{equation} w_{i+1}=w_i+h(t_i-w_i) \end{equation}
for $i=0,1,2,3$ with the initial condition,
\begin{equation}w_0=1.\end{equation}

```python
IC=1

INTITIAL_CONDITION=IC
w=np.zeros(N+1)
w[0]=INTITIAL_CONDITION
for i in range (0,N):
    w[i+1]=w[i]+h*(t[i]-w[i])
```

```python
y=2*np.exp(-t)+t-1 # Exact Solution
```

## Global Error
The upper bound of the global error is:
\begin{equation} |y(t_i)-w_i|\leq\frac{Mh}{2L} |e^{Lt_i}-1| \end{equation}
where
$|y''(t)|\leq M $ for $t \in (0,4)$ and $L$ is the Lipschitz constant.
Generally we do not have access to $M$ or $L$, but as we have the exact solution we can find $M$ and $L$.
### Local Error
The global error consistent of an exponential term and the local error.
The local error for the Euler method is
\begin{equation}
|y(t_1)-w_1|<\tau=\frac{Mh}{2}, \end{equation}

by differentiation the exact solution
$ y= 2e^{-t}+t+1$
twice
\begin{equation}
y'=-2e^{-t}+1,\end{equation}
\begin{equation}
y''=2e^{-t}.\end{equation}
Then we choose the largest possible value of $y''$ on the interval of interest $(0,4)$, which gives,

\begin{equation}|y''(t)|\leq 2e^{0}=2, \end{equation}
therefore
\begin{equation}
M=2.\end{equation}
This gives the local error
\begin{equation}
 |y(t_1)-w_1|<\tau=\frac{Mh}{2}=\frac{2h}{2}=h.\end{equation}
### Lipschitz constant
The Lipschitz constant $L$ is from the Lipschitz condition,
\begin{equation}
\left| \frac{\partial f(t,y)}{\partial y}\right|\leq L.\end{equation}
The constant can be found by taking partical derivative of $f(t,y)=t-y$ with respect to $y$
\begin{equation}
\frac{\partial f(t,y)}{\partial y}=-1,\end{equation}
\begin{equation}L=|-1|=1.\end{equation}

### Specific Global Upper Bound Formula
Puting these values into the upper bound gives:
\begin{equation} |y(t_i)-w_i|\leq\frac{2h}{2} |e^{t_i}-1|. \end{equation}

```python
Upper_bound=2*h/(2*1)*(np.exp(t)-1) # Upper Bound
```

## Results

```python
fig = plt.figure(figsize=(12,5))
# --- left hand plot
ax = fig.add_subplot(1,4,1)
plt.plot(t,w,'o:',color='k')
plt.plot(t[0],w[0],'o',color='green')

#ax.legend(loc='best')
plt.title('Numerical Solution (w) h=%s'%(h))

# --- right hand plot
ax = fig.add_subplot(1,4,2)
plt.plot(t,y,color='blue')
plt.title('Analytic Solution (y)')

#ax.legend(loc='best')
ax = fig.add_subplot(1,4,3)
plt.plot(t,np.abs(w-y),'o:',color='red')
plt.title('Exact Error |w-y|')

# --- title, explanatory text and save

ax = fig.add_subplot(1,4,4)
plt.plot(t,np.abs(w-y),'o:',color='red')
plt.plot(t,Upper_bound,color='purple')
plt.title('Upper Bound')

# --- title, explanatory text and save
fig.suptitle(r"$y'=t-y$", fontsize=20)
plt.tight_layout()
plt.subplots_adjust(top=0.85)
```

```python

d = {'time t_i': t,    'Euler (w_i) ':w,    'Exact (y_i) ':y,    'Exact Error( |y_i-w_i|) ':np.abs(y-w),'Upper Bound':Upper_bound}
df = pd.DataFrame(data=d)
df
```

      time t_i
      Euler (w_i)
      Exact (y_i)
      Exact Error( |y_i-w_i|)
      Upper Bound

      0
      0.0
      1.0
      1.000000
      0.000000
      0.000000

      1
      1.0
      0.0
      0.735759
      0.735759
      1.718282

      2
      2.0
      1.0
      1.270671
      0.270671
      6.389056

      3
      3.0
      2.0
      2.099574
      0.099574
      19.085537

      4
      4.0
      3.0
      3.036631
      0.036631
      53.598150

```python

```

Chapter 01 - Euler Methods/1_Problem Sheet/102a_Problem_Sheet.ipynb


# Intro to neural net training with autograd

In this notebook, we'll practice

* using the **autograd** Python package to compute gradients
* using gradient descent to train a basic linear regression (a NN with 0 hidden layers)
* using gradient descent to train a basic neural network for regression (NN with 1+ hidden layers)

### Requirements:

Follow the Python environment setup instructions here:
<https://www.cs.tufts.edu/comp/150BDL/2019f/setup_python_env.html>

All the specific Python packages you'll need are in the 'bdl_2019_env' conda environment:
<https://www.cs.tufts.edu/comp/150BDL/2019f/conda_envs/bdl_2019_env.yml>

```python
import pickle
import copy
import time
```

```python
## Import plotting tools

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
```

```python
## Import numpy
import numpy as np
import pandas as pd
```

```python
## Import autograd
import autograd.numpy as ag_np
import autograd
```

```python
f = np.cos
g = autograd.grad(f)
print(g(0.5))
```

# PART 1: Using autograd's 'grad' function on univariate functions

Suppose we have a mathematical function of interest $f(x)$. For now, we'll work with functions that have a scalar input and scalar output.

Then we can of course ask: what is the derivative (aka *gradient*) of this function:

$$
g(x) \triangleq \frac{\partial}{\partial x} f(x)
$$

Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.

First, we need to implement the math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': scalar float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

```python
def f(x):
    return ag_np.square(x)
```

## Create a gradient function via `autograd.grad`

Now, if `f` meeds the above requirements, we can create a Python function `g` to compute derivatives of $f(x)$ by calling `autograd.grad`:

```
g = autograd.grad(f)
```

The symbol `g` is now a **Python function** that takes the same input as `f`, but produces the derivative at a given input.

```python
g = autograd.grad(f)
```

## Using the gradient function

'g' is just a function. You can call it as usual, by providing a possible scalar float input

### Example: What is the gradient of x^2 at x=0?

```python
g(0.0)
```

### Example: What is the gradient of x^2 at x=16?

```python
g(16.0)
```

### Example: what if we provide an int type as input, not a float?

```python
g(16)
```

### Example: we can use the results of gradient function calls inside Python lists

```python
[g(-1.0), g(1.0)]

```

### Example: we can use the results of gradient calls inside Python dictionaries

```python
dict(gradient_at_4=g(4.0))
```

## Important: A Note on Importing autograd

* You might be used to importing numpy as `import numpy as np`, and then using this shorthand for `np.cos(0.0)` or `np.square(5.0)` etc.
* For autograd to work, you need to instead use **autograd's** provided numpy wrapper interface: `import autograd.numpy as ag_np`
* The `ag_np` module has the same API as `numpy`, so you can call `ag_np.cos(0.0)`, `ag_np.square(5.0)`, etc.
* For this course, we like to be careful and specify when you need the `ag_np` module and when you can get away with `np`.

## Plot to demonstrate the gradient function  side-by-side with original function

```python
x_grid_G = np.linspace(-10, 10, 100)

fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-')
subplot_grid[0,0].set_title('f(x) = x^2')

subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-')
subplot_grid[0,1].set_title('gradient of f(x)'); # trailing semi-colon eats any stdout from this line
```

## Exercise 1a:

Consider the decaying periodic function below. Can you compute its derivative using autograd and plot the result?

$$
f(x) = e^{-x/10} * cos(x)
$$

```python
def f(x):
    return 0.0 # TODO compute the function above! Should you use "np" or "ag_np"
```

```python
g = lambda x: 0.0 # TODO define g as gradient of f, using autograd
```

```python
# TODO plot the result
x_grid_G = np.linspace(-10, 10, 500)
fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)

## Left panel
subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-');
subplot_grid[0,0].set_title('f(x) = e^{-x/10} * cos(x)');

## Right panel
subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-');
subplot_grid[0,1].set_title('gradient of f(x)');
```

# PART 2: Using autograd's 'grad' function on functions with multivariate input

Now, imagine the input $x$ could be a vector of size D.

Our mathematical function $f(x)$ will map each input vector to a scalar.

We want the gradient function

\begin{align}
g(x) &\triangleq \nabla_x f(x)
\\
&= [
    \frac{\partial}{\partial x_1} f(x)
    \quad \frac{\partial}{\partial x_2} f(x)
    \quad \ldots \quad \frac{\partial}{\partial x_D} f(x)  ]
\end{align}

Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.

First, we implement math function $f(x)$ as a **Python function** `f`.

The Python function `f` needs to satisfy the following requirements:
* INPUT 'x': numpy array of float
* OUTPUT 'f(x)': scalar float
* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy

```python
def f(x_D):
    return ag_np.sum(ag_np.square(x_D))
```

## Create a gradient function via `autograd.grad`

Again, we can now create a gradient function by calling autograd:

```python
g = autograd.grad(f)
```

```python
x_D = np.zeros(4)
print(x_D)
print(f(x_D))
print(g(x_D))
```

```python
x_D = np.asarray([1., 2., 3., 4., 5., 6.])
print(x_D)
print(f(x_D))
print(g(x_D))
```

## Concept Check: Does the dimensionality of the above gradients make sense?

## Check Yourself: What do you predict the answer will be? What happened instead?

```python
x_D = np.asarray([11, -13])
print(x_D)
print(f(x_D))
print(g(x_D))
```

# Part 3: Using autograd gradients within gradient descent to solve multivariate optimization problems

### Helper function: basic gradient descent

Here's a very simple function that will perform many gradient descent steps to optimize a given function.

Make sure you understand its basic properties (the gradient descent algorithm is one of the prereqs of this course).

Quick refresher: <https://en.wikipedia.org/wiki/Gradient_descent#Description>

```python
def run_many_iters_of_gradient_descent(f, g, init_x_D=None, n_iters=100, step_size=0.001):

    # Copy the initial parameter vector
    x_D = copy.deepcopy(init_x_D)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        x_D=[],
        g_D=[])

    # Perform many iterations of gradient descent updates
    for iter_id in range(n_iters):
        if iter_id > 0:
            # Gradient descent update
            x_D = x_D - step_size * g(x_D)

        # After each update, store some info about the current state
        history['iter'].append(iter_id)
        history['f'].append(f(x_D))
        history['x_D'].append(x_D)
        history['g_D'].append(g(x_D))
    return x_D, history
```

## Worked Example 3a: Minimize f(x) = sum(square(x))

It's easy to figure out that the vector with smallest L2 norm (smallest sum of squares) is the all-zero vector.

Here's a quick example of showing that using gradient functions provided by autograd can help us solve the optimization problem:

$$
\min_x  \sum_{d=1}^D x_d^2
$$

```python
def f(x_D):
    return ag_np.sum(ag_np.square(x_D))

g = autograd.grad(f)

# Initialize at x_D = [-3, 4, -5, 6]
init_x_D = np.asarray([-3.0, 4.0, -5.0, 6.0])
```

```python
opt_x_D, history = run_many_iters_of_gradient_descent(f, g, init_x_D, n_iters=1000, step_size=0.01)
```

```python
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: f goes to zero. all x values goto zero.

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
subplot_grid[0,0].plot(history['iter'], history['x_D'])
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

## Try it Example 3b: Minimize the 'trid' function

Given a 2-dimensional vector $x = [x_1, x_2]$, the trid function is:

$$
f(x) = (x_1-1)^2 + (x_2-1)^2 - x_1 x_2
$$

Background and Picture: <https://www.sfu.ca/~ssurjano/trid.html>

Can you use autograd + gradient descent to find the optimal value $x^*$ that minimizes $f(x)$?

You can initialize your gradient descent at [+1.0, -1.0]

```python
def f(x_D):
    return 0.0 # TODO

g = f # TODO
```

```python
# TODO call run_many_iters_of_gradient_descent() with appropriate args
```

```python
# TRID example
# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations
# Expected result: ????

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
subplot_grid[0,0].plot(history['iter'], history['x_D'])
subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('x_d')

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('f(x)');
```

# Part 4: Solving linear regression with gradient descent + autograd

## Toy Data for linear regression task

We'll generate $N$ data examples (x_i, y_i)$ consisting of D-dimensional 'input' vectors $x_i$ and scalar outputs $y_i$.

We'll use an idealized linear regression model.

Each example has D=2 dimensions for x.

The first dimension is weighted by +1.337
The second dimension is weighted by -1.337

The true "bias" will be 2.0

```python
N = 300
D = 2
sigma = 0.1

true_w_D = np.asarray([1.337, -1.337])
true_bias = 10.0

train_prng = np.random.RandomState(0)

## Input data: uniform between (-5, 5)
x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))

## True response: output of our "true" linear model
f_N = np.dot(x_ND, true_w_D) + true_bias

## Add small Gaussian noise
noise_N = sigma * train_prng.randn(N)
y_N = f_N + noise_N
```

## Toy Data Visualization: Pairplots for all possible (x, y) scalar combinations

We can
* x1 vs y plot: slope is around +1 (should be +1.337)
* x2 vs y plot: slope is around -1 (should be -1.337)
* all y plots: average value (bias) is around +10.0

```python
g = sns.pairplot(
    data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));

def hide_current_axis(*args, **kwds):
    plt.gca().set_visible(False)
g.map_upper(hide_current_axis);
```

```python
## Try It Example 4a: Solve using a weights-only representation
```

For now, even though the "true bias" is large, we'll consider only the weight coefficients (and assume the bias is zero)

Consider the multivariate linear regression likelihood:

\begin{align}
y_n &\sim \mathcal{N}(w^T x_n, \sigma^2), \forall n \in 1, 2, \ldots N
\end{align}
where we assume $\sigma = 0.1$.

One non-Bayesian way to train weights would be to just compute the maximum likelihood solution:

\begin{align}
\min_{w \in \mathbb{R}^2}  - \log p(y | w, x)
\end{align}

(Remember, maximizing likelihood  is the same as minimizing negative likelihood)

## Setup the optimization problem

```python
# Define the optimization problem as an AUTOGRAD-able function wrt the weights w_D
def calc_neg_likelihood_linreg(w_D):
    return 0.5 / ag_np.square(sigma) * ag_np.sum(ag_np.square(ag_np.dot(x_ND, w_D) - y_N))
```

```python
## Test the function at an easy initial point
init_w_D = np.zeros(2)
calc_neg_likelihood_linreg(init_w_D)
```

```python
## Test the gradient at that easy point
calc_grad_wrt_w = autograd.grad(calc_neg_likelihood_linreg)
calc_grad_wrt_w(init_w_D)
```

## Perform gradient descent!

```python
# Because the gradient's magnitude is very large, use very small step size
opt_w_D, history = run_many_iters_of_gradient_descent(
    calc_neg_likelihood_linreg, autograd.grad(calc_neg_likelihood_linreg), init_w_D,
    n_iters=300, step_size=0.000001,
    )
```

```python
# LinReg worked example
# Make plots of how w_D parameter values evolve over iterations, and function values evolve over iterations
# Expected result: x

fig_h, subplot_grid = plt.subplots(
    nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)
subplot_grid[0,0].plot(history['iter'], [w_D[0] for w_D in history['x_D']], label='w_1')
subplot_grid[0,0].plot(history['iter'], [w_D[1] for w_D in history['x_D']], label='w_2')

subplot_grid[0,0].set_xlabel('iters')
subplot_grid[0,0].set_ylabel('w_d')
subplot_grid[0,0].legend()
subplot_grid[0,0].set_ylim([-3, 3])

subplot_grid[0,1].plot(history['iter'], history['f'])
subplot_grid[0,1].set_xlabel('iters')
subplot_grid[0,1].set_ylabel('-1 * log p(y | w, x)');
```

## Try it Example 4b: Solve the linear regression problem using a weights-and-bias representation

The above example only uses weights on the dimensions of $x_i$, and thus can only learn linear models that pass through the origin.

Can you instead optimize a model that **also** includes a **bias** term $b>0$?

\begin{align}
y_i &\sim \mathcal{N}(w^T x_i + b, \sigma^2), \forall i \in 1, 2, \ldots N
\end{align}
where we assume $\sigma = 0.1$.

One non-Bayesian way to train weights would be to just compute the maximum likelihood solution:

\begin{align}
\min_{w,b}  - \log p(y | w, b, x)
\end{align}

An easy way to do this is to imagine that each observation vector $x_i$ is expanded into a $\tilde{x}_i$ that contains a column of all ones.  Then, we can write the corresponding expanded weights as $\tilde{w} = [w_1 w_2 b]$.

\begin{align}
\min_{\tilde{w} \in \mathbb{R}^3}  - \log p(y | \tilde{w},\tilde{x})
\end{align}

```python
# Now, each expanded xtilde vector has size E = D+1 = 3

xtilde_NE = np.hstack([x_ND, np.ones((N,1))])
```

```python
# TODO: Define f to minimize that takes a COMBINED weights-and-bias vector wtilde_E of size E=3
```

```python
# TODO: Compute gradient of f
```

```python
# TODO run gradient descent and plot the results
```

# Part 5 setup: Autograd for functions of data structures of arrays

#### Useful Fact: autograd can take derivatives with respect to DATA STRUCTURES of parameters

This can help us when it is natural to define models in terms of several parts (e.g. NN layers).

We don't need to turn our many model parameters into one giant weights-and-biases vector. We can express our thoughts more naturally.

### Demo 1: gradient of a LIST of parameters

```python
def f(w_list_of_arr):
    return ag_np.sum(ag_np.square(w_list_of_arr[0])) + ag_np.sum(ag_np.square(w_list_of_arr[1]))

g = autograd.grad(f)
```

```python
w_list_of_arr = [np.zeros(3), np.arange(5, dtype=np.float64)]

print("Type of the gradient is: ")
print(type(g(w_list_of_arr)))

print("Result of the gradient is: ")
g(w_list_of_arr)
```

### Demo 2: gradient of DICT of parameters

```python
def f(dict_of_arr):
    return ag_np.sum(ag_np.square(dict_of_arr['weights'])) + ag_np.sum(ag_np.square(dict_of_arr['bias']))
g = autograd.grad(f)
```

```python
dict_of_arr = dict(weights=np.arange(5, dtype=np.float64), bias=4.2)

print("Type of the gradient is: ")
print(type(g(dict_of_arr)))

print("Result of the gradient is: ")
g(dict_of_arr)
```

# Part 5: Neural Networks and Autograd

### Let's use a convenient data structure for NN model parameters

Use a list of dicts of arrays.

Each entry in the list is a dict that represents the parameters of one "layer".

Each layer-specific dict has two named attributes: a vector of weights 'w' and a vector of biases 'b'

#### Here's a function to create NN params as a 'list-of-dicts' that match a provided set of dimensions

```python
def make_nn_params_as_list_of_dicts(
        n_hiddens_per_layer_list=[5],
        n_dims_input=1,
        n_dims_output=1,
        weight_fill_func=np.zeros,
        bias_fill_func=np.zeros):
    nn_param_list = []
    n_hiddens_per_layer_list = [n_dims_input] + n_hiddens_per_layer_list + [n_dims_output]

    # Given full network size list is [a, b, c, d, e]
    # For loop should loop over (a,b) , (b,c) , (c,d) , (d,e)
    for n_in, n_out in zip(n_hiddens_per_layer_list[:-1], n_hiddens_per_layer_list[1:]):
        nn_param_list.append(
            dict(
                w=weight_fill_func((n_in, n_out)),
                b=bias_fill_func((n_out,)),
            ))
    return nn_param_list
```

#### Here's a function to pretty-print any given set of NN parameters to stdout, so we can inspect

```python
def pretty_print_nn_param_list(nn_param_list_of_dict):
    """ Create pretty display of the parameters at each layer
    """
    for ll, layer_dict in enumerate(nn_param_list_of_dict):
        print("Layer %d" % ll)
        print("  w | size %9s | %s" % (layer_dict['w'].shape, layer_dict['w'].flatten()))
        print("  b | size %9s | %s" % (layer_dict['b'].shape, layer_dict['b'].flatten()))
```

## Example: NN with 0 hidden layers (equivalent to linear regression)

For univariate regression: 1D -> 1D

Will fill all parameters with zeros by default

```python
nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=1, n_dims_output=1)
pretty_print_nn_param_list(nn_params)
```

## Example: NN with 0 hidden layers (equivalent to linear regression)

For multivariate regression when |x_i| = 2: 2D -> 1D

Will fill all parameters with zeros by default

```python
nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)
pretty_print_nn_param_list(nn_params)
```

## Example: NN with 1 hidden layer of 3 hidden units

```python
nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1)
pretty_print_nn_param_list(nn_params)
```

## Example: NN with 1 hidden layer of 3 hidden units

Use 'ones' as the fill function for weights

```python
nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1,
    weight_fill_func=np.ones)
pretty_print_nn_param_list(nn_params)
```

## Example: NN with 1 hidden layer of 3 hidden units

Use random draws from standard normal as the fill function for weights

```python
nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1,
    weight_fill_func=lambda size_tuple: np.random.randn(*size_tuple))
pretty_print_nn_param_list(nn_params)
```

## Example: NN with 7 hidden layers of diff sizes

Just shows how generic this framework is!

```python
nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[3, 4, 5, 6, 5, 4, 3], n_dims_input=2, n_dims_output=1)
pretty_print_nn_param_list(nn_params)
```

## Setup: Function that performs **prediction**

```python
def predict_y_given_x_with_NN(x=None, nn_param_list=None, activation_func=ag_np.tanh):
    """ Predict y value given x value via feed-forward neural net

    Args
    ----
    x : array_like, n_examples x n_input_dims

    Returns
    -------
    y : array_like, n_examples
    """
    for layer_id, layer_dict in enumerate(nn_param_list):
        if layer_id == 0:
            if x.ndim > 1:
                in_arr = x
            else:
                if x.size == nn_param_list[0]['w'].shape[0]:
                    in_arr = x[ag_np.newaxis,:]
                else:
                    in_arr = x[:,ag_np.newaxis]
        else:
            in_arr = activation_func(out_arr)
        out_arr = ag_np.dot(in_arr, layer_dict['w']) + layer_dict['b']
    return ag_np.squeeze(out_arr)
```

### Example: Make predictions with 0-layer NN whose parameters  are filled with the 'true' params for our toy dataset

```python
true_nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)
true_nn_params[0]['w'][:] = true_w_D[:,np.newaxis]
true_nn_params[0]['b'][:] = true_bias
```

```python
yhat_N = predict_y_given_x_with_NN(x_ND, true_nn_params)
assert yhat_N.size == N

plt.plot(yhat_N, y_N, 'k.')
plt.xlabel('true y')
plt.ylabel('predicted y|x')
```

### Example: Make predictions with 0-layer NN whose parameters  are filled with all zeros

```python
zero_nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)
yhat_N = predict_y_given_x_with_NN(x_ND, zero_nn_params)
assert yhat_N.size == N

plt.plot(yhat_N, y_N, 'k.')
plt.xlabel('true y')
plt.ylabel('predicted y|x')
```

## Setup: Gradient descent implementation that can use list-of-dict parameters (not just arrays)

```python
def run_many_iters_of_gradient_descent_with_list_of_dict(
        f, g, init_x_list_of_dict=None,
        n_iters=100, step_size=0.001,
        n_steps_between_print=50):

    # Copy the initial parameter vector
    x_list_of_dict = copy.deepcopy(init_x_list_of_dict)

    # Create data structs to track the per-iteration history of different quantities
    history = dict(
        iter=[],
        f=[],
        x=[],
        g=[])
    start_time = time.time()
    for iter_id in range(n_iters):
        if iter_id > 0:
            # Gradient is a list of layer-specific dicts
            grad_list_of_dict = g(x_list_of_dict)
            for layer_id, x_layer_dict in enumerate(x_list_of_dict):
                for key in x_layer_dict.keys():
                    x_layer_dict[key] = x_layer_dict[key] - step_size * grad_list_of_dict[layer_id][key]

        fval = f(x_list_of_dict)
        history['iter'].append(iter_id)
        history['f'].append(fval)
        history['x'].append(copy.deepcopy(x_list_of_dict))
        history['g'].append(g(x_list_of_dict))

        if iter_id < 3 or (iter_id+1) % n_steps_between_print == 0:
            print("completed iter %5d/%d after %7.1f sec | loss %.6e" % (
                iter_id+1, n_iters, time.time()-start_time, fval))
    return x_list_of_dict, history
```

# Worked Exercise 5a: Train 0-layer NN via gradient descent on LINEAR toy data

```python
def nn_regression_loss_function(nn_params):
    yhat_N = predict_y_given_x_with_NN(x_ND, nn_params)
    return 0.5 / ag_np.square(sigma) * ag_np.sum(ag_np.square(y_N - yhat_N))
```

### Example 5a(i): Try first by starting the optimization at the IDEAL answer

Why? Good debugging strategy! Make sure your optimization and loss function are set up correctly. If all goes well, if we start at an intended optimal point, we shouldn't go very far (just perhaps micro optimizing for small sample effects.... the true bias is +10 but with our 100 samples maybe the best to do is +9.8)

```python
fromtrue_opt_nn_params, fromtrue_history = run_many_iters_of_gradient_descent_with_list_of_dict(
    nn_regression_loss_function,
    autograd.grad(nn_regression_loss_function),
    true_nn_params,
    n_iters=100,
    step_size=0.000001,
    n_steps_between_print=50)
```

### Pretty print the learned weights and bias... do they match the true ones used to generate the data?

```python
pretty_print_nn_param_list(fromtrue_opt_nn_params)
```

```python
plt.plot(fromtrue_history['iter'], fromtrue_history['f'], 'k.-')
```

### Example 5a(ii): Try  by starting the optimization at ALL ZEROS (so it needs to go somewhere)

```python
fromzero_opt_nn_params, fromzero_history = run_many_iters_of_gradient_descent_with_list_of_dict(
    nn_regression_loss_function,
    autograd.grad(nn_regression_loss_function),
    zero_nn_params,
    n_iters=100,
    step_size=0.000001)
```

```python
pretty_print_nn_param_list(fromzero_opt_nn_params)
```

```python
plt.plot(fromzero_history['iter'], fromzero_history['f'], 'k.-')
```

```python

```

# Exercise 5b: Create more complex non-linear toy dataset

True method *regression from QUADRATIC features*:

$$
y \sim \text{Normal}( w_1 x_1 + w_2 x_2 + w_3 x_1^2 + w_4 x_2^2 + b, \sigma^2)
$$

```python
N = 300
D = 2
sigma = 0.1

wsq_D = np.asarray([-2.0, 2.0])
w_D = np.asarray([4.2, -4.2])

train_prng = np.random.RandomState(0)
x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))
y_N = (
    np.dot(np.square(x_ND), wsq_D)
    + np.dot(x_ND, w_D)
    + sigma * train_prng.randn(N))
```

```python
sns.pairplot(
    data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));
```

```python
def nonlinear_toy_nn_regression_loss_function(nn_params):
    yhat_N = predict_y_given_x_with_NN(x_ND, nn_params)
    return 0.5 / ag_np.square(sigma) * ag_np.sum(ag_np.square(y_N - yhat_N))
```

```python
# Initialize 1-layer, 10 hidden unit network with small random noise on weights

H10_init_nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[10], n_dims_input=2, n_dims_output=1,
    weight_fill_func=lambda sz_tuple: 0.1 * np.random.randn(*sz_tuple))
```

```python
H10_opt_nn_params, H10_history = run_many_iters_of_gradient_descent_with_list_of_dict(
    nonlinear_toy_nn_regression_loss_function,
    autograd.grad(nonlinear_toy_nn_regression_loss_function),
    H10_init_nn_params,
    n_iters=300,
    step_size=0.000001)
```

#### Plot objective function vs iters

```python
plt.plot(H10_history['iter'], H10_history['f'], 'k.-')
plt.title('10 hidden units');
plt.xlabel('iter')
plt.ylabel('loss (negative log proba. of y)');
```

#### Plot predicted y vs. true y for each example as a scatterplot

```python
yhat_N = predict_y_given_x_with_NN(x_ND, H10_opt_nn_params)

plt.plot(yhat_N, y_N, 'k.');
plt.xlabel('predicted y|x');
plt.ylabel('true y');
```

```python
_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)
subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');
subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')
subplot_grid[0,0].set_xlabel('x_0');

subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');
subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')
subplot_grid[0,1].set_xlabel('x_1');
```

## More units! Try 1 layer with H=30 hidden units

```python
# Initialize 1-layer, 30 hidden unit network with small random noise on weights
H30_init_nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[30], n_dims_input=2, n_dims_output=1,
    weight_fill_func=lambda sz_tuple: 0.1 * np.random.randn(*sz_tuple))
```

```python
H30_opt_nn_params, H30_history = run_many_iters_of_gradient_descent_with_list_of_dict(
    nonlinear_toy_nn_regression_loss_function,
    autograd.grad(nonlinear_toy_nn_regression_loss_function),
    H30_init_nn_params,
    n_iters=300,
    step_size=0.000001)
```

#### Plot objective function vs iterations

```python
plt.plot(H30_history['iter'], H30_history['f'], 'k.-');
plt.title('30 hidden units');

plt.xlabel('iter')
plt.ylabel('loss (negative log proba. of y)');
```

#### Plot predicted y value vs true y value for each example

```python
yhat_N = predict_y_given_x_with_NN(x_ND, H30_opt_nn_params)

plt.plot(yhat_N, y_N, 'k.');
plt.xlabel('predicted y|x');
plt.ylabel('true y');
```

```python
_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)
subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');
subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')
subplot_grid[0,0].set_xlabel('x_0');

subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');
subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')
subplot_grid[0,1].set_xlabel('x_1');
```

## Even more units! Try 1 layer with H=100 hidden units

```python
# Initialize 1-layer, 100 hidden unit network with small random noise on weights
H100_init_nn_params = make_nn_params_as_list_of_dicts(
    n_hiddens_per_layer_list=[100], n_dims_input=2, n_dims_output=1,
    weight_fill_func=lambda sz_tuple: 0.05 * np.random.randn(*sz_tuple))
```

```python
H100_opt_nn_params, H100_history = run_many_iters_of_gradient_descent_with_list_of_dict(
    nonlinear_toy_nn_regression_loss_function,
    autograd.grad(nonlinear_toy_nn_regression_loss_function),
    H100_init_nn_params,
    n_iters=600,
    step_size=0.0000005)
```

```python
yhat_N = predict_y_given_x_with_NN(x_ND, H100_opt_nn_params)

plt.plot(yhat_N, y_N, 'k.');
plt.xlabel('predicted y|x');
plt.ylabel('true y');
```

```python
_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)
subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');
subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')
subplot_grid[0,0].set_xlabel('x_0');

subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');
subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')
subplot_grid[0,1].set_xlabel('x_1');
```

# Try it yourself!

* Can you train a prediction network on the non-linear toy data so it has ZERO training error? Is this even possible?

* Can you make the network train faster? What happens if you play with the step_size?

* What if you made the network **deeper** (more layers)?

* What other dataset would you want to try out this regression on?

```python

```


# Start-to-Finish Example: Applying Boundary Conditions in Curvilinear Coordinates for Rank-0, Rank-1, and Symmetric Rank-2 Tensors in Three Dimensions

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

## This module documents and validates basic boundary condition algorithms for curvilinear coordinate systems (e.g., Spherical, Cylindrical), based on prescription in the [SENR/NRPy+ paper](https://arxiv.org/abs/1712.07658).

## Introduction:
Following the prescription in the [SENR/NRPy+ paper](https://arxiv.org/abs/1712.07658), we will implement curvilinear boundary conditions for rank-0, rank-1, and symmetric rank-2 tensors in three dimensions; as this is the same dimension and highest rank needed for BSSN.

### Introduction to parity conditions:

Suppose we have a vector $v^\rho$ defined at ghostzone $(-\rho,\phi,z)$ ($\rho>0$) in cylindrical coordinates. This will map to an interior point at $(\rho,\phi+\pi,z)$. At this point, the direction of the $\hat{\rho}$ unit vector flips sign. Thus we cannot simply set the value of $v^\rho$ to the value it possesses at interior point $(\rho,\phi+\pi,z)$; that would result in a sign error. Instead we have
\begin{align}
v^\rho(-\rho,\phi,z)&=-v^\rho(\rho,\phi+\pi,z) \\
&= \mathbf{e}^\rho\left(-\rho,\phi,z\right) \cdot \mathbf{e}^\rho\left(\rho,\phi+\pi,z\right)v^\rho(\rho,\phi+\pi,z),
\end{align}
where $\mathbf{e}^\rho\left(\rho,\phi,z\right)$ is the $\rho$ unit vector evaluated at point $(\rho,\phi,z)$, and $\mathbf{e}^\rho\left(-\rho,\phi,z\right) \cdot \mathbf{e}^\rho\left(\rho,\phi+\pi,z\right)$ is the dot product of the two unit vectors, which must evaluate to $\pm 1$ (i.e., the **parity**). Contrast this with scalars, which do not possess a sense of direction/parity.

### Basic algorithm for applying boundary conditions in curvilinear coordinates:

At each ghost zone grid point $\mathbf{d}_{\rm gz}=(x_0,x_1,x_2)$, we will do the following:

1. Evaluate the Cartesian coordinate $\left(x(x_0,x_1,x_2),y(x_0,x_1,x_2),z(x_0,x_1,x_2)\right)$, corresponding to this grid point. Then evaluate the inverse $\mathbf{d}_{\rm new}=\left(x_0(x,y,z),x_1(x,y,z),x_2(x,y,z)\right)$.
    1. If $\mathbf{d}_{\rm new} \ne \mathbf{d}_{\rm gz}$, then the ghost zone grid point maps to a point in the grid interior, *which is exactly the case described in the above section*. To distinguish this case from an "outer boundary condition", we shall henceforth refer to it variously as an application of an "interior", "inner", or "parity" boundary condition.
    1. If $\mathbf{d}_{\rm new} \equiv \mathbf{d}_{\rm gz}$, then the ghost zone grid point is on the outer boundary of the grid, and standard outer boundary conditions should be applied.

### Applying parity conditions to arbitrary-rank tensors:

Above we presented the strategy for applying parity boundary conditions to a single component of a vector. Here we outline the generic algorithm for arbitrary-rank tensors.

Continuing the discussion from the previous section, we assume $\mathbf{d}_{\rm new} \ne \mathbf{d}_{\rm gz}$ (otherwise we would apply the *outer* boundary condition algorithm). Next suppose we are given a generic rank-$N$ tensor ($N>0$).

1. The first component of the rank-$N$ tensor corresponds to some direction with unit vector $\mathbf{e}^i$; e.g., $v^r$ corresponds to the $\mathbf{e}^r$ direction. Compute the dot product of the unit vector $\mathbf{e}^i$ evaluated at points $\mathbf{d}_{\rm gz}$ and $\mathbf{d}_{\rm new}$. Define this dot product as $P_1$ ("$P$" for "parity"):
$$
P_1 = \mathbf{e}^i\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^i\left(\mathbf{d}_{\rm new}\right).
$$
1. $P_1$ will take the value of $\pm 1$, depending on the unit-vector direction and the points $\mathbf{d}_{\rm gz}$ and $\mathbf{d}_{\rm new}$
1. Repeat the above for the remaining components of the rank-$N$ tensor $j\in \{2,3,...,N\}$, storing each $P_j$.
1. The tensor mapping from $\mathbf{d}_{\rm gz}$ to $\mathbf{d}_{\rm new}$ for this tensor $T^{ijk...}_{mnp...}$ will be given by
$$
T^{ijk...}_{lmn...}(x_0,x_1,x_2)_{\rm gz} = \prod_{\ell=1}^N P_\ell T^{ijk...}_{mnp...}(x_0,x_1,x_2)_{\rm new}.
$$

In this formulation of BSSN, we only need to deal with rank-0, rank-1, and *symmetric* rank-2 tensors. Further, our basis consists of 3 directions, so there are a total of
+ 1 parity condition (the trivial +1) for scalars (rank-0 tensors)
+ 3 parity conditions for all rank-1 tensors (corresponding to each direction)
+ 6 parity conditions for all *symmetric* rank-2 tensors (corresponding to the number of elements in the lower or upper triangle of a $3\times3$ matrix, including the diagonal)

Thus we must keep track of the behavior of **10 separate parity conditions**, which can be evaluated once the numerical grid has been set up, for all time.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Set core NRPy+ parameters for numerical grids and reference metric
1. [Step 2](#ccode_bcs): Implement C code for implementation of curvilinear coordinate boundary conditions
    1. [Step 2.a](#register_gfs): Register gridfunctions of all 10 parity types; output gridfunction aliases to `CurviBoundaryConditions/gridfunction\_defines.h`
    1. [Step 2.b](#assign_parity): Assign the correct basic parity type to each test gridfunction
        1. [Step 2.b.i](#parity_digits): Set parity type for each gridfunction, based on the digits at the end of its name
        1. [Step 2.b.ii](#dot): Set up unit-vector dot products (=parity) for each of the 10 parity condition types
    1. [Step 2.c](#gzmap): Implement a modified version of the scalar wave in curvilinear coordinates boundary condition ghost zone mapping routine
    1. [Step 2.d](#bpcs): Construct the C loops that apply boundary and parity conditions
1. [Step 3](#validate): Set up test data for Curvilinear Boundary Conditions code validation
1. [Step 4](#mainc): `CurviBC_Playground.c` : The Main C Code
1. [Step 5](#senr_compare): Validation: Compare with original SENR results
1. [Step 6](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Set core NRPy+ parameters for numerical grids and reference metric \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

We import needed NRPy+ modules and set reference_metric::CoordSystem=Spherical.

```python
# First we import needed core NRPy+ modules
from outputC import *
import NRPy_param_funcs as par
import grid as gri
import loop as lp
import indexedexp as ixp
import finite_difference as fin
import reference_metric as rfm

# Set spatial dimension (must be 3 for BSSN)
DIM = 3
par.set_parval_from_str("grid::DIM",DIM)

# Set the finite differencing order to 4; although this module doesn't compute
#   finite difference derivatives, it does define NGHOSTS, which depends
#   on finite_difference::FD_CENTDERIVS_ORDER being set.
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER",4)

# Then we set the coordinate system for the numerical grid
par.set_parval_from_str("reference_metric::CoordSystem","Spherical")
rfm.reference_metric()
```

# Step 2: Implement C code for implementation of curvilinear coordinate boundary conditions \[Back to [top](#toc)\]
$$\label{ccode_bcs}$$

## Step 2.a: Register gridfunctions of all 10 parity types; output gridfunction aliases to `CurviBoundaryConditions/gridfunction_defines.h` \[Back to [top](#toc)\]
$$\label{register_gfs}$$

Here we

1. Register within NRPy+ one gridfunction per each of the 10 parity conditions, and then
1. output to file the corresponding gridfunction aliases, so the C code can access each gridfunction by its human-friendly alias (e.g., `test_gfs[RANKONEU0GF][idx]` instead of `test_gfs[6][idx]`).

```python
# Step 2.A.1: Register gridfunctions of all 10 parity types

# 6 gridfunctions, corresponding to all unique rank-2 tensor components:
ranktwosymmDD = ixp.register_gridfunctions_for_single_rank2("AUX","ranktwosymmDD", "sym01")
# 3 gridfunctions, corresponding to all unique rank-1 tensor components:
rankoneU = ixp.register_gridfunctions_for_single_rank1("AUX","rankoneU")
# 1 rank-0 (scalar) gridfunction
rankzero = ixp.gri.register_gridfunctions("AUX","rankzero")

# Step 2.A.2. Output gridfunction #define aliases to file:
!mkdir CurviBoundaryConditions 2>/dev/null # 2>/dev/null: Don't throw an error if the directory already exists.
evolved_variables_list, auxiliary_variables_list, auxevol_variables_list = \
        gri.output__gridfunction_defines_h__return_gf_lists("CurviBoundaryConditions")
```

## Step 2.b: Assign the correct basic parity type to each test gridfunction \[Back to [top](#toc)\]
$$\label{assign_parity}$$

The appropriate symbolic dot products determining parity condition are assigned to each gridfunction based on the following numbering:

Tensor type | Parity type (Step 2.B.1) | Dot product(s) determining parity condition (Step 2.B.2)
--- | --- | ---
Scalar (Rank-0 tensor) | 0 | (*none*)
Rank-1 tensor in **i0** direction | 1 | $\mathbf{e}^0\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^0\left(\mathbf{d}_{\rm new}\right)$
Rank-1 tensor in **i1** direction | 2 | $\mathbf{e}^1\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^1\left(\mathbf{d}_{\rm new}\right)$
Rank-1 tensor in **i2** direction | 3 | $\mathbf{e}^2\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^2\left(\mathbf{d}_{\rm new}\right)$
Rank-2 tensor in **i0-i0** direction | 4 | $\left[\mathbf{e}^0\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^0\left(\mathbf{d}_{\rm new}\right)\right]^2 = 1$
Rank-2 tensor in **i0-i1** direction | 5 | $\left[\mathbf{e}^0\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^0\left(\mathbf{d}_{\rm new}\right)\right]\left[\mathbf{e}^1\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^1\left(\mathbf{d}_{\rm new}\right)\right]$
Rank-2 tensor in **i0-i2** direction | 6 | $\left[\mathbf{e}^0\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^0\left(\mathbf{d}_{\rm new}\right)\right]\left[\mathbf{e}^2\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^2\left(\mathbf{d}_{\rm new}\right)\right]$
Rank-2 tensor in **i1-i1** direction | 7 | $\left[\mathbf{e}^1\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^1\left(\mathbf{d}_{\rm new}\right)\right]^2 = 1$
Rank-2 tensor in **i1-i2** direction | 8 | $\left[\mathbf{e}^1\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^1\left(\mathbf{d}_{\rm new}\right)\right]\left[\mathbf{e}^2\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^2\left(\mathbf{d}_{\rm new}\right)\right]$
Rank-2 tensor in **i2-i2** direction | 9 | $\left[\mathbf{e}^2\left(\mathbf{d}_{\rm gz}\right) \cdot \mathbf{e}^2\left(\mathbf{d}_{\rm new}\right)\right]^2 = 1$

In [Step 2.b.i](#parity_digits) below, we set the middle column of the table for each gridfunction, and [Step 2.b.ii](#dot) sets the rightmost column.

### Step 2.b.i: Set parity type for each gridfunction, based on the digits at the end of its name \[Back to [top](#toc)\]
$$\label{parity_digits}$$

For example, if the gridfunction name ends with "01", then (based on the table above) the `set_parity_types()` function below will set the parity_type of that gridfunction to 5. We can be assured this is a robust algorithm, because `gri.register_gridfunctions()` in [grid.py](../edit/grid.py) will throw an error if a gridfunction base name ends in an integer.

After each parity type is found, we store the parity type of each gridfunction to `const int8_t arrays` `evol_gf_parity` and `aux_gf_parity`, appended to the end of `CurviBoundaryConditions/gridfunction\_defines.h`.

```python
# Step 2.B.1: set the parity conditions on all gridfunctions in gf_list,
#       based on how many digits are at the end of their names
def set_parity_types(gf_list):
    parity_type = []
    for i in range(len(gf_list)):
        varname = gf_list[i]
        parity_type__orig_len = len(parity_type)
        if  len(varname)>2:
            if   varname[-2] == "0" and varname[-1] == "0": # In Python, a[-1] points to the last
                                                            # element of a list; a[-2] the
                                                            # second-to-last element, etc.
                parity_type.append(4)
            elif varname[-2] == "0" and varname[-1] == "1":
                parity_type.append(5)
            elif varname[-2] == "0" and varname[-1] == "2":
                parity_type.append(6)
            elif varname[-2] == "1" and varname[-1] == "1":
                parity_type.append(7)
            elif varname[-2] == "1" and varname[-1] == "2":
                parity_type.append(8)
            elif varname[-2] == "2" and varname[-1] == "2":
                parity_type.append(9)
        if len(varname)>1 and len(parity_type) == parity_type__orig_len:
            if   varname[-1] == "0":
                parity_type.append(1)
            elif varname[-1] == "1":
                parity_type.append(2)
            elif varname[-1] == "2":
                parity_type.append(3)
        if varname[len(varname)-1].isdigit() == False:
            parity_type.append(0)

        if len(parity_type) == parity_type__orig_len:
            print("Error: Could not figure out parity type for evolved variable: "+varname)
            exit(1)
    return parity_type

evol_parity_type = set_parity_types(evolved_variables_list)
aux_parity_type = set_parity_types(auxiliary_variables_list)
auxevol_parity_type = set_parity_types(auxevol_variables_list)

with open("CurviBoundaryConditions/gridfunction_defines.h", "a") as file:
    file.write("\n\n/* PARITY TYPES FOR ALL GRIDFUNCTIONS.\n")
    file.write("   SEE \"Tutorial-Start_to_Finish-BSSNCurvilinear-Two_BHs_Collide.ipynb\" FOR DEFINITIONS. */\n")

    if len(evolved_variables_list) > 0:
        file.write("const int8_t evol_gf_parity[" + str(len(evolved_variables_list)) + "] = { ")
        for i in range(len(evolved_variables_list) - 1):
            file.write(str(evol_parity_type[i]) + ", ")
        file.write(str(evol_parity_type[len(evolved_variables_list) - 1]) + " };\n")

    if len(auxiliary_variables_list) > 0:
        file.write("const int8_t aux_gf_parity[" + str(len(auxiliary_variables_list)) + "] = { ")
        for i in range(len(auxiliary_variables_list) - 1):
            file.write(str(aux_parity_type[i]) + ", ")
        file.write(str(aux_parity_type[len(auxiliary_variables_list) - 1]) + " };\n")

    if len(auxevol_variables_list) > 0:
        file.write("const int8_t auxevol_gf_parity[" + str(len(auxevol_variables_list)) + "] = { ")
        for i in range(len(auxevol_variables_list) - 1):
            file.write(str(auxevol_parity_type[i]) + ", ")
        file.write(str(auxevol_parity_type[len(auxevol_variables_list) - 1]) + " };\n")

for i in range(len(evolved_variables_list)):
    print("Evolved gridfunction \""+evolved_variables_list[i]+"\" has parity type "+str(evol_parity_type[i])+".")
for i in range(len(auxiliary_variables_list)):
    print("Auxiliary gridfunction \""+auxiliary_variables_list[i]+"\" has parity type "+str(aux_parity_type[i])+".")
for i in range(len(auxevol_variables_list)):
    print("AuxEvol gridfunction \""+auxevol_variables_list[i]+"\" has parity type "+str(auxevol_parity_type[i])+".")

print("Output from CurviBoundaryConditions/gridfunction_defines.h (notice the parity types appended to the bottom):")
!cat CurviBoundaryConditions/gridfunction_defines.h
```

    Auxiliary gridfunction "rankoneU0" has parity type 1.
    Auxiliary gridfunction "rankoneU1" has parity type 2.
    Auxiliary gridfunction "rankoneU2" has parity type 3.
    Auxiliary gridfunction "ranktwosymmDD00" has parity type 4.
    Auxiliary gridfunction "ranktwosymmDD01" has parity type 5.
    Auxiliary gridfunction "ranktwosymmDD02" has parity type 6.
    Auxiliary gridfunction "ranktwosymmDD11" has parity type 7.
    Auxiliary gridfunction "ranktwosymmDD12" has parity type 8.
    Auxiliary gridfunction "ranktwosymmDD22" has parity type 9.
    Auxiliary gridfunction "rankzero" has parity type 0.
    Output from CurviBoundaryConditions/gridfunction_defines.h (notice the parity types appended to the bottom):
    /* This file is automatically generated by NRPy+. Do not edit. */

    /* EVOLVED VARIABLES: */
    #define NUM_EVOL_GFS 0

     /* AUXILIARY VARIABLES: */
    #define NUM_AUX_GFS 10
    #define RANKONEU0GF	0
    #define RANKONEU1GF	1
    #define RANKONEU2GF	2
    #define RANKTWOSYMMDD00GF	3
    #define RANKTWOSYMMDD01GF	4
    #define RANKTWOSYMMDD02GF	5
    #define RANKTWOSYMMDD11GF	6
    #define RANKTWOSYMMDD12GF	7
    #define RANKTWOSYMMDD22GF	8
    #define RANKZEROGF	9

     /* AUXEVOL VARIABLES: */
    #define NUM_AUXEVOL_GFS 0

    /* PARITY TYPES FOR ALL GRIDFUNCTIONS.
       SEE "Tutorial-Start_to_Finish-BSSNCurvilinear-Two_BHs_Collide.ipynb" FOR DEFINITIONS. */
    const int8_t aux_gf_parity[10] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 };

###  Step 2.b.ii: Set up unit-vector dot products (=parity) for each of the 10 parity condition types \[Back to [top](#toc)\]
$$\label{dot}$$

First we fill in the parity condition arrays. These take as input $(x_0,x_1,x_2)_{\rm in}$ and $(x_0,x_1,x_2)_{\rm IB}$, and output the necessary dot product(s) for each parity type. To wit (as described above), there are 10 parity types for BSSN evolved variables, which include tensors up to and including rank-2:

```python
# Step 2.B.2: Set up unit-vector dot products (=parity) for each of the 10 parity condition types
parity = ixp.zerorank1(DIM=10)
UnitVectors_inner = ixp.zerorank2()
xx0_inbounds,xx1_inbounds,xx2_inbounds = sp.symbols("xx0_inbounds xx1_inbounds xx2_inbounds", real=True)
for i in range(3):
    for j in range(3):
        UnitVectors_inner[i][j] = rfm.UnitVectors[i][j].subs(rfm.xx[0],xx0_inbounds).subs(rfm.xx[1],xx1_inbounds).subs(rfm.xx[2],xx2_inbounds)
# Type 0: scalar
parity[0] = sp.sympify(1)
# Type 1: i0-direction vector or one-form
# Type 2: i1-direction vector or one-form
# Type 3: i2-direction vector or one-form
for i in range(3):
    for Type in range(1,4):
        parity[Type] += rfm.UnitVectors[Type-1][i]*UnitVectors_inner[Type-1][i]
# Type 4: i0i0-direction rank-2 tensor
# parity[4] = parity[1]*parity[1]
# Type 5: i0i1-direction rank-2 tensor
# Type 6: i0i2-direction rank-2 tensor
# Type 7: i1i1-direction rank-2 tensor
# Type 8: i1i2-direction rank-2 tensor
# Type 9: i2i2-direction rank-2 tensor
count = 4
for i in range(3):
    for j in range(i,3):
        parity[count] = parity[i+1]*parity[j+1]
        count = count + 1

lhs_strings = []
for i in range(10):
    lhs_strings.append("parity["+str(i)+"]")
outputC(parity,lhs_strings, "CurviBoundaryConditions/set_parity_conditions.h")

print("\n\nExample: parity type 1's dot product is given by: \n"+lhs_strings[1]+" = "+str(parity[1]))
```

    Wrote to file "CurviBoundaryConditions/set_parity_conditions.h"

    Example: parity type 1's dot product is given by:
    parity[1] = sin(xx1)*sin(xx1_inbounds)*sin(xx2)*sin(xx2_inbounds) + sin(xx1)*sin(xx1_inbounds)*cos(xx2)*cos(xx2_inbounds) + cos(xx1)*cos(xx1_inbounds)

##  Step 2.c: Implement a modified version of the scalar wave in curvilinear coordinates boundary condition ghost zone mapping routine \[Back to [top](#toc)\]
$$\label{gzmap}$$

We implement a modified version of the scalar wave in curvilinear coordinates boundary condition ghost zone mapping routine, so that it also defines the parity conditions

`set_up_bc_gz_map_and_parity_conditions()` below implements a two-step algorithm:

1. Find locations to where outer ghost zone gridpoints map (this is identical to the algorithm used in the [scalar wave in curvilinear coordinates tutorial](Tutorial-Start_to_Finish-ScalarWaveCurvilinear.ipynb), which is described above in terms of $\mathbf{d}_{\rm gz}$ and $\mathbf{d}_{\rm new}$)
    1. As described in the [scalar wave in curvilinear coordinates tutorial](Tutorial-Start_to_Finish-ScalarWaveCurvilinear.ipynb), this requires first mapping from the curvilinear coordinate gridpoints $(x_0,x_1,x_2)$ in the outer ghost zones to the corresponding Cartesian grid points $(x,y,z)$ in the grid interior or outer boundary (handled by xxminmax, xxCart, and Cart_to_xx defined below).
    1. The interior gridpoint to which each ghost zone maps will be stored in the *ghostzone_map* data structure:
```C
typedef struct ghostzone_map {
   short i0,i1,i2;
} gz_map;
```
1. At each ghost zone gridpoint, find and store the correct parity condition type for each gridfunction up to rank 2 (the highest rank in the BSSN RHSs). As described above, there are a total of 10 possible parity conditions, which each have a unique behavior at a given ghost zone. Thus:
    1. at each ghost zone, we store all 10 parity conditions, in the data structure:
```C
typedef struct parity_conditions {
  int8_t parity[10];
} parity_condition;
```
    1. In the above data structure, `parity[i]` can only take a value of $\pm 1$, which is why we store it in the smallest C integer data type, `int8_t` (a one-byte, signed integer).

In the below code blocks:

1. we first output basic curvilinear$\leftrightarrow$Cartesian grid mapping C code, which is required by Step 1 of the above algorithm, and
1. then we implement Steps 1 and 2 of the above algorithm.

```python
# Step 2.C: Modified version of the scalar wave in curvilinear coordinates boundary
#           condition ghost zone mapping routine, so that it also defines the parity
#           conditions.

# First output code needed for mapping from any given curvilinear coordinate gridpoint
#  to the Cartesian coordinate in the grid interior (xxCart), and then find the
#  corresponding gridpoint index in the grid interior (Cart_to_xx; xxminmax).
# Generic coordinate NRPy+ file output, Part 1: output the conversion from (x0,x1,x2) to Cartesian (x,y,z)
outputC([rfm.xxCart[0],rfm.xxCart[1],rfm.xxCart[2]],["xCart[0]","xCart[1]","xCart[2]"],
        "CurviBoundaryConditions/xxCart.h")
# Generic coordinate NRPy+ file output, Part 2: output the coordinate bounds xxmin[] and xxmax[]:
with open("CurviBoundaryConditions/xxminmax.h", "w") as file:
    file.write("const REAL xxmin[3] = {"+str(rfm.xxmin[0])+","+str(rfm.xxmin[1])+","+str(rfm.xxmin[2])+"};\n")
    file.write("const REAL xxmax[3] = {"+str(rfm.xxmax[0])+","+str(rfm.xxmax[1])+","+str(rfm.xxmax[2])+"};\n")
# Generic coordinate NRPy+ file output, Part 3: output the conversion from Cartesian (x,y,z) to interior/OB (x0,x1,x2)
outputC([rfm.Cart_to_xx[0],rfm.Cart_to_xx[1],rfm.Cart_to_xx[2]],
        ["Cart_to_xx0_inbounds","Cart_to_xx1_inbounds","Cart_to_xx2_inbounds"],
        "CurviBoundaryConditions/Cart_to_xx.h")
```

    Wrote to file "CurviBoundaryConditions/xxCart.h"
    Wrote to file "CurviBoundaryConditions/Cart_to_xx.h"

```python
%%writefile CurviBoundaryConditions/curvilinear_parity_and_outer_boundary_conditions.h

// First we define the struct that will be used to store the 10 parity conditions at all gridpoints:
// We store the 10 parity conditions in a struct consisting of 10 integers, one for each condition.
// Note that these conditions can only take one of two values: +1 or -1.
typedef struct parity_conditions {
  int8_t parity[10];
} parity_condition;

typedef struct ghostzone_map {
  short i0,i1,i2;
} gz_map;

void set_bc_parity_conditions(REAL parity[10], const REAL xx0,const REAL xx1,const REAL xx2,
                              const REAL xx0_inbounds,const REAL xx1_inbounds,const REAL xx2_inbounds) {
    #include "set_parity_conditions.h"
}

void set_up_bc_gz_map_and_parity_conditions(const int Nxx_plus_2NGHOSTS[3], REAL *xx[3],
                                            const REAL dxx[3], const REAL xxmin[3], const REAL xxmax[3],
                                            gz_map *bc_gz_map, parity_condition *bc_parity_conditions) {
  LOOP_REGION(0,Nxx_plus_2NGHOSTS[0],0,Nxx_plus_2NGHOSTS[1],0,Nxx_plus_2NGHOSTS[2]) {
    // First find Cartesian coordinate corresponding to (x_0,x_1,x_2)=(xx[0][i0],xx[1][i1],xx[2][i2]):
    REAL xCart[3];
    xxCart(xx, i0,i1,i2, xCart);
    REAL Cartx = xCart[0];
    REAL Carty = xCart[1];
    REAL Cartz = xCart[2];

    // Next find the (i0_inbounds,i1_inbounds,i2_inbounds) corresponding to the above Cartesian coordinate.
    //   If (i0_inbounds,i1_inbounds,i2_inbounds) is in a ghost zone, then it must equal (i0,i1,i2), and
    //      the point is an outer boundary point.
    //   Otherwise (i0_inbounds,i1_inbounds,i2_inbounds) is in the grid interior, and data at (i0,i1,i2)
    //      must be replaced with data at (i0_inbounds,i1_inbounds,i2_inbounds), but multiplied by the
    //      appropriate parity condition (+/- 1).
    REAL Cart_to_xx0_inbounds,Cart_to_xx1_inbounds,Cart_to_xx2_inbounds;
#include "Cart_to_xx.h"
    int i0_inbounds = (int)( (Cart_to_xx0_inbounds - xxmin[0] - (1.0/2.0)*dxx[0] + ((REAL)NGHOSTS)*dxx[0])/dxx[0] + 0.5 );
    int i1_inbounds = (int)( (Cart_to_xx1_inbounds - xxmin[1] - (1.0/2.0)*dxx[1] + ((REAL)NGHOSTS)*dxx[1])/dxx[1] + 0.5 );
    int i2_inbounds = (int)( (Cart_to_xx2_inbounds - xxmin[2] - (1.0/2.0)*dxx[2] + ((REAL)NGHOSTS)*dxx[2])/dxx[2] + 0.5 );

    REAL xCart_orig[3]; for(int ii=0;ii<3;ii++) xCart_orig[ii] = xCart[ii];
    xxCart(xx, i0_inbounds,i1_inbounds,i2_inbounds, xCart);

#define EPS_ABS 1e-8
    if(fabs( (double)(xCart_orig[0] - xCart[0]) ) > EPS_ABS ||
       fabs( (double)(xCart_orig[1] - xCart[1]) ) > EPS_ABS ||
       fabs( (double)(xCart_orig[2] - xCart[2]) ) > EPS_ABS) {
      printf("Error. Cartesian disagreement: ( %.15e %.15e %.15e ) != ( %.15e %.15e %.15e )\n",
             (double)xCart_orig[0],(double)xCart_orig[1],(double)xCart_orig[2],
             (double)xCart[0],(double)xCart[1],(double)xCart[2]);
      exit(1);
    }

    if(i0_inbounds-i0 == 0 && i1_inbounds-i1 == 0 && i2_inbounds-i2 == 0) {
      bc_gz_map[IDX3(i0,i1,i2)].i0=-1;
      bc_gz_map[IDX3(i0,i1,i2)].i1=-1;
      bc_gz_map[IDX3(i0,i1,i2)].i2=-1;
      for(int which_parity=0; which_parity<10; which_parity++) {
        bc_parity_conditions[IDX3(i0,i1,i2)].parity[which_parity] = 1;
      }
    } else {
      bc_gz_map[IDX3(i0,i1,i2)].i0=i0_inbounds;
      bc_gz_map[IDX3(i0,i1,i2)].i1=i1_inbounds;
      bc_gz_map[IDX3(i0,i1,i2)].i2=i2_inbounds;
      const REAL xx0 = xx[0][i0];
      const REAL xx1 = xx[1][i1];
      const REAL xx2 = xx[2][i2];
      const REAL xx0_inbounds = xx[0][i0_inbounds];
      const REAL xx1_inbounds = xx[1][i1_inbounds];
      const REAL xx2_inbounds = xx[2][i2_inbounds];
      REAL REAL_parity_array[10];
      set_bc_parity_conditions(REAL_parity_array,  xx0,xx1,xx2, xx0_inbounds,xx1_inbounds,xx2_inbounds);
      for(int whichparity=0;whichparity<10;whichparity++) {
          //printf("Good? Parity %d evaluated to %e\n",whichparity,(double)REAL_parity_array[whichparity]);
          // Perform sanity check on parity array output: should be +1 or -1 to within 8 significant digits:
          if( (REAL_parity_array[whichparity]  > 0 && fabs(REAL_parity_array[whichparity] - (+1)) > 1e-8) ||
              (REAL_parity_array[whichparity] <= 0 && fabs(REAL_parity_array[whichparity] - (-1)) > 1e-8) ) {
              printf("Error. Parity evaluated to %e , which is not within 8 significant digits of +1 or -1.",REAL_parity_array[whichparity]);
              exit(1);
          }
          if(REAL_parity_array[whichparity] < 0.0) bc_parity_conditions[IDX3(i0,i1,i2)].parity[whichparity] = -1;
          if(REAL_parity_array[whichparity] > 0.0) bc_parity_conditions[IDX3(i0,i1,i2)].parity[whichparity] = +1;
      }
    }
  }
}
```

    Overwriting CurviBoundaryConditions/curvilinear_parity_and_outer_boundary_conditions.h

## Step 2.d: Construct the C loops that apply boundary and parity conditions \[Back to [top](#toc)\]
$$\label{bpcs}$$

Now that `bc_gz_map` and `bc_parity_conditions` have been set at all gridpoints for all gridfunctions, construct the C loops that apply these boundary and parity conditions.

The algorithm below loops over all gridfunctions at all boundary gridpoints, starting from the innermost ghost zone in all 6 faces of the cube and working its way out (to avoid data dependency issues).

1. At the innermost ghost zone, only gridpoints corresponding to outer boundaries are updated first, followed by another pass in which only gridpoints corresponding to inner (parity) boundaries are updated. This ensures that points on an outer boundary that actually maps to another point on the outer boundary are properly updated.
1. The above process is then repeated at the next-to-innermost ghost zone, and then the next-to-next, etc., until all ghost zones have been filled. This ordering must be obeyed, or data will be filled in with unknown values.

```python
%%writefile -a CurviBoundaryConditions/curvilinear_parity_and_outer_boundary_conditions.h

// Part P6: Declare boundary condition OB_UPDATE macro,
//          which updates a single face of the 3D grid cube
//          with
//          1. quadratic polynomial extrapolation, if the face
//             corresponds to an outer boundary, or
//          2. parity condition, if the face maps to a point
//             in the grid interior.
const int MAXFACE = -1;
const int NUL     = +0;
const int MINFACE = +1;

#define OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, i0min,i0max, i1min,i1max, i2min,i2max, FACEX0,FACEX1,FACEX2) \
  LOOP_REGION(i0min,i0max, i1min,i1max, i2min,i2max) {                  \
    const int idx3 = IDX3(i0,i1,i2);                                    \
    if(bc_gz_map[idx3].i0 == -1) {                                      \
      gfs[IDX4(which_gf,i0,i1,i2)] =                                    \
        +3.0*gfs[IDX4(which_gf,i0+1*FACEX0,i1+1*FACEX1,i2+1*FACEX2)]    \
        -3.0*gfs[IDX4(which_gf,i0+2*FACEX0,i1+2*FACEX1,i2+2*FACEX2)]    \
        +1.0*gfs[IDX4(which_gf,i0+3*FACEX0,i1+3*FACEX1,i2+3*FACEX2)];   \
    }                                                                   \
  }

#define OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, i0min,i0max, i1min,i1max, i2min,i2max, FACEX0,FACEX1,FACEX2) \
  LOOP_REGION(i0min,i0max, i1min,i1max, i2min,i2max) {                  \
    const int idx3 = IDX3(i0,i1,i2);                                    \
    if(bc_gz_map[idx3].i0 != -1) {                                      \
      gfs[IDX4(which_gf,i0,i1,i2)] =                                    \
        ( (REAL)bc_parity_conditions[idx3].parity[gfs_parity[which_gf]] )* \
        gfs[IDX4(which_gf,                                              \
                 bc_gz_map[idx3].i0,                                    \
                 bc_gz_map[idx3].i1,                                    \
                 bc_gz_map[idx3].i2)];                                  \
    }                                                                   \
  }

// Part P7: Boundary condition driver routine: Apply BCs to all six
//          boundary faces of the cube, filling in the innermost
//          ghost zone first, and moving outward.
void apply_bcs(const int Nxx[3],const int Nxx_plus_2NGHOSTS[3],
               gz_map *bc_gz_map,parity_condition *bc_parity_conditions,int num_gfs,const int8_t *gfs_parity, REAL *gfs) {
#pragma omp parallel for
  for(int which_gf=0;which_gf<num_gfs;which_gf++) {
    int imin[3] = { NGHOSTS, NGHOSTS, NGHOSTS };
    int imax[3] = { Nxx_plus_2NGHOSTS[0]-NGHOSTS, Nxx_plus_2NGHOSTS[1]-NGHOSTS, Nxx_plus_2NGHOSTS[2]-NGHOSTS };
    for(int which_gz = 0; which_gz < NGHOSTS; which_gz++) {

      // First apply OUTER boundary conditions,
      //   in case an INNER (parity) boundary point
      //   needs data at the outer boundary:
      // After updating each face, adjust imin[] and imax[]
      //   to reflect the newly-updated face extents.
      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imin[0]-1,imin[0], imin[1],imax[1], imin[2],imax[2], MINFACE,NUL,NUL); imin[0]--;
      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imax[0],imax[0]+1, imin[1],imax[1], imin[2],imax[2], MAXFACE,NUL,NUL); imax[0]++;

      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1]-1,imin[1], imin[2],imax[2], NUL,MINFACE,NUL); imin[1]--;
      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imax[1],imax[1]+1, imin[2],imax[2], NUL,MAXFACE,NUL); imax[1]++;

      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1],imax[1], imin[2]-1,imin[2], NUL,NUL,MINFACE); imin[2]--;
      OB_UPDATE_OUTER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1],imax[1], imax[2],imax[2]+1, NUL,NUL,MAXFACE); imax[2]++;

      // Then apply INNER (parity) boundary conditions:
      for(int ii=0;ii<3;ii++) {imin[ii]++; imax[ii]--;}
      // After updating each face, adjust imin[] and imax[]
      //   to reflect the newly-updated face extents.
      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imin[0]-1,imin[0], imin[1],imax[1], imin[2],imax[2], MINFACE,NUL,NUL); imin[0]--;
      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imax[0],imax[0]+1, imin[1],imax[1], imin[2],imax[2], MAXFACE,NUL,NUL); imax[0]++;

      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1]-1,imin[1], imin[2],imax[2], NUL,MINFACE,NUL); imin[1]--;
      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imax[1],imax[1]+1, imin[2],imax[2], NUL,MAXFACE,NUL); imax[1]++;

      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1],imax[1], imin[2]-1,imin[2], NUL,NUL,MINFACE); imin[2]--;
      OB_UPDATE_INNER(which_gf, bc_gz_map,bc_parity_conditions, imin[0],imax[0], imin[1],imax[1], imax[2],imax[2]+1, NUL,NUL,MAXFACE); imax[2]++;

    }
  }
}
```

    Appending to CurviBoundaryConditions/curvilinear_parity_and_outer_boundary_conditions.h

# Step 3: Set up test data for Curvilinear Boundary Conditions code validation \[Back to [top](#toc)\]
$$\label{validate}$$

We will validate this curvilinear boundary condition module by comparing its results with the original (trusted) SENR code, as follows:

* **Discrete data test**:
    1. Fill all 10 gridfunctions at each gridpoint with the unique gridpoint integer index `IDX3(i0,i1,i2)`
    1. Apply curvilinear boundary conditions
    1. Compare output data at all gridpoints with those from the original SENR code. Agreement should be perfect.

Another (future, to-be-implemented) test, which will enable us to validate coordinate systems that do not exist within the original SENR code, is described below:

* **Smooth data test** (TODO):
    1. Fill all 10 gridfunctions with data that are smooth in the Cartesian basis.
    1. Apply Jacobian transformation to all data points, to convert to curvilinear basis
    1. Apply curvilinear boundary conditions
    1. Apply Jacobian transformation to all data points, to convert back to Cartesian basis
    1. Compute difference between original Cartesian data and transformed data. Difference should be zero (to within roundoff) at all points except those that are influenced by outer boundary conditions.

```python
%%writefile CurviBoundaryConditions/CurviBC_Discrete_Test.h

void Discrete_initial_data(const int Nxx_plus_2NGHOSTS[3],REAL *in_gfs) {
    #pragma omp parallel for
    for(int i=0;i<Nxx_plus_2NGHOSTS[0]*Nxx_plus_2NGHOSTS[1]*Nxx_plus_2NGHOSTS[2]*NUM_AUX_GFS;i++) {
        in_gfs[i] = (REAL)i;
    }
}
```

    Overwriting CurviBoundaryConditions/CurviBC_Discrete_Test.h

# Step 4: `CurviBC_Playground.c`: The Main C Code \[Back to [top](#toc)\]
$$\label{mainc}$$

```python
# Part P0: Set the number of ghost cells, from NRPy+'s FD_CENTDERIVS_ORDER
with open("CurviBoundaryConditions/NGHOSTS.h", "w") as file:
    file.write("// Part P0: Set the number of ghost zones, from NRPy+'s FD_CENTDERIVS_ORDER\n")
    # Upwinding in BSSN requires that NGHOSTS = FD_CENTDERIVS_ORDER/2 + 1 <- Notice the +1.
    file.write("#define NGHOSTS "+str(int(par.parval_from_str("finite_difference::FD_CENTDERIVS_ORDER")/2)+1)+"\n")
```

```python
%%writefile CurviBoundaryConditions/CurviBC_Playground.c

// Step P1: Import needed header files
#include "NGHOSTS.h" // A NRPy+-generated file, which is set based on FD_CENTDERIVS_ORDER.
#include "stdio.h"
#include "string.h"
#include "stdlib.h"
#include "math.h"
#include "time.h"
#include "stdint.h" // Needed for Windows GCC 6.x compatibility

// Step P2: Add needed #define's to set data type, the IDX4() macro, and the gridfunctions
// Step P2a: set REAL=double, so that all floating point numbers are stored to at least ~16 significant digits.
#define REAL double

// Step P3: Set free parameters
// Step P3a: Free parameters for the numerical grid

// Spherical coordinates parameter:
// Set RMAX, the scale of the numerical domain,
//  and the max radius in Spherical coordinates.
#define RMAX 7.5
// SinhSpherical coordinates parameters:
const REAL AMPL    = RMAX;
const REAL SINHW   = 0.2;
// Cylindrical coordinates parameters:
const REAL ZMIN   = -RMAX;
const REAL ZMAX   =  RMAX;
const REAL RHOMAX =  RMAX;
// Cartesian coordinates parameters:
const REAL xmin = -RMAX, xmax = RMAX;
const REAL ymin = -RMAX, ymax = RMAX;
const REAL zmin = -RMAX, zmax = RMAX;
// SymTP coordinates parameters:
const REAL bScale =   0.5;
const REAL AMAX   =  RMAX;

// Step P4: Declare the IDX4(gf,i,j,k) macro, which enables us to store 4-dimensions of
//          data in a 1D array. In this case, consecutive values of "i"
//          (all other indices held to a fixed value) are consecutive in memory, where
//          consecutive values of "j" (fixing all other indices) are separated by
//          Nxx_plus_2NGHOSTS[0] elements in memory. Similarly, consecutive values of
//          "k" are separated by Nxx_plus_2NGHOSTS[0]*Nxx_plus_2NGHOSTS[1] in memory, etc.
#define IDX4(g,i,j,k) \
( (i) + Nxx_plus_2NGHOSTS[0] * ( (j) + Nxx_plus_2NGHOSTS[1] * ( (k) + Nxx_plus_2NGHOSTS[2] * (g) ) ) )
#define IDX3(i,j,k) ( (i) + Nxx_plus_2NGHOSTS[0] * ( (j) + Nxx_plus_2NGHOSTS[1] * (k) ) )
// Assuming idx = IDX3(i,j,k). Much faster if idx can be reused over and over:
#define IDX4pt(g,idx)   ( (idx) + (Nxx_plus_2NGHOSTS[0]*Nxx_plus_2NGHOSTS[1]*Nxx_plus_2NGHOSTS[2]) * (g) )

// Step P5: Set #define's for BSSN gridfunctions. C code generated above
#include "gridfunction_defines.h"

#define LOOP_REGION(i0min,i0max, i1min,i1max, i2min,i2max) \
  for(int i2=i2min;i2<i2max;i2++) for(int i1=i1min;i1<i1max;i1++) for(int i0=i0min;i0<i0max;i0++)

void xxCart(REAL *xx[3],const int i0,const int i1,const int i2, REAL xCart[3]) {
    REAL xx0 = xx[0][i0];
    REAL xx1 = xx[1][i1];
    REAL xx2 = xx[2][i2];
#include "xxCart.h"
}

// Step P6: Include basic functions needed to impose curvilinear
//          parity and boundary conditions.
#include "curvilinear_parity_and_outer_boundary_conditions.h"

// Step P7: Declare the function for the exact solution. time==0 corresponds to the initial data.
#include "CurviBC_Discrete_Test.h"

// main() function:
// Step 0: Read command-line input, set up grid structure, allocate memory for gridfunctions, set up coordinates
// Step 1: Write test data to gridfunctions
// Step 2: Overwrite all data in ghost zones with NaNs
// Step 3: Output relative error between numerical and exact solution.
// Step 4: Print gridfunction data after curvilinear boundary conditions have been applied:
// Step 5: Free all allocated memory
int main(int argc, const char *argv[]) {

    // Step 0a: Read command-line input, error out if nonconformant
    if(argc != 5 || atoi(argv[1]) < NGHOSTS || atoi(argv[2]) < NGHOSTS || atoi(argv[3]) < NGHOSTS) {
        fprintf(stderr,"Error: Expected one command-line argument: ./CurviBC_Playground Nx0 Nx1 Nx2 [test type: Smooth or Discrete],\n");
        fprintf(stderr,"where Nx[0,1,2] is the number of grid points in the 0, 1, and 2 directions.\n");
        fprintf(stderr,"Nx[] MUST BE larger than NGHOSTS (= %d)\n",NGHOSTS);
        exit(1);
    }
    // Step 0b: Set test type to Smooth or Discrete
    char test_type[100];
    snprintf(test_type,100,"%s",argv[4]);
    if(strncmp("Smooth",test_type,100)!=0 && strncmp("Discrete",test_type,100)!=0) {
        fprintf(stderr,"Error: test type = %s not supported. Choose Smooth or Discrete.\n",test_type);
        exit(1);
    }
    // Step 0b: Set up numerical grid structure, first in space...
    if(atoi(argv[1])%2 != 0 || atoi(argv[2])%2 != 0 || atoi(argv[3])%2 != 0) {
        fprintf(stderr,"Error: Cannot guarantee a proper cell-centered grid if number of grid cells not set to even number.\n");
        fprintf(stderr,"       For example, in case of angular directions, proper symmetry zones will not exist.\n");
        exit(1);
    }
    const int Nxx[3] = { atoi(argv[1]), atoi(argv[2]), atoi(argv[3]) };
    const int Nxx_plus_2NGHOSTS[3] = { Nxx[0]+2*NGHOSTS, Nxx[1]+2*NGHOSTS, Nxx[2]+2*NGHOSTS };
    const int Nxx_plus_2NGHOSTS_tot = Nxx_plus_2NGHOSTS[0]*Nxx_plus_2NGHOSTS[1]*Nxx_plus_2NGHOSTS[2];
    #include "xxminmax.h"

    // Step 0c: Allocate memory for gridfunctions
    REAL *test_gfs = (REAL *)malloc(sizeof(REAL) * NUM_AUX_GFS * Nxx_plus_2NGHOSTS_tot);

    // Step 0d: Set up space and time coordinates
    // Step 0d.i: Set \Delta x^i on uniform grids.
    REAL dxx[3];
    for(int i=0;i<3;i++) dxx[i] = (xxmax[i] - xxmin[i]) / ((REAL)Nxx[i]);

    // Step 0d.ii: Set up uniform coordinate grids
    REAL *xx[3];
    for(int i=0;i<3;i++) {
        xx[i] = (REAL *)malloc(sizeof(REAL)*Nxx_plus_2NGHOSTS[i]);
        for(int j=0;j<Nxx_plus_2NGHOSTS[i];j++) {
            xx[i][j] = xxmin[i] + ((REAL)(j-NGHOSTS) + (1.0/2.0))*dxx[i]; // Cell-centered grid.
        }
    }

    // Step 0e: Find ghostzone mappings and parities:
    gz_map *bc_gz_map = (gz_map *)malloc(sizeof(gz_map)*Nxx_plus_2NGHOSTS_tot);
    parity_condition *bc_parity_conditions = (parity_condition *)malloc(sizeof(parity_condition)*Nxx_plus_2NGHOSTS_tot);
    set_up_bc_gz_map_and_parity_conditions(Nxx_plus_2NGHOSTS,xx,dxx,xxmin,xxmax,  bc_gz_map, bc_parity_conditions);

    // Step 1: Write test data to gridfunctions
    if(strncmp("Discrete",test_type,100)==0) {
        Discrete_initial_data(Nxx_plus_2NGHOSTS, test_gfs);
    } else {
        fprintf(stderr,"Sorry, curvilinear boundary conditions test = %s not yet supported. Feel free to contribute!\n",test_type);
    }

    // Step 2: Overwrite all data in ghost zones with NaNs
    LOOP_REGION(0,Nxx_plus_2NGHOSTS[0], 0,Nxx_plus_2NGHOSTS[1], 0,Nxx_plus_2NGHOSTS[2]) {
        for(int gf=0;gf<NUM_AUX_GFS;gf++) {
            const int idx4 = IDX4(gf,i0,i1,i2);
            if(i0 < NGHOSTS || i0 >= Nxx_plus_2NGHOSTS[0]-NGHOSTS) test_gfs[idx4] = +(0.0 / 0.0);
            if(i1 < NGHOSTS || i1 >= Nxx_plus_2NGHOSTS[1]-NGHOSTS) test_gfs[idx4] = +(0.0 / 0.0);
            if(i2 < NGHOSTS || i2 >= Nxx_plus_2NGHOSTS[2]-NGHOSTS) test_gfs[idx4] = +(0.0 / 0.0);
        }
    }

    // Step 3: Apply curvilinear boundary conditions
    apply_bcs(Nxx, Nxx_plus_2NGHOSTS, bc_gz_map,bc_parity_conditions, NUM_AUX_GFS,aux_gf_parity, test_gfs);

    // Step 4: Print gridfunction data after curvilinear boundary conditions have been applied:
    LOOP_REGION(0,Nxx_plus_2NGHOSTS[2], 0,Nxx_plus_2NGHOSTS[1], 0,Nxx_plus_2NGHOSTS[0]) {
        printf("%d %d %d | ",i0,i1,i2);
        for(int gf=0;gf<NUM_AUX_GFS;gf++) {
            const int idx4 = IDX4(gf,i0,i1,i2);
            if(!isnan(test_gfs[idx4])) {
                printf("%d ",(int)test_gfs[idx4]);
            } else {
                fprintf(stderr,"ERROR: found NaN %d %d %d %d %d\n",gf,i0,i1,i2,NUM_AUX_GFS);
                exit(1);
            }
        }
        printf("\n");
    }

    // Step 5: Free all allocated memory
    free(bc_parity_conditions);
    free(bc_gz_map);
    free(test_gfs);
    for(int i=0;i<3;i++) free(xx[i]);
    return 0;
}
```

    Overwriting CurviBoundaryConditions/CurviBC_Playground.c

```python
import cmdline_helper as cmd
cmd.C_compile("CurviBoundaryConditions/CurviBC_Playground.c", "CurviBC_Playground")
cmd.Execute("CurviBC_Playground", "4 4 4 Discrete", "CurviBoundaryConditions/out4x4x4-Spherical-NGHOSTS4oFD.txt")
print("Finished this code cell.")
```

    Compiling executable...
    Executing `gcc -Ofast -fopenmp -march=native -funroll-loops CurviBoundaryConditions/CurviBC_Playground.c -o CurviBC_Playground -lm`...
    Finished executing in 0.821406126022 seconds.
    Finished compilation.
    Executing `taskset -c 0,1,2,3 ./CurviBC_Playground 4 4 4 Discrete`...
    Finished executing in 0.213351964951 seconds.
    Finished this code cell.

# Step 5: Validation: Compare with original SENR results \[Back to [top](#toc)\]
$$\label{senr_compare}$$

```python
import sys

import filecmp
if filecmp.cmp('CurviBoundaryConditions/out4x4x4-Spherical-NGHOSTS4oFD.txt',
               'CurviBoundaryConditions/SENRout4x4x4-Spherical_NGHOSTS4oFD.txt') == False:
    print("ERROR: Spherical boundary conditions do not agree!")
    sys.exit(1)
else:
    print("Spherical boundary condition comparison test between this tutorial notebook & trusted original SENR code: PASSED")
```

    Spherical boundary condition comparison test between this tutorial notebook & trusted original SENR code: PASSED

```python
import filecmp
import CurviBoundaryConditions.CurviBoundaryConditions as cbcs
cbcs.Set_up_CurviBoundaryConditions()

print("Running discrete CurviBC_Playground test on Spherical data, using NRPy+ CurviBoundaryConditions.CurviBoundaryConditions module.")
cmd.C_compile("CurviBoundaryConditions/CurviBC_Playground.c", "CurviBC_Playground")
cmd.Execute("CurviBC_Playground", "4 4 4 Discrete", "CurviBoundaryConditions/CBCmodule-out4x4x4-Spherical-NGHOSTS4oFD.txt")

# Then we set the coordinate system for the numerical grid
par.set_parval_from_str("reference_metric::CoordSystem","Cylindrical")
rfm.reference_metric()
cbcs.Set_up_CurviBoundaryConditions()
print("Running discrete CurviBC_Playground test on Cylindrical data, using NRPy+ CurviBoundaryConditions.CurviBoundaryConditions module.")
cmd.C_compile("CurviBoundaryConditions/CurviBC_Playground.c", "CurviBC_Playground")
cmd.Execute("CurviBC_Playground", "4 4 4 Discrete", "CurviBoundaryConditions/CBCmodule-out4x4x4-Cylindrical-NGHOSTS4oFD.txt")
print("Finished this code cell.")

print("\n")

if filecmp.cmp('CurviBoundaryConditions/CBCmodule-out4x4x4-Spherical-NGHOSTS4oFD.txt',
               'CurviBoundaryConditions/SENRout4x4x4-Spherical_NGHOSTS4oFD.txt') == False:
    print("ERROR: Spherical boundary conditions do not agree!")
    sys.exit(1)
else:
    print("Spherical boundary condition comparison test between CBC & trusted original SENR code: PASSED")

if filecmp.cmp('CurviBoundaryConditions/CBCmodule-out4x4x4-Cylindrical-NGHOSTS4oFD.txt',
               'CurviBoundaryConditions/SENRout4x4x4-Cylindrical_NGHOSTS4oFD.txt') == False:
    print("ERROR: Cylindrical boundary conditions do not agree!")
    sys.exit(1)
else:
    print("Cylindrical boundary condition comparison test between CBC & trusted original SENR code: PASSED")
```

    Wrote to file "CurviBoundaryConditions/gridfunction_defines.h"
    Wrote to file "CurviBoundaryConditions/set_parity_conditions.h"
    Wrote to file "CurviBoundaryConditions/xxCart.h"
    Wrote to file "CurviBoundaryConditions/xxminmax.h"
    Wrote to file "CurviBoundaryConditions/Cart_to_xx.h"
    Running discrete CurviBC_Playground test on Spherical data, using NRPy+ CurviBoundaryConditions.CurviBoundaryConditions module.
    Compiling executable...
    Executing `gcc -Ofast -fopenmp -march=native -funroll-loops CurviBoundaryConditions/CurviBC_Playground.c -o CurviBC_Playground -lm`...
    Finished executing in 0.821196079254 seconds.
    Finished compilation.
    Executing `taskset -c 0,1,2,3 ./CurviBC_Playground 4 4 4 Discrete`...
    Finished executing in 0.214704990387 seconds.
    Wrote to file "CurviBoundaryConditions/gridfunction_defines.h"
    Wrote to file "CurviBoundaryConditions/set_parity_conditions.h"
    Wrote to file "CurviBoundaryConditions/xxCart.h"
    Wrote to file "CurviBoundaryConditions/xxminmax.h"
    Wrote to file "CurviBoundaryConditions/Cart_to_xx.h"
    Running discrete CurviBC_Playground test on Cylindrical data, using NRPy+ CurviBoundaryConditions.CurviBoundaryConditions module.
    Compiling executable...
    Executing `gcc -Ofast -fopenmp -march=native -funroll-loops CurviBoundaryConditions/CurviBC_Playground.c -o CurviBC_Playground -lm`...
    Finished executing in 0.818950891495 seconds.
    Finished compilation.
    Executing `taskset -c 0,1,2,3 ./CurviBC_Playground 4 4 4 Discrete`...
    Finished executing in 0.215003013611 seconds.
    Finished this code cell.

    Spherical boundary condition comparison test between CBC & trusted original SENR code: PASSED
    Cylindrical boundary condition comparison test between CBC & trusted original SENR code: PASSED

# Step 6: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Start_to_Finish-Curvilinear_BCs.pdf](Tutorial-Start_to_Finish-Curvilinear_BCs.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx Tutorial-Start_to_Finish-Curvilinear_BCs.ipynb
!pdflatex -interaction=batchmode Tutorial-Start_to_Finish-Curvilinear_BCs.tex
!pdflatex -interaction=batchmode Tutorial-Start_to_Finish-Curvilinear_BCs.tex
!pdflatex -interaction=batchmode Tutorial-Start_to_Finish-Curvilinear_BCs.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    [NbConvertApp] Converting notebook Tutorial-Start_to_Finish-Curvilinear_BCs.ipynb to latex
    [NbConvertApp] Writing 110822 bytes to Tutorial-Start_to_Finish-Curvilinear_BCs.tex
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode