### Lab 3: Expectation Maximization and Variational Autoencoder

### Machine Learning 2 (2017/2018)

* The lab exercises should be made in groups of two or three people.
* The deadline is Friday, 01.06.
* Assignment should be submitted through BlackBoard! Make sure to include your and your teammates' names with the submission.
* Attach the .IPYNB (IPython Notebook) file containing your code and answers. Naming of the file should be "studentid1\_studentid2\_lab#", for example, the attached file should be "12345\_12346\_lab1.ipynb". Only use underscores ("\_") to connect ids, otherwise the files cannot be parsed.

Notes on implementation:

* You should write your code and answers in an IPython Notebook: http://ipython.org/notebook.html. If you have problems, please ask.
* Use __one cell__ for code and markdown answers only!
    * Put all code in the cell with the ```# YOUR CODE HERE``` comment and overwrite the ```raise NotImplementedError()``` line.
    * For theoretical questions, put your solution using LaTeX style formatting in the YOUR ANSWER HERE cell.
* Among the first lines of your notebook should be "%pylab inline". This imports all required modules, and your plots will appear inline.
* Large parts of you notebook will be graded automatically. Therefore it is important that your notebook can be run completely without errors and within a reasonable time limit. To test your notebook before submission, select Kernel -> Restart \& Run All.
$\newcommand{\bx}{\mathbf{x}} \newcommand{\bpi}{\mathbf{\pi}} \newcommand{\bmu}{\mathbf{\mu}} \newcommand{\bX}{\mathbf{X}} \newcommand{\bZ}{\mathbf{Z}} \newcommand{\bz}{\mathbf{z}}$

### Installing PyTorch

In this lab we will use PyTorch. PyTorch is an open source deep learning framework primarily developed by Facebook's artificial-intelligence research group. In order to install PyTorch in your conda environment go to https://pytorch.org and select your operating system, conda, Python 3.6, no cuda. Copy the text from the "Run this command:" box. Now open a terminal and activate your 'ml2labs' conda environment. Paste the text and run. After the installation is done you should restart Jupyter.

### MNIST data

In this Lab we will use several methods for unsupervised learning on the MNIST dataset of written digits. The dataset contains digital images of handwritten numbers $0$ through $9$. Each image has 28x28 pixels that each take 256 values in a range from white ($= 0$) to  black ($=1$). The labels belonging to the images are also included.
Fortunately, PyTorch comes with a MNIST data loader. The first time you run the box below it will download the MNIST data set. That can take a couple of minutes.
The main data types in PyTorch are tensors. For Part 1, we will convert those tensors to numpy arrays. In Part 2, we will use the torch module to directly work with PyTorch tensors.

```python
%pylab inline
import torch
from torchvision import datasets, transforms

train_dataset = datasets.MNIST('../data', train=True, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),
                       transforms.Normalize((0.1307,), (0.3081,))
                   ]))

train_labels = train_dataset.train_labels.numpy()
train_data = train_dataset.train_data.numpy()
# For EM we will use flattened data
train_data = train_data.reshape(train_data.shape[0], -1)

```

    Populating the interactive namespace from numpy and matplotlib

## Part 1: Expectation Maximization
We will use the Expectation Maximization (EM) algorithm for the recognition of handwritten digits in the MNIST dataset. The images are modelled as a Bernoulli mixture model (see Bishop $\S9.3.3$):
$$
p(\bx|\bmu, \bpi) = \sum_{k=1}^K  \pi_k \prod_{i=1}^D \mu_{ki}^{x_i}(1-\mu_{ki})^{(1-x_i)}
$$
where $x_i$ is the value of pixel $i$ in an image, $\mu_{ki}$ represents the probability that pixel $i$ in class $k$ is black, and $\{\pi_1, \ldots, \pi_K\}$ are the mixing coefficients of classes in the data. We want to use this data set to classify new images of handwritten numbers.

### 1.1 Binary data (5 points)
As we like to apply our Bernoulli mixture model, write a function `binarize` to convert the (flattened) MNIST data to binary images, where each pixel $x_i \in \{0,1\}$, by thresholding at an appropriate level.

```python
def binarize(X):
    ######################## ######################## ########################
    # YOUR CODE HERE

    X_ = np.around(X.astype(np.double)/255)
    return X_
```

```python
# Test test test!
bin_train_data = binarize(train_data)
assert bin_train_data.dtype == np.float
assert bin_train_data.shape == train_data.shape

```

Sample a few images of digits $2$, $3$ and $4$; and show both the original and the binarized image together with their label.

```python
# YOUR CODE HERE
######################## ######################## ########################

def plot_MNIST_digits(data):

    i = data.shape[0]

    for j in range(i):
        plt.subplot(1,i,j+1)
        plt.imshow(data[j].reshape(28,28), interpolation = 'nearest',
                   cmap ='Greys')
        plt.axis('off')
    plt.show()

def plot_bin_MNIST_digits(digits, bin_train_data, train_labels, train_data):

    for digit in digits:

        print('\n\n'+'='*50+'\n\n')
        print('plotting label {}:\n'.format(digit))

        bin_zip = zip(bin_train_data, train_labels)
        float_zip = zip(train_data, train_labels)
        bin_   = np.array([i for i,n in bin_zip if n == digit])
        float_ = np.array([i for i,n in float_zip if n == digit])

        print('... float')
        plot_MNIST_digits(np.vstack((float_[0:5])))

        print('... binary')
        plot_MNIST_digits(np.vstack((bin_[0:5])))

    print('\n\n'+'='*50+'\n\n')

plot_bin_MNIST_digits([2,3,4], bin_train_data, train_labels, train_data)

######################## ######################## ########################
```

### 1.2 Implementation (40 points)
You are going to write a function ```EM(X, K, max_iter)``` that implements the EM algorithm on the Bernoulli mixture model.

The only parameters the function has are:
* ```X``` :: (NxD) array of input training images
* ```K``` :: size of the latent space
* ```max_iter``` :: maximum number of iterations, i.e. one E-step and one M-step

You are free to specify your return statement.

Make sure you use a sensible way of terminating the iteration process early to prevent unnecessarily running through all epochs. Vectorize computations using ```numpy``` as  much as possible.

You should implement the `E_step(X, mu, pi)` and `M_step(X, gamma)` separately in the functions defined below. These you can then use in your function `EM(X, K, max_iter)`.

```python
def E_step(X, mu, pi):

    # YOUR CODE HERE
    ######################## ######################## ########################

    # expand X over axis 1 which gives us from (NxD) for X to (Nx1xD) for X_
    X_ = np.expand_dims(X,axis=1)

    # calculate unnormalized gamma
    g = pi*np.prod(mu**X_ * (1-mu)**(1-X_), axis=2)

    # calculate normalization and expend it from N to Nx1
    normal = np.expand_dims(np.sum(g,axis=1),axis = 1)

    # get normalized gamma and handle devision by zero
    gamma = np.divide(g, normal, out=np.zeros_like(g), where=normal!=0)

    ######################## ######################## ########################
    return gamma
```

```python
# Let's test on 5 datapoints
n_test = 5
X_test = bin_train_data[:n_test]
D_test, K_test = X_test.shape[1], 10

np.random.seed(2018)
mu_test = np.random.uniform(low=.25, high=.75, size=(K_test,D_test))
pi_test = np.ones(K_test) / K_test

gamma_test = E_step(X_test, mu_test, pi_test)
assert gamma_test.shape == (n_test, K_test)

```

```python
def M_step(X, gamma):

    # YOUR CODE HERE
    ######################## ######################## ########################

    N_k = np.sum(gamma, axis=0)
    pi = N_k / np.sum(N_k)

    # we expend similar to the E-step but now over the second dimension of gamma
    X_ = np.expand_dims(X,axis=1)
    X__ = np.repeat(X_,gamma.shape[1],axis=1)

    # expend from Nxk to NxKx1
    gamma_ = np.expand_dims(gamma,axis=2)

    mu_ = np.sum(X__*gamma_,axis=0)

    mu = mu_/np.expand_dims(N_k,axis=1)

    ######################## ######################## ########################
    return mu, pi
```

```python
# Oh, let's test again
mu_test, pi_test = M_step(X_test, gamma_test)

assert mu_test.shape == (K_test,D_test)
assert pi_test.shape == (K_test, )

```

```python
import time
def EM(X, K, max_iter, mu=None, pi=None, plotting = False, threshold = 1e-1):

    # YOUR CODE HERE
    ######################## ######################## ########################

    start_total = time.time()

    print('='*50+'\ninitialize EM\n')

    # init mu and pi if not given (reuse initialisation from above tests.)
    if mu is None:
        mu = np.random.uniform(low=.25, high=.75, size=(K,X_test.shape[1]))
    if pi is None:
        pi = np.ones(K) / K

    mu_ = mu
    pi_ = pi

    if plotting:
        print('initialize mu:')
        plot_MNIST_digits(mu)
        print('='*50+'\n')

    # loop over epochs
    for i in range(max_iter):

        start = time.time()

        print('iteration {}'.format(i+1))

        gamma = E_step(X,mu,pi)
        mu, pi = M_step(X,gamma)

        # plotting latent space
        if plotting: plot_MNIST_digits(mu)

        # break if differences in mu and pi are small -> converged
        if all([i>0,np.linalg.norm(mu_-mu)<threshold ,
                    np.linalg.norm(pi_-pi)<threshold]):
            print('stop because of convergence after {} iterations'.format(i+1))
            break

        mu_ = mu
        pi_ = pi

    print('\n\n'+'='*50+'\n\n{} iterations in {} min\n\n'.format(
        i+1,(time.time()-start_total)/60)+'='*50+'\n\n')

    return mu, pi

    ######################## ######################## ########################
```

### 1.3 Three digits experiment (10 points)
In analogue with Bishop $\S9.3.3$, sample a training set consisting of only __binary__ images of written digits $2$, $3$, and $4$. Run your EM algorithm and show the reconstructed digits.

```python
# YOUR CODE HERE
######################## ######################## ########################

# fast version: only use part of the data
bin_zip = zip(bin_train_data[:5000], train_labels[:5000])

X = np.array([i for i,n in bin_zip if any([n==2, n==3, n==4])])

K = 3
epochs = 10
mu,pi = EM(X,K,epochs,plotting=True)

######################## ######################## ########################
```

Can you identify which element in the latent space corresponds to which digit? What are the identified mixing coefficients for digits $2$, $3$ and $4$, and how do these compare to the true ones?

YOUR ANSWER HERE

Yes, we can identify which element in the latent space corresponds to which digit, as there is one element for each of the digits $2,3$ and $4$.

The mixing coefficients for the digits are $0.35, 0.30, 0.35$. This is slightly different to the true values, where each number should appear $\frac{1}{3}$ of the time.

```python
print(pi)
```

    [0.34954996 0.29628561 0.35416443]

### 1.4 Experiments (20 points)
Perform the follow-up experiments listed below using your implementation of the EM algorithm. For each of these, describe/comment on the obtained results and give an explanation. You may still use your dataset with only digits 2, 3 and 4 as otherwise computations can take very long.

#### 1.4.1 Size of the latent space (5 points)
Run EM with $K$ larger or smaller than the true number of classes. Describe your results.

```python
# YOUR CODE HERE
######################## ######################## ########################

# smaller
K = 2
_, _ = EM(X,K,epochs,plotting=True)

######################## ######################## ########################
```

###YOUR ANSWER HERE

When we use $K=2$, the latent space can only represent two different classes, while there are still three classes present. As a result, the latent representation starts mixing the representations for the different numbers. We can see this especially for the digits $2$ and $4$, which are merged into a joined representation. Additionally, we can see that in contrast to the setting with three classes in the latent space, the algorithm fails to converge within ten epochs.

```python
# YOUR CODE HERE
######################## ######################## ########################

# larger
K = 5
_, _ = EM(X,K,epochs,plotting=True)

######################## ######################## ########################
```

###YOUR ANSWER HERE

When we use $K=5$, the latent space can represent five different classes, while there are still three classes present. As a result, the latent representation starts splitting the representations of the given numbers. We can see this especially for the digits $2$ and $3$. For the $2$, we can see that we learn two depictions: one which encompasses a distinctive loop, while the other does not. For the $3$, we get one representation that is squeezed more than the other and one of them seems to be mixed with a representation of the digit two.

#### 1.4.2 Identify misclassifications (10 points)
How can you use the data labels to assign a label to each of the clusters/latent variables? Use this to identify images that are 'misclassified' and try to understand why they are. Report your findings.

```python
# YOUR CODE HERE
######################## ######################## ########################

def get_true_mu(bin_train_data,train_labels):

    true_mu = np.zeros((3,bin_train_data.shape[1]))

    for i in range(2,5):
        #find indices of images belonging to this label
        indx = np.argwhere(train_labels==i)

        # average over these images
        true_mu[i-2] = np.average(bin_train_data[indx],axis=0)

    return true_mu

def classify_image(image, mu):
    """
    takes image and mu as an input and maps the image to the closest mu
    """

    # L1 distance between mu and image
    dist = np.zeros((mu.shape[0],))
    for c in range(mu.shape[0]):
        dist[c] = numpy.linalg.norm(image-mu[c])

    # minimal distance is class
    classification = np.argmin(dist)

    return classification

def find_misclassification(mu, bin_train_data, train_labels, number_mc = 5):

    #find true labels of the latent space
    true_mu = get_true_mu(bin_train_data,train_labels)

    mu_label = np.zeros((mu.shape[0],))
    for i in range(len(mu)):
        mu_label[i] = classify_image(mu[i], true_mu) + 2

    #initializing arrays to save the misclassified results
    mc_pic = np.zeros((number_mc,mu.shape[1]))
    mc_pred = np.zeros((number_mc,))
    mc_label = np.zeros((number_mc,))
    i = 0
    mc_counter = 0

    while mc_counter < number_mc:
        if train_labels[i] >= 2 and train_labels[i] <= 4:      ## only consider images with true labels 2, 3 or 4
            c_pred = mu_label[classify_image(bin_train_data[i], mu)]

            if c_pred != train_labels[i]:
              mc_pic[mc_counter] = bin_train_data[i]
              mc_pred[mc_counter] = c_pred
              mc_label[mc_counter] = train_labels[i]
              mc_counter = mc_counter + 1
        i = i + 1

    print('True label - predicted label:')
    for j in range(number_mc):
        plt.subplot(1,number_mc,j+1)
        plt.imshow(mc_pic[j].reshape(28,28), interpolation = 'nearest',
                   cmap ='Greys')
        plt.title("{0:d} - {1:d}".format(int(mc_label[j]), int(mc_pred[j])))
        plt.axis('off')
    plt.show()

# using mu from the run above with the correct number of classes

find_misclassification(mu, bin_train_data, train_labels, number_mc=10)
print('Comparing to the mus found using EM:')
plot_MNIST_digits(mu)

######################## ######################## ########################
```

YOUR ANSWER HERE

We can use the data labels in order to calculate the true values of the mus by averaging over all images that are assigned with this label. Then, we compare these true values to the mus we found using the EM-algorithm and label them with the according label if they show the smallest difference between one another.

Using the same technique, we can label the images of the dataset by comparing the to the mus we found using the EM-algorithm and assigning them with the correponding label of the most similar one.  Comparing our result to the true labels, gives us a number of misclassifications as shown above.

We can see that a lot of twos get misclassified, especially if they do not encompass the loop as it is present in the latent representation. We can conclude that in general samples that are too different from the latent representation will be misclassified.

#### 1.4.3 Initialize with true values (5 points)
Initialize the three classes with the true values of the parameters and see what happens. Report your results.

```python
# YOUR CODE HERE
######################## ######################## ########################

K = 3
true_mu = get_true_mu(bin_train_data,train_labels)
_, _ = EM(X,K,epochs,mu = true_mu, plotting=True)

######################## ######################## ########################
```

YOUR ANSWER HERE

When we initialize with the true values for mu, we can see that the algorithm changes this representation only marginally. For example, the loop of the two slightly increases. The algorithm manages to converge within 3 epochs.

## Part 2: Variational Auto-Encoder

A Variational Auto-Encoder (VAE) is a probabilistic model $p(\bx, \bz)$ over observed variables $\bx$ and latent variables and/or parameters $\bz$. Here we distinguish the decoder part, $p(\bx | \bz) p(\bz)$ and an encoder part $p(\bz | \bx)$ that are both specified with a neural network. A lower bound on the log marginal likelihood $\log p(\bx)$ can be obtained by approximately inferring the latent variables z from the observed data x using an encoder distribution $q(\bz| \bx)$ that is also specified as a neural network. This lower bound is then optimized to fit the model to the data.

The model was introduced by Diederik Kingma (during his PhD at the UVA) and Max Welling in 2013, https://arxiv.org/abs/1312.6114.

Since it is such an important model there are plenty of well written tutorials that should help you with the assignment. E.g: https://jaan.io/what-is-variational-autoencoder-vae-tutorial/.

In the following, we will make heavily use of the torch module, https://pytorch.org/docs/stable/index.html. Most of the time replacing `np.` with `torch.` will do the trick, e.g. `np.sum` becomes `torch.sum` and `np.log` becomes `torch.log`. In addition, we will use `torch.FloatTensor()` as an equivalent to `np.array()`. In order to train our VAE efficiently we will make use of batching. The number of data points in a batch will become the first dimension of our data tensor, e.g. A batch of 128 MNIST images has the dimensions [128, 1, 28, 28]. To check check the dimensions of a tensor you can call `.size()`.

### 2.1 Loss function
The objective function (variational lower bound), that we will use to train the VAE, consists of two terms: a log Bernoulli loss (reconstruction loss) and a Kullback–Leibler divergence. We implement the two terms separately and combine them in the end.
As seen in Part 1: Expectation Maximization, we can use a multivariate Bernoulli distribution to model the likelihood $p(\bx | \bz)$ of black and white images. Formally, the variational lower bound is maximized but in PyTorch we are always minimizing therefore we need to calculate the negative log Bernoulli loss and Kullback–Leibler divergence.

### 2.1.1 Negative Log Bernoulli loss (5 points)
The negative log Bernoulli loss is defined as,

\begin{align}
loss = - (\sum_i^D \bx_i \log \hat{\bx_i} + (1 − \bx_i) \log(1 − \hat{\bx_i})).
\end{align}

Write a function `log_bernoulli_loss` that takes a D dimensional vector `x`, its reconstruction `x_hat` and returns the negative log Bernoulli loss. Make sure that your function works for batches of arbitrary size.

```python
def log_bernoulli_loss(x_hat, x):

    # YOUR CODE HERE
    ######################## ######################## ########################

    n = torch.mul(x, torch.log(x_hat)) + torch.mul(1 - x, torch.log(1 - x_hat))
    loss = -n.sum()

    ######################## ######################## ########################
    return loss

```

```python
### Test test test
x_test = torch.FloatTensor([[0.1, 0.2, 0.3, 0.4], [0.5, 0.6, 0.7, 0.8], [0.9, 0.9, 0.9, 0.9]])
x_hat_test = torch.FloatTensor([[0.11, 0.22, 0.33, 0.44], [0.55, 0.66, 0.77, 0.88], [0.99, 0.99, 0.99, 0.99]])

assert log_bernoulli_loss(x_hat_test, x_test) > 0.0
assert log_bernoulli_loss(x_hat_test, x_test) < 10.0

```

### 2.1.2 Negative Kullback–Leibler divergence (10 Points)
The variational lower bound (the objective to be maximized) contains a KL term $D_{KL}(q(\bz)||p(\bz))$ that can often be calculated analytically. In the VAE we assume $q = N(\bz, \mu, \sigma^2I)$ and $p = N(\bz, 0, I)$. Solve analytically!

YOUR ANSWER HERE

\begin{align*}
\newcommand{\xscalar}{x}
\newcommand{\xvec}{{\bf z}}
\newcommand{\Lmat}{{\sigma^2 \bf I}}
\newcommand{\Sigmamat}{\bf I}
\newcommand{\SigmamatInv}{\Sigmamat^{-1}}
\newcommand{\mvec}{\boldsymbol{\mu}}
        \mathcal{KL}(q||p) &= - \int q(\xvec) \log \left(\frac{p(\xvec)}{q(\xvec)}\right) d\xvec \\
        &= \int q(\xvec) \log (p(\xvec) - q(\xvec)) d\xvec \\
        &= \int q(\xvec) [ - \frac{D}{2} \log(2\pi) - \frac{1}{2} \log(|\Sigmamat|) - \frac{1}{2} (\xvec)^T \SigmamatInv(\xvec) \\
        & + \frac{D}{2} \log(2\pi) + \frac{1}{2} \log(|\Lmat|) + \frac{1}{2} (\xvec-\mvec)^T \Lmat^{-1}(\xvec-\mvec) ] d\xvec \\
        &= \frac{1}{2} \log \left(\frac{|\Lmat|}{|\Sigmamat|}\right) \int q(\xvec) d\xvec \int q(\xvec) [ - \frac{1}{2} (\xvec)^T \SigmamatInv(\xvec) \\
        & + \frac{1}{2} (\xvec-\mvec)^T \Lmat^{-1}(\xvec-\mvec) ] d\xvec
    \end{align*}
    We use $\int q(\xvec) d\xvec = 1$ and the law of the unconscious statistician to get
    \begin{align*}
        &= \frac{1}{2} \log \left(\frac{|\Lmat|}{|\Sigmamat|}\right) - \frac{1}{2} \mathbb{E}[ (\xvec)^T \SigmamatInv(\xvec)] +
        \frac{1}{2} \mathbb{E}[(\xvec-\mvec)^T \Lmat^{-1}(\xvec-\mvec)]
    \end{align*}
    Since $p$ is considered to be the original distribution, it follows that $\xvec \sim \mathcal{N}(\xvec|0, \Sigmamat)$. Therefore, we can use equation 380 from the matrix cookbook to get:
    \begin{align*}
        &= \frac{1}{2} \log \left(\frac{|\Lmat|}{|\Sigmamat|}\right) - \frac{1}{2}[\SigmamatInv + Tr(\SigmamatInv\Sigmamat)] \\
        & + \frac{1}{2}[(-\mvec)^T \Lmat^{-1}(-\mvec) + Tr(\Lmat^{-1}\Sigmamat)] \\
        &= \frac{1}{2} \left[\log \left(\frac{|\Lmat|}{|\Sigmamat|}\right) - Tr(\Sigmamat)+ (-\mvec)^T \Lmat^{-1}(-\mvec) + Tr(\Lmat^{-1}\Sigmamat)\right] \\
        &= \frac{1}{2} \left[\log \left(\frac{|\Lmat|}{|\Sigmamat|}\right) - D + (-\mvec)^T \Lmat^{-1}(-\mvec) + Tr(\Lmat^{-1}\Sigmamat)\right]\\
        \end{align*}

Write a function `KL_loss` that takes two J dimensional vectors `mu` and `logvar` and returns the negative Kullback–Leibler divergence. Where `logvar` is $\log(\sigma^2)$. Make sure that your function works for batches of arbitrary size.

```python
def KL_loss(mu, logvar):

    # YOUR CODE HERE
    ######################## ######################## ########################

    loss = -0.5 * torch.sum(1 + logvar - torch.pow(mu, 2) - torch.exp(logvar))

    ######################## ######################## ########################
    return loss

```

```python
### Test test test
mu_test = torch.FloatTensor([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
logvar_test = torch.FloatTensor([[0.01, 0.02], [0.03, 0.04], [0.05, 0.06]])

assert KL_loss(mu_test, logvar_test) > 0.0
assert KL_loss(mu_test, logvar_test) < 10.0

```

### 2.1.3 Putting the losses together (5 points)
Write a function `loss_function` that takes a D dimensional vector `x`, its reconstruction `x_hat`, two J dimensional vectors `mu` and `logvar` and returns the final loss. Make sure that your function works for batches of arbitrary size.

```python
def loss_function(x_hat, x, mu, logvar):

    # YOUR CODE HERE
    ######################## ######################## ########################

    entropy_loss = log_bernoulli_loss(x_hat, x)
    kl_loss = KL_loss(mu,logvar)
    loss = entropy_loss+kl_loss

    ######################## ######################## ########################

    return loss

```

```python
x_test = torch.FloatTensor([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9]])
x_hat_test = torch.FloatTensor([[0.11, 0.22, 0.33], [0.44, 0.55, 0.66], [0.77, 0.88, 0.99]])
mu_test = torch.FloatTensor([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
logvar_test = torch.FloatTensor([[0.01, 0.02], [0.03, 0.04], [0.05, 0.06]])

assert loss_function(x_hat_test, x_test, mu_test, logvar_test) > 0.0
assert loss_function(x_hat_test, x_test, mu_test, logvar_test) < 10.0

```

### 2.2 The model
Below you see a data structure for the VAE. The modell itself consists of two main parts the encoder (images $\bx$ to latent variables $\bz$) and the decoder (latent variables $\bz$ to images $\bx$). The encoder is using 3 fully-connected layers, whereas the decoder is using fully-connected layers. Right now the data structure is quite empty, step by step will update its functionality. For test purposes we will initialize a VAE for you. After the data structure is completed you will do the hyperparameter search.

```python
from torch import nn
from torch.nn import functional as F

class VAE(nn.Module):
    def __init__(self, fc1_dims, fc21_dims, fc22_dims, fc3_dims, fc4_dims):
        super(VAE, self).__init__()

        self.fc1 = nn.Linear(*fc1_dims)
        self.fc21 = nn.Linear(*fc21_dims)
        self.fc22 = nn.Linear(*fc22_dims)
        self.fc3 = nn.Linear(*fc3_dims)
        self.fc4 = nn.Linear(*fc4_dims)

    def encode(self, x):
        # To be implemented
        raise Exception('Method not implemented')

    def reparameterize(self, mu, logvar):
        # To be implemented
        raise Exception('Method not implemented')

    def decode(self, z):
        # To be implemented
        raise Exception('Method not implemented')

    def forward(self, x):
        # To be implemented
        raise Exception('Method not implemented')

VAE_test = VAE(fc1_dims=(784, 4), fc21_dims=(4, 2), fc22_dims=(4, 2), fc3_dims=(2, 4), fc4_dims=(4, 784))

```

### 2.3 Encoding (10 points)
Write a function `encode` that gets a vector `x` with 784 elements (flattened MNIST image) and returns `mu` and `logvar`. Your function should use three fully-connected layers (`self.fc1()`, `self.fc21()`, `self.fc22()`). First, you should use `self.fc1()` to embed `x`. Second, you should use `self.fc21()` and `self.fc22()` on the embedding of `x` to compute `mu` and `logvar` respectively. PyTorch comes with a variety of activation functions, the most common calls are `F.relu()`, `F.sigmoid()`, `F.tanh()`. Make sure that your function works for batches of arbitrary size.

```python
def encode(self, x):

    # YOUR CODE HERE
    ######################## ######################## ########################

    z = F.relu(self.fc1(x))
    mu = self.fc21(z)
    logvar = self.fc22(z)

    ######################## ######################## ########################

    return mu, logvar

```

```python
### Test, test, test
VAE.encode = encode

x_test = torch.ones((5,784))
mu_test, logvar_test = VAE_test.encode(x_test)

assert np.allclose(mu_test.size(), [5, 2])
assert np.allclose(logvar_test.size(), [5, 2])

```

### 2.4 Reparameterization (10 points)
One of the major question that the VAE is answering, is 'how to take derivatives with respect to the parameters of a stochastic variable?', i.e. if we are given $\bz$ that is drawn from a distribution $q(\bz|\bx)$, and we want to take derivatives. This step is necessary to be able to use gradient-based optimization algorithms like SGD.
For some distributions, it is possible to reparameterize samples in a clever way, such that the stochasticity is independent of the parameters. We want our samples to deterministically depend on the parameters of the distribution. For example, in a normally-distributed variable with mean $\mu$ and standard deviation $\sigma$, we can sample from it like this:

\begin{align}
\bz = \mu + \sigma \odot \epsilon,
\end{align}

where $\odot$ is the element-wise multiplication and $\epsilon$ is sampled from $N(0, I)$.

Write a function `reparameterize` that takes two J dimensional vectors `mu` and `logvar`. It should return $\bz = \mu + \sigma \odot \epsilon$.

```python
def reparameterize(self, mu, logvar):

    # YOUR CODE HERE
    ######################## ######################## ########################

    sigma = torch.sqrt(torch.exp(logvar))
    z = mu + sigma * torch.randn(mu.size())

    ######################## ######################## ########################
    return z

```

```python
### Test, test, test
VAE.reparameterize = reparameterize
VAE_test.train()

mu_test = torch.FloatTensor([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
logvar_test = torch.FloatTensor([[0.01, 0.02], [0.03, 0.04], [0.05, 0.06]])

z_test = VAE_test.reparameterize(mu_test, logvar_test)

assert np.allclose(z_test.size(), [3, 2])
assert z_test[0][0] < 5.0
assert z_test[0][0] > -5.0

```

### 2.5 Decoding (10 points)
Write a function `decode` that gets a vector `z` with J elements and returns a vector `x_hat` with 784 elements (flattened MNIST image). Your function should use two fully-connected layers (`self.fc3()`, `self.fc4()`). PyTorch comes with a variety of activation functions, the most common calls are `F.relu()`, `F.sigmoid()`, `F.tanh()`. Make sure that your function works for batches of arbitrary size.

```python
def decode(self, z):

    # YOUR CODE HERE
    ######################## ######################## ########################

    out3 = F.relu(self.fc3(z))
    x_hat = F.sigmoid(self.fc4(out3))

    ######################## ######################## ########################
    return x_hat

```

```python
# test test test
VAE.decode = decode

z_test = torch.ones((5,2))
x_hat_test = VAE_test.decode(z_test)

assert np.allclose(x_hat_test.size(), [5, 784])
assert (x_hat_test <= 1).all()
assert (x_hat_test >= 0).all()

```

### 2.6 Forward pass (10)
To complete the data structure you have to define a forward pass through the VAE. A single forward pass consists of the encoding of an MNIST image $\bx$ into latent space $\bz$, the reparameterization of $\bz$ and the decoding of $\bz$ into an image $\bx$.

Write a function `forward` that gets a a vector `x` with 784 elements (flattened MNIST image) and returns a vector `x_hat` with 784 elements (flattened MNIST image), `mu` and `logvar`.

```python
def forward(self, x):
    x = x.view(-1, 784)

    # YOUR CODE HERE
    ######################## ######################## ########################
    mu, logvar = self.encode(x)
    z = self.reparameterize(mu, logvar)
    x_hat = self.decode(z)

    ######################## ######################## ########################
    return x_hat, mu, logvar

```

```python
# test test test
VAE.forward = forward

x_test = torch.ones((5,784))
x_hat_test, mu_test, logvar_test = VAE_test.forward(x_test)

assert np.allclose(x_hat_test.size(), [5, 784])
assert np.allclose(mu_test.size(), [5, 2])
assert np.allclose(logvar_test.size(), [5, 2])

```

### 2.7 Training (15)
We will now train the VAE using an optimizer called Adam, https://arxiv.org/abs/1412.6980. The code to train a model in PyTorch is given below.

```python
from torch.autograd import Variable

def train(epoch, train_loader, model, optimizer):
    model.train()
    train_loss = 0
    for batch_idx, (data, _) in enumerate(train_loader):
        data = Variable(data)
        optimizer.zero_grad()
        recon_batch, mu, logvar = model(data)
        loss = loss_function(recon_batch, data.view(-1, 784), mu, logvar)
        loss.backward()
        train_loss += loss.data
        optimizer.step()
        if batch_idx % 100 == 0:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                100. * batch_idx / len(train_loader),
                loss.data / len(data)))

    print('====> Epoch: {} Average loss: {:.4f}'.format(
          epoch, train_loss / len(train_loader.dataset)))

```

Let's train. You have to choose the hyperparameters. Make sure your loss is going down in a reasonable amount of epochs (around 10).

```python
# Hyperparameters
# fc1_dims = (?,?)
# fc21_dims =
# fc22_dims =
# fc3_dims =
# fc4_dims =
# lr =
# batch_size =
# epochs =

# YOUR CODE HERE
######################## ######################## ########################
# using the parameters as given in: https://github.com/pytorch/examples/tree/master/vae
fc1_dims = (784,400)
fc21_dims = (400,20)
fc22_dims = (400,20)
fc3_dims = (20,400)
fc4_dims = (400,784)
lr = 1e-3
batch_size = 128
epochs = 10
######################## ######################## ########################
```

```python
# This cell contains a hidden test, please don't delete it, thx
```

Run the box below to train the model using the hyperparameters you entered above.

```python
from torchvision import datasets, transforms
from torch import nn, optim

# Load data
train_data = datasets.MNIST('../data', train=True, download=True,
                   transform=transforms.ToTensor())

train_loader = torch.utils.data.DataLoader(train_data,
                                           batch_size=batch_size, shuffle=True, **{})

# Init model
VAE_MNIST = VAE(fc1_dims=fc1_dims, fc21_dims=fc21_dims, fc22_dims=fc22_dims, fc3_dims=fc3_dims, fc4_dims=fc4_dims)

# Init optimizer
optimizer = optim.Adam(VAE_MNIST.parameters(), lr=lr)

# Train
for epoch in range(1, epochs + 1):
    train(epoch, train_loader, VAE_MNIST, optimizer)

```

    Train Epoch: 1 [0/60000 (0%)]	Loss: 549.927795
    Train Epoch: 1 [12800/60000 (21%)]	Loss: 187.774139
    Train Epoch: 1 [25600/60000 (43%)]	Loss: 151.297928
    Train Epoch: 1 [38400/60000 (64%)]	Loss: 135.530563
    Train Epoch: 1 [51200/60000 (85%)]	Loss: 133.712051
    ====> Epoch: 1 Average loss: 162.9749
    Train Epoch: 2 [0/60000 (0%)]	Loss: 129.225845
    Train Epoch: 2 [12800/60000 (21%)]	Loss: 124.249214
    Train Epoch: 2 [25600/60000 (43%)]	Loss: 119.403214
    Train Epoch: 2 [38400/60000 (64%)]	Loss: 117.094475
    Train Epoch: 2 [51200/60000 (85%)]	Loss: 110.436165
    ====> Epoch: 2 Average loss: 120.6739
    Train Epoch: 3 [0/60000 (0%)]	Loss: 114.500031
    Train Epoch: 3 [12800/60000 (21%)]	Loss: 111.776672
    Train Epoch: 3 [25600/60000 (43%)]	Loss: 114.054794
    Train Epoch: 3 [38400/60000 (64%)]	Loss: 111.648720
    Train Epoch: 3 [51200/60000 (85%)]	Loss: 111.599319
    ====> Epoch: 3 Average loss: 114.1491
    Train Epoch: 4 [0/60000 (0%)]	Loss: 110.490540
    Train Epoch: 4 [12800/60000 (21%)]	Loss: 111.640862
    Train Epoch: 4 [25600/60000 (43%)]	Loss: 110.019012
    Train Epoch: 4 [38400/60000 (64%)]	Loss: 111.838921
    Train Epoch: 4 [51200/60000 (85%)]	Loss: 110.753128
    ====> Epoch: 4 Average loss: 111.4047
    Train Epoch: 5 [0/60000 (0%)]	Loss: 111.950302
    Train Epoch: 5 [12800/60000 (21%)]	Loss: 109.241913
    Train Epoch: 5 [25600/60000 (43%)]	Loss: 107.332108
    Train Epoch: 5 [38400/60000 (64%)]	Loss: 110.472519
    Train Epoch: 5 [51200/60000 (85%)]	Loss: 109.508560
    ====> Epoch: 5 Average loss: 109.7302
    Train Epoch: 6 [0/60000 (0%)]	Loss: 110.585701
    Train Epoch: 6 [12800/60000 (21%)]	Loss: 108.586479
    Train Epoch: 6 [25600/60000 (43%)]	Loss: 107.371269
    Train Epoch: 6 [38400/60000 (64%)]	Loss: 105.153595
    Train Epoch: 6 [51200/60000 (85%)]	Loss: 105.446297
    ====> Epoch: 6 Average loss: 108.5776
    Train Epoch: 7 [0/60000 (0%)]	Loss: 109.831375
    Train Epoch: 7 [12800/60000 (21%)]	Loss: 110.046555
    Train Epoch: 7 [25600/60000 (43%)]	Loss: 104.801361
    Train Epoch: 7 [38400/60000 (64%)]	Loss: 103.717628
    Train Epoch: 7 [51200/60000 (85%)]	Loss: 111.997604
    ====> Epoch: 7 Average loss: 107.8022
    Train Epoch: 8 [0/60000 (0%)]	Loss: 107.858582
    Train Epoch: 8 [12800/60000 (21%)]	Loss: 108.593163
    Train Epoch: 8 [25600/60000 (43%)]	Loss: 108.067123
    Train Epoch: 8 [38400/60000 (64%)]	Loss: 110.446304
    Train Epoch: 8 [51200/60000 (85%)]	Loss: 105.854263
    ====> Epoch: 8 Average loss: 107.1581
    Train Epoch: 9 [0/60000 (0%)]	Loss: 104.720139
    Train Epoch: 9 [12800/60000 (21%)]	Loss: 106.817406
    Train Epoch: 9 [25600/60000 (43%)]	Loss: 104.800179
    Train Epoch: 9 [38400/60000 (64%)]	Loss: 105.337631
    Train Epoch: 9 [51200/60000 (85%)]	Loss: 109.797195
    ====> Epoch: 9 Average loss: 106.6367
    Train Epoch: 10 [0/60000 (0%)]	Loss: 103.187050
    Train Epoch: 10 [12800/60000 (21%)]	Loss: 106.347107
    Train Epoch: 10 [25600/60000 (43%)]	Loss: 105.634209
    Train Epoch: 10 [38400/60000 (64%)]	Loss: 108.781830
    Train Epoch: 10 [51200/60000 (85%)]	Loss: 108.425758
    ====> Epoch: 10 Average loss: 106.2243

Run the box below to check if the model you trained above is able to correctly reconstruct images.

```python
### Let's check if the reconstructions make sense
# Set model to test mode
VAE_MNIST.eval()

# Reconstructed
train_data_plot = datasets.MNIST('../data', train=True, download=True,
                   transform=transforms.ToTensor())

train_loader_plot = torch.utils.data.DataLoader(train_data_plot,
                                           batch_size=1, shuffle=False, **{})

for batch_idx, (data, _) in enumerate(train_loader_plot):
    x_hat, mu, logvar = VAE_MNIST(data)
    plt.imshow(x_hat.view(1,28,28).squeeze().data.numpy(), cmap='gray')
    plt.title('%i' % train_data.train_labels[batch_idx])
    plt.show()
    if batch_idx == 3:
        break

```

### 2.8 Visualize latent space (20 points)
Now, implement the auto-encoder now with a 2-dimensional latent space, and train again over the MNIST data. Make a visualization of the learned manifold by using a linearly spaced coordinate grid as input for the latent space, as seen in  https://arxiv.org/abs/1312.6114 Figure 4.

```python
# YOUR CODE HERE
######################## ######################## ########################
fc1_dims = (784,400)
fc21_dims = (400,2)
fc22_dims = (400,2)
fc3_dims = (2,400)
fc4_dims = (400,784)
lr = 1e-3
batch_size = 128
epochs = 10

# Init model
VAE_MNIST = VAE(fc1_dims=fc1_dims, fc21_dims=fc21_dims, fc22_dims=fc22_dims, fc3_dims=fc3_dims, fc4_dims=fc4_dims)

# Init optimizer
optimizer = optim.Adam(VAE_MNIST.parameters(), lr=lr)

# Train
for epoch in range(1, epochs + 1):
    train(epoch, train_loader, VAE_MNIST, optimizer)

######################## ######################## ########################
```

    Train Epoch: 1 [0/60000 (0%)]	Loss: 547.188660
    Train Epoch: 1 [12800/60000 (21%)]	Loss: 190.724518
    Train Epoch: 1 [25600/60000 (43%)]	Loss: 183.475037
    Train Epoch: 1 [38400/60000 (64%)]	Loss: 178.626740
    Train Epoch: 1 [51200/60000 (85%)]	Loss: 181.308685
    ====> Epoch: 1 Average loss: 190.5039
    Train Epoch: 2 [0/60000 (0%)]	Loss: 173.734924
    Train Epoch: 2 [12800/60000 (21%)]	Loss: 171.191605
    Train Epoch: 2 [25600/60000 (43%)]	Loss: 172.238831
    Train Epoch: 2 [38400/60000 (64%)]	Loss: 155.479813
    Train Epoch: 2 [51200/60000 (85%)]	Loss: 168.183075
    ====> Epoch: 2 Average loss: 166.9514
    Train Epoch: 3 [0/60000 (0%)]	Loss: 170.649994
    Train Epoch: 3 [12800/60000 (21%)]	Loss: 162.990494
    Train Epoch: 3 [25600/60000 (43%)]	Loss: 158.472961
    Train Epoch: 3 [38400/60000 (64%)]	Loss: 166.178955
    Train Epoch: 3 [51200/60000 (85%)]	Loss: 159.786560
    ====> Epoch: 3 Average loss: 163.5314
    Train Epoch: 4 [0/60000 (0%)]	Loss: 162.653625
    Train Epoch: 4 [12800/60000 (21%)]	Loss: 167.145065
    Train Epoch: 4 [25600/60000 (43%)]	Loss: 160.325531
    Train Epoch: 4 [38400/60000 (64%)]	Loss: 162.656601
    Train Epoch: 4 [51200/60000 (85%)]	Loss: 171.967133
    ====> Epoch: 4 Average loss: 161.6073
    Train Epoch: 5 [0/60000 (0%)]	Loss: 158.492798
    Train Epoch: 5 [12800/60000 (21%)]	Loss: 161.623306
    Train Epoch: 5 [25600/60000 (43%)]	Loss: 154.071381
    Train Epoch: 5 [38400/60000 (64%)]	Loss: 158.868637
    Train Epoch: 5 [51200/60000 (85%)]	Loss: 147.898712
    ====> Epoch: 5 Average loss: 160.2289
    Train Epoch: 6 [0/60000 (0%)]	Loss: 159.335892
    Train Epoch: 6 [12800/60000 (21%)]	Loss: 161.691116
    Train Epoch: 6 [25600/60000 (43%)]	Loss: 156.913315
    Train Epoch: 6 [38400/60000 (64%)]	Loss: 157.479507
    Train Epoch: 6 [51200/60000 (85%)]	Loss: 162.400757
    ====> Epoch: 6 Average loss: 159.0768
    Train Epoch: 7 [0/60000 (0%)]	Loss: 163.747528
    Train Epoch: 7 [12800/60000 (21%)]	Loss: 154.155258
    Train Epoch: 7 [25600/60000 (43%)]	Loss: 157.765182
    Train Epoch: 7 [38400/60000 (64%)]	Loss: 152.845505
    Train Epoch: 7 [51200/60000 (85%)]	Loss: 157.339355
    ====> Epoch: 7 Average loss: 157.9870
    Train Epoch: 8 [0/60000 (0%)]	Loss: 154.260925
    Train Epoch: 8 [12800/60000 (21%)]	Loss: 158.054810
    Train Epoch: 8 [25600/60000 (43%)]	Loss: 151.418488
    Train Epoch: 8 [38400/60000 (64%)]	Loss: 152.102722
    Train Epoch: 8 [51200/60000 (85%)]	Loss: 157.490784
    ====> Epoch: 8 Average loss: 157.0406
    Train Epoch: 9 [0/60000 (0%)]	Loss: 163.825485
    Train Epoch: 9 [12800/60000 (21%)]	Loss: 155.307663
    Train Epoch: 9 [25600/60000 (43%)]	Loss: 157.243729
    Train Epoch: 9 [38400/60000 (64%)]	Loss: 148.323654
    Train Epoch: 9 [51200/60000 (85%)]	Loss: 157.200607
    ====> Epoch: 9 Average loss: 156.2460
    Train Epoch: 10 [0/60000 (0%)]	Loss: 150.685394
    Train Epoch: 10 [12800/60000 (21%)]	Loss: 162.164215
    Train Epoch: 10 [25600/60000 (43%)]	Loss: 157.070526
    Train Epoch: 10 [38400/60000 (64%)]	Loss: 153.022583
    Train Epoch: 10 [51200/60000 (85%)]	Loss: 148.663284
    ====> Epoch: 10 Average loss: 155.4347

```python
######################## ######################## ########################

VAE_MNIST.eval()

size = 5
f, axarr = plt.subplots(size*2,size*2)

for i in range(-size,size):
    for j in range(-size,size):
        pos = torch.zeros((1,2))
        pos[0,0] = i
        pos[0,1] = j
        x_hat = VAE_MNIST.decode(torch.autograd.Variable(pos))
        axarr[i+size,j+size].imshow(x_hat.view(1,28,28).squeeze().data.numpy(), interpolation = 'nearest',
                       cmap ='Greys')
        axarr[i+size,j+size].axis('off')

plt.show()

######################## ######################## ########################
```

### 2.8 Amortized inference (10 points)
What is amortized inference? Where in the code of Part 2 is it used? What is the benefit of using it?

YOUR ANSWER HERE

In amortized inference, we share the variational parameters $\lambda$ across datapoints. In the Variational Autoencoder in Part 2, this is the case, since we use the same parameters, i.e. the same weights and biases in the encoding network, for all datapoints, i.e. samples from the dataset.

If we see a new datapoint, we can use the learned network in order to approximate its posterior $q(z)$, which can be an advantage over the alternative mean-field approach, where we would have to run variational inference for each new datapoint again.


# Sommerfeld Boundary Conditions

 Validated for all coordinate systems available in NRPy+

## Authors: Terrence Pierre Jacques & Zach Etienne

## Abstract

The aim of this notebook is to describe the mathematical motivation behind the Sommerfeld boundary condition, document how it is implemented within the NRPy+ infrastructure in Cartesian coordinates, and record ongoing validation tests against the [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk).

**Notebook Status:**  Validated against the Einstein Toolkit

**Validation Notes:** The Sommerfeld boundary condition as implemented in NRPy+ has been validated against [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk)(ETK) for the case of a scalar wave propagating across a 3D Cartesian grid. We have agreement to roundoff error with the Einstein Toolkit. Specifically, we have achieved:

1. Roundoff level agreement for the wave propagating toward each of the individual faces

1. Roundoff level agreement using any combination of input parameters available in the [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk) (variable value at infinity, radial power, and wave speed)

[comment]: <> (Introduction: TODO)

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#initializenrpy): Set core NRPy+ parameters for numerical grids
1. [Step 2](#sbc): Definition and mathematical motivation
    1. [Step 2.a](#intro): Introduction & background mathematics
    1. [Step 2.b](#sbc_prelims): Preliminaries - The scalar wave equation in curvilinear coordinates
    1. [Step 2.c](#sbc_ansatz): Sommerfeld boundary condition ansatz
    1. [Step 2.d](#sbc_ansatz_dtf): Applying the ansatz to $\partial_t f$
    1. [Step 2.e](#curvicoords): Implementation in generic curvilinear coordinates
        1. [Step 2.e.i](#cartcoords_byhand): Sommerfeld boundary conditions implementation in Cartesian coordinates, derived by hand
        1. [Step 2.e.ii](#cartcoords_bynrpysympy): Sommerfeld boundary conditions implementation in Cartesian coordinates, derived by NRPy+/SymPy
1. [Step 3](#numalg): Numerical algorithm overview
    1. [Step 3.a](#class): Sommerfeld python class and parameters
    1. [Step 3.b](#partial_rf): Calculate $\partial_r f$
    1. [Step 3.c](#cfunc): `apply_bcs_sommerfeld()` C function
    1. [Step 3.d](#k): Solving for the subdominant radial falloff proportionality constant $k$
    1. [Step 3.e](#innerbcs): Inner Boundary Conditions
1. [Step 4](#py_validate): Python file validation
1. [Step 5](#interface): NRPy+ Interface for Applying Sommerfeld Boundary Conditions
1. [Step 6](#etk_validation): Validation against the [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk)
1. [Latex Output](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Set core NRPy+ parameters for numerical grids \[Back to [top](#toc)\]
$$\label{initializenrpy}$$

Import needed NRPy+ core modules and set working directories.

```python
# Step P1: Import needed NRPy+ core modules:
import NRPy_param_funcs as par   # NRPy+: Parameter interface
import reference_metric as rfm  # NRPy+: Reference metric support
import grid as gri               # NRPy+: Functions having to do with numerical grids
import cmdline_helper as cmd     # NRPy+: Multi-platform Python command-line interface
import shutil, os, sys           # Standard Python modules for multiplatform OS-level functions

# Create C code output directory:
Ccodesdir = os.path.join("SommerfeldBoundaryCondition_Validate")
# First remove C code output directory if it exists
# Courtesy https://stackoverflow.com/questions/303200/how-do-i-remove-delete-a-folder-that-is-not-empty
shutil.rmtree(Ccodesdir, ignore_errors=True)
# Then create a fresh directory
cmd.mkdir(Ccodesdir)

par.set_parval_from_str("grid::DIM",3)
DIM = par.parval_from_str("grid::DIM")
```

# Step 2: Sommerfeld Boundary Conditions \[Back to [top](#toc)\]
$$\label{sbc}$$

## Step 2.a: Introduction & background mathematics \[Back to [top](#toc)\]
$$\label{intro}$$

When we numerically solve an initial value problem, appropriate initial data must be provided, coupled to a technique for evolving the initial data forward in time (to construct the solution at $t>0$), and we must impose boundary conditions.

The subject of this notebook is implementation of Sommerfeld boundary conditions, which are often used when solving hyperbolic systems of partial differential equations (PDEs). Sommerfeld boundary conditions are also referred to as a radiation or transparent boundary conditions.

The essential idea of a transparent boundary is creating a boundary in which wave fronts can pass through with minimal reflections. In other words, the boundary condition acts to map our numerical solution to outside our numerical domain in a smooth fashion. Because this mapping is assumed to be linear, this treatment occurs at the same time-level as calculations within our numerical domain. This point will be revisited later.

Suppose we have a dynamical variable $f$; i.e., a variable in our hyperbolic system of PDEs that satisfies the equation

$$
\partial_t f = \text{something},
$$
where generally we refer to "something" as the "right-hand side" or "RHS" of the hyperbolic PDE, where we formulate the PDE such that the RHS contains no explicit time derivatives, but generally does contain spatial derivatives (typically computed in NRPy+ using finite differencing, though may in general be computed using other, e.g., pseudospectral techniques).

To construct the solution at times after the initial data, we adopt the [Method of Lines (MoL)](Tutorial-ScalarWave.ipynb) approach, which integrates the equations forward *in time* using standard explicit techniques typically used when solving *ordinary* differential equations. In doing so, MoL evaluates the RHS of the PDE at all points in our numerical domain (typically using finite difference derivatives), except the ghost zones (i.e., the gridpoints neighboring the precise point needed to evaluate a derivative using e.g., a finite difference derivative).

After each RHS evaluation we must fill in the data on the boundaries (i.e., the ghost zones), so that data exist at all gridpoints (including the boundaries) at the next MoL substep. In doing so, we have two options:

1. Perform an MoL substep to push the *interior* solution $f$ forward in time one substep, and then update the boundary values of $f$.
1. During the MoL substep, immediately after evaluating the RHS of the $\partial t f$ equation, update the boundary values of the RHS of the $\partial t f$ equation. Then push the solution $f$ forward in time *at all gridpoints, including ghost zones* by one substep.

Our implementation of the Sommerfeld boundary condition implements the second option, filling in the data for $\partial_t f$ (cf., our [extrapolation boundary condition implementation](Tutorial-Start_to_Finish-Curvilinear_BCs.ipynb)) on the boundaries (i.e., ghost zones).

## Step 2.b: Preliminaries - The scalar wave equation in curvilinear coordinates \[Back to [top](#toc)\]
$$\label{sbc_prelims}$$

Our Sommerfeld boundary condition implementation assumes $f$ behaves as a spherically symmetric, outgoing wave at the boundaries. Under these assumptions, the waves satisfy the wave equation in spherical coordinates with angular parts set to zero. I.e., under these assumptions the wavefunction $u(r,t)$ will satisfy:

\begin{align}
0 = \Box\, u &= \hat{D}^{\nu} \hat{D}_{\nu} u \\
&= \hat{g}^{\mu\nu} \hat{D}_{\mu} \hat{D}_{\nu} u \\
&= \hat{g}^{\mu\nu} \hat{D}_{\mu} \partial_{\nu} u \\
&= \hat{g}^{\mu\nu} \left[\partial_{\mu} (\partial_{\nu} u) - \hat{\Gamma}^\alpha_{\mu\nu} (\partial_{\alpha} u) \right],
\end{align}
here the hatted metric is defined as $\hat{g}^{tt} = -1/v^2$ (where $v$ is the wavespeed), $\hat{g}^{rr}=1$, $\hat{g}^{\theta\theta}=1/r^2$, and $\hat{g}^{\phi\phi}=1/(r^2\sin^2\theta)$ are the only nonzero metric terms for the spherical (contravariant) metric. However, the fact that $u=u(r,t)$ does not depend on the angular pieces greatly simplifies the expression:

\begin{align}
\Box\, u
&= \hat{g}^{\mu\nu} \left[\partial_{\mu} (\partial_{\nu} u) - \hat{\Gamma}^\alpha_{\mu\nu} (\partial_{\alpha} u) \right] \\
&= \left(-\frac{1}{v^2}\partial_t^2 + \partial_r^2\right)u - \hat{\Gamma}^\alpha_{\mu\nu} (\partial_{\alpha} u) \\
&= \left(-\frac{1}{v^2}\partial_t^2 + \partial_r^2\right)u - \hat{g}^{\mu\nu} \left[\hat{\Gamma}^t_{\mu\nu} \partial_{t} + \hat{\Gamma}^r_{\mu\nu} \partial_{r}\right]u.
\end{align}

We will now refer the reader to the [scalar wave in curvilinear coordinates notebook](Tutorial-ScalarWaveCurvilinear.ipynb) for the remainder of the derivation. The bottom line is, after removing terms implying angular variation in $u$, one obtains the wave equation for spherically symmetric waves:

$$
\frac{1}{v^2} \partial_t^2 u = \partial_r^2 u + \frac{2}{r} \partial_r u,
$$
which has the general solution

$$
u(r,t) = A \frac{u(r + vt)}{r} + B \frac{u(r - vt)}{r},
$$
where the (left) right term represents an (ingoing) outgoing wave.

## Step 2.c: Sommerfeld boundary condition ansatz \[Back to [top](#toc)\]
$$\label{sbc_ansatz}$$

Inspired by the solution to the scalar wave equation, our Sommerfeld boundary condition will assume the solution $f(r,t)$ acts as an *outgoing* spherical wave ($A=0$), with an asymptotic value $f_0$ at $r\to\infty$ and a correction term for incoming waves or transient non-wavelike behavior at the boundaries, with $r^n$ falloff (ignoring higher-order radial falloffs):

$$
f = f_0 + \frac{u(r-vt)}{r} + \frac{c}{r^n},
$$
where $c$ is a constant.

## Step 2.d: Applying the ansatz to $\partial_t f$ \[Back to [top](#toc)\]
$$\label{sbc_ansatz_dtf}$$

As described in the above section, we will not apply Sommerfeld boundary conditions to $f$ but $\partial_t f$ instead:

$$
\partial_t f = -v \frac{u'(r-vt)}{r}.
$$

To get a better understanding of the $u'(r-vt)$ term, let's compute the radial partial derivative as well:

\begin{align}
\partial_r f &= \frac{u'(r-vt)}{r} - \frac{u(r-vt)}{r^2} - n \frac{c}{r^{n+1}} \\
\implies \frac{u'(r-vt)}{r} &= \partial_r f + \frac{u(r-vt)}{r^2} + n \frac{c}{r^{n+1}}
\end{align}

Thus we get

\begin{align}
\partial_t f &= -v \frac{u'(r-vt)}{r} \\
&= -v \left[\partial_r f + \frac{u(r-vt)}{r^2} + n \frac{c}{r^{n+1}} \right]
\end{align}

To take care of the (as-yet) unknown $\frac{u(r-vt)}{r^2}$ term, notice our ansatz

$$
f = f_0 + \frac{u(r-vt)}{r} + \frac{c}{r^n}
$$
implies that

\begin{align}
\frac{f - f_0}{r} &= \frac{u(r-vt)}{r^2} + \frac{c}{r^{n+1}} \\
\implies \frac{u(r-vt)}{r^2} &= \frac{f - f_0}{r} - \frac{c}{r^{n+1}}
\end{align}

so we have

\begin{align}
\partial_t f &= -v \left[\partial_r f + \frac{u(r-vt)}{r^2} + n \frac{c}{r^{n+1}} \right]\\
&= -v \left[\partial_r f + \frac{f - f_0}{r} - \frac{c}{r^{n+1}} + n \frac{c}{r^{n+1}} \right] \\
&= -v \left[\partial_r f + \frac{f - f_0}{r}\right] + \frac{k}{r^{n+1}},
\end{align}
where $k=-v c(n-1)$ is just another constant.

Thus we have derived our boundary condition:

$$
\boxed{
\partial_t f = -\frac{v}{r} \left[r \partial_r f + (f - f_0)\right] + \frac{k}{r^{n+1}}.
}
$$

## Step 2.e: Implementation in generic curvilinear coordinates \[Back to [top](#toc)\]
$$\label{curvicoords}$$

The core equation
$$
\boxed{
\partial_t f = -\frac{v}{r} \left[r \partial_r f + (f - f_0)\right] + \frac{k}{r^{n+1}}.
}
$$

is implemented in NRPy+ using its `reference_metric.py` module ([Tutorial notebook](Tutorial-Reference_Metric.ipynb)), as this module requires *all* coordinate systems to define the spherical coordinate $r$ in terms of input quantities `(xx0,xx1,xx2)`. Thus we need only rewrite the above equation in terms of `(xx0,xx1,xx2)`. Defining $x^i$=`(xx0,xx1,xx2)`, we have, using the chain rule:

\begin{align}
\partial_t f &= -\frac{v}{r} \left[r \partial_r f + (f - f_0)\right] + \frac{k}{r^{n+1}} \\
&= -\frac{v}{r(x^i)} \left[r \frac{\partial x^i}{\partial r} \partial_i f + (f - f_0)\right] + \frac{k}{r^{n+1}}.
\end{align}

$\frac{\partial x^i}{\partial r}$ can be impossible to compute directly, as we are given $r(x^i)$ but not necessarily $x^i(r)$. The key here is to note that we are actually given $x^j_{\rm Sph} = (r(x^i),\theta(x^i),\phi(x^i))$ for all coordinate systems, so we can define the Jacobian

$$\frac{\partial x^j_{\rm Sph}(x^i)}{\partial x^i},$$

and NRPy+ can invert this matrix to give us

$$\frac{\partial x^i}{\partial x^j_{\rm Sph}},$$

In summary, the implementation of Sommerfeld boundary conditions in arbitrary curvilinear coordinates $x^i=$`(xx0,xx1,xx2)` is given by
$$
\boxed{
\partial_t f = -\frac{v}{r(x^i)} \left[r(x^i) \frac{\partial x^i}{\partial r} \partial_i f + (f - f_0)\right] + \frac{k}{r^{n+1}}.
}
$$

In the next subsections, we'll work through implementation of this general equation in the special case of Cartesian coordinates, first by hand, and then *automatically*, for Cartesian *or other curvilinear coordinate systems supported by NRPy+* using `reference_metric.py`.

### Step 2.e.i: Sommerfeld boundary conditions implementation in Cartesian coordinates, derived by hand \[Back to [top](#toc)\]
$$\label{cartcoords_byhand}$$

Let's now work this out for Cartesian coordinates in NRPy+, first by hand, and then using `reference_metric.py`:

In Cartesian coordinates $\frac{\partial f}{\partial r}$ may be expanded as

$$
\frac{\partial f}{\partial r} = \frac{\partial x}{\partial r} \partial_x f + \frac{\partial y}{\partial r}\partial_y f + \frac{\partial z}{\partial r}\partial_z f.
$$

Defining $x^i$ to the the $i$th component of the Cartesian reference metric, we have

\begin{align}
x^0 = x &= r\sin\theta \cos\phi \implies \frac{\partial x}{\partial r}=\frac{x}{r}, \\
x^1 = y &= r\sin\theta \sin\phi \implies \frac{\partial y}{\partial r}=\frac{y}{r}, \\
x^2 = z &= r\cos\theta \implies \frac{\partial z}{\partial r}=\frac{z}{r}.
\end{align}

Based on this, we can rewrite the above as

\begin{align}
\frac{\partial f}{\partial r}
&= \frac{x}{r} \partial_x f +\frac{y}{r} \partial_y f +\frac{z}{r} \partial_z f \\
&= \frac{x^0}{r} \partial_0 f +\frac{x^1}{r} \partial_1 f +\frac{x^2}{r} \partial_2 f \\
&= \frac{x^i}{r} \partial_i f,
\end{align}

yielding the Sommerfeld boundary condition in *Cartesian coordinates*

$$
\partial_t f = -\frac{v}{r} \left[x^i \partial_i f + \left( f - f_0 \right) \right] + \frac{k}{r^{n+1}}.
$$

### Step 2.e.ii: Sommerfeld boundary conditions implementation in Cartesian coordinates, derived automatically by NRPy+/SymPy \[Back to [top](#toc)\]
$$\label{cartcoords_bynrpysympy}$$

Now let's use NRPy+'s `reference_metric.py` to obtain the same expression for $\frac{\partial f}{\partial r}$ in Cartesian coordinates; i.e.,

$$
\frac{\partial f}{\partial r} = \frac{x^i}{r} \partial_i f,
$$
but using the generic coordinate system interface (`CoordSystem=Cartesian`; feel free to change it to another coordinate system of your choice). Note in the above that me we must also know the functional form of $r(x^i)$, so we use NRPy's `outputC` module to output the C code to calculate $r(x^i)$ and $\frac{\partial x^i}{\partial r} \partial_i f$, as shown below.

# Step 3: Numerical algorithm overview \[Back to [top](#toc)\]
$$\label{numalg}$$

To implement the above Sommerfeld boundary condition, we must specify for each dynamical variable:

* its wave speed at the boundary,
* its asymptotic value at infinity, and
* its $r^{n+1}$ power that must be applied for the non-wavelike behavior at the boundary.

Note in our ansatz

$$
f = f_0 + \frac{u(r-vt)}{r} + \frac{c}{r^n},
$$

in this expansion it would be natural to have $n = 2$, but in the boundary condition we have

$$
\partial_t f = -\frac{v}{r(x^i)} \left[r(x^i) \frac{\partial x^i}{\partial r} \partial_i f + (f - f_0)\right] + \frac{k}{r^{n+1}}.
$$

Thus, the exponent in the $k$ term should be 3. And indeed, in our own implementations we have found that $n=2$ exhibits the best results for minimal reflections. The [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk) documentation page also states similar results, writing "empirically, we have found that taking (exponent $ = 3$) almost completely eliminates the bad transient caused by the radiative boundary condition on its own". The following set of code cells implement the above equation, excluding the $\frac{k}{r^{n+1}}$ term, for a Cartesian grid.

Our procedure in implementing this boundary condition is as follows:

0) Define data for $f$ at **all** points in our numerical domain

1) Evaluate $\frac{df}{dt}$ at points in the **interior** only, using the       prescribed equations to evaluate the RHS and ghost zones for the finite         differences at the outer boundaries

2) For points in the ghost zones, apply the Sommerfeld condition to obtain      $\frac{df}{dt}$ in the ghost zones, assuming advection and radial fall off behavior. When evaluating the spatial derivatives, use forward or backward finite differences for directions perpendicular to the outer boundary face being considered, and centered derivatives for the directions which lie in the plane of the outer boundary face. Furthermore, to minimize error we loop through inner boundary points and work outwards, so that the forward or backward stencils never include untreated points.

3) Account for non-wavelike evolution of our numerical solution at the boundary

4) Perform RK-update for all points (interior + ghost zones). Since the mapping to outside our numerical domain is linear, treatment of points in the interior and points on the boundary must be at the same time level.

## Step 3.a: Sommerfeld python class and parameters \[Back to [top](#toc)\]
$$\label{class}$$

First we define the python class __sommerfeld_boundary_condition_class__, where we store all variables and functions related to the our impementation of the Sommerfeld boundary condition. When calling the class, users can set default values for each dynamical variable's value at infinity, radial fall-off power, and wave speed at the boundary.

Next we define the function _sommerfeld_params_ which writes these parameters to lists in C, which will be used by the main C code.

```python
%%writefile $Ccodesdir/Sommerfeld.py
class sommerfeld_boundary_condition_class():
    """
    Class for generating C code to apply Sommerfeld boundary conditions
    """
    # class variables should be the resulting dicts
    # Set class variable default values
    # radial falloff power n = 3 has been found to yield the best results
    #  - see Tutorial-SommerfeldBoundaryCondition.ipynb Step 2 for details
    def __init__(self, fd_order=2, vars_at_inf_default = 0., vars_radial_falloff_power_default = 3., vars_speed_default = 1.):
        evolved_variables_list, _, _ = gri.gridfunction_lists()

        # set class finite differencing order
        self.fd_order = fd_order

        NRPy_FD_order = par.parval_from_str("finite_difference::FD_CENTDERIVS_ORDER")

        if NRPy_FD_order < fd_order:
            print("ERROR: The global central finite differencing order within NRPy+ must be greater than or equal to the Sommerfeld boundary condition's finite differencing order")
            sys.exit(1)

        # Define class dictionaries to store sommerfeld parameters for each EVOL gridfunction

        # EVOL gridfunction asymptotic value at infinity
        self.vars_at_infinity = {}

        # EVOL gridfunction wave speed at outer boundaries
        self.vars_speed = {}

        # EVOL gridfunction radial falloff power
        self.vars_radial_falloff_power = {}

        # Set default values for each specific EVOL gridfunction
        for gf in evolved_variables_list:
            self.vars_at_infinity[gf.upper() + 'GF'] = vars_at_inf_default
            self.vars_radial_falloff_power[gf.upper() + 'GF'] = vars_radial_falloff_power_default
            self.vars_speed[gf.upper() + 'GF'] = vars_speed_default

    def sommerfeld_params(self):
        # Write parameters to C file

        # Creating array for EVOL gridfunction values at infinity
        var_at_inf_string = "{"
        for _gf,val in self.vars_at_infinity.items():
            var_at_inf_string += str(val) + ", "
        var_at_inf_string = var_at_inf_string[:-2] + "};"

        # Creating array for EVOL gridfunction values of radial falloff power
        vars_radial_falloff_power_string = "{"
        for _gf,val in self.vars_radial_falloff_power.items():
            vars_radial_falloff_power_string += str(val) + ", "
        vars_radial_falloff_power_string = vars_radial_falloff_power_string[:-2] + "};"

        # Creating array for EVOL gridfunction values of wave speed at outer boundaries
        var_speed_string = "{"
        for _gf,val in self.vars_speed.items():
            var_speed_string += str(val) + ", "
        var_speed_string = var_speed_string[:-2] + "};"

        # Writing to values to sommerfeld_params.h file
        out_str = """
// Sommerfeld EVOL grid function parameters
const REAL evolgf_at_inf[NUM_EVOL_GFS] = """+var_at_inf_string+"""
const REAL evolgf_radial_falloff_power[NUM_EVOL_GFS] = """+vars_radial_falloff_power_string+"""
const REAL evolgf_speed[NUM_EVOL_GFS] = """+var_speed_string+"""
"""
        return out_str
```

    Writing SommerfeldBoundaryCondition_Validate/Sommerfeld.py

## Step 3.b: Calculate $\partial_r f$ \[Back to [top](#toc)\]
$$\label{partial_rf}$$

Next we generate the C code for calculating $\partial_r f$ for each dynamical variable $f$ in our coordinate system of choice.

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

    @staticmethod
    def dfdr_function(fd_order):
        # function to write c code to calculate dfdr term in Sommerfeld boundary condition

        # Read what # of dimensions being usded
        DIM = par.parval_from_str("grid::DIM")

        # Set up the chosen reference metric from chosen coordinate system, set within NRPy+
        CoordSystem = par.parval_from_str("reference_metric::CoordSystem")
        rfm.reference_metric()

        # Simplifying the results make them easier to interpret.
        do_simplify = True
        if "Sinh" in CoordSystem:
            # Simplification takes too long on Sinh* coordinate systems
            do_simplify = False

        # Construct Jacobian matrix, output Jac_dUSph_dDrfmUD[i][j] = \partial x_{Sph}^i / \partial x^j:
        Jac_dUSph_dDrfmUD = ixp.zerorank2()
        for i in range(3):
            for j in range(3):
                Jac_dUSph_dDrfmUD[i][j] = sp.diff(rfm.xxSph[i],rfm.xx[j])

        # Invert Jacobian matrix, output to Jac_dUrfm_dDSphUD.
        Jac_dUrfm_dDSphUD, dummyDET = ixp.generic_matrix_inverter3x3(Jac_dUSph_dDrfmUD)

        # Jac_dUrfm_dDSphUD[i][0] stores \partial x^i / \partial r
        if do_simplify:
            for i in range(3):
                Jac_dUrfm_dDSphUD[i][0] = sp.simplify(Jac_dUrfm_dDSphUD[i][0])

        # Declare \partial_i f, which is actually computed later on
        fdD = ixp.declarerank1("fdD") # = [fdD0, fdD1, fdD2]
        contraction = sp.sympify(0)
        for i in range(3):
            contraction += fdD[i]*Jac_dUrfm_dDSphUD[i][0]

        if do_simplify:
            contraction = sp.simplify(contraction)

        r_str_and_contraction_str = outputC([rfm.xxSph[0],contraction],
                 ["*_r","*_partial_i_f"],filename="returnstring",params="includebraces=False")
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

Here we generate the C code used to calculate all relevant spatial derivatives $\partial_i f$, using second order accurate finite differences. Specifically, if our ghost zone point lies on one of the faces, on an edge or corner, we use forward or backward differences depending on the specific direction, and centered differences otherwise. Note that all derivatives along the normal of the boundary faces are forward or backward, to minimize using non-updated points in the derivative calculations.

For example, consider some point with Cartesian coordinates $(i,j,k)$ on our grid, the derivative of $f$ along the $x$ direction will be the forward (backward with change of signs on coefficients)

$$
\frac{\partial f_{ijk}}{\partial x} \approx \frac{1}{2\Delta x} \left( -3f_{i,j,k} + 4f_{i+1,j,k} - f_{i+2,j,k} \right),
$$

or the ceneterd difference approximation

$$
\frac{\partial f_{ijk}}{\partial x} \approx \frac{1}{2\Delta x} \left( f_{i+1,j,k} - f_{i-1,j,k} \right).
$$

We determine the signs of the coefficents (corresponding to using either a foward or backward difference) by determining what face the point lies within. The above is applied for all three Cartesian directions. Note the use if the `SHIFTSTENCIL` variable, which helps determine when to use forward/backward difference to take derivatives along normals to boundary faces, or when to use central differences to either take derivatives parallel to the faces or at points on edges and corners.

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

        def gen_central_2oFD_stencil_str(intdirn):
            if intdirn == 0:
                return "(gfs[IDX4S(which_gf,i0+1,i1,i2)]-gfs[IDX4S(which_gf,i0-1,i1,i2)])*0.5"  # Does not include the 1/dx multiplication
            if intdirn == 1:
                return "(gfs[IDX4S(which_gf,i0,i1+1,i2)]-gfs[IDX4S(which_gf,i0,i1-1,i2)])*0.5"  # Does not include the 1/dy multiplication
            return "(gfs[IDX4S(which_gf,i0,i1,i2+1)]-gfs[IDX4S(which_gf,i0,i1,i2-1)])*0.5"  # Does not include the 1/dz multiplication

        def gen_central_4oFD_stencil_str(intdirn):
            if intdirn == 0:
                return """(-c2*gfs[IDX4S(which_gf,i0+2,i1,i2)]
                         +c1*gfs[IDX4S(which_gf,i0+1,i1,i2)]
                         -c1*gfs[IDX4S(which_gf,i0-1,i1,i2)]
                         +c2*gfs[IDX4S(which_gf,i0-2,i1,i2)])"""  # Does not include the 1/dx multiplication
            if intdirn == 1:
                return """(-c2*gfs[IDX4S(which_gf,i0,i1+2,i2)]
                         +c1*gfs[IDX4S(which_gf,i0,i1+1,i2)]
                         -c1*gfs[IDX4S(which_gf,i0,i1-1,i2)]
                         +c2*gfs[IDX4S(which_gf,i0,i1-2,i2)])"""  # Does not include the 1/dy multiplication
            return """(-c2*gfs[IDX4S(which_gf,i0,i1,i2+2)]
                         +c1*gfs[IDX4S(which_gf,i0,i1,i2+1)]
                         -c1*gfs[IDX4S(which_gf,i0,i1,i2-1)]
                         +c2*gfs[IDX4S(which_gf,i0,i1,i2-2)])"""  # Does not include the 1/dz multiplication

        def gen_central_6oFD_stencil_str(intdirn):
            if intdirn == 0:
                return """( c3*gfs[IDX4S(which_gf,i0+3,i1,i2)]
                         -c2*gfs[IDX4S(which_gf,i0+2,i1,i2)]
                         +c1*gfs[IDX4S(which_gf,i0+1,i1,i2)]
                         -c1*gfs[IDX4S(which_gf,i0-1,i1,i2)]
                         +c2*gfs[IDX4S(which_gf,i0-2,i1,i2)]
                         -c3*gfs[IDX4S(which_gf,i0-3,i1,i2)])"""  # Does not include the 1/dx multiplication

            if intdirn == 1:
                return """( c3*gfs[IDX4S(which_gf,i0,i1+3,i2)]
                         -c2*gfs[IDX4S(which_gf,i0,i1+2,i2)]
                         +c1*gfs[IDX4S(which_gf,i0,i1+1,i2)]
                         -c1*gfs[IDX4S(which_gf,i0,i1-1,i2)]
                         +c2*gfs[IDX4S(which_gf,i0,i1-2,i2)]
                         -c3*gfs[IDX4S(which_gf,i0,i1-3,i2)])"""  # Does not include the 1/dy multiplication

            return """( c3*gfs[IDX4S(which_gf,i0,i1,i2+3)]
                         -c2*gfs[IDX4S(which_gf,i0,i1,i2+2)]
                         +c1*gfs[IDX4S(which_gf,i0,i1,i2+1)]
                         -c1*gfs[IDX4S(which_gf,i0,i1,i2-1)]
                         +c2*gfs[IDX4S(which_gf,i0,i1,i2-2)]
                         -c3*gfs[IDX4S(which_gf,i0,i1,i2-3)])"""  # Does not include the 1/dz multiplication

        def gen_central_fd_stencil_str(intdirn, fd_order):
            if fd_order==2:
                return gen_central_2oFD_stencil_str(intdirn)
            if fd_order==4:
                return gen_central_4oFD_stencil_str(intdirn)
            return gen_central_6oFD_stencil_str(intdirn)

        def output_dfdx(intdirn, fd_order):
            dirn = str(intdirn)
            dirnp1 = str((intdirn+1)%3)  # if dirn='0', then we want this to be '1'; '1' then '2'; and '2' then '0'
            dirnp2 = str((intdirn+2)%3)  # if dirn='0', then we want this to be '2'; '1' then '0'; and '2' then '1'

            preface = """
// On a +x"""+dirn+""" or -x"""+dirn+""" face, do up/down winding as appropriate:
if(abs(FACEXi["""+dirn+"""])==1 || i"""+dirn+"""+NGHOSTS >= Nxx_plus_2NGHOSTS"""+dirn+""" || i"""+dirn+"""-NGHOSTS <= 0) {
    int8_t SHIFTSTENCIL"""+dirn+""" = FACEXi["""+dirn+"""];
    if(i"""+dirn+"""+NGHOSTS >= Nxx_plus_2NGHOSTS"""+dirn+""") SHIFTSTENCIL"""+dirn+""" = -1;
    if(i"""+dirn+"""-NGHOSTS <= 0)                  SHIFTSTENCIL"""+dirn+""" = +1;
    SHIFTSTENCIL"""+dirnp1+""" = 0;
    SHIFTSTENCIL"""+dirnp2+""" = 0;
"""
            if fd_order == 2:
                return preface + """

    fdD"""+dirn+"""
        = SHIFTSTENCIL"""+dirn+"""*(-1.5*gfs[IDX4S(which_gf,i0+0*SHIFTSTENCIL0,i1+0*SHIFTSTENCIL1,i2+0*SHIFTSTENCIL2)]
                         +2.*gfs[IDX4S(which_gf,i0+1*SHIFTSTENCIL0,i1+1*SHIFTSTENCIL1,i2+1*SHIFTSTENCIL2)]
                         -0.5*gfs[IDX4S(which_gf,i0+2*SHIFTSTENCIL0,i1+2*SHIFTSTENCIL1,i2+2*SHIFTSTENCIL2)]
                        )*invdx"""+dirn+""";

// Not on a +x"""+dirn+""" or -x"""+dirn+""" face, using centered difference:
} else {
    fdD"""+dirn+""" = """+gen_central_fd_stencil_str(intdirn, 2)+"""*invdx"""+dirn+""";
}
"""
            if fd_order == 4:
                return preface + """

    fdD"""+dirn+"""
        = SHIFTSTENCIL"""+dirn+"""*(u0*gfs[IDX4S(which_gf,i0+0*SHIFTSTENCIL0,i1+0*SHIFTSTENCIL1,i2+0*SHIFTSTENCIL2)]
                         +u1*gfs[IDX4S(which_gf,i0+1*SHIFTSTENCIL0,i1+1*SHIFTSTENCIL1,i2+1*SHIFTSTENCIL2)]
                         +u2*gfs[IDX4S(which_gf,i0+2*SHIFTSTENCIL0,i1+2*SHIFTSTENCIL1,i2+2*SHIFTSTENCIL2)]
                         +u3*gfs[IDX4S(which_gf,i0+3*SHIFTSTENCIL0,i1+3*SHIFTSTENCIL1,i2+3*SHIFTSTENCIL2)]
                         +u4*gfs[IDX4S(which_gf,i0+4*SHIFTSTENCIL0,i1+4*SHIFTSTENCIL1,i2+4*SHIFTSTENCIL2)]
                        )*invdx"""+dirn+""";

// Not on a +x"""+dirn+""" or -x"""+dirn+""" face, using centered difference:
} else {
    fdD"""+dirn+""" = """+gen_central_fd_stencil_str(intdirn, 4)+"""*invdx"""+dirn+""";
}
"""
            if fd_order == 6:
                return preface + """

    fdD"""+dirn+"""
        = SHIFTSTENCIL"""+dirn+"""*(u0*gfs[IDX4S(which_gf,i0+0*SHIFTSTENCIL0,i1+0*SHIFTSTENCIL1,i2+0*SHIFTSTENCIL2)]
                         +u1*gfs[IDX4S(which_gf,i0+1*SHIFTSTENCIL0,i1+1*SHIFTSTENCIL1,i2+1*SHIFTSTENCIL2)]
                         +u2*gfs[IDX4S(which_gf,i0+2*SHIFTSTENCIL0,i1+2*SHIFTSTENCIL1,i2+2*SHIFTSTENCIL2)]
                         +u3*gfs[IDX4S(which_gf,i0+3*SHIFTSTENCIL0,i1+3*SHIFTSTENCIL1,i2+3*SHIFTSTENCIL2)]
                         +u4*gfs[IDX4S(which_gf,i0+4*SHIFTSTENCIL0,i1+4*SHIFTSTENCIL1,i2+4*SHIFTSTENCIL2)]
                         +u5*gfs[IDX4S(which_gf,i0+5*SHIFTSTENCIL0,i1+5*SHIFTSTENCIL1,i2+5*SHIFTSTENCIL2)]
                         +u6*gfs[IDX4S(which_gf,i0+6*SHIFTSTENCIL0,i1+6*SHIFTSTENCIL1,i2+6*SHIFTSTENCIL2)]
                        )*invdx"""+dirn+""";

// Not on a +x"""+dirn+""" or -x"""+dirn+""" face, using centered difference:
} else {
    fdD"""+dirn+""" = """+gen_central_fd_stencil_str(intdirn, 6)+"""*invdx"""+dirn+""";
}
"""

            print("Error: fd_order = "+str(fd_order)+" currently unsupported.")
            sys.exit(1)

        contraction_term_func = """

// Function to calculate the radial derivative of a grid function
void contraction_term(const paramstruct *restrict params, const int which_gf, const REAL *restrict gfs, REAL *restrict xx[3],
           const int8_t FACEXi[3], const int i0, const int i1, const int i2, REAL *restrict _r, REAL *restrict _partial_i_f) {

#include "RELATIVE_PATH__set_Cparameters.h" /* Header file containing correct #include for set_Cparameters.h;
                                             * accounting for the relative path */

// Initialize derivatives to crazy values, to ensure that
//   we will notice in case they aren't set properly.
REAL fdD0=1e100;
REAL fdD1=1e100;
REAL fdD2=1e100;

REAL xx0 = xx[0][i0];
REAL xx1 = xx[1][i1];
REAL xx2 = xx[2][i2];

int8_t SHIFTSTENCIL0;
int8_t SHIFTSTENCIL1;
int8_t SHIFTSTENCIL2;

"""
        if fd_order == 4:
            contraction_term_func +="""
// foward/backward finite difference coefficients
const REAL u0 =-25./12.;
const REAL u1 = 4.;
const REAL u2 = -3.;
const REAL u3 = 4./3.;
const REAL u4 = -1./4.;

// central finite difference coefficients
const REAL c1 = 2./3.;
const REAL c2 = 1./12.;

"""
        if fd_order == 6:
            contraction_term_func +="""
// foward/backward finite difference coefficients
const REAL u0 = -49./20.;
const REAL u1 =  6.;
const REAL u2 = -15./2.;
const REAL u3 =  20./3.;
const REAL u4 = -15./4.;
const REAL u5 =  6./5.;
const REAL u6 = -1./6.;

// central finite difference coefficients
const REAL c1 = 3./4.;
const REAL c2 = 3./20.;
const REAL c3 = 1./60;

"""
        for i in range(DIM):
            if "fdD"+str(i) in r_str_and_contraction_str:
                contraction_term_func += output_dfdx(i, fd_order)

        contraction_term_func += "\n" + r_str_and_contraction_str

        contraction_term_func +="""
} // END contraction_term function
"""
        return contraction_term_func
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

## Step 3.c: `apply_bcs_sommerfeld()` C function \[Back to [top](#toc)\]
$$\label{cfunc}$$

Here, we build up the main C code and define the function `apply_bcs_sommerfeld()` to be used by NRPy's MoL time stepping algorithm.

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

    def write_sommerfeld_main_Ccode(self, Ccodesdir):
        main_Ccode = """
// Boundary condtion driver routine: Apply BCs to all
// boundary faces of the 3D numerical domain, filling in the
// outer boundary ghost zone layers, starting with the innermost
// layer and working outward.
"""
        main_Ccode += self.sommerfeld_params()
        main_Ccode += self.dfdr_function(self.fd_order)

        main_Ccode += """
void apply_bcs_sommerfeld(const paramstruct *restrict params, REAL *restrict xx[3],
                          const bc_struct *restrict bcstruct, const int NUM_GFS,
                          const int8_t *restrict gfs_parity, REAL *restrict gfs,
                          REAL *restrict rhs_gfs) {

    #pragma omp parallel for
        for(int which_gf=0;which_gf<NUM_GFS;which_gf++) {
          const REAL char_speed             = evolgf_speed[which_gf];
          const REAL var_at_infinity        = evolgf_at_inf[which_gf];
          const REAL radial_falloff_power = evolgf_radial_falloff_power[which_gf];

          #include "RELATIVE_PATH__set_Cparameters.h" /* Header file containing correct #include for set_Cparameters.h;
                                                       * accounting for the relative path */

            for(int which_gz = 0; which_gz < NGHOSTS; which_gz++) {
                for(int pt=0;pt<bcstruct->num_ob_gz_pts[which_gz];pt++) {
                    const int i0 = bcstruct->outer[which_gz][pt].outer_bc_dest_pt.i0;
                    const int i1 = bcstruct->outer[which_gz][pt].outer_bc_dest_pt.i1;
                    const int i2 = bcstruct->outer[which_gz][pt].outer_bc_dest_pt.i2;
                    const int8_t FACEX0 = bcstruct->outer[which_gz][pt].FACEi0;
                    const int8_t FACEX1 = bcstruct->outer[which_gz][pt].FACEi1;
                    const int8_t FACEX2 = bcstruct->outer[which_gz][pt].FACEi2;

                    const int8_t FACEXi[3] = {FACEX0, FACEX1, FACEX2};

                    // Initialize derivatives to crazy values, to ensure that
                    //   we will notice in case they aren't set properly.
                    REAL r = 1e100;
                    REAL partial_i_f = 1e100;
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

Finally, we calculate $\frac{df}{dt}$ without the $\frac{k}{r^{n+1}}$ term;

$$
\frac{\partial f}{\partial t} = -\frac{v}{r} \left( x^i \partial_i f + \left( f - f_0 \right) \right).
$$

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

                    contraction_term(params, which_gf, gfs, xx, FACEXi, i0, i1, i2, &r, &partial_i_f);

                    const REAL invr = 1./r;

                    const REAL source_rhs = -char_speed*(partial_i_f + invr*(gfs[IDX4S(which_gf,i0,i1,i2)] - var_at_infinity));
                    rhs_gfs[IDX4S(which_gf,i0,i1,i2)] = source_rhs;
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

## Step 3.d: Solving for $k$ \[Back to [top](#toc)\]
$$\label{k}$$

 Here we formulate a way to approximate $k$. If our solution satisfies the advection equation both at the ghost zones and at interior points close to the boundaries, then we may find $k$ by determining the portion of $f$ that is not accounted for by the equation

$$
\frac{\partial f}{\partial t} = \left[\frac{\partial f}{\partial t} \right]_{adv}= -\frac{v}{r} \left( x^i \partial_i f + \left( f - f_0 \right) \right).
$$

The above is the advection equation we arrive at assuming $f$ behaves purely as an outgoing spherical wave. For an interior point directly adjacent to a ghost zone point, $f$ must satisfy **both** the time evolution equation for prescribed points within the interior, $\left[\frac{\partial f}{\partial t} \right]_{evol}$, and the advection equation $\left[\frac{\partial f}{\partial t} \right]_{adv}$. We then find the difference as

$$
\delta = \left[\frac{\partial f}{\partial t} \right]_{evol} - \left[\frac{\partial f}{\partial t} \right]_{adv} = \frac{k}{r^{n+1}},
$$

i.e. $\delta$ represents the numerical departure from the expected purely wave-like behavior at that point. We solve for $\delta$ at this interior point and express $k$ as

$$
k = \delta r^{n+1}_{int}.
$$

Thus, the $\frac{k}{r^{n+1}}$ term for the associated ghost zone point may be expressed as

$$
\frac{k}{r^{n+1}_{gz}} = \delta \left( \frac{r_{int}}{r_{gz}} \right) ^{n+1}.
$$

We approximate $k$ in this fashion using the code below. Note that we activate this term only when *radial_falloff_power* > 0, which set to 3 by default.

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

                    /************* For radial falloff and the extrapolated k term *************/
                    if (radial_falloff_power > 0) {

                      // Move one point away from gz point to compare pure advection to df/dt|interior

                      const int i0_offset = i0+FACEX0;
                      const int i1_offset = i1+FACEX1;
                      const int i2_offset = i2+FACEX2;

                      // Initialize derivatives to crazy values, to ensure that
                      //   we will notice in case they aren't set properly.
                      REAL r_offset = 1e100;
                      REAL partial_i_f_offset = 1e100;

                      contraction_term(params, which_gf, gfs, xx, FACEXi, i0_offset, i1_offset, i2_offset, &r_offset, &partial_i_f_offset);

                      const REAL invr_offset = 1./r_offset;

                      // Pure advection: [FIXME: Add equation (appearing in Jupyter notebook documentation)]
                      const REAL extrap_rhs = char_speed*(partial_i_f_offset + invr_offset*(gfs[IDX4S(which_gf,i0_offset,i1_offset,i2_offset)] - var_at_infinity));

                      // Take difference between pure advection and df/dt|interior
                      const REAL diff_between_advection_and_f_rhs =
                          rhs_gfs[IDX4S(which_gf,i0_offset,i1_offset,i2_offset)] + extrap_rhs;

                      // Solve for k/(r_gz)^n+1 term
                      rhs_gfs[IDX4S(which_gf,i0,i1,i2)] += diff_between_advection_and_f_rhs*pow(r_offset*invr,radial_falloff_power);

                  }
                } // END for(int pt=0;pt<num_ob_gz_pts[which_gz];pt++)
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

## Step 3.e: Inner Boundary Conditions \[Back to [top](#toc)\]
$$\label{innerbcs}$$

Finally, we apply parity conditions for inner boundary conditions. Since the Sommerfeld boundary condition treats the right hand sides, these data are thus copied over to the right hand sides of points in the inner boundaries, according to appropriate parity conditions. For a detailed disussion on inner boundaries and parity conditions, see [Tutorial-Start_to_Finish-Curvilinear_BCs](Tutorial-Start_to_Finish-Curvilinear_BCs.ipynb).

```python
%%writefile -a $Ccodesdir/Sommerfeld.py

            // Apply INNER (parity) boundary conditions:
                for(int pt=0;pt<bcstruct->num_ib_gz_pts[which_gz];pt++) {
                    const int i0dest = bcstruct->inner[which_gz][pt].inner_bc_dest_pt.i0;
                    const int i1dest = bcstruct->inner[which_gz][pt].inner_bc_dest_pt.i1;
                    const int i2dest = bcstruct->inner[which_gz][pt].inner_bc_dest_pt.i2;
                    const int i0src  = bcstruct->inner[which_gz][pt].inner_bc_src_pt.i0;
                    const int i1src  = bcstruct->inner[which_gz][pt].inner_bc_src_pt.i1;
                    const int i2src  = bcstruct->inner[which_gz][pt].inner_bc_src_pt.i2;

                    rhs_gfs[IDX4S(which_gf,i0dest,i1dest,i2dest)] =
                            bcstruct->inner[which_gz][pt].parity[gfs_parity[which_gf]] * rhs_gfs[IDX4S(which_gf, i0src,i1src,i2src)];
            } // END for(int pt=0;pt<num_ib_gz_pts[which_gz];pt++)
        } // END for(int which_gz = 0; which_gz < NGHOSTS; which_gz++)
    } // END for(int which_gf=0;which_gf<NUM_GFS;which_gf++)
} // END function
"""
```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

Here we write the entire function to a C file.

```python
%%writefile -a $Ccodesdir/Sommerfeld.py
        with open(os.path.join(Ccodesdir,"boundary_conditions/apply_bcs_sommerfeld.h"),"w") as file:
            file.write(main_Ccode)

    def write_sommerfeld_file(self, Ccodesdir):
        self.write_sommerfeld_main_Ccode(Ccodesdir)
        print("""\nSuccessfully generated Sommerfeld boundary condition C code""")

```

    Appending to SommerfeldBoundaryCondition_Validate/Sommerfeld.py

# Step 4: Python file validation \[Back to [top](#toc)\]
$$\label{py_validate}$$

Here we validate the python code generated by this notebook, [SommerfeldBoundaryCondition_Validate/Sommerfeld.py](../edit/SommerfeldBoundaryCondition_Validate/Sommerfeld.py), against the trusted code in [CurviBoundaryConditions/CurviBoundaryConditions.py](../edit/CurviBoundaryConditions/CurviBoundaryConditions.py) line by line. Passing corresponds to complete agreement between the files.

Note that there is more content [CurviBoundaryConditions/CurviBoundaryConditions.py](../edit/CurviBoundaryConditions/CurviBoundaryConditions.py) relating to more than just the Sommerfeld boundary condition, so we start the comparison where the Sommerfeld code begins.

```python
# Then compare all files generated by this notebook
#      (output moved in previous code cell to validate/)
#      and the separate Python module (output to Baikal
#      and BaikalVacuum).
import difflib

def compare_two_files(filepath1,filepath2, file1_idx1=None):

    with open(filepath1) as file1, open(filepath2) as file2:
        # Read the lines of each file
        file1_lines = file1.readlines()
        file2_lines = file2.readlines()

        if file1_idx1!=None:
            file1_lines = file1_lines[file1_idx1:]
#         print(file1_lines)

        num_diffs = 0
        file1_lines_noleadingwhitespace = []
        for line in file1_lines:
            if line.strip() == "": # If the line contains only whitespace, remove all leading whitespace
                file1_lines_noleadingwhitespace.append(line.lstrip())
            else:
                file1_lines_noleadingwhitespace.append(line)
        file2_lines_noleadingwhitespace = []
        for line in file2_lines:
            if line.strip() == "": # If the line contains only whitespace, remove all leading whitespace
                file2_lines_noleadingwhitespace.append(line.lstrip())
            else:
                file2_lines_noleadingwhitespace.append(line)
        for line in difflib.unified_diff(file1_lines_noleadingwhitespace, file2_lines_noleadingwhitespace,
                                         fromfile=filepath1,
                                         tofile  =filepath2):
            sys.stdout.writelines(line)
            num_diffs = num_diffs + 1
        if num_diffs == 0:
            print("PASSED: "+filepath2+" matches trusted version")
        else:
            print("FAILED (see diff above): "+filepath2+" does NOT match trusted version")

import os

notebook_cfile = 'SommerfeldBoundaryCondition_Validate/Sommerfeld.py'
nrpy_cfile = 'CurviBoundaryConditions/CurviBoundaryConditions.py'

idx1=258
compare_two_files(nrpy_cfile, notebook_cfile, idx1)

```

    PASSED: SommerfeldBoundaryCondition_Validate/Sommerfeld.py matches trusted version

# Step 5: NRPy+ Interface for Applying Sommerfeld Boundary Conditions \[Back to [top](#toc)\]
$$\label{interface}$$

To apply the Sommerfled boundary condition to any given grid function, its wave speed at the boundaries, asymptotic value at infinity, and radial exponent of the k term (*radial_falloff_power*) must be specified. In general, a (*radial_falloff_power*) of 3 has been found to yield the best results, i.e. minimal initial transients and reflections.

Here we showcase the features of the NRPy+ interface for implementing this boundary condition and defining these values. The interface is a python class structure that allows the user to specify default values for each grid function, which may then be changed. This may be useful when the user wants to define the default values for several grid functions but wishes to specifically alter few others.

To begin, we define our global NRPy finite differencing order and our coordinate system of choice. Our Sommerfeld boundary condition driver check that the finite differencing chosen for the driver is less than or equal to the global finite differencing order.

```python
import finite_difference as fin       # NRPy+: Finite difference C code generation module
FD_order = 6
par.set_parval_from_str("finite_difference::FD_CENTDERIVS_ORDER",FD_order)

# Set the coordinate system for the numerical grid
# Choices are: Spherical, SinhSpherical, SinhSphericalv2, Cylindrical, SinhCylindrical,
#              SymTP, SinhSymTP
CoordSystem     = "Spherical"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
```

Next, let's define a few grid functions. Note that the boundary condition should be used only for variables defined in three spatial dimensions.

```python
# Step P3: Defining a couple of grid functions
uu, vv, ww, xx, yy, zz, = gri.register_gridfunctions("EVOL",["uu","vv","ww","xx","yy","zz"])

```

NRPy+ can now access these grid function names, and will store default values for the boundary condition. First we import `CurviBoundaryConditions.CurviBoundaryConditions`, and define a class instance _bcs_ using `sommerfeld_boundary_condition_class`.

```python
import CurviBoundaryConditions.CurviBoundaryConditions as cbcs
cbcs.Set_up_CurviBoundaryConditions(os.path.join(Ccodesdir,"boundary_conditions/"),
                    Cparamspath=os.path.join("../"), BoundaryCondition='Sommerfeld')

bcs = cbcs.sommerfeld_boundary_condition_class(fd_order=4,
                                             vars_radial_falloff_power_default=3,
                                             vars_speed_default=1.,
                                             vars_at_inf_default=1.)
# bcs.vars_radpower.items()
bcs.vars_at_infinity['VVGF'] = 0.0
bcs.write_sommerfeld_file(Ccodesdir)
```

    Wrote to file "SommerfeldBoundaryCondition_Validate/boundary_conditions/parity_conditions_symbolic_dot_products.h"
    Evolved parity: ( uu:0, vv:0, ww:0, xx:0, yy:0, zz:0 )

    Wrote to file "SommerfeldBoundaryCondition_Validate/boundary_conditions/EigenCoord_Cart_to_xx.h"

    Successfully generated Sommerfeld boundary condition C code

Using the instance _bcs_ we may change these default values using the grid function names, since NRPy+ stores these values in python dictionaries. We then print out the contents of the dictionaries.

```python
# Changing values for uu
bcs.vars_at_infinity['UUGF'] = 5.
bcs.vars_speed['UUGF']       = 0.5

# Changing values for zz
bcs.vars_at_infinity['ZZGF'] = 4.
bcs.vars_speed['ZZGF']       = 2.**0.5

print('GF values at infinity =', bcs.vars_at_infinity.items())
print('GF speeds = ', bcs.vars_speed.items())
print('GF radial powers = ' ,bcs.vars_radial_falloff_power.items())
```

    GF values at infinity = dict_items([('UUGF', 5.0), ('VVGF', 0.0), ('WWGF', 1.0), ('XXGF', 1.0), ('YYGF', 1.0), ('ZZGF', 4.0)])
    GF speeds =  dict_items([('UUGF', 0.5), ('VVGF', 1.0), ('WWGF', 1.0), ('XXGF', 1.0), ('YYGF', 1.0), ('ZZGF', 1.4142135623730951)])
    GF radial powers =  dict_items([('UUGF', 3), ('VVGF', 3), ('WWGF', 3), ('XXGF', 3), ('YYGF', 3), ('ZZGF', 3)])

Finally, we write these values into the `sommerfeld_params.h` file, and generate the `radial_derivative.h` file, which defines the function used to calculate the $\partial_r f$ term, which our C code may read from later, using the function write_sommerfeld_files(), which takes the C codes directory path and finite differencing order as inputs. __Only second and fourth order finite differences are supported at this time.__

# Step 6: Validation against the [Einstein Toolkit's NewRad boundary condition driver](https://www.einsteintoolkit.org/EinsteinEvolve/thornguide/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk) \[Back to [top](#toc)\]
$$\label{etk_validation}$$

Here we showcase some validation results of our Sommerfeld boundary condition as implemented in NRPy+ against [ETK's NewRad boundary condition driver](https://www.einsteintoolkit.org/thornguide/EinsteinEvolve/NewRad/documentation.html#XEinsteinEvolve_NewRad_Alcubierre:2002kk), storing the ETK data and subsequent plots in the [SommerfeldBoundaryCondition folder](SommerfeldBoundaryCondition). Specifically, we do so by

1. Generating plane wave initial data for ETK using the [ETK_thorn-IDScalarWaveNRPy notebook](Tutorial-ETK_thorn-IDScalarWaveNRPy.ipynb), with the NRPy+ code generation documented [here](Tutorial-ScalarWave.ipynb)

2. Generating the ETK evolution C codes using the [ETK_thorn-WaveToyNRPy notebook](Tutorial-ETK_thorn-WaveToyNRPy.ipynb), with the NRPy+ code generation generation also documented [here](Tutorial-ScalarWave.ipynb)

3. Compare results to [Tutorial-Start_to_Finish-ScalarWaveCurvilinear](Tutorial-Start_to_Finish-ScalarWaveCurvilinear.ipynb), adding 1e-16 to the log$_{10}$(relative error) plots to avoid taking the log$_{10}$ of zero where we have perfect agreement. $t$=0.3 represents 6 time steps into the systems

For both the evolution thorn and NRPy+ code we define the gridfunctions __UUGF__ and __VVGF__, use RK4 time stepping, and fourth order finite differencing. For the boundary condition parameters, we set uu_at_infinity = 2.0, vv_at_infinity = 0.0, and char_speed = 1.0, and var_radial_falloff_power = 3.0 for both evolution variables.

First we show validation results for the case of a scalar wave propagating in the +x direction (initial data, documented [here](Tutorial-ScalarWave.ipynb), with kk0 = 1, kk1 = kk2 = 0), overlaying $u\left(x, y=0,z=0,t=0.3\right)$ from the ETK thorn and from NRPy+, and plotting the relative difference between the two.

```python
from IPython.display import Image
from IPython.display import display

x_axis_plot = Image("SommerfeldBoundaryCondition/NRPy_vs_ETK_x-axis.png", width=400, height=400)
E_relx_axis_plot = Image("SommerfeldBoundaryCondition/E_rel_x-axis.png", width=400, height=400)
display(x_axis_plot,E_relx_axis_plot)
```

Lastly we show validation results for the case of a scalar wave propagating along the -x, -y, +z diagonal, and taking a slice along the y-axis at x=z=0 (initial data, documented [here](Tutorial-ScalarWave.ipynb), with kk0 = -1, kk1 = -1, kk2 = 1), overlaying $u\left(x=0, y,z=0,t=0.3\right)$ from the ETK thorn and from NRPy+, and plotting the relative difference between the two.

```python
diagonal = Image("SommerfeldBoundaryCondition/NRPy_vs_ETK_diagonal.png", width=400, height=400)
E_rel_diagonal = Image("SommerfeldBoundaryCondition/E_rel_diagonal.png", width=400, height=400)
display(diagonal, E_rel_diagonal)
```

# Step 7: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-SommerfeldBoundaryCondition.pdf](Tutorial-SommerfeldBoundaryCondition.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-SommerfeldBoundaryCondition")
```

    Created Tutorial-SommerfeldBoundaryCondition.tex, and compiled LaTeX file
        to PDF file Tutorial-SommerfeldBoundaryCondition.pdf