# $\text{interp_sphgrid_MO_ETK}$: An Einstein Toolkit Module for Interpolation to Spherical Grids

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

## This module is designed to interpolate arbitrary quantities on [Einstein Toolkit](https://einsteintoolkit.org/) Adaptive-Mesh Refinement (AMR) grids (using the [Carpet](https://carpetcode.org/) AMR infrastructure) to numerical grids with spherical sampling.

**This module has not yet undergone validation testing.**

## Introduction:
Given some set of $N$ quantities $\mathbf{Q}=\{Q_0,Q_1,Q_2,...,Q_{N-2},Q_{N-1}\}$, this module performs the following for each $Q_i$:

1. Evaluate $Q_i$ at all gridpoints that are not ghost zones. Sometimes $Q_i$ is computed using finite difference derivatives, so this is necessary.
1. Call upon Carpet's interpolation and interprocessor synchronization functions to fill in $Q_i$ at all ghost zones, *except* at the outer boundary. We do not generally trust $Q_i$ at the outer boundary due to errors associated with the approximate outer boundary conditions.
1. At this point, $Q_i$ is set at all gridpoints except ghost zones at the outer boundary. Interpolate $Q_i$ to the spherical grids, **maintaining the Cartesian basis for all vectors and tensors**, and append the result to a file.

This tutorial module takes a three-part structure. First, all the needed core Einstein Toolkit (ETK) C routines for interpolation are presented. Second, NRPy+ is used to output gridfunctions needed on the spherical grids. Third, the needed files for interfacing this module with the rest of the Einstein Toolkit (ccl files) are specified.

# Table of Contents:
$$\label{toc}$$

This module is organized as follows

1. [Step 1](#etkmodule): Setting up the Core C Code for the Einstein Toolkit Module
    1. [Setp 1.a](#etk_interp): Low-Level Einstein Toolkit Interpolation Function
    1. [Step 1.b](#sphericalgridnotes): Setting up the Spherical Grids
    1. [Step 1.c](#fileformat): Outputting to File
    1. [Step 1.d](#maininterpolator): The Main Interpolation Driver Function
1. [Step 2](#nrpy): Use NRPy+ C Output to Set All Output Gridfunctions
    1. [Step 2.a](#nrpy_list_of_funcs_interp): Set up NRPy-based "list_of_functions_to_interpolate.h"
        1. [Step 2.a.i](#nrpygrmhd): GRMHD quantities (IN PROGRESS)
        1. [Step 2.a.ii](#nrpy4metric): Compute all 10 components of the 4-metric $g_{\mu\nu}$
        1. [Step 2.a.iii](#nrpy4christoffels): Compute all 40 4-Christoffels $\Gamma^{\mu}_{\nu\delta}$
    1. [Step 2.b](#nrpy_c_callingfunction): C code calling function for the NRPy+ C output
    1. [Step 2.c](#nrpygetgfname): The `get_gf_name()` function
    1. [Step 2.d](#nrpy_interp_counter): C Code for Initializing and incrementing "InterpCounter"
1. [Step 3](#cclfiles): Interfacing with the rest of the Einstein Toolkit; Setting up CCL files
    1. [Step 3.a](#makecodedefn): $\text{make.code.defn}$
    1. [Step 3.b](#interfaceccl): $\text{interface.ccl}$
    1. [Step 3.c](#paramccl): $\text{param.ccl}$
    1. [Step 3.d](#scheduleccl): $\text{schedule.ccl}$
1. [Step 4](#readingoutputfile): Python Script for Reading the Output File
1. [Step 5](#latex_pdf_output): Output this module to $\LaTeX$-formatted PDF

# Step 1: Setting up the Core C Code for the Einstein Toolkit Module \[Back to [top](#toc)\]
$$\label{etkmodule}$$

First we set up the output directories for the ETK module:

```python
!mkdir interp_sphgrid_MO_ETK     2>/dev/null # 2>/dev/null: Don't throw an error or warning if the directory already exists.
!mkdir interp_sphgrid_MO_ETK/src 2>/dev/null # 2>/dev/null: Don't throw an error or warning if the directory already exists.
```

## Step 1.a: Low-Level ETK Interpolation Function \[Back to [top](#toc)\]
$$\label{etk_interp}$$

We start by writing the low-level interpolation function **Interpolate_to_sph_grid()**, which  to file.

**Interpolate_to_sph_grid()** takes as input
* **cctkGH**: Information about the underlying Cactus/Carpet grid hierarchy.
* **interp_num_points**: Number of destination interpolation points
* **point_x_temp, point_y_temp, point_z_temp**: Cartesian $(x,y,z)$ location for each of the **interp_num_points** interpolation points.
* **input_array_names[1]**: List of input gridfunction names to interpolate. We will do this only one gridfunction at a time, for gridfunction $Q_i$, as described above.

**Interpolate_to_sph_grid()** outputs:
* **output_f[1]**: The gridfunction **input_array_names[1]** interpolated to the set of **interp_num_points** specified in the input.

```python
%%writefile interp_sphgrid_MO_ETK/src/Interpolate_to_sph_grid.h

void Interpolate_to_sph_grid(cGH *cctkGH,CCTK_INT interp_num_points, CCTK_INT interp_order,
                             CCTK_REAL *point_x_temp,CCTK_REAL *point_y_temp,CCTK_REAL *point_z_temp,
                             const CCTK_STRING input_array_names[1], CCTK_REAL *output_f[1]) {
  DECLARE_CCTK_PARAMETERS;
  CCTK_INT ierr;

  const CCTK_INT NUM_INPUT_ARRAYS=1;
  const CCTK_INT NUM_OUTPUT_ARRAYS=1;

  CCTK_STRING coord_system = "cart3d";

  // Set up handles
  const CCTK_INT coord_system_handle = CCTK_CoordSystemHandle(coord_system);
  if (coord_system_handle < 0) {
    CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING,
        "can't get coordinate system handle for coordinate system \"%s\"!",
               coord_system);
  }

  const CCTK_INT operator_handle = CCTK_InterpHandle(interpolator_name);
  if (operator_handle < 0)
    CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING,
               "couldn't find interpolator \"%s\"!",
               interpolator_name);

  char interp_order_string[10];
  snprintf(interp_order_string, 10, "order=%d", interp_order);
  CCTK_STRING interpolator_pars = interp_order_string;
  CCTK_INT param_table_handle = Util_TableCreateFromString(interpolator_pars);
  if (param_table_handle < 0) {
    CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING,
               "bad interpolator parameter(s) \"%s\"!",
               interpolator_pars);
  }

  CCTK_INT operand_indices[NUM_INPUT_ARRAYS]; //NUM_OUTPUT_ARRAYS + MAX_NUMBER_EXTRAS];
  for(int i = 0 ; i < NUM_INPUT_ARRAYS  ; i++) {
    operand_indices[i] = i;
  }
  Util_TableSetIntArray(param_table_handle, NUM_OUTPUT_ARRAYS,
                        operand_indices, "operand_indices");

  CCTK_INT opcodes[NUM_INPUT_ARRAYS];
  for(int i = 0 ; i < NUM_INPUT_ARRAYS  ; i++) {
    opcodes[i] = 0;
  }
  Util_TableSetIntArray(param_table_handle, NUM_OUTPUT_ARRAYS,
                        opcodes, "opcodes");

  const void* interp_coords[3]
    = { (const void *) point_x_temp,
        (const void *) point_y_temp,
        (const void *) point_z_temp };

  CCTK_INT input_array_indices[NUM_INPUT_ARRAYS];
  for(int i = 0 ; i < NUM_INPUT_ARRAYS ; i++) {
    input_array_indices[i] = CCTK_VarIndex(input_array_names[i]);
    if(input_array_indices[i] < 0) {
      CCTK_VWarn(0, __LINE__, __FILE__, CCTK_THORNSTRING,
        "COULD NOT FIND VARIABLE '%s'.",
        input_array_names[i]);
      exit(1);
    }
  }

  CCTK_INT output_array_types[NUM_OUTPUT_ARRAYS];
  for(int i = 0 ; i < NUM_OUTPUT_ARRAYS ; i++) {
    output_array_types[i] = CCTK_VARIABLE_REAL;
  }

  void * output_arrays[NUM_OUTPUT_ARRAYS]
    = { (void *) output_f[0] };

  // actual interpolation call
  ierr = CCTK_InterpGridArrays(cctkGH,
                               3, // number of dimensions
                               operator_handle,
                               param_table_handle,
                               coord_system_handle,
                               interp_num_points,
                               CCTK_VARIABLE_REAL,
                               interp_coords,
                               NUM_INPUT_ARRAYS, // Number of input arrays
                               input_array_indices,
                               NUM_OUTPUT_ARRAYS, // Number of output arrays
                               output_array_types,
                               output_arrays);
  if (ierr<0) {
    CCTK_WARN(1,"interpolation screwed up");
    Util_TableDestroy(param_table_handle);
    exit(1);
  }

  ierr = Util_TableDestroy(param_table_handle);
  if (ierr != 0) {
    CCTK_WARN(1,"Could not destroy table");
    exit(1);
  }
}
```

    Overwriting interp_sphgrid_MO_ETK/src/Interpolate_to_sph_grid.h

## Step 1.b: Setting up the Spherical Grids \[Back to [top](#toc)\]
$$\label{sphericalgridnotes}$$

* By default, we set logarithmic radial coordinates: $r(x_{0,i}) = R_0 + e^{x_{0,i}}$, where
  + $x_{0,i} = x_{0, \mathrm{beg}} + \left(i+\frac{1}{2}\right) \Delta x_0$
  + $x_{0, {\mathrm{beg}}} = \log\left( R_{\mathrm{in}} - R_0 \right)$
  + $\Delta x_0 = \frac{1}{N_0}\log\left(\frac{R_\mathrm{out} - R_0}{R_\mathrm{in} - R_0}\right)$
* As for the polar angle $\theta$, there are two options:
  + **Option 1**:
  $$
  \theta(x_{1,j})  \, = \, \theta_c \, + \, \left( \pi - 2 \theta_c \right) x_{1,j} \,
+ \, \xi \, \sin\left(2 \pi x_{1,j} \right) \text{, where}
$$
    + $x_{1,j} = x_{1, \mathrm{beg}} + \left(j+\frac{1}{2}\right) \Delta x_1$
    + $\Delta x_1 = \frac{1}{N_1}$
  + **Option 2**:
  $$
  \theta(x_{1,j}) = \frac{\pi}{2} \left[  1  + \left(1-\xi \right) \left(2 x_{1,j} - 1 \right) + \left( \xi - \frac{2 \theta_c}{\pi} \right) \left( 2 x_{1,j} - 1 \right)^n \right] \text{, where}
  $$
    + $n$ is odd
    + $x_{1,j} = x_{1, \mathrm{beg}} + \left(j+\frac{1}{2}\right) \Delta x_1$
    + $\Delta x_1 = \frac{1}{N_1}$
* The azimuthal angle $\phi$ is uniform, so that $\phi(x_{2,k}) = x_{2,k}$:
  + $x_{2,k} \in [0,2\pi]$
  + $x_{2,k} = x_{2, \mathrm{beg}} + \left(k+\frac{1}{2}\right)\Delta x_{2}$
  + $\Delta x_{2} = \frac{ 2 \pi }{N_2}$

```python
%%writefile interp_sphgrid_MO_ETK/src/Set_up_interp_points_on_sph_grid.h

void sph_grid_Interpolate_many_pts__set_interp_pts(CCTK_ARGUMENTS) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  CCTK_REAL dx0 = log( (Rout - R0) / (Rin - R0) ) / ((CCTK_REAL)N0);
  CCTK_REAL dx1 =      1.0 / ((CCTK_REAL)N1);
  CCTK_REAL dx2 = 2.0*M_PI / ((CCTK_REAL)N2);
  CCTK_REAL x0_beg = log( Rin - R0 );
  CCTK_INT which_pt = 0;
  for(CCTK_INT k=0;k<N2;k++) for(CCTK_INT j=0;j<N1;j++) for(CCTK_INT i=0;i<N0;i++) {
    CCTK_REAL x0_i = x0_beg + ((CCTK_REAL)i + 0.5)*dx0;
    CCTK_REAL rr = R0 + exp(x0_i);

    CCTK_REAL x1_j = x1_beg + ((CCTK_REAL)j + 0.5)*dx1;
    CCTK_REAL th = -1e300;
    if(theta_option == 1) {
       th = th_c + (M_PI - 2.0*th_c)*x1_j + xi*sin(2.0*M_PI*x1_j);
    } else if (theta_option == 2) {
       th = M_PI/2.0 * ( 1.0 + (1.0 - xi)*(2.0*x1_j - 1.0) + (xi - 2.0*th_c/M_PI)*pow(2.0*x1_j - 1.0 ,th_n) );
    } else {
       printf("Error: theta_option = %d NOT SUPPORTED.",theta_option);
       exit(1);
    }

    CCTK_REAL x2_k = x2_beg + ((CCTK_REAL)k + 0.5)*dx2;
    CCTK_REAL ph = x2_k;

    points_x[which_pt] = rr*sin(th)*cos(ph);
    points_y[which_pt] = rr*sin(th)*sin(ph);
    points_z[which_pt] = rr*cos(th);
    which_pt++;
  }
}
```

    Overwriting interp_sphgrid_MO_ETK/src/Set_up_interp_points_on_sph_grid.h

## Step 1.c: Outputting to File (File format notes) \[Back to [top](#toc)\]
$$\label{fileformat}$$

Since they take almost no space relative to the data chunks, we attach the entire metadata to each interpolated function that is output:

```python
%%writefile interp_sphgrid_MO_ETK/src/output_to_file.h

#include "define_NumInterpFunctions.h"

// output_to_file() starts order and InterpCounter both with the value 1
void output_to_file(CCTK_ARGUMENTS,char gf_name[100],int *order,CCTK_REAL *output_f[1]) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  char filename[100];
  sprintf (filename, "%s/interp_sph_grids_MO.dat", out_dir);
  FILE *file;
  if(*InterpCounter == 1 && *order==1) {
    file = fopen (filename,"w");
  } else {
    file = fopen (filename,"a+");
  }
  if (! file) {
    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
                "interp_sph_grid__ET_thorn: Cannot open output file '%s'", filename);
    exit(1);
  }

  fwrite(gf_name, 100*sizeof(char), 1, file);
  fwrite(order, sizeof(CCTK_INT), 1, file);

  fwrite(&N0, sizeof(CCTK_INT), 1, file);
  fwrite(&R0, sizeof(CCTK_REAL), 1, file);
  fwrite(&Rin, sizeof(CCTK_REAL), 1, file);
  fwrite(&Rout, sizeof(CCTK_REAL), 1, file);

  fwrite(&N1, sizeof(CCTK_INT), 1, file);
  fwrite(&x1_beg, sizeof(CCTK_REAL), 1, file);
  fwrite(&theta_option, sizeof(CCTK_INT), 1, file);
  fwrite(&th_c, sizeof(CCTK_REAL), 1, file);
  fwrite(&xi, sizeof(CCTK_REAL), 1, file);
  fwrite(&th_n, sizeof(CCTK_INT), 1, file);

  fwrite(&N2, sizeof(CCTK_INT), 1, file);
  fwrite(&x2_beg, sizeof(CCTK_REAL), 1, file);

  CCTK_REAL magic_number = 1.130814081305130e-21;
  fwrite(&magic_number, sizeof(CCTK_REAL), 1, file);
  fwrite(&cctk_iteration, sizeof(CCTK_INT), 1, file);
  fwrite(&cctk_time, sizeof(CCTK_REAL), 1, file);
  for(CCTK_INT i=0;i<1;i++) {
    fwrite(output_f[i], sizeof(CCTK_REAL)*N0*N1*N2, 1, file);
  }

  fclose(file);
}
```

    Overwriting interp_sphgrid_MO_ETK/src/output_to_file.h

## Step 1.d: The Main Interpolation Driver Function \[Back to [top](#toc)\]
$$\label{maininterpolator}$$

The **Interpolate_to_sph_grid_main_function()** function calls the above functions as follows:
1. **sph_grid_Interpolate_many_pts__set_interp_pts()**: First set up the spherical grids
1. **Interpolate_to_sph_grid()**: Output

```python
%%writefile interp_sphgrid_MO_ETK/src/main_function.cc

// Include needed ETK & C library header files:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
// Needed for dealing with Cactus/ETK infrastructure
#include "cctk.h"
#include "cctk_Arguments.h"
#include "cctk_Parameters.h"
// Needed for low-level interpolation functions
#include "util_Table.h"
#include "util_String.h"

// Include locally-defined C++ functions:
#include "Set_up_interp_points_on_sph_grid.h"
#include "Interpolate_to_sph_grid.h"
#include "output_to_file.h"
#include "get_gf_name.h"

void Interpolate_to_sph_grid_main_function(CCTK_ARGUMENTS) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  // Perform interpolation only at iteration == interp_out_iteration:
  if(cctk_iteration != interp_out_iteration) return;

  // Set up spherically sampled interpolation grid arrays points_x,points_y,points_z:
  sph_grid_Interpolate_many_pts__set_interp_pts(CCTK_PASS_CTOC);

  // Set up output array:
  CCTK_REAL *output_f[1];
  output_f[0] = output_interped;
  // The name of the input gridfunction is always "interp_sphgrid_MO_ETK::interped_gf":
  const CCTK_STRING input_array_names[1] = { "interp_sphgrid_MO_ETK::interped_gf" };

  // Perform interpolation!
  for(int order=1; order <= 4; order *=2) {
      char gf_name[100];
      get_gf_name(*InterpCounter,gf_name);
      printf("Interpolating\033[1m %s \033[0m... using interpolation order = %d\n",gf_name,order);
      Interpolate_to_sph_grid(cctkGH, N0*N1*N2, order,
                                 points_x,points_y,points_z, input_array_names, output_f);

      if(CCTK_MyProc(cctkGH)==0) {
        for(int i=0;i<N0*N1*N2;i++) {
            if(output_f[0][i] > 1e20) {
                printf("BAD POINT: %s %d %e %e %e %e\n",gf_name,i,points_x[i],points_y[i],points_z[i], output_f[0][i]);
            }
        }
        output_to_file(CCTK_PASS_CTOC,gf_name,&order,output_f);
        printf("Interpolate_to_sph_grid_main_function(): Just output to file at iteration %d\n",cctk_iteration);
      } else {
        printf("Interpolate_to_sph_grid_main_function(): Process !=0 waiting for file output at iteration %d\n",cctk_iteration);
      }
  }
}
```

    Overwriting interp_sphgrid_MO_ETK/src/main_function.cc

# Step 2: Use NRPy+ C Output to Set All Output Gridfunctions \[Back to [top](#toc)\]
$$ \label{nrpy}$$

```python
# Step 2: Import needed NRPy+ parameters
import indexedexp as ixp
import grid as gri
import finite_difference as fin
from outputC import *
import sympy as sp
import NRPy_param_funcs as par
import loop

par.set_parval_from_str("grid::GridFuncMemAccess","ETK")

from collections import namedtuple
gf_interp = namedtuple('gf_interp', 'gf_description')
gf_interp_list = []
gf_interp_list.append(gf_interp("dummy -- used because this is a 1-offset array"))

interped_gf = gri.register_gridfunctions("AUX","interped_gf")

def interp_fileout(which_InterpCounter, expression, filename):
    kernel = fin.FD_outputC("returnstring",lhrh(lhs=gri.gfaccess("out_gfs","interped_gf"),rhs=expression),"outCverbose=False")
    output_type="a"
    if which_InterpCounter == 1:
        output_type="w"

    with open(filename, output_type) as file:
        file.write("if(*InterpCounter == "+str(which_InterpCounter)+") {\n")
        file.write(loop.loop(["i2","i1","i0"],
                             ["cctk_nghostzones[2]","cctk_nghostzones[1]","cctk_nghostzones[0]"],\
                             ["cctk_lsh[2]-cctk_nghostzones[2]",
                              "cctk_lsh[1]-cctk_nghostzones[1]",
                              "cctk_lsh[0]-cctk_nghostzones[0]"],\
                             ["1","1","1"],\
                             ["#pragma omp parallel for","",""],"   ",kernel))
        file.write("}\n")
    # If successful, return incremented which_InterpCounter:
    return which_InterpCounter+1
```

## Step 2.a: Set up NRPy-based "list_of_functions_to_interpolate.h" \[Back to [top](#top)\]
$$\label{nrpy_list_of_funcs_interp}$$

**First specify NRPy+ output file and initialize which_InterpCounter, which keeps track of the number of interpolated functions on the grid**

```python
NRPyoutfilename = "interp_sphgrid_MO_ETK/src/list_of_functions_to_interpolate.h"

which_InterpCounter = 1
```

### Step 2.a.i: GRMHD quantities (*IN PROGRESS; still working on adding vector potential*) \[Back to [top](#toc)\]
$$\label{nrpygrmhd}$$

These include
* $\rho_b$, the baryonic density (i.e., the HydroBase variable $\verb|rho|$)
* $P$, the total gas pressure (i.e., the HydroBase variable $\verb|press|$)
* $\Gamma v_{(n)}^i$, the Valencia 3-velocity times the Lorentz factor (i.e., the HydroBase 3-gridfuntion $\verb|vel|$, multiplied by the Lorentz factor). This definition of velocity has the advantage that after interpolation, it will not violate $u^\mu u_\mu = -1$. In terms of the IllinoisGRMHD 3-velocity $v^i = u^i / u^0$, the Valencia 3-velocity is given by (Eq. 11 of [Etienne *et al*](https://arxiv.org/pdf/1501.07276.pdf)):
$$
v_{(n)}^i = \frac{1}{\alpha} \left(v^i + \beta^i\right).
$$
Further, $\Gamma = \alpha u^0$ is given by (as shown [here](Tutorial-u0_smallb_Poynting-Cartesian.ipynb)):
$$
\Gamma = \alpha u^0 = \sqrt{\frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}}}.
$$
Therefore, $\Gamma v_{(n)}^i$ is given by
$$
\Gamma v_{(n)}^i = \frac{1}{\alpha} \left(v^i + \beta^i\right) \sqrt{\frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}}}.
$$
* $A_i$, the *unstaggered* magnetic vector potential.
* $B^i$, the *unstaggered* magnetic field vector (output only for validation purposes).

```python
# INPUT GRIDFUNCTIONS: The AUX or EVOL designation is *not* used in diagnostic modules.
gammaDD = ixp.register_gridfunctions_for_single_rank2("AUX","gammaDD", "sym01")
betaU = ixp.register_gridfunctions_for_single_rank1("AUX","betaU")
alpha = gri.register_gridfunctions("AUX","alpha")

DIM=3

gf_interp_list.append(gf_interp("IGM density primitive"))
rho_b       = gri.register_gridfunctions("AUX","rho_b")
interp_expr = rho_b
which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)

gf_interp_list.append(gf_interp("IGM pressure primitive"))
P = gri.register_gridfunctions("AUX","P")
interp_expr = P
which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)
```

Next we implement:
$$
v_{(n)}^i = \frac{1}{\alpha} \left(v^i + \beta^i\right),
$$
and
$$
\Gamma v_{(n)}^i = \sqrt{\frac{1}{1 - \gamma_{ij}v^i_{(n)}v^j_{(n)}}} v_{(n)}^i.
$$

```python
IGMvU = ixp.register_gridfunctions_for_single_rank1("AUX","IGMvU")
Valenciav = ixp.zerorank1()
for i in range(DIM):
    Valenciav[i] = 1/alpha * (IGMvU[i] + betaU[i])
v_dot_v = sp.sympify(0)
for i in range(DIM):
    for j in range(DIM):
        v_dot_v += gammaDD[i][j]*Valenciav[i]*Valenciav[j]

Gamma_times_ValenciavU = ixp.zerorank1()
for i in range(DIM):
    Gamma_times_ValenciavU[i] = sp.sqrt(1/(1 - v_dot_v))*Valenciav[i]
    gf_interp_list.append(gf_interp("Lorentz factor, times Valencia vU"+str(i)))
    interp_expr = Gamma_times_ValenciavU[i]
    which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)

# For testing:
# gf_interp_list.append(gf_interp("Lorentz factor"))
# interp_expr = v_dot_v
# which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)

# for i in range(DIM):
#     gf_interp_list.append(gf_interp("Valencia vU"+str(i)))
#     interp_expr = Valenciav[i]
#     which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)

BU = ixp.register_gridfunctions_for_single_rank1("AUX","BU")
for i in range(DIM):
    gf_interp_list.append(gf_interp("IGM magnetic field component B"+str(i)))
    interp_expr = BU[i]
    which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)
```

### Step 2.a.ii: Compute all 10 components of the 4-metric $g_{\mu\nu}$ \[Back to [top](#toc)\]
$$\label{nrpy4metric}$$

We are given $\gamma_{ij}$, $\alpha$, and $\beta^i$ from ADMBase, and the 4-metric is given in terms of these quantities as
$$
g_{\mu\nu} = \begin{pmatrix}
-\alpha^2 + \beta^k \beta_k & \beta_i \\
\beta_j & \gamma_{ij}
\end{pmatrix}.
$$

```python
# Eq. 2.121 in B&S
betaD = ixp.zerorank1()
for i in range(DIM):
    for j in range(DIM):
        betaD[i] += gammaDD[i][j]*betaU[j]

# Now compute the beta contraction.
beta2 = sp.sympify(0)
for i in range(DIM):
    beta2 += betaU[i]*betaD[i]

# Eq. 2.122 in B&S
g4DD = ixp.zerorank2(DIM=4)
g4DD[0][0] = -alpha**2 + beta2
for i in range(DIM):
    g4DD[i+1][0] = g4DD[0][i+1] = betaD[i]
for i in range(DIM):
    for j in range(DIM):
        g4DD[i+1][j+1] = gammaDD[i][j]

for mu in range(4):
    for nu in range(mu,4):
        gf_interp_list.append(gf_interp("4-metric component g4DD"+str(mu)+str(nu)))
        interp_expr = g4DD[mu][nu]
        which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)
```

### Step 2.a.iii:  Compute all 40 4-Christoffels $\Gamma^{\mu}_{\nu\delta}$ \[Back to [top](#toc)\]
$$\label{nrpy4christoffels}$$

By definition,
$$
\Gamma^{\mu}_{\nu\delta} = \frac{1}{2} g^{\mu\eta} \left(g_{\eta\nu,\delta} + g_{\eta\delta,\nu} - g_{\nu\delta,\eta}  \right)
$$

Recall that $g_{\mu\nu}$ is given from $\gamma_{ij}$, $\alpha$, and $\beta^i$ via
$$
g_{\mu\nu} = \begin{pmatrix}
-\alpha^2 + \beta^k \beta_k & \beta_i \\
\beta_j & \gamma_{ij}
\end{pmatrix}.
$$

The derivatives $g_{\mu\nu,\eta}$ are then computed in terms of finite-difference derivatives of the input ADM gridfunctions $\gamma_{ij}$, $\alpha$, and $\beta^i$, **assuming that the 4-metric is static, so that $\partial_t g_{\mu\nu}=0$ for all $\mu$ and $\nu$**.

To compute $g^{\mu\nu}$, we use the standard formula (Eq. 4.49 in [Gourgoulhon](https://arxiv.org/pdf/gr-qc/0703035.pdf)):
$$
g^{\mu\nu} = \begin{pmatrix}
-\frac{1}{\alpha^2} & \frac{\beta^i}{\alpha^2} \\
\frac{\beta^i}{\alpha^2} & \gamma^{ij} - \frac{\beta^i\beta^j}{\alpha^2}
\end{pmatrix},
$$
where $\gamma^{ij}$ is given by the inverse of $\gamma_{ij}$.

```python
betaDdD = ixp.zerorank2()
gammaDD_dD = ixp.declarerank3("gammaDD_dD","sym01")
betaU_dD   = ixp.declarerank2("betaU_dD","nosym")
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            # Recall that betaD[i] = gammaDD[i][j]*betaU[j] (Eq. 2.121 in B&S)
            betaDdD[i][k] += gammaDD_dD[i][j][k]*betaU[j] + gammaDD[i][j]*betaU_dD[j][k]

# Eq. 2.122 in B&S
g4DDdD = ixp.zerorank3(DIM=4)
alpha_dD   = ixp.declarerank1("alpha_dD")
for i in range(DIM):
    # Recall that g4DD[0][0] = -alpha^2 + betaU[i]*betaD[i]
    g4DDdD[0][0][i+1] += -2*alpha*alpha_dD[i]
    for j in range(DIM):
        g4DDdD[0][0][i+1] += betaU_dD[j][i]*betaD[j] + betaU[j]*betaDdD[j][i]

for i in range(DIM):
    for j in range(DIM):
        # Recall that g4DD[i][0] = g4DD[0][i] = betaD[i]
        g4DDdD[i+1][0][j+1] = g4DDdD[0][i+1][j+1] = betaDdD[i][j]
for i in range(DIM):
    for j in range(DIM):
        for k in range(DIM):
            # Recall that g4DD[i][j] = gammaDD[i][j]
            g4DDdD[i+1][j+1][k+1] = gammaDD_dD[i][j][k]

gammaUU, dummyDET = ixp.symm_matrix_inverter3x3(gammaDD)

g4UU = ixp.zerorank2(DIM=4)
g4UU[0][0] = -1 / alpha**2
for i in range(DIM):
    g4UU[0][i+1] = g4UU[i+1][0] = betaU[i]/alpha**2
for i in range(DIM):
    for j in range(DIM):
        g4UU[i+1][j+1] = gammaUU[i][j] - betaU[i]*betaU[j]/alpha**2
```

Again, we are to compute:
$$
\Gamma^{\mu}_{\nu\delta} = \frac{1}{2} g^{\mu\eta} \left(g_{\eta\nu,\delta} + g_{\eta\delta,\nu} - g_{\nu\delta,\eta}  \right)
$$

```python
Gamma4UDD = ixp.zerorank3(DIM=4)
for mu in range(4):
    for nu in range(4):
        for delta in range(4):
            for eta in range(4):
                Gamma4UDD[mu][nu][delta] += sp.Rational(1,2)*g4UU[mu][eta]*\
                (g4DDdD[eta][nu][delta] + g4DDdD[eta][delta][nu] - g4DDdD[nu][delta][eta])

# Now output the 4-Christoffels to file:
for mu in range(4):
    for nu in range(4):
        for delta in range(nu,4):
            gf_interp_list.append(gf_interp("4-Christoffel GammaUDD"+str(mu)+str(nu)+str(delta)))
            interp_expr = Gamma4UDD[mu][nu][delta]
            which_InterpCounter = interp_fileout(which_InterpCounter,interp_expr,NRPyoutfilename)
```

## Step 2.b: C code calling function for the NRPy+ C output \[Back to [top](#toc)\]
$$\label{nrpy_c_callingfunction}$$

In the above blocks, we wrote and appended to a file "list_of_functions_to_interpolate.h". Here we write the calling function for this C code.

```python
%%writefile interp_sphgrid_MO_ETK/src/construct_function_to_interpolate__store_to_interped_gf.cc
#include <stdio.h>
#include <stdlib.h>
#include "cctk.h"
#include "cctk_Arguments.h"
#include "cctk_Parameters.h"

// Set the gridfunction interped_gf, according to the interpolation counter variable interp_counter.
//    For example, we might interpolate "IllinoisGRMHD::rho_b" if interp_counter==0. The following
//    function takes care of these
void list_of_functions_to_interpolate(cGH *cctkGH,const CCTK_INT *cctk_lsh,const CCTK_INT *cctk_nghostzones,
                                     const CCTK_REAL invdx0,const CCTK_REAL invdx1,const CCTK_REAL invdx2,
                                     const CCTK_INT *InterpCounter,
                                     const CCTK_REAL *rho_bGF,const CCTK_REAL *PGF,
                                     const CCTK_REAL *IGMvU0GF,const CCTK_REAL *IGMvU1GF,const CCTK_REAL *IGMvU2GF,
                                     const CCTK_REAL *BU0GF,const CCTK_REAL *BU1GF,const CCTK_REAL *BU2GF,
                                     const CCTK_REAL *gammaDD00GF,const CCTK_REAL *gammaDD01GF,const CCTK_REAL *gammaDD02GF,
                                     const CCTK_REAL *gammaDD11GF,const CCTK_REAL *gammaDD12GF,const CCTK_REAL *gammaDD22GF,
                                     const CCTK_REAL *betaU0GF,const CCTK_REAL *betaU1GF,const CCTK_REAL *betaU2GF,
                                     const CCTK_REAL *alphaGF,   CCTK_REAL *interped_gfGF) {
#include "list_of_functions_to_interpolate.h"
}

void construct_function_to_interpolate__store_to_interped_gf(CCTK_ARGUMENTS) {
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;
  const CCTK_REAL invdx0 = 1.0 / CCTK_DELTA_SPACE(0);
  const CCTK_REAL invdx1 = 1.0 / CCTK_DELTA_SPACE(1);
  const CCTK_REAL invdx2 = 1.0 / CCTK_DELTA_SPACE(2);
  list_of_functions_to_interpolate(cctkGH,cctk_lsh,cctk_nghostzones,invdx0,invdx1,invdx2,
                                   InterpCounter,
                                   rho_b,P,
                                   vx,vy,vz,
                                   Bx,By,Bz,
                                   gxx,gxy,gxz,gyy,gyz,gzz,
                                   betax,betay,betaz,alp, interped_gf);
// interped_gf will be interpolated across AMR boundaries, meaning that
//    it must be prolongated. Only gridfunctions with 3 timelevels stored
//    may be prolongated (provided time_interpolation_order is set to the
//    usual value of 2). We should only call this interpolation routine
//    at iterations in which all gridfunctions are on the same timelevel
//    (usually a power of 2), which will ensure that the following
//    "filling of the timelevels" is completely correct.
#pragma omp parallel for
    for(int i=0;i<cctk_lsh[0]*cctk_lsh[1]*cctk_lsh[2];i++) {
        interped_gf_p[i]   = interped_gf[i];
        interped_gf_p_p[i] = interped_gf[i];
    }
}
```

    Overwriting interp_sphgrid_MO_ETK/src/construct_function_to_interpolate__store_to_interped_gf.cc

## Step 2.c: The `get_gf_name()` function \[Back to [top](#toc)\]
$$\label{nrpygetgfname}$$

```python
with open("interp_sphgrid_MO_ETK/src/get_gf_name.h", "w") as file:
    file.write("void get_gf_name(const int InterpCounter,char gf_name[100]) {\n")
    for i in range(1,which_InterpCounter):
        file.write("    if(InterpCounter=="+str(i)+") { snprintf(gf_name,100,\""+gf_interp_list[i].gf_description+"\"); return; }\n")
    file.write("    printf(\"Error. InterpCounter = %d unsupported. I should not be here.\\n\",InterpCounter); exit(1);\n")
    file.write("}\n")
```

## Step 2.d: C Code for Initializing and incrementing "InterpCounter" \[Back to [top](#toc)\]
$$\label{nrpy_interp_counter}$$
The gridfunctions are interpolated one at a time based on the current value of the index quantity "InterpCounter". Here we write the C code needed for initializing and incrementing this variable.

```python
with open("interp_sphgrid_MO_ETK/src/define_NumInterpFunctions.h", "w") as file:
    file.write("#define NumInterpFunctions "+str(which_InterpCounter)+"\n")
```

```python
%%writefile interp_sphgrid_MO_ETK/src/interp_counter.cc
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <ctype.h>
#include "cctk.h"
#include "cctk_Arguments.h"
#include "cctk_Parameters.h"

#include "define_NumInterpFunctions.h"

void SphGrid_InitializeInterpCounterToZero(CCTK_ARGUMENTS)
{
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;
  *InterpCounter = 0;

  if(verbose==2) printf("interp_sphgrid_MO_ETK: Just set InterpCounter to %d\n",*InterpCounter);
}

void SphGrid_InitializeInterpCounter(CCTK_ARGUMENTS)
{
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

  if(cctk_iteration == interp_out_iteration) {
    *InterpCounter = 1;
    if(verbose==2) printf("interp_sphgrid_MO_ETK: Just set InterpCounter to %d ; ready to start looping over interpolated gridfunctions!\n",
                          *InterpCounter);
  }
}

// This function increments InterpCounter if we are at the interp_out_iteration until
// it hits NumInterpFunctions. At this iteration, InterpCounter is set to zero, which
// exits the loop.
void SphGrid_IncrementInterpCounter(CCTK_ARGUMENTS)
{
  DECLARE_CCTK_ARGUMENTS;
  DECLARE_CCTK_PARAMETERS;

    if(*InterpCounter == NumInterpFunctions-1) {
        *InterpCounter = 0;
        if(verbose==2) printf("interp_sphgrid_MO_ETK: Finished! Just zeroed InterpCounter.\n");
    } else {
        (*InterpCounter)++;
        if(verbose==2) printf("interp_sphgrid_MO_ETK: Just incremented InterpCounter to %d of %d\n",*InterpCounter,NumInterpFunctions-1);
    }
}
```

    Overwriting interp_sphgrid_MO_ETK/src/interp_counter.cc

# Step 3: Define how this module interacts and interfaces with the larger Einstein Toolkit infrastructure \[Back to [top](#toc)\]
$$\label{cclfiles}$$

Writing a module ("thorn") within the Einstein Toolkit requires that three "ccl" files be constructed, all in the root directory of the thorn:

1. $\text{interface.ccl}$: defines the gridfunction groups needed, and provides keywords denoting what this thorn provides and what it should inherit from other thorns.
1. $\text{param.ccl}$: specifies free parameters within the thorn.
1. $\text{schedule.ccl}$: allocates storage for gridfunctions, defines how the thorn's functions should be scheduled in a broader simulation, and specifies the regions of memory written to or read from gridfunctions.

## Step 3.a: $\text{make.code.defn}$ \[Back to [top](#toc)\]
$$\label{makecodedefn}$$

Before writing the "ccl" files, we first add Einstein Toolkit's equivalent of a Makefile, the make.code.defn file:

```python
%%writefile interp_sphgrid_MO_ETK/src/make.code.defn
# Main make.code.defn file for thorn interp_sphgrid_MO_ETK

# Source files in this directory
SRCS =  main_function.cc interp_counter.cc construct_function_to_interpolate__store_to_interped_gf.cc
```

    Overwriting interp_sphgrid_MO_ETK/src/make.code.defn

## Step 3.b: $\text{interface.ccl}$ \[Back to [top](#toc)\]
$$\label{interfaceccl}$$

Let's now write $\text{interface.ccl}$. The [official Einstein Toolkit (Cactus) documentation](http://cactuscode.org/documentation/referencemanual/ReferenceManual.html) defines what must/should be included in an interface.ccl file [**here**](http://cactuscode.org/documentation/referencemanual/ReferenceManualch8.html#x12-260000C2.2).

```python
%%writefile interp_sphgrid_MO_ETK/interface.ccl

# With "implements", we give our thorn its unique name.
implements: interp_sphgrid_MO_ETK

# By "inheriting" other thorns, we tell the Toolkit that we
#   will rely on variables/function that exist within those
#   functions.
inherits:   admbase IllinoisGRMHD Grid

# Tell the Toolkit that we want "interped_gf" and "InterpCounter"
#    and invariants to NOT be visible to other thorns, by using
#    the keyword "private". Note that declaring these
#    gridfunctions here *does not* allocate memory for them;
#    that is done by the schedule.ccl file.
private:
CCTK_REAL interpolation_gf type=GF timelevels=3 tags='Checkpoint="no"'
{
  interped_gf
} "Gridfunction containing output from interpolation."

int InterpCounterVar type = SCALAR tags='checkpoint="no"'
{
  InterpCounter
} "Counter that keeps track of which function we are interpolating."

CCTK_REAL interp_pointcoords_and_output_arrays TYPE=ARRAY DISTRIB=CONSTANT DIM=1 SIZE=N0*N1*N2 tags='checkpoint="no"'
{
  points_x,points_y,points_z,
  output_interped
}
```

    Overwriting interp_sphgrid_MO_ETK/interface.ccl

## Step 3.c: $\text{param.ccl}$ \[Back to [top](#toc)\]
$$\label{paramccl}$$

We will now write the file $\text{param.ccl}$. This file allows the listed parameters to be set at runtime. We also give allowed ranges and default values for each parameter. More information on this file's syntax can be found in the [official Einstein Toolkit documentation](http://cactuscode.org/documentation/referencemanual/ReferenceManualch8.html#x12-265000C2.3).

```python
%%writefile interp_sphgrid_MO_ETK/param.ccl

# Output the interpolated data to the IO::out_dir directory:
shares: IO
USES STRING out_dir

restricted:

########################################
# BASIC THORN STEERING PARAMETERS
CCTK_INT interp_out_iteration "Which iteration to interpolate to spherical grids?" STEERABLE=ALWAYS
{
  0:* :: ""
} 960000

## Interpolator information
CCTK_STRING interpolator_name "Which interpolator to use?" STEERABLE=ALWAYS
{
  ".+" :: "Any nonempty string; an unsupported value will throw an error."
} "Lagrange polynomial interpolation"

CCTK_INT verbose "Set verbosity level: 1=useful info; 2=moderately annoying (though useful for debugging)" STEERABLE=ALWAYS
{
  0:2 :: "0 = no output; 1=useful info; 2=moderately annoying (though useful for debugging)"
} 2
########################################
# SPHERICAL COORDINATE SYSTEM PARAMETERS
CCTK_INT N0 "Number of points in r direction" STEERABLE=ALWAYS
{
  0:* :: ""
} 96

CCTK_INT N1 "Number of points in theta direction" STEERABLE=ALWAYS
{
  0:* :: ""
} 96

CCTK_INT N2 "Number of points in phi direction" STEERABLE=ALWAYS
{
  0:* :: ""
} 96

##########
# Cartesian position of center of spherical grid (usually center of BH) -- CURRENTLY UNSUPPORTED!
CCTK_REAL x_center "x-position of center." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0

CCTK_REAL y_center "y-position of center." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0

CCTK_REAL z_center "z-position of center." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0

##########
# Radial parameters:
CCTK_REAL R0 "Radial offset: r(x0) = R_0 + exp(x0). Probably should keep it set to zero." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0

CCTK_REAL Rin "x0 offset: x0 = log(Rin-R0) + (i + 0.5)Dx0." STEERABLE=ALWAYS
{
  0:* :: ""
} 1.08986052555408

CCTK_REAL Rout "Dx0 = log( (Rout-R0) / (Rin-R0) )/N0" STEERABLE=ALWAYS
{
  0:* :: ""
} 80.0

##########
# Theta parameters:
CCTK_REAL x1_beg "x1 offset: x1 = x1_beg + (j + 0.5)Dx1. Probably should keep it set to zero." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0

CCTK_INT theta_option "Which prescription for theta should be used? 1 or 2?" STEERABLE=ALWAYS
{
  1:2 :: ""
} 1

CCTK_REAL th_c "theta_c: Angular cutout size for theta = 0 and pi" STEERABLE=ALWAYS
{
  0:* :: ""
} 0.053407075111026485 # 0.017*pi

CCTK_REAL xi "Amplitude of nonlinear part of the theta distribution." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.25

CCTK_INT th_n "Power of nonlinear part of theta distribution. Only for theta_option=2" STEERABLE=ALWAYS
{
  0:* :: ""
} 9

##########
# Phi parameters:
CCTK_REAL x2_beg "x2 offset: x2 = x2_beg + (k + 0.5)Dx2. Probably should keep it set to zero." STEERABLE=ALWAYS
{
  0:* :: ""
} 0.0
########################################
```

    Overwriting interp_sphgrid_MO_ETK/param.ccl

## Step 3.d: $\text{schedule.ccl}$ \[Back to [top](#toc)\]
$$\label{scheduleccl}$$

Finally, we will write the file $\text{schedule.ccl}$; its official documentation is found [here](http://cactuscode.org/documentation/referencemanual/ReferenceManualch8.html#x12-268000C2.4).

This file declares storage for variables declared in the $\text{interface.ccl}$ file and specifies when the various parts of the thorn will be run:

```python
%%writefile interp_sphgrid_MO_ETK/schedule.ccl

STORAGE: interpolation_gf[3]
STORAGE: InterpCounterVar
STORAGE: interp_pointcoords_and_output_arrays

#############################
SCHEDULE SphGrid_InitializeInterpCounterToZero AT CCTK_INITIAL
{
  LANG: C
  OPTIONS: GLOBAL
} "Initialize InterpCounter variable to zero"

SCHEDULE SphGrid_InitializeInterpCounterToZero AT CCTK_POST_RECOVER_VARIABLES
{
  LANG: C
  OPTIONS: GLOBAL
} "Initialize InterpCounter variable to zero"

SCHEDULE SphGrid_InitializeInterpCounter before SphGrid_InterpGroup AT CCTK_ANALYSIS
{
  LANG: C
  OPTIONS: GLOBAL
} "Initialize InterpCounter variable"
##################

SCHEDULE GROUP SphGrid_InterpGroup AT CCTK_ANALYSIS BEFORE CarpetLib_printtimestats BEFORE CarpetLib_printmemstats AFTER Convert_to_HydroBase WHILE interp_sphgrid_MO_ETK::InterpCounter
{
} "Perform all spherical interpolations. This group is only actually scheduled at cctk_iteration==interp_out_iteration."

SCHEDULE construct_function_to_interpolate__store_to_interped_gf in SphGrid_InterpGroup before DoSum
{
  STORAGE: interpolation_gf[3],InterpCounterVar,interp_pointcoords_and_output_arrays
  OPTIONS: GLOBAL,LOOP-LOCAL
  SYNC: interpolation_gf
  LANG: C
} "Construct the function to interpolate"

SCHEDULE Interpolate_to_sph_grid_main_function in SphGrid_InterpGroup after construct_function_to_interpolate__store_to_interped_gf
{
  OPTIONS: GLOBAL
  LANG: C
} "Perform interpolation and output result to file."
#######
SCHEDULE SphGrid_IncrementInterpCounter in SphGrid_InterpGroup after Interpolate_to_sph_grid_main_function
{
  LANG: C
  OPTIONS: GLOBAL
} "Increment InterpCounter variable, or set to zero once loop is complete."
##################
```

    Overwriting interp_sphgrid_MO_ETK/schedule.ccl

# Step 4: Python Script for Reading the Output File \[Back to [top](#toc)\]
$$\label{readingoutputfile}$$

Here is a Python code for reading the output file generated by this thorn. It is based on a collection of Python scripts written by Bernard Kelly, available [here](https://bitbucket.org/zach_etienne/nrpy/src/master/mhd_diagnostics/).

After generating the output file "interp_sphgrid_MO_ETK.dat" using the Einstein Toolkit thorn above, this script will read in all the data. Processing can then be done by straightforward modification of this script. Save the script as Interp_Sph_ReadIn.py, and run it using the command

**python Interp_Sph_ReadIn.py interp_sphgrid_MO_ETK.dat 58 outfile**

Currently the last parameter "outfile" is required but not used.

```python
"""
interp_sphgrid_MO_ETK.dat File Reader. Compatible with Python 2.7+ and 3.6+ at least.

Zachariah B. Etienne

Based on Python scripts written by Bernard Kelly:
https://bitbucket.org/zach_etienne/nrpy/src/master/mhd_diagnostics/

Find the latest version of this reader at the bottom of this Jupyter notebook:
https://github.com/zachetienne/nrpytutorial/blob/master/Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids.ipynb

Usage instructions:

From the command-line, run via:
python Interp_Sph_ReadIn.py interp_sphgrid_MO_ETK.dat [number of gridfunctions (58 or so)] [outfile]

Currently the last parameter "outfile" is required but not actually used.
"""
import numpy as np
import struct
import sys
import argparse

parser = argparse.ArgumentParser(description='Read file.')
parser.add_argument("datafile", help="main data file")
parser.add_argument("number_of_gridfunctions", help="number of gridfunctions")

parser.add_argument("outfileroot", help="root of output file names")

args = parser.parse_args()

datafile = args.datafile
outfileroot = args.outfileroot
number_of_gridfunctions = int(args.number_of_gridfunctions)

print("reading from "+str(datafile))

"""
read_char_array():
Reads a character array of size="size"
from a file (with file handle = "filehandle")
and returns the character array as a proper
Python string.
"""
def read_char_array(filehandle,size):
    reached_end_of_string = False
    chartmp = struct.unpack(str(size)+'s', filehandle.read(size))[0]

    #https://docs.python.org/3/library/codecs.html#codecs.decode
    char_array_orig = chartmp.decode('utf-8',errors='ignore')

    char_array = ""
    for i in range(len(char_array_orig)):
        char = char_array_orig[i]
        # C strings end in '\0', which in Python-ese is '\x00'.
        #   As characters read after the end of the string will
        #   generally be gibberish, we no longer append
        #   to the output string after '\0' is reached.
        if   sys.version_info[0]==3 and bytes(char.encode('utf-8')) == b'\x00':
            reached_end_of_string = True
        elif sys.version_info[0]==2 and char ==  '\x00':
            reached_end_of_string = True

        if reached_end_of_string == False:
            char_array += char
        else:
            pass # Continue until we've read 'size' bytes
    return char_array

"""
read_header()
Reads the header from a file.
"""
def read_header(filehandle):
    # This function makes extensive use of Python's struct.unpack
    # https://docs.python.org/3/library/struct.html
    # First store gridfunction name and interpolation order used:
    # fwrite(gf_name, 100*sizeof(char), 1, file);
    gf_name = read_char_array(filehandle,100)
    # fwrite(order, sizeof(CCTK_INT), 1, file);
    order = struct.unpack('i',filehandle.read(4))[0]

    # Then the radial grid parameters:
    # fwrite( & N0, sizeof(CCTK_INT), 1, file);
    N0 =    struct.unpack('i',filehandle.read(4))[0]
    # fwrite( & R0, sizeof(CCTK_REAL), 1, file);
    R0 =    struct.unpack('d',filehandle.read(8))[0]
    # fwrite( & Rin, sizeof(CCTK_REAL), 1, file);
    Rin =   struct.unpack('d',filehandle.read(8))[0]
    # fwrite( & Rout, sizeof(CCTK_REAL), 1, file);
    Rout =  struct.unpack('d',filehandle.read(8))[0]

    # Then the grid parameters related to the theta coordinate:
    # fwrite( & N1, sizeof(CCTK_INT), 1, file);
    N1           = struct.unpack('i', filehandle.read(4))[0]
    # fwrite( & x1_beg, sizeof(CCTK_REAL), 1, file);
    x1_beg       = struct.unpack('d', filehandle.read(8))[0]
    # fwrite( & theta_option, sizeof(CCTK_INT), 1, file);
    theta_option = struct.unpack('i', filehandle.read(4))[0]
    # fwrite( & th_c, sizeof(CCTK_REAL), 1, file);
    th_c         = struct.unpack('d', filehandle.read(8))[0]
    # fwrite( & xi, sizeof(CCTK_REAL), 1, file);
    xi           = struct.unpack('d', filehandle.read(8))[0]
    # fwrite( & th_n, sizeof(CCTK_INT), 1, file);
    th_n         = struct.unpack('i', filehandle.read(4))[0]

    # Then the grid parameters related to the phi coordinate:
    # fwrite( & N2, sizeof(CCTK_INT), 1, file);
    N2     = struct.unpack('i', filehandle.read(4))[0]
    # fwrite( & x2_beg, sizeof(CCTK_REAL), 1, file);
    x2_beg = struct.unpack('d', filehandle.read(8))[0]

    magic_number_check = 1.130814081305130e-21
    # fwrite( & magic_number, sizeof(CCTK_REAL), 1, file);
    magic_number = struct.unpack('d', filehandle.read(8))[0]
    if magic_number != magic_number_check:
        print("Error: Possible file corruption: Magic number mismatch. Found magic number = "+str(magic_number)+" . Expected "+str(magic_number_check))
        exit(1)
    # fwrite( & cctk_iteration, sizeof(CCTK_INT), 1, file);
    cctk_iteration = struct.unpack('i', filehandle.read(4))[0]
    # fwrite( & cctk_time, sizeof(CCTK_REAL), 1, file);
    cctk_time      = struct.unpack('d', filehandle.read(8))[0]

    return gf_name,order,N0,R0,Rin,Rout,N1,x1_beg,theta_option,th_c,xi,th_n,N2,x2_beg,cctk_iteration,cctk_time

# Now open the file and read all the data
with open(datafile,"rb") as f:
    # Main loop over all gridfunctions
    for i in range(number_of_gridfunctions):
        # Data are output in chunks, one gridfunction at a time, with metadata
        #    for each gridfunction stored at the top of each chunk
        # First read in the metadata:
        gf_name, order, N0, R0, Rin, Rout, N1, x1_beg, theta_option, th_c, xi, th_n, N2, x2_beg, cctk_iteration, cctk_time = read_header(f)
        print("\nReading gridfunction "+gf_name+", stored at interp order = "+str(order))
        data_chunk_size = N0*N1*N2*8 # 8 bytes per double-precision number
        # Next read in the full gridfunction data
        bytechunk = f.read(data_chunk_size)
        # Process the data using NumPy's frombuffer() function:
        #   https://docs.scipy.org/doc/numpy/reference/generated/numpy.frombuffer.html
        buffer_res = np.frombuffer(bytechunk)
        # Reshape the data into a 3D NumPy array:
        #   https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html
        this_data = buffer_res.reshape(N0,N1,N2)

        # Sanity check: Make sure the output in the "middle" of the grid looks reasonable.
        ii = int(N0/2)
        jj = int(N1/2)
        kk = int(N2/2)
        with open("output-gf"+str(i)+".txt","w") as file:
            for ii in range(N0):
                for kk in range(N2):
                    r  = ii*1.0/N0
                    th = (jj*1.0)*np.pi/N1
                    ph = (kk*1.0)*2.0*np.pi/N2
                    xx = r*np.sin(th)*np.cos(ph)
                    yy = r*np.sin(th)*np.sin(ph)
                    zz = r*np.cos(th)
                    file.write(str(xx)+" "+str(yy)+" "+str(zz)+" "+str(this_data[kk,jj,ii])+"\n")

```

# Step 5: Output this module to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.pdf](Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
!jupyter nbconvert --to latex --template latex_nrpy_style.tplx Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.ipynb
!pdflatex -interaction=batchmode Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.tex
!pdflatex -interaction=batchmode Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.tex
!pdflatex -interaction=batchmode Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.tex
!rm -f Tut*.out Tut*.aux Tut*.log
```

    [NbConvertApp] Converting notebook Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.ipynb to latex
    [NbConvertApp] Writing 142601 bytes to Tutorial-ETK_thorn-Interpolation_to_Spherical_Grids_multi_order.tex
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode
    This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
     restricted \write18 enabled.
    entering extended mode


These lecture notes are provided under Creative Commons Attribution license, CC-BY.
All code is subject to the terms of the FSF-approved BSD-3 license.
(c) 2018 Rachel B. Getman, Sapna Sarupria, Ulf D. Schiller

```python
from IPython.display import YouTubeVideo
```

# MSE 8900 / CHE 8450 - Multiscale Modeling

Rachel B. Getman, Sapna Sarupria, Ulf D. Schiller

Clemson University

## Lecture 3.5: Lattice Boltzmann Methods

1. The lattice Boltzmann equation
2. Fluctuating lattice Boltzmann
3. Chapman-Enskog expansion

### References
1. Sauro Succi, The Lattice Boltzmann Equation for Complex States of Flowing Matter, Oxford University Press, 2018.
2. B. Duenweg and A. J. C. Ladd, Lattice Boltzmann Simualtions of Soft Matter, Adv. Polym. Sci. 221, 89, 2009.

### From Newton to Navier-Stokes

### Kinetic Theory: Boltzmann Equation

* evolution equation for the (one-)particle distribution function
$$
\left( \frac{\partial}{\partial t} + \vec{v}\cdot\frac{\partial}{\partial\vec{r}} + \frac{\vec{F}}{m}\cdot\frac{\partial}{\partial\vec{v}} \right) f(\vec{r},\vec{v},t) = \mathcal{C}[f]
$$

* Boltzmann collision operator
$$
\mathcal{C}[f] = \int d\vec{v}_1 \int d\Omega \, \sigma(v_\text{rel},\Omega) \, v_\text{rel} \left[ f(\vec{r},\vec{V}',t) f(\vec{r},\vec{v}_1',t) - f(\vec{r},\vec{v},t) f(\vec{r},\vec{v}_1,t) \right]
$$

* Detailed balance
$$
f(\vec{r},\vec{v}_1',t) f(\vec{r},\vec{v}_2',t) = f(\vec{r},\vec{v}_1,t) f(\vec{r},\vec{v}_2,t)
$$

* Equilibrium distribution (Maxwell-Boltzmann) $f=f^\text{eq} + f^\text{neq}$
\begin{equation*}
\ln f^\text{eq} = \gamma_0 + \gamma\,\vec{v} + \gamma_4 \vec{v}^2
\end{equation*}

### Hydrodynamic Fields

* "average" of polynomials $\psi(\vec{v})$ in components of $\vec{v}$
$$\begin{aligned}
m_\psi(\vec{r},t) = \int \psi(\vec{v}) \, f(\vec{r},\vec{v},t) d\vec{v}
\end{aligned}$$

* density, momentum density, stress tensor
$$\begin{aligned}
\rho(\vec{r}, t) &= m \int f d\vec{v} \\
\vec{j}(\vec{r}, t) &= m \int \vec{v} f d\vec{v} \\
\Pi(\vec{r} ,t) &= m \int \vec{v} \otimes \vec{v} f d\vec{v}
\end{aligned}$$

### Separation of scales

* Observation: not all $m_\psi$ show up in the macroscopic equations of motion

* $\rho$, $\vec{j}$ (and $e$) are collisional invariants
\begin{equation*}
\int d\vec{r} d\vec{v} \frac{\delta m_{\rho,\vec{j},e}(f)}{\delta f} \mathcal{C}[f] = 0
\end{equation*}

* local equilibrium (Maxwell-Boltzmann) $f^\text{eq}(\rho,\vec{j},e)$

* Hydrodynamics describes variation of $\rho$ and $\vec{j}$ (and $e$) through \emph{transport} (over a macroscopic distance $\sim L$)

* all other variables relax rapidly through \emph{collisions} ($\sim \lambda$ mean free path)

* scale separation: $\epsilon \sim Kn = \frac{\lambda}{L} \ll 1$ \qquad Knudsen number $Kn=\frac{\lambda}{L}$

#### How can we exploit the scale separation?

* we are only interested in the dynamics of the slow variables up to a certain order
* the dynamics of the fast variables beyond that order is unimportant
* any set of fast variables that leaves the slow dynamics unchanged will do
* the number of degrees of freedom can be greatly reduced!
* Caveat: imperfect scale separation $\rightarrow$ fast variables can couple to slow dynamics

### Discretization a la Grad

* Truncated Hermite expansion
$$f^N(\vec{r},\vec{v},t)=\omega(\vec{v}) \sum_{n=0}^N \frac{1}{n!}\mathsf{a}^{(n)}(\vec{r},t) \mathcal{H}^{(n)}(\vec{v})$$

($a^{(0)}=\rho$, $a^{(1)}=\vec{j}$, $a^{(2)} = \Pi - \rho\mathsf{I}$, $\dots$)

* Gauss-Hermite quadrature
$$\begin{aligned}
\mathsf{a}^{(n)} &= \int \mathcal{H}^{(n)}(\vec{v}) f^N(\vec{r},\vec{v},t) \, d\vec{v}
= \sum_{i} w_i \, \frac{\mathcal{H}^{(n)}(\vec{c}_i) f^N(\vec{r},\vec{c}_i,t)}{\omega(\vec{c}_i)}\\
&= \sum \mathcal{H}^{(n)}(\vec{c}_i) f_i(\vec{r},t)
\end{aligned}$$

$$\int \omega(\vec{v}) \, p(\vec{r},\vec{v},t) \, d\vec{v} = \sum_{i} w_i \, p(\vec{r},\vec{c}_i,t)$$

* Discrete velocity model (DVM)

$$\left( \frac{\partial}{\partial t} + \vec{c}_i \cdot \frac{\partial}{\partial\vec{r}} \right) f_i = - \sum_j \Omega_{ij} (f_j - f_j^\text{eq})$$

### Quadratures

| Quadrature | LB model | $q$ | $b_q$ | $w_q$ | $\vec{c}_q$ |
|---|---|---|---|---|---|
| $E_{1,5}^{3}$ | D1Q3 | 0 | 1 | $\frac{2}{3}$  | 0 |
|               |      | 1 | 2 | $\frac{1}{6}$  | $\pm\sqrt{3}$ |
| $E_{2,5}^{9}$ | D2Q9 | 0 | 1 | $\frac{4}{9}$  | $(0,0)$ |
|               |      | 1 | 4 | $\frac{1}{9}$  | $(\pm\sqrt{3},0)$, $(0,\pm\sqrt{3})$ |
|               |      | 2 | 4 | $\frac{1}{36}$ | $(\pm\sqrt{3},\pm\sqrt{3})$ |
| $E_{3,5}^{15}$ | D3Q15 | 0 | 1 | $\frac{2}{9}$ | $(0,0,0)$ |
|                | | 1  |  6 | $\frac{1}{9}$  | $(\pm\sqrt{3},0,0)$, $(0,\pm\sqrt{3},0)$, $(0,0,\sqrt{3})$ |
|                | | 3  |  8 | $\frac{1}{72}$ | $(\pm\sqrt{3},\pm\sqrt{3},\pm\sqrt{3})$ |
| $E_{3,5}^{19}$ | D3Q19 | 0 | 1 | $\frac{1}{3}$ | $(0,0,0)$ |
|                | | 1 |  6 | $\frac{1}{18}$ | $(\pm\sqrt{3},0,0)$, $(0,\pm\sqrt{3},0)$, $(0,0,\sqrt{3})$ |
|                | | 2 | 12 | $\frac{1}{36}$ | $(\pm\sqrt{3},\pm\sqrt{3},0)$, $(\pm\sqrt{3},0,\pm\sqrt{3})$, $(0,\pm\sqrt{3},\pm\sqrt{3})$ |
| $E_{3,5}^{27}$ | D3Q27 | 0 | 1 | $\frac{8}{27}$ | $(0,0,0)$ |
|                | | 1 |  6 | $\frac{2}{27}$ | $(\pm\sqrt{3},0,0)$, $(0,\pm\sqrt{3},0)$, $(0,0,\sqrt{3})$ |
|                | | 2 | 12 | $\frac{1}{54}$ | $(\pm\sqrt{3},\pm\sqrt{3},0)$, $(\pm\sqrt{3},0,\pm\sqrt{3})$, $(0,\pm\sqrt{3},\pm\sqrt{3})$ |
|                | | 3 |  8 | $\frac{1}{216}$ | $(\pm\sqrt{3},\pm\sqrt{3},\pm\sqrt{3})$

Notation $E^n_{D,d}$: $D$ dimensions, $d$ degree, $n$ abscissae
$q$: neighbor shell, $b_q$: number of neighbors, $w_q$ weights, $\vec{c}_q$ velocities

\begin{equation*}
\mathsf{T}^{(n)} = \sum_i w_i \vec{c}_i \dots \vec{c}_i =
\begin{cases}
0 & n \text{ odd} \\
{\delta}^{(n)} & n \text{ even}
\end{cases}, \quad \forall n \leq d.
\end{equation*}

### Models with polynomial equilibrium

* Ansatz: expansion in the velocities $\vec{u}$ (Euler stress is quadratic in $\vec{u}$)
\begin{equation*}
f_i^\text{eq}(\rho,\vec{u}) = w_i \rho \left[ 1 + A \vec{u}\cdot\vec{c}_i + B (\vec{u}\cdot\vec{c}_i)^2 + C\vec{u}^2 \right]
\end{equation*}

* cubic symmetry of lattice tensors $\mathsf{T}^{(n)}$
  \begin{alignat*}{2}
    &\sum_i w_i = 1 &
    &\sum_i w_i c_{i \alpha} = 0 \\
    &\sum_i w_i c_{i \alpha} c_{i \beta}
       =  \sigma_2 \, \delta_{\alpha \beta} &
    &\sum_i w_i c_{i \alpha} c_{i \beta} c_{i \gamma} = 0 \\
    &\sum_i w_i c_{i \alpha} c_{i \beta} c_{i \gamma} c_{i \delta}
      =  \kappa_4 \, \delta_{\alpha \beta \gamma \delta} &&
      + \sigma_4  \left(
          \delta_{\alpha \beta}  \delta_{\gamma \delta}
        + \delta_{\alpha \gamma} \delta_{\beta \delta}
        + \delta_{\alpha \delta} \delta_{\beta \gamma}
        \right)
\end{alignat*}

* at least three shells required to satisfy the conditions
\begin{alignat*}{4}
\sum_i w_i &= 1 & \qquad
\kappa_4 &= 0 & \qquad
\sigma_4 &= \sigma_2^2 & \qquad
c_s^2 &= \sigma_2
\end{alignat*}

### The D3Q19 model

* Equilibrium distribution:
\begin{equation*}
f_i^\text{eq}(\rho,\vec{u}) = w_i \rho \left[ 1 + \frac{\vec{u}\cdot\vec{c}_i}{c_s^2} + \frac{\vec{u}\vec{u} \colon (\vec{c}_i\vec{c}_i - c_s^2 \mathsf{I})}{2c_s^4} \right]
\end{equation*}

* Moments:
\begin{align*}
\sum_i f_i^\text{eq} &= \rho \\
\sum_i f_i^\text{eq}\vec{c}_i &= \rho\vec{u} \\
\sum_i f_i^\text{eq}\vec{c}_i\vec{c}_i &= \rho c_s^2 \mathsf{I} + \rho\vec{u}\vec{u}
\end{align*}

* Weight coefficients:
\begin{equation*}
w_i =1/3 \quad \text{for } |\vec{c}_i|=0, \qquad
w_i = 1/18 \quad \text{for } |\vec{c}_i|=1, \qquad
w_i = 1/36 \quad \text{for } |\vec{c}_i|=\sqrt{2}
\end{equation*}
* Speed of sound: $c_s = \frac{1}{\sqrt3}\left(\frac{a}{h}\right)$

### The lattice Boltzmann equation

* recall the Boltzmann equation with linearised collisions

\begin{equation*}
\left( \frac{\partial}{\partial t} + \vec{v} \cdot \frac{\partial}{\partial\vec{r}} \right) f(\vec{r},\vec{v},t) = - \Omega \left[ f(\vec{r},\vec{v},t) - f^{\text{eq}}(\vec{v}) \right]
\end{equation*}

* systematic discretization $\rightarrow$ lattice Boltzmann equation

\begin{equation*}
\bar f_i(\vec{r}+h\vec{c}_i,t+h) = \bar f_i^*(\vec{r},t) = \bar f_i(\vec{r},t) + \sum_j \Lambda_{ij} \left[ \bar f_j(\vec{r},t) - f_j^{\text{eq}}(\rho,\vec{u}) \right]
\end{equation*}

* But this is only first-order in time!?

### Splitting of the discrete Boltzmann equation

* Discrete velocity model (now with force term)

\begin{equation*}
%\frac{df_i}{dt} + {\lambda} f_i = {\lambda} f_i^\text{eq}.
\frac{\partial}{\partial t} f_i = - \vec{c}_i \cdot \frac{\partial}{\partial\vec{r}} f_i - \sum_j \Omega_{ij} (f_j - f_j^\text{eq}) + G_i = \left( \mathcal{S} + \mathcal{C} + \mathcal{F} \right) f_i
\end{equation*}

* Formal solution

$$f_i(t+h) = \exp\left[ {h(\mathcal{S} + \mathcal{C} + \mathcal{F})} \right] \, f_i(t)$$

* Baker-Campbell-Hausdorff (for sufficiently differentiable operators)

\begin{equation*}
\exp \left[ h(\mathcal{S} + \mathcal{C} + \mathcal{F}) \right] = e^{ \frac{h}{2} \left( \mathcal{C} + \mathcal{F} \right) } e^{  h \mathcal{S} } e^{ \frac{h}{2} \left( \mathcal{C} + \mathcal{F} \right) } + \mathcal{O}(h^3) \approx \mathsf{C}^\frac12 \mathsf{S} \mathsf{C}^\frac12
\end{equation*}

* Second-order in time lattice Boltzmannn update

\begin{equation*}
\mathbf{f}(t+h) = \mathsf{C}^\frac12 \mathsf{S} \mathsf{C}^\frac12 \, \mathbf{f}(t)
\end{equation*}

P. Dellar, Comp. Math. App. 65, 129-141 (2013)
UDS, Comp. Phys. Comm. 185, 2586-2597 (2014)

### Splitting of the discrete Boltzmann equation

* Streaming (Integration along characteristic)

\begin{equation*}
f_i(\vec{r}+\vec{c}_i,t+h) = f_i(\vec{r},t)
\end{equation*}

* Collisions (Crank-Nicolson rule)

\begin{equation*}
  \mathbf{f}^\text{nq}(t+h) = \left( \mathsf{I} + \frac{h}{2} \mathsf{\Omega} \right)^{-1}
  \left( \mathsf{I} - \frac{h}{2} \mathsf{\Omega} \right) \mathbf{f}^\text{nq}(t) + \mathcal{O}((h/\tau)^3)
\end{equation*}

* The error $\mathcal{O}((h/\tau)^3)$ can cause non-linear instabilities!

* Why not exact solution?

\begin{equation*}
  \mathbf{f}^\text{nq}(t+h) = \exp [ - h \mathsf{\Omega} ] \, \mathbf{f}^\text{nq}(t)
\end{equation*}

* in practice less accurate!

P. Dellar, Comp. Math. App. 65, 129-141 (2013)
Brownlee et al., Phys. Rev. E 75, 036711 (2007)

### The classical LB algorithm

1. streaming step: move $f_i^*(\vec{r},t)$ along $\vec{c}_i$ to the next lattice site, increment $t$ by $h$

$$
f_i(\vec{r}+h\vec{c}_i,t+h) = f_i^*(\vec{r},t)
$$

2. collision step: apply $\Lambda_{ij}$ and compute the post-collisional $f_i^*(\vec{r},t)$ on every lattice site

$$
f_i^*(\vec{r},t) = f(\vec{r},t) + \sum_j \Lambda_{ij} \left[ f_j(\vec{r},t) - f_j^\text{eq}(\rho,\vec{u}) \right]
$$

### Macroscopic moments in lattice Boltzmann

* macroscopic fields are velocity moments of the populations

\begin{align*}
\rho &= \sum_i f_i & \rho\vec{u} &= \sum_i f_i\vec{c}_i+\frac{h}{2}\mathbf{f} & \mathsf{\Pi} &= \sum_i \frac{f_i+f_i^*}{2} \vec{c}_i\otimes\vec{c}_i
\end{align*}

* construct orthogonal basis $e_{ki}$ for moments (recall $\psi(\vec{v})$ and $m_\psi$)
\begin{equation*}
m_k = \sum_i e_{ki} f_i
\end{equation*}

$0 \leq k \leq 9$: hydrodynamic modes (slow), $k\geq10$: kinetic modes (fast)

* collision matrix is diagonal in mode space
\begin{equation*}
\Lambda (\mathbf{f} - \mathbf{f}^\text{eq}) = \mathsf{M}^{-1} {\left( \mathsf{M}\Lambda\mathsf{M}^{-1} \right)}{ \mathsf{M} (\mathbf{f} - \mathbf{f}^\text{eq})} = \mathsf{M}^{-1} {\hat\Lambda}{ (\mathbf{m}-\mathbf{m}^\text{eq}) }
\end{equation*}

* MRT model
\begin{align*}
(m_k - m_k^\text{eq})^* = \gamma_k (m_k - m_k^\text{eq})
\end{align*}

### Choice of the moment basis

\begin{align*}
& m_0 = \rho = \sum_i f_i       && \text{mass}\\
& m_1 = j_x = \sum_i f_i c_{ix} && \text{momentum $x$}\\
& m_2 = j_y = \sum_i f_i c_{iy} && \text{momentum $y$}\\
& m_3 = j_z = \sum_i f_i c_{iz} && \text{momentum $z$}\\
& m_4 = \text{tr}(\mathsf{\Pi})             && \text{bulk stress}\\
& m_5, \ldots, m_9  \simeq \overline{\mathsf{\Pi}} && \text{shear stresses}\\
& m_{10}, \ldots, m_{18}        && \text{"kinetic modes", "ghost modes"}\\
\end{align*}

### Multiple relaxation time model (MRT)

\begin{align*}
&\gamma_0 = \gamma_1 = \gamma_2 = \gamma_3 = 0 \quad \text{mass and momentum conservation} \\
&\gamma_4 = \gamma_b \quad \text{bulk stress} \\
&\gamma_5 = \ldots = \gamma_9 = \gamma_s \quad \text{shear stress} \\
&\gamma_{10} = \ldots = \gamma_{18} = 0 \quad \text{simplest choice, careful with boundaries!}
\end{align*}

* Remark: we could also relax the populations directly:
\begin{equation*}
f_i^{\text{nq}*} = \sum_j \Lambda_{ij} f_j^\text{nq}
\end{equation*}
* simplest choice $\Lambda_{ij}=\lambda^{-1}\delta_{ij}$ $\rightarrow$ lattice BGK
* not a particularly good choice (less stable, less accurate)

### Lattice moments vs. hydrodynamic moments

* Concatenation of lattice Boltzmann updates

\begin{equation*}
\mathbf{f}(t+N h) = \mathsf{C}^\frac12 \left( \mathsf{S} \mathsf{C} \right)^N \mathsf{C}^{-\frac{1}{2}} \, \mathbf{f}(t)
\end{equation*}

* density
$$
\rho = \sum_i ( \mathsf{C}^\frac12 \mathbf{f} )_i = \sum_i f_i
$$

* momentum density
$$
\vec{j} = \sum_i ( \mathsf{C}^\frac12 \mathbf{f} )_i \vec{c}_i
= \sum_i f_i \vec{c}_i + \frac{h}{2} \mathbf{f}
$$

* viscous stress

\begin{equation*}
\begin{split}
\sigma &= - \sum_i ( \mathsf{C}^\frac12 \mathbf{f}^\text{nq} )_i \vec{c}_i\vec{c}_i
%= \mathsf{C}^\frac12 \Pi^\text{nq}  \\
%&\approx \frac{1}{2} \left( \1 + \mathsf{C} \right) \Pi^\text{nq}
= - \left( \mathsf{I}+ \frac{h}{2} \Omega \right)^{-1} \sum_i f^\text{nq}_i \vec{c}_i\vec{c}_i %\Pi^\text{nq}
\end{split}
\end{equation*}

* {Note: if $\mathsf{C}^\frac12$ includes fluctuations, it may not be invertible!}

### Viscous stress relaxation

$$ {\Pi} = \overline{{\Pi}} + \frac{1}{3} \text{tr}({\Pi}) \mathsf{I}$$

* recall: collision step applies linear relaxation to the moments

\begin{eqnarray*}
\overline{{\Pi}}^{*\text{neq}} &=& \gamma_s \overline{{\Pi}}^{\text{neq}} \\
\text{tr}({\Pi}^{*\text{neq}}) &=& \gamma_b \text{tr}({\Pi}^{\text{neq}})
\end{eqnarray*}

* Chapman-Enskog expansion

\begin{equation*}
-\frac{1}{2} \left ( \mathsf{\Pi}^{*\text{neq}} + \mathsf{\Pi}^\text{neq} \right) = \sigma = \eta \left( \nabla\vec{u} + (\nabla\vec{u})^t \right) + \left( \zeta - \frac{2}{3} \eta \right) (\nabla\cdot\vec{u}) \mathsf{I}
\end{equation*}

* shear and bulk viscosities are determined by the relaxation parameters

\begin{align*}
\eta &= \frac{\rho c_s^2 h}{2} \frac{1+\gamma_s}{1-\gamma_s}& \eta_b &= \frac{\rho c_s^2 h}{3} \frac{1+\gamma_b}{1-\gamma_b}
\end{align*}

### Viscosity of the lattice Boltzmann fluid

* incompressible Navier-Stokes equation is recovered

* $-1 \leq \gamma_s \leq 1$ $\Leftrightarrow$ positive viscosities

* any viscosity value is accessible

### The force term in LB

* force term $G_i$ in the discrete velocity model has the same
  first moments as the acceleration term $-\frac{1}{\rho} \vec{F} \cdot
  \nabla_\vec{c} f$

\begin{align*}\label{eq:force-moments}
\sum_i G_i &= 0, & \sum_i \vec{c}_i G_i &= \vec{F}, & \sum_i \vec{c}_i \vec{c}_i G_i &= \vec{F} \vec{u} + \vec{u} \vec{F},
\end{align*}

* second-order accuracy requires to transform the force term

\begin{equation*}
\bar{\vec{G}} = \left( \mathsf{I} + \frac{h}{2} \Omega \right)^{-1} h \Omega \, \vec{G} = \left( \mathsf{I} - \frac12 \Lambda \right) \vec{G}
\end{equation*}

* leads to a self-consistency problem when forces are velocity dependent

* collide-stream-collide scheme

### Units in LB

* grid spacing $a$, time step $h$, particle mass $m_p$

* these merely control the _accuracy_ and _stability_ of LB!

* physically relevant: mass density $\rho$, viscosity $\eta$, temperature $k_BT$

* recall:

\begin{align*}
c_s^2 &= \frac{1}{3} \frac{a^2}{h^2} = \hat{c}_s^2 \frac{{a^2}}{{h^2}}\\
\eta &= \frac{\rho c_s^2 h}{2} \frac{1+\gamma_s}{1-\gamma_s} = \hat{\rho} \hat{c}_s^2 \hat{\eta} \frac{m_p}{{a}{h}} \\
k_BT &= m_p c_s^2 = m_p \hat{c}_s^2 \frac{{a^2}}{{h^2}}
\end{align*}

* always make sure you are simulating the right _physics_!

* for comparison with experiments: match dimensionless numbers!
($Ma$, $Re$, $Pe$, $Sc$, $Kn$, $Pr$, $Wi$, $De$, ...)

### Dimensionless numbers and LB parameters

* connection between dimensionless numbers and simulation parameters

\begin{equation*}
\frac{1+\gamma}{1-\gamma} = \frac{2\nu}{c_s^2 h} = 2 \sqrt{3} \hat{L} \frac{Ma}{Re} > 0
\end{equation*}

* Knudsen number

\begin{equation*}
Kn\propto\frac{Ma}{Re} = \frac{\eta}{\rho c_s L} = \frac{c_s h}{2 L} \frac{1 + \gamma}{1 - \gamma} = \frac{1}{\sqrt{3} \hat L} \left( \frac{1}{\lambda} - \frac{1}{2} \right)
\end{equation*}

* diffusive scaling: $a \sim \epsilon L$ \quad $h \sim \epsilon^2 T$

\begin{align*}
\frac{{a^2}}{{h}} &= \text{const.} &
\frac{{a}}{{h}} \sim \frac{1}{\epsilon}\frac{L}{T} \underset{\epsilon\rightarrow 0}{\rightarrow} \infty
\end{align*}

* $Ma \rightarrow 0$ at fixed Reynolds number

* Mach number annealing: $h \sim \epsilon T$

* run simulation with higher Mach number until transients have decayed

### Thermal fluctuations

* so far the LB model is athermal and entirely deterministic

* for Brownian motion, we need fluctuations!

#### Do we need fluctuations?

* Ideal gas, temp. $T$, particle mass $m_p$, sound speed $c_s$:

$$  k_B T = m_p c_s^2$$

* $c_s \sim a / h$ ($a$ lattice spacing, $h$ time step)
* $c_s$ as small as possible

* Example (water)

|||
|---|---|
| sound speed realistic:  | $1.5 \times 10^3 m / s$ |
| sound speed artificial: | $c_s = 10^2 m / s$      |
| temperature             | $T \approx 300 K$, $k_B T = 4 \times 10^{-21}$ |
| particle mass:          | $m_p = 4 \times 10^{-25} kg$ |

|   | macroscopic scale | molecular scale |
|---|---|---|
| lattice spacing    | $a = 1 mm$             | $a = 1 nm$             |
| time step          | $h = 10^{-5} s$        | $h = 10^{-11} s$       |
| mass of a site     | $m_a = 10^{-6} kg$     | $m_a = 10^{-24} kg$    |
| "Boltzmann number" | $Bo = (m_p/m_a)^{1/2}$ | $Bo = (m_p/m_a)^{1/2}$ |
|                    | $ = 6 \times 10^{-10}$ | $ = 0.6$               |

### Low Mach number physics

* LB requires $u \ll c_i$ hence $u \ll c_s$
* low Mach number $Ma=u/c_s \ll 1$
* compressibility does not matter
* equation of state does not matter $\rightarrow$ choose ideal gas!

$m_p$ particle mass

\begin{equation*}
  p = \frac{\rho}{m_p} k_B T
\end{equation*}

\begin{equation*}
  c_s^2 = \frac{\partial p}{\partial \rho} = \frac{1}{m_p} k_B T
\end{equation*}

\begin{equation*}
  p = \rho c_s^2
\end{equation*}

\begin{equation*}
  k_B T = m_p c_s^2
\end{equation*}

### Generalized lattice gas model (GLG)

* consider integer population numbers ($m_p$ mass of an LB particle)

\begin{equation*}
\nu_i = \frac{f_i}{\mu} \qquad \mu=\frac{m_p}{a^3} \qquad \mu \nu_i = {w_i}\rho
\end{equation*}

* each lattice site in contact with a heat bath

* Poisson + constraints

\begin{equation*}
     P \left( \left\{ \nu_i \right\} \right)
     \propto
     \prod_i
     \frac{ {\bar \nu}_i^{\nu_i} }{\nu_i !} e^{-{\bar \nu}_i}
     \delta \left( \mu \sum_i \nu_i - \rho \right)
     \delta \left( \mu \sum_i \nu_i \vec c_i - \vec j \right)
\end{equation*}

[B. Duenweg, UDS, A. J. C. Ladd, Phys. Rev. E 76, 036704 (2007)]

### Entropy

* associated entropy
\begin{equation*}
    P % \left( \left\{ \nu_i \right\} \right)
    \propto
    \exp \left[S \left( \left\{ \nu_i \right\} \right)\right]
    \delta \left( \mu \sum_i \nu_i - \rho \right)
    \delta \left( \mu \sum_i \nu_i \vec c_i - \vec j \right)
\end{equation*}

* Stirling: $\nu_i! = \exp \left( \nu_i \ln \nu_i - \nu_i \right)$
\begin{align*}
    S \left( \left\{ \nu_i \right\} \right)
    &= - \sum_i \left( \nu_i \ln \nu_i -\nu_i - \nu_i \ln {\bar \nu}_i
    + {\bar \nu}_i \right) \\
    &= \frac{1}{\mu} \sum_i \rho w_i \left( \frac{f_i}{\rho w_i}-\frac{f_i}{\rho w_i} \ln\frac{f_i}{\rho w_i} - 1 \right)
\end{align*}

* $\mu$ controls the mean square fluctuations (degree of coarse-graining)

### Maximum entropy principle

* maximize entropy $S$ subject to constraints for mass and momentum conservation

\begin{equation*}
\frac{\partial S}{\partial \nu_i} + \chi
+ \lambda \cdot \vec c_i  =  0 \qquad
\mu \sum_i \nu_i - \rho  =  0 \qquad
\mu \sum_i \nu_i \vec c_i - \vec{j}  =  0
\end{equation*}

* formal solution

\begin{equation*}
f_i^{\text{eq}} = w_i \, \rho \exp \left( \chi + \lambda \cdot \vec{c}_i \right)
\end{equation*}

* expansion up to $\mathcal{O}(u^2)$

\begin{equation*}
f_i^\text{eq}(\rho,\vec{u}) = w_i \rho \left[ 1 + \frac{\vec{u}\cdot\vec{c}_i}{c_s^2} + \frac{\vec{u}\vec{u} \colon (\vec{c}_i\vec{c}_i - c_s^2 \mathsf{I})}{2c_s^4} \right]
\end{equation*}

### Fluctuations around equilibrium

* Gauss distribution for non-equilibrium part
\begin{equation*}
    P % \left( \left\{ f_i^{neq} \right\} \right)
    \propto
    \exp \left( -  \sum_i \frac{ \left( f_i^{\text{nq}} \right)^2 }
                               { 2 \mu \rho w_i } \right)
    \delta \left( \sum_i f_i^{\text{nq}} \right)
    \delta \left( \sum_i \vec c_i \, f_i^{\text{nq}} \right)
\end{equation*}

* transform to the modes ($b_k = \sum_i w_i e_{ki}^2$, Basis $e_{ki}$)
\begin{equation*}
    P\left( \left\{ {m}_k^{\text{nq}} \right\} \right)
    \propto \exp \left( - \sum_{k \ge 4}
    \frac{\left( {m}_k^{\text{nq}} \right)^2}{2 \mu \rho b_k} \right)
\end{equation*}

* more convenient: ortho-_normal_ modes
\begin{equation*}
    \hat{m}_k  =  \sum_i \hat{e}_{ki}
    \frac{f_i}{\sqrt{w_i \mu \rho}}
\end{equation*}

### Implementation of the fluctuations

* introduce stochastic term into the collision step
\begin{equation*}
    {m}_k^{* \text{nq}} = \gamma_k {m}_k^{\text{nq}} + \varphi_k r_k
\end{equation*}
$r_k$ random number from normal distribution

* ensure detailed balance (like in Monte-Carlo)
\begin{equation*}
\frac{p(m \rightarrow m^*)}{p(m^* \rightarrow m)} = \frac{\exp(-m^{*2}/2)}{\exp(-m^2/2)} \qquad \Rightarrow \qquad {\varphi_k = \sqrt{\mu\rho b_k \left( 1 - \gamma_k^2 \right)}}
\end{equation*}

* $\varphi_k \neq 0$ for \emph{all} non-conserved modes
* all modes have to be thermalized

[A. J. C. Ladd, J. Fluid Mech. 271, 285-309 (1994)]
[Adhikari et al., Europhys. Lett. 71, 473-479 (2005)]
[B. Duenweg, UDS, A. J. C. Ladd, Phys. Rev. E 76, 036704 (2007)]

### Eliminating fast variables: Chapman-Enskog

* introduce length and time scales
\begin{alignat*}{3}
&\text{coarse-grained length:} &\quad& \vec{r}_1 = \epsilon \vec{r} &\quad\rightarrow\quad& \frac{\partial}{\partial\vec{r}}=\epsilon\frac{\partial}{\partial\vec{r}_1} \\
&\text{convective time scale:} &\quad& t_1 = \epsilon t \\
&\text{diffusive time scale:}  &\quad& t_2 = \epsilon^2 t &\quad\rightarrow\quad& \frac{\partial}{\partial t} = \epsilon \frac{\partial}{\partial t_1} + \epsilon^2 \frac{\partial}{\partial t_2}
\end{alignat*}

* use $\epsilon$ as a perturbation parameter and expand $f$
\begin{eqnarray*}
f &=& f^{(0)} + \epsilon f^{(1)} + \epsilon^2 f^{(2)} + \dots \\
\mathcal{C}[f] &=& \mathcal{C}[f^{(0)}] + \epsilon \int d\vec{r} d\vec{v} \frac{\delta C[f]}{\delta f} f^{(1)}(\vec{r},\vec{v}) + \dots
\end{eqnarray*}

* solve for each order in $\epsilon$

### Chapman-Enskog expansion

* $\epsilon^0$: yields the collisional invariants, and the equilibrium distribution $f^{(0)}=f^\text{eq}$

* $\epsilon^1$: yields the Euler equations, and the first order correction $f^{(1)}$

\begin{equation}
\tag{*}
\left( \frac{\partial}{\partial t_1} + \vec{c}\frac{\partial}{\partial\vec{r}_1} \right) f^\text{(0)} = \int d\vec{r}' d\vec{c}' \frac{\delta C[f^{(0)}]}{\delta f^{(0)}} f^{(1)}(\vec{r}',\vec{c}')
\end{equation}

* $\epsilon^2$: adds viscous terms to the Euler equation

* Navier-Stokes!

* the "only" difficulty is: no explicit solution of (*) is known...\\ (except for Maxwell molecules)

### Chapman-Enskog expansion for LB

* original LBE
\begin{equation*}
f_i(\vec{r}+h\vec{c}_i,t+h) - f_i(\vec{r},t) = \delta_i
\end{equation*}

* recall: coarse-grained length $\vec{r}_1$, convective time scale $t_1$, diffusive time scale~$t_2$

\begin{equation*}
f_i(\vec{r}_1+\epsilon h\vec{c}_i,t_1+\epsilon h,t_2+\epsilon^2h) - f_i(\vec{r}_1,t_1,t_2) = \delta_i
\end{equation*}

* Taylor expansion:

\begin{equation*}
\epsilon h \left( \frac{\partial}{\partial t_1} + \vec{c}_i\cdot\frac{\partial}{\partial\vec{r}_1} \right) f_i + \epsilon^2h \left[ \frac{\partial}{\partial t_2} + \frac{h}{2} \left( \frac{\partial}{\partial t_1} + \vec{c}_i\cdot\frac{\partial}{\partial\vec{r}_1}\right) \right] f_i = \delta_i
\end{equation*}

###  Chapman-Enskog expansion for LB

* expand $f_i$ and $\Delta_i$ in powers of $\epsilon$
\begin{eqnarray*}
f_i &=& f_i^{(0)} + \epsilon f_i^{(1)} + \epsilon^2 f_i^{(2)} + \dots \\
\Delta_i &=& \Delta_i^{(0)} + \epsilon \Delta_i^{(1)} + \dots
%&=& \Delta_i^{(0)} + \epsilon \sum_j \textcolor{blue}{\left. \frac{\partial \Delta_i}{\partial f_j}\right|_{f_j=f_j^{(0)}}} f_j^{(1)} + \dots \\
%&=:& \epsilon \sum_j \textcolor{blue}{\Lambda_{ij}} f_j^{(1)}
\end{eqnarray*}

* hierarchy of equations at different powers of $\epsilon$
\begin{align*}
\mathcal{O}(\epsilon^0): \quad & \Delta_i^{(0)} = 0 \\
\mathcal{O}(\epsilon^1): \quad & \left( \frac{\partial}{\partial t_1} + \vec{c}_i\cdot\frac{\partial}{\partial\vec{r}_1} \right) f_i^{(0)} = \frac{1}{h} \Delta_i^{(1)} \\
\mathcal{O}(\epsilon^2): \quad &  \left[ \frac{\partial}{\partial t_2} + \frac{h}{2} \left(  \frac{\partial}{\partial t_1} + \vec{c}_i\cdot\frac{\partial}{\partial\vec{r}_1} \right)^2 \right]  f_i^{(0)}  +  \left( \frac{\partial}{\partial t_1} + \vec{c}_i\cdot\frac{\partial}{\partial\vec{r}_1} \right) f_i^{(1)} = \frac{1}{h} \Delta_i^{(2)}
\end{align*}

### Zeroth order $\epsilon^0$

* no expansion for conserved quantities!
\begin{eqnarray*}
f_i^{(0)} &=& f_i^\text{eq} \\
\rho^{(0)} &=& \rho = \sum_i f_i^\text{eq} \\
\vec{j}^{(0)} &=& \vec{j} = \sum_i f_i^\text{eq} \vec{c}_i
\end{eqnarray*}

* linear part of collision operator
\begin{eqnarray*}
\Delta_i &=& \epsilon \Delta_i^{(1)} = \epsilon \sum_j {\left. \frac{\partial \Delta_i}{\partial f_j} \right|_{f^{(0)}}} f_j^{(1)} = \sum_j {\Lambda_{ij}} f_j^{(1)}
\end{eqnarray*}

### Equations for the mass density

*
\begin{align*}
&\frac{\partial}{\partial t_1} \rho + \frac{\partial}{\partial\vec{r}_1} \cdot \vec{j} = 0 \\
&\frac{\partial}{\partial t_2} \rho = 0
\end{align*}

* continuity equation holds!

### Equations for the momentum density

*
\begin{align*}
&\frac{\partial}{\partial t_1} \vec{j} + \frac{\partial}{\partial\vec{r}_1} \cdot {\mathsf{\Pi}^{(0)}} = 0 \\
&\frac{\partial}{\partial t_2} \vec{j} + \frac{1}{2} \frac{\partial}{\partial\vec{r}_1} \cdot \left( {\mathsf{\Pi}^{*(1)} + \mathsf{\Pi}^{(1)}} \right) = 0
\end{align*}

* Euler stress
\begin{equation*}
{\Pi}^{(0)} = \rho c_s^2 \mathsf{I} + \rho \vec{u}\vec{u} = \mathsf{\Pi}^\text{eq}
\end{equation*}

* Newtonian viscous stress
\begin{equation*}
\frac{\epsilon}{2} \left( \mathsf{\Pi}^{*(1)} + \mathsf{\Pi}^{(1)} \right) = - \mathsf{\Pi}^\text{visc}
\end{equation*}

* incompressible Navier-Stokes equation holds!

### Diggin' deeper: Third moment

* the third moment ${\Phi}^{(0)} = \sum_i f_i^{(0)} \vec{c}_i\vec{c}_i\vec{c}_i$ enters through its equilibrium part!
\begin{equation*}
\frac{\partial}{\partial t_1} \mathsf{\Pi}^{(0)} + \frac{\partial}{\partial\vec{r}_1} \cdot {{\Phi}^{(0)}} = \frac{1}{h} \sum_i \Delta_i^{(1)} \vec{c}_i \vec{c}_i = \frac{1}{h} \left( \mathsf{\Pi}^{*(1)} - \mathsf{\Pi}^{(1)} \right)
\end{equation*}

* explicit form
\begin{equation*}
{\Phi}^{(0)}_{\alpha\beta\gamma} = \rho c_s^2 \left( u_\alpha \delta_{\beta\gamma} + u_\beta\delta_{\alpha\gamma} + u_\gamma\delta_{\alpha\beta} \right)
\end{equation*}

* from continuity and Euler equation
\begin{equation*}
\frac{\partial}{\partial t_1} \mathsf{\Pi}^{(0)} = \frac{\partial}{\partial t_1} \left( \rho c_s^2 \mathsf{I} + \rho \vec{u}\vec{u} \right) = \dots
\end{equation*}

* neglecting terms of $\mathcal{O}(u^3)$
\begin{equation*}
\mathsf{\Pi}^{*(1)} - \mathsf{\Pi}^{(1)} = \rho c_s^2 h \left( \nabla\vec{u} + (\nabla\vec{u})^t \right)
\end{equation*}

### Suitable LB models

* equilibrium values of the moments up to ${\Phi}^\text{eq}$
\begin{eqnarray*}
\rho^\text{eq}&=&\rho \\
\vec{j}^\text{eq}&=&\vec{j} \\
{\Pi}^\text{eq}&=&\rho c_s^2 \mathsf{I} + \rho \vec{u} \vec{u} \\
{\Phi}^\text{eq}_{\alpha\beta\gamma} &=& \rho c_s^2 \left( u_\alpha \delta_{\beta\gamma} + u_\beta\delta_{\alpha\gamma} + u_\gamma\delta_{\alpha\beta} \right)
\end{eqnarray*}

* collision operator
\begin{align*}
\sum_i \Delta_i &= 0 &
\sum_i \Delta_i \vec{c}_i &= 0 \\
\overline{{\Pi}}^{*\text{neq}} &= \gamma_s \overline{{\Pi}}^{\text{neq}} &
\text{tr}({\Pi}^{*\text{neq}}) &= \gamma_b \text{tr}({\Pi}^{\text{neq}})
\end{align*}

### Hands-On Activity

1. Play around with the LB code available at https://gist.github.com/uschille/8f65dd40572b2d943409.

### References

1. Schiller, U. D., Krüger, T. & Henrich, O. Mesoscopic modelling and simulation of soft matter. Soft Matter 14, 9–26 (2017).
4. Schiller, U. D. A unified operator splitting approach for multi-scale fluid–particle coupling in the lattice Boltzmann method. Comput. Phys. Commun. 185, 2586–2597 (2014).
2. Dünweg, B., Schiller, U. D. & Ladd, A. J. C. Progress in the understanding of the fluctuating lattice Boltzmann equation. Comput. Phys. Commun. 180, 605–608 (2009).
5. Dünweg, B., Schiller, U. D. & Ladd, A. J. C. Statistical mechanics of the fluctuating lattice Boltzmann equation. Phys. Rev. E 76, 036704 (2007).
3. Schiller, U. D. Thermal fluctuations and boundary conditions in the lattice Boltzmann method. (Johannes Gutenberg University Mainz, 2008).