# Quantum Integer Programming (QuIP) 47-779. Fall 2020, CMU

This notebook contains material from the Quantum Integer Programming Lecture at CMU Fall 2020 by David Bernal (bernalde at cmu.edu), Sridhar Tayur (stayur at cmu.edu) and Davide Venturelli; the content is available on **[Github](https://github.com/bernalde/QuIP)**. The text is released under the **[CC-BY-NC-ND-4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode) license, and code is released under the **[MIT license](https://opensource.org/licenses/MIT).*

Run in **[Google Colab](https://colab.research.google.com/github/bernalde/QuIP/blob/master/notebooks/Notebook%205%20-%20QUBO.ipynb)**

## Quadratic Unconstrained Binary Optimization
This notebook will explain the basics of the QUBO modeling. In order to implement the different QUBOs we will use D-Wave's packages **[dimod](https://github.com/dwavesystems/dimod)**, and then solve them using **[neal](https://github.com/dwavesystems/dwave-neal)**'s implementation of simulated annealing.
We will also leverage the use of D-Wave's package **[dwavebinarycsp](https://github.com/dwavesystems/dwavebinarycsp)** to translate constraint satisfaction problems to QUBOs. Finally, for Groebner basis computations we will use **[Sympy](https://www.sympy.org/)** for symbolic computation in Python and **[Networkx](https://networkx.github.io/)** for network models/graphs.

#### Problem statement
We define a QUBO as the following optimization problem:
$$
\min_{x \in \{0,1 \}^n} \sum_{(ij) \in E(G)} Q_{ij}x_i x_j + \sum_{i \in V(G)}Q_{ii}x_i + c_Q = \min_{x \in \{0,1 \}^n}  x^\top Q x + c_Q
$$
where we optimize over binary variables $x \in \{ 0,1 \}^n$, on a constrained graph $G(V,E)$ defined by an adjacency matrix $Q$. We also include an arbitrary offset  $c_Q$.

### Example
Suppose we want to solve the following problem via QUBO
$$
\min_{\mathbf{x}} 2𝑥_0+4𝑥_1+4𝑥_2+4𝑥_3+4𝑥_4+4𝑥_5+5𝑥_6+4𝑥_7+5𝑥_8+6𝑥_9+5𝑥_{10} \\
s.t. \begin{bmatrix}
1 & 0 & 0 & 1 & 1 & 1 & 0 & 1 & 1 & 1 & 1\\
0 & 1 & 0 & 1 & 0 & 1 & 1 & 0 & 1 & 1 & 1\\
0 & 0 & 1 & 0 & 1 & 0 & 1 & 1 & 1 & 1 & 1
\end{bmatrix}\mathbf{x}=
\begin{bmatrix}
1\\
1\\
1
\end{bmatrix} \\
\mathbf{x} \in \{0,1 \}^{11}
$$

```python
# If using this on Google collab, we need to install the packages
try:
  import google.colab
  IN_COLAB = True
except:
  IN_COLAB = False

# Let's install with dimod and neal
if IN_COLAB:
    !pip install dimod
    !pip install dwave-neal
```

```python
# Import the Dwave packages dimod and neal
import dimod
import neal
# Import Matplotlib to generate plots
import matplotlib.pyplot as plt
# Import numpy and scipy for certain numerical calculations below
import numpy as np
from scipy.special import gamma
import math
from collections import Counter
import pandas as pd
from itertools import chain
import time
import networkx as nx
```

First we would write this problem as a an unconstrained one by penalizing the linear constraints as quadratics in the objective. Let's first define the problem parameters

```python
A = np.array([[1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1],
            [0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1],
            [0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1]])
b = np.array([1, 1, 1])
c = np.array([2, 4, 4, 4, 4, 4, 5, 4, 5,6, 5])
```

In order to define the $Q$ matrix, we write the problem
$$
\min_{\mathbf{x}} \mathbf{c}^\top \mathbf{x}\\
s.t. \mathbf{A}\mathbf{x}=\mathbf{b} \\
\mathbf{x} \in \{0,1 \}^{11}
$$
as follows:
$$
\min_{\mathbf{x}} \mathbf{c}^\top \mathbf{x} + \rho(\mathbf{A}\mathbf{x}-\mathbf{b})^\top (\mathbf{A}\mathbf{x}-\mathbf{b}) \\
\mathbf{x} \in \{0,1 \}^{11}
$$
Exploting the fact that $x^2=x$ for $x \in \{0,1\}$, we can make the linear terms appear in the diagonal of the $Q$ matrix.
$$
\rho(\mathbf{A}\mathbf{x}-\mathbf{b})^\top (\mathbf{A}\mathbf{x}-\mathbf{b}) = \rho( \mathbf{x}^\top (\mathbf{A}^\top \mathbf{A}) \mathbf{x} - 2\mathbf{b}^\top \mathbf{A} \mathbf{x} + \mathbf{b}^\top \mathbf{b} )
$$
For this problem in particular, ona can prove the the penalization factor is given by $\rho > \sum_{i=1}^n |c_i|$, therefore we choose this bound + 1.

```python
rho = np.sum(np.abs(c)) + 1
Q = rho*np.matmul(A.T,A)
Q += np.diag(c)
Q -= rho*2*np.diag(np.matmul(b.T,A))
cQ = rho*np.matmul(b.T,b)
print(Q)
print(cQ)

```

    [[ -46    0    0   48   48   48    0   48   48   48   48]
     [   0  -44    0   48    0   48   48    0   48   48   48]
     [   0    0  -44    0   48    0   48   48   48   48   48]
     [  48   48    0  -92   48   96   48   48   96   96   96]
     [  48    0   48   48  -92   48   48   96   96   96   96]
     [  48   48    0   96   48  -92   48   48   96   96   96]
     [   0   48   48   48   48   48  -91   48   96   96   96]
     [  48    0   48   48   96   48   48  -92   96   96   96]
     [  48   48   48   96   96   96   96   96 -139  144  144]
     [  48   48   48   96   96   96   96   96  144 -138  144]
     [  48   48   48   96   96   96   96   96  144  144 -139]]
    144

We can visualize the graph that defines this instance using the Q matrix as the adjacency matrix of a graph.

```python
G = nx.from_numpy_matrix(Q)
nx.draw(G, with_labels=True)

```

Let's define a QUBO model and then solve it via simulated annealing.

```python
model = dimod.BinaryQuadraticModel.from_qubo(Q, offset=cQ)

```

```python
def plot_enumerate(results, title=None):

    plt.figure()

    energies = [datum.energy for datum in results.data(
        ['energy'], sorted_by=None)]

    if results.vartype == 'Vartype.BINARY':
        samples = [''.join(c for c in str(datum.sample.values()).strip(
            ', ') if c.isdigit()) for datum in results.data(['sample'], sorted_by=None)]
        plt.xlabel('bitstring for solution')
    else:
        samples = np.arange(len(energies))
        plt.xlabel('solution')

    plt.bar(samples,energies)
    plt.xticks(rotation=90)
    plt.ylabel('Energy')
    plt.title(str(title))
    print("minimum energy:", min(energies))

def plot_energies(results, title=None):
    energies = results.data_vectors['energy']
    occurrences = results.data_vectors['num_occurrences']
    counts = Counter(energies)
    total = sum(occurrences)
    counts = {}
    for index, energy in enumerate(energies):
        if energy in counts.keys():
            counts[energy] += occurrences[index]
        else:
            counts[energy] = occurrences[index]
    for key in counts:
        counts[key] /= total
    df = pd.DataFrame.from_dict(counts, orient='index').sort_index()
    df.plot(kind='bar', legend=None)

    plt.xlabel('Energy')
    plt.ylabel('Probabilities')
    plt.title(str(title))
    plt.show()
    print("minimum energy:", min(energies))
```

Let's now solve this problem using Simulated Annealing

```python
simAnnSampler = neal.SimulatedAnnealingSampler()
simAnnSamples = simAnnSampler.sample(model, num_reads=1000)

```

```python
plot_enumerate(simAnnSamples, title='Simulated annealing in default parameters')
plot_energies(simAnnSamples, title='Simulated annealing in default parameters')
```

Notice that this is the same example we have been solving earlier (via Integer Programming in the Quiz 1 and via Ising model in the Notebook 4).

## Let's go back to the slides

Let's solve the graph coloring problem that we solved via Groebner basis in the Notebook 2 now using QUBO.

$\mathcal{G}(V, E)$ has 12 vertices and 23 edges. We ask if the graph is $3$–colorable. Let’s first encode $V$ and $E$ using Python’s built–in data structures:

This second tutorial is heavily inspired in D-Wave's Map coloring of Canada found **[here](https://docs.ocean.dwavesys.com/en/stable/examples/map_coloring.html)**.

```python
# Let's install with dimod and neal
if IN_COLAB:
    !pip install dwavebinarycsp
    !pip install dwavebinarycsp[maxgap]
    !pip install dwavebinarycsp[mip]

import dwavebinarycsp
```

```python
V = range(1, 12+1)
E = [(1,2),(2,3),(1,4),(1,6),(1,12),(2,5),(2,7),(3,8),(3,10),(4,11),(4,9),(5,6),(6,7),(7,8),(8,9),(9,10),(10,11),(11,12),(5,12),(5,9),(6,10),(7,11),(8,12)]
layout = {i: [np.cos((2*i+1)*np.pi/8),np.sin((2*i+1)*np.pi/8)] for i in np.arange(5,13)}
layout[1] = [-1.5,1.5]
layout[2] = [1.5,1.5]
layout[3] = [1.5,-1.5]
layout[4] = [-1.5,-1.5]
G = nx.Graph()
G.add_edges_from(E)
nx.draw(G, with_labels=True, pos=layout)
```

```python
# Function for the constraint that two nodes with a shared edge not both select
# one color
def not_both_1(v, u):
    return not (v and u)

# Valid configurations for the constraint that each node select a single color, in this case we want to use 3 colors
one_color_configurations = {(0, 0, 1), (0, 1, 0), (1, 0, 0)}
colors = len(one_color_configurations)

# Create a binary constraint satisfaction problem
csp = dwavebinarycsp.ConstraintSatisfactionProblem(dwavebinarycsp.BINARY)

# Add constraint that each node select a single color
for node in V:
    variables = ['x'+str(node)+','+str(i) for i in range(colors)]
    csp.add_constraint(one_color_configurations, variables)

# Add constraint that each pair of nodes with a shared edge not both select one color
for edge in E:
    v, u = edge
    for i in range(colors):
        variables = ['x'+str(v)+','+str(i), 'x'+str(u)+','+str(i)]
        csp.add_constraint(not_both_1, variables)
```

Defining the Binary Quandratic model (QUBO) using the CSP library we have:

```python
bqm = dwavebinarycsp.stitch(csp)
```

```python
simAnnSamples = simAnnSampler.sample(bqm, num_reads=1000)

```

```python
plot_enumerate(simAnnSamples, title='Simulated annealing in default parameters')
plot_energies(simAnnSamples, title='Simulated annealing in default parameters')
```

Because of precision issues in the translation to BQM, we obtain very tiny coefficeints that should be zero. In any case, since this is a constraint satisfaction problem, any of the solutions with energy ~0 is a valid coloring.

```python
# Check that a good solution was found
sample = simAnnSamples.first.sample     # doctest: +SKIP
if not csp.check(sample):           # doctest: +SKIP
        print("Failed to color map. Try sampling again.")
else:
        print(sample)
```

    {'x1,0': 1, 'x1,1': 0, 'x1,2': 0, 'x10,0': 1, 'x10,1': 0, 'x10,2': 0, 'x11,0': 0, 'x11,1': 0, 'x11,2': 1, 'x12,0': 0, 'x12,1': 1, 'x12,2': 0, 'x2,0': 0, 'x2,1': 0, 'x2,2': 1, 'x3,0': 0, 'x3,1': 1, 'x3,2': 0, 'x4,0': 0, 'x4,1': 1, 'x4,2': 0, 'x5,0': 1, 'x5,1': 0, 'x5,2': 0, 'x6,0': 0, 'x6,1': 0, 'x6,2': 1, 'x7,0': 0, 'x7,1': 1, 'x7,2': 0, 'x8,0': 1, 'x8,1': 0, 'x8,2': 0, 'x9,0': 0, 'x9,1': 0, 'x9,2': 1}

```python
# Function that plots a returned sample
def plot_map(sample):
    # Translate from binary to integer color representation
    color_map = {}
    for node in V:
          for i in range(colors):
            if sample['x'+str(node)+','+str(i)]:
                color_map[node] = i
    # Plot the sample with color-coded nodes
    node_colors = [color_map.get(node) for node in G.nodes()]
    nx.draw(G, with_labels=True, pos=layout, node_color=node_colors)
    plt.show()
```

```python
plot_map(sample)
```

## Let's go back to the slides

This last tutorial implements an example of integer factorization using QUBO. This was published by Rauf Dridi and Hedayat Alghassi (both former postdocs at the CMU Quantum Computing group) in Nature
This notebook is drawn from an implementation found in **[Github](https://github.com/michael-L-i/DWave/blob/master/Factoring.ipynb)**.

The factoring problem: Given a  large number $M$, the gaol is find the two prime factors $p$ and $q$ such that
$$
M = p q.
$$
For instance, if $M=15$ then $p=3$ and $q=5$. To solve this problem on D-Wave (using quantum annealing) we have to map the problem of finding $p$ and $q$ from the integer domain to the binary domain. Let's say, we would like to factor $M=15$. First, we write
$$
 p = p_0 + 2 p_1 + 4 p_2\, \mbox{and}\, q = q_0 + 2 q_1 + 4 q_2.
$$
Now the prolem becomes: find the $p_i$ and $q_i$ such that
$$
M = (p_0 + 2 p_1 + 4 p_2) (q_0 + 2 q_1 + 4 q_2).
$$
Let's do the calculations:

```python
from sympy import *
p = IndexedBase("p")
q = IndexedBase("q")
```

```python
eq = 15 - ((p[0] + 2* p[1] + 4 *p[2]) * (q[0] + 2 *q[1] + 4 *q[2]) )
expand(eq)
```

$\displaystyle - {p}_{0} {q}_{0} - 2 {p}_{0} {q}_{1} - 4 {p}_{0} {q}_{2} - 2 {p}_{1} {q}_{0} - 4 {p}_{1} {q}_{1} - 8 {p}_{1} {q}_{2} - 4 {p}_{2} {q}_{0} - 8 {p}_{2} {q}_{1} - 16 {p}_{2} {q}_{2} + 15$

The goal is now to solve $eq =0$. As the title of this section indicates, it is possible to solve this algebraically {\it without a quantum computer} (but not efficiently). Let's explain how this before we move ahead with quantum annealing: Let's state the problem one more time:

Find $p_i$s and $q_i$s such that:

*   $eq=0$ where $eq$ is given in the above cell
*   all $p_i$s and $q_i$s are binary:
$p_i (p_i-1)=0$ and $q_i (q_i-1)=0$. So we ended up with a set of algebraic equations
$$
  \mathcal S = \{eq=0, p_0 (p_0-1)=0,p_1 (p_1-1)=0,p_2 (p_2-1)=0, q_0 (q_0-1)=0, q_1 (q_1-1)=0, q_2 (q_2-1)=0 \}.
$$
This system can be constructed as follows:

```python
S = [expand(eq)]
for i in range(3):
  S += [p[i]*(p[i]-1), q[i]*(q[i]-1)]
S
```

    [-p[0]*q[0] - 2*p[0]*q[1] - 4*p[0]*q[2] - 2*p[1]*q[0] - 4*p[1]*q[1] - 8*p[1]*q[2] - 4*p[2]*q[0] - 8*p[2]*q[1] - 16*p[2]*q[2] + 15,
     (p[0] - 1)*p[0],
     (q[0] - 1)*q[0],
     (p[1] - 1)*p[1],
     (q[1] - 1)*q[1],
     (p[2] - 1)*p[2],
     (q[2] - 1)*q[2]]

To solve the system $\mathcal S$ we use Groebner bases technique:

```python
groebner(S, p[0], p[1], p[2], q[0], q[1], q[2] , order='lex')
```

$\displaystyle GroebnerBasis\left(\left( {p}_{0} - 1, \  {p}_{1} - {q}_{2}, \  {p}_{2} + {q}_{2} - 1, \  {q}_{0} - 1, \  {q}_{1} + {q}_{2} - 1, \  {q}_{2}^{2} - {q}_{2}\right), \left( {p}_{0}, \  {p}_{1}, \  {p}_{2}, \  {q}_{0}, \  {q}_{1}, \  {q}_{2}\right)\right)$

```python
solve([p[0] - 1, p[1] - q[2], p[2] + q[2] - 1, q[0] - 1, q[1] + q[2] - 1, q[2]**2 - q[2]], p[0], p[1], p[2], q[0], q[1], q[2] )
```

    [{p[0]: 1, p[1]: 0, p[2]: 1, q[0]: 1, q[1]: 1, q[2]: 0},
     {p[0]: 1, p[1]: 1, p[2]: 0, q[0]: 1, q[1]: 0, q[2]: 1}]

From this we obtain that the factors are $p= 1*1 + 2*1 = 3$, $q = 1*1 + 4*1 = 5$.

Next we generalize this procedure:

```python
from math import log2
from math import *
```

Notice that $M=pq$ implies that $log_2(M) = log_2(p) + log_2(q)$, which means the number of bits allocated to $p$ and $q$ adds up to the number of bit in $M$ (the number of bits being $\lceil \log_2(x) \rceil$). This will guarantee we are not injecting more variables than we are suposed to.

```python
def factorBiPrime(M):
  ceiling = ceil

  p_eq =0
  q_eq = 0
  varlist = []
  S = []

  Mb = ceiling(log2(M))
  Pb = ceiling(log2(M)/2)+1
  Qb = Mb-Pb +1

  for i in range (Pb):
    p_eq += p[i]*2**i
    varlist += [p[i]]
    S += [p[i]*(p[i]-1)]

  for i in range(Qb):
    q_eq += q[i]*2**i
    varlist += [q[i]]
    S += [q[i]*(q[i]-1)]

  eq = M - (p_eq)*(q_eq)
  S += [eq]

  sol = (solve(groebner(S, varlist , order='lex'), varlist))

  for x in sol.items():
    p_eq = p_eq.subs(x[0], x[1])
    q_eq = q_eq.subs(x[0], x[1])
  return p_eq, q_eq
```

```python
factorBiPrime(77)
```

    (11, 7)

#### Solving it as a QUBO
The goal is to solve $eq=0$ using QUBO. Since we look for the minimum of the cost function, we have to map our problem of finding the zeros (solutions) of the equation $eq=0$ into finding the minimum of a certain function. We do this with the following observation:

The solutions ($pi$ and $qi$) of the equation eq =0 are exactly the minima of the cost function $Q = eq^2$. Recall
$$
eq = 15 - ((p_0 + 2 p_1) (q_0 + 2 q_1 + 4q_2) ).
$$

```python
eq = 15 - ((p[0] + 2* p[1] + 4 *p[2]) * (q[0] + 2 *q[1] + 4 *q[2]) )
```

```python
preQ = expand(eq**2)
preQ
```

$\displaystyle {p}_{0}^{2} {q}_{0}^{2} + 4 {p}_{0}^{2} {q}_{0} {q}_{1} + 8 {p}_{0}^{2} {q}_{0} {q}_{2} + 4 {p}_{0}^{2} {q}_{1}^{2} + 16 {p}_{0}^{2} {q}_{1} {q}_{2} + 16 {p}_{0}^{2} {q}_{2}^{2} + 4 {p}_{0} {p}_{1} {q}_{0}^{2} + 16 {p}_{0} {p}_{1} {q}_{0} {q}_{1} + 32 {p}_{0} {p}_{1} {q}_{0} {q}_{2} + 16 {p}_{0} {p}_{1} {q}_{1}^{2} + 64 {p}_{0} {p}_{1} {q}_{1} {q}_{2} + 64 {p}_{0} {p}_{1} {q}_{2}^{2} + 8 {p}_{0} {p}_{2} {q}_{0}^{2} + 32 {p}_{0} {p}_{2} {q}_{0} {q}_{1} + 64 {p}_{0} {p}_{2} {q}_{0} {q}_{2} + 32 {p}_{0} {p}_{2} {q}_{1}^{2} + 128 {p}_{0} {p}_{2} {q}_{1} {q}_{2} + 128 {p}_{0} {p}_{2} {q}_{2}^{2} - 30 {p}_{0} {q}_{0} - 60 {p}_{0} {q}_{1} - 120 {p}_{0} {q}_{2} + 4 {p}_{1}^{2} {q}_{0}^{2} + 16 {p}_{1}^{2} {q}_{0} {q}_{1} + 32 {p}_{1}^{2} {q}_{0} {q}_{2} + 16 {p}_{1}^{2} {q}_{1}^{2} + 64 {p}_{1}^{2} {q}_{1} {q}_{2} + 64 {p}_{1}^{2} {q}_{2}^{2} + 16 {p}_{1} {p}_{2} {q}_{0}^{2} + 64 {p}_{1} {p}_{2} {q}_{0} {q}_{1} + 128 {p}_{1} {p}_{2} {q}_{0} {q}_{2} + 64 {p}_{1} {p}_{2} {q}_{1}^{2} + 256 {p}_{1} {p}_{2} {q}_{1} {q}_{2} + 256 {p}_{1} {p}_{2} {q}_{2}^{2} - 60 {p}_{1} {q}_{0} - 120 {p}_{1} {q}_{1} - 240 {p}_{1} {q}_{2} + 16 {p}_{2}^{2} {q}_{0}^{2} + 64 {p}_{2}^{2} {q}_{0} {q}_{1} + 128 {p}_{2}^{2} {q}_{0} {q}_{2} + 64 {p}_{2}^{2} {q}_{1}^{2} + 256 {p}_{2}^{2} {q}_{1} {q}_{2} + 256 {p}_{2}^{2} {q}_{2}^{2} - 120 {p}_{2} {q}_{0} - 240 {p}_{2} {q}_{1} - 480 {p}_{2} {q}_{2} + 225$

This long expression is a multilinear in terms of binary variables, therefore we can quadratize it. The challenge is on!

```python

```


| | Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section 2.2, écoulement d'un film de fluide Newtonien sur un plan incliné.

> Dans cette section il est important de revoir les concepts de flux de quantité de mouvement vus au chapitre 1.  Voici un résumé des points qui seront utilisés dans la section 2.2:

http://pierreproulx.espaceweb.usherbrooke.ca/images/GCH200_Ch1_resume.pdf

### Film d'un fluide newtonien, section 2.2 Transport Phenomena

>Le développement fait dans Transport Phenomena sera répété ici en développant les solutions avec le calculateur formel sympy et en traçant la solution avec sympy.plot.

>>Un fluide Newtonien s'écoule sous l'effet de la gravité sur un plan incliné:

>>Le bilan fait sur le film de fluide est schématisé ainsi:

>Plusieurs des termes montrés sur la figure ci-haut sont nuls, :

$
\begin{equation*}
  \boxed{ v_x=0 \quad  v_y=0 \quad \tau_{zz}=0 \quad \tau_{yz}=0 }
  \end{equation*}
$

>et de plus, on a que:

$
\begin{equation*}
  \boxed{ \rho v_z v_z \quad  et \quad p }
  \end{equation*}
$

>ne varient pas en fonction de z, ce qui fera que

$
\begin{equation*}
  \boxed{\phi_{zz}=constante }
  \end{equation*}
$

>Dans le traitement qui sera fait dans sympy on utilisera $\phi_{zz} = C $.

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique

import sympy as sp               # pour utiliser sympy, on lui donne l'alias sp
from IPython.display import *    # pour utiliser display qui formate les équations
sp.init_printing(use_latex=True) # pour utiliser LaTeX, formattage de grande qualité

import matplotlib.pyplot as plt  # pout utiliser matplotlib, outils de représentations graphiques
```

### On pourrait aussi utiliser
> sp.init_printing(use_latex=False)
### Le résultat sera le même, mais le formattage des équations sera beaucoup moins élégant,

```python
#
# définition des variables symboliques
#
x,delta_x,L,W,rho,g,beta,mu,delta=sp.symbols('x,delta_x,L,W,rho,g,beta,mu,delta')
tau_xz = sp.symbols('tau_xz')
phi_xz = sp.symbols('phi_xz')
phi_zz = sp.symbols('phi_zz')
C1,C2  = sp.symbols('C1,C2')
```

#### On pose le bilan:

```python
# Bilan des forces
dV  = W*L*delta_x
dAx = L*W
dAz = W*delta_x
bilan = dAx*(phi_xz(x)-phi_xz(x+delta_x))+dAz*(phi_zz(0)-phi_zz(L))+dV*rho*g*sp.cos(beta)
bilan = bilan/(L*W*delta_x)
# mais phi_zz est le même en z=0 et en z=L, donc le terme de phi_zz(0) - phi_zz(L) =0
bilan = bilan.subs((phi_zz(0)-phi_zz(L)), 0)
display(bilan)
```

> *Dans le bilan obtenu ci-haut on prend la limite quand $\delta x \rightarrow 0 $ On le fait avec **sympy** mais notez bien  que ce sont des manipulations de calcul algébrique qu'on ferait facilement à la main. Suivez bien les développements en faisant les mêmes sur une feuille de papier à côté. Vous obtiendrez le même résultat, même si votre notation peut être un peu différente et si vos manipulations ne sont pas exactement les mêmes. L'utilité du calculateur symbolique deviendra de plus en plus évidente au fur et à mesure que les problèmes deviendront plus complexes. *

```python
eq1 = sp.limit(bilan, delta_x, 0)
display(eq1)
```

```python
eq1 = eq1.doit()    # fais la substitution de la variable à la limite
display(eq1)
```

```python
eq1 = eq1.subs(phi_xz(x), tau_xz(x))    # remplacer phi par tau car la convection est nulle
display(eq1)
```

On insère ensuite la loi de Newton pour remplacer $\tau_{xz}$

```python
# Loi de Newton
vz     = sp.Function('v_z')(x)
newton = -mu*sp.Derivative(vz,x)

# On la substitue dans le bilan
eq2 = sp.Eq(eq1.subs(tau_xz(x), newton))
display(eq2)
```

```python
# La solution est faite avec la fonction dsolve de sympy
eq3 = sp.dsolve(eq2, vz)
display(eq3)
```

>> Conditions aux limites,
>> en x = $\delta$

>>> ${v_{z}}{\left (\delta \right )} = C_{1} + C_{2} (\delta) - \frac{g \rho (\delta)^{2}}{2 \mu} \cos{\left (\beta \right )}$  $=0$

>> et en x=0

>>>$\frac  {{dv_{z}}}{dx} =  C_{2} - 2 \frac{g \rho (0)}{2 \mu} \cos{\left (\beta \right )}$  $=0$

> exercice facile à faire à la main, afin de nous préparer à des problèmes plus complexes à venir, voyons comment le résoudre avec **sympy**

```python
# Le rhs est la partie de droite de l'équation, c'est la partie qui nous intéresse
eq4=eq3.rhs
display('eq3 et eq4 (qui est eq3.rhs)',eq3,eq4)
"""
Pose et solution des 2 équations de conditions aux limites pour C1 et C2
    La forme générale:
    sp.solve([equation1, equation2, ...],('variable1, variable2,...'))
"""
conditions = [  sp.Eq(eq4.diff(x).subs(x, 0),   0),  #L'équation 4 dérivée, quand x=0, est égale à 0
                sp.Eq(eq4.subs(x, delta),       0) ] #l'équation 4 quand x=delta est égale à 0
display('Les deux conditions aux limites',*conditions)

constantes=sp.solve(conditions, ('C1,C2'))            # pour trouver C1 et C2
display('les constantes C1 et C2', constantes)
vz=eq4.subs(constantes)
display('Le profil est donc, en substituant C1 et C2 dans eq4', vz)
vzp=eq3.subs(constantes)
display('ou dans eq3', vzp)
```

```python
# Et le profil est obtenu.
display(vz.simplify())
#
# Qu'on peut maintenant tracer, en donnant des valeurs numériques bien choisies aux paramètres
#
dico = {'beta': sp.pi/4,
        'delta':0.001,
        'g':    9.81,
        'rho':  1000,
        'mu':   0.001}

vzmax = vz.subs(x, 0)
display(9.81*1000*0.001**2*.7/2/0.001)
vzmax = vzmax.subs(dico)

plt.rcParams['figure.figsize'] = 10,8

goptions={'title' : 'Profil de vitesse parabolique',
          'ylabel': 'V/Vmax',
          'xlabel': 'r/R'}

sp.plot(vz.subs(dico)/vzmax,(x,0,0.001),**goptions);
```

### À partir du profil de vitesses on peut calculer:

* La force que le fluide exerce sur la plaque:

$
\begin{equation*}
  \boxed{ F = -\mu \bigg [\frac {dv_z}{dx}\bigg ]_{x=\delta} WL}
  \end{equation*}
$
* Le débit volumétrique

$
\begin{equation*}
  \boxed{ Q =\int_0 ^{W} \int_0 ^{\delta} v_z dx dy }
  \end{equation*}
$

* La vitesse moyenne

$
\begin{equation*}
  \boxed{ v_{z_{moyen}} = \frac {\int_0 ^{W} \int_0 ^{\delta} v_z dx dy }
                        {\int_0 ^{W} \int_0 ^{\delta} dx dy     }}
  \end{equation*}
$

### Utilisons sympy

```python
force = -mu*vz.diff(x).subs(x, delta)*L*W
display(force)
```

>>> Quel est le poids du film de liquide?

```python
debit = sp.integrate(vz, (x, 0, delta))*W
display(debit)
```

```python
vzmoyen=debit/(delta*W)
display(vzmoyen)
```

```python
vzmax=vz.subs(x, 0)
display(vzmax)
```

```python
display(vzmoyen/vzmax)
```


# The heat equation in 1D

$$
\renewcommand{\DdQq}[2]{{\mathrm D}_{#1}{\mathrm Q}_{#2}}
\renewcommand{\drondt}{\partial_t}
\renewcommand{\drondx}{\partial_x}
\renewcommand{\drondtt}{\partial_{tt}}
\renewcommand{\drondxx}{\partial_{xx}}
\renewcommand{\drondyy}{\partial_{yy}}
\renewcommand{\dx}{\Delta x}
\renewcommand{\dt}{\Delta t}
\renewcommand{\grandO}{{\mathcal O}}
\renewcommand{\density}[2]{\,f_{#1}^{#2}}
\renewcommand{\fk}[1]{\density{#1}{\vphantom{\star}}}
\renewcommand{\fks}[1]{\density{#1}{\star}}
\renewcommand{\moment}[2]{\,m_{#1}^{#2}}
\renewcommand{\mk}[1]{\moment{#1}{\vphantom{\star}}}
\renewcommand{\mke}[1]{\moment{#1}{e}}
\renewcommand{\mks}[1]{\moment{#1}{\star}}
$$

In this tutorial, we test a very classical lattice Boltzmann scheme $\DdQq{1}{3}$ on the heat equation.

The problem reads
$$
\begin{gathered} \drondt u = \mu \drondxx u, \quad t>0, \quad x\in(0,1),\\ u(0) = u(1) = 0, \end{gathered}
$$
where $\mu$ is a constant scalar.

```python

from six.moves import range
%matplotlib inline
```

## The scheme $\DdQq{1}{3}$

The numerical simulation of this equation by a lattice Boltzmann scheme consists in the approximatation of the solution on discret points of $(0,1)$ at discret instants.

To simulate this system of equations, we use the $\DdQq{1}{3}$ scheme given by

* three velocities $v_0=0$, $v_1=1$, and $v_2=-1$, with associated distribution functions $\fk{0}$, $\fk{1}$, and $\fk{2}$,
* a space step $\dx$ and a time step $\dt$, the ration $\lambda=\dx/\dt$ is called the scheme velocity,
* three moments
  $$ \mk{0}=\sum_{i=0}^{2} \fk{i}, \quad \mk{1}= \sum_{i=0}^{2} v_i \fk{i}, \quad \mk{2}= \frac{1}{2} \sum_{i=0}^{2} v_i^2 \fk{i},$$

  and their equilibrium values $\mke{0}$, $\mke{1}$, and $\mke{2}$.
* two relaxation parameters $s_1$ and $s_2$ lying in $[0,2]$.

In order to use the formalism of the package pylbm, we introduce the three polynomials that define the moments: $P_0 = 1$, $P_1=X$, and $P_2=X^2/2$, such that
$$ \mk{k} = \sum_{i=0}^2 P_k(v_i) \fk{i}.$$

The transformation $(\fk{0}, \fk{1}, \fk{2})\mapsto(\mk{0},\mk{1}, \mk{2})$ is invertible if, and only if, the polynomials $(P_0,P_1,P_2)$ is a free set over the stencil of velocities.

The lattice Boltzmann method consists to compute the distribution functions $\fk{0}$, $\fk{1}$, and $\fk{2}$ in each point of the lattice $x$ and at each time $t^n=n\dt$.
A step of the scheme can be read as a splitting between the relaxation phase and the transport phase:

* relaxation:
$$
    \begin{aligned}\mks{1}(t,x)&=(1-s_1)\mk{1}(t,x)+s_1\mke{1}(t,x),\\ \mks{2}(t,x)&=(1-s_2)\mk{2}(t,x)+s_2\mke{2}(t,x).\end{aligned}
$$

* m2f:
$$
    \begin{aligned}\fks{0}(t,x)&\;=\mk{0}(t,x)-2\mks{2}(t,x), \\ \fks{1}(t,x)&\;=\mks{1}(t,x)/2+\mks{2}(t,x), \\ \fks{2}(t,x)&\;=-\mks{1}(t,x)/2+\mks{2}(t,x).\end{aligned}
    $$

* transport:
$$
    \begin{aligned} \fk{0}(t+\dt, x)&\;=\fks{0}(t,x), \\ \fk{1}(t+\dt, x)&\;=\fks{1}(t,x-\dx), \\ \fk{2}(t+\dt, x)&\;=\fks{2}(t,x+\dx). \end{aligned}
    $$

* f2m:
$$
    \begin{aligned}\mk{0}(t+\dt,x)&\;=\fk{0}(t+\dt,x)+\fk{1}(t+\dt,x)+\fk{2}(t+\dt,x), \\ \mk{1}(t+\dt,x)&\;=\fk{1}(t+\dt,x)-\fk{2}(t+\dt,x), \\ \mk{2}(t+\dt,x)&\;=\tfrac{1}{2}\fk{1}(t+\dt,x)+\tfrac{1}{2}\fk{2}(t+\dt,x).\end{aligned}
$$

The moment of order $0$, $\mk{0}$, being conserved during the relaxation phase,
a diffusive scaling $\dt=\dx^2$, yields to the following equivalent equation
$$\drondt\mk{0} = 2\bigl(\tfrac{1}{s_1}-\tfrac{1}{2}\bigr) \drondxx\mke{2} + \grandO(\dx^2),$$

if $\mke{1}=0$.
In order to be consistent with the heat equation, the following choice is done:
$$\mke{2}=\tfrac{1}{2}u, \qquad s_1 = \frac{2}{1+2\mu}, \qquad s_2=1.$$

## Using pylbm

pylbm uses Python dictionary to describe the simulation. In the following, we will build this dictionary step by step.

### The geometry

In pylbm, the geometry is defined by a box and a label for the boundaries.

```python
import pylbm
import numpy as np
xmin, xmax = 0., 1.
dico_geom = {
    'box': {'x': [xmin, xmax], 'label':0},
}
geom = pylbm.Geometry(dico_geom)
print(geom)
geom.visualize(viewlabel=True)
```

### The stencil

pylbm provides a class stencil that is used to define the discret velocities of the scheme. In this example, the stencil is composed by the velocities $v_0=0$, $v_1=1$ and $v_2=-1$ numbered by $[0,1,2]$.

```python
dico_sten = {
    'dim':1,
    'schemes':[{'velocities':list(range(3))}],
}
sten = pylbm.Stencil(dico_sten)
print(sten)
sten.visualize()
```

### The domain

In order to build the domain of the simulation, the dictionary should contain the space step $\dx$ and the stencils of the velocities (one for each scheme).

We construct a domain with $N=10$ points in space.

```python
N = 10
dx = (xmax-xmin)/N
dico_dom = {
    'box': {'x': [xmin, xmax], 'label':0},
    'space_step':dx,
    'schemes':[
        {
            'velocities':list(range(3)),
        }
    ],
}
dom = pylbm.Domain(dico_dom)
print(dom)
dom.visualize()
```

### The scheme

In pylbm, a simulation can be performed by using several coupled schemes. In this example, a single scheme is used and defined through a list of one single dictionary. This dictionary should contain:

* 'velocities': a list of the velocities
* 'conserved_moments': a list of the conserved moments as sympy variables
* 'polynomials': a list of the polynomials that define the moments
* 'equilibrium': a list of the equilibrium value of all the moments
* 'relaxation_parameters': a list of the relaxation parameters ($0$ for the conserved moments)
* 'init': a dictionary to initialize the conserved moments

(see the documentation for more details)

The scheme velocity could be taken to $1/\dx$ and the inital value of $u$ to

$$ u(t=0,x) = \sin(\pi x).$$

```python
import sympy as sp

def solution(x, t):
    return np.sin(np.pi*x)*np.exp(-np.pi**2*mu*t)

# parameters
mu = 1.
la = 1./dx
s1 = 2./(1+2*mu)
s2 = 1.
u, X = sp.symbols('u, X')

dico_sch = {
    'dim':1,
    'scheme_velocity':la,
    'schemes':[
        {
            'velocities':list(range(3)),
            'conserved_moments':u,
            'polynomials':[1, X, X**2/2],
            'equilibrium':[u, 0., .5*u],
            'relaxation_parameters':[0., s1, s2],
            'init':{u:(solution, (0.,))},
        }
    ],
}

sch = pylbm.Scheme(dico_sch)
print(sch)
```

    [0] WARNING  pylbm.scheme in function __init__ line 194
    The value 'space_step' is not given or wrong.
    The scheme takes default value: dx = 1.

    Scheme informations
    	 spatial dimension: dim=1
    	 number of schemes: nscheme=1
    	 number of velocities:
        Stencil.nv[0]=3
    	 velocities value:
        v[0] = (0: 0), (1: 1), (2: -1),
    	 polynomials:
        P[0] = 1, X, X**2/2,
    	 equilibria:
        EQ[0] = u, 0.0, 0.5*u,
    	 relaxation parameters:
        s[0] = 0.0, 0.666666666666667, 1.00000000000000,
    	 moments matrices
    M      = Matrix([[1, 1, 1], [0, 10.0000000000000, -10.0000000000000], [0, 50.0000000000000, 50.0000000000000]])
    M^(-1) = Matrix([[1.00000000000000, 0, -0.0200000000000000], [0, 0.0500000000000000, 0.0100000000000000], [0, -0.0500000000000000, 0.0100000000000000]])

### The simulation

A simulation is built by defining a correct dictionary.

We combine the previous dictionaries to build a simulation. In order to impose the homogeneous Dirichlet conditions in $x=0$ and $x=1$, the dictionary should contain the key 'boundary_conditions' (we use pylbm.bc.Anti_bounce_back function).

```python
dico = {
    'box':{'x':[xmin, xmax], 'label':0},
    'space_step':dx,
    'scheme_velocity':la,
    'schemes':[
        {
            'velocities':list(range(3)),
            'conserved_moments':u,
            'polynomials':[1, X, X**2/2],
            'equilibrium':[u, 0., .5*u],
            'relaxation_parameters':[0., s1, s2],
            'init':{u:(solution,(0.,))},
        }
    ],
    'boundary_conditions':{
        0:{'method':{0:pylbm.bc.AntiBounceBack,}, 'value':None},
    },
    'generator': 'numpy'
}

sol = pylbm.Simulation(dico)
print(sol)
```

    Simulation informations:
    Domain informations
    	 spatial dimension: 1
    	 space step: dx= 1.000e-01
    Scheme informations
    	 spatial dimension: dim=1
    	 number of schemes: nscheme=1
    	 number of velocities:
        Stencil.nv[0]=3
    	 velocities value:
        v[0] = (0: 0), (1: 1), (2: -1),
    	 polynomials:
        P[0] = 1, X, X**2/2,
    	 equilibria:
        EQ[0] = u, 0.0, 0.5*u,
    	 relaxation parameters:
        s[0] = 0.0, 0.666666666666667, 1.00000000000000,
    	 moments matrices
    M      = Matrix([[1, 1, 1], [0, 10.0000000000000, -10.0000000000000], [0, 50.0000000000000, 50.0000000000000]])
    M^(-1) = Matrix([[1.00000000000000, 0, -0.0200000000000000], [0, 0.0500000000000000, 0.0100000000000000], [0, -0.0500000000000000, 0.0100000000000000]])

### Run a simulation

Once the simulation is initialized, one time step can be performed by using the function one_time_step.

We compute the solution of the heat equation at $t=0.1$. And, on the same graphic, we plot the initial condition, the exact solution and the numerical solution.

```python
import numpy as np
import sympy as sp
import pylab as plt
import pylbm

u, X, LA = sp.symbols('u, X, LA')

def solution(x, t):
    return np.sin(np.pi*x)*np.exp(-np.pi**2*mu*t)

xmin, xmax = 0., 1.
N = 128
mu = 1.
Tf = .1
dx = (xmax-xmin)/N # spatial step
la = 1./dx
s1 = 2./(1+2*mu)
s2 = 1.
dico = {
    'box':{'x':[xmin,xmax], 'label':0},
    'space_step':dx,
    'scheme_velocity':la,
    'schemes':[
        {
            'velocities':list(range(3)),
            'conserved_moments':u,
            'polynomials':[1, X/LA, X**2/(2*LA**2)],
            'equilibrium':[u, 0., .5*u],
            'relaxation_parameters':[0., s1, s2],
            'init':{u:(solution,(0.,))},
        }
    ],
    'boundary_conditions':{
        0:{'method':{0:pylbm.bc.AntiBounceBack,}, 'value':None},
    },
    'parameters':{LA: la},
    'generator': 'numpy'
}

sol = pylbm.Simulation(dico)
x = sol.domain.x
y = sol.m[u]

plt.figure(1)
plt.plot(x, y,'k', label='initial')

while sol.t < 0.1:
    sol.one_time_step()

plt.plot(x, sol.m[u],'b', label=r'$D_1Q_3$')
plt.plot(x, solution(x, sol.t),'r', label='exact')
plt.title('Heat equation t={0:5.3f}'.format(sol.t))
plt.legend()
```

```python

```


# Group-level analyses
This week's lab is about group-level models, multiple comparison correction (MCC), and region-of-interest (ROI) analysis. In this notebook, we will focus on group-level models, which we'll demonstrate and explain using both FSL and Python.

We'll focus on the "summary statistics" approach again, in which we'll demonstrate how we average $c\beta$-terms across runs (in run-level analyses) and subjects (in grouplevel analyses) using the GLM. Then, we're going to show you how to test more extensive hypotheses in grouplevel models.

**What you'll learn**: after this lab, you'll ...

- understand the concept of the summary statistics approach
- be able to construct different grouplevel models (in FSL)

**Estimated time needed to complete**: 2 hours

```python
# Some imports
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

## What are group-level models?
Last week, we discussed the multilevel models and the summary statistics approach. Specifically, we focused on how data from different runs are usually (in non-fMRI contexts) are analyzed in a single multilevel GLM by "concatenating" the data. And how, in fMRI, we usually don't take this approach due to the computational burden and use the summary statistics approach, which analyzes each run separately and subsequently aggregates the data in a second, run-level GLM.

In this notebook, we will extend this idea of analyzing data from multiple levels by looking at data from multiple subjects and how to analyze this data in group-level models. We will look at two "flavors" of group-level analyses: parametric and non-parametric.

## Parametric analyses
The most often used "flavor" of fMRI (group-level) analyses are *parametric*: it assumes that the data can be modeled using specific probability distributions. For example, we assume that the results of statistical tests of parameters (i.e., $t$-values), given that null hypothesis is true, are distributed according to the Students $t$-distribution (with a particular degrees-of-freedom):

\begin{align}
t_{c\hat{\beta}} \sim \mathcal{T}(\mathrm{df})
\end{align}

where you can read the $\sim$ symbol as "is distributed as". Importantly, the validity of the computed $p$-values depends on whether the choice of distribution is appropriate. If not, you might risk inflated type 1 or type 2 errors.

The first-level and run-level GLMs that we have discussed so far are examples of parametric analyses. There are also *non-parametric* versions of the GLM that do not assume any particular form of distribution; while somewhat more computationally expensive, this is become a more and more popular alternative to (group-level) parametric analyses. Importantly, the difference between parametric and non-parametric analyses is only important for the *inference* (not the *estimation*) aspect of the (group-level) analyses.

Now, let's focus on the parametric version of group-level analyses. Basically, this amounts to doing the same thing as we did last week with the run-level analyses, but this time, the results from our run-level analyses ($c\hat{\beta}^{*}$) across different subjects will become our target ($y^{\dagger}$). Note that we will use the "dagger" ($^{\dagger}$) superscript to denote that the mathematical terms belong to the group-level model (just like the $^{*}$ superscript in our notebooks refers to terms belonging to the run-level models).

To reiterate, the results from our run-level analyses ($c\hat{\beta}^{*}$), or first-level analyses if we only have a single run, become our dependent variable in our group-level analysis ($y^{\dagger}$):

\begin{align}
y^{\dagger} = c\hat{\beta}^{*}
\end{align}

Again, the group-level represents a GLM with a particular design matrix ($\mathbf{X}^{\dagger}$) and parameters ($\beta^{\dagger}$):

\begin{align}
y^{\dagger} = \mathbf{X}^{\dagger}\beta^{\dagger} + \epsilon^{\dagger}
\end{align}

And the group-level parameters can be estimated with OLS:

\begin{align}
\hat{\beta}^{\dagger} = (\mathbf{X}^{\dagger\ T} \mathbf{X}^{\dagger})^{-1}\mathbf{X}^{\dagger}y^{\dagger}
\end{align}

As mentioned last week, the parameter estimation procedure (i.e., estimating $\hat{\beta}^{\dagger}$) is relatively straightforward, but the inference procedure depends on the specific variance approach: fixed, random, or mixed effects. If your aim is to perform inference to the population, *you should never use a fixed-effects type of analysis across subjects*, as this will typically inflate your type 1 error greatly.

That leaves us with random effects and mixed effects GLMs.

### Random effects
Let's first focus on a random effects type of analysis (which is commonly used in group-level analysis in the SPM software package, for example), as this is relatively easy to understand. For now, let's use simulated data. Suppose that we want to do a group-analysis of our run-level `flocBLOCKED` data. Specifically, we are interested in the difference between faces and places ($H_{0}: \beta_{face}^{*} = \beta_{house}^{*} = 0$, $H_{a}: \beta_{face}^{*} > \beta_{house}^{*}$). As such, we'll use the corresponding $c\hat{\beta}^{*}$ terms from our run-level analysis (i.e., the contrasts against baseline, COPE1 and COPE2) as our new target $y^{\dagger}$ as follows:

\begin{align}
y^{\dagger} = \begin{bmatrix}
c\hat{\beta}^{*}_{1, F>0} \\
c\hat{\beta}^{*}_{2, F>0} \\
\vdots \\
c\hat{\beta}^{*}_{N, F>0} \\
c\hat{\beta}^{*}_{1, P>0} \\
c\hat{\beta}^{*}_{2, P>0} \\
\vdots \\
c\hat{\beta}^{*}_{N, P>0}
\end{bmatrix}
\end{align}

For our simulation, we'll assume that we have 20 subjects ($N=20$).

    ToThink (0 points): You might think, why compute the difference contrast at the group-level, while we could have computed this already in the first-level analysis (and subsequently average this in the run-level and group-level analyses)? In fact, this is mathematically (largely) equivalent. However, most people prefer to postpone this step to the group-level analysis. Can you think of a reason why?

Before we are going to simulate the data, let's construct our design-matrix, $\mathbf{X}^{*}$. Assuming we have 20 subjects and two types of inputs ($c\hat{\beta}^{*}_{F>0}$ and $c\hat{\beta}^{*}_{P>0}$, our design matrix will be:

```python
N = 20
X = np.zeros((N * 2, 2))
X[:N, 0] = 1
X[N:, 1] = 1

print(X)
```

    ToThink (1 point): In the above design matrix, we did not specify an intercept (i.e., a vector of ones). With good reason, because our group-level GLM will crash if we did. Explain shortly why an intercept for this particular design matrix (i.e., as represented by the variable <tt>X</tt> above) is not only redundant, but will make the GLM impossible to estimate. Hint: try adding an intercept, run a GLM, and see what happens.

YOUR ANSWER HERE

We are going to simulate data for a single voxel, which shows a slight preference (in run-level $c\hat{\beta}^{*}$ estimates across subjects) for faces relative to places. Specifically, we'll assume that the voxel activates on average with $\beta^{\dagger}_{faces} = 5$ for faces, while it activates on average with $\beta^{\dagger}_{places} = 4.5$ for places:

\begin{align}
y^{*} = \mathbf{X}^{\dagger}_{faces}\cdot 5 + \mathbf{X}^{\dagger}_{places} \cdot 4.5 + \epsilon^{\dagger}
\end{align}

where the noise, $\epsilon^{\dagger}$, is normally distributed with mean 0 and a standard deviation of 1:

\begin{align}
\epsilon^{\dagger} \sim \mathcal{N}(0, 1)
\end{align}

(The symbol $\sim$ stands for "is distributed as".) We'll implement this in code below:

```python
N = 20
beta_f = 5
beta_p = 4.5
std_noise = 1
np.random.seed(42)
noise = np.random.normal(0, std_noise, size=N * 2)

y_sim = X @ np.array([beta_f, beta_p]) + noise
```

And let's plot it:

```python
plt.figure(figsize=(5, 16))
plt.plot(y_sim[:N], np.arange(y_sim.size // 2), marker='o', ms=10)
plt.plot(y_sim[N:], np.arange(y_sim.size // 2, y_sim.size), marker='o', ms=10)
plt.ylim(y_sim.size, -1)
plt.legend([r"$c\hat{\beta}^{*}_{face}$", r"$c\hat{\beta}^{*}_{place}$"], fontsize=15)
plt.ylabel("Subjects", fontsize=20)
plt.xlabel(r"$c\hat{\beta}^{*}$", fontsize=20)
plt.grid()
plt.title(r"$y^{\dagger}$ (run-level estimates)", fontsize=25)
plt.show()
```

Note that these $c\hat{\beta}^{*}$ estimates are the only thing you need for a random effects analysis! We're simply ignoring the lower-level variance terms.

    ToDo (2 points): Run a group-level random-effects GLM (using the variable <tt>X</tt> as design matrix and <tt>y_sim</tt> as dependent variable). Store the estimated parameters in a variable named <tt>gl_params</tt>. Then, compute the $t$-value for the contrast $\beta^{\dagger}_{face} > \beta^{\dagger}_{place}$ and store this in a variable named <tt>gl_tvalue</tt>.

```python
''' Implement the ToDo here. '''
from numpy.linalg import inv

# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
from niedu.tests.nii.week_6 import test_rfx_glm
test_rfx_glm(X, y_sim, gl_params, gl_tvalue)
```

### Mixed-effects (in FSL)
Proper mixed-effects models are a bit too complex to implement ourselves in Python, so we'll show you how to do it in FSL! We ran the (fixed-effects) run-level models for the `flocBLOCKED` data for you already, which we download below:

```python
import os
data_dir = os.path.join(os.path.expanduser("~"), 'NI-edu-data')

print("Downloading run-level FSL FEAT results (+- 450MB) ...")
!aws s3 sync --no-sign-request s3://openneuro.org/ds003477 {data_dir} --exclude "*" --include "derivatives/fsl/sub-*/flocBLOCKED/runlevel.gfeat/*"
print("\nDone!")
```

Within the `flocBLOCKED` directory, the first-level (`ses-1.feat` and `ses-2.feat`) and run-level results (`runlevel.gfeat`) are stored for each subject.

    ToDo (ungraded): Start a remote desktop environment, open a terminal, navigate to the <tt>runlevel.gfeat</tt> directory of the task <tt>flocBLOCKED</tt> of subject <tt>sub-03</tt> (so <tt>sub-03/flocBLOCKED/runlevel.gfeat</tt>), and print the directory's contents to the terminal.

(Note that the following ToDos will be checked later using test-cells.)

You should see that the `runlevel.gfeat` directory contains multiple `cope*.feat` directories. These directories correspond to the results of the run-level (intercept-only) analyses of the seven different first-level contrasts. In contrast to what you did yourself last week (in which you concatenated the COPEs from the $c\hat{\beta}_{\mathrm{face}}$ and $c\hat{\beta}_{\mathrm{place}}$ contrasts in a single GLM), we were lazy and used the "Inputs are lower-level FEAT directories" option in FEAT. This runs the exact same GLM (using an intercept-only run-level design matrix) for all first-level contrasts separately.

For example, the data in the `runlevel.gfeat/cope1.feat` directory corresponds to the first contrast from the first-level analysis ($\beta_{face} > 0$), the data in the `runlevel.gfeat/cope2.feat` directory corresponds to the second contrast from the first-level analysis ($\beta_{place} > 0$), etc. etc.

In our intercept-only run-level analysis, we only computed a run-level contrast reflecting the average across the two sessions, i.e., using the contrast vector $[1]$. The results of this run-level contrast, for each first-level contrast, are stored in the `cope1.nii.gz` file.

    ToDo (ungraded): Open Feat, change "First-level analysis" to "Higher-level analysis" and change the input type to "Inputs are 3D cope images from feat directories". Then, click on the "Stats" tab.

Under the "Stats" tab, you can change the type of inference you want FEAT to do. By default, it uses a particular mixed-effects approach that FSL calls "FLAME 1". In general, we recommend using this type of inference. The other mixed-effects approach, "FLAME 1+2", is also a good choice, but may take much longer to run. The "Mixed effects: Simple OLS" is, unlike the same suggests, the random-effects option, which discards the lower-level variance estimates. Lastly, the "Randomise" option is a non-parametric (random-effects) variant, that will be discussed in more detail later.

    ToDo (ungraded): Set the option to "Mixed effects: FLAME 1". Enable the "use automatic outlier de-weighting" option (should be yellow). Then, go to post-stats and set the "Thresholding" option to "Cluster" with a Z threshold of 3.7 and a "Cluster P threshold" of 0.05. This refers to a particular type of multiple comparison correction that will be discussed later.

Now, let's define the dependent variable ($y^{\dagger}$). Suppose, just like we did with the simulated data, that we have the following null and alternative hypothesis:

$H_{0}: \beta_{face}^{*} - \beta_{house}^{*} = 0$

$H_{a}: \beta_{face}^{*} - \beta_{house}^{*} > 0$

Note that the contrast evaluating the difference between these two parameters was *not* computed in the first-level or run-level analyses, so we have to create this ourselves by concatenating the COPEs corresponding to the $\beta_{face}^{*} > 0$ and $\beta_{house}^{*} > 0$ run-level contrasts, just like you did last week.

Doing this for all subjects becomes a bit cumbersome though, so you only need to do this for `sub-02`, `sub-03`, and `sub-04`. Make sure the first three inputs correspond to the face-against-baseline COPE (COPE nr. 1) and the last three inputs correspond to the place-against-baseline COPE (COPE nr. 2).

\begin{align}
y^{\dagger} = \begin{bmatrix}
c\hat{\beta}^{*}_{\mathrm{face>0,\ sub-02}} \\
c\hat{\beta}^{*}_{\mathrm{face>0,\ sub-03}} \\
c\hat{\beta}^{*}_{\mathrm{face>0,\ sub-04}} \\
c\hat{\beta}^{*}_{\mathrm{place>0,\ sub-02}} \\
c\hat{\beta}^{*}_{\mathrm{place>0,\ sub-03}} \\
c\hat{\beta}^{*}_{\mathrm{place>0,\ sub-04}} \\
\end{bmatrix}
\end{align}

Note that you need the actual `cope*.nii.gz` files — *not* the `.feat` directory itself.

(Realize, though,  that having only 3 subjects for your group-analysis would be a hopelessly underpowered study.)

    ToDo (1 point): In the "Data" tab, use the option "Inputs are 3D cope images from FEAT directories", set the number of inputs to 6, and select the correct files. Note that within the "Select input data" window, you can copy/paste the paths using CTRL+C / CTRL+V, which might make the process a little easier/faster (not forget to actually modify the paths after copy/pasting).

    ToDo (ungraded): In the "Data" tab, set the output directory to your home folder + <tt>grouplevel_task-flockBLOCKED</tt>.

    ToDo (1 point): In the “Stats” tab, click on "Full model setup". Create two predictors ("EVs") that model the mean of the $c\beta^{*}_{face} > 0$ (first EV) and the $c\beta^{*}_{place} > 0$ (second EV). Then, add another predictor that models the effect of age on the $c\beta^{*}_{face}$ contrast *only*. The ages of the three participants (from sub-02 to sub-04) are: $[21, 32, 25]$. Importantly, make sure you center (de-mean) the age values before entering them in FEAT, as this improves the interpretation of the other predictors (you can compute the de-meaned age values, for example, in a new code cell).

    ToDo (1 point): In the "Contrasts & F-tests" tab, define the following group-level contrasts:

1. $H_{0}: \beta^{\dagger}_{face} = 0$
$H_{a}: \beta^{\dagger}_{face} > 0$

2. $H_{0}: \beta^{\dagger}_{place} = 0$
$H_{a}: \beta^{\dagger}_{place} > 0$

3. $H_{0}: \beta^{\dagger}_{face} - \beta^{\dagger}_{place} = 0$
$H_{a}: \beta^{\dagger}_{face} - \beta^{\dagger}_{place} > 0$

4. $H_{0}: \beta^{\dagger}_{age} = 0$
$H_{a}: \beta^{\dagger}_{age} > 0$

5. $H_{0}: \beta^{\dagger}_{age} = 0$
$H_{a}: \beta^{\dagger}_{age} < 0$

    ToDo (ungraded): Save your design (using the "Save" button in the bottom of the FEAT window) in your `week_6` directory and name it <tt>setup_feat_grouplevel.fsf</tt>.

Below, you can test your implementation.

```python
''' Tests your setup_feat.fsf file: inputs'''
import os
import os.path as op
import niedu

here = op.abspath('')
par_dir = op.basename(op.dirname(here))
home = op.expanduser("~")

fsl_deriv_dir = op.join(os.path.expanduser("~"), 'NI-edu-data', 'derivatives', 'fsl')

if par_dir != 'solutions':  # must be a student-account
    fsf = 'setup_feat_grouplevel.fsf'
    if not op.isfile(fsf):
        print("Couldn't find a 'setup_feat_group.fsf' file in your week_6 directory!")

    with open(fsf, 'r') as f:
        fsf = f.readlines()

    fsf = [txt.replace('\n', '') for txt in fsf if txt != '\n']
    fsf = [txt for txt in fsf if txt[0] != '#']  # remove commnts

    from niedu.tests.nii.week_6 import test_grouplevel_fsf_inputs
    test_grouplevel_fsf_inputs(fsf, fsl_deriv_dir)
```

```python
''' Tests your setup_feat.fsf file: Evs'''
if par_dir != 'solutions':
    from niedu.tests.nii.week_6 import test_grouplevel_fsf_evs
    test_grouplevel_fsf_evs(fsf)
```

```python
''' Tests your setup_feat.fsf file: contrasts'''
if par_dir != 'solutions':
    from niedu.tests.nii.week_6 import test_grouplevel_fsf_cons
    test_grouplevel_fsf_cons(fsf)
```

## Non-parametric analyses
*This is an optional section*

Before we go on with the topic of "multiple comparison correction" in the next notebook, let's focus on another type of inference in statistical models: permutation-based non-parametric analyses. Unlike parametric analyses (which we discussed in the previous section), non-parametric analyses do not assume anything about how our statistics-of-interest are distributed. Instead, it uses the data itself to estimate a distribution of your test-statistic under the null hypothesis. Note that this type of inference is also available in FSL, where it is called "randomise".

Permutation tests work (roughly) as follows:
1. The statistic (e.g., $t$-value) is computed as normal;
2. The rows of design matrix ($\mathbf{X}$) are shuffled and the statistic is (re-)computed, which is iterated a number of times (e.g., 1000);
3. The ("non-parametric") $p$-value is computed

Let's do this for our previously simulated data (the variables `X` and `y_sim`).

### Computing the "observed" statistic
First, we need to compute our "observed" statistic. In the framework of the GLM, this is often a $t$-statistic (or $F$-statistic). But note that permutation-based tests work for any kind of statistics. (In fact, permutation-based tests are especially useful for statistics with unknown distributions!)

We are sure that you, by now, know how to compute $t$-values, so we are going to use a predefined function `compute_tvalue` with three arguments: `X`, `y`, and `c`. We'll compute the $t$-value for our data (`X` and `y_sim`) and store it in a variable called `t_obs` (for "observed").

```python
from niedu.utils.nii import compute_tvalue
t_obs = compute_tvalue(X, y_sim, c=np.array([1, -1]))
```

### Run the permutations
The second step is to, iteratively, permute the data and (re-)compute the test statistic. In most cases (assuming that the datapoints, $y$, are completely independent, i.e., there is no "autocorrelation"), permutation within the GLM framework is done by shuffling the rows of the design matrix ($\mathbf{X}$). This is usually done a larger number of times (e.g., 1000 or more).

    ToDo (ungraded): Below, we started implementing the permutation loop. Can you finish it? You may want to use the np.random.permutation function. Make sure to save the permuted t-value each iteration (by storing it in the <tt>t_perm</tt> array).

```python
""" Implement the ToDo here. """
np.random.seed(42)
iters = 1000
t_perm = np.zeros(iters)

for i in range(iters):
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
''' Tests the ToDo above. '''
np.testing.assert_almost_equal(t_perm.mean(), 0, decimal=1)
print("Well done!")
```

### Calculate the p-value
The last step is to calculate the $p$-value. This is done by dividing the number of permuted statistics *larger* than the observed statistic plus one by the total number of permutations plus one. Or, mathematically, for $P$ permutations, the $p$-value for any statistic $s$ with observed value $s^{\mathrm{obs}}$ and vector of permuted values $\mathbf{s}^{\mathrm{perm}}$ is defined as:

\begin{align}
p = \frac{\sum_{i=1}^{P}\mathbf{I}(\mathbf{s}^{\mathrm{perm}}_{i} \geq s^{\mathrm{obs}}) + 1}{P + 1}
\end{align}

where $\mathbf{I}$ is an indicator function, returning $1$ when the condition is true and $0$ otherwise.

We can also visualize it:

```python
plt.figure(figsize=(10, 5))
plt.hist(t_perm)
plt.axvline(t_obs, ls='--', c='red', lw=3)
plt.legend(['observed', 'permuted'])
plt.xlabel(r'$t$-value', fontsize=20)
plt.ylabel('Frequency', fontsize=20)
plt.show()
```

In the above figure, the number of values right of the dashed red line ($t^{obs}$) plus 1 divided by the number of permutations plus 1 ($P+1$) is our $p$-value! Now, try computing it yourself!

    ToDo (ungraded): Compute the $p$-value corresponding to our observed $t$-statistic (<tt>t_obs</tt>) and store it in a variable named <tt>p_perm</tt>.

```python
# Implement your ToDo here
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo.
Note: this may differ slightly due to the exact implementation of the permutation procedure
'''
np.testing.assert_almost_equal(p_perm, 0.03, decimal=2)
print("Well done!")
```

    ToDo (ungraded): Shuffling rows works for most designs, but what if you have an intercept-only model (i.e., a one-sample $t$-test), where a design matrix consisting only of a vector of ones? Permuting the rows won't work there. An alternative for this scenario is "sign flipping": randomly changing ones to minus ones (with replacement).

Below, we simulate some new data (<tt>y_sim2</tt> and <tt>X2</tt>). Can you compute the non-parametric $p$-value for this one-sample $t$-test? Use a 1000 permutations and store the $p$-value in a new variable named <tt>p_perm2</tt>.

Hint: to generate random "signs", take a look here.

```python
np.random.seed(17)
y_sim2 = np.random.normal(0, 1, size=100)
X2 = np.ones(100)[:, np.newaxis]

iters = 1000

# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
# The p-value might slightly differ from 0.17 depending on the
# exact method of generating random signs.
np.testing.assert_almost_equal(p_perm2, 0.17, decimal=2)
print("Well done!")
```

### Threshold-Free Cluster Enhancement (TFCE)
Apart from (or perhaps because of) the fact that non-parametric analyses are blind to the distribution of effects, non-parametric analyses have an additional advantage: they can use a "trick" called "Threshold-Free Cluster Enhancement" (TFCE). This method allows for voxel-based inference while taking into account the spatial size of the effect. You could see TFCE as a method to "boost" high-amplitude voxels that are located within a cluster of other high-amplitude voxels.

However, we don't know how these "boosted" values are distributed (they might or might not be $\mathcal{T}$ distributed)! Fortunately, as said before, non-parameteric analyses don't care how the computed statistic is distributed (under the null). How TFCE works mathematically is beyond the scope of the course (but you can learn more in [this video](https://www.youtube.com/watch?v=q7cWw8WC0Ws)), but in fact recommend using this technique in combination with FSL's `randomise` tool, also because it provides an elegant and effective way to control for multiple comparisons (the topic of section 3).

##  Checking out results from group-level FEAT analyses
We have run a group-level model for a couple of our lower-level contrasts. The results from the group-level analysis that we are going to take a look at will be downloaded below:

```python
print("Downloading group-level FSL FEAT results (+- 67MB) ...")
!aws s3 sync --no-sign-request s3://openneuro.org/ds003477 {data_dir} --exclude "*" --include "derivatives/fsl/grouplevel_task-flocBLOCKED/contrast-faceGTother_method-FLAME1_thresh-uncorr05.gfeat/*"
print("\nDone!")
```

```python
gl_dir = os.path.join(fsl_deriv_dir, 'grouplevel_task-flocBLOCKED', 'contrast-faceGTother_method-FLAME1_thresh-uncorr05.gfeat')
print(f"Data is located here: {gl_dir}")
```

This contains the results from the group-level analysis of the lower-level $4 \cdot \beta_{face} - \beta_{place} - \beta_{body} - \beta_{character} - \beta_{object} > 0$ contrast. (The "faceGTother" stands for "face greater than other conditions".)

    ToDo (ungraded): Open a terminal, navigate to the above directory, and print its contents to the terminal window.

You should see the usual files: HTML-reports, thresholded results (as nifti files), and files with information about the design (`design.con`, `design.fsf`, etc.).

    ToDo (ungraded): Open the <tt>report.html</tt> file (with the command <tt>firefox report.html</tt>). Click on "Inputs", which will show you the data that we used as inputs (i.e., $y^{\dagger}$) for our group-level analysis.

As you can see, the inputs for our group-level analysis are not individual `cope*.nii.gz` files, but entire `.feat` directories. This is because we used the "Inputs are lower-level FEAT directories" option.

    ToDo (ungraded): Click on "Results". This will show you a page with the simple "intercept-only" design (i.e., a design for a one-sample $t$-test). Now, click on "Lower-level contrast 1 (average)" to view in the group-level result from this particular run-level contrast.

On this page, you can view the results, as $z$-statistic images, from the group-level analysis plotted on a set of axial slices. Below the brain plot, you can see the "time series plot", which does not actually show a "time series", but rather the input (i.e., $y^{\dagger}$) of the voxel with the highest $z$-value. Thus, the "time series" here do not reflect datapoints over time (as in first-level analyses), but run-level $c\hat{\beta}^{*}$ estimates of different subjects!

    ToDo (ungraded): In your terminal, open <tt>FSLeyes</tt>. Then, click on <tt>File</tt> → <tt>Add standard</tt> → select <tt>MNI152_T1_2mm_brain.nii.gz</tt> → click on <tt>Open</tt>.

Then, click on <tt>File</tt> → <tt>Add from directory</tt> → navigate to and select the <tt>contrast-faceGTothers_method-FLAME1_thresh-uncorr05.gfeat/cope1.feat</tt> directory → click <tt>Open</tt>.

This automatically opens the `filtered_func_data.nii.gz` file: a 4D nifti file with the run-level $c\hat{\beta}^{*}$ estimates of different subjects across the fourth dimension (i.e., data from different subjects are represented by different volumes, our new $y^{\dagger}$).

    ToDo (ungraded): Set the minimum value to 100 (in the top menu) and change the colormap to "Red-Yellow". Note that the minimum value of 100 is rather arbitrary, but it removes some of the "visual clutter".

Now, go to voxel $[26, 39, 25]$, which is (roughly) located in the right fusiform gyrus (the "FFA") and, for this participant (<tt>sub-02</tt>), seems to activate relatively strongly in response to faces (rather than to other images).

    ToDo (ungraded): Increase the volume number from 0 to 1 (in the "Location" panel), which will show you the run-level $c\hat{\beta}^{*}$ estimates from a different subject (i.e., <tt>sub-03</tt>). You'll notice that this participant's "FFA" is in a slightly different location! In fact, when you view the other volumes (i.e., the $c\hat{\beta}^{*}$ maps of other subjects), you see that the anatomical location of the effects varies substantially!

    ToThink (1 point): Name two reasons why an effect (i.e., a significant "blob") may differ in anatomical location across subjects.

YOUR ANSWER HERE

Alright, now let's look at some real group-level results in FSLeyes!

    ToDo (ungraded): In FSLeyes, delete the <tt>cope1: filtered_func_data</tt> overlay (or make it invisible by clicking on the eye icon). Then, open the <tt>thresh_zstat1.nii.gz</tt> (<tt>File</tt> → <tt>Add from file</tt>) and set the colormap to "Red-Yellow".

Here, you see the "thresholded" $z$-values plotted on the template (MNI152, 2mm) that FSL by default uses for its group-analyses.

    ToDo/ToThink (1 point): Go to the voxel at location $[37, 60, 28]$. According to the Harvard-Oxford Subcortical structural atlas, to what anatomical brain region does this voxel most likely belong? Write this down in the text cell below. Check out the notebook from last week if you forgot how to open the Atlas panel.

YOUR ANSWER HERE

The red-yellow voxels that you see are those that have $z$-values corresponding to $p$-values smaller than 0.05 (the other voxels with $p$ > 0.05 are set to 0). This particular thresholding procedure gives what people often refer to as "uncorrected results". Although they are thresholded at a particular $p$-value, the results are referred to as "uncorrected" because they have not been corrected for multiple comparisons — the topic of the next notebook (`MCC.ipynb`).

    Tip!
    Before handing in your notebooks, we recommend restarting your kernel (Kernel → Restart & Clear Ouput) and running all your cells again (manually, or by Cell → Run all). By running all your cells one by one (from "top" to "bottom" of the notebook), you may spot potential errors that are caused by accidentally overwriting your variables or running your cells out of order (e.g., defining the variable 'x' in cell 28 which you then use in cell 15).


# Stochastic Processes:
Data Analysis and Computer Simulation

# Brownian motion 2: computer simulation

# 3. Simulations with on-the-fly animation

# 3.1. Simulation code with on-the-fly animation

## Import libraries

```python
% matplotlib nbagg
import numpy as np # import numpy library as np
import matplotlib.pyplot as plt # import pyplot library as plt
import matplotlib.mlab as mlab # import mlab module to use MATLAB commands with the same names
import matplotlib.animation as animation # import animation modules from matplotlib
from mpl_toolkits.mplot3d import Axes3D # import Axes3D from mpl_toolkits.mplot3d
plt.style.use('ggplot') # use "ggplot" style for graphs
```

## Define `init` function for `FuncAnimation`

```python
def init():
    global R,V,W,Rs,Vs,Ws,time
    R[:,:] = 0.0 # initialize all the variables to zero
    V[:,:] = 0.0 # initialize all the variables to zero
    W[:,:] = 0.0 # initialize all the variables to zero
    Rs[:,:,:] = 0.0 # initialize all the variables to zero
    Vs[:,:,:] = 0.0 # initialize all the variables to zero
    Ws[:,:,:] = 0.0 # initialize all the variables to zero
    time[:]   = 0.0 # initialize all the variables to zero
    title.set_text(r'') # empty title
    line.set_data([],[]) # set line data to show the trajectory of particle n in 2d (x,y)
    line.set_3d_properties([]) # add z-data separately for 3d plot
    particles.set_data([],[]) # set position current (x,y) position data for all particles
    particles.set_3d_properties([]) # add current z data of particles to get 3d plot
    return particles,title,line # return listed objects that will be drawn by FuncAnimation
```

## Define `animate` function for `FuncAnimation`

```python
def animate(i):
    global R,V,W,Rs,Vs,Ws,time # define global variables
    time[i]=i*dt # store time in each step in an array time
    W = std*np.random.randn(nump,dim) # generate an array of random forces accordingly to Eqs.(F10) and (F11)
    R, V = R + V*dt, V*(1-zeta/m*dt)+W/m # update R & V via Eqs.(F5)&(F9)
    Rs[i,:,:]=R # accumulate particle positions at each step in an array Rs
    Vs[i,:,:]=V # accumulate particle velocitys at each step in an array Vs
    Ws[i,:,:]=W # accumulate random forces at each step in an array Ws
    title.set_text(r"t = "+str(time[i])) # set the title to display the current time
    line.set_data(Rs[:i+1,n,0],Rs[:i+1,n,1]) # set the line in 2D (x,y)
    line.set_3d_properties(Rs[:i+1,n,2]) # add z axis to set the line in 3D
    particles.set_data(R[:,0],R[:,1]) # set the current position of all the particles in 2d (x,y)
    particles.set_3d_properties(R[:,2]) # add z axis to set the particle in 3D
    return particles,title,line # return listed objects that will be drawn by FuncAnimation
```

## Set parameters and initialize variables

```python
dim  = 3 # system dimension (x,y,z)
nump = 1000 # number of independent Brownian particles to simulate
nums = 1024 # number of simulation steps
dt   = 0.05 # set time increment, \Delta t
zeta = 1.0 # set friction constant, \zeta
m    = 1.0 # set particle mass, m
kBT  = 1.0 # set temperatute, k_B T
std  = np.sqrt(2*kBT*zeta*dt) # calculate std for \Delta W via Eq.(F11)
np.random.seed(0) # initialize random number generator with a seed=0
R = np.zeros([nump,dim]) # array to store current positions and set initial condition Eq.(F12)
V = np.zeros([nump,dim]) # array to store current velocities and set initial condition Eq.(F12)
W = np.zeros([nump,dim]) # array to store current random forcces
Rs = np.zeros([nums,nump,dim]) # array to store positions at all steps
Vs = np.zeros([nums,nump,dim]) # array to store velocities at all steps
Ws = np.zeros([nums,nump,dim]) # array to store random forces at all steps
time = np.zeros([nums]) # an array to store time at all steps
```

## Perform and animate the simulation using `FuncAnimation`

```python
fig = plt.figure(figsize=(10,10)) # set fig with its size 10 x 10 inch
ax = fig.add_subplot(111,projection='3d') # creates an additional axis to the standard 2D axes
box = 40 # set draw area as box^3
ax.set_xlim(-box/2,box/2) # set x-range
ax.set_ylim(-box/2,box/2) # set y-range
ax.set_zlim(-box/2,box/2) # set z-range
ax.set_xlabel(r"x",fontsize=20) # set x-lavel
ax.set_ylabel(r"y",fontsize=20) # set y-lavel
ax.set_zlabel(r"z",fontsize=20) # set z-lavel
ax.view_init(elev=12,azim=120) # set view point
particles, = ax.plot([],[],[],'ro',ms=8,alpha=0.5) # define object particles
title = ax.text(-180.,0.,250.,r'',transform=ax.transAxes,va='center') # define object title
line, = ax.plot([],[],[],'b',lw=1,alpha=0.8) # define object line
n = 0 # trajectry line is plotted for the n-th particle
anim = animation.FuncAnimation(fig,func=animate,init_func=init,
            frames=nums,interval=5,blit=True,repeat=False)
## If you have ffmpeg installed on your machine
## you can save the animation by uncomment the last line
## You may install ffmpeg by typing the following command in command prompt
## conda install -c menpo ffmpeg
##
# anim.save('movie.mp4',fps=50,dpi=100)
```

    <IPython.core.display.Javascript object>

## Summary of simulation methods

### Original differential equation

\begin{equation}
\frac{d\mathbf{R}(t)}{dt}=\mathbf{V}(t)\tag{F1}
\end{equation}

\begin{equation}
m\frac{d\mathbf{V}(t)}{dt}=\color{black}{-\zeta\mathbf{V}(t)}+\color{black}{\mathbf{F}(t)}
\tag{F2}
\end{equation}
$\hspace{80mm}$with
\begin{equation}
\langle \mathbf{F}(t)\rangle=\mathbf{0}
\tag{F3}
\end{equation}

\begin{equation}
\langle \mathbf{F}(t)\mathbf{F}(0)\rangle = {2k_B T\zeta}\mathbf{I}\delta(t)
\tag{F4}
\end{equation}

### Euler method

$$
\mathbf{V}_{i+1}
=\left(1-\frac{\zeta}{m}\Delta t\right)\mathbf{V}_i + \frac{1}{m} {\Delta \mathbf{W}_i}
\tag{F9}
$$

$$
\mathbf{R}_{i+1}=\mathbf{R}_i+\mathbf{V}_i \Delta t \hspace{15mm}\tag{B3}
$$
$\hspace{80mm}$with
\begin{equation}
\langle \Delta \mathbf{W}_i\rangle=\mathbf{0}
\tag{F10}
\end{equation}

\begin{equation}
\langle \Delta \mathbf{W}_i\Delta \mathbf{W}_j\rangle = {2k_B T\zeta}\Delta t\mathbf{I}\delta_{ij}
\tag{F11}
\end{equation}

### 2nd order Runge-Kutta method

$$
\mathbf{V}'_{i+\frac{1}{2}}
=\mathbf{V}_i-\frac{\zeta}{m}\frac{\Delta t}{2}\mathbf{V}_{i}
=\left(1-\frac{\zeta}{m}\frac{\Delta t}{2}\right)\mathbf{V}_{i}
\tag{F12}
$$

$$
\mathbf{V}_{i+1}
=\mathbf{V}_i-\frac{\zeta}{m}\Delta t\mathbf{V}'_{i+\frac{1}{2}} + \frac{1}{m} {\Delta \mathbf{W}_i}
\tag{F13}
$$

$$
\mathbf{R}_{i+1}=\mathbf{R}_i+\mathbf{V}'_{i+\frac{1}{2}} \Delta t \hspace{15mm}
\tag{F14}
$$

### 4th order Runge-Kutta method

$$
\mathbf{V}'_{i+\frac{1}{2}}
=\mathbf{V}_i-\frac{\zeta}{m}\frac{\Delta t}{2}\mathbf{V}_{i}
\tag{F15}
$$

$$
\mathbf{V}''_{i+\frac{1}{2}}
=\mathbf{V}_i-\frac{\zeta}{m}\frac{\Delta t}{2}\mathbf{V}'_{i+\frac{1}{2}}
\tag{F16}
$$

$$
\mathbf{V}'''_{i+1}
=\mathbf{V}_i-\frac{\zeta}{m}{\Delta t}\mathbf{V}''_{i+\frac{1}{2}}
\tag{F17}
$$

$$
\mathbf{V}_{i+1}
=\mathbf{V}_i-\frac{\zeta}{m}\frac{\Delta t}{6}\left(\mathbf{V}+\mathbf{V}'_{i+\frac{1}{2}}+\mathbf{V}''_{i+\frac{1}{2}}+\mathbf{V}'''_{i+1}\right) + \frac{1}{m} {\Delta \mathbf{W}_i}
\tag{F18}
$$

$$
\mathbf{R}_{i+1}=\mathbf{R}_i+
\frac{\Delta t}{6}\left(\mathbf{V}+\mathbf{V}'_{i+\frac{1}{2}}+\mathbf{V}''_{i+\frac{1}{2}}+\mathbf{V}'''_{i+1}\right)
\hspace{15mm}
\tag{F19}
$$


# Flux Vector Splitting

The flux vector splitting (FVS) method is a means of solving a hyperbolic system of equations in an upwind fashion. In this note, we will establish an implementation framework for computing the flux at a face using FVS.

## Preamble

* Define "vec" command for LaTeX $\newcommand{vec}[1]{\boldsymbol{#1}}$

```python
import sympy
sympy.init_printing(use_unicode=True)
```

## Conservation Equations
Consider the Euler Equations in 3D:

\begin{equation}
    \partial_t \vec{w} + \partial_i \vec{f}_i = 0
\end{equation}

Where

\begin{equation}
    \vec{w} = \begin{bmatrix}
        \rho     \\
        \rho u_1 \\
        \rho u_2 \\
        \rho u_3 \\
        \rho E
    \end{bmatrix}
    \qquad
    \vec{f}_i = \begin{bmatrix}
        \rho u_i                    \\
        \rho u_1 u_i + \delta_{1i}p \\
        \rho u_2 u_i + \delta_{2i}p \\
        \rho u_3 u_i + \delta_{3i}p \\
        \rho H u_i
    \end{bmatrix}
\end{equation}

With the constituitive relations

\begin{align}
    p &= \rho R T
       = \left( \gamma - 1 \right) \rho e \\
    e &= c_vT = E - k \\
    k &= \frac{1}{2}u_i u_i \\
    H &= E + \frac{p}{\rho} \\
\end{align}

## Flux Jacobians

Many computational methods require linearization of the flux vectors, which can be defined for a generalized coordinate direction $\vec{n} = [n_1, n_2, n_3]$ as follows:

\begin{equation}
    \vec{A}_n = \partial_\vec{w} \vec{f}_i n_i
        = \partial_\vec{w} \begin{bmatrix}
            \rho u_i n_i             \\
            \rho u_1 u_i n_i + n_1 p \\
            \rho u_2 u_i n_i + n_2 p \\
            \rho u_3 u_i n_i + n_3 p \\
            \rho H u_i n_i
        \end{bmatrix}
        = \partial_\vec{w} \left(\vec{w} u_i n_i \right) + \partial_\vec{w} p \begin{bmatrix}
            0   \\
            n_1 \\
            n_2 \\
            n_3 \\
            u_i n_i
        \end{bmatrix}
\end{equation}

Defining $u_n = u_i n_i$ and noting that $\partial_\vec{w} u_n = \rho^{-1} \cdot \left[ -u_n, n_1, n_2, n_3, 0 \right]$, the equation above becomes

\begin{equation}
    \vec{A}_n = \vec{I} u_n
    + \begin{bmatrix} 1 \\ u_1 \\ u_2 \\ u_3 \\ H \end{bmatrix}
      \begin{bmatrix} -u_n, n_1, n_2, n_3, 0 \end{bmatrix}
    + \begin{bmatrix} 0 \\ n_1 \\ n_2 \\ n_3 \\ u_n \end{bmatrix} \partial_\vec{w} p
\end{equation}

Note that is deriving the formula, we have not yet invoked any assumptions about the thermochemical characteristics of the gas (ideal, calorically perfect, etc.). Furthermore, the expression can be implemented with reasonable efficiency since we're simply summing a diagonal plus two outer products.

If we assume the gas is in thermochemical equilibrium, the equation of state for the mixture can be written in the form $p = p(\rho,\rho e)$. Therefore, the partial derivative of pressure with respect to our state variables may be written as:

\begin{equation}
    \partial_{\vec{w}}p = \left. \frac{\partial p}{\partial\rho}\right|_{\rho e} \partial_\vec{w} \rho
                        + \left. \frac{\partial p}{\partial\rho e}\right|_\rho \partial_\vec{w} \rho e
                        = \alpha \cdot \partial_\vec{w} \rho + \beta \cdot \partial_\vec{w} \rho e
\end{equation}

Recalling that $\rho e = \rho E - \rho k$, we have:

\begin{align}
    \partial_\vec{w} \rho   &= \begin{bmatrix} 1, &0,  &0,  &0,  &0 \end{bmatrix} \\
    \partial_\vec{w} \rho e &= \begin{bmatrix} k, &-u_1, &-u_2, &-u_3, &1 \end{bmatrix} \\
    \partial_\vec{w} p      &= \beta \begin{bmatrix} k + \beta^{-1}\alpha, &-u_1, &-u_2, &-u_3, &1 \end{bmatrix}
\end{align}

Substituting this into our previous expression for $\vec{A}_n$:

\begin{equation}
    \vec{A}_n = \vec{I} u_n
    + \begin{bmatrix} 1 \\ u_1 \\ u_2 \\ u_3 \\ H \end{bmatrix}
      \begin{bmatrix} -u_n, n_1, n_2, n_3, 0 \end{bmatrix}
    + \beta
      \begin{bmatrix} 0 \\ n_1 \\ n_2 \\ n_3 \\ u_n \end{bmatrix}
      \begin{bmatrix} k + \beta^{-1}\alpha, -u_1, -u_2, -u_3, 1 \end{bmatrix}
\end{equation}

In practice, it may not be possible to parameterize the equation of state in terms of $\rho, \rho e$. For example, the equilibrium air curve fits of Reference [1] utilize an equation of state of the form

\begin{equation}
    p = \rho e \cdot \left( \gamma \left( \ln{\rho}, \ln{e} \right) - 1 \right)
\end{equation}

With this form, it is simpler to compute partial derivatives with respect to $\rho, e$:

\begin{align}
    \left.\frac{\partial p}{\partial \rho}\right|_{e} &= e\left( \gamma-1 + \frac{\partial \gamma}{\partial \ln{\rho}}\right) \\
    \left.\frac{\partial p}{\partial e}\right|_{\rho} &= \rho\left( \gamma-1 + \frac{\partial\gamma}{\partial \ln{e}}\right)
\end{align}

This can then be related to $\alpha, \beta$ by expanding $\delta \rho e$ in the expression for the differential change in pressure:

\begin{align}
    \delta p &= \alpha \cdot \delta\rho + \beta \cdot \delta\rho e \\
             &= \alpha \cdot \delta\rho + \beta \cdot \left( e \delta\rho + \rho \delta e \right) \\
             &= \left(\alpha + e\beta \right) \cdot \delta\rho + \rho \beta \cdot \delta e \\
\end{align}

From which we can surmize that

\begin{align}
    \beta  &= \frac{1}{\rho}\left. \frac{ \partial p }{ \partial e } \right|_\rho \\
    \alpha &= \left. \frac{ \partial p }{ \partial \rho } \right|_e - e\beta \\
\end{align}

For the case of an ideal, calorically perfect gas governed by the constituitive relations shown above, the partial derivatives of the equation of state are $\left(\alpha,\beta\right) = \left(0, \gamma-1\right)$. As a result,

\begin{equation}
    \partial_\vec{w}p = \left( \gamma - 1 \right) \cdot \left[ k, -u_1, -u_2, -u_3, 1 \right]
\end{equation}

Which leads to the following general expression for the flux Jacobian for the calorically perfect Euler equations:

\begin{equation}
    \vec{A}_n = \vec{I} u_n
    + \begin{bmatrix} 1 \\ u_1 \\ u_2 \\ u_3 \\ H \end{bmatrix}
      \begin{bmatrix} -u_n, n_1, n_2, n_3, 0 \end{bmatrix}
    + \left(\gamma - 1\right)
      \begin{bmatrix} 0 \\ n_1 \\ n_2 \\ n_3 \\ u_n \end{bmatrix}
      \begin{bmatrix} k, -u_1, -u_2, -u_3, 1 \end{bmatrix}
\end{equation}

The code below computes the calorically perfect Jacobian for $\vec{n} = [1, 0, 0]$, which I have verified matches the Jacobian published in Reference [2].

```python
from sympy.matrices import Matrix, eye

# Symbols
u1, u2, u3 = sympy.symbols('u_1 u_2 u_3')
n1, n2, n3 = sympy.symbols('n_1 n_2 n_3')
H, e, k, γ = sympy.symbols('H e k γ')
un = sympy.symbols('u_n')

# General Jacobian matrix
v1 = Matrix([  1,  u1,  u2,  u3,   H])
v2 = Matrix([-un,  n1,  n2,  n3,   0])
v3 = Matrix([  0,  n1,  n2,  n3,  un])
v4 = (γ-1)*Matrix([k, -u1, -u2, -u3, 1])
An = un*eye(5) + v1*v2.transpose() + v3*v4.transpose()
An.simplify()
An
```

```python
# Assume n = (1,0,0)
a = sympy.symbols('a')
A = An.subs({n1:1, n2:0, n3:0, un:u1})
A = A.expand()
for i,entry in enumerate(A):
    A[i] = sympy.collect(A[i], u1)
    A[i] = sympy.collect(A[i], u2)
    A[i] = sympy.collect(A[i], u3)
    A[i] = sympy.collect(A[i], k)
#A.simplify()
A # Matches Ref [1], Eq 2.15
```

## Eigen-Structure
For many numerical schemes it is necessary to diagonalize the flux Jacobian.

```python
A = A.subs(k, u1*u1 + u2*u2 + u3*u3)
L,Λ = A.diagonalize()
L
```

## References

1. S. Srinivasan, J. Tannehill, K. Weilmuenster. "Simplified Curve Fits for the Thermodynamic Properties of Equilibrium Air". NASA Reference Publication 1181. August 1987.


# Removing `if` Statements from Expressions

## Author: Patrick Nelson

### NRPy+ Source Code for this module:
* [Min_Max_and_Piecewise_Expressions.py](../edit/Min_Max_and_Piecewise_Expressions.py) Contains functions that can be used to compute the minimum or maximum of two values and to implement piecewise-defined expressions

## Introduction:

Conditional statements are a critical tool in programming, allowing us to control the flow through a program to avoid pitfalls, code piecewise-defined functions, and so forth. However, there are times when it is useful to work around them. It takes a processor time to evaluate the whether or not to execute the code block, so for some expressions, performance can be improved by rewriting the expression to use an absolute value function in a manner upon which we will expand in this tutorial. Even more relevant to NRPy+ are piecewise-defined functions. These inherently involve `if` statements, but NRPy+'s automatic code generation cannot handle these by itself, requiring hand-coding to be done. However, if it is possible to rewrite the expression in terms of absolute values, then NRPy+ can handle the entire thing itself.

The absolute value is a function that simply returns the magnitude of its argument, a positive value. That is,
\begin{align}
|x|&= \left \{ \begin{array}{lll}x & \mbox{if} & x \geq 0 \\
-x & \mbox{if} & x \leq 0 \end{array} \right. \\
\end{align}

In C, this is implemented as `fabs()`, which merely has to make the first bit of a double-precision floating point number 0, and is thus quite fast.

There are myriad uses for these tricks in practice. One example comes from GRMHD (and, by extension, the special cases of GRFFE and GRHD), in which it is necessary to limit the velocity of the plasma in order to keep the simulations stable. This is done by calculating the Lorentz factor $\Gamma$ of the plasma and comparing to some predefined maximum $\Gamma_\max$. Then, if
$$
R = 1-\frac{1}{\Gamma^2} > 1-\frac{1}{\Gamma_{\max}^2} = R_\max,
$$
we rescale the velocities by $\sqrt{R_\max/R}$. In NRPy+, we instead always rescale by
$$
\sqrt{\frac{\min(R,R_\max)}{R+\epsilon}},
$$
which has the same effect while allowing the entire process to be handled by NRPy+'s automatic code generation. ($\epsilon$ is some small number chosen to avoid division by zero without affecting the results otherwise.) See [here](Tutorial-GRHD_Equations-Cartesian.ipynb#convertvtou) for more information on this.

# Table of Contents
$$\label{toc}$$

This notebook is organized as follows

1. [Step 1](#min_max): Minimum and Maximum
    1. [Step 1.a](#confirm): Confirm that these work for real numbers
1. [Step 2](#piecewise): Piecewise-defined functions
1. [Step 3](#sympy): Rewrite functions to work with symbolic expressions
1. [Step 4](#validation): Validation against `Min_Max_and_Piecewise_Expressions` NRPy+ module
1. [Step 5](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Minimum and Maximum \[Back to [top](#toc)\]
$$\label{min_max}$$

Our first job will be to rewrite minimum and maximum functions without if statements. For example, the typical implementation of `min(a,b)` will be something like this:
```python
def min(a,b):
    if a<b:
        return a
    else:
        return b
```
However, to take full advantage of NRPy+'s automated function generation capabilities, we want to write this without the `if` statements, replacing them with calls to `fabs()`. We will define these functions in the following way:
$$\boxed{
\min(a,b) = \tfrac{1}{2} \left( a+b - \lvert a-b \rvert \right)\\
\max(a,b) = \tfrac{1}{2} \left( a+b + \lvert a-b \rvert \right).}
$$

## Step 1.a: Confirm that these work for real numbers \[Back to [top](#toc)\]
$$\label{confirm}$$

For real numbers, these operate exactly as expected. In the case $a>b$,
\begin{align}
\min(a,b) &= \tfrac{1}{2} \left( a+b - (a-b) \right) = b \\
\max(a,b) &= \tfrac{1}{2} \left( a+b + (a-b) \right) = a, \\
\end{align}
and in the case $a<b$, the reverse holds:
\begin{align}
\min(a,b) &= \tfrac{1}{2} \left( a+b - (b-a) \right) = a \\
\max(a,b) &= \tfrac{1}{2} \left( a+b + (b-a) \right) = b, \\
\end{align}

In code, we will represent this as:
```
min_noif(a,b) = sp.Rational(1,2)*(a+b-nrpyAbs(a-b))
max_noif(a,b) = sp.Rational(1,2)*(a+b+nrpyAbs(a-b))
```
For demonstration purposes, we will use `np.absolute()` and floating point numbers.

```python
import numpy as np               # NumPy: Python module specializing in numerical computations
import matplotlib.pyplot as plt  # matplotlib: Python module specializing in plotting capabilities

thismodule = "Min_Max_and_Piecewise_Expressions"
# First, we'll write the functions. Note that we are not using sympy right now. For NRPy+ code generation,
# use the expressions above.
def min_noif(a,b):
    return 0.5 * (a+b-np.absolute(a-b))

def max_noif(a,b):
    return 0.5 * (a+b+np.absolute(a-b))

# Now, let's put these through their paces.
a_number = 5.0
another_number = 10.0
print("The minimum of "+str(a_number)+" and "+str(another_number)+" is "+str(min_noif(a_number,another_number)))
```

    The minimum of 5.0 and 10.0 is 5.0

Feel free to test other cases above if you'd like. Note that we use a suffix, `_noif`, to avoid conflicts with other functions. When using this in NRPy+, make sure you use `sp.Rational()` and the `nrpyAbs()` function, which will always be interpreted as the C function `fabs()` (Sympy's `sp.Abs()` may get interpreted as $\sqrt{zz^*}$, for instance).

# Step 2: Piecewise-defined functions \[Back to [top](#toc)\]
$$\label{piecewise}$$

Next, we'll define functions to represent branches of a piecewise-defined function. For example, consider the function
\begin{align}
f(x) &= \left \{ \begin{array}{lll} \frac{1}{10}x^2+1 & \mbox{if} & x \leq 0 \\
\exp(\frac{x}{5}) & \mbox{if} & x > 0 \end{array} \right. , \\
\end{align}
which is continuous, but not differentiable at $x=0$.

To solve this problem, let's add the two parts together, multiplying each part by a function that is either one or zero depending on $x$. To define $x \leq 0$, this can be done by multiplying by the minimum of $x$ and $0$. We also will need to normalize this. To avoid putting a zero in the denominator, however, we will add some small $\epsilon$ to the denominator, i.e.,
$$
\frac{\min(x,0)}{x-\epsilon}
$$
This $\epsilon$ corresponds `TINYDOUBLE` in NRPy+; so, we will define the variable here with its default value, `1e-100`. Additionally, to get the correct behavior on the boundary, we shift the boundary by $\epsilon$, giving us
$$
\frac{\min(x-\epsilon,0)}{x-\epsilon}
$$

The corresponding expression for $x > 0$ can be written as
$$
\frac{\max(x,0)}{x+\epsilon},
$$
using a positive small number to once again avoid division by zero.
When using these for numerical relativity codes, it is important to consider the relationship between $\epsilon$, or `TINYDOUBLE`, and the gridpoints in the simulation. As long as $\epsilon$ is positive and large enough to avoid catastrophic cancellation, these functional forms avoid division by zero, as proven [below](#proof).

So, we'll code NumPy versions of these expressions below. Naturally, there are many circumstances in which one will want the boundary between two pieces of a function to be something other than 0; if we let that boundary be $x^*$, this can easily be done by passing $x-x^*$ to the maximum/minimum functions. For the sake of code readability, we will write the functions to pass $x$ and $x^*$ as separate arguments. Additionally, we code separate functions for $\leq$ and $<$, and likewise for $\geq$ and $>$. The "or equal to" versions add a small offset to the boundary to give the proper behavior on the desired boundary.

```python
TINYDOUBLE = 1.0e-100
def coord_leq_bound(x,xstar):
    # Returns 1.0 if x <= xstar, 0.0 otherwise.
    # Requires appropriately defined TINYDOUBLE
    return min_noif(x-xstar-TINYDOUBLE,0.0)/(x-xstar-TINYDOUBLE)

def coord_geq_bound(x,xstar):
    # Returns 1.0 if x >= xstar, 0.0 otherwise.
    # Requires appropriately defined TINYDOUBLE
    return max_noif(x-xstar+TINYDOUBLE,0.0)/(x-xstar+TINYDOUBLE)

def coord_less_bound(x,xstar):
    # Returns 1.0 if x < xstar, 0.0 otherwise.
    # Requires appropriately defined TINYDOUBLE
    return min_noif(x-xstar,0.0)/(x-xstar-TINYDOUBLE)

def coord_greater_bound(x,xstar):
    # Returns 1.0 if x > xstar, 0.0 otherwise.
    # Requires appropriately defined TINYDOUBLE
    return max_noif(x-xstar,0.0)/(x-xstar+TINYDOUBLE)

# Now, define our the equation and plot it.
x_data = np.arange(start = -10.0, stop = 11.0, step = 1.0)
y_data = coord_less_bound(x_data,0.0)*(0.1*x_data**2.0+1.0)\
        +coord_geq_bound(x_data,0.0)*np.exp(x_data/5.0)

plt.figure()
a = plt.plot(x_data,y_data,'k',label="Piecewise function")
b = plt.plot(x_data,0.1*x_data**2.0+1.0,'b.',label="y=0.1*x^2+1")
c = plt.plot(x_data,np.exp(x_data/5.0),'g.',label="y=exp(x/5)")
plt.legend()
plt.xlabel("x")
plt.ylabel("y")
plt.show()

```

The plot above shows the expected piecewise-defined function. It is important in applying these functions that each greater-than be paired with a less-than-or-equal-to, or vice versa. Otherwise, the way these are written, a point on the boundary will be set to zero or twice the expected value.

These functions can be easily combined for more complicated piecewise-defined functions; if a piece of a function is defined as $f(x)$ on $x^*_- \leq x < x^*_+$, for instance, simply multiply by both functions, e.g.
```
coord_geq_bound(x,x_star_minus)*coord_less_bound(x,x_star_plus)*f(x)
```

# Step 3: Rewrite functions to work with symbolic expressions \[Back to [top](#toc)\]
$$\label{sympy}$$

In order to use this with sympy expressions in NRPy+, we will need to rewrite the `min` and `max` functions with slightly different syntax. Critically, we will change `0.5` to `sp.Rational(1,2)` and calls to `np.absolute()` to `nrpyAbs()`. We will also need to import `outputC.py` here for access to `nrpyAbs()`. The other functions will not require redefinition, because they only call specific combinations of the `min` and `max` function.

In practice, we want to use `nrpyAbs()` and *not* `sp.Abs()` with our symbolic expressions, which will force `outputC` to use the C function `fabs()`, and not try to multiply the argument by its complex conjugate and then take the square root.

```python
from outputC import nrpyAbs     # NRPy+: Core C code output module

def min_noif(a,b):
    # Returns the minimum of a and b
    if a==sp.sympify(0):
        return sp.Rational(1,2) * (b-nrpyAbs(b))
    if b==sp.sympify(0):
        return sp.Rational(1,2) * (a-nrpyAbs(a))
    return sp.Rational(1,2) * (a+b-nrpyAbs(a-b))

def max_noif(a,b):
    # Returns the maximum of a and b
    if a==sp.sympify(0):
        return sp.Rational(1,2) * (b+nrpyAbs(b))
    if b==sp.sympify(0):
        return sp.Rational(1,2) * (a+nrpyAbs(a))
    return sp.Rational(1,2) * (a+b+nrpyAbs(a-b))
```

# Step 4: Validation against `Min_Max_and_Piecewise_Expressions` NRPy+ module \[Back to [top](#toc)\]
$$\label{validation}$$

As a code validation check, we will verify agreement in the SymPy expressions for plane-wave initial data for the Scalar Wave equation between
1. this tutorial and
2. the NRPy+ [Min_Max_and_Piecewise_Expressions](../edit/Min_Max_and_Piecewise_Expressions.py) module.

```python
# Reset & redefine TINYDOUBLE for proper comparison
%reset_selective -f TINYDOUBLE

import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends
import NRPy_param_funcs as par   # NRPy+: parameter interface
TINYDOUBLE = par.Cparameters("REAL", thismodule, "TINYDOUBLE", 1e-100)

import Min_Max_and_Piecewise_Expressions as noif
all_passed=0

def comp_func(expr1,expr2,basename,prefixname2="noif."):
    passed = 0
    if str(expr1-expr2)!="0":
        print(basename+" - "+prefixname2+basename+" = "+ str(expr1-expr2))
        passed = 1
    return passed

a,b = sp.symbols("a b")

here = min_noif(a,b)
there = noif.min_noif(a,b)
all_passed += comp_func(here,there,"min_noif")

here = max_noif(a,b)
there = noif.max_noif(a,b)
all_passed += comp_func(here,there,"max_noif")

here = coord_leq_bound(a,b)
there = noif.coord_leq_bound(a,b)
all_passed += comp_func(here,there,"coord_leq_bound")

here = coord_geq_bound(a,b)
there = noif.coord_geq_bound(a,b)
all_passed += comp_func(here,there,"coord_geq_bound")

here = coord_less_bound(a,b)
there = noif.coord_less_bound(a,b)
all_passed += comp_func(here,there,"coord_less_bound")

here = coord_greater_bound(a,b)
there = noif.coord_greater_bound(a,b)
all_passed += comp_func(here,there,"coord_greater_bound")

import sys
if all_passed==0:
    print("ALL TESTS PASSED!")
else:
    print("ERROR: AT LEAST ONE TEST DID NOT PASS")
    sys.exit(1)
```

    ALL TESTS PASSED!

# Step 5: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Min_Max_and_Piecewise_Expressions.pdf](Tutorial-Min_Max_and_Piecewise_Expressions.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-Min_Max_and_Piecewise_Expressions")
```

    Created Tutorial-Min_Max_and_Piecewise_Expressions.tex, and compiled LaTeX
        file to PDF file Tutorial-Min_Max_and_Piecewise_Expressions.pdf