# SINDy
> Discovering governing equations using the SINDy algorithm

- toc: true
- hide: false
- branch: master
- search_exclude: false
- badges: true
- comments: true
- categories: [differential equations, machine learning]

```python
# hide
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

from scipy.integrate import odeint

from sklearn.metrics import r2_score, mean_absolute_error
from sklearn.metrics import mean_squared_error, median_absolute_error
```

```python
# hide
def mean_absolute_percentage_error(y_true, y_pred):
    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100

def symmetric_mape(y_true, y_pred, eps = 1e-8):
    summ = ((np.abs(y_true) + np.abs(y_pred)) + eps)
    return np.mean(np.abs(y_pred - y_true) / summ) * 100
```

```python
# hide
def print_scores(y_test, y_pred):
    print(f"R2 score: {r2_score(y_test, y_pred)}")
    print(f"MSE score: {mean_squared_error(y_test, y_pred)}")
    print(f"MAE score: {mean_absolute_error(y_test, y_pred)}")
    print(f"Median AE score: {median_absolute_error(y_test, y_pred)}")
    print(f"MAPE score: {mean_absolute_percentage_error(y_test, y_pred)}")
    print(f"SMAPE score: {symmetric_mape(y_test, y_pred)}")
```

Sparse Identification of Nonlinear Dynamical Systems (SINDy) is an algorithm to discover governing dynamical equations for time series ${\bf x}(t)$. The key idea is to construct a differential equation:

$$ \frac{d{\bf x}}{d t} = \Theta({\bf x}^T) {\bf x}$$,

where the time derivative is computed from the time series data and $\Theta({\bf x})$ is a non-linear basis.

Steps:
 - Compute time derivative: Perhaps the trickiest part, especially for noisy time series although one can use total variation regularized derivatives for such cases as in [here](https://github.com/stur86/tvregdiff)
 - Choose basis: Some non-linear basis constructed from the time series data, for example, polynomial basis.
 - Apply regularized linear regression: Apply Lasso/Ridge with one step or sequential thresholding least squares.

Paper: https://www.pnas.org/content/113/15/3932

Once the underlying dynamical equations are discovered, **forecasting** becomes a lot easier.

**Extensions**:

Knowing the right coordinates and basis functions to use is often difficult. The most interesting extension of SINDy so far has been to use latent basis / coordinates instead of physical space basis. In latent space,

$$ {\bf x} \longrightarrow {\bf z} = Encoder({\bf x}).$$

A non-linear basis in ${\bf z}$ is used to perform SINDy. The full cost function also takes into account the physical space SINDy expression. For more details, see the paper:

Paper: https://www.pnas.org/content/116/45/22445

## Example

### Create synthetic dataset

```python
# hide

# **REMEMBER**: change `dx1dt`, `dx2dt` also if you change the ranfun below.
def fun(y, t):
    x1, x2 = y
    dxdt = [-x1**3 - x2, x1 - x2**3]
    return dxdt

t_full = np.linspace(0, 15, 1501)

# The following two functions work great, with alpha = 0.0001, they are being correctly classified.
#y0 = [0.1, 0.05] # LINEAR: small x1, x2 => Linear coupled oscillator
y0 = [0.5, 0.5] # NON-LINEAR: Getting close to triggering non-linear effects
```

**Solve** the ODE:

$$
\begin{align}
\frac{dx_1}{dt} &= -x_1^3 - x_2 \\
\frac{dx_2}{dt} &= x_1 - x_2^3
\end{align}
$$

```python
# hide
sol = odeint(fun, y0, t_full)
sol_new = sol[:1001]
```

Plot the solutions

```python
# hide_input
t_discover = t_full[:1001]

plt.plot(t_discover, sol_new[:, 0], 'b', label='x1(t)')
plt.plot(t_discover, sol_new[:, 1], 'g', label='x2(t)')
plt.legend(loc='best')
plt.xlabel('t')
plt.grid()
# plt.savefig('time_series.png', bbox_inches='tight')
```

## Compute the time derivative.

```python
x1 = sol_new[:,0]
x2 = sol_new[:,1]
```

**Actual** time derivatives

```python
dx1dt = -x1**3 - x2
dx2dt =  x1    - x2**3
```

**Numerically** computed derivatives from data

```python
# hide
dt = t_discover[1] - t_discover[0]
dx1dt_data = np.gradient(x1, dt)
dx2dt_data = np.gradient(x2, dt)
# or using TVRegDiff
# dx1dt = TVRegDiff(x1, 10, 0.1, dx=dt, plotflag=0)
# See https://github.com/stur86/tvregdiff
# for information on different parameters
```

```python
# hide
print_scores(dx1dt, dx1dt_data)
```

    R2 score: 0.9999999965191242
    MSE score: 2.8269156140981004e-10
    MAE score: 4.206507155221408e-06
    Median AE score: 3.6181352647546294e-06
    MAPE score: 0.004477342875542224
    SMAPE score: 0.0022391799382288345

```python
# hide
print_scores(dx2dt, dx2dt_data)
```

    R2 score: 0.9999996026514059
    MSE score: 2.195307457531667e-08
    MAE score: 9.892101759080555e-06
    Median AE score: 4.016199533329878e-06
    MAPE score: 0.009102822948966595
    SMAPE score: 0.004535741601342113

```python
# hide
plt.plot(dx1dt[-50:], label='Actual', alpha=0.4)
plt.plot(dx1dt_data[-50:], ':', lw=2, label='from_data')

plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$dx_1/dt$', fontsize=14)
plt.legend()
```

```python
# hide
plt.plot(dx2dt[-50:], label='Actual', alpha=0.4)
plt.plot(dx2dt_data[-50:], ':', lw=2, label='from_data')

plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$dx_2/dt$', fontsize=14)
plt.legend()
```

```python
# hide
X = np.zeros((sol_new.shape[0], sol_new.shape[1]))

X[:, 0] = x1 # dx1/dt
X[:, 1] = x2 # dx2/dt
```

## Construct a basis: polynomial (or trig)

```python
# collapse
from sklearn.preprocessing import PolynomialFeatures

dum_data = pd.DataFrame({'x1': x1, 'x2': x2})

deg = 3 # Polynomial degree to use

p = PolynomialFeatures(degree=deg,include_bias=True).fit(dum_data)
xpoly = p.fit_transform(dum_data)
newdf = pd.DataFrame(xpoly, columns = p.get_feature_names(dum_data.columns))

print("Feature names:", list(newdf))#newdf.columns.values.tolist())
print("Feature array shape:", newdf.shape)
```

    Feature names: ['1', 'x1', 'x2', 'x1^2', 'x1 x2', 'x2^2', 'x1^3', 'x1^2 x2', 'x1 x2^2', 'x2^3']
    Feature array shape: (1001, 10)

## Regression using Lasso

(or Ridge/OLS with sequential thresholding)

Lasso does regularized linear regression with L1-norm. **alpha** is a hyperparameter.
 - low alpha -> OLS
 - high alpha -> most features zero

```python
from sklearn.linear_model import Lasso
mod = Lasso(alpha=0.0001)
mod
```

    Lasso(alpha=0.0001)

```python
# hide
# Prepare training data
newdf_train, newdf_test = newdf[:800], newdf[800:]
dx1dt_train, dx1dt_test = dx1dt[:800], dx1dt[800:]
dx2dt_train, dx2dt_test = dx2dt[:800], dx2dt[800:]
```

### $dx_1 / dt$

```python
# hide
mod.fit(newdf_train, dx1dt_train)
print(mod.coef_) # should give the 3rd (x2) + 4th (x1^2) argument non-zero
print(mod.intercept_)
mod.score(newdf_test, dx1dt_test)
```

    [ 0.         -0.05576026 -0.99852418  0.         -0.         -0.01819165
     -0.50202469 -0.          0.         -0.        ]
    0.0033302444886464527

    0.9943287924302203

```python
# hide
fit_dx1 = pd.DataFrame(columns=newdf.columns)
fit_dx1.loc[0] = mod.coef_
fit_dx1.abs().sort_values(by=0, axis=1, ascending=False)
```

      x2
      x1^3
      x1
      x2^2
      1
      x1^2
      x1 x2
      x1^2 x2
      x1 x2^2
      x2^3

      0
      0.998524
      0.502025
      0.05576
      0.018192
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

```python
# hide
ypred_x1 = mod.predict(newdf_test)
print_scores(dx1dt_test, ypred_x1)
```

    R2 score: 0.9943287924302203
    MSE score: 8.443005835967624e-05
    MAE score: 0.009127412469564932
    Median AE score: 0.00942351242559783
    MAPE score: 59.78370313040213
    SMAPE score: 9.034670809972091

Identified ODE:

$dx_1/dt \sim -x_2 - x_1^3$

with minor contributions from other terms that could be gotten rid of by applying a threshold

```python
# Drop features with absolute values less than 0.1
dx1_thr = fit_dx1[fit_dx1.columns[fit_dx1.abs().max() > 0.1]]
dx1_thr
```

      x2
      x1^3

      0
      -0.998524
      -0.502025

```python
# hide_input
# PLOT results
t_test  = np.linspace(8, 10, 201)

#plt.plot(t, sol_new[:, 1], label='Actual', alpha = 0.4)
#plt.plot(ypred, label='Actual')
plt.plot(t_test, ypred_x1, 'r--', lw = 2, label='Prediction')
plt.plot(t_discover, dx1dt, 'g', label='Actual', alpha=0.4)
plt.axvline(x = 8, color='k', linestyle='--')
plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$dx_1/dt$', fontsize=14)
plt.grid()
plt.legend()
# plt.savefig('dx1dt_fit.png', bbox_inches='tight')
```

### $dx_2 / dt$

```python
# hide
mod.fit(newdf_train, dx2dt_train)
print(mod.coef_) # should give the 2nd (x1) + last (x2^3) argument non-zero
print(mod.intercept_)
mod.score(newdf_test, dx2dt_test)
```

    [ 0.          0.99491832 -0.02198579 -0.         -0.         -0.04791986
      0.          0.         -0.         -0.82689872]
    0.0022126622685510813

    0.9965028656692735

```python
# hide
fit_dx2 = pd.DataFrame(columns=newdf.columns)
fit_dx2.loc[0] = mod.coef_
fit_dx2.abs().sort_values(by=0, axis=1, ascending=False)
```

      x1
      x2^3
      x2^2
      x2
      1
      x1^2
      x1 x2
      x1^3
      x1^2 x2
      x1 x2^2

      0
      0.994918
      0.826899
      0.04792
      0.021986
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

```python
# hide
ypred_x2 = mod.predict(newdf_test)
print_scores(dx2dt_test, ypred_x2)
```

    R2 score: 0.9965028656692735
    MSE score: 1.3128657613492703e-05
    MAE score: 0.003284761866157678
    Median AE score: 0.0036951175861815455
    MAPE score: 1.962931213051045
    SMAPE score: 0.9952953131151899

Identified ODE:

$dx_2/dt \sim x_1 - x_2^3$

with minor contributions from other terms that could be gotten rid of by applying a threshold

```python
# Drop features with absolute values less than 0.1
dx2_thr = fit_dx2[fit_dx2.columns[fit_dx2.abs().max() > 0.1]]
dx2_thr
```

      x1
      x2^3

      0
      0.994918
      -0.826899

```python
# hide_input
# PLOT results
t_test  = np.linspace(8, 10, 201)

#plt.plot(t, sol_new[:, 0], label='Actual', alpha = 0.4)
#plt.plot(ypred, label='Actual')
plt.plot(t_test, ypred_x2, 'r--', lw = 2, label='Prediction')
plt.plot(t_discover, dx2dt, 'g', label='Actual', alpha = 0.4)
plt.axvline(x = 8, color='k', linestyle='--')
plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$dx_2/dt$', fontsize=14)
plt.grid()
plt.legend()
# plt.savefig('dx2dt_fit.png', bbox_inches='tight')
```

## Forecasting using SINDy

Now that we have discovered the differential equations, we can use the ODE solver to forecast the future.

```python
# Manually entering values for coefficients but this can be automated
def fun_forecast(y, t):
    x1, x2 = y
    dxdt = [-0.5*(x1**3) - 0.9985*x2,
            0.995*x1     - 0.8269*x2**3]
    return dxdt

t_forecast = np.linspace(10, 15, 500)

y0 = [x1[-1], x2[-1]]

sol_forecast = odeint(fun_forecast, y0, t_forecast)
```

```python
# hide
plt.plot(t_forecast, sol_forecast[:, 0], 'b', label='x1(t)')
plt.plot(t_forecast, sol_forecast[:, 1], 'g', label='x2(t)')
plt.legend(loc='best')
plt.xlabel('t')
plt.grid()
```

```python
# hide_input
plt.plot(t_forecast, sol_forecast[:, 0], 'b--', label='Forecast')
plt.plot(t_full, sol[:, 0], 'g', label='Actual', alpha=0.4)
plt.axvline(x = 10, color='k', linestyle='--')
plt.legend(loc='best')
plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$x_1$', fontsize=14)
plt.grid()
```

```python
# hide_input
plt.plot(t_forecast, sol_forecast[:, 1], 'b--', label='Forecast')
plt.plot(t_full, sol[:, 1], 'g', label='Actual', alpha=0.4)
plt.axvline(x = 10, color='k', linestyle='--')
plt.legend(loc='best')
plt.xlabel('Time', fontsize=14)
plt.ylabel(r'$x_2$', fontsize=14)
plt.grid()
```

```python
# hide
y_pred1, y_pred2 = sol_forecast[:, 0], sol_forecast[:, 1]
y_test1, y_test2 = sol[1001:, 0],      sol[1001:, 1]
```

```python
# hide
for (y_test, y_pred) in zip([y_test1, y_test2], [y_pred1, y_pred2]):
    print(f"R2 score: {r2_score(y_test, y_pred)}")
    print(f"MSE score: {mean_squared_error(y_test, y_pred)}")
    print(f"MAE score: {mean_absolute_error(y_test, y_pred)}")
    print(f"Median AE score: {median_absolute_error(y_test, y_pred)}")
    print(f"MAPE score: {mean_absolute_percentage_error(y_test, y_pred)}")
    print(f"SMAPE score: {symmetric_mape(y_test, y_pred)}")
```

    R2 score: 0.9986014512322046
    MSE score: 2.9194295629861532e-05
    MAE score: 0.004498499556886438
    Median AE score: 0.004095611629683124
    MAPE score: 8.356195293001417
    SMAPE score: 3.427959986309787
    R2 score: 0.9988787480251635
    MSE score: 2.8082044451838518e-05
    MAE score: 0.004387715903581068
    Median AE score: 0.0032235369101030545
    MAPE score: 8.17055503299457
    SMAPE score: 3.0643658219157226

```python

```


# Characterization of Discrete Systems

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Comunications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Impulse Response

The concept of the so-called impulse response of discrete linear time-invariant (LTI) system and its connection to linear difference equations is introduced in the following.

### Output Signal

The response $y[k] = \mathcal{H} \{ x[k] \}$ of an LTI system to an arbitrary input signal $x[k]$ is derived. By applying the [sifting-property of the discrete Dirac impulse](../discrete_signals/standard_signals.ipynb#Dirac-Impulse), the input signal can be represented as

\begin{equation}
x[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \delta[k-\kappa]
\end{equation}

The output signal of the system is given by

\begin{equation}
y[k] = \mathcal{H} \left\{ \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \delta[k-\kappa] \right\}
\end{equation}

The summation and system response operator $\mathcal{H}$ can be exchanged under the assumption that the system is linear

\begin{equation}
y[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot \mathcal{H} \left\{ \delta[k-\kappa] \right\}
\end{equation}

where $\mathcal{H} \{\cdot\}$ was only applied to the Dirac impulse, since $x[\kappa]$ can be regarded as constant factor with respect to the index $k$.

The response of a system to a Dirac impulse as input is termed [*impulse response*](https://en.wikipedia.org/wiki/Impulse_response). It is defined as

\begin{equation}
h[k] = \mathcal{H} \left\{ \delta[k] \right\}
\end{equation}

If the system is time-invariant, the response to a shifted Diract impulse is $\mathcal{H} \left\{ \delta[k-\kappa]) \right\} = h[k-\kappa]$. Hence, for a discrete LTI system we finally get

\begin{equation}
y[k] = \sum_{\kappa = -\infty}^{\infty} x[\kappa] \cdot h[k-\kappa] = y[k] = x[k] * h[k]
\end{equation}

This operation is termed as linear [*convolution*](https://en.wikipedia.org/wiki/Convolution) and commonly abbreviated by $*$. The properties of an LTI system are entirely characterized by its impulse response. The response $y[k]$ of a system to an arbitrary input signal $x[k]$ is given by the convolution of the input signal $x[k]$ with its impulse response $h[k]$.

### Relation to Difference Equation

The impulse response $h[k] = \mathcal{H} \{ \delta[k] \}$ is the response of an LTI system to an Dirac impulse at the input. It can be derived from the coefficients of a [linear differential equation representing the LTI system](difference_equation.ipynb) by computing the output signal for the input signal $x[k] = \delta[k]$. Introducing this into the [solution of the difference equation](difference_equation.ipynb#Computation-of-the-Output-Signal) yields

\begin{equation}
h[k] = \frac{1}{a_0} \left( \sum_{m=0}^{M} b_m \; \delta[k-m] - \sum_{n=1}^{N} a_n \; h[k-n] \right)
\end{equation}

### Finite Impulse Response

Let's consider the case of a [non-recursive system](difference_equation.ipynb#Recursive-and-Non-Recursive-Systems) with $a_n = 0$ for $n > 0$. Without loss of generality it can be assumed that $a_0 = 1$, since $\frac{1}{a_0}$ can be incorporated into the other coefficients. The impulse response is given as

\begin{equation}
h[k] = \sum_{m=0}^{M} b_m \; \delta[k-m] = \begin{cases} b_k & \text{for } 0 \leq k < M \\ 0 & \text{otherwise} \end{cases}
\end{equation}

The impulse response of a non-recursive system is of finite length $M$. Its values are given by the coefficients $b_m$ of the system. An impulse response of finite length is commonly termed [finite impulse response (FIR)](https://en.wikipedia.org/wiki/Finite_impulse_response). The term FIR (system) is used synonymously to non-recursive system. The former relates to the length of the impulse response and the latter to the structure of the system. Note that the summation in above formula constitutes a convolution.

**Example**

According to above result, the impulse response of the [moving average filter](difference_equation.ipynb#Moving-Average) is given as

\begin{equation}
h[k] = \frac{1}{N} \cdot \text{rect}_N[k]
\end{equation}

As alternative to the [solution of the difference equation illustrated before](difference_equation.ipynb#Moving-Average), the output signal $y[k] = \mathcal{H} \{ y[k] \}$ is computed by convolution for the same input signal.

```python
%matplotlib inline
import numpy as np
from scipy import signal
import matplotlib.pyplot as plt

def rect(k, N):
    return np.where((k>=0) & (k<N), 1.0, 0.0)

np.random.seed(seed=0)

N = 10

k = np.arange(0, 60)
h = 1/N * rect(k, N)
x = np.cos(2*np.pi/30 * k) + .2 * np.random.normal(size=(len(k)))
y = np.convolve(x, h)

plt.figure(figsize=(10,4))
plt.subplot(121)
plt.stem(k, x)
plt.xlabel('$k$')
plt.ylabel(r'$x[k]$')
plt.ylim([-1.5, 1.5])

plt.subplot(122)
plt.stem(k, y[0:len(x)])
plt.xlabel('$k$')
plt.ylabel('$y[k]$')
plt.ylim([-1.5, 1.5])
plt.tight_layout()
```

**Exercise**

* Compare above output signal $y[k]$ derived by convolution with the output signal derived by [solution of the difference equation](difference_equation.ipynb#Moving-Average).

### Infinite Impulse Response

Now the general case of a [recursive system](difference_equation.ipynb#Recursive-and-Non-Recursive-Systems) is regarded. By inspection of above formula for the computation of the impulse response from the coefficients of a linear difference equation it becomes clear that the impulse response $h[k]$ for time-instant $k$ depends on the impulse response of past time instants $(k-1) \dots (k-N+1)$. This feedback generally results in an impulse response of infinite length. Such an impulse response is commonly termed as [infinite impulse response (IIR)](https://en.wikipedia.org/wiki/Infinite_impulse_response). The term IIR (system) is used synonymously to recursive system. The former relates to the length of the impulse response and the latter to the structure of the system.

For a finite-length input signal $x[k]$ the output signal of a recursive system cannot be computed by a linear convolution $y[k] = x[k] * h[k]$ in practice. This is due to the infinite length of its impulse response. As practical solution, the impulse response is often truncated to a finite length after its has decayed reasonably.

**Example**

The impulse response $h[k]$ of the previously introduced [second-order recursive LTI system](difference_equation.ipynb#Second-Order-System) with the difference equation

\begin{equation}
y[k] - y[k-1] + \frac{1}{2} y[k-2] = x[k]
\end{equation}

is derived by solution of the difference equation for a Dirac impulse as input signal. As can be deduced from its difference equation, the second-order recursive system has an IIR. Its first 256 samples are computed and plotted in the following. Note, less samples have been shown for ease of illustration.

```python
def dirac(k):
    return np.where(k==0, 1.0, 0.0)

a = [1.0, -1.0, 1/2]
b = [1.0]

k = np.arange(0, 256)
x = dirac(k)
h = signal.lfilter(b, a, x)

plt.figure(figsize=(6,3))
plt.stem(k, h)
plt.xlabel('$k$')
plt.ylabel(r'$h[k]$');
plt.axis([0, 20, -.5, 1.2]);
```

In order to illustrate the decay of the impulse response, its magnitude $A[k]$ is plotted in [decibel (dB)](https://en.wikipedia.org/wiki/Decibel)

\begin{equation}
A[k] = 20 \cdot \log_{10} ( |h[k]| ) \quad \text{in dB}
\end{equation}

```python
import warnings
warnings.filterwarnings('ignore', 'divide by zero encountered in log10')

plt.figure(figsize=(10,4))
plt.stem(k, 20*np.log10(np.abs(h)))
plt.xlabel('$k$')
plt.ylabel(r'$|h[k]|$ in dB')
plt.axis([0, k[-1], -800, 0])
plt.grid();
```

Its obvious that the magnitude of the impulse response has decayed to quite small values after 256 samples. The truncated impulse response can be used to calculate the output signal $y[k]$ by convolution with the input signal $x[k] = \text{rect}_{20}[k]$. The resulting output signal is plotted for illustration.

```python
def rect(k, N):
    return np.where((k>=0) & (k<N), 1.0, 0.0)

x = rect(k, 20)
y_ir = np.convolve(h, x)

plt.figure(figsize=(6,3))
plt.stem(k, y_ir[:len(k)])
plt.xlabel('$k$')
plt.ylabel(r'$y[k]$')
plt.axis([0, 40, -.7, 2.6]);
```

**Exercise**

* Compare the output signal computed by convolution with the one [computed by solution of the difference equation](difference_equation.ipynb#Second-Order-System).

* Investigate the effect of truncating an IIR on the output signal computed by convolution

    * Split the IIR $h[k]$ into two parts: a first part holding the samples of the truncated impulse response and second part holding the remaining samples
    * Split the output signal into to parts by splitting the convolution using the splitted impulse response

### Copyright

  To the extent possible under law,
  the person who associated CC0
  with this work has waived all copyright and related or neighboring
  rights to this work.


# How various formats can deal with LaTeX math

 **Hans Petter Langtangen**, Simula Research Laboratory and University of Oslo

Date: **May 29, 2016**

**Summary.** The purpose of this document is to test LaTeX math in DocOnce with
various output formats.  Most LaTeX math constructions are renedered
correctly by MathJax in plain HTML, but some combinations of
constructions may fail.  Unfortunately, only a subset of what works in
html MathJax also works in sphinx MathJax. The same is true for
markdown MathJax expresions (e.g., Jupyter notebooks).  Tests and
examples are provided to illustrate what may go wrong.

The recommendation for writing math that translates to MathJax in
html, sphinx, and markdown is to stick to the environments `\[
... \]`, `equation`, `equation*`, `align`, `align*`, `alignat`, and
`alignat*` only. Test the math with sphinx output; if it works in that
format, it should work elsewhere too.

The current version of the document is translated from DocOnce source
to the format **ipynb**.

# Test of equation environments

## Test 1: Inline math

We can get an inline equation
`$u(t)=e^{-at}$` rendered as $u(t)=e^{-at}$.

## Test 2: A single equation with label

An equation with number,

        !bt
        \begin{equation} u(t)=e^{-at} \label{eq1a}\end{equation}
        !et

looks like

$$
\begin{equation} u(t)=e^{-at} \label{eq1a} \tag{1}\end{equation}
$$

Maybe this multi-line version is what we actually prefer to write:

        !bt
        \begin{equation}
        u(t)=e^{-at}
        \label{eq1b}
        \end{equation}
        !et

The result is the same:

$$
\begin{equation}
u(t)=e^{-at} \label{eq1b} \tag{2}
\end{equation}
$$

We can refer to this equation through its label `eq1b`: [(2)](#eq1b).

## Test 3: Multiple, aligned equations without label and number

MathJax has historically had some problems with rendering many LaTeX
math environments, but the `align*` and `align` environments have
always worked.

        !bt
        \begin{align*}
        u(t)&=e^{-at}\\
        v(t) - 1 &= \frac{du}{dt}
        \end{align*}
        !et

Result:

$$
\begin{align*}
u(t)&=e^{-at}\\
v(t) - 1 &= \frac{du}{dt}
\end{align*}
$$

## Test 4: Multiple, aligned equations with label

Here, we use `align` with user-prescribed labels:

        !bt
        \begin{align}
        u(t)&=e^{-at}
        \label{eq2b}\\
        v(t) - 1 &= \frac{du}{dt}
        \label{eq3b}
        \end{align}
        !et

Result:

$$
\begin{equation}
u(t)=e^{-at}
\label{eq2b} \tag{3}
\end{equation}
$$

$$
\begin{equation}
v(t) - 1 = \frac{du}{dt}
\label{eq3b} \tag{4}
\end{equation}
$$

We can refer to the last equations as the system [(3)](#eq2b)-[(4)](#eq3b).

## Test 5: Multiple, aligned equations without label

In LaTeX, equations within an `align` environment is automatically
given numbers.  To ensure that an html document with MathJax gets the
same equation numbers as its latex/pdflatex companion, DocOnce
generates labels in equations where there is no label prescribed. For
example,

        !bt
        \begin{align}
        u(t)&=e^{-at}
        \\
        v(t) - 1 &= \frac{du}{dt}
        \end{align}
        !et

is edited to something like

        !bt
        \begin{align}
        u(t)&=e^{-at}
        \label{_auto5}\\
        v(t) - 1 &= \frac{du}{dt}
        \label{_auto6}
        \end{align}
        !et

and the output gets the two equation numbered.

$$
\begin{equation}
u(t)=e^{-at}
\label{_auto1} \tag{5}
\end{equation}
$$

$$
\begin{equation}
v(t) - 1 = \frac{du}{dt}
\label{_auto2} \tag{6}
\end{equation}
$$

## Test 6: Multiple, aligned equations with multiple alignments

The `align` environment can be used with two `&` alignment characters, e.g.,

        !bt
        \begin{align}
        \frac{\partial u}{\partial t} &= \nabla^2 u, & x\in (0,L),
        \ t\in (0,T]\\
        u(0,t) &= u_0(x), & x\in [0,L]
        \end{align}
        !et

The result in ipynb becomes

$$
\begin{equation}
\frac{\partial u}{\partial t} = \nabla^2 u,  x\in (0,L),
\ t\in (0,T]
\label{_auto3} \tag{7}
\end{equation}
$$

$$
\begin{equation}
u(0,t) = u_0(x),  x\in [0,L]
\label{_auto4} \tag{8}
\end{equation}
$$

A better solution is usually to use an `alignat` environment:

        !bt
        \begin{alignat}{2}
        \frac{\partial u}{\partial t} &= \nabla^2 u, & x\in (0,L),
        \ t\in (0,T]\\
        u(0,t) &= u_0(x), & x\in [0,L]
        \end{alignat}
        !et

with the rendered result

$$
\begin{equation}
\frac{\partial u}{\partial t} = \nabla^2 u,  x\in (0,L),
\ t\in (0,T]
\label{_auto5} \tag{9}
\end{equation}
$$

$$
\begin{equation}
u(0,t) = u_0(x),  x\in [0,L]
\label{_auto6} \tag{10}
\end{equation}
$$

**align/alignat environments with equation numbers are anti-aligned!**

In the `sphinx`, `ipynb`, and `pandoc` output formats, DocOnce rewrites
the equations in an `alignat` environment as individual equations in
`equation` environments (or more precisely, `sphinx` can work with
`alignat*` so only numbered `alignat` equations get rewritten as individual
equations). If the alignment is somewhat important, try the best with a
manual rewrite in terms of separate `equation` environments, and stick to
`align*` and `alignat*` in `sphinx`.

If DocOnce had not rewritten the above equations, they would be
rendered in ipynb as

$$
\begin{alignat}{2}
\frac{\partial u}{\partial t} &= \nabla^2 u, & x\in (0,L),
\ t\in (0,T]\\
u(0,t) &= u_0(x), & x\in [0,L]
\end{alignat}
$$

## Test 7: Multiple, aligned eqnarray equations without label

Let us try the old `eqnarray*` environment.

        !bt
        \begin{eqnarray*}
        u(t)&=& e^{-at}\\
        v(t) - 1 &=& \frac{du}{dt}
        \end{eqnarray*}
        !et

which results in

$$
\begin{eqnarray*}
u(t)&=& e^{-at}\\
v(t) - 1 &=& \frac{du}{dt}
\end{eqnarray*}
$$

## Test 8: Multiple, eqnarrayed equations with label

Here we use `eqnarray` with labels:

        !bt
        \begin{eqnarray}
        u(t)&=& e^{-at}
        \label{eq2c}\\
        v(t) - 1 &=& \frac{du}{dt}
        \label{eq3c}
        \end{eqnarray}
        !et

which results in

$$
\begin{eqnarray}
u(t)&=& e^{-at} \label{eq2c} \tag{11}\\
v(t) - 1 &=& \frac{du}{dt} \label{eq3c} \tag{12}
\end{eqnarray}
$$

Can we refer to the last equations as the system [(11)](#eq2c)-[(12)](#eq3c)
in the ipynb format?

## Test 9: The `multiline` environment with label and number

The LaTeX code

        !bt
        \begin{multline}
        \int_a^b f(x)dx = \sum_{j=0}^{n} \frac{1}{2} h(f(a+jh) +
        f(a+(j+1)h)) \\
        =\frac{h}{2}f(a) + \frac{h}{2}f(b) + \sum_{j=1}^n f(a+jh)
        \label{multiline:eq1}
        \end{multline}
        !et

gets rendered as

$$
\begin{multline}
\int_a^b f(x)dx = \sum_{j=0}^{n} \frac{1}{2} h(f(a+jh) +
f(a+(j+1)h)) \\
=\frac{h}{2}f(a) + \frac{h}{2}f(b) + \sum_{j=1}^n f(a+jh)
\label{multiline:eq1} \tag{13}
\end{multline}
$$

and we can hopefully refer to the Trapezoidal rule
as the formula [(13)](#multiline:eq1).

## Test 10: Splitting equations using a split environment

Although `align` can be used to split too long equations, a more obvious
command is `split`:

        !bt
        \begin{equation}
        \begin{split}
        \int_a^b f(x)dx = \sum_{j=0}^{n} \frac{1}{2} h(f(a+jh) +
        f(a+(j+1)h)) \\
        =\frac{h}{2}f(a) + \frac{h}{2}f(b) + \sum_{j=1}^n f(a+jh)
        \end{split}
        \end{equation}
        !et

The result becomes

$$
\begin{equation}
\begin{split}
\int_a^b f(x)dx = \sum_{j=0}^{n} \frac{1}{2} h(f(a+jh) +
f(a+(j+1)h)) \\
=\frac{h}{2}f(a) + \frac{h}{2}f(b) + \sum_{j=1}^n f(a+jh)
\end{split}
\label{_auto7} \tag{14}
\end{equation}
$$

## Test 11: Newcommands and boldface bm vs pmb

First we use the plain old pmb package for bold math. The formula

        !bt
        \[ \frac{\partial\u}{\partial t} +
        \u\cdot\nabla\u = \nu\nabla^2\u -
        \frac{1}{\varrho}\nabla p,\]
        !et

and the inline expression `$\nabla\pmb{u} (\pmb{x})\cdot\pmb{n}$`
(with suitable newcommands using pmb)
get rendered as

$$
\frac{\partial\pmb{u}}{\partial t} +
\pmb{u}\cdot\nabla\pmb{u} = \nu\nabla^2\pmb{u} -
\frac{1}{\varrho}\nabla p,
$$

and $\nabla\pmb{u} (\pmb{x})\cdot\pmb{n}$.

Somewhat nicer fonts may appear with the more modern `\bm` command:

        !bt
        \[ \frac{\partial\ubm}{\partial t} +
        \ubm\cdot\nabla\ubm = \nu\nabla^2\ubm -
        \frac{1}{\varrho}\nabla p,\]
        !et

(backslash `ubm` is a newcommand for bold math $u$), for which we get

$$
\frac{\partial\boldsymbol{u}}{\partial t} +
\boldsymbol{u}\cdot\nabla\boldsymbol{u} = \nu\nabla^2\boldsymbol{u} -
\frac{1}{\varrho}\nabla p.
$$

Moreover,

        $\nabla\boldsymbol{u}(\boldsymbol{x})\cdot\boldsymbol{n}$

becomes $\nabla\boldsymbol{u}(\boldsymbol{x})\cdot\boldsymbol{n}$.

**Warning.**

Note: for the ipynb format, `\bm` was substituted by DocOnce
to `\boldsymbol`.

# Problematic equations

Finally, we collect some problematic formulas in MathJax. They all work
fine in LaTeX. Most of them look fine in html too, but some fail in
sphinx, ipynb, or markdown.

## Colored terms in equations

The LaTeX code

        !bt
        \[ {\color{blue}\frac{\partial\u}{\partial t}} +
        \nabla\cdot\nabla\u = \nu\nabla^2\u -
        \frac{1}{\varrho}\nabla p,\]
        !et

results in

$$
{\color{blue}\frac{\partial\pmb{u}}{\partial t}} +
\nabla\cdot\nabla\pmb{u} = \nu\nabla^2\pmb{u} -
\frac{1}{\varrho}\nabla p,
$$

but correct rendering in ipynb requires omitting the `\color` command:

$$
\frac{\partial\pmb{u}}{\partial t} +
\nabla\cdot\nabla\pmb{u} = \nu\nabla^2\pmb{u} -
\frac{1}{\varrho}\nabla p,
$$

## Bar over symbols

Sometimes one must be extra careful with the LaTeX syntax to get sphinx MathJax
to render a formula correctly. Consider the combination of a bar over a
bold math symbol:

        !bt
        \[ \bar\f = f_c^{-1}\f,\]
        !et

which for ipynb output results in

$$
\bar\boldsymbol{f} = f_c^{-1}\boldsymbol{f}.
$$

With sphinx, this formula is not rendered. However, using curly braces for the bar,

        !bt
        \[ \bar{\f} = f_c^{-1}\f,\]
        !et

makes the output correct also for sphinx:

$$
\bar{\boldsymbol{f}} = f_c^{-1}\boldsymbol{f},
$$

## Matrix formulas

Here is an `align` environment with a label and the `pmatrix`
environment for matrices and vectors in LaTeX.

        !bt
        \begin{align}
        \begin{pmatrix}
        G_2 + G_3 & -G_3 & -G_2 & 0 \\
        -G_3 & G_3 + G_4 & 0 & -G_4 \\
        -G_2 & 0 & G_1 + G_2 & 0 \\
        0 & -G_4 & 0 & G_4
        \end{pmatrix}
        &=
        \begin{pmatrix}
        v_1 \\
        v_2 \\
        v_3 \\
        v_4
        \end{pmatrix}
        + \cdots
        \label{mymatrixeq}\\
        \begin{pmatrix}
        C_5 + C_6 & -C_6 & 0 & 0 \\
        -C_6 & C_6 & 0 & 0 \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0
        \end{pmatrix}
        \frac{d}{dt} &=
        \begin{pmatrix}
        v_1 \\
        v_2 \\
        v_3 \\
        v_4
        \end{pmatrix} =
        \begin{pmatrix}
        0 \\
        0 \\
        0 \\
        -i_0
        \end{pmatrix}
        \end{align}
        !et

which becomes

$$
\begin{equation}
\begin{pmatrix}
G_2 + G_3 & -G_3 & -G_2 & 0 \\
-G_3 & G_3 + G_4 & 0 & -G_4 \\
-G_2 & 0 & G_1 + G_2 & 0 \\
0 & -G_4 & 0 & G_4
\end{pmatrix}
=
\begin{pmatrix}
v_1 \\
v_2 \\
v_3 \\
v_4
\end{pmatrix}
+ \cdots
\label{mymatrixeq} \tag{15}
\end{equation}
$$

$$
\begin{equation}
\begin{pmatrix}
C_5 + C_6 & -C_6 & 0 & 0 \\
-C_6 & C_6 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix}
\frac{d}{dt} =
\begin{pmatrix}
v_1 \\
v_2 \\
v_3 \\
v_4
\end{pmatrix} =
\begin{pmatrix}
0 \\
0 \\
0 \\
-i_0
\end{pmatrix}
\label{_auto8} \tag{16}
\end{equation}
$$

The first equation do not render correctly in ipynb.

The same matrices without labels in an `align*` environment:

        !bt
        \begin{align*}
        \begin{pmatrix}
        G_2 + G_3 & -G_3 & -G_2 & 0 \\
        -G_3 & G_3 + G_4 & 0 & -G_4 \\
        -G_2 & 0 & G_1 + G_2 & 0 \\
        0 & -G_4 & 0 & G_4
        \end{pmatrix}
        &=
        \begin{pmatrix}
        v_1 \\
        v_2 \\
        v_3 \\
        v_4
        \end{pmatrix}
        + \cdots \\
        \begin{pmatrix}
        C_5 + C_6 & -C_6 & 0 & 0 \\
        -C_6 & C_6 & 0 & 0 \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0
        \end{pmatrix}
        \frac{d}{dt} &=
        \begin{pmatrix}
        v_1 \\
        v_2 \\
        v_3 \\
        v_4
        \end{pmatrix} =
        \begin{pmatrix}
        0 \\
        0 \\
        0 \\
        -i_0
        \end{pmatrix}
        \end{align*}
        !et

The rendered result becomes

$$
\begin{align*}
\begin{pmatrix}
G_2 + G_3 & -G_3 & -G_2 & 0 \\
-G_3 & G_3 + G_4 & 0 & -G_4 \\
-G_2 & 0 & G_1 + G_2 & 0 \\
0 & -G_4 & 0 & G_4
\end{pmatrix}
&=
\begin{pmatrix}
v_1 \\
v_2 \\
v_3 \\
v_4
\end{pmatrix}
+ \cdots \\
\begin{pmatrix}
C_5 + C_6 & -C_6 & 0 & 0 \\
-C_6 & C_6 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0
\end{pmatrix}
\frac{d}{dt} &=
\begin{pmatrix}
v_1 \\
v_2 \\
v_3 \\
v_4
\end{pmatrix} =
\begin{pmatrix}
0 \\
0 \\
0 \\
-i_0
\end{pmatrix}
\end{align*}
$$