[@ggruszczynski](https://github.com/ggruszczynski)

# LBM - some theory

In this tutorial the reader will get the idea of:

* symbolic code generation
* a bit more advanced LBM models

It will be shown how the moments of equilibrium distribution function and the transformation matrices can be calculated.
Let us begin with some repetition for the newcomers...

### Repetition: Discrete Boltzmann equation

$$
  \underbrace{ f_i(\boldsymbol{x} + \boldsymbol{e_i} \Delta {\boldsymbol{x}}, t +  \Delta {t} ) }_{Streaming} =
  \underbrace{ f_i(\boldsymbol{x}, t ) - \frac{1}{\tau } ( f_i - f_i^{eq}) + F_i(\boldsymbol{x}, t ) }_{Collision} \nonumber
$$

where:
* $\tau = \tau(\nu)$ relaxation parameter, $\nu$ is the kinematic viscosity
* $f_i$ - discrete probability distribution function
* $F_i$ - source term (ex. gravity force)

### Repetition: Algorithm

1. Initialize $ \enspace f_i ^{in} $

2. Compute density
$ \rho = \sum_{i=0}^{8} f_i ^{in}(\boldsymbol{x},t)$
 and velocity
$ \boldsymbol{u}(\boldsymbol{x},t) = \frac{1}{\rho} \sum_{i=0}^{8} \, f_i ^{in}(\boldsymbol{x},t) \boldsymbol{e}_i +  \frac{\textbf{F}}{2 \rho} \delta t  $

3. Compute equilibrium distribution function
$  f_i ^{eq}(\boldsymbol{x},t) =  w_i \rho
 \left[ 1 + \frac{\boldsymbol{e}_i \boldsymbol{u}}{c_s^2 e^2} + \frac{ (\boldsymbol{e}_i \boldsymbol{u})^2}{2 c_s^4 e^4} - \frac{\boldsymbol{u}^2 } {2c_s^2 e^2} \right] $

4. Collision
$  f_i ^{out}(\boldsymbol{x},t) = f_i^{in}(\boldsymbol{x},t) - \frac{1}{\tau_f} \bigg[ f_i^{in}(\boldsymbol{x},t) - f_i^{eq}(\boldsymbol{x},t) \bigg] + F_i(\boldsymbol{x}, t ) $

5. Streaming
$  f_i ^{in}(\boldsymbol{x} + \boldsymbol{e}_i ,t+1) =  f_i^{out} (\boldsymbol{x},t) $

### Repetition: moments

$$
m_0 = M = \int r^0 \rho(r) d \Omega \\
m_1 = \mu = \frac{1}{M}\int r^1 \rho(r) d \Omega \\
m_2 = I_{zz'} = \int r^2 \rho(r) d \Omega \\
\sigma ^2 =  I_{xx'}\int (r - \mu)^2 \rho(r) d \Omega
$$

### The raw moments and central moments in LBM
$$
\kappa_{mn} = \sum_{i}(e_{i, x})^m ( e_{i, y})^n f_{i} \\
\tilde{\kappa}_{mn} = \sum_{i} ( e_{i, x} - u_x)^m ( e_{i, y} - u_y)^n f_{i}
$$
Physical interpretation:
$$
\rho = \kappa_{00} = \sum_i f_i \nonumber % \hspace{2em} \text{- normalized pressure}
 \\
\rho \textbf{u} = \rho [ u_x, u_y]^\top = [ \kappa_{10}, \kappa_{01}]^\top
= \sum_i f_i \textbf{e}_i + \frac{\textbf{F}}{2} \delta t
$$

### Algorithm - revisited

1. Initialize $ \enspace f_i ^{in} $

2. Compute velocity
$\textbf{u} = [u_x, u_y]^\top = [ \kappa_{10}, \kappa_{01}]^\top
= \dfrac{1}{\rho} \sum_i f_i \textbf{e}_i + \frac{\textbf{F}}{2 \rho} \delta t \nonumber  $

3. Compute (central) moments

$ \boldsymbol{\tilde{\Upsilon}}(\boldsymbol{x},t) =\mathbb{N}\mathbb{M} \textbf{f}(\boldsymbol{x},t) \\
 \boldsymbol{\tilde{\Upsilon}}^{eq} =
    [\rho,
     0,
     0,
     c_s^2 \rho,
     c_s^2 \rho,
     0,
     0,
     0,
     \sigma^2 \rho] ^\top \\
\tilde{\boldsymbol{F}}  =
[
     0,
     F_x /\rho ,
     F_y /\rho ,
     0,
     0,
     0,
     c_s^2 F_y /\rho ,
     c_s^2 F_x /\rho ,
     0]^\top \nonumber
$

4. Collision
$  \boldsymbol{\tilde{\Upsilon}}(\textbf{x}, t + \delta t ) = \boldsymbol{\tilde{\Upsilon}}  - \mathbb{S} (\boldsymbol{\tilde{\Upsilon}} - \boldsymbol{\tilde{\Upsilon}}^{eq}) + (\mathbb{1} - \mathbb{S}/2)\tilde{\textbf{F}} $

5. Streaming
$  f_i(\textbf{x} + \textbf{e}\delta t, t + \delta t )
 =
\mathbb{M}^{-1} \mathbb{N}^{-1} \boldsymbol{\tilde{\Upsilon}}_{i}(\textbf{x}, t + \delta t )  $

## Moments of Equilibrium Distribution Function

Now, we are going to calculate the (central) moments of equilibrium distribution function.

The formulas for the discrete equilibrium distribution function $ f^{eq}_i $
comes from a discretization of the continous Maxwell-Boltzmann distribution function.
The Maxwell-Boltzmann equilibrium distribution function in a continuous, velocity space is known as:

$$
\Psi^{\textit{M-B, eq}} =
\Psi^{\textit{M-B, eq}}(\psi, \boldsymbol{\xi}, \boldsymbol{u}) =
\dfrac{\psi}{(2 \pi c_s^2)^{D/2}}
exp \left[
-\frac{(\boldsymbol{\xi}-\boldsymbol{u})^2}{2 c_s^2}
\right]
$$

Where $ \psi $  is the quantity of interest (like fluid density or enthalpy), $c_s^2$ is the lattice speed of sound (aka variance of the distribution) and $ D $ is the number of dimensions.
The continuous definition of the central moments is:

$$
\tilde{\kappa}_{mn} = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty}
(\xi_x - u_x)^m (\xi_y -u_y)^n
\Psi(\psi, \boldsymbol{\xi}, \boldsymbol{u})
d \xi_x d \xi_y
$$

```python
from sympy import Symbol, exp, pi, integrate, oo
from sympy import simplify, Float, preorder_traversal
from sympy.matrices import Matrix, eye, diag
from sympy.interactive.printing import init_printing
from sympy import ccode
import sympy as sp
import numpy as np

# init_printing()
```

```python
ex_D2Q9 = Matrix([0, 1, 0, -1, 0, 1, -1, 1, -1])
ey_D2Q9 = Matrix([0, 0, 1, 0, -1, 1, 1, -1, -1])

# Let us choose the following order of moments
# one can denote the variables as f[0], f[1], f[2], f[3]...
# or f_00, f_10, f_01, f_20
# We will use the latter notation.
# observe that f[3]=f_20. It is streamed from direction e[-1,0].

order_of_moments = [
    (0, 0),
    (1, 0),
    (0, 1),
    (2, 0),
    (0, 2),
    (1, 1),
    (2, 1),
    (1, 2),
    (2, 2)]

dzeta_x = Symbol('dzeta_x', real=True)
dzeta_y = Symbol('dzeta_y', real=True)

dzeta2D = Matrix([dzeta_x, dzeta_y])

ux = Symbol('u.x')  # don't set real=True for velocity as it freezes the test suite :/
uy = Symbol('u.y')

u2D = Matrix([ux, uy])

# rho = Symbol(r'\rho', positive=True)
# cs2 = Symbol(r'\sigma', positive=True)

rho = Symbol('rho', positive=True)
cs2 = 1./3.
```

```python
def round_and_simplify(stuff):
    simplified_stuff = simplify(stuff)
    rounded_stuff = simplified_stuff

    for a in preorder_traversal(rounded_stuff):
        if isinstance(a, Float):
            rounded_stuff = rounded_stuff.subs(a, round(a, 10))

    rounded_and_simplified_stuff = simplify(rounded_stuff)
    return rounded_and_simplified_stuff
```

## Task

Fill the body of `get_Maxwellian_DF` function and run the script to calculate (central) moments.

```python
class ContinuousCMTransforms:
    def __init__(self, dzeta, u, rho, cs2):
        """
        :param dzeta: direction (x,y,z)
        :param u: velocity (x,y,z) i.e., mean of the distribution
        :param rho: density (not necessarily m00, for instance in multiphase flows)
        :param cs2: variance of the distribution = (speed of sound)^2,
                    for isothermal LB cs2=1./3;
                    otherwise  cs2 = Symbol('RT', positive=True)

        """
        self.dzeta = dzeta
        self.u = u
        self.rho = rho
        self.sigma2 = cs2

    def get_Maxwellian_DF(self):
        """
        :return: continuous, local Maxwell-Boltzmann distribution
        """
        dzeta_minus_u = self.dzeta - self.u
        dzeta_u2 = dzeta_minus_u.dot(dzeta_minus_u)

        # df = self.rho / pow(2 * sp.pi * self.cs2, 2 / 2)  # this is to difficult for sympy :/
        df = self.rho / (2 * sp.pi * self.sigma2)  # 2D version hack

        df *= exp(-dzeta_u2 / (2 * self.sigma2))
        return df

    def get_m(self, mno):
        fun = self.get_Maxwellian_DF()
        for dzeta_i, mno_i in zip(self.dzeta, mno):
            fun *= pow(dzeta_i, mno_i)

        lim = [(dim, -oo, oo) for dim in self.dzeta]
        result = integrate(fun, *lim)
        return round_and_simplify(result)

    def get_cm(self, mno):
        fun = self.get_Maxwellian_DF()
        for dzeta_i, u_i, mno_i in zip(self.dzeta, self.u, mno):
            fun *= pow((dzeta_i - u_i), mno_i)

        lim = [(dim, -oo, oo) for dim in self.dzeta]
        result = integrate(fun, *lim)
        return round_and_simplify(result)

```

```python
# here the zeroth moment is calculated
ccmt = ContinuousCMTransforms(dzeta2D, u2D, rho=rho, cs2=cs2)
row0 = order_of_moments[0]
moment0 = ccmt.get_cm(row0)

moment0
```

$\displaystyle 1.0 \rho$

```python
# write a line of code to calculate the whole vector of moments

m_eq = Matrix([ccmt.get_m(row) for row in order_of_moments])
m_eq
```

$\displaystyle \left[\begin{matrix}1.0 \rho\\1.0 \rho u.x\\1.0 \rho u.y\\\rho \left(1.0 u.x^{2} + 0.3333333333\right)\\\rho \left(1.0 u.y^{2} + 0.3333333333\right)\\1.0 \rho u.x u.y\\\rho u.y \left(1.0 u.x^{2} + 0.3333333333\right)\\\rho u.x \left(1.0 u.y^{2} + 0.3333333333\right)\\\rho \left(1.0 u.x^{2} u.y^{2} + 0.3333333333 u.x^{2} + 0.3333333333 u.y^{2} + 0.1111111111\right)\end{matrix}\right]$

```python
# and the vector of central moments

cm_eq = Matrix([ccmt.get_cm(row) for row in order_of_moments])
cm_eq
```

$\displaystyle \left[\begin{matrix}1.0 \rho\\0\\0\\0.3333333333 \rho\\0.3333333333 \rho\\0\\0\\0\\0.1111111111 \rho\end{matrix}\right]$

```python
# next, print is as 'C' code

def print_code(order_of_moments, lhs,rhs):
    for moment, expr in zip(order_of_moments, rhs):
        mstr = [str(m) for m in moment]
        mstr = ''.join(mstr)
        print(f"double {lhs}_{mstr} = {ccode(expr)};")

print_code(order_of_moments, "cm_eq", cm_eq)
```

    double cm_eq_00 = 1.0*rho;
    double cm_eq_10 = 0;
    double cm_eq_01 = 0;
    double cm_eq_20 = 0.33333333329937886*rho;
    double cm_eq_02 = 0.33333333329937886*rho;
    double cm_eq_11 = 0;
    double cm_eq_21 = 0;
    double cm_eq_12 = 0;
    double cm_eq_22 = 0.11111111110039928*rho;

## Moments of non-equlibrium Distribution Function

The discrete distribution function are streamed along the lattice links, which are defined by a set of discrete velocities,$\textbf{e}$.
Using the Euleran basis and a D2Q9 space, the discrete velocities read,

$$
\textbf{e} = [\textbf{e}_x, \textbf{e}_y], \\
\textbf{e}_x = [0,1,0,-1,0,1,-1,-1,1]^\top, \\
\textbf{e}_y = [0,0,1,0,-1,1,1,-1,-1]^\top, \\
$$

The discrete, raw and central moments are introduced based on the work of Geier et al. [^5] as,

$$ k_{mn} = \sum_{\alpha}(e_{\alpha x})^m ( e_{\alpha y})^n \Psi_{\alpha} $$

while the central moments are calculated in a moving reference frame i.e., with respect to the fluid velocity:

$$ \tilde{k}_{mn} = \sum_{\alpha} ( e_{\alpha x} - u_x)^m ( e_{\alpha y} - u_y)^n \Psi_{\alpha} $$

where $ \Psi_{\alpha} $ is the distribution function of interest (for example hydrodynamic or enthalpy).

Notice, that the equations can be expressed by matrix transformations [^1][^2][^3][^4].

$$
\boldsymbol{\Upsilon} = \mathbb{M} \boldsymbol{\Psi} \\
\boldsymbol{\tilde{\Upsilon}} = \mathbb{N} \boldsymbol{\Upsilon} = \underbrace{\mathbb{N} \mathbb{M}}_{\mathbb{T}} \boldsymbol{\Psi}
$$

where $\boldsymbol{\Upsilon}$ and $\boldsymbol{\tilde{\Upsilon}}$ denote the raw and central moments, respectively.
From the computational point of view, it is preferred to perform the transformations in two steps as in above (without explicit usage of the $\mathbb{T}$ matrix).

Rows of the transformation matrices are calculated analogously to $k$ and $\tilde{k}$,
$$
M_{mn} = [ (\textbf{e}_x)^m (\textbf{e}_y)^n ]^\top, \\
T_{mn} = [ (\textbf{e}_x - \mathbb{1} u_x)^m (\textbf{e}_y - \mathbb{1} u_y)^n ]
$$
Then, the matrices are assembled row by row as,

$$
\mathbb{M}
 =
 \left[
 M_{00},
 M_{10},
 M_{01},
 M_{20},
 M_{02},
 M_{11},
 M_{10},
 M_{01},
 M_{22}
 \right]
  \\
\mathbb{T} =
 \left[
 T_{00},
 T_{10},
 T_{01},
 T_{20},
 T_{02},
 T_{11},
 T_{10},
 T_{01},
 T_{22}
 \right]
$$

The $\mathbb{N}$ matrix can be found as $\mathbb{N} = \mathbb{T} \mathbb{M}^{-1} $.

Observe that $ \mathbb{M} $ is a fixed matrix while $ \mathbb{N} $ depends on the fluid velocity, $ \textbf{u} $.

Finally, the set of the central moments can be expressed in vector form as,

$$
\boldsymbol{\tilde{\Upsilon}} =
[\tilde{k}_{00}, \tilde{k}_{10}, \tilde{k}_{01}, \tilde{k}_{20}, \tilde{k}_{02}, \tilde{k}_{11}, \tilde{k}_{21}, \tilde{k}_{12}, \tilde{k}_{22}]^\top.
$$

The physical interpretation of the raw, zeroth and first order moments of the hydrodynamic  DF corresponds to the values of density, $ \rho $ and momentum $  \rho \textbf{u} $.

```python

class MatrixGenerator:
    def __init__(self, ex, ey, order_of_moments):
        self.ex = ex
        self.ey = ey
        self.order_of_moments = order_of_moments

    def __matrix_maker(self, row_maker_fun):
        M = [row_maker_fun(*row) for row in self.order_of_moments]
        return M

    def get_raw_moments_matrix(self):
        """
        :return: transformation matrix from DF to raw moments
        """

        def get_row(m, n):
            row = [pow((self.ex[i]), m) * pow((self.ey[i]), n)  for i in range(0, 9)]
            return row

        m_ = self.__matrix_maker(get_row)
        # M = [get_row(*row) for row in self.order_of_moments] # same as
        return Matrix(m_)

    def get_T_matrix(self):
        """
        :return: transformation matrix from DF to central moments
        """

        def get_row(m, n):
            row = [pow((self.ex[i] - ux), m) * pow((self.ey[i] - uy), n)  for i in range(0, 9)]
            row = [round_and_simplify(r) for r in row] # simplify the elements in each row
            return row

        m_ = self.__matrix_maker(get_row)
        return Matrix(m_)

```

```python
matrixGenerator = MatrixGenerator(ex_D2Q9, ey_D2Q9, order_of_moments)

Mraw = matrixGenerator.get_raw_moments_matrix()
Mraw
```

$\displaystyle \left[\begin{matrix}1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1\\0 & 1 & 0 & -1 & 0 & 1 & -1 & 1 & -1\\0 & 0 & 1 & 0 & -1 & 1 & 1 & -1 & -1\\0 & 1 & 0 & 1 & 0 & 1 & 1 & 1 & 1\\0 & 0 & 1 & 0 & 1 & 1 & 1 & 1 & 1\\0 & 0 & 0 & 0 & 0 & 1 & -1 & -1 & 1\\0 & 0 & 0 & 0 & 0 & 1 & 1 & -1 & -1\\0 & 0 & 0 & 0 & 0 & 1 & -1 & 1 & -1\\0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 1\end{matrix}\right]$

```python
Traw = matrixGenerator.get_T_matrix()
Nraw = Traw * Mraw.inv()

Nraw = Matrix([round_and_simplify(Nraw[i,:]) for i in range(9)])
Nraw
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\- u.x & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\- u.y & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0\\u.x^{2} & - 2 u.x & 0 & 1 & 0 & 0 & 0 & 0 & 0\\u.y^{2} & 0 & - 2 u.y & 0 & 1 & 0 & 0 & 0 & 0\\u.x u.y & - u.y & - u.x & 0 & 0 & 1 & 0 & 0 & 0\\- u.x^{2} u.y & 2 u.x u.y & u.x^{2} & - u.y & 0 & - 2 u.x & 1 & 0 & 0\\- u.x u.y^{2} & u.y^{2} & 2 u.x u.y & 0 & - u.x & - 2 u.y & 0 & 1 & 0\\u.x^{2} u.y^{2} & - 2 u.x u.y^{2} & - 2 u.x^{2} u.y & u.y^{2} & u.x^{2} & 4 u.x u.y & - 2 u.y & - 2 u.x & 1\end{matrix}\right]$

## Task
We have just generate the matrix of transformation.
Now, let as create the vector of variables which are going to be transformed.
Implement the `get_symbols` function. It shall return a vector (1-D Matrix, i.e. `Matrix([stuff])` ) having the following form $ [f_{00}, f_{10}, f_{01}, f_{20}, f_{02}, etc...] $

```python

def get_symbols(name, directions):
    print_symbols = []

    for direction in directions:
        direction = [str(d) for d in direction]
        direction = ''.join(direction)
        print_symbols.append(f"{name}_{direction}")

    return Matrix(print_symbols)

fs = get_symbols("f", order_of_moments)
fs
```

$\displaystyle \left[\begin{matrix}f_{00}\\f_{10}\\f_{01}\\f_{20}\\f_{02}\\f_{11}\\f_{21}\\f_{12}\\f_{22}\end{matrix}\right]$

```python
m = Mraw * fs
m
```

$\displaystyle \left[\begin{matrix}f_{00} + f_{01} + f_{02} + f_{10} + f_{11} + f_{12} + f_{20} + f_{21} + f_{22}\\f_{10} + f_{11} + f_{12} - f_{20} - f_{21} - f_{22}\\f_{01} - f_{02} + f_{11} - f_{12} + f_{21} - f_{22}\\f_{10} + f_{11} + f_{12} + f_{20} + f_{21} + f_{22}\\f_{01} + f_{02} + f_{11} + f_{12} + f_{21} + f_{22}\\f_{11} - f_{12} - f_{21} + f_{22}\\f_{11} - f_{12} + f_{21} - f_{22}\\f_{11} + f_{12} - f_{21} - f_{22}\\f_{11} + f_{12} + f_{21} + f_{22}\end{matrix}\right]$

```python
print("//raw moments from density-probability functions")
print_code(order_of_moments, "m", m)
```

    //raw moments from density-probability functions
    double m_00 = f_00 + f_01 + f_02 + f_10 + f_11 + f_12 + f_20 + f_21 + f_22;
    double m_10 = f_10 + f_11 + f_12 - f_20 - f_21 - f_22;
    double m_01 = f_01 - f_02 + f_11 - f_12 + f_21 - f_22;
    double m_20 = f_10 + f_11 + f_12 + f_20 + f_21 + f_22;
    double m_02 = f_01 + f_02 + f_11 + f_12 + f_21 + f_22;
    double m_11 = f_11 - f_12 - f_21 + f_22;
    double m_21 = f_11 - f_12 + f_21 - f_22;
    double m_12 = f_11 + f_12 - f_21 - f_22;
    double m_22 = f_11 + f_12 + f_21 + f_22;

```python
ms = get_symbols("m", order_of_moments)
cm = Nraw * ms
cm
```

$\displaystyle \left[\begin{matrix}m_{00}\\- m_{00} u.x + m_{10}\\- m_{00} u.y + m_{01}\\m_{00} u.x^{2} - 2 m_{10} u.x + m_{20}\\m_{00} u.y^{2} - 2 m_{01} u.y + m_{02}\\m_{00} u.x u.y - m_{01} u.x - m_{10} u.y + m_{11}\\- m_{00} u.x^{2} u.y + m_{01} u.x^{2} + 2 m_{10} u.x u.y - 2 m_{11} u.x - m_{20} u.y + m_{21}\\- m_{00} u.x u.y^{2} + 2 m_{01} u.x u.y - m_{02} u.x + m_{10} u.y^{2} - 2 m_{11} u.y + m_{12}\\m_{00} u.x^{2} u.y^{2} - 2 m_{01} u.x^{2} u.y + m_{02} u.x^{2} - 2 m_{10} u.x u.y^{2} + 4 m_{11} u.x u.y - 2 m_{12} u.x + m_{20} u.y^{2} - 2 m_{21} u.y + m_{22}\end{matrix}\right]$

```python
print("//central moments from raw moments")
print_code(order_of_moments, "cm", cm)
```

    //central moments from raw moments
    double cm_00 = m_00;
    double cm_10 = -m_00*u.x + m_10;
    double cm_01 = -m_00*u.y + m_01;
    double cm_20 = m_00*pow(u.x, 2) - 2*m_10*u.x + m_20;
    double cm_02 = m_00*pow(u.y, 2) - 2*m_01*u.y + m_02;
    double cm_11 = m_00*u.x*u.y - m_01*u.x - m_10*u.y + m_11;
    double cm_21 = -m_00*pow(u.x, 2)*u.y + m_01*pow(u.x, 2) + 2*m_10*u.x*u.y - 2*m_11*u.x - m_20*u.y + m_21;
    double cm_12 = -m_00*u.x*pow(u.y, 2) + 2*m_01*u.x*u.y - m_02*u.x + m_10*pow(u.y, 2) - 2*m_11*u.y + m_12;
    double cm_22 = m_00*pow(u.x, 2)*pow(u.y, 2) - 2*m_01*pow(u.x, 2)*u.y + m_02*pow(u.x, 2) - 2*m_10*u.x*pow(u.y, 2) + 4*m_11*u.x*u.y - 2*m_12*u.x + m_20*pow(u.y, 2) - 2*m_21*u.y + m_22;

```python
# RELAXATION MATRIX
omega_v = Symbol('omega_nu', positive=True)
omega_b = Symbol('omega_bulk', positive=True)

s_plus_D2Q9 = (omega_b + omega_v) / 2
s_minus_D2Q9 = (omega_b - omega_v) / 2

S_relax_hydro_D2Q9 = diag(1, 1, 1, s_plus_D2Q9, s_plus_D2Q9, omega_v, 1, 1, 1)
S_relax_hydro_D2Q9[3, 4] = s_minus_D2Q9
S_relax_hydro_D2Q9[4, 3] = s_minus_D2Q9
```

```python
cm_after_collision = eye(9) * cm + S_relax_hydro_D2Q9 * (cm_eq - cm)
print("//collision in central moments space")
print_code(order_of_moments, "cm_after_collision", cm_after_collision)
```

    //collision in central moments space
    double cm_after_collision_00 = 1.0*rho;
    double cm_after_collision_10 = 0;
    double cm_after_collision_01 = 0;
    double cm_after_collision_20 = m_00*pow(u.x, 2) - 2*m_10*u.x + m_20 + ((1.0/2.0)*omega_bulk - 1.0/2.0*omega_nu)*(-m_00*pow(u.y, 2) + 2*m_01*u.y - m_02 + 0.33333333329937886*rho) + ((1.0/2.0)*omega_bulk + (1.0/2.0)*omega_nu)*(-m_00*pow(u.x, 2) + 2*m_10*u.x - m_20 + 0.33333333329937886*rho);
    double cm_after_collision_02 = m_00*pow(u.y, 2) - 2*m_01*u.y + m_02 + ((1.0/2.0)*omega_bulk - 1.0/2.0*omega_nu)*(-m_00*pow(u.x, 2) + 2*m_10*u.x - m_20 + 0.33333333329937886*rho) + ((1.0/2.0)*omega_bulk + (1.0/2.0)*omega_nu)*(-m_00*pow(u.y, 2) + 2*m_01*u.y - m_02 + 0.33333333329937886*rho);
    double cm_after_collision_11 = m_00*u.x*u.y - m_01*u.x - m_10*u.y + m_11 + omega_nu*(-m_00*u.x*u.y + m_01*u.x + m_10*u.y - m_11);
    double cm_after_collision_21 = 0;
    double cm_after_collision_12 = 0;
    double cm_after_collision_22 = 0.11111111110039928*rho;

## Summary

That's the magic - you have learned how perform symbolic computations and generate code from it.
The back-tranformation from central moments to moments, then from moments to distribution function follow the same way.

References:

[^1]: Linlin Fei, Kai Hong Luo, 'Cascaded lattice Boltzmann method for incompressible thermal flows with heat sources and general thermal boundary conditions' Computers and Fluids (2018).

[^2]: Linlin Fei, Kai Hong Luo, Chuandong Lin, Qing Li, 'Modeling incompressible thermal flows using a central-moments-based lattice Boltzmann method' International Journal of Heat and Mass Transfer (2017).

[^3]: Linlin Fei and Kai Hong Luo, 'Consistent forcing scheme in the cascaded lattice Boltzmann method' Physical Review E 96, 053307 (2017).

[^4]: Linlin Fei, Kai H. Luo and Qing Li, 'Three-dimensional cascaded lattice Boltzmann method: Improved implementation and consistent forcing scheme' Physical Review E 97, 053309 (2018)

[^5]: M. Geier, A. Greiner, J. G. Korvink, 'Cascaded digital lattice Boltzmann automata for high Reynolds number flow' Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 73 (2006).

[^6]: Xiaoyi He, Xiaowen Shan, and Gary D. Doolen, 'Discrete Boltzmann equation model for nonideal gases' in Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics (1998).


# Simulación Monte Carlo: Distribución de Rayleigh

#### Descripción

En este trabajo se estudia la duracción de un dispositivo constituido por $11$ componentes, que supondremos variables de *Rayleigh*, organizados en distintas configuraciones (serie, paralelo, stand-by, etc.). Para ello, utilizaremos el método de simulación *Monte Carlo*.

El contenido de este trabajo se encuentra alojado en el siguiente repositorio: https://github.com/garciparedes/monte-carlo-simulation-rayleigh-distribution

#### Contenidos

  * [Introducción](#Introducción)
  * [Apartado A - Simulación](#Apartado-A)
    * [Apartado A.1 - Serie](#Apartado-A.1)
    * [Apartado A.2 - Paralelo](#Apartado-A.2)
    * [Apartado A.3 - Stand By](#Apartado-A.3)
    * [Apartado A.4 - "Al menos 6"](#Apartado-A.4)
    * [Apartado A.5 - Promedio](#Apartado-A.5)
  * [Apartado B - Aproximaciones Puntuales](#Apartado-B)
    * [Apartado B.1 - Serie](#Apartado-B.1)
    * [Apartado B.2 - Paralelo](#Apartado-B.2)
    * [Apartado B.3 - Stand By](#Apartado-B.3)
    * [Apartado B.4 - "Al menos 6"](#Apartado-B.4)
    * [Apartado B.5 - Promedio](#Apartado-B.5)
  * [Apartado C - Estudio Analítico](#Apartado-C)
    * [Apartado C.1 - Serie](#Apartado-C.1)
    * [Apartado C.2 - Paralelo](#Apartado-C.2)
  * [Conclusiones](#Conclusiones)

#### Autor

  * Sergio García Prado - [garciparedes.me](https://garciparedes.me)

#### Fecha

  Mayo de 2018

#### Agradecimientos

Me gustaría agradecer al profesor [Alfonso Gordaliza](http://www.eio.uva.es/infor/personas/alfonso.html) la revisión y correcciones sobre este trabajo.

```R
rm(list = ls())
```

```R
library(ggplot2, warn.conflicts = FALSE)
library(gridExtra, warn.conflicts = FALSE)
library(tidyr, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)
library(magrittr, warn.conflicts = FALSE)
```

## Introducción

*Unos determinados componentes de hardware informático tienen una vida útil que se comporta según la distribución de Rayleigh, de parámetro $\sigma = 1$ año, según información fiable recibida del fabricante. La función de densidad del modelo de Rayleigh es sobradamente conocida por todos:*

$$f(x) = \frac{x}{\sigma^2}exp\left(-\frac{x^2}{2\sigma^2}\right), \ x >0$$

*Se tiene un equipo informático en cuya fabricación se incorpora un dispositivo formado **$11$ componentes** de los anteriores, que pueden estar acoplados según diferentes esquemas según se indica má adelante. Se supone que el equipo informático está funcionando ininterrumpidamente desde su puesta en marcha y que, idealmente, solo son posibles los fallos de los componetes que se están estudiando.*

Uno de los primeros pasos antes de comenzar a realizar el ejercicio es el de definir la función de densidad de la distribución de Rayleigh, para poder trabajar con ella, así como estudiar su forma:

```R
rayleigh.density <- function(x, sigma = 1) {
    x / sigma ^ 2 * exp(- x ^ 2 / (2 * sigma ^ 2))
}
```

Para la transformación cuantil, necesitaremos apoyarnos en la función de distribución de la variable aleatoria de estudio, la cual es fácil de obtener a partir de la función de densidad:

\begin{align}
F(x) &= \int_0^x f(t) \,dt \\
&= \int_0^x \frac{t}{\sigma^2}exp\left(-\frac{t^2}{2\sigma^2}\right) \,dt \\
&= \left(-exp\left(-\frac{t^2}{2\sigma^2}\right)\right]_0^x \\
&= 1 -exp\left(-\frac{x^2}{2\sigma^2}\right), \ x >0
\end{align}

```R
rayleigh.distribution <- function(x, sigma = 1) {
    1 - exp(- x ^ 2 / (2 * sigma ^ 2))
}
```

Ahora, necesitamos invertir la función de distribución para poder utilizarla en la transformación cuantil, la cual utilizaremos para llevar a cabo la simulación Monte Carlo. A continuación se indican los pasos necesarios para la inversión de $F(x)$:

\begin{align}
y  &= 1 -exp \left( - \frac{x^2}{2 \sigma ^ 2}\right) \\
1 - y  &= exp \left(- \frac{x^2}{2 \sigma ^ 2}\right) \\
ln\left(1 - y \right) &= -\frac{x ^ 2}{2 \sigma ^ 2} \\
-2\sigma^2ln\left(1 - y \right)  &= x^2 \\
\sigma\sqrt{-2ln\left(1 - y \right)}  &= x \\
\end{align}

Una vez invertida, la transformación cuantil para la distribución de Rayleigh es la siguiente:

\begin{equation}
F^{-1}(x) = \sigma\sqrt{-2ln\left(1 - x \right)}, \ x \in [0, 1]
\end{equation}

```R
rayleigh.quantile <- function(x, sigma = 1) {
    sigma * sqrt(- 2 * log(1 - x))
}
```

Para comprender mejor el comportamiento de esta distribución, es muy util representar gráficamente las funciones descritas superiormente:

```R
options(repr.plot.height = 3, repr.plot.width = 9)
grid.arrange(
    ggplot(data.frame(x=c(0, 6)), aes(x)) +
        stat_function(fun = rayleigh.density, col = "red") +
        ylab("f(x)") +
        ggtitle("Función de Densidad"),
    ggplot(data.frame(x=c(0, 6)), aes(x)) +
        stat_function(fun = rayleigh.distribution, col = "red") +
        ylab("F(x)") +
        ggtitle("Función de Distribución"),
    ggplot(data.frame(x=c(0, 1)), aes(x)) +
        stat_function(fun = rayleigh.quantile, col = "red") +
        ylab("F^(-1)(x)") +
        ggtitle("Transformación Cuantil"),
    ncol = 3)
```

Tal y como se puede apreciar en los gráficos, la Distribución de Rayleigh no es simétrica. Su forma recuerda a la de distribución de la $\chi^2$ con una cola más pesada hacía la derecha. En nuestro caso, se ha fijado el parámetro de escala $\sigma = 1$ por lo que la mayoría de masa de probabilidad se localiza en dicha zona del soporte. La función de distribución refleja lo que se ha indicado anteriormente, pero además deja ver claramente el nivel de asimetría de la variable. La transformación cuantil es la función inversa de la de densidad, por lo que "refleja" dicho comportamiento.

La distribución de *Rayleigh* surge como un caso particular de la distribución de *Weibull*, la cual permite modelizar un gran número de situaciones temporales. En concreto, la relación entre estas dos distribuciones es la siguiente:

$$
Rayleigh\left(\sigma\right) \equiv Weibull\left(\sqrt{2}\sigma, 2\right)
$$

Donde el primer parámetro de *Weibull* se refiere a la escala y el segundo a la forma.

## Apartado A

El enunciado del apartado es:

*Realizar un experimento de simulación Montecarlo para tener una aproximación razonable del comportamiento de las variables que se describen a continuación.*

*En el trabajo hay que describir y justificar razonadamente todos los pasos que se dan en el experimento de simulación.
Se recomienda recordar lo estudiado acerca de la transformación cuantil y la simulación de variables aleatorias en el Capítulo 4 del material de *del Barrio & Gordaliza*.*

*Se recomienda simular entre $5000$ y $10000$ muestras y presentar sendos histogramas como aproximaciones empíricas al modelo probabilístico (función de densidad) de cada una de las variables indicadas en los cinco subapartados.*

La simulación *Monte Carlo* se basa en la generación de valores de una determinada variable aleatoria con distribución seguida mediante la función inversa de la función de distribución y la generación aleatoria de valores de manera uniforme en el intervalo $[0, 1]$.

La intuicción detrás de este procedimiento es la siguiente: Si "elegimos" al azar y de manera uniforme cuantiles de una determinada distribución $F$, entonces al final estaremos obteniendo realizaciones de dicha distribución.

Esto, se puede llevar a cabo de manera automática si se generan números aleatorios de una distribución $U \sim U(0, 1)$ y después se transforman mediante la función de distribución inversa (transformación cuantil) que se acaba de calcular. Entonces si aplicamos $X = F^{-1}(U)$ se puede demostrar desarrollando la función de densidad de la variable $X$, que habremos conseguido $X \sim F$.

Por tanto, esta será la intuicción que utilizaremos para realizar una simulación sobre variables de *Rayleigh*. En concreto, y siguiendo el enunciado propuesto diseñaremos el experimento de la siguiente manera:

Sea:

$$U_i = \left(U_{i1}, ..., U_{ij}, ..., U_{i11}\right) \ v.a.i.i.d. \ | \ U_{ij} \sim U\left(0,1\right) \ \forall i \in \{1, ..., 7500\} $$

Nuestras variables aleatorias (en este caso vector por referirse a 11 componentes) de partida, entonces realizaremos la transformación cuantil sobre todas ellas de tal manera que:

\begin{align}
F^{-1}\left(U_i\right) &= \left(F^{-1}\left(U_{i1}\right), ..., F^{-1}\left(U_{ij}\right), ..., F^{-1}\left(U_{i11}\right)\right) \\
 &= \left(X_{i1}, ..., X_{ij}, ..., X_{i11}\right) \\
 &= X_i
\end{align}

Por lo que habremos llegado a nuestra simulación de variables de *Rayleigh* mediante la transformación cuantil de variables de una distribución *Uniforme* en el intervalo $[0,1]$ tal y como se indica a continuación:

$$X_i = \left(X_{i1}, ..., X_{ij}, ..., X_{i11}\right) \ v.a.i.i.d. \ | \ X_{ij} \sim F \ \forall i \in \{1, ..., 7500\}$$

Además, debido a la suposición de independencia entre variables, es posible generar todos los valores como una muestra aleatoria simple y después agruparlos en vectores de $11$ componentes. A continuación se incluye el código necesario para realizar dicha generación de variables con distribución *Uniforme* así como la tranformación cuantil:

```R
n.s <- 7500
n.c <- 11
x.limits <- c(0, 6)

uniform.sample <-
    matrix(runif(n.s * n.c, min = 0, max = 1), ncol = n.c, nrow = n.s) %>%
    as.data.frame() %>%
    set_colnames(sprintf("C%02d", 1:n.c))

rayleigh.sample <-
    rayleigh.quantile(uniform.sample)
```

Los histogramas resultante de las variables con distribución $Uniforme$ sobre cada componente se puede visualizar a continuación:

```R
options(repr.plot.height = 5, repr.plot.width = 9)
uniform.sample %>%
    gather() %>%
    ggplot(aes(value)) +
        scale_x_continuous(limits = c(0, 1)) +
        geom_histogram(aes(y = ..density..), binwidth = 0.05) +
        stat_function(fun = dunif, col = "red") +
        facet_wrap(~ key, scales = "free")
```

Así como los histogramas de cada componente tras la realización de la transformación cuantil, resultando en simulaciones de la variables de $Rayleigh$:

```R
options(repr.plot.height = 5, repr.plot.width = 9)
rayleigh.sample %>%
    select(starts_with("C")) %>%
    gather() %>%
    ggplot(aes(x = value)) +
        scale_x_continuous(limits = x.limits) +
        geom_histogram(aes(y = ..density..), binwidth = 0.1) +
        stat_function(fun = rayleigh.density, col = "red") +
        facet_wrap(~ key)

```

Una vez generada la simulación resultante, se procederá a resolver las distintas cuestiones sobre distintas suposiciones sobre la forma en que se conectan los componentes del dispositivo. Para ello, el procedimiento que se seguirá es el siguiente: Primero se justificará el procedimiento utilizado para posteriormente realizar la pertinente transformación de los datos, que se representará de manera gráfica.

### Apartado A.1

El enunciado del apartado es:

*La duración del dispositivo si los componentes están acoplados **en serie** (el dispositivo funciona mientras todos los componentes estén en funcionamiento).*

Como es obvio, diremos que la duracción de un dispositivo acoplado en serie es la del mínimo de los componentes que la componen. Por tanto, en cada observación de la muestra, seleccionaremos el mínimo de entre sus componentes.

```R
rayleigh.sample$series <-
    rayleigh.sample %>%
    dplyr::select(dplyr::starts_with("C")) %>%
    apply(1, min)
```

```R
options(repr.plot.height = 4, repr.plot.width = 9)
rayleigh.sample %>%
    ggplot(aes(series)) +
    scale_x_continuous(limits = x.limits) +
    geom_histogram(bins = 60)
```

En el [Apartado B.1](#Apartado-B.1) se calculan distintos estadísticos sobre la muestra generada.

### Apartado A.2

El enunciado del apartado es:

*La duración del dispositivo si los componentes están acoplados **en paralelo** (el dispositivo funciona mientras alguno de los componentes estén funcionamiento)*

Como es obvio, diremos que la duracción de un dispositivo acoplado en paralelo es la del máximo de los componentes que la componen. Por tanto, en cada observación de la muestra, seleccionaremos el máximo de entre sus componentes.

```R
rayleigh.sample$parallel <-
    rayleigh.sample %>%
    dplyr::select(dplyr::starts_with("C")) %>%
    apply(1, max)
```

```R
options(repr.plot.height = 4, repr.plot.width = 9)
rayleigh.sample %>%
    ggplot(aes(parallel)) +
    scale_x_continuous(limits = x.limits) +
    geom_histogram(bins = 60)
```

En el [Apartado B.2](#Apartado-B.2) se calculan distintos estadísticos sobre la muestra generada.

### Apartado A.3

El enunciado del apartado es:

*La duración del dispositivo si los componentes están **acoplados en "stand by"**, según se explica para esta modalidad de acoplamiento en el material de *del Barrio & Gordaliza* en el capítulo "Modelos de fiabilidad. La ley de Weibull."*

Como es obvio, diremos que la duracción de un dispositivo acoplado en "stand by" es la suma de las duraciones de cada componentes si entendemos que no puede funcionar ningún componente de manera simultanea y cuando uno de estos falla, comienza a funcionar otro de manera inmediata.Por tanto, en cada observación de la muestra, seleccionaremos la suma de sus componentes.

```R
rayleigh.sample$stand.by <-
    rayleigh.sample %>%
    dplyr::select(dplyr::starts_with("C")) %>%
    apply(1, sum)
```

```R
options(repr.plot.height = 4, repr.plot.width = 9)
rayleigh.sample %>%
    ggplot(aes(stand.by)) +
    scale_x_continuous(limits = x.limits * n.c / 2) +
    geom_histogram(bins = 60)
```

En el [Apartado B.3](#Apartado-B.3) se calculan distintos estadísticos sobre la muestra generada.

### Apartado A.4

El enunciado del apartado es:

*La duración del dispositivo si los componentes están acoplados de tal forma que **el dispositivo funciona mientras al menos seis de los componentes estén en funcionamiento**.*

Para estudiar el fenómeno referido a que se mantengan funcionando de manera correcta al menos $6$ de los componentes, es equivalente a estudiar el estadístico ordenado situado en la sexta posición, es decir, $X_{(6)}$. Por tanto, en este caso se utilizará el tiempo de fallo del sexto componente para cada observación de la simulada.

```R
rayleigh.sample$least.6 <-
    rayleigh.sample %>%
    dplyr::select(dplyr::starts_with("C")) %>%
    apply(1, sort) %>%
    {t(.)[, 6]}
```

```R
options(repr.plot.height = 4, repr.plot.width = 9)
rayleigh.sample %>%
    ggplot(aes(least.6)) +
    scale_x_continuous(limits = x.limits) +
    geom_histogram(bins = 60)
```

En el [Apartado B.4](#Apartado-B.4) se calculan distintos estadísticos sobre la muestra generada.

### Apartado A.5

El enunciado del apartado es:

*La **duración promedio de los once componentes** si se supone que todos comienzan su funcionamiento a la vez y el dispositivo se mantiene activo hasta que todos han fallado.*

En este caso se pide estudiar el promedio de la duración de cada componente. Por tanto, aplicaremos el promedio sobre cada observación de la muestra.

```R
rayleigh.sample$mean <-
    rayleigh.sample %>%
    dplyr::select(dplyr::starts_with("C")) %>%
    apply(1, mean)
```

```R
options(repr.plot.height = 4, repr.plot.width = 9)
rayleigh.sample %>%
    ggplot(aes(mean)) +
    scale_x_continuous(limits = x.limits) +
    geom_histogram(bins = 60)
```

En el [Apartado B.5](#Apartado-B.5) se calculan distintos estadísticos sobre la muestra generada.

## Apartado B

El enunciado del apartado es:

*En todos los casos que se decriben en el apartado anterior, a partir de las simulaciones obtener una aproximación (estimación puntual) de los siguientes parámetros de la variable estudiaba: la media, la desviación típica, el percentil $2$ y la probabilidad de que la variable supere el valor $1.5$ añós.*

Para el cálculo los parámetros, se va a construir una función que simplifique la implementación en *R*:

```R
ApproximateParameter <- function(x) {
    list(
        mean = mean(x),
        sd = sd(x),
        quantile.02 = quantile(x, 0.02, names = FALSE),
        p.upper.1.5 = sum(x > 1.5) / length(x)
    )
}
```

### Apartado B.1

En este caso se utilizarán los datos transformados en el [Apartado A.1](#Apartado-A.1), y referidos a *La duración del dispositivo si los componentes están acoplados **en serie** (el dispositivo funciona mientras todos los componentes estén en funcionamiento).*

```R
ApproximateParameter(rayleigh.sample$series)
```

$mean

0.379743849192924

$sd

0.199021277761433

$quantile.02

0.0595719486436585

$p.upper.1.5

0

Estos valores son coherentes con el experimento que se pretende a estudiar, así como con la simulación realizada, estando la distribución muy próximos a  $0$ en el soporte, tanto que la probabilidad aproximada de que el dispositivo dure más de un año y medio se convierte en $0$

### Apartado B.2

En este caso se utilizarán los datos transformados en el [Apartado A.2](#Apartado-A.2), y referidos a *La duración del dispositivo si los componentes están acoplados **en paralelo** (el dispositivo funciona mientras alguno de los componentes estén funcionamiento)*

```R
ApproximateParameter(rayleigh.sample$parallel)
```

$mean

2.41004122115448

$sd

0.488574929628598

$quantile.02

1.55413833435374

$p.upper.1.5

0.987466666666667

En el caso de la configuración en paralelo, la duración del dispositivo es mucho mayor, por ser necesario que todos los componentes fallen antes de que el dispositivo deje de funcionar. Además, en esta configuración la probabilidad de que el dispositivo dure más de un año y medio es alta.

### Apartado B.3

En este caso se utilizarán los datos transformados en el [Apartado A.3](#Apartado-A.3), y referidos a *La duración del dispositivo si los componentes están **acoplados en "stand by"**, según se explica para esta modalidad de acoplamiento en el material de *del Barrio & Gordaliza* en el capítulo "Modelos de fiabilidad. La ley de Weibull."*

```R
ApproximateParameter(rayleigh.sample$stand.by)
```

$mean

13.8145687726395

$sd

2.16623011963433

$quantile.02

9.55368421000001

$p.upper.1.5

1

Para el caso de la configuración de componentes en modo "stand by", la duración del dispositivo es muy alta (como era de esperar), siendo mayor que 10 años con probabilidad alta (ya que el cuantil 0.02 se encuentra casi en dicho valor). Por estas razones, la probabilidad de que el dispositivo dure más de un año es técnicamente un suceso seguro.

### Apartado B.4

En este caso se utilizarán los datos transformados en el [Apartado A.4](#Apartado-A.4), y referidos a *La duración del dispositivo si los componentes están acoplados de tal forma que **el dispositivo funciona mientras al menos seis de los componentes estén en funcionamiento**.*

```R
ApproximateParameter(rayleigh.sample$least.6)
```

$mean

1.19043357233194

$sd

0.248675222239707

$quantile.02

0.722444299705511

$p.upper.1.5

0.112133333333333

En esta configuración, nos encontramos en un caso intermedio entre los dos anterior (componentes conectados en serie y los componentes conectados en paralelo), lo cual que se refleja en cierta manera en la forma de la distribución obtenida.

### Apartado B.5

En este caso se utilizarán los datos transformados en el [Apartado A.5](#Apartado-A.5), y referidos a *La **duración promedio de los once componentes** si se supone que todos comienzan su funcionamiento a la vez y el dispositivo se mantiene activo hasta que todos han fallado.*

```R
ApproximateParameter(rayleigh.sample$mean)
```

$mean

1.25586988842177

$sd

0.196930010875848

$quantile.02

0.868516746363638

$p.upper.1.5

0.107466666666667

En este caso se está estudiando el tiempo promedio de los componentes, lo cual nos permite conocer más en detalle la duración de cada uno de ellos. Sin embargo, en este caso no nos podemos fijar en la desviación estándar de los datos, ya que está sesgada por el número de componentes promediados. Estudiando los valores de los parámetros sabemos que el tiempo promedio de duración de un componente es de un año y cuatro meses, así como que la decima parte de ellos no dura más de un año y medio.

## Apartado C

El enunciado del apartado es:

*Para los casos descritos en los apartados [A.1](#Apartado-A.1) y [A.2](#Apartado-A.2), obtener también una expresión analítica de la función de densidad de las variables siguiendo las indicaciones dadas en el material de *del Barrio & Gordaliza* en el capítulo "Modelos de fiabilidad. La ley de Weibull". Hacer una representación de dichas funciones superpuestas sobre los histogramas obtenidos por las simulaciones de Montecarlo.*

### Apartado C.1

En este caso se va a estudiar el supuesto en el [Apartado A.1](#Apartado-A.1), el cual se refiere a la elección de una configuración en serie. Tal y como indicamos en dicho apartado, esto se refiere a la búsqueda del componente de menor duracción. Es decir,

Sea:
$$
X_1, ..., X_{n} \ v.a.i.i.d. \ | \ X_i \sim Rayleigh(\sigma)
$$

Entonces nuestra variable de estudio es:

\begin{align}
X_S
&= min\left(\left\{X_1, ..., X_{11}\right\}\right) \\
&= X_{(1)}
\end{align}

Como sabemos:

\begin{align}
F_S(x)
&= F_{X_{(1)}}\left( x \right) \\
&= 1 - \left(1 - F_{X_i}\left(x\right) \right) ^ n \\
&= 1 - \left(1 - \left(1 - exp\left( - \frac{x ^ 2}{2 \sigma ^ 2}\right) \right) \right) ^ n \\
&= 1 - exp\left(-\frac{nx^2}{2\sigma^2}\right)
\end{align}

Ahora calculamos la función de densidad simplemente derivando la función de distribución obtenida:

\begin{align}
f_S(x)
&= \frac{\partial}{\partial x} F_S(x) \\
&= \frac{\partial}{\partial x} 1 - exp\left(-\frac{nx^2}{2\sigma^2}\right) \\
&= \frac{nx}{\sigma^2} \cdot exp\left(-\frac{nx^2}{2\sigma^2}\right)
\end{align}

Si nos paramos a estudiar la relación entre esta variable y la de las variables de partida no es difícil darse cuenta de que:

$$
X_{S} \sim Rayleigh\left(\frac{\sigma}{\sqrt{n}}\right)
$$

Para nuestro caso en particular tenemos que:

$$
X_{S} \sim Rayleigh\left(\frac{1}{\sqrt{11}}\right)
$$

Por tanto, tan solo tendremos que cambiar el parámetro sigma por su valor adecuado.

A continuación se muestra una representación gráfica del histograma de la variable junto con la sobreposición de la función de densidad desarrollada de manera analítica en el párrafo anterior:

```R
rayleigh.sample %>%
    ggplot(aes(x = series)) +
        scale_x_continuous(limits = x.limits) +
        geom_histogram(aes(y = ..density..), binwidth = 0.1) +
        stat_function(fun = rayleigh.density, args = list(sigma = 1 / sqrt(n.c)), col = "red")
```

### Apartado C.2

En este caso se va a estudiar el supuesto en el [Apartado A.2](#Apartado-A.2), el cual se refiere a la elección de una configuración en serie. Tal y como indicamos en dicho apartado, esto se refiere a la búsqueda del componente de menor duracción. Es decir,

Sea:
$$
X_1, ..., X_{n} \ v.a.i.i.d. \ | \ X_i \sim Rayleigh(\sigma)
$$

Entonces nuestra variable de estudio es:

\begin{align}
X_P \\
&= max\left(\left\{X_1, ..., X_{11}\right\}\right) \\
&= X_{(n)}
\end{align}

Como sabemos:

\begin{align}
F_P(x)
&= F_{ X_{(n)} } \left( x \right) \\
&= \left( F_{X_i} \left( x \right) \right) ^ n \\
&= \left(1 - exp\left( - \frac{x ^ 2}{2 \sigma ^ 2}\right) \right) ^ n
\end{align}
Ahora calculamos la función de densidad simplemente derivando la función de distribución obtenida:

\begin{align}
f_P(x)
&= \frac{\partial}{\partial x} F_P(x) \\
&= \frac{\partial}{\partial x} \left(F_{X_i}\left(x\right) \right) ^ n \\
&= n \ f_{X_i}\left(x\right) \left(F_{X_i}\left(x\right) \right) ^ {n - 1}  \\
&= \frac{n x}{\sigma^2}exp\left(-\frac{x^2}{2\sigma^2}\right) \left(1 -exp\left(-\frac{x^2}{2\sigma^2}\right)\right) ^ {n - 1}
\end{align}

En este caso no parece surgir ninguna función conocida a partir de la transformación. A continuación se incluye el código *R* necesario para la implementación de la función:

```R
rayleigh.max.density <- function(x, sigma = 1, n = 11) {
   n * rayleigh.density(x) * rayleigh.distribution(x) ^ (n - 1)
}
```

A continuación se muestra una representación gráfica del histograma de la variable junto con la sobreposición de la función de densidad desarrollada de manera analítica en el párrafo anterior:

```R
rayleigh.sample %>%
    ggplot(aes(x = parallel)) +
        scale_x_continuous(limits = x.limits) +
        geom_histogram(aes(y = ..density..), binwidth = 0.1) +
        stat_function(fun = rayleigh.max.density, col = "red")
```

## Conclusiones

En este trabajo se ha podido comprobar la potencia de la *transformación cuantil* para simulación de variables, lo cual se conoce como el método de *Monte Carlo*. Esta estrategia de simulación permite generar muestras de cualquier distribución conocida de tal manera que ningún método conocido permita distinguir entre observaciones reales de la distribución y las aquí simuladas.

Gracias a métododos de simulación como el aquí estudiado es fácil darse cuenta del gran ahorro económico que se puede tiene frente a simulaciones físicas, cuyas dificultades a nivel del tiempo necesario para ser llevadas a cabo es en muchas situaciones impracticable. Otras circunstancias en que los métodos de simulación suponen ventajas frente a métodos analíticos son aquellos en que el suceso de estudio es complicado de desarrollar por dicha vía (o incluso imposible debido a dificultades matemáticas). Por tanto, existen un gran número de ocasiones en que los métodos de simulación otorgan una buena aproximación al comportamiento real de ciertos fenómenos.

En cuanto a la distribución simulada en este caso, las variables de *Rayleigh* se utilizan para modelizar fenómenos de duración del tiempo de vida de determinados componentes físicos, ya sean materiales o incluso biológicos, por lo que es enriquecedor conocer y manejar dicha distribución.

## Referencias
  * [WI09] H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2009.
  * [WI11] Hadley Wickham (2011). The Split-Apply-Combine Strategy for Data Analysis. Journal of Statistical Software, 40(1), 1-29. URL http://www.jstatsoft.org/v40/i01/.
  * [BA12] Baptiste Auguie (2012). gridExtra: functions in Grid graphics. R package version 0.9.1. http://CRAN.R-project.org/package=gridExtra
  * [TRC13] R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.
  * [SA18] Del Barrio & Gordaliza. Modelos Probabilísticos, 2017/18. Facultad de Ciencias: Departamento de Estadística e Investigación Operativa.


Stability of defection, optimisation of strategies and the limits of memory in the PD.
----------------------

## 2. Stability of defection

```python
import opt_mo

import numpy as np
import sympy as sym
import itertools
import axelrod as axl
import matplotlib.pyplot as plt
```

    //anaconda3/envs/opt-mo/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
      warnings.warn(msg, category=DeprecationWarning)

```python
from fractions import Fraction
```

```python
from axelrod.action import Action

C, D = Action.C, Action.D
```

```python
import tqdm
```

```python
import matplotlib.transforms as transforms
import matplotlib
```

```python
import warnings; warnings.simplefilter('ignore')
```

```python
import operator
import functools
```

```python
sym.init_printing()
```

```python
font = {"size": 10, "weight": "bold"}
matplotlib.rc("font", **font)
```

Analytical check
----------------

```python
p_1, p_2, p_3, p_4 = sym.symbols("p_1, p_2, p_3, p_4")
q_1, q_2, q_3, q_4 = sym.symbols("q_1, q_2, q_3, q_4")
k_1, k_2, k_3, k_4 = sym.symbols("k_1, k_2, k_3, k_4")

p, q, k = (p_1, p_2, p_3, p_4), (q_1, q_2, q_3, q_4), (k_1, k_2, k_3, k_4)
```

```python
def get_Q_N_derivative(player, opponent):
    x = np.array(player)

    Q = opt_mo.utility.quadratic_term_numerator(opponent)
    c = opt_mo.utility.linear_term_numerator(opponent)

    return np.dot(x, Q) + c
```

```python
def get_Q_N(player, opponent):
    x = np.array(player)

    Q = opt_mo.utility.quadratic_term_numerator(opponent)
    c = opt_mo.utility.linear_term_numerator(opponent)
    a = opt_mo.utility.constant_term_numerator(opponent)

    return np.dot(x, Q.dot(x.T) * 1 / 2) + np.dot(c, x.T) + a
```

```python
def get_Q_D(player, opponent):
    x = np.array(player)

    Q_bar = opt_mo.utility.quadratic_term_denominator(opponent)
    c_bar = opt_mo.utility.linear_term_denominator(opponent)
    a_bar = opt_mo.utility.constant_term_denominator(opponent)

    return np.dot(x, Q_bar.dot(x.T) * 1 / 2) + np.dot(c_bar, x.T) + a_bar
```

```python
def get_Q_D_derivative(player, opponent):
    x = np.array(player)

    Q_bar = opt_mo.utility.quadratic_term_denominator(opponent)
    c_bar = opt_mo.utility.linear_term_denominator(opponent)
    a_bar = opt_mo.utility.constant_term_denominator(opponent)

    return np.dot(x, Q_bar) + c_bar
```

**Check quadratic derivative of**

$$\frac{1}{2}pQp^T + cp +a $$

```python
expr = get_Q_N(p, q)
```

```python
diff = [sym.diff(expr, i) for i in p]
```

```python
derivatives = get_Q_N_derivative(p, q)
```

```python
for i in range(4):
    assert (diff[i] - derivatives[i]).simplify() == 0
```

**Check derivative of utility**

```python
def get_written_derivative_of_utility(player, opponents):
    sums = 0
    for opponent in opponents:
        numerator = (get_Q_N_derivative(player, opponent) * get_Q_D(player, opponent) -
                      get_Q_D_derivative(player, opponent) * get_Q_N(player, opponent))

        denominator = get_Q_D(player, opponent) ** 2

        sums += numerator / denominator
    return (sums) * (1 / len(opponents))
```

```python
for seed in range(100):
    num_players = 5
    np.random.seed(seed)
    opponents = [[np.random.random() for _ in range(4)] for _ in range(num_players)]

    np.random.seed(seed + 1000)
    player = [np.random.random() for _ in range(4)]

    written_derivative = get_written_derivative_of_utility(player, opponents)

    utility = opt_mo.tournament_utility(p, opponents)
    utility_derivative = [sym.diff(utility, i) for i in p]
    utility_derivative = [expr.subs({p_1: player[0], p_2: player[1], p_3: player[2], p_4: player[3]}) for expr
                          in utility_derivative]

    differences = written_derivative - utility_derivative
    for difference in differences:
        assert np.isclose(round(difference, 10), 0)
```

```python
opponents = [q, k]
```

```python
utility = opt_mo.tournament_utility(p, opponents)
derivative_of_utility = [sym.diff(utility, i) for i in p]
```

```python
written_derivative = get_written_derivative_of_utility(p, opponents)
```

```python
for i in tqdm.tqdm(range(4)):
    assert (written_derivative[i] - derivative_of_utility[i]).simplify() == 0
```

      0%|          | 0/4 [00:00<?, ?it/s]
     25%|██▌       | 1/4 [00:06<00:18,  6.28s/it]
     50%|█████     | 2/4 [02:27<01:33, 46.69s/it]
     75%|███████▌  | 3/4 [02:33<00:34, 34.56s/it]
    100%|██████████| 4/4 [02:34<00:00, 24.63s/it]

**Stability of defection**

**Check condition for defection stability**

```python
opponents = [q, k]
```

```python
utility = opt_mo.tournament_utility(p, opponents)
```

```python
diff_utility = [sym.diff(utility, i) for i in p]
```

```python
diff_utility_at_zero = [expr.subs({p_1: 0, p_2: 0, p_3: 0, p_4: 0}).expand() for expr in diff_utility]
```

```python
def get_derivate_for_p_zeros(opponents):
    sums = 0
    for opponent in opponents:
        lhs = opt_mo.utility.linear_term_numerator(opponent) * opt_mo.utility.constant_term_denominator(opponent)
        rhs = opt_mo.utility.linear_term_denominator(opponent) * opt_mo.utility.constant_term_numerator(opponent)

        denominator = opt_mo.utility.constant_term_denominator(opponent) ** 2

        sums +=  (lhs - rhs) / denominator

    return (sums) * (1 / len(opponents))
```

```python
expression = get_derivate_for_p_zeros(opponents)
```

```python
for i in tqdm.tqdm(range(4)):
    assert (diff_utility_at_zero[i] - expression[i]).simplify() == 0
```

    100%|██████████| 4/4 [00:02<00:00,  1.54it/s]

```python
def stackplot(plot, eco, logscale=True):

    populations = eco.population_sizes
    figure, ax = plt.subplots(figsize=(10, 8))
#     figure = ax.get_figure()
    figure.patch.set_facecolor('#ffffff')
    figure.patch.set_alpha(0.2)

    turns = range(len(populations))
    pops = [
        [populations[iturn][ir] for iturn in turns]
        for ir in plot.result_set.ranking
    ]
    ax.stackplot(turns, *pops)

    ax.yaxis.tick_left()
    ax.yaxis.set_label_position("right")
    ax.yaxis.labelpad = 25.0

    ax.set_ylim([0.0, 1.0])
    #ax.set_xlim([0.0, 10 ** 3])
    ax.set_ylabel("Relative population size", fontweight='bold')
    ax.set_xlabel("Turn", fontweight='bold')

    trans = transforms.blended_transform_factory(ax.transAxes, ax.transData)
    ticks = []
    for i, n in enumerate(plot.result_set.ranked_names):
        x = -0.01
        y = (i + 0.5) * 1 / plot.result_set.num_players
        if n != 'Defector':
            opponent_strings = (n.split(':')[1].replace('[', '').replace(']', '')).split(",")
            opponent = [Fraction(float(op)).limit_denominator() for op in opponent_strings]
            label = '$q^{(%s)}$'% i
#             for p in range(3):
#                 label += r'\frac{' + str(opponent[p].numerator) + '}{' + str(opponent[p].denominator) + '},'
#             label += r'\frac{' + str(opponent[3].numerator) + '}{' + str(opponent[3].denominator) + '})$'
#             print(label)
            n = label
        ax.annotate(
            n,
            xy=(x, y),
            xycoords=trans,
            clip_on=False,
            va="center",
            ha="right",
            fontsize=10,
        )
        ticks.append(y)
    ax.set_yticks(ticks)
    ax.tick_params(direction="out")
    ax.set_yticklabels([])
    if logscale:
        ax.set_xscale("log")
    for tick in ax.yaxis.get_majorticklabels():  # example for xaxis
        tick.set_fontsize(20)

    plt.tight_layout()
    plt.xlim(0, 10 **2)
    return figure.savefig('../img/population_defection_takes_over.pdf', facecolor=figure.get_facecolor(),
                          edgecolor='none', bbox_inches='tight')
```

```python
for seed in range(2, 10):
    np.random.seed(seed)
    opponents = [[np.round(np.random.random(), 5) for _ in range(4)] for _ in range(3)]

    derivative = get_derivate_for_p_zeros(opponents)

    if all([el < 0 for el in derivative]):
        print('Found at: %s' % seed)
        break
```

    Found at: 5

```python
defection_stable_opponents_set = [axl.MemoryOnePlayer(ps, initial=D) for ps in opponents] + [axl.Defector()]
```

```python
tournament = axl.Tournament(defection_stable_opponents_set)
results = tournament.play(progress_bar=False)
eco = axl.Ecosystem(results)
eco.reproduce(500)
```

```python
color = '#ffffff'
```

```python
plot = axl.Plot(results)
p = stackplot(plot, eco)
```

```python
populations = eco.population_sizes
turns = range(len(populations))
pops = [
    [populations[iturn][ir] for iturn in turns]
    for ir in plot.result_set.ranking
]
```

```python
figure, ax = plt.subplots(figsize=(10, 8))

ax.stackplot(turns, *pops)
ax.yaxis.tick_left()
ax.yaxis.set_label_position("right")
ax.yaxis.labelpad = 25.0

ax.set_ylim([0.0, 1.0])
# ax.set_ylabel("Relative population size", fontweight='bold', color=color)
ax.set_xlabel("Turn", fontweight='bold', color=color)

trans = transforms.blended_transform_factory(ax.transAxes, ax.transData)

ticks = []
for i, n in enumerate(plot.result_set.ranked_names):
    x = -0.01
    y = (i + 0.5) * 1 / plot.result_set.num_players
    if n != 'Defector':
        opponent_strings = (n.split(':')[1].replace('[', '').replace(']', '')).split(",")
        opponent = [Fraction(float(op)).limit_denominator() for op in opponent_strings]
        label = '$q^{(%s)}$'% i
        n = label
    ax.annotate(
        n,
        xy=(x, y),
        xycoords=trans,
        clip_on=False,
        va="center",
        ha="right",
        fontsize=17,
        color=color
    )
    ticks.append(y)
ax.set_yticks(ticks)
ax.tick_params(direction="out")
ax.set_yticklabels([])
ax.set_xscale("log")

ax.spines['bottom'].set_color(color)
ax.spines['top'].set_color(color)
ax.spines['right'].set_color(color)
ax.spines['left'].set_color(color)

ax.tick_params(axis='x', colors=color)
ax.tick_params(axis='y', colors=color)

for tick in ax.yaxis.get_majorticklabels():  # example for xaxis
    tick.set_fontsize(20)

plt.tight_layout()
plt.xlim(0, 10 **2);

figure.savefig('/Users/storm/src/talks/talks/2020-02-26-Max-Planck/static/population_defection_takes_over.png',
            bbox_inches='tight',
            transparent=True, dpi=100)
```

```python
p.savefig('../img/population_defection_takes_over.pdf', facecolor=fig.get_facecolor(), edgecolor='none',
          bbox_inches='tight')
```

```python
for seed in range(5000):
    np.random.seed(4)
    opponents = [[round(np.random.random(), 5) for _ in range(4)] for _ in range(3)]

    np.random.seed(seed)
    other_opponent =[[np.random.random() for _ in range(4)]]

    derivative = get_derivate_for_p_zeros(opponents + other_opponent)
    if all([el < 0 for el in derivative]):
        print('Found at: %s' % seed)
        break
```

```python
defection_stable_opponents_set = [axl.MemoryOnePlayer(ps, initial=C) for ps in opponents]
defection_stable_opponents_set += [axl.Defector()]
```

```python
tournament = axl.Tournament(defection_stable_opponents_set)
results = tournament.play(progress_bar=False)
eco = axl.Ecosystem(results)
eco.reproduce(50000)
```

```python
populations = eco.population_sizes
turns = range(len(populations))
pops = [
    [populations[iturn][ir] for iturn in turns]
    for ir in plot.result_set.ranking
]
```

```python
figure, ax = plt.subplots(figsize=(10, 8))

ax.stackplot(turns, *pops)
ax.yaxis.tick_left()
ax.yaxis.set_label_position("right")
ax.yaxis.labelpad = 25.0

ax.set_ylim([0.0, 1.0])
# ax.set_ylabel("Relative population size", fontweight='bold', color=color)
ax.set_xlabel("Turn", fontweight='bold', color=color)

trans = transforms.blended_transform_factory(ax.transAxes, ax.transData)

ticks = []
for i, n in enumerate(plot.result_set.ranked_names):
    x = -0.01
    y = (i + 0.5) * 1 / plot.result_set.num_players
    if n != 'Defector':
        opponent_strings = (n.split(':')[1].replace('[', '').replace(']', '')).split(",")
        opponent = [Fraction(float(op)).limit_denominator() for op in opponent_strings]
        label = '$q^{(%s)}$'% i
        n = label
    ax.annotate(
        n,
        xy=(x, y),
        xycoords=trans,
        clip_on=False,
        va="center",
        ha="right",
        fontsize=17,
        color=color
    )
    ticks.append(y)
ax.set_yticks(ticks)
ax.tick_params(direction="out")
ax.set_yticklabels([])
ax.set_xscale("log")

ax.spines['bottom'].set_color(color)
ax.spines['top'].set_color(color)
ax.spines['right'].set_color(color)
ax.spines['left'].set_color(color)

ax.tick_params(axis='x', colors=color)
ax.tick_params(axis='y', colors=color)

for tick in ax.yaxis.get_majorticklabels():  # example for xaxis
    tick.set_fontsize(20)

plt.tight_layout()
plt.xlim(0, 10 **2);

figure.savefig('/Users/storm/src/talks/talks/2020-02-26-Max-Planck/static/population_defection_fails.png',
            bbox_inches='tight',
            transparent=True, dpi=100)
```

```python
plot = axl.Plot(results)
p = stackplot(plot, eco)

p.savefig('../img/population_defection_fails.pdf', bbox_inches='tight')
```

```python

```

```python

```


# SINDy-PI Feature Overview

This notebook provides a simple overview of the basic functionality of SINDy-PI implemented in PySINDy, and is written by Alan Kaptanoglu and Kadierdan Kaheman. SINDy-PI is used to handle generic implicit ODEs of the form,
$$ \frac{d}{dt}\mathbf{x} = \mathbf{f}(\mathbf{x}, \dot{\mathbf{x}}) \approx \mathbf{\Theta}(\mathbf{x}, \dot{\mathbf{x}})\mathbf{\Xi}$$
This is based off of the paper "Kaheman, K., Kutz, J. N., & Brunton, S. L. (2020). SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics. Proceedings of the Royal Society A, 476(2242), 20200279."

We should clarify that building SINDy libraries $\mathbf{\Theta}(\mathbf{x}, \dot{\mathbf{x}})$ depending on both $\mathbf{x}$ and $\dot{\mathbf{x}}$ is also possible with any of the PySINDy libraries. However, in that case one must be sure to omit the linear $\dot{\mathbf{x}}$ term in the library (since then the SINDy model will just fit $\dot{\mathbf{x}} = \dot{\mathbf{x}}$) and otherwise be careful about the matrix conditioning. The benefit of not using the SINDy-PI functionality is that you can avoid using SymPy (see below).

An interactive version of this notebook is available on binder
[](https://mybinder.org/v2/gh/dynamicslab/pysindy/v1.6.1?filepath=examples/9_sindypi_with_sympy.ipynb)

```python
# Import libraries.
import numpy as np
from matplotlib import pyplot as plt
import pysindy as ps
from pysindy.differentiation import FiniteDifference
from pysindy.utils import enzyme, bacterial
import sympy as sp
from scipy.integrate import solve_ivp

# ignore user warnings
import warnings
warnings.filterwarnings("ignore", category=UserWarning)

# For reproducibility
np.random.seed(100)

integrator_keywords = {}
integrator_keywords['rtol'] = 1e-12
integrator_keywords['method'] = 'LSODA'
integrator_keywords['atol'] = 1e-12
```

# Example to show library functionality
Below, we define custom libraries for x and x_dot, showing that the resulting full library and library names are a tensor product of the two independent libraries. Note that to generate the constant function 1 without generating repeated entries, we need to use include_bias=True.

```python
t = np.linspace(0, 1, 5)
x = np.ones((5, 2))

# Functions to be applied to the data x
functions = [lambda x: np.exp(x), lambda x, y: np.sin(x + y)]

# Functions to be applied to the data x_dot
x_dot_functions = [lambda x: x]

lib = ps.SINDyPILibrary(
    library_functions=functions,
    x_dot_library_functions=x_dot_functions,
    t=t,
).fit(x)
lib.transform(x)
print("Without function names: ")
print(lib.get_feature_names(), "\n")

# Function names includes both the x and x_dot functions
function_names = [
    lambda x: "exp(" + x + ")",
    lambda x, y: "sin(" + x + y + ")",
    lambda x: x,
]
lib = ps.SINDyPILibrary(
    library_functions=functions,
    x_dot_library_functions=x_dot_functions,
    function_names=function_names,
    t=t,
).fit(x)
lib.transform(x)
print("With function names: ")
print(lib.get_feature_names(), "\n")

# Now include a bias term (constant term)
lib = ps.SINDyPILibrary(
    library_functions=functions,
    x_dot_library_functions=x_dot_functions,
    function_names=function_names,
    t=t,
    include_bias=True,
).fit(x)
lib.transform(x)
print("With function names and bias term: ")
print(lib.get_feature_names())

```

    Without function names:
    ['f0(x0)', 'f0(x1)', 'f1(x0,x1)', 'f_dot0(x0_dot)', 'f_dot0(x1_dot)', 'f0(x0)f_dot0(x0_dot)', 'f0(x1)f_dot0(x0_dot)', 'f1(x0,x1)f_dot0(x0_dot)', 'f0(x0)f_dot0(x1_dot)', 'f0(x1)f_dot0(x1_dot)', 'f1(x0,x1)f_dot0(x1_dot)']

    With function names:
    ['exp(x0)', 'exp(x1)', 'sin(x0x1)', 'x0_dot', 'x1_dot', 'exp(x0)x0_dot', 'exp(x1)x0_dot', 'sin(x0x1)x0_dot', 'exp(x0)x1_dot', 'exp(x1)x1_dot', 'sin(x0x1)x1_dot']

    With function names and bias term:
    ['1', 'exp(x0)', 'exp(x1)', 'sin(x0x1)', 'x0_dot', 'x1_dot', 'exp(x0)x0_dot', 'exp(x1)x0_dot', 'sin(x0x1)x0_dot', 'exp(x0)x1_dot', 'exp(x1)x1_dot', 'sin(x0x1)x1_dot']

### Solve 1D Michelson-Menten enzyme model as first demonstration
The Michelson-Menten enzyme model is,
$$\dot{x} = 0.6 - \frac{1.5 x}{0.3 + x}.$$
Note that, instead of dealing with fractional terms, it can be rearranged into
$$\dot{x} = \frac{3}{5} -3 x - \frac{10}{3} x\dot{x},$$
which is an implicit ODE, since the right-hand-side now depends on $\dot{x}$! In order to handle this scenario, SINDy-PI builds a candidate library with terms depending on both $x$ and $\dot{x}$, and solves the optimization problem,
$$\|\Theta(x, \dot{x}) - \Theta(x, \dot{x})\Xi\|_2 + \lambda\|\Xi\|_1, \quad s.t. \text{diag}(\Xi) = 0.$$
This reason this modified optimization is required is that linear terms in $\dot{x}$ are no longer guaranteed, so every candidate library term is fit with a different model to find the best combination of implicit terms. Note $\text{diag}(\Xi) = 0$ is required because without this, the model would just fit every candidate library term with itself, $\dot{x} = \dot{x}$, $x\dot{x} = x\dot{x}$, $x^2\dot{x} = x^2\dot{x}$, and so on.

```python
# define parameters
r = 1
dt = 0.001
T = 4
t = np.arange(0, T + dt, dt)
t_span = (t[0], t[-1])
x0_train = [0.55]
x_train = solve_ivp(enzyme, t_span, x0_train, t_eval=t, **integrator_keywords).y.T

# Initialize custom SINDy library so that we can have x_dot inside it.
x_library_functions = [
    lambda x: x,
    lambda x, y: x * y,
    lambda x: x ** 2,
    lambda x, y, z: x * y * z,
    lambda x, y: x * y ** 2,
    lambda x: x ** 3,
    lambda x, y, z, w: x * y * z * w,
    lambda x, y, z: x * y * z ** 2,
    lambda x, y: x * y ** 3,
    lambda x: x ** 4,
]
x_dot_library_functions = [lambda x: x]

# library function names includes both
# the x_library_functions and x_dot_library_functions names
library_function_names = [
    lambda x: x,
    lambda x, y: x + y,
    lambda x: x + x,
    lambda x, y, z: x + y + z,
    lambda x, y: x + y + y,
    lambda x: x + x + x,
    lambda x, y, z, w: x + y + z + w,
    lambda x, y, z: x + y + z + z,
    lambda x, y: x + y + y + y,
    lambda x: x + x + x + x,
    lambda x: x,
]

# Need to pass time base to the library so can build the x_dot library from x
sindy_library = ps.SINDyPILibrary(
    library_functions=x_library_functions,
    x_dot_library_functions=x_dot_library_functions,
    t=t[1:-1],
    function_names=library_function_names,
    include_bias=True,
)

# Use the SINDy-PI optimizer, which relies on CVXPY.
# Note that if LHS of the equation fits the data poorly,
# CVXPY often returns failure.
sindy_opt = ps.SINDyPI(
    threshold=1e-6,
    tol=1e-8,
    thresholder="l1",
    max_iter=20000,
)
model = ps.SINDy(
    optimizer=sindy_opt,
    feature_library=sindy_library,
    differentiation_method=ps.FiniteDifference(drop_endpoints=True),
)
model.fit(x_train, t=t)
model.print()
```

    (4002, 1) 4002
    Solver failed on model  3 , setting coefs to zeros
    1 = 5.000 x0 + 1.667 x0_dot + 5.556 x0x0_dot
    x0 = 0.200 1 + -0.333 x0_dot + -1.111 x0x0_dot
    x0x0 = 0.198 x0 + 0.007 x0x0x0 + -0.338 x0x0_dot + -1.099 x0x0x0_dot
    x0x0x0 = 0.000
    x0x0x0x0 = -0.001 1 + 0.363 x0x0x0 + 0.041 x0x0_dot + -1.205 x0x0x0x0x0_dot
    x0_dot = 0.600 1 + -3.000 x0 + -3.333 x0x0_dot
    x0x0_dot = 0.180 1 + -0.900 x0 + -0.300 x0_dot
    x0x0x0_dot = -0.004 1 + 0.136 x0 + -0.508 x0x0 + -0.344 x0x0x0 + -0.102 x0x0_dot + -0.219 x0x0x0x0x0_dot
    x0x0x0x0_dot = 0.003 1 + 0.001 x0 + -0.391 x0x0x0 + -0.247 x0x0x0x0 + -0.108 x0x0_dot
    x0x0x0x0x0_dot = 0.001 1 + -0.670 x0x0x0x0 + -0.005 x0_dot + 0.029 x0x0_dot + -0.271 x0x0x0_dot

### Okay, so how to choose which model to use?
So SINDy-PI has generated a different dynamical model for each of the candidate library terms, but how do we choose which one to use? This can be decided in many ways, but an easy choice is picking the model that minimizes the mean-squared error of the $\dot{x}$ testing data. There are many other ways to choose, and the SINDy-PI paper noted that bad models tend to also be less sparse, so sparsity is another metric for selection. Below, we simply integrate forward all the models and on this testing trajectory, they basically all perform fine.

```python
# Plot the fit of the derivative
x_dot_train = model.differentiate(x_train, t)
x_dot_train_pred = sindy_opt.Theta_ @ sindy_opt.coef_.T
plt.figure()
for i in range(r):
    plt.subplot(1, r, i + 1)
    plt.plot(t, x_dot_train[:, i], 'k')
    plt.plot(t[1:-1], x_dot_train_pred[:, 5], 'r--')
```

### Now we need to reformat all the feature strings for conversion into symbolic variables
So we have all these nice models but these are implicit ODEs, and the default Python solvers (solve_ivp and odeint) cannot solve implicit ODEs. There are two options although we only illustrate the first option below; we must convert the equations into symbolic variables and then either (1) use SymPy to symbolically solve the equation for $\dot{x}$, after which the equation is no longer implicit and can be solved with the default Python solves, or (2) convert the equations into a symbolic differential algebraic equation (DAE), which can be numerically solved with the diffeqpy Python package (but requires Julia to be installed).

```python
features = model.get_feature_names()
features[0] = "1"
features_copy = list(np.copy(features))
nfeatures = len(features)
features_formatted = []
# Need to put multiplication between terms for sympy
for i in range(nfeatures):
    for j in range(r):
        # Overkill to make sure all the x0, x1, etc. get replaced
        temp_string = features[i].replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        features[i] = temp_string
    features_formatted.append(temp_string)
features = features_copy
print(features)
print(features_formatted)
coefs = model.coefficients()
sym_features = [sp.symbols(feature) for feature in features]
sym_theta = [sp.symbols(feature) for feature in features]
```

    ['1', 'x0', 'x0x0', 'x0x0x0', 'x0x0x0x0', 'x0_dot', 'x0x0_dot', 'x0x0x0_dot', 'x0x0x0x0_dot', 'x0x0x0x0x0_dot']
    ['1', 'x0', 'x0 * x0', 'x0 * x0 * x0', 'x0 * x0 * x0 * x0', 'x0_dot', 'x0 * x0_dot', 'x0 * x0 * x0_dot', 'x0 * x0 * x0 * x0_dot', 'x0 * x0 * x0 * x0 * x0_dot']

### Check all the model fits produce sensible models

```python
sym_equations = []
sym_equations_rounded = []
for i in range(nfeatures):
    sym_equations.append(
        sp.solve(
            sp.Eq(sym_theta[i], sym_theta @ np.around(coefs[i], 10)), sym_features[i]
        )
    )
    sym_equations_rounded.append(
        sp.solve(
            sp.Eq(sym_theta[i], sym_theta @ np.around(coefs[i], 2)), sym_features[i]
        )
    )
    print(sym_theta[i], " = ", sym_equations_rounded[i][0])

```

    1  =  5.0*x0 + 1.67*x0_dot + 5.56*x0x0_dot
    x0  =  0.2*1 - 0.33*x0_dot - 1.11*x0x0_dot
    x0x0  =  0.2*x0 - 0.34*x0x0_dot + 0.01*x0x0x0 - 1.1*x0x0x0_dot
    x0x0x0  =  0
    x0x0x0x0  =  0.04*x0x0_dot + 0.36*x0x0x0 - 1.21*x0x0x0x0x0_dot
    x0_dot  =  0.6*1 - 3.0*x0 - 3.33*x0x0_dot
    x0x0_dot  =  0.18*1 - 0.9*x0 - 0.3*x0_dot
    x0x0x0_dot  =  0.14*x0 - 0.51*x0x0 - 0.1*x0x0_dot - 0.34*x0x0x0 - 0.22*x0x0x0x0x0_dot
    x0x0x0x0_dot  =  -0.11*x0x0_dot - 0.39*x0x0x0 - 0.25*x0x0x0x0
    x0x0x0x0x0_dot  =  -0.01*x0_dot + 0.03*x0x0_dot - 0.27*x0x0x0_dot - 0.67*x0x0x0x0

### Now we will:
1. reformat all the equations
2. solve symbolically for x_dot
3. integrate and plot the new equations for x_dot

```python
# Define the ODE symbol variables
t_sym = sp.symbols("t_sym")
x_sym = sp.symbols("x:%d" % r)
x_dot_sym = sp.symbols("x:%d_dot" % r)

# Need to format the above equations so that there are space between x0 * x0, x0 * x_dot0, and so on.
sym_equations_formatted = []
sym_equations_rounded_formatted = []
for i in range(nfeatures):
    for j in range(r):
        # Overkill to make sure all the x0, x1, etc. get replaced
        temp_string = str(sym_equations[i]).replace(
            "x" + str(j) + "x", "x" + str(j) + " * x"
        )
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_rounded_string = str(sym_equations_rounded[i]).replace(
            "x" + str(j) + "x", "x" + str(j) + " * x"
        )
        temp_rounded_string = temp_rounded_string.replace(
            "x" + str(j) + "x", "x" + str(j) + " * x"
        )
        temp_rounded_string = temp_rounded_string.replace(
            "x" + str(j) + "x", "x" + str(j) + " * x"
        )
        temp_rounded_string = temp_rounded_string.replace(
            "x" + str(j) + "x", "x" + str(j) + " * x"
        )
        sym_equations_formatted.append(temp_string)
        sym_equations_rounded_formatted.append(temp_rounded_string)

# Now that the equations are mathematically formatted,
# solve for x_dot0 in the algebraic equation.
sym_equations_simplified = []
sym_equations_rounded_simplified = []
for i in range(nfeatures):
    sym_equations_simplified.append(
        sp.solve(
            sp.Add(
                sp.sympify(sym_equations_formatted)[i][0],
                -sp.sympify(features_formatted[i]),
            ),
            x_dot_sym,
        )
    )
    rounded = sp.solve(
        sp.Add(
            sp.sympify(sym_equations_rounded_formatted)[i][0],
            -sp.sympify(features_formatted[i]),
        ),
        x_dot_sym,
    )
    if len(rounded) != 0:
        rounded_temp = rounded[0]
        for a in sp.preorder_traversal(rounded):
            if isinstance(a, sp.Float):
                rounded_temp = rounded_temp.subs(a, round(a, 2))
        sym_equations_rounded_simplified.append(rounded_temp)
    else:
        sym_equations_rounded_simplified.append([])

# Plot the results for each of the models
plt.figure(figsize=(16, 3))
x0_test = np.random.rand(1)
x_test = solve_ivp(enzyme, t_span, x0_test, t_eval=t).y.T
plt.plot(t, x_test, color="k", linestyle="solid", label="True")
for i in range(nfeatures):
    ax = plt.gca()
    if i != nfeatures - 1:
        ax.set_xticklabels([])
    if len(sym_equations_simplified[i]) != 0:
        ODE_Func = sp.lambdify([t_sym, x_sym], sym_equations_simplified[i][0])

        # Now simulate the system we identified
        x_test_sim = solve_ivp(ODE_Func, t_span, x0_test, t_eval=t).y.T
        if (
            np.linalg.norm(x_test_sim) < 1e3
            and sym_equations_rounded_simplified[i] != 0
        ):
            plt.plot(
                t,
                x_test_sim,
                linestyle="dashed",
                label=str(sp.sympify(features_formatted[i]))
                + " = "
                + str(sym_equations_rounded_simplified[i]),
            )
plt.grid(True)
plt.legend(fontsize=8)
```

### We will now try a 2D example, the bacterial competence system (Mangan et al. 2016)
$$ \begin{aligned} \dot{x} &= a_1 + \frac{a_2x^2}{a_3 + x^2} - \frac{x}{1 + x + y} \\  \dot{y} &= \frac{b_1}{1 + b_2 x^5} - \frac{y}{1 + x + y} \end{aligned} $$
with $a_1=0.004$, $a_2=0.07$, $a_3=0.04$, $b_1=0.82$, $b_2=1854.5$. Note that multi-variable SymPy will require some changes, and multi-variable SINDy-PI typically requires many trajectories to correctly identify models.

```python
# define parameters
r = 2
dt = 0.01
T = 20
t_train = np.arange(0, T + dt, dt)
t_train_span = (t_train[0], t_train[-1])

# Typically need at least 20-40 trajectories for a decent model.
# Note that this can be fairly time-consuming.
n_trajectories = 10
x_train_multi = []
t_train_multi = []
x0 = np.random.rand(n_trajectories, 2)
for i in range(n_trajectories):
    x_train_multi.append(solve_ivp(bacterial, t_train_span,
                                   x0[i, :], t_eval=t_train,
                                   **integrator_keywords).y.T)
    t_train_multi.append(t_train[1:-1])

# Initialize custom SINDy library up to sixth order polynomials
x_library_functions = [
    lambda x: x,
    lambda x, y: x * y,
    lambda x: x ** 2,
    lambda x, y, z: x * y * z,
    lambda x, y: x * y ** 2,
    lambda x: x ** 3,
    lambda x, y, z, w: x * y * z * w,
    lambda x, y, z: x * y * z ** 2,
    lambda x, y: x * y ** 3,
    lambda x: x ** 4,
    lambda x: x ** 5,
    lambda x: x ** 6,
]
x_dot_library_functions = [lambda x: x]

library_function_names = [
    lambda x: x,
    lambda x, y: x + y,
    lambda x: x + x,
    lambda x, y, z: x + y + z,
    lambda x, y: x + y + y,
    lambda x: x + x + x,
    lambda x, y, z, w: x + y + z + w,
    lambda x, y, z: x + y + z + z,
    lambda x, y: x + y + y + y,
    lambda x: x + x + x + x,
    lambda x: x + x + x + x + x,
    lambda x: x + x + x + x + x + x,
    lambda x: x,
]

# Get the indices of the equations for x0_dot and x1_dot
lib = ps.SINDyPILibrary(
    library_functions=x_library_functions,
    x_dot_library_functions=x_dot_library_functions,
    t=t_train,
    function_names=library_function_names,
    include_bias=True,
).fit(np.asarray(x_train_multi)[0, :, :])
lib.transform(np.asarray(x_train_multi)[0, :, :])
features = np.asarray(lib.get_feature_names())
ind1 = int(np.ravel(np.where(features == "x0_dot"))[0] - 1)  # -1 because bias term
ind2 = int(np.ravel(np.where(features == "x1_dot"))[0] - 1)

# Need to flatten t_train_multi before passing time base to SINDyPILibrary
sindy_library = ps.SINDyPILibrary(
    library_functions=x_library_functions,
    x_dot_library_functions=x_dot_library_functions,
    t=np.ravel(np.asarray(t_train_multi)),
    function_names=library_function_names,
)

# Note, using model_subset (list of indices) to only compute two of the fits
sindy_opt = ps.SINDyPI(
    threshold=10,
    tol=1e-5,
    thresholder="l1",
    max_iter=20000,
    model_subset=[ind1, ind2],
)
model = ps.SINDy(
    optimizer=sindy_opt,
    feature_library=sindy_library,
    differentiation_method=ps.FiniteDifference(drop_endpoints=True),
)
model.fit(x_train_multi, t=dt, multiple_trajectories=True)
model.print()
```

    x0 = 0.000
    x1 = 0.000
    x0x1 = 0.000
    x0x0 = 0.000
    x1x1 = 0.000
    x0x1x1 = 0.000
    x0x0x0 = 0.000
    x1x1x1 = 0.000
    x0x1x1x1 = 0.000
    x0x0x0x0 = 0.000
    x1x1x1x1 = 0.000
    x0x0x0x0x0 = 0.000
    x1x1x1x1x1 = 0.000
    x0x0x0x0x0x0 = 0.000
    x1x1x1x1x1x1 = 0.000
    x0_dot = -0.410 x0 + 0.007 x1 + 0.021 x0x1
    x1_dot = 0.010 x0 + 0.244 x1 + -0.332 x1x1 + 0.082 x1x1x1x1 + -0.020 x1x1x1x1x1 + 0.001 x1x1x1x1x1x1 + 0.773 x0x1_dot + 1.044 x1x1_dot + -0.009 x1x1x1x1x1x1x1_dot
    x0x0_dot = 0.000
    x1x0_dot = 0.000
    x0x1x0_dot = 0.000
    x0x0x0_dot = 0.000
    x1x1x0_dot = 0.000
    x0x1x1x0_dot = 0.000
    x0x0x0x0_dot = 0.000
    x1x1x1x0_dot = 0.000
    x0x1x1x1x0_dot = 0.000
    x0x0x0x0x0_dot = 0.000
    x1x1x1x1x0_dot = 0.000
    x0x0x0x0x0x0_dot = 0.000
    x1x1x1x1x1x0_dot = 0.000
    x0x0x0x0x0x0x0_dot = 0.000
    x1x1x1x1x1x1x0_dot = 0.000
    x0x1_dot = 0.000
    x1x1_dot = 0.000
    x0x1x1_dot = 0.000
    x0x0x1_dot = 0.000
    x1x1x1_dot = 0.000
    x0x1x1x1_dot = 0.000
    x0x0x0x1_dot = 0.000
    x1x1x1x1_dot = 0.000
    x0x1x1x1x1_dot = 0.000
    x0x0x0x0x1_dot = 0.000
    x1x1x1x1x1_dot = 0.000
    x0x0x0x0x0x1_dot = 0.000
    x1x1x1x1x1x1_dot = 0.000
    x0x0x0x0x0x0x1_dot = 0.000
    x1x1x1x1x1x1x1_dot = 0.000

```python
# Plot the fit of the derivative
x_train = np.asarray(x_train_multi)[0, :, :]
x_dot_train = model.differentiate(x_train, t_train)
x_dot_train_pred = sindy_opt.Theta_[:len(t_train) - 2, :] @ sindy_opt.coef_.T
plt.figure()
inds = [ind1, ind2]
for i in range(r):
    plt.subplot(1, r, i + 1)
    plt.plot(t_train, x_dot_train[:, i], 'k')
    plt.plot(t_train[1:-1], x_dot_train_pred[:, inds[i]], 'r--')
```

#### SINDy-PI model is not good here, but we proceed anyways to illustrate how to do the symbolic solve + integration in more than 1 dimensions.

```python
# Repeat formatting the feature names and building the symbolic equations
features = model.get_feature_names()
nfeatures = len(features)
features[0] = "1"
features_copy = list(np.copy(features))
nfeatures = len(features)
features_formatted = []
# Need to put multiplication between terms for sympy
for i in range(nfeatures):
    for j in range(r):
        # Overkill to make sure all the x0, x1, etc. get replaced
        temp_string = features[i].replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        temp_string = temp_string.replace("x" + str(j) + "x", "x" + str(j) + " * x")
        features[i] = temp_string
    features_formatted.append(temp_string)
features = features_copy

coefs = model.coefficients()
sym_features = [sp.symbols(feature) for feature in features]
sym_theta = [sp.symbols(feature) for feature in features]
sym_equations = []
sym_equations_rounded = []
for i in range(nfeatures):
    if np.any(np.around(coefs[i], 2) != 0.0):
        sym_equation = sp.solve(
            sp.Eq(sym_theta[i], sym_theta @ np.around(coefs[i], 6)), sym_features[i]
        )
        sym_equations.append(sym_equation)
        sym_equation_rounded = sp.solve(
            sp.Eq(sym_theta[i], sym_theta @ np.around(coefs[i], 2)), sym_features[i]
        )
        sym_equations_rounded.append(sym_equation_rounded)

```

```python
# Define the ODE symbol variables
t_sym = sp.symbols('t_sym')
x_sym = sp.symbols('x:%d' % r)
x_dot_sym = sp.symbols('x:%d_dot' % r)

# Need to format the above equations so that there are space between x0 * x0, x0 * x_dot0, and so on.
sym_equations_formatted = []
for i in range(len(sym_equations)):
    for j in range(r):
        # Overkill to make sure all the x0, x1, etc. get replaced
        temp_string = str(sym_equations[i]).replace('x' + str(j) + 'x', 'x' + str(j) + ' * x')
        temp_string = temp_string.replace('x' + str(j) + 'x', 'x' + str(j) + ' * x')
        temp_string = temp_string.replace('x' + str(j) + 'x', 'x' + str(j) + ' * x')
        temp_string = temp_string.replace('x' + str(j) + 'x', 'x' + str(j) + ' * x')
        sym_equations[i] = temp_string
    sym_equations_formatted.append(temp_string)

# IMPORTANT change for multi-variable implicit equations:
# Now that the equations are mathematically formatted,
# solve for x0_dot and x1_dot in the algebraic equation.
# Have two unknowns, (x0_dot, x1_dot),
# so need to solve using two of ANY of the equations,
# but for simplicity we choose the equations for x0_dot and x1_dot.
sym_equations_simplified = sp.solve(
    (sp.Add(
        sp.sympify(sym_equations_formatted)[0][0],
        - sp.sympify(features_formatted[ind1])),
    sp.Add(
        sp.sympify(sym_equations_formatted)[1][0],
        - sp.sympify(features_formatted[ind2]))),
    x_dot_sym)
print('Solved for xdot equations = ', sym_equations_simplified)

# Not the cleanest notation for multiple variables, but it works
ODE_Func = sp.lambdify([x_sym, t_sym],
                       [sym_equations_simplified[x_dot_sym[0]],
                        sym_equations_simplified[x_dot_sym[1]]])

```

    Solved for xdot equations =  {x1_dot: (-1110.0*x1**6 + 20212.0*x1**5 - 82372.0*x1**4 + 331681.0*x1**2 - 243970.0*x1 - 9959.0)/(772811.0*x0 - 8754.0*x1**6 + 1044332.0*x1 - 1000000.0), x0_dot: 0.021224*x0*x1 + 3.1e-5*x1**6 - 0.000477*x1**4 + 0.007338*x1 - 0.410064}

#### We can simulate the system we identified but tends to be unstable -- need more memory to train on additional trajectories.


# 7. Vertical Vibration of Quarter Car Model

This notebook introduces the base excitation system by examning the behavior of a quarter car model.

After the completion of this assignment students will be able to:

- excite a system with a sinusoidal input
- understand the difference in transient and steady state solutions
- create a frequency response plot
- define resonance and determine the parameters that cause resonance

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib notebook
from resonance.linear_systems import SimpleQuarterCarSystem
```

```python
sys = SimpleQuarterCarSystem()
```

The simple quarter car model has a suspension stiffness and damping, along with the sprung car mass in kilograms, and a travel speed parameter in meters per second.

```python
sys.constants
```

```python
sys.coordinates
```

```python
sys.speeds
```

# A sinusoidal road

The road is described as:

$$y(t) = Ysin\omega_b t$$

where $Y$ is the amplitude of the sinusoidal road undulations and $\omega_b$ is the frequency of the a function of the car's speed. If the distance between the peaks (amplitude 0.01 meters) of the sinusoidal road is 6 meters and the car is traveling at 7.5 m/s calculate what the frequency will be.

```python
Y = 0.01  # m
v = sys.constants['travel_speed']
bump_distance = 6  # m
wb = v / bump_distance * 2 * np.pi  # rad /s
```

```python

```

```python
print(wb)
```

Now with the amplitude and frequency set you can use the `sinusoidal_base_displacing_response()` function to simulate the system.

```python
traj = sys.sinusoidal_base_displacing_response(Y, wb, 20.0)
```

```python
traj.head()
```

```python
traj.plot(subplots=True);
```

We've written an animation for you. You can play it with:

```python
sys.animate_configuration(fps=20)
```

**Exercise**

Try different travel speeds and see what kind of behavior you can observe. Make sure to set the `travel_speed` constant and the frequency value for `sinusoidal_base_displacing_response()` to be consistent.

# Transmissibility

When designing a car the designer wants the riders to feel comfortable and to isolate them from the road's bumps. There are two important aspects to investigate. The first is called *displacement transmissibility* and is a ratio between the ampitude of the steady state motion and the ampitude of the sinusoidal base displacement. So in our case this would be:

$$ \frac{X}{Y}(\omega_b) = \frac{\textrm{Steady State Amplitude}}{\textrm{Base Displacement Amplitude}} $$

This can be plotted as a function of the base displacement frequency. A car suspension designer may want this ratio to be an optimal value for rider comfort. Maybe they'd like to make the ratio 1 or maybe even less than one if possible.

**Exercise**

Use the curve fitting technique from the previous notebook to plot $X/Y$ for a range of frequencies. Your code  should look something like:

```python
from scipy.optimize import curve_fit

def cosine_func(times, amp, freq, phase_angle):
    return amp * np.cos(freq * times - phase_angle)

frequencies = np.linspace(1.0, 20.0, num=100)

amplitudes = []

for omega in frequencies:
    # your code here

amplitudes = np.array(amplitudes)

fig, ax = plt.subplots(1, 1, sharex=True)
ax.set_xlabel('$\omega_b$ [rad/s]')
ax.set_ylabel('Displacement Transmissibility')

ax.axvline(, color='black') # natural frequency
ax.plot()#?
ax.grid();
```

```python
from scipy.optimize import curve_fit
def cosine_func(times, amp, freq, phase_angle):
    return amp * np.cos(freq * times - phase_angle)
frequencies = np.linspace(1.0, 20.0, num=100)

amplitudes = []

for omega in frequencies:
    traj = sys.sinusoidal_base_displacing_response(Y, omega, 20.0)
    popt, pcov = curve_fit(cosine_func,
                           traj[10:].index, traj[10:].car_vertical_position,
                           p0=(Y, omega, 0.05))
    amplitudes.append(abs(popt[0]))

amplitudes = np.array(amplitudes)

fig, ax = plt.subplots(1, 1, sharex=True)
ax.set_xlabel('$\omega_b$ [rad/s]')
ax.set_ylabel('Displacement Transmissibility')

ax.axvline(np.sqrt(sys.constants['suspension_stiffness'] / sys.constants['sprung_mass']), color='black')
ax.plot(frequencies, amplitudes / Y)
ax.grid();
```

```python
# write you answer here
```

The second thing to investigate is the *force transmissibility*. This is the ratio of the force applied by the suspension to the sprung car. Riders will feel this force when the car travels over bumps. Reducing this is also preferrable. The force applied to the car can be compared to the

**Excersice**

Create a measurement to calculate the force applied to the car by the suspension. Simulate the system with $Y=0.01$ m, $v = 10$ m/s, and the distance between bump peaks as $6$ m. Plot the trajectories.

```python
def force_on_car(suspension_damping, suspension_stiffness,
                 car_vertical_position, car_vertical_velocity, travel_speed, time):
    # write this code

sys.add_measurement('force_on_car', force_on_car)

# write code for Y and omega_b, etc
```

```python
Y = 0.01  # m
bump_distance = 6  # m

def force_on_car(suspension_damping, suspension_stiffness,
                 car_vertical_position, car_vertical_velocity,
                 travel_speed, time):

    wb = travel_speed / bump_distance * 2 * np.pi

    y = Y * np.sin(wb * time)
    yd = Y * wb * np.cos(wb * time)

    return (suspension_damping * (car_vertical_velocity - yd) +
            suspension_stiffness * (car_vertical_position - y))

sys.add_measurement('force_on_car', force_on_car)

v = 10.0
sys.constants['travel_speed'] = v
wb = v / bump_distance * 2 * np.pi  # rad /s
```

```python
traj = sys.sinusoidal_base_displacing_response(Y, wb, 10.0)
traj[['car_vertical_position', 'car_vertical_velocity', 'force_on_car']].plot(subplots=True)
```

```python
# write your answer here
```

```python
sys.animate_configuration(fps=30)
```

Force transmissibility will be visited more in your next homework.

#  Arbitrary Periodic Forcing (Fourier Series)

Fourier discovered that any periodic function with a period $T$ can be described by an infinite series of sums of sines and cosines. See the wikipedia article for more info (https://en.wikipedia.org/wiki/Fourier_series). The key equation is this:

$$ F(t) = \frac{a_0}{2} + \sum_{n=1}^\infty (a_n \cos n\omega_T t + b_n \sin n \omega_T t)$$

The terms $a_0, a_n, b_n$ are called the Fourier coefficients and are defined as such:

$$ a_0 = \frac{2}{T} \int_0^T F(t) dt$$

$$ a_n = \frac{2}{T} \int_0^T F(t) \cos n \omega_T t dt \quad \textrm{for} \quad n = 1, 2, \ldots $$

$$ b_n = \frac{2}{T} \int_0^T F(t) \sin n \omega_T t dt \quad \textrm{for} \quad n = 1, 2, \ldots $$

## Introduction to SymPy

SymPy is a Python package for symbolic computing. It can do many symbolic operations, for instance, integration, differentiation, linear algebra, etc. See http://sympy.org for more details of the features and the documentation. Today we will cover how to do integrals using SymPy and use it to find the Fourier series that represents a sawtooth function.

```python
import sympy as sm
```

The function `init_printing()` enables LaTeX based rendering in the Jupyter notebook of all SymPy objects.

```python
sm.init_printing()
```

Symbols can be created by using the `symbols()` function.

```python
x, y, z = sm.symbols('x, y, z')
```

```python
x, y, z
```

The `integrate()` function allows you to do symbolic indefinite or definite integrals. Note that the constants of integration are not included in indefinite integrals.

```python
sm.integrate(x * y, x)
```

The `Integral` class creates and unevaluated integral, where as the `integrate()` function automatically evaluates the integral.

```python
expr = sm.Integral(x * y, x)
expr
```

To evaluate the unevaluated form you call the `.doit()` method. Note that all unevaluated SymPy objects have this method.

```python
expr.doit()
```

This shows how to create an unevaluated definite integral, store it in a variable, and then evaluate it.

```python
expr = sm.Integral(x * y, (x, 0, 5))
expr
```

```python
expr.doit()
```

# Fourier Coefficients for the Sawtooth function

Now let's compute the Fourier coefficients for a saw tooth function. The function that describes the saw tooth is:

$$
F(t) =
\begin{cases}
      A \left( \frac{4t}{T} - 1 \right) & 0 \leq t \leq T/2 \\
      A \left( 3 - \frac{4t}{t} \right) & T/2 \leq t \leq T
\end{cases}
$$

where:

- $A$ is the amplitude of the saw tooth
- $T$ is the period of the saw tooth
- $\omega_T$ is the frequency of the saw tooth, i.e. $\omega_T = \frac{2\pi}{T}$
- $t$ is time

This is a piecewise function with two parts from $t=0$ to $t=T$.

```python
A, T, wT, t = sm.symbols('A, T, omega_T, t', real=True, positive=True)
A, T, wT, t
```

The first Fourier coefficient $a_0$ describes the average value of the periodic function. and is:

$$a_0 = \frac{2}{T} \int_0^T F(t) dt$$

This integral will have to be done in two parts:

$$a_0 = a_{01} + a_{02} = \frac{2}{T} \int_0^{T/2} F(t) dt + \frac{2}{T} \int_{T/2}^T F(t) dt$$

These two integrals are evaluated below. Note that $a_0$ evaluates to zero. This is because the average of our function is 0.

```python
ao_1 = 2 / T * sm.Integral(A * (4 * t / T - 1), (t, 0, T / 2))
ao_1
```

```python
ao_1.doit()
```

```python
ao_2 = 2 / T * sm.Integral(A * (3 - 4 * t / T), (t, T / 2, T))
ao_2
```

```python
ao_2.doit()
```

But SymPy can also handle piecewise directly. The following shows how to define a piecewise function.

```python
F_1 = A * (4 * t / T - 1)
F_2 = A * (3 - 4 * t / T)
```

```python
F = sm.Piecewise((F_1, t<=T/2),
                 (F_2, T/2<t))
F
```

```python
F_of_t_only = F.xreplace({A: 0.01, T: 2 * sm.pi / wb})
F_of_t_only
```

```python
sm.plot(F_of_t_only, (t, 0, 2 * np.pi / wb))
```

The integral can be taken of the entire piecewise function in one call.

```python
sm.integrate(F, (t, 0, T))
```

Now the Fourier coefficients $a_n$ and $b_n$ can be computed.

$$
a_n = \frac{2}{T}\int_0^T F(t) \cos n\omega_Tt dt \\
b_n = \frac{2}{T}\int_0^T F(t) \sin n\omega_Tt dt
$$

```python
n = sm.symbols('n', real=True, positive=True)
```

For $a_n$:

```python
an = 2 / T * sm.Integral(F * sm.cos(n * wT * t), (t, 0, T))
an
```

```python
an.doit()
```

This can be simplified:

```python
an = an.doit().simplify()
an
```

Now substitute the $2\pi/T$ for $\omega_T$.

```python
an = an.subs({wT: 2 * sm.pi / T})
an
```

Let's see how this function varies with increasing $n$. We will use a loop but the SymPy expressions will not automatically display because they are inside a loop. So we need to use SymPy's `latex()` function and the IPython display tools. SymPy's `latex()` function  transforms the SymPy expression into a string of matching LaTeX commands.

```python
sm.latex(an, mode='inline')
```

The `display()` and `LaTeX()` functions then turn the LaTeX string in to a displayed version.

```python
from IPython.display import display, Latex
```

Now we can see how $a_n$ varies with $n=1,2,\ldots$.

```python
for n_i in range(1, 6):
    ans = an.subs({n: n_i})
    display(Latex('$a_{} = $'.format(n_i) + sm.latex(ans, mode='inline')))
```

For even $n$ values the coefficient is zero and for even values it varies with the inverse of $n^2$. More precisely:

$$
a_n =
\begin{cases}
0 & \textrm{if }n\textrm{ is even} \\
-\frac{8A}{n^2\pi^2} & \textrm{if }n\textrm{ is odd}
\end{cases}
$$

SymPy can actually reduce this further if your set the assumption that $n$ is an integer.

```python
n = sm.symbols('n', real=True, positive=True, integer=True)
an = 2 / T * sm.Integral(F * sm.cos(n * wT * t), (t, 0, T))
an = an.doit().simplify()
an.subs({wT: 2 * sm.pi / T})
```

The odd and even versions can be computed by setting the respective assumptions.

```python
n = sm.symbols('n', real=True, positive=True, integer=True, odd=True)
an = 2 / T * sm.Integral(F * sm.cos(n * wT * t), (t, 0, T))
an = an.doit().simplify()
an.subs({wT: 2 * sm.pi / T})
```

Note that $b_n$ is always zero:

```python
bn = 2 / T * sm.Integral(F * sm.sin(n * wT * t), (t, 0, T))
bn
```

```python
bn.doit().simplify().subs({wT: 2 * sm.pi / T})
```

# Numerical evalution of the Fourier Series

Now the Fourier coefficients can be used to plot the approximation of the saw tooth forcing function.

```python
import numpy as np
```

The following function plots the actual sawtooth function. It does it all in one line by cleverly using the absolute value and the modulo functions.

```python
def sawtooth(A, T, t):
    return (4 * A / T) * (T / 2 - np.abs(t % T - T / 2) ) - A
```

```python
A = 1
T = 2
t = np.linspace(0, 5, num=500)
```

```python
plt.figure()

plt.plot(t, sawtooth(A, T, t));
```

# Exercise

Write a function that computes the Fourier approximation of the sawtooth function for a given value of $n$, i.e. using a finite number of terms. Then plot it for $n=2, 4, 6, 8, 10$ on top of the actual sawtooth function. How many terms of the infinite series are needed to get a good sawtooth?

```python
def sawtooth_approximation(n, A, T, t):
    # code here
    return f

# plot sawtooth
f = sawtooth(A, T, t)

plt.figure()
plt.plot(t, f, color='k', label='true sawtooth')

for n in np.arange(2, 12, 2):
    f_approx = sawtooth_approximation(n, A, T, t)
    plt.plot(t, f_approx, label='n = {}'.format(n))

plt.legend()
# zoom in a bit on the interesting bit
plt.xlim(0, T)
```

```python
def sawtooth_approximation(n, A, T, t):
    # odd values of indexing variable up to n
    n = np.arange(1, n+1)[:, np.newaxis]
    # cos coefficients
    an = A *(8 * (-1)**n - 8) / 2 / np.pi**2 / n**2
    # sawtooth frequency
    wT = 2 * np.pi / T
    # sum of n cos functions
    f = np.sum(an * np.cos(n * wT * t), axis=0)
    return f

# plot sawtooth
f = sawtooth(A, T, t)

plt.figure()
plt.plot(t, f, color='k', label='true sawtooth')

for n in np.arange(2, 12, 2):
    f_approx = sawtooth_approximation(n, A, T, t)
    plt.plot(t, f_approx, label='n = {}'.format(n))

plt.legend()
# zoom in a bit on the interesting bit
plt.xlim(0, T)
```

```python
# write answer here
```

Below is a interactive plot that shows the same thing as above.

```python
A = 1
T = 2
t = np.linspace(0, 5, num=500)

fig, ax = plt.subplots(1, 1)

f = sawtooth(A, T, t)

saw_tooth_lines = ax.plot(t, f, color='k')

n = 2

f_approx = sawtooth_approximation(n, A, T, t)

approx_lines = ax.plot(t, f_approx)

leg = ax.legend(['true', 'approx, n = {}'.format(n)])

# zoom in a bit on the interesting bit
plt.xlim(0, 2 * T)

def update(n=0):
    f_approx = sawtooth_approximation(n, A, T, t)
    approx_lines[0].set_ydata(f_approx)
    leg.get_texts()[1].set_text('approx, n = {}'.format(n))
```

```python
from ipywidgets import interact

interact(update, n=(0, 20, 2))
```

# Apply the sawtooth to the quarter car

Now that you know the Fourier series coefficients. Calculate them for a suitable number of terms and simulate them with the `sys.periodic_base_displacing_response()` function.

Your code should look something like:

```python
def fourier_coeffs(A, T, N):
    # write your code here

a0, an, bn = fourier_coeffs(?)

traj = sys.periodic_base_displacing_response(?)
```

```python
def fourier_coeffs(A, T, N):
    n = np.arange(1, N+1)
    an = A *(8 * (-1)**n - 8) / 2 / np.pi**2 / n**2
    return 0, an, np.zeros_like(an)

a0, an, bn = fourier_coeffs(0.01, 2 * np.pi / wb, 100)

traj = sys.periodic_base_displacing_response(a0, an, bn, wb, 20.0)
```

```python
traj.plot(subplots=True)
```

```python
sys.animate_configuration(fps=30)
```