```python
# https://colab.research.google.com/github/kassbohm/tm-snippets/blob/master/ipynb/TM_A/TM_3/lie.ipynb

from sympy import *
from sympy.physics.units import *

t, l = var("t, l")

pprint("\nt=0 s:")
sub_list = [(t, 0*second),]
# pprint("\nt=15 s:")
# sub_list = [(t, 15*second),]

tt = t / (60 * second)
p = 2 * pi * tt
l = 10*cm *(1 + tt)

C, S = cos(p), sin(p)
R = Matrix([
    [C, -S, 0],
    [S,  C, 0],
    [0,  0, 1]
    ])

vx, vy, vz = l*C, l*S, 0

pprint("\n(v'x, v'y, v'z) / (cm/s):")
v = Matrix([vx, vy, vz])
tmp = diff(v, t)
tmp = tmp.subs(sub_list)
tmp /= cm/second
pprint(tmp)

pprint("\n(v'x̄, v'ȳ, v'z̄)")
v = Matrix([l, 0, 0])
w = Matrix([0, 0, diff(p, t)])
tmp = diff(v, t) + w.cross(v)
tmp = tmp.subs(sub_list)
tmp /= cm/second
pprint(tmp)

pprint("\nCheck: (v'x, v'y, v'z) / (cm/s):")
tmp = diff(v, t) + w.cross(v)
tmp = R*tmp
tmp = tmp.subs(sub_list)
tmp /= cm/second
pprint(tmp)

# t=0 s:
#
# (v'x, v'y, v'z) / (cm/s):
# ⎡1/6⎤
# ⎢   ⎥
# ⎢ π ⎥
# ⎢ ─ ⎥
# ⎢ 3 ⎥
# ⎢   ⎥
# ⎣ 0 ⎦
#
# (v'x̄, v'ȳ, v'z̄)
# ⎡1/6⎤
# ⎢   ⎥
# ⎢ π ⎥
# ⎢ ─ ⎥
# ⎢ 3 ⎥
# ⎢   ⎥
# ⎣ 0 ⎦
#
# Check: (v'x, v'y, v'z) / (cm/s):
# ⎡1/6⎤
# ⎢   ⎥
# ⎢ π ⎥
# ⎢ ─ ⎥
# ⎢ 3 ⎥
# ⎢   ⎥
# ⎣ 0 ⎦

```


# Dreading tales of SVD and PCA
## Explore the maths behind with python interpretation
How many times I may walk over this super popular ML topic but it never ceases to amaze (or you may say scare me) with its ability to project itself to different spaces altogether. Let’s deconstruct and reconstruct this once again for one and all.
Here’s a short introduction of PCA for those who never found it friendly enough (you’re not to be blamed). Greet it with overwhelming features and it will help you by reducing the feature space to what could be absorbed by your favorite models. Mind you, its no normal reduction but projection to different set of orthogonal coordinates where no correlation can exist (can you foresee elimination of multicollinearity here!).

### Start treading the unknown path to unravel the mystery behind it.
In order to understand the maths behind, assume you have a matrix A of n rows and d columns. Our objective is to project this data to a different set of orthogonal coordinates with minimum loss of information.

Let’s pause for a minute to check out the beauty of the orthogonal system. We are designing our system in a way that there won’t be any correlation that will exist between axes once we have projected our data in all the perpendicular axes independently.

As these orthogonal axes (that we will soon be computing) are directional vectors so these are basically the unit vectors and thus we can further extend the constraints imposed on the system to make it an orthonormal i.e., $|V^TV|=1$ system.

Let’s start with the estimation of the first axis which can be transcended as finding the best fit line to the data.  In case of our matrix A, consider each row as different vectors which are to be projected on the 1-D vector such that the sum of the squares of the perpendicular distances of the points to this 1-D vector are minimized (We are talking about projecting rows as basic matrix operation of Av implies dot product of rows with vector v).

If we consider, projecting a row xi of the data onto a line through the origin. Then,

$x_{i1}^2 + x_{i2}^2 +………………+ x_{id}^2 = (length \space of\space projection)^2 + (distance\space of \space point\space to \space line)^2$

$(distance\space of \space point\space to \space line)^2 = x_{i1}^2 + x_{i2}^2 +………………+ x_{id}^2  -  (length \space of\space projection)^2$

Here, minimizing the sum of the squares of the distances is equivalent to maximizing
the sum of the squares of the lengths of the projections onto the line. This way we can find the first basis vector v1 capturing the maximum variance of the data.

Next, we want to find 2-dimensional subspace that could be the best fit for the data when we have already determined the first basis v1 by above formulation. Since v2 will be orthogonal to v1, it can be proved that instead of finding best 2-dimensional subspace, we can directly find the best fit unit vector perpendicular to the first basis vector that could maximize the length of the projections onto the line.

So, in short in order to determine best fit r-dimensional subspace spanning the entire dataset, a greedy approach is adopted to estimate the higher dimensional subspace which will be orthogonal to already estimated lower dimensional subspace.

To summarize, we are just estimating best fit unit vector orthogonal to each other. Here best is described as minimizing the error when we project data points to these vectors or maximizing the projected lengths.

### Time to unravel the mystery behind these freshly estimated orthogonal vectors

When we compute $Av_i$, we actually get the list of lengths (with signs) of the projections of
the rows of A onto $v_i$. Think of $|Av_i| = \sigma_i (A)$ as the component of the matrix A along
$v_i$. Here, $\sigma_i (A)$ is called as singular values of A.

So, each row of the matrix A is projected on the vector vi which is like finding the component of matrix A along the vector $v_i$.

Consider one row, say $a_j$, of A. Since $v_1,v_2....v_r$ span the space of all rows of A, and as already mentioned umpteen times above these vectors are orthogonal so whatever was projected on $v_1$ will be unrelated to what will be projected to other vectors. Thus, we can say for each row $a_j$, $\sum_{i=1}^{r}(a_j.v_i )^2 =|a_j |^2$

Summing over all rows j,

$\sum_{j=1}^{n}(a_j)^2 =\sum_{i=1}^{r}\sum_{j=1}^{n} (a_j.v_i )^2 = \sum_{i=1}^{r} Av_i= \sum_{i=1}^{r} \sigma_i(A)^2$

Here, when we compute Av, we along with length of the projected points of the data obtain sign of these length indicating direction. If we try to normalize Avi using $\sigma_i (A)$, we can obtain another set of orthogonal vectors:

$u_i=\frac{Av_i}{\sigma_i(A)}$

The vectors $u_1,u_2.....u_r$ are called the left singular vectors of A. The $v_i$are called the right singular vectors.

### Figure out how unit vectors $u_i$  and $v_i$ will be computed:
All the calculation that we have doing till now were under the assumption that $v_i$ are orthonormal. For $u_i$ we only know that it will be a unit vector.

Taking advantage of the property of unit vector we know for sure that $u_i^T u_i = 1$, therefore

$Av_i=u_i\sigma_i$

Multiplying both sides by $(Av_i)^T $

\begin{equation}
(Av_i)^TAv_i=(u_i\sigma_i)^Tu_i\sigma_i
\\
v_i^TA^TAv_i=\sigma_i^2u_i^Tu_i \qquad    [u_i^Tu_i=1]
\\
v_i^TA^TAv_i=\sigma_i^2
\end{equation}

Here we want to find the optimal vector $v_i$ such that above singular value is maximized which will be equal to eigen vector of $A^TA$. It can be proved below using lagrange multipliers (assume B=$A^TA$ is a symmetric matrix):

Taking the lagrangian:
$L(x,λ)=x^TBx−λx^Tx−λ=x^T(B−λI)x−λ$
Now, if you take derivative with respect to x, and set to zero you get:

$\frac{\partial L(x,λ)}{\partial x}=\frac{\partial (x^TBx−λx^Tx−λ)}{\partial x}$

$ \implies 2(B−λI)x=0$

$\implies Bx=λx$

Here, λ should be the eigen value of B, and x should be an eigen vector

For $u_i$,

\begin{equation}
Av_i=u_i\sigma_i \qquad    ..............eq[1]
\\
Av_j=u_j\sigma_j \qquad    ..............eq[2]
\end{equation}

Multiplying eq1 and eq2:
\begin{equation}
v_i^TA^TAv_j=\sigma_iu_i^Tu_j\sigma_j
\\
v_i^T\sigma_j^2v_j=\sigma_iu_i^Tu_j\sigma_j  \qquad    [v_i^Tv_j=0]
\\
0=\sigma_iu_i^Tu_j\sigma_j
\end{equation}

Since eigen values are not always zero, it can be stated that $u_i^Tu_j=0$. Thus, $u_i$ are also orthonormal.
Similarly as above, it can be shown that $u_i$ are the eigen vectors of $AA^T$.

Thus, matrix A can be factorized into product of three different matrices known as singular value decomposition. It can be written as: $A=UDV^T$ where the columns of U and V are orthonormal and the matrix D is diagonal with positive real entries.

As discussed above, we would need to project the data to lower dimensions for PCA and in order to do that we are estimating eigen vectors of the covariance matrix. Here, the covariance matrix of a data set encodes the information concerning the possible linear dependencies and inter-relationships among the data. Off diagnal elements tell us amount of dependency between 2 columns and so if 2 columns are highly related then these values will be higher symmetrically. So, highly correlated columns can be projected to same axis. We can easily reduce the dimension by selecting only required vectors that capture maximum variance of the data.

To summarize, we tried to find dimensions along which maximum information is projected. This can be determined by the singular values in the diagonal cells which are placed in descending order. Now in order to reduce the dimension, top components are selected which captures maximum variance of the data.

Below is a python notebook showing above calculation with sklearn PCA:

```python
import pandas as pd
import numpy as np
from sklearn.decomposition import PCA
import scipy
import matplotlib.pyplot as plt
```

```python
X = np.array([[3,1,1],[-1, 1,0],[-2,-2,-1]])

#Creating a highly correlated columns
X=np.append(X,(X[:,0]*0.9).reshape(-1,1),axis=1)

```

```python
pca = PCA()
pca.fit(X)
```

    PCA(copy=True, iterated_power='auto', n_components=None, random_state=None,
        svd_solver='auto', tol=0.0, whiten=False)

#### Difference between singular values and explained in variance

```python
#singular values are length of rows projected on principal axes
print("Singular value:",pca.singular_values_)

#explained variance is square of singular value divided by (no. of rows-1)
print("explained variance computed from singular value:",np.square(pca.singular_values_[0])/(X.shape[0]-1))

print("explained variance:",pca.explained_variance_)

```

    Singular value: [5.49383072e+00 1.77702673e+00 1.05908890e-16]
    explained variance computed from singular value: 15.091087995282484
    explained variance: [1.50910880e+01 1.57891200e+00 5.60834647e-33]

#### Almost 90% of the variance is captured by the 1st component

```python
plt.plot(range(pca.n_components_),pca.explained_variance_ratio_)
plt.xlabel('number of components')
plt.ylabel('cumulative explained variance');
```

```python
# pca.components are the right singular vector which is the eigen vector of the covariance matrix
pca.components_
```

    array([[ 0.79781054,  0.43703573,  0.30871157,  0.27784041],
           [-0.52081239,  0.8466091 ,  0.08144918,  0.07330426],
           [-0.28269852, -0.28269852,  0.38013956,  0.83406056]])

#### Contribution of individual variables on principal components

```python
#computing weights of features on each component:
#where x1=a1u1+a2u2 (some combination of transformed vectors)
#to compute a1 we can just take dot product of x1 with u1
load=np.matmul(X.T,pca.transform(X))
```

```python
pca_feature= pd.DataFrame(load.T,columns=["X1","X2","X3","X4"],index=["PC1","PC2","PC3"])
plt.figure(figsize=(7,4))
sns.heatmap(pca_feature,cmap="YlGnBu")
```

#### We observe X1 and X4 variables which are highly correlated are projected on PC1

```python
pca.components_.T
```

    array([[ 0.67492573, -0.28208378, -0.67969265],
           [ 0.33440717,  0.91170163, -0.02745311],
           [ 0.25233323,  0.15740446,  0.10981243],
           [ 0.60743316, -0.2538754 ,  0.7247106 ]])

#### Visualize the direction of variables with respect to the components

```python
def loading_plot(coeff, labels):
    n = coeff.shape[0]
    for i in range(n):
        plt.arrow(0, 0, coeff[i,0], coeff[i,1], head_width = 0.05, head_length = 0.05, color = '#21918C',alpha = 0.5)
        plt.text(coeff[i,0]* 1.15, coeff[i,1] * 1.15, labels[i], color = '#21918C', ha = 'center', va = 'center')
    plt.xlim(-1,1)
    plt.ylim(-1,1)
    plt.xlabel('PC1')
    plt.ylabel('PC2')
    plt.grid()
```

```python
fig, ax = plt.subplots(figsize = (7,7))
loading_plot(pca.components_.T, ["X1","X2","X3","X4"])
```


# Operator splitting methods

Operator splitting is a natural and old idea. When a PDE or system of PDEs
contains different terms expressing different physics, it is natural to
use different numerical methods for different physical processes. This can
optimize and simplify the overall solution process. The idea was
especially popularized in the context of the Navier-Stokes equations
and reaction-diffusion PDEs. Common names for the technique are *operator
splitting*, *fractional step* methods, and *split-step* methods. We shall
stick to the former name.
In the context of nonlinear differential equations, operator splitting
can be used to isolate nonlinear terms and simplify the solution methods.

A related technique, often known as dimensional splitting or
alternating direction implicit (ADI) methods, is to split the spatial
dimensions and solve a 2D or 3D problem as two or three
consecutive 1D problems, but this type of splitting
is not to be further considered here.

## Ordinary operator splitting for ODEs

Consider first an ODE where the right-hand side is split into two
terms:

$$
\begin{equation}
u' = f_0(u) + f_1(u)\thinspace .
\label{_auto1} \tag{1}
\end{equation}
$$

In case $f_0$ and $f_1$ are linear functions of $u$, $f_0=au$ and $f_1=bu$,
we have $u(t)=Ie^{(a+b)t}$, if $u(0)=I$.
When going one time step of length $\Delta t$ from $t_n$ to $t_{n+1}$, we have

$$
u(t_{n+1}) = u(t_n)e^{(a+b)\Delta t}\thinspace .
$$

This expression can be also be written as

$$
u(t_{n+1}) = u(t_n)e^{a\Delta t}e^{b\Delta t},
$$

or

$$
\begin{equation}
u^\stepzero = u(t_n)e^{a\Delta t},
\label{nonlin:splitting:ODE:step1} \tag{2}
\end{equation}
$$

$$
\begin{equation}
u(t_{n+1}) = u^\stepzero e^{b\Delta t}
\label{nonlin:splitting:ODE:step2} \tag{3}
\end{equation}
$$

The first step ([2](#nonlin:splitting:ODE:step1)) means solving
$u'=f_0$ over a time interval $\Delta t$ with $u(t_n)$ as start value.
The second step ([3](#nonlin:splitting:ODE:step2)) means solving
$u'=f_1$ over a time interval $\Delta t$ with the value at the end
of the first step as start value.
That is, we progress the solution in two steps and solve
two ODEs $u'=f_0$ and $u'=f_1$. The order of the equations is not
important. From the derivation above we see that solving $u'=f_1$
prior to $u'=f_0$ can equally well be done.

The technique is exact if the ODEs are linear. For nonlinear ODEs it is
only an approximate method with error $\Delta t$. The technique can
be extended to an arbitrary number of steps; i.e., we may split the PDE
system into any number of subsystems. Examples will illuminate this principle.

## Strang splitting for ODEs

The accuracy of the splitting method in the section [Ordinary operator splitting for ODEs](#nonlin:splitting:ODE)
can be improved from $\Oof{\Delta t}$ to $\Oof{\Delta t^2}$ using so-called
*Strang splitting*, where we take frac{1}{2} a step with the $f_0$ operator,
a full step with the $f_1$ operator, and finally frac{1}{2} another step with
the $f_0$ operator. During a time interval $\Delta t$ the algorithm can
be written as follows.

$$
\begin{align*}
\frac{du^{\stepzero}}{dt} &= f_0(u^{\stepzero}),
\quad u^{\stepzero}(t_n)=u(t_n),
\quad t\in [t_n,t_n+\frac{1}{2}\Delta t],\\
\frac{du^{\stephalf}}{dt} &= f_1(u^{\stephalf}),
\quad u^{\stephalf}(t_n)=u^{\stepzero}(t_{n+\frac{1}{2}}),
\quad t\in [t_n,t_n+\Delta t],\\
\frac{du^{\stepone}}{dt}  &= f_0(u^{\stepone}),
\quad u^{\stepone}(t_{n+\frac{1}{2}})=u^{\stephalf}(t_{n+1}),
\quad t\in [t_n+\frac{1}{2}\Delta t, t_n+\Delta t]\thinspace .
\end{align*}
$$

The global solution is set as $u(t_{n+1}) = u^{\stepone}(t_{n+1})$.

There is no use in combining higher-order methods with
ordinary splitting since the error due to splitting is $\Oof{\Delta
t}$, but for Strang splitting it makes sense to use schemes of order
$\Oof{\Delta t^2}$.

With the notation introduced for Strang splitting, we may express
ordinary first-order splitting as

$$
\begin{align*}
\frac{du^{\stepzero}}{dt} &= f_0(u^{\stepzero}),\quad u^{\stepzero}(t_n)=u(t_n),
\quad t\in [t_n,t_n+\Delta t],\\
\frac{du^{\stepone}}{dt} &= f_1(u^{\stepone}),\quad u^{\stepone}(t_n)=u^{\stepzero}(t_{n+1}),
\quad t\in [t_n,t_n+\Delta t],
\end{align*}
$$

with global solution set as $u(t_{n+1}) = u^{\stepone}(t_{n+1})$.

## Example: Logistic growth

Let us split the (scaled) logistic equation

$$
u'=u(1-u),\quad u(0)=0.1,
$$

with solution $u=(9e^{-t}+1)^{-1}$, into

$$
u'=u - u^2 = f_0(u) + f_1(u), \quad f_0(u)=u,\quad f_1(u)=-u^2\thinspace .
$$

We solve $u'=f_0(u)$ and $u'=f_1(u)$ by a Forward Euler step.
In addition, we add a method where we solve $u'=f_0(u)$ analytically,
since the equation is actually $u'=u$ with solution $e^t$.
The software that accompanies the following methods is the file
[`split_logistic.py`](${src_nonlin}/split_logistic.py).

### Splitting techniques

Ordinary splitting takes a Forward Euler step for each of the ODEs
according to

$$
\begin{equation}
\frac{u^{\stepzero,n+1} - u^{\stepzero,n}}{\Delta t} =
f_0(u^{\stepzero,n}),\quad
u^{\stepzero,n} = u(t_n),\quad t\in [t_n,t_n+\Delta t],
\label{_auto2} \tag{4}
\end{equation}
$$

$$
\begin{equation}
\frac{u^{\stepone,n+1} - u^{\stepone, n}}{\Delta t} = f_1(u^{\stepone,n}),\quad
u^{\stepone,n} = u^{\stepzero,n+1},\quad t\in [t_n,t_n+\Delta t],
\label{_auto3} \tag{5}
\end{equation}
$$

with $u(t_{n+1}) = u^{\stepone,n+1}$.

Strang splitting takes the form

$$
\begin{equation}
\frac{u^{\stepzero,n+\frac{1}{2}} - u^{\stepzero,n}}{\frac{1}{2}\Delta t} =
f_0(u^{\stepzero,n}),\quad
u^{\stepzero,n} = u(t_n),\quad t\in [t_n,t_n+\frac{1}{2}\Delta t],
\label{_auto4} \tag{6}
\end{equation}
$$

$$
\begin{equation}
\frac{u^{\stephalf,n+1}-u^{\stephalf,n}}{\Delta t} =
f_1(u^{\stephalf,n}),\quad
u^{\stephalf,n} = u^{\stepzero, n+\frac{1}{2}},\quad t\in [t_n,t_n+\Delta t],
\label{_auto5} \tag{7}
\end{equation}
$$

$$
\begin{equation}
\frac{u^{\stepone, n+1} - u^{\stepone, n+\frac{1}{2}}}{\frac{1}{2}\Delta t} =
f_0(u^{\stepone,n+\frac{1}{2}}),\quad
u^{\stepone,n+\frac{1}{2}} = u^{\stephalf,n+1},\quad
t\in [t_n+\frac{1}{2}\Delta t, t_n+\Delta t]\thinspace .
\label{_auto6} \tag{8}
\end{equation}
$$

### Verbose implementation

The following function computes four solutions arising from the Forward
Euler method, ordinary splitting, Strang splitting, as well as Strang splitting
with exact treatment of $u'=f_0(u)$:

```python
import numpy as np

def solver(dt, T, f, f_0, f_1):
    """
    Solve u'=f by the Forward Euler method and by ordinary and
    Strang splitting: f(u) = f_0(u) + f_1(u).
    """
    Nt = int(round(T/float(dt)))
    t = np.linspace(0, Nt*dt, Nt+1)
    u_FE = np.zeros(len(t))
    u_split1 = np.zeros(len(t))  # 1st-order splitting
    u_split2 = np.zeros(len(t))  # 2nd-order splitting
    u_split3 = np.zeros(len(t))  # 2nd-order splitting w/exact f_0

    # Set initial values
    u_FE[0] = 0.1
    u_split1[0] = 0.1
    u_split2[0] = 0.1
    u_split3[0] = 0.1

    for n in range(len(t)-1):
        # Forward Euler method
        u_FE[n+1] = u_FE[n] + dt*f(u_FE[n])

        # --- Ordinary splitting ---
        # First step
        u_s_n = u_split1[n]
        u_s = u_s_n + dt*f_0(u_s_n)
        # Second step
        u_ss_n = u_s
        u_ss = u_ss_n + dt*f_1(u_ss_n)
        u_split1[n+1] = u_ss

        # --- Strang splitting ---
        # First step
        u_s_n = u_split2[n]
        u_s = u_s_n + dt/2.*f_0(u_s_n)
        # Second step
        u_sss_n = u_s
        u_sss = u_sss_n + dt*f_1(u_sss_n)
        # Third step
        u_ss_n = u_sss
        u_ss = u_ss_n + dt/2.*f_0(u_ss_n)
        u_split2[n+1] = u_ss

        # --- Strang splitting using exact integrator for u'=f_0 ---
        # First step
        u_s_n = u_split3[n]
        u_s = u_s_n*np.exp(dt/2.)  # exact
        # Second step
        u_sss_n = u_s
        u_sss = u_sss_n + dt*f_1(u_sss_n)
        # Third step
        u_ss_n = u_sss
        u_ss = u_ss_n*np.exp(dt/2.)  # exact
        u_split3[n+1] = u_ss

    return u_FE, u_split1, u_split2, u_split3, t
```

### Compact implementation

We have used quite many lines for the steps in the splitting methods.
Many will prefer to condense the code a bit, as done here:

```python
# Ordinary splitting
u_s = u_split1[n] + dt*f_0(u_split1[n])
u_split1[n+1] = u_s + dt*f_1(u_s)

# Strang splitting
u_s = u_split2[n] + dt/2.*f_0(u_split2[n])
u_sss = u_s + dt*f_1(u_s)
u_split2[n+1] = u_sss + dt/2.*f_0(u_sss)

# Strang splitting using exact integrator for u'=f_0
u_s = u_split3[n]*np.exp(dt/2.)  # exact
u_ss = u_s + dt*f_1(u_s)
u_split3[n+1] = u_ss*np.exp(dt/2.)
```

### Results

[Figure](#nonlin:splitting:ODE:logistic:fig) shows that the impact of
splitting is significant. Interestingly, however,
the Forward Euler method applied to the entire problem directly is much more
accurate than any of the splitting schemes. We also see that Strang
splitting is definitely more accurate than ordinary splitting and that
it helps a bit to use an exact solution of $u'=f_0(u)$.  With a large
time step ($\Delta t = 0.2$, left plot in [Figure](#nonlin:splitting:ODE:logistic:fig)), the asymptotic values are off
by 20-30%.  A more reasonable time step ($\Delta t = 0.05$, right plot
in [Figure](#nonlin:splitting:ODE:logistic:fig)) gives better
results, but still the asymptotic values are up to 10% wrong.

As technique for solving nonlinear ODEs, we realize that the present
case study is not particularly promising, as the Forward Euler method
both linearizes the original problem and provides a solution that is
much more accurate than any of the splitting techniques.
In complicated multi-physics settings, on the other hand, splitting
may be the only feasible way to go, and sometimes you really need to apply
different numerics to different parts of a PDE problem.
But in very simple problems, like the logistic ODE, splitting is just an
inferior technique. Still, the logistic ODE is ideal for introducing all
the mathematical details and for investigating the behavior.

Effect of ordinary and Strang splitting for the logistic equation.

## Reaction-diffusion equation

Consider a diffusion equation coupled to chemical reactions modeled by
a nonlinear term $f(u)$:

$$
\frac{\partial u}{\partial t} = \dfc\nabla^2u + f(u)\thinspace .
$$

This is a physical process composed of two individual processes:
$u$ is the concentration of a substance that is locally generated by
a chemical reaction $f(u)$, while $u$ is spreading in space because
of diffusion. There are obviously two time scales: one for the chemical
reaction and one for diffusion. Typically, fast chemical reactions require
much finer time stepping than slower diffusion processes.
mathcal{I}_t could therefore be advantageous to split
the two physical effects in separate models and use different numerical methods
for the two.

A natural spitting in the present case is

$$
\begin{equation}
\frac{\partial u^{\stepzero}}{\partial t} = \dfc\nabla^2 u^{\stepzero},
\label{nonlin:splitting:RD:eq:diffu} \tag{9}
\end{equation}
$$

$$
\begin{equation}
\frac{\partial u^{\stepone}}{\partial t} = f(u^{\stepone})
\label{nonlin:splitting:RD:eq:decay} \tag{10}\thinspace .
\end{equation}
$$

Looking at these familiar problems,
we may apply a $\theta$ rule (implicit) scheme for
([9](#nonlin:splitting:RD:eq:diffu)) over one time step
and avoid dealing with nonlinearities
by applying an explicit scheme for ([10](#nonlin:splitting:RD:eq:decay))
over the same time step.

Suppose we have some solution $u$ at time level $t_n$. For flexibility,
we define a $\theta$ method for the diffusion part
([9](#nonlin:splitting:RD:eq:diffu)) by

$$
[D_t u^\stepzero =
\dfc (D_xD_x u^\stepzero + D_y D_y u^\stepzero)]^{n+\theta}\thinspace .
$$

We use $u^{n}$ as initial condition for $u^\stepzero$.

The reaction part, which is defined at each mesh point (without coupling
values in different mesh points), can employ any scheme for an ODE.
Here we use an Adams-Bashforth method of order 2. Recall that the overall
accuracy of the splitting method
is maximum $\Oof{\Delta t^2}$ for Strang splitting,
otherwise it is just $\Oof{\Delta t}$. Higher-order methods for ODEs will
therefore be a waste of work. The 2nd-order Adams-Bashforth method reads

$$
\begin{equation}
u^{\stepone,n+1}_{i,j} = u^{\stepone,n}_{i,j} +
\frac{1}{2}\Delta t\left( 3f(u^{\stepone, n}_{i,j}, t_n) -
f(u^{\stepone, n-1}_{i,j}, t_{n-1})
\right)
\thinspace .
\label{_auto7} \tag{11}
\end{equation}
$$

We can use a Forward Euler step to start the method, i.e, compute
$u^{\stepone,1}_{i,j}$.

The algorithm goes like this:

1. Solve the diffusion problem for one time step as usual.

2. Solve the reaction ODEs at each mesh point in $[t_n,t_n+\Delta t]$,
   using the diffusion solution in 1. as initial condition.
   The solution of the ODEs constitutes the solution of the original problem
   at the end of each time step.

We may use a much smaller time step when solving the reaction part, adapted
to the dynamics of the problem $u'=f(u)$. This gives great flexibility in
splitting methods.

[hpl 1: Can you make an example? I have started in `src/split_diffu_react.py`.]

## Example: Reaction-Diffusion with linear reaction term

The methods above may be explored in detail through a specific
computational example in which we compute the convergence rates
associated with four different solution approaches for the
reaction-diffusion equation with a linear reaction term,
i.e. $f(u)=-bu$.  The methods comprise solving without splitting (just
straight Forward Euler), ordinary splitting, first order Strang
splitting, and second order Strang splitting.  In all four methods, a
standard centered difference approximation is used for the spatial
second derivative. The methods share the error model $E = C h^r$,
while differing in the step $h$ (being either $\Delta x^2$ or $\Delta
x$) and the convergence rate $r$ (being either 1 or 2).

All code commented below is found in the file
[`split_diffu_react.py`](${src_nonlin}/split_diffu_react.py). When executed,
a function `convergence_rates` is called, from which all convergence
rate computations are handled:

```python
def convergence_rates(scheme='diffusion'):

    F = 0.5     # Upper limit for FE (stability). For CN, this
                # limit does not apply, but for simplicity, we
                # choose F = 0.5 as the initial F value.
    T = 1.2
    a = 3.5
    b = 1
    L = 1.5
    k = np.pi/L

    def exact(x, t):
        '''exact sol. to: du/dt = a*d^2u/dx^2 - b*u'''
        return np.exp(-(a*k**2 + b)*t) * np.sin(k*x)

    def f(u, t):
        return -b*u

    def I(x):
        return exact(x, 0)

    global error    # error computed in the user action function
    error = 0

    # Convergence study
    def action(u, x, t, n):
        global error
        if n == 1:      # New simulation, - reset error
            error = 0
        else:
            error = max(error, np.abs(u - exact(x, t[n])).max())

    E = []
    h = []
    Nx_values = [10, 20, 40, 80]   # i.e., dx halved each time
    for Nx in Nx_values:
        dx = L/Nx
        if scheme == 'Strang_splitting_2ndOrder':
            print 'Strang splitting with 2nd order schemes...'
            # In this case, E = C*h**r (with r = 2) and since
            # h = dx = K*dt, the ratio dt/dx must be constant.
            # To fulfill this demand, we must let F change
            # when dx changes. From F = a*dt/dx**2, it follows
            # that halving dx AND doubling F assures dt/dx const.
            # Initially, we simply choose F = 0.5.

            dt = F/a*dx**2
            #print 'dt/dx:', dt/dx
            Nt = int(round(T/float(dt)))
            t = np.linspace(0, Nt*dt, Nt+1)   # global time
            Strang_splitting_2ndOrder(I=I, a=a, b=b, f=f, L=L, dt=dt,
                                      dt_Rfactor=1, F=F, t=t, T=T,
                                      user_action=action)
            h.append(dx)
            # prepare for next iteration (make F match dx/2)
            F = F*2       # assures dt/dx const. when dx = dx/2
        else:
            # In these cases, E = C*h**r (with r = 1) and since
            # h = dx**2 = K*dt, the ratio dt/dx**2 must be constant.
            # This is fulfilled by choosing F = 0.5 (for FE stability)
            # and make sure that F, dx and dt comply to F = a*dt/dx**2.
            dt = F/a*dx**2
            Nt = int(round(T/float(dt)))
            t = np.linspace(0, Nt*dt, Nt+1)   # global time
            if scheme == 'diffusion':
                print 'FE on whole eqn...'
                diffusion_theta(I, a, f, L, dt, F, t, T,
                                step_no=0, theta=0,
                                u_L=0, u_R=0, user_action=action)
                h.append(dx**2)
            elif scheme == 'ordinary_splitting':
                print 'Ordinary splitting...'
                ordinary_splitting(I=I, a=a, b=b, f=f, L=L, dt=dt,
                                   dt_Rfactor=1, F=F, t=t, T=T,
                                   user_action=action)
                h.append(dx**2)
            elif scheme == 'Strang_splitting_1stOrder':
                print 'Strang splitting with 1st order schemes...'
                Strang_splitting_1stOrder(I=I, a=a, b=b, f=f, L=L, dt=dt,
                                          dt_Rfactor=1, F=F, t=t, T=T,
                                          user_action=action)
                h.append(dx**2)
            else:
                print 'Unknown scheme requested!'
                sys.exit(0)

            #print 'dt/dx**2:', dt/dx**2

        E.append(error)
        Nx *= 2         # Nx doubled gives dx/2

    print 'E:', E
    print 'h:', h

    # Convergence rates
    r = [np.log(E[i]/E[i-1])/np.log(h[i]/h[i-1])
         for i in range(1,len(Nx_values))]
    print 'Computed rates:', r

if __name__ == '__main__':

    schemes = ['diffusion',
               'ordinary_splitting',
               'Strang_splitting_1stOrder',
               'Strang_splitting_2ndOrder']

    for scheme in schemes:
        convergence_rates(scheme=scheme)
```

Now, with respect to the error ($E = C h^r$), the Forward Euler
scheme, the ordinary splitting scheme and first order Strang splitting
scheme are all first order ($r = 1$), with a step $h = \Delta x^2 =
K^{-1}\Delta t$, where $K$ is some constant. This implies that the
*ratio* $\frac{\Delta t}{\Delta x^2}$ must be held constant during
convergence rate calculations. Furthermore, the Fourier number $F =
\frac{\alpha \Delta t}{\Delta x^2}$ is upwards limited to $F = 0.5$,
being the stability limit with explicit schemes. Thus, in these cases,
we use the fixed value of $F$ and a given (but changing) spatial
resolution $\Delta x$ to compute the corresponding value of $\Delta t$
according to the expression for $F$.  This assures that $\frac{\Delta
t}{\Delta x^2}$ is kept constant. The loop in `convergence_rates` runs
over a chosen set of grid points (`Nx_values`) which gives a doubling
of spatial resolution with each iteration ($\Delta x$ is halved).

For the second order Strang splitting scheme, we have $r = 2$ and a
step $h = \Delta x = K^{-1}\Delta t$, where $K$ again is some
constant. In this case, it is thus the ratio $\frac{\Delta t}{\Delta
x}$ that must be held constant during the convergence rate
calculations. From the expression for $F$, it is clear then that $F$
must change with each halving of $\Delta x$. In fact, if $F$ is
doubled each time $\Delta x$ is halved, the ratio $\frac{\Delta
t}{\Delta x}$ will be constant (this follows, e.g., from the
expression for $F$). This is utilized in our code.

A solver `diffusion_theta` is used in each of the four solution approaches:

```python
def diffusion_theta(I, a, f, L, dt, F, t, T, step_no, theta=0.5,
                    u_L=0, u_R=0, user_action=None):
    """
    Full solver for the model problem using the theta-rule
    difference approximation in time (no restriction on F,
    i.e., the time step when theta >= 0.5).     Vectorized
    implementation and sparse (tridiagonal) coefficient matrix.
    Note that t always covers the whole global time interval, whether
    splitting is the case or not. T, on the other hand, is
    the end of the global time interval if there is no split,
    but if splitting, we use T=dt. When splitting, step_no
    keeps track of the time step number (for lookup in t).
    """

    Nt = int(round(T/float(dt)))
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)       # Mesh points in space
    # Make sure dx and dt are compatible with x and t
    dx = x[1] - x[0]
    dt = t[1] - t[0]

    u   = np.zeros(Nx+1)   # solution array at t[n+1]
    u_1 = np.zeros(Nx+1)   # solution at t[n]

    # Representation of sparse matrix and right-hand side
    diagonal = np.zeros(Nx+1)
    lower    = np.zeros(Nx)
    upper    = np.zeros(Nx)
    b        = np.zeros(Nx+1)

    # Precompute sparse matrix (scipy format)
    Fl = F*theta
    Fr = F*(1-theta)
    diagonal[:] = 1 + 2*Fl
    lower[:] = -Fl  #1
    upper[:] = -Fl  #1
    # Insert boundary conditions
    diagonal[0] = 1
    upper[0] = 0
    diagonal[Nx] = 1
    lower[-1] = 0

    diags = [0, -1, 1]
    A = scipy.sparse.diags(
        diagonals=[diagonal, lower, upper],
        offsets=[0, -1, 1], shape=(Nx+1, Nx+1),
        format='csr')
    #print A.todense()

    # Allow f to be None or 0
    if f is None or f == 0:
        f = lambda x, t: np.zeros((x.size)) \
            if isinstance(x, np.ndarray) else 0

    # Set initial condition
    if isinstance(I, np.ndarray):   # I is an array
        u_1 = np.copy(I)
    else:                           # I is a function
        for i in range(0, Nx+1):
            u_1[i] = I(x[i])

    if user_action is not None:
        user_action(u_1, x, t, step_no+0)

    # Time loop
    for n in range(0, Nt):
        b[1:-1] = u_1[1:-1] + \
                  Fr*(u_1[:-2] - 2*u_1[1:-1] + u_1[2:]) + \
                  dt*theta*f(u_1[1:-1], t[step_no+n+1]) + \
                  dt*(1-theta)*f(u_1[1:-1], t[step_no+n])
        b[0] = u_L; b[-1] = u_R  # boundary conditions
        u[:] = scipy.sparse.linalg.spsolve(A, b)

        if user_action is not None:
            user_action(u, x, t, step_no+(n+1))

        # Update u_1 before next step
        u_1, u = u, u_1

    # u is now contained in u_1 (swapping)
    return u_1
```

For the no splitting approach with Forward Euler in time, this solver
handles both the diffusion and the reaction term. When splitting,
`diffusion_theta` takes care of the diffusion term only, while the
reaction term is handled either by a Forward Euler scheme in
`reaction_FE`, or by a second order Adams-Bashforth scheme from
Odespy. The `reaction_FE` function covers one complete time step `dt`
during ordinary splitting, while Strang splitting (both first and
second order) applies it with `dt/2` twice during each time step
`dt`. Since the reaction term typically represents a much faster
process than the diffusion term, a further refinement of the time step
is made possible in `reaction_FE`. mathcal{I}_t was implemented as

```python
def reaction_FE(I, f, L, Nx, dt, dt_Rfactor, t, step_no,
                user_action=None):
    """Reaction solver, Forward Euler method.
    Note the at t covers the whole global time interval.
    dt is either one complete,or one frac{1}{2}, of the step in the
    diffusion part, i.e. there is a local time interval
    [0, dt] or [0, dt/2] that the reaction_FE
    deals with each time it is called. step_no keeps
    track of the (global) time step number (required
    for lookup in t).
    """

    u = np.copy(I)
    dt_local = dt/float(dt_Rfactor)
    Nt_local = int(round(dt/float(dt_local)))
    x = np.linspace(0, L, Nx+1)

    for n in range(Nt_local):
        time = t[step_no] + n*dt_local
        u[1:Nx] = u[1:Nx] + dt_local*f(u[1:Nx], time)

    # BC already inserted in diffusion step, i.e. no action here
    return u
```

With the ordinary splitting approach, each time step `dt` is covered
twice. First computing the impact of the reaction term, then the
contribution from the diffusion term:

```python
def ordinary_splitting(I, a, b, f, L, dt,
                       dt_Rfactor, F, t, T,
                       user_action=None):
    '''1st order scheme, i.e. Forward Euler is enough for both
    the diffusion and the reaction part. The time step dt is
    given for the diffusion step, while the time step for the
    reaction part is found as dt/dt_Rfactor, where dt_Rfactor >= 1.
    '''
    Nt = int(round(T/float(dt)))
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)       # Mesh points in space
    u = np.zeros(Nx+1)

    # Set initial condition u(x,0) = I(x)
    for i in range(0, Nx+1):
        u[i] = I(x[i])

    # In the following loop, each time step is "covered twice",
    # first for reaction, then for diffusion
    for n in range(0, Nt):
        # Reaction step (potentially many smaller steps within dt)
        u_s = reaction_FE(I=u, f=f, L=L, Nx=Nx,
                        dt=dt, dt_Rfactor=dt_Rfactor,
                        t=t, step_no=n,
                        user_action=None)

        u = diffusion_theta(I=u_s, a=a, f=0, L=L, dt=dt, F=F,
                              t=t, T=dt, step_no=n, theta=0,
                              u_L=0, u_R=0, user_action=None)

        if user_action is not None:
            user_action(u, x, t, n+1)

    return
```

For the two Strang splitting approaches, each time step `dt` is
handled by first computing the reaction step for (the first) `dt/2`,
followed by a diffusion step `dt`, before the reaction step is treated
once again for (the remaining) `dt/2`. Since first order Strang
splitting is no better than first order accurate, both the reaction
and diffusion steps are computed explicitly. The solver was
implemented as

```python
def Strang_splitting_1stOrder(I, a, b, f, L, dt, dt_Rfactor,
                              F, t, T, user_action=None):
    '''Strang splitting while still using FE for the reaction
    step and for the diffusion step. Gives 1st order scheme.
    The time step dt is given for the diffusion step, while
    the time step for the reaction part is found as
    0.5*dt/dt_Rfactor, where dt_Rfactor >= 1. Introduce an
    extra time mesh t2 for the reaction part, since it steps dt/2.
    '''
    Nt = int(round(T/float(dt)))
    t2 = np.linspace(0, Nt*dt, (Nt+1)+Nt)   # Mesh points in diff
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)
    u = np.zeros(Nx+1)

    # Set initial condition u(x,0) = I(x)
    for i in range(0, Nx+1):
        u[i] = I(x[i])

    for n in range(0, Nt):
        # Reaction step (1/2 dt: from t_n to t_n+1/2)
        # (potentially many smaller steps within dt/2)
        u_s = reaction_FE(I=u, f=f, L=L, Nx=Nx,
                        dt=dt/2.0, dt_Rfactor=dt_Rfactor,
                        t=t2, step_no=2*n,
                        user_action=None)
        # Diffusion step (1 dt: from t_n to t_n+1)
        u_sss = diffusion_theta(I=u_s, a=a, f=0, L=L, dt=dt, F=F,
                              t=t, T=dt, step_no=n, theta=0,
                              u_L=0, u_R=0, user_action=None)
        # Reaction step (1/2 dt: from t_n+1/2 to t_n+1)
        # (potentially many smaller steps within dt/2)
        u = reaction_FE(I=u_sss, f=f, L=L, Nx=Nx,
                        dt=dt/2.0, dt_Rfactor=dt_Rfactor,
                        t=t2, step_no=2*n+1,
                        user_action=None)

        if user_action is not None:
            user_action(u, x, t, n+1)

    return
```

The second order version of the Strang splitting approach utilizes a
second order Adams-Bashforth solver for the reaction part and a
Crank-Nicolson scheme for the diffusion part. The solver has the same
structure as the one for first order Strang splitting and was
implemented as

```python
def Strang_splitting_2ndOrder(I, a, b, f, L, dt, dt_Rfactor,
                              F, t, T, user_action=None):
    '''Strang splitting using Crank-Nicolson for the diffusion
    step (theta-rule) and Adams-Bashforth 2 for the reaction step.
    Gives 2nd order scheme. Introduce an extra time mesh t2 for
    the reaction part, since it steps dt/2.
    '''
    import odespy
    Nt = int(round(T/float(dt)))
    t2 = np.linspace(0, Nt*dt, (Nt+1)+Nt)   # Mesh points in diff
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)
    u = np.zeros(Nx+1)

    # Set initial condition u(x,0) = I(x)
    for i in range(0, Nx+1):
        u[i] = I(x[i])

    reaction_solver = odespy.AdamsBashforth2(f)

    for n in range(0, Nt):
        # Reaction step (1/2 dt: from t_n to t_n+1/2)
        # (potentially many smaller steps within dt/2)
        reaction_solver.set_initial_condition(u)
        t_points = np.linspace(0, dt/2.0, dt_Rfactor+1)
        u_AB2, t_ = reaction_solver.solve(t_points) # t_ not needed
        u_s = u_AB2[-1,:]  # pick sol at last point in time

        # Diffusion step (1 dt: from t_n to t_n+1)
        u_sss = diffusion_theta(I=u_s, a=a, f=0, L=L, dt=dt, F=F,
                              t=t, T=dt, step_no=n, theta=0.5,
                              u_L=0, u_R=0, user_action=None)
        # Reaction step (1/2 dt: from t_n+1/2 to t_n+1)
        # (potentially many smaller steps within dt/2)
        reaction_solver.set_initial_condition(u_sss)
        t_points = np.linspace(0, dt/2.0, dt_Rfactor+1)
        u_AB2, t_ = reaction_solver.solve(t_points) # t_ not needed
        u = u_AB2[-1,:]  # pick sol at last point in time

        if user_action is not None:
            user_action(u, x, t, n+1)

    return
```

When executing `split_diffu_react.py`, we find that the estimated
convergence rates are as expected. The second order Strang splitting
gives the least error (about $4e^{-5}$) and has second order
convergence ($r = 2$), while the remaining three approaches have first
order convergence ($r = 1$).

## Analysis of the splitting method

Let us address a linear PDE problem for which we can develop analytical
solutions of the discrete equations, with and without splitting, and discuss
these. Choosing $f(u)=-\beta u$ for a constant $\beta$ gives a linear
problem. We use the Forward Euler method for both the PDE and ODE problems.

We seek a 1D Fourier wave component solution of the problem, assuming
homogeneous Dirichlet conditions at $x=0$ and $x=L$:

$$
u = e^{-\dfc k^2 t - \beta t}\sin kx,\quad k = \frac{\pi}{L}\thinspace .
$$

This component fits the 1D PDE problem ($f=0$). On complex form we can
write

$$
u = e^{-\dfc k^2 t - \beta t + ikx},
$$

where $i=\sqrt{-1}$ and the imaginary part is taken as the physical solution.

We refer to the section "Analysis of schemes for
the diffusion equation": "" in [[Langtangen_Linge]](#Langtangen_Linge) and to
the book [[Langtangen_decay]](#Langtangen_decay) for a discussion of exact numerical
solutions to diffusion and decay problems, respectively.  The key idea
is to search for solutions $A^ne^{ikx}$ and determine $A$.  For the
diffusion problem solved by a Forward Euler method one has

$$
A = 1 - 4F\sin^p,
$$

where $F=\dfc\Delta t/\Delta x^2$ is the mesh Fourier number and $p=k\Delta x/2$
is a dimensionless number reflecting the spatial resolution (number of points
per wave length in space). For the decay problem $u'=-\beta u$, we have
$A=1 - q$, where $q$ is a dimensionless parameter for the resolution
in the decay problem: $q = \beta\Delta t$.

The original model problem can also be discretized by a Forward Euler scheme,

$$
[D^+_t u = \dfc D_xD_x u - \beta u]^n_i\thinspace .
$$

Assuming $A^ne^{ikx}$ we find that

$$
u^n_i = (1 - 4F\sin^p -q)^n\sin kx\thinspace .
$$

We are particularly interested in what happens at one time step. That is,

$$
u^{n}_{i} = (1-4F\sin^2 p)u^{n-1}_i\thinspace .
$$

In the two stage algorithm, we first compute the diffusion step

$$
u^{\stepzero,n+1}_i = (1 - 4F\sin^2 p)u^{n-1}_i\thinspace .
$$

Then we use this as input to the decay algorithm and arrive at

$$
u^{\stepone,n+1} = (1-q)u^{\stepzero,n+1} = (1-q)(1-4F\sin^2 p) u^{n-1}_i\thinspace .
$$

The splitting approximation over one step is therefore

$$
E = 1 - 4F\sin^p -q - (1-q)(1-4F\sin^2 p) = -q(2 - F\sin^2 p))\thinspace .
$$


Alumno: David Cabezas Berrido

DNI: 20079906D

# Números naturales. Inducción. Recursividad.

## Naturales

En esta sección vamos a trabajar con $\mathbb N $

Pero antes, una breve introducción a Jupyter notebook:

1. La tecla a crea una nueva celda encima de la seleccionada.
1. Si pulsamos m, la celda pasa a ser de texto.
1. Dos veces d sobre una celda, la elimina.
1. Shift + tab o ? al final muestran una pequeña ayuda de las funciones.
1. El tabulador autocompleta.
1. Jupyter notebook acepta programación en python y LaTex, como puede
   comprobarse en la $\mathbb N $ de naturales o en la enumeración de esta lista.

Dicho esto, comencemos.

En `python` podemos utilizar `isinstance` para determinar si un objeto es un entero

```python
isinstance(4,int)
```

    True

```python
isinstance([3.4],int)
```

    False

Podemos con esta función definir otra que detecte si un número es natural

```python
def isnatural(n):
    if not isinstance(n,int):         # Descartamos decimales
        return False
    return n>=0                       # Nos aseguramos de que no sea negativo
```

```python
isnatural(3)
```

    True

```python
isnatural(-3)
```

    False

La funciones sucesor y predecesor quedarían como sigue

```python
sucesor = lambda x: x+1   # lambda permite definir funciones rápidamente
```

```python
sucesor(2)
```

    3

Generador de errores:

```python
def prec(n):
    if not isnatural(n):
        raise TypeError("El argumento debe ser un número natural")
    if n==0:
        raise ValueError("El 0 no tiene predecesor")
    return n-1
```

```python
prec(1)
```

    0

```python
prec(27)
```

    26

```python
prec(0)
```

La función predecesor está bastante completa, sin embargo la función sucesor es mejorable, ahora sabemos como hacerla.

```python
sucesor(-1.5)
```

    -0.5

```python
def suc(n):
    if not isnatural(n):
        raise TypeError("Debe introducir un número natural")
    return n+1
```

```python
suc(1.5)
```

```python
sucesor(28)
```

    29

Así mejor, ahora podemos definir la suma recursivamente, como hemos visto en teoría

```python
def suma(m,n):
    if n == 0:
        return m
    return sucesor(suma(m,prec(n)))
```

```python
suma(2,3)
```

    5

```python
suma(1000,1)
```

    1001

Pero no cantemos victoria tan pronto.

```python
suma(1,1000)
```

    1001

Definamos también el producto, también de forma poco eficiente

```python
def producto(m,n):
    if n == 0:
        return 0
    return suma(producto(m,prec(n)),m)
```

```python
producto(7,6)
```

    42

```python
producto(500,500)
```

    250000

Tal como dijimos, poco eficiente. Veremos la recursividad en profundidad más adelante.

## Sucesiones, inducción

Con la librería `sympy` podemos hacer cálculo simbólico. En particular, podemos calcular el valor de algunas sumas con parámetros.

```python
import sympy as sp                # Tendre que poner sp. antes de las funciones
```

```python
from sympy import *               # Ya no
```

Cuando vamos a utilizar un símbolo tenemos que declararlo primero. Para el ejemplo que sigue vamos a utilizar `n` como entero e `i` como contador

```python
n, i = symbols("n, i", integer = True)
```

### Algunas sumatorias

Calculemos por ejemplo $\sum_{i=1}^n i$

```python
s = Sum(i,(i,1,n))
```

```python
s
```

    Sum(i, (i, 1, n))

Si queremos calcular el "valor" de esta sumatoria, podemos utilizar el método `doit`

```python
s.doit()
```

    n**2/2 + n/2

```python
pprint(_)   # Esta función lo imprime de forma más bonita, _ se refiere al último resultado obtenido
```

     2
    n    n
    ── + ─
    2    2

```python
summation(2,(i,1,n))    # Esta función lo hace directamente, en este caso estamos sumando 2 n veces
```

    2*n

Y una suma de potencias

```python
s=Sum(i**2,(i,1,n))  # Suma de los cuadrados de 1 a n
```

```python
s.doit()
```

    n**3/3 + n**2/2 + n/6

```python
s=Sum(i**30,(i,1,n))    # Suma de los número del 1 a n elevados a 30
```

```python
s.doit()
```

    n**31/31 + n**30/2 + 5*n**29/2 - 203*n**27/6 + 1131*n**25/2 - 16965*n**23/2 + 216775*n**21/2 - 2304485*n**19/2 + 19959975*n**17/2 - 137514723*n**15/2 + 731482225*n**13/2 - 31795091601*n**11/22 + 8053785025*n**9/2 - 102818379585*n**7/14 + 15626519181*n**5/2 - 23749461029*n**3/6 + 8615841276005*n/14322

```python
a = Symbol("a")
```

```python
Sum(a**i,(i,1,n)).doit()   # a^1 + a^2 + ... + a^n
```

    Piecewise((n, Eq(a, 1)), ((a - a**(n + 1))/(-a + 1), True))

### Inducción

Veamos cómo podemos utilizar `sympy` para hacer algunos ejemplos de inducción

Por ejemplo, veamos que para todo $6\mid 7^n-1$ para todo $n\in \mathbb N$. Empezamos definiendo una función que nos devuelva $7^n-1$

```python
f = lambda n: 7**n -1
```

Primero veamos qué valor tiene en el 0

```python
f(0) % 6     # Se cumple para 0
```

    0

Si por hipótesis de inducción $f(n)$ es un múltiplo de 6, entonces para probar que $f(n+1)$ también lo es, bastará demostrar que la diferencia $f(n+1)-f(n)$ es un múltiplo de 6

```python
simplify((f(n+1)-f(n)) % 6)
```

    6*Mod(7**n, 1)

Esto significa que es igual a 6 por el resto de dividir $7^n$ entre 1, que obviamente será 0, luego $f(n+1)-f(n)$ se diferencian en un múltiplo de 6. Se confirma así que para cualquier $n\in \mathbb N$, $7^n -1$ en múltiplo de 6.

Ahora demostremos que $7^{2n}+16n-1$ es múltiplo de 64 para cuarquier $n\in \mathbb N$

```python
f = lambda n: 7**(2*n)+16*n-1
```

```python
f(0)
```

    0

```python
simplify((f(n+1)-f(n)) % 64)
```

    16*Mod(3*7**(2*n) + 1, 4)

Esta vez $3\times 7^{2n}+1$ debe de ser múltiplo de $4$, lo cual no es tan obvio habremos acabado. Nueva inducción:

```python
g = lambda n: 3*7**(2*n)+1
```

```python
g(0)
```

    4

```python
simplify((g(n+1)-g(n))%4)
```

    0

$3\times7^{2n}+1$ es múltiplo de 4 para todo $n\in \mathbb N$ y como consecuencia $7^{2n}+16n-1$ lo es de 64

Pongamos otro ejemplo: Demostrar que para todo número impar k, el resto de dividir $2^k$ entre 3 es
2.

Podemos escribir ese k como 2n+1

```python
f = lambda n: 2**(2*n+1)
```

```python
f(0) % 3      # 0 la cumple
```

    2

```python
simplify(f(n+1) - f(n))
```

    6*2**(2*n)

```python
Mod(_,3)
```

    0

Si $f(n+1)$ y $f(n)$ se diferencian en un múltiplo de 3, el resto de dividir $f(n+1)$ entre 3 será el mismo que el de dividir $f(n)$ entre 3, Por lo que si se cumple para un $n\in \mathbb N$, se cumplirá para $n+1$.

Vamos con el último. Demostrar que la sumatoria de las quintas potencias más las séptimas potencias de los naturales del 1 al n es igual a $2\times (\frac{n^2+n}{2})^4$ para cualquier n natural mayor o igual a 1.

```python
k = Symbol("k",integer=true)
```

```python
f = lambda n: 2*((n**2+n)/2)**4
```

```python
g = lambda n: summation(k**5+k**7,(k,1,n))
```

```python
simplify((f(n+1) - f(n)) - (g(n+1) - g(n)))
```

    0

Luego $f(n+1) - f(n)$ es igual a $g(n+1) - g(n)$. Y suponiendo $f(n) = g(n)$ se llega a $f(n+1) = g(n+1)$. Lo que termina la demostración.

### Recursividad e iteración

Veamos cómo las funciones recursivas se pueden acelerar usando memorización de los pasos anteriores

```python
fib = lambda n: n if n<2 else fib(n-2)+fib(n-1)    # Definimos una función recursiva de fibonacci
```

```python
fib(10)
```

    55

```python
fibonacci(10)
```

    55

Parece que funciona igual que la original, pero sólo para el lento tiempo de reacción de las personas y para números pequeños.

```python
import time   # Muestra tiempo

```

```python
start=time.time()
fib(35)
time.time()-start

```

    3.521246910095215

```python
start=time.time()
fibonacci(35)
time.time()-start
```

    0.0009148120880126953

Un posible solución a este problema es ir almacenando los resultados previos de cálculo, lo cual es conocido como [memoization](http://stackoverflow.com/questions/1988804/what-is-memoization-and-how-can-i-use-it-in-python)

```python
import functools
```

```python
@functools.lru_cache(maxsize=None)
def fibo(num):
    if num < 2:
        return num
    else:
        return fibo(num-1) + fibo(num-2)
```

```python
start=time.time()
fibo(35)
time.time()-start
```

    0.00027561187744140625

Hemos definido una función recursiva mucho más eficiente, pero requiere de almacenamiento en la memoria, luego es mejor pensar razonadamente y definir una función de fibonacci iterativa.

```python
def fibon(n):
    if (not isnatural(n)) or n == 0:                                        # Generador de errores
        raise TypeError("Debe introducir un número natural distinto de 0")
    else:
        if n == 1 or n == 2:              # Términos 1 y 2 de la sucesión
            return 1
        else:                             # Cálculo del término n para n >=3
            a = 1
            b = 1
            for i in range(n-2):          # Sumamos los dos anteriores términos n-2 veces
                f = a + b
                b = a                     # El número pasa a ser el anterior
                a = f                     # El anterior pasa a ser el anterior del anterior
        return f                          # Devuelvo el término n-ésimo

```

```python
fibon(35)
```

    9227465

```python
start=time.time()
fibon(35)
time.time()-start
```

    7.843971252441406e-05

Con esto queda confirmado que la recursividad es bastante mala en términos de eficiencia, aunque más sencilla de programar.
Antes continuar, definamos una función potencia igual a la vista en teoría.

```python
def potencia(m,n):
    if n == 0:
        return 1
    else:
        return producto(potencia(m,prec(n)),m)
```

```python
potencia(5,0)
```

    1

```python
potencia(2,7)
```

    128

Funciona, pero es una función recursiva que llama a la función producto, también recursiva y que a su vez llama a la función suma, también recursiva. Por lo que esta función es el colmo de la ineficiencia.

```python
potencia(3,9)
```

## Resolución de ecuaciones recursivas

Para sistemas de ecuaciones lineales, ulilizaremos linsolve().

Para sistemas de ecuaciones no lineales, utilizaremos solve(), que también resuelve las lineales pero tarda más por tener que determinar su tipo.

Para ecuaciones en recurrencia, utilizaremos rsolve(), del paquete `simpy`.

Empecemos con un ejemplo:
- $a(0)=0$,
- $a(1)=1$,
- $a(n+2)=5a(n+1)-6a(n)$.

Empezamos declarando `a` como función

```python
a = Function('a')
```

Tendríamos que resolver la ecuación $x^2-5x+6=0$, cuyas soluciones son 2 y 3. Comprobémoslo.

```python
rsolve(a(n+2)-5*a(n+1)+6*a(n),a(n))
```

    2**n*C0 + 3**n*C1

Sólo nos queda hallar la solución particular, aunque el mismo comando rsolve se puede encargar de ello si le damos
también los términos iniciales como parámetros.

```python
rsolve(a(n+2)-5*a(n+1)+6*a(n),a(n), {a(0):0,a(1):1})  # Diccionario
```

    -2**n + 3**n

En este caso serían C0=-1, C1=1.

También podemos usar el comando solve para resolverla mediante el polinomio característico

```python
x=Symbol("x")
solve(x**2-5*x+6)
```

    [2, 3]

Y por tanto la solución general será de la forma $a_n=u 2^n + v 3^n$, para ciertas constantes $u$ y $v$ que tenemos que encontrar a partir de las condiciones iniciales

Imponemos por tanto que $a_0=0=u+v$ y que $a_1=1=2u+3v$

```python
u,v = symbols("u,v")
solve([u+v,2*u+3*v-1])
```

    {u: -1, v: 1}

Por lo que $a_n=-2^n+3^n$, como habíamos visto arriba

Veamos un ejemplo de una que no sea homogénea

$a(n+2)=5a(n+1)-6a(n)+n$

```python
rsolve(a(n+2)-5*a(n+1)+6*a(n)-n,a(n))
```

    2**n*C0 + 3**n*C1 + C0*n

Como vemos es bastante rápido, ahora hagamos otro más paso a paso.

$a(0)=0$, $a(1)=1$, $a(n+2)=5a(n+1)-6a(n)+n\times3^n$

Cuando la ecuación está igualada a un número elevado a n por un polinomio en n, se añade multiplicando al polinomio característico $(x-b)^{r+1}$ donde b es el número y r el grado del polinomio en n.
El polinomio característico sería $(x^2-5x+6)(x-3)^2$

```python
factor(x**2-5*x+6)
```

    (x - 3)*(x - 2)

Las soluciones serían x=3 (sol. triple) y x=2. Por lo que la solución general tendría esta forma: $(An^2+Bn+C)\times 3^n + D\times 2^n$

Esta ecuación tiene las soluciones deseadas, pero también tiene otras que hemos añadido al obtener el polinomio característico, por lo que para simplificarla tomaremos parámetros generales en las condiciones iniciales.

- $a(0) = p = C + D$
- $a(1) = q = (A+B+C)\times 3 + D\times 2$
- $a(2) = 5\times a(1) - 6\times a(0) + 2\times 3^2 = 5q-6p+18 = (4A+2B+C)\times 9 + D\times 4$
- $a(3) = 5\times a(2) - 6\times a(1) + 3\times 3^3 = 5\times (5q-6p+18)-6q+81=(9A+3B+C)\times 27 + D\times 8$

Resolvemos el sistema y comprobamos que dos parámetros ahora son fijos.

```python
A, B, C, D, p, q = symbols("A, B, C, D, p, q")
```

```python
linsolve([C+D-p,(A+B+C)*3+D*2-q,(4*A+2*B+C)*9+D*4-(5*q-6*p+18),(9*A+3*B+C)*27+D*8-(5*(5*q-6*p+18)-6*q+81)],[A,B,C,D])
```

    {(3/2, -9/2, -2*p + q + 9, 3*p - q - 9)}

Ahora la ecuación general (la que tiene sólo las soluciones que nos interesan) tiene esta forma:

```python
g = lambda n:simplify(((3/2)*n**2+(-9/2)*n+(-2*p+q+9))*3**n+(3*p-q-9)*2**n)
```

```python
solve([g(0),g(1)-1],[p,q])
```

    {p: 0, q: 1}

Introducimos las condiciones especiales particulares y ya podemos hallar la solución particular.

```python
p = lambda n:simplify(((3/2)*n**2+(-9/2)*n+(-2*0+1+9))*3**n+(3*0-1-9)*2**n)
```

```python
p(n)       # Aquí la tenemos
```

    -10*2**n + 3**n*(1.5*n**2 - 4.5*n + 10)

Hagamos otro ejemplo: $a(0)+a(1)=2, a(1)+a(2)=4, a(n)+a(n-1)=2n$

Esta vez las condiciones iniciales vienen dadas de una forma peculiar. Aunque de eso nos ocuparemos luego, para homogeneizar y eliminar el $2n$ restaremos dos términos consecutivos de la ecuación.

```python
n = Symbol("n", integer=true)
a = Function('a')
```

```python
rsolve(a(n)-a(n-1)-n+1,a(n),{a(1):0})
```

    n*(n/2 - 1/2)

```python
pprint(_)
```

      ⎛n   1⎞
    n⋅⎜─ - ─⎟
      ⎝2   2⎠

```python
a(n)+a(n-1)-2*n - (a(n-1)+a(n-2)-2*(n-1))
```

    a(n) - a(n - 2) - 2

Ahora tenemos un molesto 2, que podemos eliminar repitiendo el proceso.

```python
a(n)-a(n-2)-2 -(a(n-1)-a(n-3)-2)
```

    a(n) + a(n - 3) - a(n - 2) - a(n - 1)

```python
solve(x**3-x**2-x+1,x)   # Esto no nos vale, no tiene en cuenta las soluciones múltiples
```

    [-1, 1]

```python
factor(x**3-x**2-x+1,x) # Ahora sí
```

    (x - 1)**2*(x + 1)

```python
g = lambda n: A*(-1)**n +(B*n+C)  # Solución general
```

Ahora debemos buscar una solución particular que cumpla   $a(1)+a(0)=2,   a(2)+a(1)=4$

Para esto, resolveremos el siguiente sistema:

- $a(0)+a(1) = 2$

- $a(1)+a(2) = 4$

```python
solve([g(0)+g(1)-2,g(2)+g(1)-4],[A,B,C])
```

    {B: 1, C: 1/2}

```python
p = lambda n: A(-1)**n + n + 1/2
```

```python
p(n)
```

    n + A(-1)**n + 0.5

Esta solución particular es válida para cualquier valor de A.

Ahora pasemos a ver la utilidad de esto, con ejemplo más prácticos.

El juego de las Torres de Hanói se trata de pasar una serie de discos de una varilla a otra, disponiendo de una tercera para auxliarse, reglas:
1. Sólo se puede mover un disco cada vez.
2. Un disco de mayor tamaño no puede descansar sobre uno más pequeño que él mismo.
3. Sólo puedes desplazar el disco que se encuentre arriba en cada varilla.

Calculemos el mínimo número de movimientos posibles para realizar el juego con n discos, $a(n)$.

Cuenta una leyenda que Dios encargó a los monjes de un monasterio de Hanói pasar 64 discos entre tres varillas de diamante, y que cuando acabaran, el mundo terminaría.

Para desplazar los $n$ discos, primero tendremos que desplazar todos menos el mayor a una de las varillas, $a(n)$; a continuación, desplazar el mayor de los discos a la otra varilla que queda libre ($1$); y por último, volver a colocar los $n-1$ primeros encima del mayor. Luego nos queda la siguiente ecuación de recurrencia:

$a(n)=2a(n-1)+1$

```python
rsolve(a(n)-2*a(n-1)-1,a(n),{a(0):0})   # Aunque no sea homogénea, rsolve puede hacerla directamente
```

    2**n - 1

```python
h = lambda n: 2**n - 1
```

```python
h(64)
```

    18446744073709551615

Si los monjes movieran un disco por segundo, tardarían en completar el juego 117 el tiempo de exitencia de la Tierra y 41.78 veces el tiempo de vida del universo. Así que aún queda mundo para rato.

Ahora contemos las regiones en las que queda subdividido un plano al trazar n rectas no paralelas y sin que 3 rectas se corten en el mismo punto:

- 0 rectas -> 1 región
- 1 recta -> 2 regiones
- 2 rectas -> 4 regiones
- 3 rectas -> 7 regiones

Nos queda la ecuación de recurrencia:
$a(n)=a(n-1)+n$, con $a(0)=1$

```python
rsolve(a(n)-a(n-1)-n,a(n),{a(0):1})
```

    n*(n + 1)/2 + 1

```python
pprint(_)
```

    n⋅(n + 1)
    ───────── + 1
        2

```python
r = lambda n: n*(n+1)/2+1
```

```python
r(100)
```

    5051.0

Sería difícil contar las regiones tras trazar 100 líneas, pero sabemos que saldrían 5051.

Ahora un ejemplo bastante conocido, que permitirá hallar una fórmula iterativa para calcular un término de la sucesión de fibonacci, este lo haremos paso a paso. $a(n)=a(n-1)+a(n-2), a(0)=0, a(1)=1$

```python
solve(x**2-x-1)    # Las soluciones del polinomio característico son el número phi y el inverso de su opuesto.
```

    [1/2 + sqrt(5)/2, -sqrt(5)/2 + 1/2]

```python
g = lambda n: A*(1/2+sqrt(5)/2)**n + B*(1/2-sqrt(5)/2)**n
```

```python
solve([g(0),g(1)-1],[A,B])   # Se puede hacer a mano por reducción fácilmente y se obtiene A = 1/sqrt(5), B = -1/sqrt(5)
```

    {A: 0.447213595499958, B: -0.447213595499958}

```python
fibo = lambda n: (1/sqrt(5))*(1/2+sqrt(5)/2)**n - (1/sqrt(5))*(1/2-sqrt(5)/2)**n
```

```python
fibo(10)
```

    -sqrt(5)*(-sqrt(5)/2 + 0.5)**10/5 + sqrt(5)*(0.5 + sqrt(5)/2)**10/5

Es curioso como una expresión tan compleja siempre acaba valiendo un número natural. Pero eso no es nada, en el siguiente y último ejemplo, incluso se utilizan números imaginarios.

$a(1)=1, a(2)=2, a(n)=-a(n-2)$

```python
solve(x**2+1,x)     # Empezamos bien...
```

    [-I, I]

```python
g = lambda n: A*I**n + B*(-I)**n
```

```python
solve([g(1)-1,g(2)-2],[A,B])
```

    {A: -1 - I/2, B: -1 + I/2}

```python
p = lambda n: (-I/2-1)*I**n + (I/2-1)*(-I)**n
```

```python
simplify(p(1)), simplify(p(8)), simplify(p(10789423))   # Le ocurre lo mismo que al anterior
```

    (1, -2, -1)

## Combinatoria

#### Principio de inclusión-exclusión

Para contar los elementos de la unión de dos o más conjuntos cuando es más fácil calcular la intersección que la unión. Se usa el principio de inclusión-exclusión. Consiste en sumar los cardinales de los conjuntos y a continuación restar las intersecciones de cada par de conjuntos, posteriormente la de cada trío de conjuntos... y así hasta sumar (si es un número impar de conjuntos) o restar(si es un número par de conjuntos) la intersección de todos.
Por ejemplo:

```python
X={1,2,3,4,5,6}
Y={4,5,6,7}
Z={6,7,8,9,10,11}
```

```python
len(X|Y|Z)     # len() dice el cardinal, y | significa unión.
```

    11

Python cuenta con una función que lo hace, pero, ¿y si no la tuviéramos?

```python
len(X)+len(Y)+len(Z)-len(X&Y)-len(X&Z)-len(Y&Z)+len(X&Y&Z)   # & hace la intersección de conjuntos
```

    11

Ahora un ejemplo práctico, veamos cuántos números binarios de 6 cifras no continen la secuencia 101. Para ello es más fácil empezar calulando los que sí la contienen. La primera cifra ha de ser un 1, luego tendremos que considerar 4 conjuntos, que son las 4 formas diferentes de que aparezca la secuencia:

$
1. A:   1 0 1 _ _ _     (8)
2. B:   1 1 0 1 _ _     (4)
3. C:   1 _ 1 0 1 _     (4)
4. D:   1 _ _ 1 0 1     (4)
$

A la derecha está el cardinal de cada conjunto, que corresponde al número de posibilidades en cada posición (2, 1 ó 0) elevado al número de posiciones. Ahora consideremos las intersecciones de de cada par de conjuntos. Por ejemplo, el cardinal de A&C es 2, ya que sólo queda un hueco libre ($2^1$), el de A&D es 1 ($2^0$) y el de A$B es 0 porque no existen números binarios que tengan el 0 y el 1 a la vez en la misma posición.

$
- A&B:             (0)
- A&C: 1 0 1 0 1 _ (2)
- A&D: 1 0 1 1 0 1 (1)
- B&C:             (0)
- B&D: 1 1 0 1 0 1 (1)
- C&D:             (0)
- A&B&C, A&B&D, A&C&D, B&C&D    (0)
- A&B&CD           (0)
$

El cálculo que necesitamos hacer será |A|+|B|+|C|+|D|-|A&B|-|A&C|-|A&D|-|B&C|-|B&D|-|C&D|+|A&B&C|+|A&B&D|+|A&C&D|+|B&C&D|-|A&B&C&D|

```python
8+4+4+4-0-2-1-0-1-0+0+0+0+0-0
```

    16

Como hay 32 números binarios de 6 cifras (5 huecos libres, $2^5$) y 16 contienen la secuencia 101, 32-16 = 16 números binarios de 6 cifras no la contendrán.

#### Listas

En un conjunto de $n$ elementos, podemos formar $n^k$ listas de k elementos diferentes. Si en cada lista no se pueden repetir elementos, para el número es $n$ elevado a $k$ descendente, es decir, $n\times(n-1)\times...\times(n-k)\times(n-k+1)$. Definamos una función que calcula las variaciones simples (sin repetición) de $k$ entre $n$ elementos.

```python
def V(n,k):
    v = n;
    for i in range(1,k):   # Hace iteraciones con i=1, i=2,...,i=k-1
        v = v*(n-i)        # n(n-1)(n-2)...(n-(k-1))
    return v

```

```python
V(7,3)    # 7*6*5
```

    210

```python
V(7,9)    # Hay 0 formas de escoger 9 elementos diferentes en un conjunto de 7 (el bucle llega a 0 y empieza a multiplicar 0)
```

    0

```python
def perm(n): return V(n,n)    # Serían las posibles permutaciones, n!
```

```python
perm(6)
```

    720

```python
def perm_circ(n): return perm(n-1)     # Permutaciones circulares, (n-1)!
```

```python
perm_circ(6)
```

    120

#### Combinaciones

Las combinaciones son listas en las que no importa el orden, por lo que tendremos que dividir el número de listas entre las formas de ordenarlas, las combinaciones simples equivalen a los números combinatorios.

```python
def combs(n,k):
    if k == 0:
        return 1
    else:
        return V(n,k)/perm(k)  # Es lo mismo que n!/(k!(n-k)!)
```

```python
combs(4,2)     # 4 sobre 2
```

    6.0

```python
def combs_rep(n,k): return factorial(n+k-1)/(factorial(k)*factorial(n-1))    # Ahora las combinaciones con repetición
```

```python
combs_rep(3,3)
```

    10

```python
a,b = symbols("a,b")
```

```python
expand((a+b)**5)
```

    a**5 + 5*a**4*b + 10*a**3*b**2 + 10*a**2*b**3 + 5*a*b**4 + b**5

Podemos definir una función que dada una lista de exponentes calcule el coeficiente multinomial que les corresponde.

```python
def coef_multinomial(X):
    dividendo = 0
    for k in X:
        dividendo += k
    dividendo = factorial(dividendo)
    divisor = 1
    for k in X:
        divisor *= factorial(k)
    return dividendo/divisor
```

```python
coef_multinomial([2,2,1])
```

    30

```python
x,y,z,c,d = symbols("x,y,z,c,d")
```

```python
pprint(coef_multinomial([a,b,c,d]))
```

    (a + b + c + d)!
    ────────────────
      a!⋅b!⋅c!⋅d!

```python
expand((x+y+z)**5)
```

    x**5 + 5*x**4*y + 5*x**4*z + 10*x**3*y**2 + 20*x**3*y*z + 10*x**3*z**2 + 10*x**2*y**3 + 30*x**2*y**2*z + 30*x**2*y*z**2 + 10*x**2*z**3 + 5*x*y**4 + 20*x*y**3*z + 30*x*y**2*z**2 + 20*x*y*z**3 + 5*x*z**4 + y**5 + 5*y**4*z + 10*y**3*z**2 + 10*y**2*z**3 + 5*y*z**4 + z**5

```python
pprint(_)  # Efectivamente podemos comprobar que el coeficiente del término x^2 y^2 z es 30
```

     5      4        4         3  2       3           3  2       2  3       2  2
    x  + 5⋅x ⋅y + 5⋅x ⋅z + 10⋅x ⋅y  + 20⋅x ⋅y⋅z + 10⋅x ⋅z  + 10⋅x ⋅y  + 30⋅x ⋅y ⋅z

           2    2       2  3        4         3           2  2           3
     + 30⋅x ⋅y⋅z  + 10⋅x ⋅z  + 5⋅x⋅y  + 20⋅x⋅y ⋅z + 30⋅x⋅y ⋅z  + 20⋅x⋅y⋅z  + 5⋅x⋅z

    4    5      4         3  2       2  3        4    5
      + y  + 5⋅y ⋅z + 10⋅y ⋅z  + 10⋅y ⋅z  + 5⋅y⋅z  + z

Esto significa que existen 30 formas de repartir 5 objetos diferentes en 3 cajas con 2 objetos en la primera, dos en la segunda y uno en la última. Tendría ${5\choose 2}$ opciones para la primera caja, y a partir de ahí ${3\choose 2}$ para la segunda, obviamente en la tercera va el que quede.

```python
combs(5,2) * combs(3,2)
```

    30.0

#### Particiones

Los números de Stirling de segunda clase representan las formas de dividir un conjunto en varios subconjuntos disjuntos, no vacíos tales que su unión es el total. No es difícil definir una función recursiva para calcularlo.

```python
def Stirling2(n,k):
    if k > n:
        raise TypeError("El segundo argumento nunca puede se mayor que el primero")
    if n == k or k == 1:
        return 1
    else:
        return Stirling2(n-1,k-1) + k*Stirling2(n-1,k)
```

```python
Stirling2(5,3)
```

    25

También podemos definir una función (no muy eficiente) que calcule el n-ésimo número de Bell, que representa el número de particiones de un conjunto con n elementos.

```python
def Bell(n):
    bell = 0
    for k in range(1,n+1):
        bell += Stirling2(n,k)
    return bell

```

Ahora definamos una función que calcule las particiones de un número natural n con k o menos sumandos.

```python
import functools           # Utilizaremos la memoization, porque esta función no va a ser nada eficiente.
```

```python
@functools.lru_cache(maxsize=None)
def part_nat(n,k):
    if k == 1:
        return 1
    if k == n:
        return part_nat(n,n-1) + 1
    if k > n:
        return part_nat(n,n)
    if k < n:
        return part_nat(n,k-1) + part_nat(k,n-k)
```

```python
part_nat(7,7)
```

    15

#### Aplicaciones

```python
n, k = symbols("n, k")
```

Consideremos las aplicaciones de un conjunto A, de cardinal $n$ en un conjunto B, de cardinal $k$.

El número de aplicaciones totales es de $k^n$, ya que cada aplicación es una lista de $n$ elementos elegidos entre $k$.

El número de aplicaciones inyectivas es el mismo que el de listas sin repetición, requiere que $n$ sea menor o igual que $k$.

El número de aplicaciones biyectivas es el mismo que el de las permutaciones de cualquiera de los conjuntos ($n!$), requiere que $k$ sea igual a $n$.

Para las aplicaciones sobreyectivas, la fórmula es más complicada y hace uso del principio de inclusión-exclusión:

$T(n,k)=\sum_{i=0}^{k-1}{{k \choose i}\times (-1)^i\times (k-i)^n}$

Posteriormente definiremos una función para calcularla.

```python
V(5,2)          # Un ejemplo de aplicaciones inyectivas de un conjunto de 2 elementos en otro de 5
```

    20

```python
def T(n,k):
    t = 0
    for i in range(0,k):
        t += combs(k,i)*(-1)**i*(k-i)**n
    return t
```

```python
T(5,3)
```

    150.0

También se puede calcular como $S(n,k)\times k!$

```python
def T_(n,k):
    return perm(k)*Stirling2(n,k)
```

```python
T_(5,3)
```

    150

#### Desbarajustes

Los desbarajustes de $n$ son las formas de ordenar n elementos de forma que ninguno de ellos quede en su sitio, cumplen la ecuación:

$D(1)=0, D(n)=n\times D(n-1) + (-1)^n$

```python
def D(n):
    if n <= 1:
        return 0
    return n*D(n-1)+(-1)**n

```

```python
D(7)
```

    1854

Aunque podemos definirla por una fórmula iterativa:

$D(n)=n!\times \sum_{i=0}^{n}{\frac{(-1)^i}{i!}}$

```python
def D(n):
    suma = 0
    for i in range(0,n+1):
        suma += (-1)**i/factorial(i)
    return factorial(n)*suma
```

```python
D(7)
```

    1854


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, G.F. Forsyth. Partly based on David Ketcheson's pendulum lesson, also under CC-BY.

# Phugoid Oscillation

Welcome back! This is the second IPython Notebook of the series _"The phugoid model of glider flight"_, the first learning module of the course [**"Practical Numerical Methods with Python."**](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about)

In the first notebook, _"Phugoid Motion"_, we described the physics of an aircraft's oscillatory trajectory, seen as an exchange of kinetic and potential energy. This analysis goes back to Frederick Lanchester, who published his book _"Aerodonetics"_ on aircraft stability in 1909. We concluded that first exposure to our problem of interest by plotting the flight paths predicted by Lanchester's analysis, known as _phugoids_.

Here, we will look at the situation when an aircraft is initially moving on the straight-line phugoid (obtained with the parameters $C=2/3$, $\cos\theta=1$, and $z=z_t$ in the previous analysis), and experiences a small upset, a wind gust that slightly perturbs its path. It will then enter into a gentle oscillation around the previous straight-line path: a _phugoid oscillation_.

If the aircraft experiences an upward acceleration of $-d^2z/dt^2$, and we assume that the perturbation is small, then $\cos\theta=1$ is a good approximation and Newton's second law in the vertical direction is:

$$\begin{equation}
L - W = - \frac{W}{g}\frac{d^2 z}{dt^2}
\end{equation}$$

In the previous notebook, we saw that the following relation holds for the ratio of lift to weight, in terms of the trim velocity $v_t$:

$$\begin{equation}
\frac{L}{W}=\frac{v^2}{v_t^2}
\end{equation}$$

This will be useful: we can divide Equation (1) by the weight and use Equation (2) to replace $L/W$. Another useful relation from the previous notebook expressed the conservation of energy (per unit mass) as $v^2 = 2 gz$. With this, Equation (1) is rearranged as:

$$\begin{equation}
\frac{d^2z}{dt^2} + \frac{gz}{z_t} = g
\end{equation}$$

Look at Equation (3) for a moment. Does it ring a bell? Do you recognize it?

If you remember from your physics courses the equation for _simple harmonic motion_, you should see the similarity!

Take the case of a simple spring. Hooke's law is $F=-kx$, where $F$ is a restoring force, $x$ the displacement from a position of equilibrium and $k$ the spring constant. This results in the following ordinary differential equation for the displacement:

$$\begin{equation}
 \frac{d^2 x}{dt^2}= -\frac{k}{m}x
\end{equation}$$

which has the solution $x(t) = A \cos(\omega t- \phi)$, representing simple harmonic motion with an angular frequency $\omega=\sqrt{k/m}=2\pi f$ and phase angle $\phi$.

Now look back at Equation (3): it has nearly the same form and it represents simple harmonic motion with angular frequency $\omega=\sqrt{g/z_t}$ around mean height $z_t$.

Think about this for a moment ... we can immediately say what the period of the oscillation is: exactly $2 \pi \sqrt{z_t/g}$ — or, in terms of the trim velocity, $\pi \sqrt{2} v_t/g$.

_This is a remarkable result!_ Think about it: we know nothing about the aircraft, or the flight altitude, yet we can obtain the period of the phugoid oscillation simply as a function of the trim velocity. For example, if trim velocity is 200 knots, we get a phugoid period of about 47 seconds—over that time, you really would not notice anything if you were flying in that aircraft.

Next, we want to be able to compute the trajectory of the aircraft for a given initial perturbance. We will do this by numerically integrating the equation of motion.

## Prepare to integrate

We want to integrate the differential equation and plot the trajectory of the aircraft. Are you ready?

The equation for the phugoid oscillation is a second-order, ordinary differential equation (ODE). Let's represent the time derivative with a prime, and write it like this:

$$\begin{equation}
z(t)'' + \frac{g \,z(t)}{z_t}=g
\end{equation}$$

There's a convenient trick when we work with ODEs: we can turn this 2nd-order equation into a system of two 1st-order equations. Like this:

$$\begin{eqnarray}
z'(t) &=& b(t)\\
b'(t) &=& g\left(1-\frac{z(t)}{z_t}\right)
\end{eqnarray}$$

Are you following? Make sure you are following the derivations, even if it means writing the equations down in your own notes! (Yes, the old-fashioned paper way.)

Another way to look at a system of two 1st-order ODEs is by using vectors. You can make a vector with your two independent variables,

$$\begin{equation}
\vec{u}  = \begin{pmatrix} z \\ b \end{pmatrix}
\end{equation}$$

and write the differential system as a single vector equation:

\begin{equation}
\vec{u}'(t)  = \begin{pmatrix} b\\ g-g\frac{z(t)}{z_t} \end{pmatrix}
\end{equation}

If you call the right-hand-side $\vec{f}(\vec{u})$, then the equation is very short: $\vec{u}'(t) = \vec{f}(\vec{u})$—but let's drop those arrows to denote vectors from now on, as they are a bit cumbersome: just remember that $u$ and $f$ are vectors in the phugoid equation of motion.

Next, we'll prepare to solve this problem numerically.

## Initial value problems

Let's step back for a moment. Suppose we have a first-order ODE $u'=f(u)$. You know that if we were to integrate this, there would be an arbitrary constant of integration. To find its value, we do need to know one point on the curve $(t, u)$. When the derivative in the ODE is with respect to time, we call that point the _initial value_ and write something like this:

$$u(t=0)=u_0$$

In the case of a second-order ODE, we already saw how to write it as a system of first-order ODEs, and we would need an initial value for each equation: two conditions are needed to determine our constants of integration. The same applies for higher-order ODEs: if it is of order $n$, we can write it as $n$ first-order equations, and we need $n$ known values. If we have that data, we call the problem an _initial value problem_.

Remember the definition of a derivative? The derivative represents the slope of the tangent at a point of the curve $u=u(t)$, and the definition of the derivative $u'$ for a function is:

$$u'(t) = \lim_{\Delta t\rightarrow 0} \frac{u(t+\Delta t)-u(t)}{\Delta t}$$

If the step $\Delta t$ is already very small, we can _approximate_ the derivative by dropping the limit. We can write:

$$\begin{equation}
u(t+\Delta t) \approx u(t) + u'(t) \Delta t
\end{equation}$$

With this equation, and because we know $u'(t)=f(u)$, if we have an initial value, we can step by $\Delta t$ and find the value of $u(t+\Delta t)$, then we can take this value, and find $u(t+2\Delta t)$, and so on: we say that we _step in time_, numerically finding the solution $u(t)$ for a range of values: $t_1, t_2, t_3 \cdots$, each separated by $\Delta t$. The numerical solution of the ODE is simply the table of values $t_i, u_i$ that results from this process.

## Discretization

In order to execute the process described above and find the numerical solution of the ODE, we start by choosing the values $t_1,t_2,t_3 \cdots t_n$—we call these values our *grid* in time. The first point of the grid is given by our _initial value_, and the small difference between two consecutive times is called the _time step_, denoted by $\Delta t$.  The solution value at time $t_n$ is denoted by $u_n$.

Let's build a time grid for our problem. We first choose a final time $T$ and the time step $\Delta t$. In code, we'll use readily identifiable variable names: `T` and `dt`, respectively. With those values set, we can calculate the number of time steps that will be needed to reach the final time; we call that variable `N`.

Let's write some code. The first thing we do in Python is load our favorite libraries: NumPy for array operations, and the Pyplot module in Matplotlib, to later on be able to plot the numerical solution. The line `%matplotlib inline` tells IPython Notebook to show the plots inline.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
```

Now, initialize `T` and `dt`, calculate `N` and build a NumPy array with all the values of time that make up the grid.

```python
T = 100.0
dt = 0.02
N = int(T/dt)+1
t = numpy.linspace(0.0, T, N)
```

We have our grid! Now it's time to apply the numerical time stepping represented by Equation (10).

##### Challenge!

* Write the code above using the NumPy function `arange()` instead of `linspace()`. If you need to, read the documentation for these functions.

##### Pro tip:

Enter a question mark followed by any function, e.g., `?numpy.linspace`, into a code cell and execute it, to get a help pane on the notebook.

## Euler's method

The approximate solution at time $t_n$ is $u_n$, and the numerical solution of the differential equation consists of computing a sequence of approximate solutions by the following formula, based on Equation (10):

$$u_{n+1} = u_n + \Delta t \,f(u_n).$$

This formula is called **Euler's method**.

For the equations of the phugoid oscillation, Euler's method gives the following algorithm that we need to implement in code:

\begin{align}
z_{n+1} & = z_n + \Delta t \, b_n \\
b_{n+1} & = b_n + \Delta t \left(g - \frac{g}{z_t} \, z_n \right).
\end{align}

### And solve!

To apply the numerical solution method, we need to set things up in code: define the parameter values needed in the model, initialize a NumPy array to hold the discrete solution values, and initialize another array for the elevation values.

```python
# initial conditions
z0 = 100.  #altitude
b0  = 10.  #upward velocity resulting from gust
zt = 100.
g  = 9.81

u = numpy.array([z0, b0])

# initialize an array to hold the changing elevation values
z = numpy.zeros(N)
z[0] = z0
```

You should pay attention to a couple of things: (1) See how there is a dot after the numbers used to define our parameters? We just want to be explicit (as a good habit) that these variables are real numbers, called "floats." (2) We both _created_ and _initialized_ with zeros everywhere the solution vector `z`. Look up the documentation for the handy NumPy function `zeros()`, if you need to. (3) In the last line above, we assign the _initial value_ to the first element of the solution vector: `z[0]`.

Now we can step in time using Euler's method. Notice how we are time stepping the two independent variables at once in the time iterations.

```python
# time-loop using Euler's method
for n in range(1,N):
    u = u + dt*numpy.array([u[1], g*(1-u[0]/zt)])
    z[n] = u[0]
```

Make sure you understand what this code is doing. This is a basic pattern in numerical methods: iterations in a time variable that apply a numerical scheme at each step.

## Plot the solution

If the code is correct, we have stored in the array `z` the position of the glider at each time. Let's use Matplotlib to examine the flight path of the aircraft.

You should explore the [Matplotlib tutorial](http://matplotlib.org/users/pyplot_tutorial.html) (if you need to) and familiarize yourself with the command-style functions that control the size, labels, line style, and so on. Creating good plots is a useful skill: it is about communicating your results effectively.

Here, we set the figure size, the limits of the vertical axis, the format of tick-marks, and axis labels. The final line actually produces the plot, with our chosen line style (continuous black line).

```python
pyplot.figure(figsize=(10,4))   #set plot size
pyplot.ylim(40,160)             #y-axis plot limits
pyplot.tick_params(axis='both', labelsize=14) #increase font size for ticks
pyplot.xlabel('t', fontsize=14) #x label
pyplot.ylabel('z', fontsize=14) #y label
pyplot.plot(t,z, 'k-');
```

##### Explore and think

Try changing the value of `v` in the initial conditions.

* What happens when you have a larger gust?
* What about a smaller gust?
* What happens if there isn't a gust (`v = 0`)?

## Exact solution

The equation for phugoid oscillations is a 2nd-order, linear ODE and it has an exact solution of the following form:

$$
\begin{equation}
z(t) = A \sin \left(\sqrt{\frac{g}{z_t}} t \right) + B \cos \left(\sqrt{\frac{g}{z_t}} t \right) + z_t
\end{equation}
$$

where $A$ and $B$ are constants that we solve for using initial conditions.

Our numerical solution used the initial conditions:

$$\begin{eqnarray}
z(0) = z_0 \\
b(0) = b_0
\end{eqnarray}$$

Applying these to the exact solution and solving for $A$ and $B$, we get:

$$\begin{equation}
z(t) = b_0 \sqrt{\frac{z_t}{g}} \sin \left(\sqrt{\frac{g}{z_t}} t \right) + (z_0-z_t) \cos \left(\sqrt{\frac{g}{z_t}} t \right) + z_t
\end{equation}$$

We already defined all of these variables for our numerical solution, so we can immediately compute the exact solution.

##### Pro tip:

The expression is a  bit long —if you don't feel like scrolling left and right, you can insert a line break in the code using a backslash '\' _(without any trailing spaces)_ and Python will treat the next line as a continuation of the first.

```python
z_exact = b0*(zt/g)**.5*numpy.sin((g/zt)**.5*t)+\
            (z0-zt)*numpy.cos((g/zt)**.5*t)+zt
```

## Plot the exact solution

Now we can plot our exact solution!  Even better, we can plot _both_ the numerical solution *and* the exact solution to see how well Euler's method approximated the phugoid oscillations.

To add another curve to a plot, simply type a second `pyplot.plot()` statement.  We also added a legend using the format

```Python
pyplot.legend(["line 1","line 2"])
```

```python
pyplot.figure(figsize=(10,4))
pyplot.ylim(40,160)             #y-axis plot limits
pyplot.tick_params(axis='both', labelsize=14) #increase font size for ticks
pyplot.xlabel('t', fontsize=14) #x label
pyplot.ylabel('z', fontsize=14) #y label
pyplot.plot(t,z)
pyplot.plot(t, z_exact)
pyplot.legend(['Numerical Solution','Analytical Solution']);
```

That looks like pretty good agreement, but what's happening towards the end? We'll come back to this. For now, re-run the previous steps with a different timestep, say $dt=0.01$ and pay attention to the difference.

Euler's method, like all numerical methods, introduces some errors.  If the method is *convergent*, the approximation will get closer and closer to the exact solution as we reduce the size of the step, $\Delta t$. The error in the numerical method should tend to zero, in fact, when $\Delta t\rightarrow 0$—when this happens, we call the method _consistent_. We'll define these terms more carefully in the theory components of this course.

## Convergence

To compare the two solutions, we need to use a **norm** of the difference, like the $L_1$ norm, for example.

$$ E = \Delta t \sum_{n=0}^N \left|z(t_n) - z_n\right|$$

The $L_1$ norm is the sum of the individual differences between the exact and the numerical solutions, at each mesh point. In other words, $E$ is the discrete representation of the integral over the interval $T$ of the (absolute) difference between the computed $z$ and $z_{\rm exact}$:

$$ E = \int \vert z-z_\rm{exact}\vert dt $$

We check for convergence by calculating the numerical solution using progressively smaller values of `dt`. We already have most of the code that we need.  We just need to add an extra loop and an array of different $\Delta t$ values to iterate through.

You should read the documentation for the function [enumerate](https://docs.python.org/2/library/functions.html#enumerate) and make sure you understand how we're using it below.

##### Warning

The cell below can take a little while to finish (the last $\Delta t$ value alone requires 1 million iterations!).  If the cell is still running, the input label will say `In [*]`.  When it finishes, the `*` will be replaced by a number.

```python
# time-increment array
dt_values = numpy.array([0.1, 0.05, 0.01, 0.005, 0.001, 0.0001])

# array that will contain solution of each grid
z_values = numpy.empty_like(dt_values, dtype=numpy.ndarray)

for i, dt in enumerate(dt_values):
    N = int(T/dt)+1    # number of time-steps
    ### discretize the time using numpy.linspace() ###
    t = numpy.linspace(0.0, T, N)

    # initial conditions
    u = numpy.array([z0, b0])
    z = numpy.empty_like(t)
    z[0] = z0

    # time loop - Euler method
    for n in range(1,N):
        ### compute next solution using Euler method ###
        u = u + dt*numpy.array([u[1], g*(1-u[0]/zt)])
        z[n] = u[0]   # store the elevation at time-step n+1

    z_values[i] = z.copy()    # store the total elevation calculation grid i
```

### Calculate the error

We now have numerical solutions for each $\Delta t$ in the array `z_values`.  To calculate the error corresponding to each $\Delta t$, we can write a function!

```python
def get_error(z, dt):
    """Returns the error relative to analytical solution using L-1 norm.

    Parameters
    ----------
    z : array of float
        numerical solution.
    dt : float
        time increment.

    Returns
    -------
    err : float
        L_{1} norm of the error with respect to the exact solution.
    """
    N = len(z)
    t = numpy.linspace(0.0, T, N)

    z_exact = b0*(zt/g)**.5*numpy.sin((g/zt)**.5*t)+\
                (z0-zt)*numpy.cos((g/zt)**.5*t)+zt

    return dt * numpy.sum(numpy.abs(z-z_exact))
```

**Note**: in the last line of the function, we perform an 'array operation':

`z - z_exact`

We are *not* subtracting one value from another.  Instead, we are taking the difference between elements at each corresponding index in both arrays.  Here is a quick example:

```python
a = numpy.array([1, 2, 3])
b = numpy.array([4, 4, 4])

b - a
```

    array([3, 2, 1])

Now, we iterate through each $\Delta t$ value and calculate the corresponding error.

```python
error_values = numpy.empty_like(dt_values)

for i, dt in enumerate(dt_values):
    ### call the function get_error() ###
    error_values[i] = get_error(z_values[i], dt)
```

Remember, *if* the method is convergent then the error should get smaller as  $\Delta t$ gets smaller.  To visualize this, let's plot $\Delta t$ vs. error.  If you use `pyplot.plot` you won't get a very useful result.  Instead, use `pyplot.loglog` to create the same plot with a log-log scale.  This is what we do almost always to assess the errors of a numerical scheme graphically.

```python
pyplot.figure(figsize=(10, 6))
pyplot.tick_params(axis='both', labelsize=14) #increase tick font size
pyplot.grid(True)                         #turn on grid lines
pyplot.xlabel('$\Delta t$', fontsize=16)  #x label
pyplot.ylabel('Error', fontsize=16)       #y label
pyplot.loglog(dt_values, error_values, 'ko-')  #log-log plot
pyplot.axis('equal')                      #make axes scale equally;
```

This is the kind of result we like to see!  As $\Delta t$ shrinks (towards the left), the error gets smaller and smaller, like it should.

##### Challenge!

We calculated the error for several different timestep sizes using two nested `for` loops.  That worked, but whenever possible, we like to re-use code (and not just copy and paste it!).

Create a function that implements Euler's method and re-write the error analysis cell to use your function.

---

###### The cell below loads the style of this notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "PT Mono";
    font-size: 90%;
}

</style>