```python
import numpy as np

##################################################
##### Matplotlib boilerplate for consistency #####
##################################################
from ipywidgets import interact
from ipywidgets import FloatSlider
from matplotlib import pyplot as plt

%matplotlib inline

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('svg')

global_fig_width = 8
global_fig_height = global_fig_width / 1.61803399
font_size = 12

plt.rcParams['axes.axisbelow'] = True
plt.rcParams['axes.edgecolor'] = '0.8'
plt.rcParams['axes.grid'] = True
plt.rcParams['axes.labelpad'] = 8
plt.rcParams['axes.linewidth'] = 2
plt.rcParams['axes.titlepad'] = 16.0
plt.rcParams['axes.titlesize'] = font_size * 1.4
plt.rcParams['figure.figsize'] = (global_fig_width, global_fig_height)
plt.rcParams['font.sans-serif'] = ['Computer Modern Sans Serif', 'DejaVu Sans', 'sans-serif']
plt.rcParams['font.size'] = font_size
plt.rcParams['grid.color'] = '0.8'
plt.rcParams['grid.linestyle'] = 'dashed'
plt.rcParams['grid.linewidth'] = 2
plt.rcParams['lines.dash_capstyle'] = 'round'
plt.rcParams['lines.dashed_pattern'] = [1, 4]
plt.rcParams['xtick.labelsize'] = font_size
plt.rcParams['xtick.major.pad'] = 4
plt.rcParams['xtick.major.size'] = 0
plt.rcParams['ytick.labelsize'] = font_size
plt.rcParams['ytick.major.pad'] = 4
plt.rcParams['ytick.major.size'] = 0
##################################################
```

# Course Structure

- Lecture 1: Introduction to Bayesian Inference and Pints
- **Lecture 2: Maximum Likelihood Estimation**
   - **The Likelihood function**
   - **Maximum Likelihood Estimation**
   - **Maximum a Posteriori Esimation**
- Lecture 3: MCMC sampling
- Lecture 4: Hierarchical models

# Likelihood

- Recall the likelihood appears in Bayes' Theorem

$$P(\theta | data) = \frac{\color{red}{{P(data|\theta)}} P(\theta)}{P(data)}$$

- Remember not a probability distribution because $\theta$ varies
- Most important choice, derived from the statistical model of the underlying process
- Encapsulates many subjective judgements about analysis.

# Equivalence relation

- A notation often seen in the literature is

$$\mathcal{L}(\theta | data) = P(data | \theta)$$

Therefore, a likelihood of $\theta$ for a particular data sample is equivalent to the probability of that data sample for that value of $\theta$. We call the above an *equivalence relation*

# Example:  frequency of lift malfunctioning

- Imagine we want to create a model for the frequency a lift (elevator) breaks down in a given year, $X$.
- Assume a range of unpredictable and uncorrelated factors (temperature, lift usage, etc.) affect the functioning of the lift.
- therefore, $X ∼ \text{Poisson}(\theta)$, where $\theta$ is the mean number of times the lift breaks in one year.
- we don’t a priori know the true value of $\theta$, our model defines collection of probability models; one for each value of $\theta$.
- We call this collection of models the Likelihood.

```python
import math
from scipy.stats import poisson
def lift_likelihood(theta, data):
    x = np.arange(0, 20)
    plt.plot(x, poisson.pmf(x, theta), 'bo', ms=8, label='poisson pmf')
    plt.vlines(x, 0, poisson.pmf(x, theta), colors='b', lw=5, alpha=0.5)
    print(data)
    if data is not None:
        plt.vlines([data], 0, poisson.pmf(data, theta), colors='r', lw=5, alpha=0.9)
    plt.ylim(0,0.25)
    plt.ylabel(r'$P(X | \theta=%s)$'%str(theta))
    plt.xlabel('X')
    plt.show()
def lift_likelihood_no_data(theta):
    lift_likelihood(theta,None)
def lift_likelihood_five_data(theta):
    lift_likelihood(theta,5)

def lift_likelihood_w_theta(k):
    theta = np.linspace(0, 20, 100)
    plt.plot(theta, np.exp(-theta)*theta**k/(math.factorial(k)))
    plt.ylabel(r'$P(k=%s | \theta)$'%str(k))
    plt.xlabel(r'mean number of breakdowns $\theta$')
    plt.show()
```

```python
# Display our collection of models
widget = FloatSlider(value=5.0, min=0.0, max=15.0, step=1.0, continuous_update=False)
interact(lift_likelihood_no_data, theta=widget, continuous_update=False);
```

    interactive(children=(FloatSlider(value=5.0, continuous_update=False, description='theta', max=15.0, step=1.0)…

To calculate the likelihood:
- fix the data (say number of breakdowns is measured at 5)
- find the corresponding probability for each of the models

```python
interact(lift_likelihood_five_data, theta=widget, continuous_update=False);
```

    interactive(children=(FloatSlider(value=5.0, continuous_update=False, description='theta', max=15.0, step=1.0)…

Using the equation for a Poisson distribution, and $k=5$ as the number of breakdowns, the likelihood function is:

$$\mathcal{L}(\theta | k) = P(k | \theta) = \exp(-\theta)\frac{\theta^k}{k!}$$

```python
lift_likelihood_w_theta(5)
```

# Example: ODE-based model

- Models can often take the form of differential equations evolving in time
- Likelihood is often straightforward to derive and relativly cheap to calculate due to *independent* (in time) measurement noise
- Lets consider the case of an ordinary differential equation (ODE) model, the reversible reaction model in the previous lecture:

$$\dot{y}(t) = k_1 (1 - y) - k_2 y,$$

where $k_1$ represents a forward reaction rate, $k_2$ is the backward reaction rate, and $y$ represents the concentration of a chemical solute.

- In an experiment, we take $N$ measurements of the system $z_i$, with $i = 0...N-1$, that are modelled with independent Gaussian measurement noise:

$$z_i \sim y(t_i) + N(0, \sigma)$$

- Assuming that $\sigma$ is unknown, we now have **three** model parameters: $\boldsymbol{\theta} = [k_1, k_2, \sigma]$, rather than one in the previous lift example

Lets look at a random realisations of this experiment, with $N=50$, $k_1=5$, $k_2=3$, and $\sigma=0.02$

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import odeint

def r(y, t, p):
    k1 = p[0]
    k2 = p[1]
    dydt = k1 * (1 - y) - k2 * y
    return dydt

def random(p):
    y0 = 0.1
    times = np.linspace(0, 1, 50)
    values = odeint(r, y0, times, (p,))
    values += np.random.normal(0, 0.02, values.shape)
    plt.ylabel('Concentration')
    plt.scatter(times, values)
    plt.show()
```

```python
random([5,3])
```

- As before, this model describes an infinite family of probability distributions governed by the three parameters $\boldsymbol{\theta} = [k_1, k_2, \sigma]$
- However, now we have $N$ outputs due to the $N$ time samples that were measured, therefore a probabilty distribution around each time point $t_i$

```python

from scipy.stats import norm
def family_of_models(k1, k2, sigma):
    y0 = 0.1
    times = np.linspace(0, 1, 50)
    index = int(len(times)/2)
    values = odeint(r, y0, times, ([k1, k2],)).reshape(-1)
    global_fig_width = 8
    f, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 8/1.618))
    ax1.set_ylabel('Concentration')
    ax1.set_xlabel(r'$t$')
    ax1.plot(times, values)
    ax1.fill_between(times, values-sigma, values+sigma, alpha=0.5)
    ax1.axvline(times[index],color='r')

    concentrations = np.linspace(0, 1, 100)
    ax2.plot(concentrations, norm.pdf(concentrations, values[index], sigma), color='r')
    ax2.set_xlabel('concentration')
    ax2.set_ylabel(r'$P(y(t_i) | \theta)$')
    plt.show()

k1_widget = FloatSlider(value=5.0, min=0.0, max=15.0, step=1.0, continuous_update=False)
k2_widget = FloatSlider(value=3.0, min=0.0, max=15.0, step=1.0, continuous_update=False)
sigma_widget = FloatSlider(value=0.1, min=0.0, max=0.2, step=0.01, continuous_update=False)
```

```python
# Display our collection of models
interact(family_of_models, k1=k1_widget, k2=k2_widget, sigma=sigma_widget, continuous_update=False);
```

    interactive(children=(FloatSlider(value=5.0, continuous_update=False, description='k1', max=15.0, step=1.0), F…

# Likelihood of ODE model

- we assume that the errors at each time point are independent, therefore the conditional probability density of observing the whole experimental trace from time sample 1 to time sample N is simply the product of the probability density functions at each time point

$$\mathcal{L}(\theta \vert \mathbf{y}) = \prod_{i=1}^{N} P(z_i | \boldsymbol{\theta}).$$

- With our further assumption that the experimental noise is also normally distributed with a mean of zero and variance of $\sigma^2$ , the likelihood can be expressed as

\begin{equation}
\mathcal{L}(\boldsymbol{\theta} \vert \mathbf{z}) =  \prod_{i=1}^{N} \mathcal{N}(z_i \vert y(t, \boldsymbol{\theta}),\sigma^2) = \prod_{i=0}^{N} \frac{1}{\sqrt{2 \pi \sigma^2}} \exp \left(-\frac{\left(z_i -y(t_i, \boldsymbol{\theta})\right)^2}{2\sigma^2}\right),
\end{equation}

- Note that we denote the solution of the ODE as $y(t_i, \boldsymbol{\theta})$, to emphasis that the solution is a function of the parameters $\boldsymbol(\theta)$

- Normally work with the log-likelihood:

\begin{align}
l(\boldsymbol{\theta} \vert \mathbf{z}) &= \log(\mathcal{L}(\boldsymbol{\theta} \vert \mathbf{z})) \nonumber \\
&= -\frac{N}{2}\log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^N (z_i - y(t_i, \boldsymbol{\theta}))^2 \nonumber \\
&= -\frac{N}{2}\log(2\pi) - N\log(\sigma) - \frac{1}{2\sigma^2}\sum_{i=1}^N (z_i - y (t_i, \boldsymbol{\theta}))^2
\end{align}

- Notice the similarities with classical sum of squares error function (for fixed $\sigma$ anyway)!

```python
def calculate_log_likelihood(k1, k2, sigma, data, times, y0):
    N = len(times)
    values = odeint(r, y0, times, ([k1, k2],)).reshape(-1)
    return -N/2.0*np.log(2*np.pi) - N*np.log(sigma) - (1.0/(2.0*sigma**2)) * np.sum((data - values)**2)
```

```python
def calculate_log_likelihood_all(K1, K2, sigma):
    N = 50
    y0 = 0.1
    times = np.linspace(0, 1, N)
    result = np.empty_like(K1)
    data = odeint(r, y0, times, ([5, 3],)).reshape(-1) + np.random.normal(0, sigma, times.shape)
    for i,(k1,k2) in enumerate(zip(K1,K2)):
        result[i] = calculate_log_likelihood(k1, k2, sigma, data, times, y0)
    return result

def show_ode_log_likelihood(sigma):
    k1 = np.linspace(0, 15, 50)
    k2 = np.linspace(0, 15, 50)
    K1, K2 = np.meshgrid(k1, k2)
    log_likelihood = calculate_log_likelihood_all(K1.flat, K2.flat, sigma).reshape(K1.shape)

    plt.contourf(K1, K2, log_likelihood, 20, cmap='RdGy', levels=30);
    plt.xlabel(r'$k_1$')
    plt.ylabel(r'$k_2$')
    plt.title('log-likelihood for ODE model')
    plt.colorbar()
    plt.show()
```

```python
interact(show_ode_log_likelihood, sigma=FloatSlider(value=0.01, min=0.001, max=0.1, step=0.001, continuous_update=False));
```

    interactive(children=(FloatSlider(value=0.01, continuous_update=False, description='sigma', max=0.1, min=0.001…

# Maximum Likelihood Estimation

- In the previous lecture we worked out the posterior *distribution* a given parameter, given the available data
- Often only interested in the most *likely* parameter value
- Can use maximum likelihood estimation, simply find the value of $\theta$ that maximises the likelihood
- a **frequentist** approach, uses a likelihood function (not a valid probability distribution)

$$\theta_{mle} = \text{arg max}_{\theta \in \Omega} P(data|\theta)$$

# Coin example

- Back to the coin example. We perform an experiment of $n$ flips, and it lands heads up $h$ times

- Likelihood is:

$$\mathcal{L}(\theta \vert h\times H) = \theta^h (1-\theta)^{n-h}$$

# Finding the parameters that maximise the Likelihood

- For tractable likelihood functions, we can simply find the derivative of the likelihood and set to zero:

$$\frac{\partial \mathcal{L}(\theta \vert h\times H)}{\partial \theta} = \theta^{h - 1} (h - n \theta) (1-\theta)^{n -h - 1}$$

- Set $\frac{\partial \mathcal{L}(\theta \vert h\times H)}{\partial \theta} = 0$ to find the MLE estimator for $\theta$

$$\theta_{mle} = \frac{h}{n}$$

```python
def show_mle_coin(h):
    n = 10
    theta = np.linspace(0,1,100)
    L = theta**h * (1-theta)**(n-h)
    plt.plot(theta,L, label='loss function')
    plt.xlabel(r'Probability of landing heads $\theta$')
    plt.ylabel(r'MLE loss function $L(\theta)$')
    max_L = h / n
    plt.vlines(max_L, 0, np.max(L), color='r', linestyle='dashed', linewidth=2)
    plt.scatter([max_L],[np.max(L)], color='r', label='MLE estimate', s=100)
    plt.legend()
    plt.show()

h_widget = FloatSlider(value=8.0, min=0.0, max=10.0, step=1.0, continuous_update=False)
```

```python
interact(show_mle_coin, h=h_widget);
```

    interactive(children=(FloatSlider(value=8.0, continuous_update=False, description='h', max=10.0, step=1.0), Ou…

## Fisher information and the MLE estimator

- A measure of information (for $\theta$) contained in the data $X$ is given by the Fisher information $I(\theta)$

$$I(\theta) = -E\left[\frac{\partial^2 l(\theta | X)}{\partial \theta^2}\right]$$

- For large number of samples $N$, the MLE estimator converges to the true $\theta_0$ by

$$\theta_{mle} \sim \mathcal{N} \left(\theta_0, \frac{1}{NI(\theta)} \right)$$

- Corresponding approximate 95% confidence intervals for $\theta_0$ are

$$\theta_{mle} - 1.96 \sqrt{\frac{1}{nI(\theta)}} \le \theta_0 \le \theta_{mle} + 1.96 \sqrt{\frac{1}{nI(\theta)}}$$

# Non-linear optimisation

- Often the analytical approach is not feasible, can then turn to one of many non-linear optimisation algorithms
- Derivative-free Direct methods (aka Search methods)
  - Brute-force exploration (look at the landscape and pick the lowest point)
  - Random search (e.g. Simulated annealing)
  - Coordinate search (aka Coordinate descent, aka Compass search)
  - Search & poll / Pattern search
  - Simplex methods (e.g. Nelder-Mead)
  - Tabu search
  - Dividing rectangles algorithm (DIRECT)
  - Powell's conjugate direction method
- Multi-start methods
  - Multi-level single-linkage (MLSL)
  - Basin-hopping
- Derivative-free evolutionary methods and metaheuristics
  - Genetic algorithms (GA)
  - Differential evolution
  - Evolution strategies
  - Controlled random search (CRS)
  - Swarm algorithms
  - Metaheuristics
- Bayesian optimistation
- Gradient-estimating methods
  - Finite difference methods
  - Simplex gradient methods / Implicit filtering
  - Natural evolution strategies (NES)
- Surrogate-model methods
  - Trust-region methods
  - Data-based Online Nonlinear Extremumseeker (DONE)
- Methods requiring the 1st-order gradient
  - Root finding methods (e.g. Newton's method, BFGS)
  - Gradient descent (aka Steepest descent)
  - Stochastic gradient descent
  - Continuation

# Some useful non-linear optimisation packages:

- [NLopt](https://nlopt.readthedocs.io)
- [Pagmo/Pygmo](http://esa.github.io/pygmo/)
- [Scipy](https://docs.scipy.org/doc/scipy/reference/optimize.html)
- [Pints](https://pints.readthedocs.io/en/latest/)

# Example using PINTS

- We will use a popular model of population growth, the logistic equation:

    $$ \frac{dy(t)}{dt} = r y(t) \frac{k - y(t)}{k}$$

- Two parameters, the carrying capacity $k$, and the rate of growth $r$
- We will assume an unknown measurement noise $\sigma$, which gives the parameter set $\boldsymbol{\theta} = [r, k, \sigma]$
- The `pints.GaussianLogLikelihood` in PINTS implements the independent Gaussian noise log-likelihood derived earlier

\begin{align}
l(\boldsymbol{\theta} \vert \mathbf{z}) &= \log(L(\boldsymbol{\theta} \vert \mathbf{z})) \nonumber \\
&= -\frac{N}{2}\log(2\pi) - N\log(\sigma) - \frac{1}{2\sigma^2}\sum_{i=1}^N (z_i - y (t_i, \boldsymbol{\theta}))^2
\end{align}

```python
import pints
import pints.toy
import matplotlib.pyplot as plt
import numpy as np

p0 = 1  # initial population; initial value
model = pints.toy.LogisticModel(p0)

# Define the 'true' parameters
true_parameters = [0.1, 50, 5]

# Run a simulation to get test data
times = np.linspace(0, 100, 100)
values = model.simulate(true_parameters[:-1], times)

# Add some noise
values += np.random.normal(0, true_parameters[-1], values.shape)

```

```python
# Show the test data
plt.figure()
plt.xlabel('Time')
plt.ylabel(r'Population $y(t)$')
plt.scatter(times, values, label='data')
plt.plot(times, model.simulate(true_parameters[:-1], times), color='r', lw=3, label='true parameters')
plt.legend()
plt.show()
```

```python
# Create an object with links to the model and time series
problem = pints.SingleOutputProblem(model, times, values)

# Create the log-likelihood function
log_likelihood = pints.GaussianLogLikelihood(problem)

# Select some boundaries
boundaries = pints.RectangularBoundaries([0, 0, 0], [100, 100, 100])

# Select a starting point
x0 = [50, 50, 50]

# Perform an optimization using XNES.
found_parameters, found_value = pints.optimise(log_likelihood, x0, boundaries=boundaries, method=pints.XNES)
print('log_likelihood at true solution:')
print(log_likelihood(true_parameters))
```

    Maximising LogPDF
    using Exponential Natural Evolution Strategy (xNES)
    Running in sequential mode.
    Population size: 7
    Iter. Eval. Best      Time m:s
    0     7     -481.7851   0:00.0
    1     14    -449.7718   0:00.0
    2     21    -443.2462   0:00.0
    3     28    -439.0006   0:00.0
    20    147   -438.9613   0:00.0
    40    287   -438.9469   0:00.1
    60    427   -438.9469   0:00.1
    80    567   -438.9469   0:00.1
    100   707   -438.9469   0:00.1
    120   847   -438.9469   0:00.1
    140   987   -438.9469   0:00.1
    160   1127  -438.9469   0:00.1
    180   1267  -438.9469   0:00.2
    200   1407  -438.9469   0:00.2
    220   1547  -438.9469   0:00.2
    240   1687  -438.9469   0:00.2
    260   1827  -438.9469   0:00.2
    280   1967  -438.9469   0:00.2
    284   1988  -438.9469   0:00.2
    Halting: No significant change for 200 iterations.
    log_likelihood at true solution:
    -288.64023040857603

```python
# Show the results
plt.figure()
plt.xlabel('Time')
plt.ylabel(r'Population $y(t)$')
found_mean = model.simulate(found_parameters[:-1], times)
plt.fill_between(times, found_mean - found_parameters[-1], found_mean + found_parameters[-1],
                 color='gray', alpha=0.3)
plt.plot(times, found_mean, color='r', label='found parameters')
plt.scatter(times, values, alpha=0.5, label='data')
plt.legend()
plt.show()
```

# Maximum a posteriori (MAP) estimation

- Rather than a likelihood, can also maximise the unnormalised posterior:

$$P(\theta | data) \sim P(data|\theta) P(\theta)$$

$$\theta_{map} = \text{arg max}_{\theta \in \Omega}  P(data|\theta) P(\theta)$$

- MLE is a particular case of MAP, using a Uniform prior (just multiplies the likelihood by a constant)
- This method is a useful method of incorporporating domain knowledge on the parameters
- Related to regularisation in non-linear and linear optimisation

# Coin example with a Gaussian prior

- Reasonable assumption is that the coin is likely to be fair, lets use a Gaussian prior $N(0.5, \sigma)$

- Maximum a posteriori loss function $L(\theta)$ is therefore:

\begin{align*}
L(\theta) &= P(data|\theta) P(\theta) \\
&= \theta^h (1-\theta)^{n-h} \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{(\theta - 0.5)^2}{2 \sigma^2}}
\end{align*}

```python
def show_map_coin(h, sigma):
    n = 10
    theta = np.linspace(0,1,100)
    L = theta**h * (1-theta)**(n-h) * np.exp(-(theta - 0.5)**2/(2*sigma**2)) / np.sqrt(2*np.pi*sigma**2)
    plt.plot(theta,L, label='loss function')
    plt.xlabel(r'Probability of landing heads $\theta$')
    plt.ylabel(r'MAP loss function $L(\theta)$')
    max_i = np.argmax(L)
    plt.scatter([theta[max_i]],[L[max_i]], color='r', label='MAP estimate', s=100)
    plt.legend()
    plt.show()

h_widget = FloatSlider(value=8.0, min=0.0, max=10.0, step=1.0, continuous_update=False)
sigma_widget = FloatSlider(value=0.3, min=0.0, max=0.3, step=0.01, continuous_update=False)
```

```python
interact(show_map_coin, h=h_widget, sigma=sigma_widget, continuous_update=False);
```

    interactive(children=(FloatSlider(value=8.0, continuous_update=False, description='h', max=10.0, step=1.0), Fl…

# Logistic growth example

Recall the logistic equation:

$$f(t) = \frac{k}{1+(k/p_0 - 1) \exp(-r t)}$$

- Anyone familiar with this equation could estimate a value of the carrying capacity $k$ from a plot
- Would be reasonable to therefore use a Gaussian Prior for $k$

```python
def show_logistic_estimate():
    plt.xlabel('Time')
    plt.ylabel(r'Population $y(t)$')
    plt.scatter(times, values, label='data')
    plt.plot([0, 100], [50, 50], c='k', ls='--', lw=3, label='estimate for $k$')
    plt.legend()
    plt.show()
```

```python
show_logistic_estimate()
```

## Finding MAP estimator in PINTS

```python
# Create the log-likelihood function (using the problem defined earlier)
log_likelihood = pints.GaussianLogLikelihood(problem)

# Create a uniform prior over r
log_prior_r = pints.UniformLogPrior([0],[100])

# Create a gaussian prior over k
log_prior_k = pints.GaussianLogPrior(50,10)

# Create a uniform prior over sigma
log_prior_sigma = pints.UniformLogPrior([0],[100])

# Create a composed prior
log_prior = pints.ComposedLogPrior(log_prior_r, log_prior_k, log_prior_sigma)
```

```python
# Create a posterior log-likelihood (log(likelihood * prior))
log_posterior = pints.LogPosterior(log_likelihood, log_prior)

# Select some boundaries
boundaries = pints.RectangularBoundaries([0, 0, 0], [100, 100, 100])

# Select a starting point
x0 = [50, 50, 50]

# Perform an optimization using Particle Swarm Optimisation (PSO).
found_parameters, found_value = pints.optimise(log_likelihood, x0, boundaries=boundaries, method=pints.PSO)
print('posterior log-likelihood at true solution:')
print(log_posterior(true_parameters))
```

    Maximising LogPDF
    using Particle Swarm Optimisation (PSO)
    Running in sequential mode.
    Population size: 7
    Iter. Eval. Best      Time m:s
    0     13    -489.3967   0:00.0
    1     26    -489.3967   0:00.0
    2     39    -475.6101   0:00.0
    3     52    -463.3096   0:00.0
    20    273   -439.1372   0:00.0
    40    533   -438.9546   0:00.0
    60    793   -438.9546   0:00.1
    80    1053  -400.392    0:00.1
    100   1313  -399.0795   0:00.1
    120   1573  -397.7643   0:00.1
    140   1833  -396.5449   0:00.1
    160   2093  -383.5393   0:00.1
    180   2353  -330.0047   0:00.1
    200   2613  -311.9692   0:00.1
    220   2873  -286.0045   0:00.2
    240   3133  -286.0045   0:00.2
    260   3393  -286.0045   0:00.2
    280   3653  -286.0045   0:00.2
    300   3913  -286.0045   0:00.2
    320   4173  -286.0045   0:00.2
    340   4433  -286.0045   0:00.2
    360   4693  -286.0045   0:00.3
    380   4953  -286.0045   0:00.3
    400   5213  -286.0045   0:00.3
    408   5304  -286.0045   0:00.3
    Halting: No significant change for 200 iterations.
    posterior log-likelihood at true solution:
    -301.0720944067509

```python
# Show the results
plt.figure()
plt.xlabel('Time')
plt.ylabel(r'Population $y(t)$')
found_mean = model.simulate(found_parameters[:-1], times)
plt.fill_between(times, found_mean - found_parameters[-1], found_mean + found_parameters[-1],
                 color='gray', alpha=0.2)
plt.plot(times, found_mean)
plt.plot(times, values)
plt.show()
```

# Electrochemistry example - POMS

- three unresolved two-electron surface-confined polyoxometalate reduction processes by AC voltammetry

**(left)** Molecular structure of $[\text{PMo}_{12}\text{O}_{40}]^{3-}$    **(right)** Experimental AC voltammetry trace

- The sequence of six electron transfer steps are modelled by the following quasi-reversible reactions

\begin{align}
    A + e^- \underset{k^1_{ox}(t)}{\overset{k^1_{red}(t)}{\rightleftarrows}} B,
    \\
    B + e^- \underset{k^2_{ox}(t)}{\overset{k^2_{red}(t)}{\rightleftarrows}} C,
    \\
    C + e^- \underset{k^3_{ox}(t)}{\overset{k^3_{red}(t)}{\rightleftarrows}} D,
    \\
    D + e^- \underset{k^4_{ox}(t)}{\overset{k^4_{red}(t)}{\rightleftarrows}} E,
    \\
    E + e^- \underset{k^5_{ox}(t)}{\overset{k^5_{red}(t)}{\rightleftarrows}} F,
    \\
    F + e^-  \underset{k^6_{ox}(t)}{\overset{k^6_{red}(t)}{\rightleftarrows}} G,
\end{align}

where the forward $k_{red}$ and backwards $k_{ox}$ reaction rates are
given by
the Butler-Volmer
relationships

\begin{align}\label{eq:rate1}
    k^i_{red}(t) &= k^0_i \exp\left(-\frac{\alpha_i F}{RT} [E_r(t) - E^0_i]
    \right), \\
    k^i_{ox}(t) &= k^0_i \exp\left((1-\alpha_i)\frac{F}{RT} [E_r(t) - E^0_i]
\right).  \label{eq:rate2}
\end{align}

- This can be modelled by an ordinary differential equation containing 17 parameters to be estimated

$$
\mathbf{p} =
(E^0_1,E^0_2,E^0_3,E^0_4,E^0_5,E^0_6,k^0_1,k^0_2,k^0_3,k^0_4,k^0_5,k^0_6,
         \alpha_1,
         \alpha_2,
         R_u,
         C_{dl},
         \Gamma).
$$

- The effect of the $E^0_i$ parameters on the simulated current is highly non-linear.
- In such a high dimensional space all non-linear optimisers we tried failed to find the global minimum
- But approximate values of $E^0_i$ can be easily read off the experimental current trace.... **solution:** put a Gaussian prior on all $E^0_i$ parameters

**standard deviation** of the Gaussian prior (i.e. confidence of the estimation of $E^0_i$), required to be $<= 0.1$ V for **reliable parameter estimation**

## Summary

- The likelihood function is derived from your model of the underlying process, and the experimental data
- It is often the most important choice, encapsulates many **subjective judgements**
- **Maximum likelihood estimation** (MLE) finds the *most likely* parameters that lead to the data
- **Maximum a posteriori estimation** (MAP) finds the most likely parameters that lead to the data, *given additional information* encoded in the prior
- **Non-linear Optimisation** methods are useful in both MLE and MAP estimation for non-trivial models.

```python

```


# Plotting with Matplotlib

## Preparations

```
import numpy as np
import scipy as sp
import sympy

# the convienient Matplotib plotting interface pyplot (the tidy/right way)
# use this for building scripts
import matplotlib.pyplot as plt

# for using the matplotlib API directly (the hard and verbose way)
# use this when building applications, and/or backends
import matplotlib as mpl
```

## Scaling: getting font sizes right for publications

```
# some sample data
x = np.arange(0,10,0.1)
```

```
page_width_cm = 13
dpi = 200
inch = 2.54 # inch in cm

plt.rc('font', family='serif')
plt.rc('xtick', labelsize=12) # tick labels
plt.rc('ytick', labelsize=20) # tick labels
plt.rc('axes', labelsize=20)  # axes labels
# If you don’t need LaTeX, don’t use it. It is slower to plot, and text
# looks just fine without. If you need it, e.g. for symbols, then use it.
plt.rc('text', usetex=True)
```

```
# chose the backend: 'gtk', 'inline', 'osx', 'qt', 'qt4', 'tk', 'wx'
%matplotlib qt
# create a figure instance, note that figure size is given in inches!
fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(8,6))
# set the big title (note aligment relative to figure)
fig.suptitle("suptitle 16, figure alignment", fontsize=16)
# add axes to plot manually
# left, bottom, width, height (range 0 to 1)
ax = fig.add_axes([0.1, 0.1, 0.5, 0.8])
# actual plotting
ax.plot(x, x**2, label="label 12")
# set axes title (note aligment relative to axes)
ax.set_title("title 14, axes alignment", fontsize=14)
# axes labels
ax.set_xlabel('xlabel 12')
ax.set_ylabel(r'$y_{\alpha}$ 12', fontsize=8)
# legend
ax.legend(fontsize=12, loc="best")
fig.savefig('figure-%03i.png' % dpi, dpi=dpi)
fig.savefig('figure.svg')
fig.savefig('figure.eps')

# figure scope
y_ = 0.7
ax.annotate('',  xy=(0.0, y_), xycoords='figure fraction',
             xytext=(0.1, y_), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))
ax.annotate('',  xy=(0.1, y_), xycoords='figure fraction',
             xytext=(0.6, y_), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))
ax.annotate('',  xy=(0.6, y_), xycoords='figure fraction',
             xytext=(1.0, y_), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))

x_ = 0.15
ax.annotate('',  xy=(x_, 0), xycoords='figure fraction',
             xytext=(x_, 0.1), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))
ax.annotate('',  xy=(x_, 0.1), xycoords='figure fraction',
             xytext=(x_, 0.9), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))
ax.annotate('',  xy=(x_, 0.9), xycoords='figure fraction',
             xytext=(x_, 1.0), textcoords='figure fraction',
             arrowprops=dict(arrowstyle="<->"))
```

    <matplotlib.text.Annotation at 0x6a57e90>

```
stext = u'$\\sigma = \\frac{33 \\cdot 10^3 \\Omega}{5 \\epsilon^5}$'
# bbox is a dictionary of matplotlib.patches.Rectangle properties:
# http://matplotlib.org/api/artist_api.html#matplotlib.patches.Rectangle
bbox = dict(boxstyle="round", alpha=0.5, edgecolor=(1., 0.5, 0.5),
            facecolor=(1., 0.8, 0.8),)
ax.text(6, 40, stext, fontsize=24, verticalalignment='bottom',
         horizontalalignment='center', bbox=bbox)
# we have to update the figure
ax.plot(x, x**2, 'ks', label="label 12", alpha=0.3)
fig.canvas.draw()
```

```
# what is the current backend?
%matplotlib
fig = plt.figure(1, figsize=(8,6))
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
ax.plot(x, x**2, label="label 12")
```

    Using matplotlib backend: Qt4Agg

    [<matplotlib.lines.Line2D at 0x5588450>]

```
# switch to inline plotting
%matplotlib inline
fig = plt.figure(1, figsize=(8,6))
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
ax.plot(x, x**2, label="label 12")
```

```
%matplotlib qt
fig = plt.figure(1, figsize=(8,6))
ax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
ax.plot(x, x**2, label="label 12")
```

    [<matplotlib.lines.Line2D at 0x5589710>]

```
%matplotlib?
```

```
for i in range(5):
    plt.figure(i)
    plt.plot(x, i*x**2)
```

```
def serialplots():
    for i in range(5):
        plt.figure(i)
        plt.plot(x, i*x**2)
        plt.show()

def serialplots2():
    for i in range(5):
        plt.figure(i)
        plt.plot(x, i*x**2)
    plt.show()

def serialplots3():
    for i in range(5):
        figure(i)
        plot(x, i*x**2)
        show()

def serialplots4():
    for i in range(5):
        figure(i)
        plot(x, i*x**2)
    show()
```

```
%matplotlib qt
serialplots4()
```


# Deep Reinforcement Learning

Reinforcement learning is one of the hottest (if not the hottest) area of deep learning. It is possibly the closest that we have come to achieving general AI at this point in time. The reason it is touted as getting close to general AI is that we can use the same framework repeatedly in different environments to maximise future rewards. Apart from the ATARI games in which [Deep Reinforcement Learning got famous](https://web.stanford.edu/class/psych209/Readings/MnihEtAlHassibis15NatureControlDeepRL.pdf) for and beating the current world GO chamption, it is now being used in finance and [teaching robots to walk](https://www.youtube.com/watch?v=JFJkpVWTQVM).

The general concept of RL is to simulate game a large number of times and to learn which actions constitutes a good move. One of the biggest challenges that RL faces is to infer what constitutes a good move. A good move at the current state might be a bad move for future states. This is definitely the case in playing chess for example where you constantly sacrifice pieces for larger future rewards.

## Belman Equation

The goal of reinforcement learning is to maximise its return. In this tutorial we get 'envirnonments' from OpenAI which provides us a game, the physics/ rules by which the game is controlled by and outputs the **rewards** achieved for a given **action**, at the current **state** of the game.

The bellman equation plays a crucial role in reinforcement learning.
\begin{align}
Q(S_t)(a_t) = r_t + \gamma\max_{a_{t+1}}Q(S_{t+1})(a_{t+1})
\end{align}
where $Q$ is a quality function, which depends on state $S_t$ and outputs the maximum possible future reward for action $a_t$. $\gamma$ is a discount factor. Most texts would prefer to depict the Quality function as $Q(S_t, a_t)$, however I wish to stress that $Q(S_t)$ outputs the maximum discounted future reward for all actions. $a_t$ is in this tutorial, a one-hot encoded variable which helps to choose a reward for all possible actions. **Notice how could recursively expand this function.**

## Pseudocode for DQN
Reference: https://github.com/yenchenlin/DeepLearningFlappyBird
```
Initialize replay memory D to size N
Initialize action-value function Q with random weights
for episode = 1, M do
    Initialize state s_1
    for t = 1, T do
        With probability ϵ select random action a_t
        otherwise select a_t=max_a  Q(s_t,a)
        Execute action a_t in emulator and observe r_t and s_(t+1) and if terminated
        Store transition (s_t,a_t,r_t,s_(t+1)) in D
        if episode > observation episodes:
            Sample a minibatch of transitions (s_j,a_j,r_j,s_(j+1)) from D
            Set y_j:=
                r_j for terminal s_(j+1)
                r_j+γ*max_(a^' )  Q(s_(j+1),a'; θ_i) for non-terminal s_(j+1)
            Perform a gradient step on (y_j-Q(s_j,a_j))^2 (and change model Q)
    end for
end for
```

## OpenAI functions

- env = gym.make(environment_name) <- sets up the environment
- env.reset() <- resets the environment to starting point
- env.step(action) <- takes action and goes to state $S_{t+1}$, returns $r_t$, $S_{t+1}$ and if game terminated.
- env.render() <- render the output

## Reference:

- https://keon.io/deep-q-learning/
- http://mckinziebrandon.me/TensorflowNotebooks/2016/12/21/openai.html
- https://github.com/yenchenlin/DeepLearningFlappyBird
- [Playing Atari with Deep Reinforcement Learning](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf)
- [OpenAI environments](https://gym.openai.com/envs/)
- https://www.youtube.com/watch?v=JFJkpVWTQVM

```python
!pip install gym
!conda install -y JSAnimation
```

    Requirement already satisfied: gym in /Users/sachin/anaconda/lib/python3.5/site-packages
    Requirement already satisfied: six in /Users/sachin/anaconda/lib/python3.5/site-packages (from gym)
    Requirement already satisfied: numpy>=1.10.4 in /Users/sachin/anaconda/lib/python3.5/site-packages (from gym)
    Requirement already satisfied: pyglet>=1.2.0 in /Users/sachin/anaconda/lib/python3.5/site-packages (from gym)
    Requirement already satisfied: requests>=2.0 in /Users/sachin/anaconda/lib/python3.5/site-packages (from gym)
    Requirement already satisfied: idna<2.7,>=2.5 in /Users/sachin/anaconda/lib/python3.5/site-packages (from requests>=2.0->gym)
    Requirement already satisfied: certifi>=2017.4.17 in /Users/sachin/anaconda/lib/python3.5/site-packages (from requests>=2.0->gym)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Users/sachin/anaconda/lib/python3.5/site-packages (from requests>=2.0->gym)
    Requirement already satisfied: urllib3<1.23,>=1.21.1 in /Users/sachin/anaconda/lib/python3.5/site-packages (from requests>=2.0->gym)
    Fetching package metadata .............
    Solving package specifications: .

    # All requested packages already installed.
    # packages in environment at /Users/sachin/anaconda:
    #
    jsanimation               0.1                      py35_0    conda-forge

```python
%matplotlib inline
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

import re

from keras.models import Sequential, Model
from keras.layers import Dense, Input, Dot
from keras.models import load_model, model_from_json
from keras.optimizers import Adam

import gym

from collections import deque

import time

%matplotlib inline

# Imports specifically so we can render outputs in Jupyter.
from JSAnimation.IPython_display import display_animation
from matplotlib import animation
from IPython.display import display
```

    Using TensorFlow backend.

```python
def create_model(n_states, n_actions):
    # Maximum future discounted reward
    # Q(S_t)
    state = Input(shape=(n_states,))
    x1 = Dense(4, activation='relu')(state)
    x2 = Dense(4, activation='relu')(x1)
    out = Dense(n_actions)(x2)
    # Q(S_t)(a_t)
    actions = Input(shape=(n_actions,))
    out2 = Dot(axes=-1)([out, actions])

    # wrap the above in Keras Model class
    model = Model(inputs=[state, actions], outputs=out2)
    model.compile(loss='mse', optimizer='rmsprop')

    model2 = Model(inputs=state, outputs=out)

    return model, model2
```

```python
def train_data(minibatch, model):
    s_j_batch = np.array([d[0] for d in minibatch])
    a_batch = np.array([d[1] for d in minibatch])
    r_batch = np.array([d[2] for d in minibatch])
    s_j1_batch = np.array([d[3] for d in minibatch])
    terminal_batch = np.array([d[4] for d in minibatch])

    readout_j1_batch = model.predict(s_j1_batch, batch_size=BATCH)
    y_batch = r_batch + GAMMA * np.max(readout_j1_batch, axis=1)
    y_batch[terminal_batch] = r_batch[terminal_batch]
    return s_j_batch, a_batch, y_batch
```

```python
env = gym.make('CartPole-v0')
STATES, ACTIONS = env.observation_space.shape[0], env.action_space.n
model, out = create_model(STATES, ACTIONS)
INITIAL_EPSILON = 1e-1
FINAL_EPSILON = 1e-4
DECAY = 0.9
GAMMA = 0.9 # decay rate of past observations
OBSERVE = 5000. # timesteps to observe before training
REPLAY_MEMORY = 5000 # number of previous transitions to remember
TIME_LIMIT = 100000
BATCH = 128
# open up a game state to communicate with emulator
state = env.reset()

# store the previous observations in replay memory
D = deque(maxlen=REPLAY_MEMORY)
loss = []

a_t = np.zeros(ACTIONS)
a_t[np.random.choice(ACTIONS)] = 1
s_t, r_0, terminal, _ = env.step(np.argmax(a_t))

# start training
epsilon = INITIAL_EPSILON

up_time = [0]
start = time.time()
for t in range(TIME_LIMIT):
    # choose an action
    readout_t = out.predict(s_t[None, :])
    a_t = np.zeros([ACTIONS])
    if np.random.random() <= epsilon:
        a_t[np.random.choice(ACTIONS)] = 1
    else:
        a_t[np.argmax(readout_t)] = 1

    # scale down epsilon
    if epsilon > FINAL_EPSILON and t > OBSERVE:
        epsilon *= DECAY

    # run the selected action and observe next state and reward
    # store the transition in D
    s_t1, r_t, terminal, _ = env.step(np.argmax(a_t))
    D.append((s_t, a_t, r_t, s_t1, terminal))

    if terminal:
        up_time.append(0)
        env.reset()
    else:
        up_time[-1] += r_t

    # only train if done observing
    if t > OBSERVE:
        # sample a minibatch to train on
        idx = np.random.choice(REPLAY_MEMORY, BATCH, replace=False)
        minibatch = [D[i] for i in idx]
        # get the batch variables
        s_t_batch, a_batch, y_batch = train_data(minibatch, out)
        # perform gradient step
        loss.append(model.train_on_batch([s_t_batch, a_batch], y_batch))

    # update the old state
    s_t = s_t1

    if t%(TIME_LIMIT/20)==0:
        print('Episode :', t, ', time taken: ', time.time() - start, 's, average up time: ',np.mean(up_time[-100:]))
        start = time.time()
```

    [2017-10-29 15:57:44,293] Making new env: CartPole-v0

    Episode : 0 , time taken:  0.04322099685668945 s, average up time:  1.0
    Episode : 5000 , time taken:  1.8772480487823486 s, average up time:  97.0784313725
    Episode : 10000 , time taken:  14.008546113967896 s, average up time:  36.98
    Episode : 15000 , time taken:  13.076855897903442 s, average up time:  70.53
    Episode : 20000 , time taken:  12.869633913040161 s, average up time:  104.74
    Episode : 25000 , time taken:  12.837337017059326 s, average up time:  133.49
    Episode : 30000 , time taken:  12.839324951171875 s, average up time:  146.25
    Episode : 35000 , time taken:  13.132809162139893 s, average up time:  144.52
    Episode : 40000 , time taken:  12.963682889938354 s, average up time:  167.73
    Episode : 45000 , time taken:  13.547950029373169 s, average up time:  181.39
    Episode : 50000 , time taken:  14.607528924942017 s, average up time:  170.48
    Episode : 55000 , time taken:  15.984797954559326 s, average up time:  151.86
    Episode : 60000 , time taken:  16.933092832565308 s, average up time:  146.43
    Episode : 65000 , time taken:  17.319180965423584 s, average up time:  163.32
    Episode : 70000 , time taken:  17.957252979278564 s, average up time:  184.96
    Episode : 75000 , time taken:  21.94894814491272 s, average up time:  198.13
    Episode : 80000 , time taken:  24.839221000671387 s, average up time:  171.12
    Episode : 85000 , time taken:  24.760560035705566 s, average up time:  136.11
    Episode : 90000 , time taken:  20.755791902542114 s, average up time:  116.67
    Episode : 95000 , time taken:  21.71216917037964 s, average up time:  97.53

```python
plt.figure(figsize=(12, 5))
plt.plot(up_time)
plt.show()
```

```python
plt.figure(figsize=(12, 5))
plt.plot(loss)
plt.show()
```

**Only run the following if you are running it locally and not in Docker**

```python
def display_frames_as_gif(frames):
    """
    Displays a list of frames as a gif, with controls
    """
    #plt.figure(figsize=(frames[0].shape[1] / 72.0, frames[0].shape[0] / 72.0), dpi = 72)
    patch = plt.imshow(frames[0])
    plt.axis('off')

    def animate(i):
        patch.set_data(frames[i])

    anim = animation.FuncAnimation(plt.gcf(), animate, frames = len(frames), interval=50)
    display(display_animation(anim, default_mode='once'))
```

```python
env = gym.make('CartPole-v0')

# Run a demo of the environment
observation = env.reset()
cum_reward = 0
frames = []
for t in range(5000):
    # Render into buffer.
    frames.append(env.render(mode = 'rgb_array'))
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)
    if done:
        break
env.render(close=True)
display_frames_as_gif(frames)
print(t)
```

    [2017-10-29 16:04:01,470] Making new env: CartPole-v0

    <input id="_anim_sliderUYKANTEOQDEJFLEV" type="range" style="width:350px" name="points" min="0" max="1" step="1" value="0" onchange="animUYKANTEOQDEJFLEV.set_frame(parseInt(this.value));"></input>

    <button onclick="animUYKANTEOQDEJFLEV.slower()">–</button>
    <button onclick="animUYKANTEOQDEJFLEV.first_frame()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.previous_frame()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.reverse_animation()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.pause_animation()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.play_animation()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.next_frame()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.last_frame()"></button>
    <button onclick="animUYKANTEOQDEJFLEV.faster()">+</button>
  <form action="#n" name="_anim_loop_selectUYKANTEOQDEJFLEV" class="anim_control">
    <input type="radio" name="state" value="once" checked> Once </input>
    <input type="radio" name="state" value="loop" > Loop </input>
    <input type="radio" name="state" value="reflect" > Reflect </input>
  </form>

    15

```python
# Run a demo of the environment
observation = env.reset()
cum_reward = 0
frames = []
a_t = np.zeros([ACTIONS])
a_t[env.action_space.sample()] = 1
for t in range(5000):
    # Render into buffer.
    s_t, reward, done, info = env.step(np.argmax(a_t))
    frames.append(env.render(mode = 'rgb_array'))
    readout_t = out.predict(s_t[None, :])
    a_t = np.zeros([ACTIONS])
    a_t[np.argmax(readout_t)] = 1

    if done:
        break
env.render(close=True)
display_frames_as_gif(frames)
print(t)
```

    <input id="_anim_sliderHQTEPCSOBUZKDJTL" type="range" style="width:350px" name="points" min="0" max="1" step="1" value="0" onchange="animHQTEPCSOBUZKDJTL.set_frame(parseInt(this.value));"></input>

    <button onclick="animHQTEPCSOBUZKDJTL.slower()">–</button>
    <button onclick="animHQTEPCSOBUZKDJTL.first_frame()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.previous_frame()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.reverse_animation()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.pause_animation()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.play_animation()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.next_frame()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.last_frame()"></button>
    <button onclick="animHQTEPCSOBUZKDJTL.faster()">+</button>
  <form action="#n" name="_anim_loop_selectHQTEPCSOBUZKDJTL" class="anim_control">
    <input type="radio" name="state" value="once" checked> Once </input>
    <input type="radio" name="state" value="loop" > Loop </input>
    <input type="radio" name="state" value="reflect" > Reflect </input>
  </form>

    114

```python

```

deepschool.io/Lesson 20 - Deep Q Learning - Solutions.ipynb


```python
import os
import fnmatch

import numpy as np
import pandas as pd

import pint
import sympy

import sissotools
from sissotools import io
from sissotools import dimensional_analysis
from sissotools import regression
from sissotools import plotting
```

```python
try:
    from tqdm import tqdm
except ImportError:
    # Optional import
    class tqdm:
        """Progress bar placeholder"""
        def __init__(self, iterable=None, total=0):
            self.count = 0
            self.total = total
            if iterable is not None:
                self.iterable = iterable
                if hasattr(iterable, "__len__"):
                    self.total = len(iterable)
            else:
                self.iterable = []

        def __iter__(self):
                for obj in self.iterable:
                    self.count += 1
                    print(self.count, "/", self.total)
                    yield obj

        def update(self, n=1):
            self.count += n
            print(self.count, "/", self.total)
```

```python
%matplotlib inline
```

# I/O Options

```python
input_filename = "./data_AD.txt"
working_directory = "./demo"
```

```python
features_fname = os.path.join(working_directory, "feature_space/Uspace.name")

file_list = os.listdir(os.path.join(working_directory, "models"))
models_fname = fnmatch.filter(file_list, "top*0_001d").pop()
models_fname = os.path.join(working_directory, "models", models_fname)

coeff_fname = fnmatch.filter(file_list, "top*_001d_coeff").pop()
coeff_fname = os.path.join(working_directory, "models", coeff_fname)

file_checks = [os.path.isfile(fname) for fname
               in [features_fname, models_fname, coeff_fname]]
if not all(file_checks):
    raise RuntimeError("Please ensure demo_inputs.ipynb has completed.")
```

# Read Files

```python
# parse tab-separated-value file
df_train = io.read_tsv(input_filename)

# parse SISSO outputs
df_features = io.read_features(features_fname)
df_models = io.read_models(models_fname)
df_coeff = io.read_coefficients(coeff_fname)
```

```python
df_features.head()  # feature_space/Uspace.name
```

      Feature
      Correlation

      1
      (((mus*wlog)-wlog)*((lambd-mus)-sqrt(mus)))
      0.9933

      2
      (((lambd-mus)-sqrt(mus))/(exp(mus)/wlog))
      0.9933

      3
      (((lambd-mus)-sqrt(mus))/((wlog)^-1+(mus/wlog)))
      0.9933

      4
      ((log(wlog)*(mus*wlog))-(wlog*(mus+lambd)))
      0.9931

      5
      (((wlog*lambd)/sqrt(mus))-(wlog/(mus+lambd)))
      0.9930

```python
df_models.head()  # models/"top*0_001d"
```

      rmse
      mae
      feature id

      model index

      1
      0.251624
      0.655874
      13

      2
      0.251782
      0.746856
      11

      3
      0.262698
      0.807561
      4

      4
      0.267091
      0.731557
      31

      5
      0.267385
      0.666787
      41

```python
df_coeff.head()  # models/"top*0_001d_coeff"
```

      intercept
      slope

      model index

      1
      0.0
      0.095264

      2
      0.0
      0.023109

      3
      0.0
      -0.105638

      4
      0.0
      -0.101614

      5
      0.0
      -0.094123

# Filtering by units

```python
# Define units per feature
unit_ref = {'wlog': 'K',
            'w2': 'K',
            'lambd': 'dimensionless',
            'mus': 'dimensionless'}

# Set desired units
target_units = "[temperature]"
target_degree = 1
```

```python
filtered_indices = []
for model_row in df_models.itertuples():
    # loop over models, sorted by RMSE
    idx, rmse, mae, feature_id = model_row
    intercept, slope = df_coeff.loc[idx]
    feature, correlation = df_features.loc[feature_id]

    dims, degree = dimensional_analysis.check_dimensionality(
        feature, unit_ref)

    if dims == target_units and degree == target_degree:
        filtered_indices.append(idx)

print("Filtered by units. Remaining: {} / {}".format(
    len(filtered_indices), len(df_models)))
```

    /home/sxiexie/anaconda3/envs/sru/lib/python3.6/site-packages/pint/quantity.py:1238: RuntimeWarning: divide by zero encountered in double_scalars
      magnitude = magnitude_op(new_self._magnitude, other._magnitude)
    /home/sxiexie/anaconda3/envs/sru/lib/python3.6/site-packages/pint/quantity.py:1238: RuntimeWarning: invalid value encountered in double_scalars
      magnitude = magnitude_op(new_self._magnitude, other._magnitude)

    Filtered by units. Remaining: 345 / 1000

    /home/sxiexie/anaconda3/envs/sru/lib/python3.6/site-packages/pint/quantity.py:1486: RuntimeWarning: divide by zero encountered in double_scalars
      magnitude = new_self._magnitude ** exponent

# Filtering by other constraints

```python
feature_columns = ["wlog", "lambd", "mus"]
```

```python
lambda_res = 401  # resolution: 0.01
mus_res = 61  # resolution: 0.01
wlog_res = 100  # resolution: 10 K

lambda_range = np.linspace(0, 4, lambda_res)
mus_range = np.linspace(0.1, 0.16, mus_res)
wlog_range = np.linspace(10, 1000, wlog_res)

lambda_range[0] += 1e-9  # evaluate one-sided limit

validation_meshes = np.meshgrid(wlog_range, lambda_range, mus_range)
validation_space = np.vstack([mesh.flatten() for mesh
                              in validation_meshes]).T
df_val = pd.DataFrame(columns=feature_columns,
                      data=validation_space)
print("Validation points:", len(validation_space))
```

    Validation points: 2446100

```python
transform = "({}) / wlog"  # normalize

idx_pass = []
checks_fail = {}
for idx in tqdm(filtered_indices):
    feature_id = df_models.loc[idx, "feature id"]
    expr = df_features.loc[feature_id, "Feature"]
    expr = transform.format(expr)
    coeff = tuple(df_coeff.loc[idx][::-1])

    values, _ = regression.evaluate_model(expr,
                                          df_val,
                                          feature_columns,
                                          coeff)

    lambda_sort = np.argsort(validation_space[:, 1])
    unique_lambda, counts = np.unique(validation_space[:, 1],
                                      return_counts=True)
    split_indices = np.cumsum(counts)[:-1]

    split_values = np.array_split(values[lambda_sort], split_indices)
    lower_curve = np.array([np.min(batch) for batch in split_values])
    upper_curve = np.array([np.max(batch) for batch in split_values])

    lower_gradient = np.gradient(lower_curve)
    upper_gradient = np.gradient(upper_curve)

    checks = [np.all(np.isfinite(values)),  # finite
              np.all(values > 0),           # strictly positive
              np.all(lower_gradient>=0),    # monotonic
              np.all(upper_gradient>=0),    # monotonic
              lower_gradient[0] < 1e-3,     # lambda limit
              upper_gradient[0] < 1e-3,     # lambda limit
              ]

    if np.all(checks):
        idx_pass.append(idx)
    else:
        checks_fail[idx] = checks
```

    1 / 345
    2 / 345
    3 / 345
    4 / 345
    5 / 345
    6 / 345
    7 / 345
    8 / 345
    9 / 345
    10 / 345
    11 / 345
    12 / 345
    13 / 345
    14 / 345
    15 / 345
    16 / 345
    17 / 345
    18 / 345
    19 / 345
    20 / 345
    21 / 345
    22 / 345
    23 / 345
    24 / 345
    25 / 345
    26 / 345
    27 / 345
    28 / 345
    29 / 345
    30 / 345

    <string>:3: RuntimeWarning: divide by zero encountered in true_divide
    /home/sxiexie/anaconda3/envs/sru/lib/python3.6/site-packages/numpy/lib/function_base.py:1068: RuntimeWarning: invalid value encountered in subtract
      out[tuple(slice1)] = (f[tuple(slice4)] - f[tuple(slice2)]) / (2. * ax_dx)

    31 / 345
    32 / 345
    33 / 345
    34 / 345
    35 / 345
    36 / 345
    37 / 345
    38 / 345
    39 / 345
    40 / 345
    41 / 345
    42 / 345
    43 / 345
    44 / 345
    45 / 345
    46 / 345
    47 / 345
    48 / 345
    49 / 345
    50 / 345
    51 / 345
    52 / 345
    53 / 345
    54 / 345
    55 / 345

    <string>:3: RuntimeWarning: divide by zero encountered in true_divide

    56 / 345
    57 / 345
    58 / 345
    59 / 345
    60 / 345
    61 / 345
    62 / 345
    63 / 345
    64 / 345
    65 / 345
    66 / 345
    67 / 345
    68 / 345
    69 / 345
    70 / 345
    71 / 345
    72 / 345
    73 / 345
    74 / 345
    75 / 345
    76 / 345
    77 / 345
    78 / 345
    79 / 345
    80 / 345
    81 / 345
    82 / 345
    83 / 345
    84 / 345
    85 / 345
    86 / 345
    87 / 345
    88 / 345
    89 / 345
    90 / 345
    91 / 345
    92 / 345
    93 / 345
    94 / 345
    95 / 345
    96 / 345
    97 / 345
    98 / 345
    99 / 345
    100 / 345
    101 / 345
    102 / 345
    103 / 345
    104 / 345
    105 / 345
    106 / 345
    107 / 345
    108 / 345
    109 / 345
    110 / 345
    111 / 345
    112 / 345
    113 / 345
    114 / 345
    115 / 345
    116 / 345
    117 / 345
    118 / 345
    119 / 345
    120 / 345
    121 / 345
    122 / 345
    123 / 345
    124 / 345
    125 / 345
    126 / 345
    127 / 345
    128 / 345
    129 / 345
    130 / 345
    131 / 345
    132 / 345
    133 / 345
    134 / 345
    135 / 345
    136 / 345
    137 / 345
    138 / 345
    139 / 345
    140 / 345
    141 / 345
    142 / 345
    143 / 345
    144 / 345
    145 / 345
    146 / 345
    147 / 345
    148 / 345
    149 / 345
    150 / 345
    151 / 345
    152 / 345
    153 / 345
    154 / 345
    155 / 345
    156 / 345
    157 / 345
    158 / 345
    159 / 345
    160 / 345
    161 / 345
    162 / 345
    163 / 345
    164 / 345
    165 / 345
    166 / 345
    167 / 345
    168 / 345
    169 / 345
    170 / 345
    171 / 345
    172 / 345
    173 / 345
    174 / 345
    175 / 345
    176 / 345
    177 / 345
    178 / 345
    179 / 345
    180 / 345
    181 / 345
    182 / 345
    183 / 345
    184 / 345
    185 / 345
    186 / 345
    187 / 345
    188 / 345
    189 / 345
    190 / 345
    191 / 345
    192 / 345
    193 / 345
    194 / 345
    195 / 345
    196 / 345
    197 / 345
    198 / 345
    199 / 345
    200 / 345
    201 / 345

    <string>:3: RuntimeWarning: invalid value encountered in sqrt

    202 / 345
    203 / 345
    204 / 345
    205 / 345
    206 / 345
    207 / 345

    <string>:3: RuntimeWarning: invalid value encountered in sqrt
    <string>:3: RuntimeWarning: invalid value encountered in sqrt

    208 / 345
    209 / 345
    210 / 345
    211 / 345
    212 / 345
    213 / 345
    214 / 345
    215 / 345
    216 / 345
    217 / 345
    218 / 345
    219 / 345
    220 / 345
    221 / 345
    222 / 345
    223 / 345
    224 / 345
    225 / 345

    <string>:3: RuntimeWarning: divide by zero encountered in true_divide

    226 / 345
    227 / 345
    228 / 345
    229 / 345
    230 / 345
    231 / 345
    232 / 345
    233 / 345
    234 / 345
    235 / 345
    236 / 345
    237 / 345
    238 / 345
    239 / 345
    240 / 345
    241 / 345
    242 / 345
    243 / 345
    244 / 345
    245 / 345

    <string>:3: RuntimeWarning: divide by zero encountered in true_divide

    246 / 345
    247 / 345
    248 / 345
    249 / 345
    250 / 345
    251 / 345
    252 / 345
    253 / 345
    254 / 345
    255 / 345
    256 / 345
    257 / 345
    258 / 345
    259 / 345
    260 / 345
    261 / 345
    262 / 345
    263 / 345
    264 / 345
    265 / 345
    266 / 345
    267 / 345
    268 / 345
    269 / 345
    270 / 345
    271 / 345
    272 / 345
    273 / 345
    274 / 345
    275 / 345
    276 / 345
    277 / 345
    278 / 345
    279 / 345
    280 / 345
    281 / 345
    282 / 345
    283 / 345
    284 / 345
    285 / 345
    286 / 345
    287 / 345
    288 / 345
    289 / 345
    290 / 345
    291 / 345
    292 / 345
    293 / 345
    294 / 345
    295 / 345
    296 / 345
    297 / 345
    298 / 345
    299 / 345
    300 / 345
    301 / 345
    302 / 345
    303 / 345
    304 / 345
    305 / 345
    306 / 345
    307 / 345
    308 / 345
    309 / 345
    310 / 345
    311 / 345
    312 / 345
    313 / 345
    314 / 345
    315 / 345
    316 / 345
    317 / 345
    318 / 345
    319 / 345
    320 / 345
    321 / 345
    322 / 345
    323 / 345
    324 / 345
    325 / 345
    326 / 345
    327 / 345
    328 / 345
    329 / 345
    330 / 345
    331 / 345
    332 / 345
    333 / 345
    334 / 345
    335 / 345
    336 / 345
    337 / 345
    338 / 345
    339 / 345
    340 / 345
    341 / 345
    342 / 345
    343 / 345
    344 / 345
    345 / 345

```python
print("Filtered by physical constraints. Remaining: {} / {}".format(
    len(idx_pass), len(filtered_indices)))
```

    Filtered by physical constraints. Remaining: 82 / 345

# Summarizing filtered models

```python
rmse_list = df_models.loc[idx_pass, "rmse"]
features_idx = df_models.loc[idx_pass, "feature id"].values
features_list = df_features.loc[features_idx, "Feature"]
slope_list = df_coeff.loc[idx_pass, "slope"]

```

```python
data = np.vstack([rmse_list, slope_list]).T
df_summary = pd.DataFrame(data=data,
                          columns=["RMSE", "Coeff."],
                          index=features_list)
```

```python
df_summary
```

      RMSE
      Coeff.

      Feature

      (((lambd)^3*(wlog*lambd))/((lambd)^3+sqrt(mus)))
      0.251624
      0.095264

      ((exp(-lambd)-exp(mus))*(log(mus)*(wlog*lambd)))
      0.345276
      0.041190

      ((exp(-lambd)-exp(mus))*((wlog*lambd)/sqrt(mus)))
      0.350186
      -0.029923

      (((wlog*lambd)/exp(lambd))-(wlog*lambd))
      0.354138
      -0.108653

      ((log(mus)*(lambd)^2)/((wlog)^-1+(lambd/wlog)))
      0.361629
      -0.060127

      ...
      ...
      ...

      (((lambd)^3*(wlog*lambd))/((lambd)^3+(mus+lambd)))
      0.680469
      0.125574

      (((lambd)^2*(wlog/mus))/(lambd+(lambd)^-1))
      0.694425
      0.011984

      (((lambd)^2*(wlog/mus))/((lambd)^-1+(mus+lambd)))
      0.718787
      0.012481

      (((lambd)^3*(wlog*lambd))/((lambd)^3+sqrt(lambd)))
      0.750743
      0.117304

      (((lambd)^3*(wlog/mus))/((lambd)^2+(lambd)^-1))
      0.969725
      0.010934

82 rows × 2 columns

# Visualizing filtered models

```python
lambda_res = 401  # resolution: 0.01
lambda_range = np.linspace(0, 4, lambda_res)
lambda_range[0] = 1e-9
wlog_range = [50]
mus_range = [0.1]

validation_meshes = np.meshgrid(wlog_range, lambda_range, mus_range)
validation_space = np.vstack([mesh.flatten() for mesh
                              in validation_meshes]).T
df_val = pd.DataFrame(columns=feature_columns,
                      data=validation_space)
print("Plotting points:", len(validation_space))
```

    Plotting points: 401

```python
import matplotlib.pyplot as plt

plt.figure(figsize=(3.5, 3), dpi=150)
for i, (expr, slope) in enumerate(zip(features_list, slope_list)):
    coeff = (slope, 0)

    expr = transform.format(expr)

    values, _ = regression.evaluate_model(expr,
                                          df_val,
                                          feature_columns,
                                          coeff)
    if i == 0:
        plt.plot(lambda_range,
                 values,
                 linewidth=3,
                 color='cornflowerblue',
                 zorder=10)
    else:
        plt.plot(lambda_range, values, linewidth=1, color='k', alpha=0.5)
plt.plot([], [], linewidth=3, color='cornflowerblue', label="Best Equation")
plt.plot([], [], linewidth=1, color='k', alpha=0.1, label="Eq. 2 ~ {}".format(len(features_list)))
plt.legend(loc='lower right')
plt.xlim(0, 4)
plt.ylim(0, 0.34)
plt.xlabel("$\lambda$")
plt.ylabel("$T_C / \omega_{log}$")
plt.tight_layout()
```

```python

```


# Neuromatch Academy: Week 1, Day 3, Tutorial 4
# Model Fitting: Multiple linear regression

#Tutorial Objectives

This is Tutorial 4 of a series on fitting models to data. We start with simple linear regression, using least squares optimization (Tutorial 1) and Maximum Likelihood Estimation (Tutorial 2). We will use bootstrapping to build confidence intervals around the inferred linear model parameters (Tutorial 3). We'll finish our exploration of linear models by  generalizing to multiple linear regression (Tutorial 4). We then move on to polynomial regression (Tutorial 5). We end by learning how to choose between these various models. We discuss the bias-variance trade-off (Tutorial 6) and two common methods for model selection, AIC and Cross Validation (Tutorial 7).

In this tutorial, we will generalize our linear model to incorporate multiple linear features.
- Learn how to structure our inputs for multiple linear regression using the 'Design Matrix'
- Generalize the MSE for multiple features using the ordinary least squares estimator
- Visualize our data and model fit in multiple dimensions

# Setup

```python
# @title  Imports
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
```

```python
#@title Figure Settings
%matplotlib inline
fig_w, fig_h = (8, 6)
plt.rcParams.update({'figure.figsize': (fig_w, fig_h)})
%config InlineBackend.figure_format = 'retina'
```

# Multiple Linear Regression

```python
#@title Video: Multiple Linear Regression
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="uQjKnlhGEVY", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=uQjKnlhGEVY

Now that we have considered the univariate case and how to produce confidence intervals for our estimator, we turn now to the general linear regression case, where we can have more than one regressor, or feature, in our input.

Recall that our original univariate linear model was given as

\begin{align}
y = \theta_0 + \theta_1 x
\end{align}

where $\theta_0$ is the intercept, $\theta_1$ is the slope. We can easily extend this to the multivariate scenario by adding another parameter for each additional feature

\begin{align}
y = \theta_0 + \theta_1 x_1 + \theta_1 x_2 + ... +\theta_d x_d
\end{align}

where $d$ is the dimensionality (number of features) in our input.

We can condense this succinctly using vector notation for a single data point

\begin{align}
y_i = \boldsymbol{\theta}^{\top}\mathbf{x}_i
\end{align}

and fully in matrix form

\begin{align}
\mathbf{y} = \mathbf{X}\boldsymbol{\theta}
\end{align}

where $\mathbf{y}$ is a vector of measurements, $\mathbf{X}$ is a matrix containing the feature values (columns) for each input sample (rows), and $\boldsymbol{\theta}$ is our parameter vector.

This matrix $\mathbf{X}$ is often referred to as the "[design matrix](https://en.wikipedia.org/wiki/Design_matrix)".

For this tutorial we will focus on the two-dimensional case ($d=2$), which allows us to fully explore the multivariate case while still easily visualizing our results. As an example, think of a situation where a scientist records the spiking response of a retinal ganglion cell to patterns of light signals that vary in contrast and in orientation.

In this case our model can be writen as

\begin{align}
y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \epsilon
\end{align}

or in matrix form where

\begin{align}
\mathbf{X} =
\begin{bmatrix}
1 & x_{1,1} & x_{1,2} \\
1 & x_{2,1} & x_{2,2} \\
\vdots & \vdots & \vdots \\
1 & x_{n,1} & x_{n,2}
\end{bmatrix},
\boldsymbol{\theta} =
\begin{bmatrix}
\theta_0 \\
\theta_1 \\
\theta_2 \\
\end{bmatrix}
\end{align}

For our actual exploration dataset we shall set $\boldsymbol{\theta}=[0, -2, -3]$ and draw $N=40$ noisy samples from $x \in [-2,2)$. Note that setting the value of $\theta_0 = 0$ effectively ignores the offset term.

```python
np.random.seed(121)
theta = [0, -2, -3]
n_samples = 40
n_regressors = len(theta)
x = np.random.uniform(-2, 2, (n_samples, n_regressors))
noise = np.random.randn(n_samples)
y = x @ theta + noise
```

Now that we have our dataset, we want to find an optimal vector of paramters $\boldsymbol{\hat\theta}$. Recall our analytic solution to minimizing MSE for a single regressor:

\begin{align}
\hat\theta = \sum_i \frac{x_i y_i}{x_i^2}.
\end{align}

The same holds true for the multiple regressor case, only now expressed in matrix form

\begin{align}
\boldsymbol{\hat\theta} = (\mathbf{X}^\top\mathbf{X})^{-1}\mathbf{X}^\top\mathbf{y}.
\end{align}

This is called the [ordinary least squares](https://en.wikipedia.org/wiki/Ordinary_least_squares) (OLS) estimator.

### Exercise: Ordinary Least Squares Estimator

In this exercise you will implement the OLS approach to estimating $\boldsymbol{\hat\theta}$ from the design matrix $\mathbf{X}$ and measurement vector $\mathbf{y}$. You can the `@` symbol for matrix multiplication, `.T` for transpose, and `np.linalg.inv` for matrix inversion.

```python
def ordinary_least_squares(x, y):
  """Ordinary least squares estimator for linear regression.

  Args:
    x (ndarray): design matrix of shape (n_samples, n_regressors)
    y (ndarray): vector of measurements of shape (n_samples)

  Returns:
    ndarray: estimated parameter values of shape (n_regressors)
  """
  ######################################################################
  ## TODO for students: solve for the optimal parameter vector using OLS
  ######################################################################
  theta_hat = np.linalg.inv(x.T @ x) @ x.T @ y

  # comment this out when you've filled
  #raise NotImplementedError("Student excercise: solve for theta_hat vector using OLS")
  return theta_hat
```

```python
# to_remove solution
def ordinary_least_squares(x, y):
  """Ordinary least squares estimator for linear regression.

  Args:
    x (ndarray): design matrix of shape (n_samples, n_regressors)
    y (ndarray): vector of measurements of shape (n_samples)

  Returns:
    ndarray: estimated parameter values of shape (n_regressors)
  """
  theta_hat = np.linalg.inv(x.T @ x) @ x.T @ y
  return theta_hat
```

```python
theta_hat = ordinary_least_squares(x, y)
theta_hat
```

    array([ 0.27846561, -2.01651235, -3.14249005])

Now that we have our $\mathbf{\hat\theta}$, we can obtain $\mathbf{\hat y}$ and thus our mean squared error.

```python
y_hat = x @ theta_hat
print(f"MSE = {np.mean((y - y_hat)**2):.2f}")
```

    MSE = 0.57

Finally, the following code will plot a geometric visualization of the data points (blue) and fitted plane. You can see that the residuals (green bars) are orthogonal to the hyperplane.

```python
xx, yy = np.mgrid[-2:2:50j, -2:2:50j]
y_hat_grid = np.array([xx.flatten(), yy.flatten()]).T @ theta_hat[1:]
y_hat_grid = y_hat_grid.reshape((50, 50))

ax = plt.subplot(projection='3d')
ax.plot(x[:,1], x[:,2], y, '.')
ax.plot_surface(xx, yy, y_hat_grid, linewidth=0, alpha=0.5, color='C1',
                cmap=plt.get_cmap('coolwarm'))

for i in range(len(x)):
  ax.plot((x[i, 1], x[i, 1]),
          (x[i, 2], x[i, 2]),
          (y[i], y_hat[i]),
          'g-', alpha=.5)

ax.set(
    xlabel='$x_1$',
    ylabel='$x_2$',
    zlabel='y'
)
plt.tight_layout()

```

# Summary
- linear regression generalizes naturally to multiple dimensions
- linear algebra affords us the mathematical tools to reason and solve such problems beyond the two dimensional case

**NOTE** in practice, multidimensional least squares problems can be solve very efficiently (thanks to numerical routines such as LAPACK).

```python

```


Computational Seismology

Finite Volume Method - 1D Elastic Wave Equation

Seismo-Live: http://seismo-live.org

##### Authors:
* David Vargas ([@dvargas](https://github.com/davofis))
* Heiner Igel ([@heinerigel](https://github.com/heinerigel))

## Basic Equations

The source free elastic wave equation can be written in terms of a coupled first-order system

\begin{align}
\partial_t \sigma - \mu \partial_x v & = 0 \\
\partial_t v - \frac{1}{\rho} \partial_x \sigma & = 0
\end{align}

with $\rho$ the density and $\mu$ the shear modulus. This equation in matrix-vector notation follows

\begin{equation}
\partial_t \mathbf{Q} + \mathbf{A} \partial_x \mathbf{Q} = 0
\end{equation}

where $\mathbf{Q} = (\sigma, v)$ is the vector of unknowns and the matrix $\mathbf{A}$ contains the parameters $\rho$ and $\mu$. The above matrix equation is analogous to the advection equation $\partial_t q + a \partial_x q = 0$. Although it is a coupled system, diagonalization of $\mathbf{A} = \mathbf{R}^{-1}\mathbf{\Lambda}\mathbf{R}$ allows us to implement all elements developed for the solution of the advection equation in terms of fluxes. It turns out that the decoupled version is

\begin{equation}
\partial_t \mathbf{W} + \mathbf{\Lambda} \partial_x \mathbf{W} = 0
\end{equation}

where the eigenvector matrix $\mathbf{R}$ and the diagonal matrix of eigenvalues $\mathbf{\Lambda}$ are given

\begin{equation}
\mathbf{W} = \mathbf{R}^{-1}\mathbf{Q}
\qquad\text{,}\qquad
\mathbf{\Lambda}=
     \begin{pmatrix}
       -c & 0 \\
        0 & c
     \end{pmatrix}
\qquad\text{,}\qquad
\mathbf{R} =
     \begin{pmatrix}
       Z & -Z \\
       1 &  1
     \end{pmatrix}
\qquad\text{and}\qquad
\mathbf{R}^{-1} = \frac{1}{2Z}
     \begin{pmatrix}
       1 & Z \\
      -1 & Z
     \end{pmatrix}
\end{equation}

Here $Z = \rho c$ with $c = \sqrt{\mu/\rho}$ represents the seismic impedance.

This notebook implements both upwind and Lax-Wendroff schemes for solving the free source version of the elastic wave equation in a homogeneous media. To keep the problem simple we use as spatial initial condition a Gauss function with half-width $\sigma$

\begin{equation}
Q(x,t=0)  = e^{-1/\sigma^2 (x - x_{o})^2}
\end{equation}

```python
# Import all necessary libraries, this is a configuration step for the exercise.
# Please run it before the simulation code!
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation

# Show the plots in the Notebook.
plt.switch_backend("nbagg")
```

### 1. Initialization of setup

```python
# Initialization of setup
# --------------------------------------------------------------------------
nx    = 800        # number of grid points
c     = 2500       # acoustic velocity in m/s
ro    = 2500       # density in kg/m^3
Z     = ro*c       # impedance
mu    = ro*c**2    # shear modulus

xmax  = 10000      # Length in m
eps   = 0.5        # CFL
tmax  = 2.0        # simulation time in s
isnap = 10         # plotting rate
sig   = 200        # argument in the inital condition
x0    = 5000       # position of the initial condition

imethod = 'upwind'  # 'Lax-Wendroff', 'upwind'

# Initialize Space
x, dx = np.linspace(0,xmax,nx,retstep=True)

# use wave based CFL criterion
dt = eps*dx/c    # calculate time step from stability criterion

# Simulation time
nt = int(np.floor(tmax/dt))

# Initialize wave fields
Q = np.zeros((2,nx))
Qnew = np.zeros((2,nx))
Qa = np.zeros((2,nx))
```

### 2. Initial condition

Seismic disturbances are introduced through specification of a particular spatial initial condition, in this case we use a Gaussian function.

#### Exercise 1
Implement the spatial initial condition given by:

\begin{equation}
Q(x,t=0)  = e^{-1/\sigma^2 (x - x_{o})^2}
\end{equation}

Then, visualize the initial condition in a given plot.

```python
# Initial condition
# --------------------------------------------------------------------------
sx = np.exp(-1./sig**2 * (x-x0)**2)
Q[0,:] = sx

# ---------------------------------------------------------------
# Plot initial condition
# ---------------------------------------------------------------
plt.plot(x, sx, color='r', lw=2, label='Initial condition')
plt.ylabel('Amplitude', size=16)
plt.xlabel('x', size=16)
plt.legend()
plt.grid(True)
plt.show()
```

### 3. Solution for the homogeneous problem

#### Upwind finite volume scheme
We decompose the solution into right propagating $\mathbf{\Lambda}^{+}$ and left propagating eigenvalues $\mathbf{\Lambda}^{-}$ where

\begin{equation}
\mathbf{\Lambda}^{+}=
     \begin{pmatrix}
       -c & 0 \\
        0 & 0
     \end{pmatrix}
\qquad\text{,}\qquad
\mathbf{\Lambda}^{-}=
     \begin{pmatrix}
        0 & 0 \\
        0 & c
     \end{pmatrix}
\qquad\text{and}\qquad
\mathbf{A}^{\pm} = \mathbf{R}^{-1}\mathbf{\Lambda}^{\pm}\mathbf{R}
\end{equation}

This strategy allows us to formulate an upwind finite volume scheme for any hyperbolic system as

\begin{equation}
\mathbf{Q}_{i}^{n+1}  = \mathbf{Q}_{i}^{n} - \frac{dt}{dx}(\mathbf{A}^{+}\Delta\mathbf{Q}_{l} - \mathbf{A}^{-}\Delta\mathbf{Q}_{r})
\end{equation}

with corresponding flux term given by

\begin{equation}
\mathbf{F}_{l} =  \mathbf{A}^{+}\Delta\mathbf{Q}_{l}
\qquad\text{,}\qquad
\mathbf{F}_{r}  =  \mathbf{A}^{-}\Delta\mathbf{Q}_{r}
\end{equation}

#### Lax-Wendroff finite volume scheme

The upwind solution presents a strong diffusive behaviour. In this sense, the Lax-Wendroff perform better, with the advantage that it is not needed to decompose the eigenvalues into right and left propagations. Here the matrix $\mathbf{A}$ can be used in its original form. The Lax-Wendroff follows

\begin{equation}
\mathbf{Q}_{i}^{n+1}  = \mathbf{Q}_{i}^{n} - \frac{dt}{2dx}\mathbf{A}(\mathbf{Q}_{i+1}^{n} - \mathbf{Q}_{i-1}^{n}) + \frac{1}{2}\Big(\frac{dt}{dx}\Big)^2\mathbf{A}^2(\mathbf{Q}_{i+1}^{n} - 2\mathbf{Q}_{i}^{n} + \mathbf{Q}_{i-1}^{n})
\end{equation}

#### Exercise 2
Initialize all relevant matrices, i.e $R$, $R^{-1}$, $\mathbf{\Lambda}^{+}$, $\mathbf{\Lambda}^{-}$, $\mathbf{A}^{+}$, $\mathbf{A}^{-}$, $\mathbf{A}$.

```python
# Initialize all matrices
# --------------------------------------------------------------------------

R = np.array([[Z, -Z],[1, 1]])        # Eq. 8.43
Rinv = np.linalg.inv(R)               # Eq. 8.43
Lp = np.array([[0, 0], [0, c]])       # Eq. 8.49
Lm = np.array([[-c, 0], [0, 0]])      # Eq. 8.49
Ap = R @ Lp @ Rinv                    # Eq. 8.50
Am = R @ Lm @ Rinv                    # Eq. 8.50
A = np.array([[0, -mu], [-1/ro, 0]])  # Eq. 8.35
```

### 4. Finite Volumes solution

```python
# Initialize animated plot
# ---------------------------------------------------------------
fig = plt.figure(figsize=(10,6))
ax1 = fig.add_subplot(2,1,1)
ax2 = fig.add_subplot(2,1,2)
line1 = ax1.plot(x, Q[0,:], 'k', x, Qa[0,:], 'r--')
line2 = ax2.plot(x, Q[1,:], 'k', x, Qa[1,:], 'r--')
ax1.set_ylabel('Stress')
ax2.set_ylabel('Velocity')
ax2.set_xlabel(' x ')
plt.suptitle('Homogeneous F. volume - %s method'%imethod, size=16)

plt.ion()    # set interective mode
plt.show()

# ---------------------------------------------------------------
# Time extrapolation
# ---------------------------------------------------------------
for i in range(nt):
    if imethod =='Lax-Wendroff':
        for j in range(1,nx-1):
            dQ1 = Q[:,j+1] - Q[:,j-1]
            dQ2 = Q[:,j-1] - 2*Q[:,j] + Q[:,j+1]
            Qnew[:,j] = Q[:,j] - 0.5*dt/dx*(A @ dQ1)\
            + 1./2.*(dt/dx)**2 * (A @ A) @ dQ2    # Eq. 8.56

        # Absorbing boundary conditions
        Qnew[:,0] = Qnew[:,1]
        Qnew[:,nx-1] = Qnew[:,nx-2]

    elif imethod == 'upwind':
        for j in range(1,nx-1):
            dQl = Q[:,j] - Q[:,j-1]
            dQr = Q[:,j+1] - Q[:,j]
            Qnew[:,j] = Q[:,j] - dt/dx * (Ap @ dQl + Am @ dQr)  # Eq. 8.54

        # Absorbing boundary conditions
        Qnew[:,0] = Qnew[:,1]
        Qnew[:,nx-1] = Qnew[:,nx-2]
    else:
        raise NotImplementedError

    Q, Qnew = Qnew, Q

    # --------------------------------------
    # Animation plot. Display solution
    if not i % isnap:
        for l in line1:
            l.remove()
            del l
        for l in line2:
            l.remove()
            del l
        # --------------------------------------
        # Analytical solution (stress i.c.)
        Qa[0,:] = 1./2.*(np.exp(-1./sig**2 * (x-x0 + c*i*dt)**2)\
        + np.exp(-1./sig**2 * (x-x0-c*i*dt)**2))

        Qa[1,:] = 1/(2*Z)*(np.exp(-1./sig**2 * (x-x0+c*i*dt)**2)\
        - np.exp(-1./sig**2 * (x-x0-c*i*dt)**2))

        # --------------------------------------
        # Display lines
        line1 = ax1.plot(x, Q[0,:], 'k', x, Qa[0,:], 'r--', lw=1.5)
        line2 = ax2.plot(x, Q[1,:], 'k', x, Qa[1,:], 'r--', lw=1.5)
        plt.legend(iter(line2), ('F. Volume', 'Analytic'))
        plt.gcf().canvas.draw()

```

```python

```


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

```python
# Imports
from sympy import init_printing, Matrix, symbols, sqrt, Rational, acos
from numpy import matrix, transpose, sqrt
from numpy.linalg import pinv, inv, det, svd, norm
from scipy.linalg import pinv2
init_printing()
```

# An overview of key ideas

## Moving from vectors to matrices

Consider two position vectors $\underline{u} , \underline{v} \in \mathbb{R}^{3}$.  They can be expressed as column vectors.

$$ \underline{u}=\begin{bmatrix}1\\-1\\0\end{bmatrix} \\ \underline{v}=\begin{bmatrix}0\\1\\-1\end{bmatrix} \tag{1} $$

We can add constant scalar multiples of these vectors, with the scalars $x_1$ and $x_2$.

$$ {x}_{1}\underline{u}+{x}_{2}\underline{v}=\underline{b} \tag{2} $$

This is simple vector addition.  It's easy to visualize that if we combine all possible combinations, that we start filling a plane through the origin.  Adding a third vector that is not in this plane will, using all possible linear combinations, extend to fill all of three-dimensional space.

$$ \underline{w}=\begin{bmatrix}0\\0\\1\end{bmatrix} \tag{3} $$

We now have (4) below.

$$ {x}_{1}\underline{u}+{x}_{2}\underline{v}+{x}_{3}\underline{w}={b} \tag{4} $$

We can write the coefficients of the vectors as a vector of unknowns, $\underline{x}$, shown in (5).

$$ \underline{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}\tag{5}$$

Let us now write (4) is the form $A \underline{x} = \underline{b}$.

$$ \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{bmatrix}\begin{bmatrix} { x }_{ 1 } \\ { x }_{ 2 } \\ { x }_{ 3 } \end{bmatrix}=\begin{bmatrix} { x }_{ 1 } \\ { x }_{ 2 }-{ x }_{ 1 } \\ { x }_{ 3 }-{ x }_{ 2 } \end{bmatrix} \tag{6} $$

We create matrix $A$ in code below.

```python
A = Matrix([[1, 0, 0], [-1, 1, 0], [0, -1, 1]]) # Creating a matrix and putting
# it into a computer variable called C
A # Displaying it to the screen
```

This is the column-view of matrix-vector multiplication as opposed to the row view.  Matrices are seen as column, representing vectors.  Each element of the column vector $\underline{x}$ is a scalar multiple of the corresponding column in the matrix $A$.

$$ { x }_{ 1 }\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}+{ x }_{ 2 }\begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix}+{ x }_{ 3 }\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}=\begin{matrix} { x }_{ 1 } \\ -{ x }_{ 1 }+{ x }_{ 2 } \\ -{ x }_{ 2 }+{ x }_{ 3 } \end{matrix} = {x}_{1}\underline{u}+{x}_{2}\underline{v}+{x}_{3}\underline{w} \tag{7}$$

If we consider the solution vector, $\underline{b}$, then we can write (8).

$$ \begin{bmatrix} { x }_{ 1 } \\ { x }_{ 2 }-{ x }_{ 1 } \\ { x }_{ 3 }-{ x }_{ 2 } \end{bmatrix} = \begin{bmatrix}{b}_{1}\\{b}_{2}\\{b}_{3}\end{bmatrix} \tag{8} $$

By substitution this can be _converted_ to (9).

$$ \begin{bmatrix} { x }_{ 1 } \\ { x }_{ 2 } \\ { x }_{ 3 } \end{bmatrix}=\begin{bmatrix} { b }_{ 1 } \\ { b }_{ 1 }+{ b }_{ 2 } \\ { b }_{ 1 }+{ b }_{ 2 }+{ b }_{ 2 } \end{bmatrix} \tag{9} $$

This is, in fact, a contrived example.  Looking closely, we can write the right-hand side of (9) as a matrix times $\underline{b}$.

$$ \begin{bmatrix}1&0&0\\1&1&0\\1&1&1\end{bmatrix}\begin{bmatrix}{b}_{1}\\{b}_{2}\\{b}_{3}\end{bmatrix} \tag{10} $$

This matrix is the inverse of $A$ such that $\underline{x} = A^{-1} \underline{b}$.  We will learn much more about matrix inverses and whether matrices are invertible.  The `sympy` library has a method called `.inv()` that will calculate the inverse.

```python
A.det()
```

```python
A.inv()  # Inverse of matrix A
```

Something else we will come across is _lower triangular form_.  Notice that all the values above the _main diagonal_ (values from the top-left to the bottom-right) as zero value.  This matrix is _lower triangular_.

Now, let's replace $\underline{w}$ and create a matrix $M$.  Unlike our original $\underline{w}$, this one is _special_ (for our purposes).

```python
x1, x2, x3, b1, b2, b3 = symbols('x1, x2, x3, b1, b2, b3') # Creating algebraic symbols
# This reserves these symbols so as not to see them as computer variable names
```

```python
M = Matrix([[1, 0, -1], [-1, 1, 0], [0, -1, 1]]) # Creating a matrix and putting
# it into a computer variable called M
M # Displaying it to the screen
```

And a vector of unknowns.

```python
x_vect = Matrix([[x1], [x2], [x3]]) # Giving this columns vector a computer
# variable name
x_vect
```

Now, $M \underline{x}$ can be calculated.

```python
M * x_vect
```

For the solution vector, we have three equations that we can create.

$$ { x }_{ 1 }-{ x }_{ 3 }={ b }_{ 1 }\\ { x }_{ 2 }-{ x }_{ 1 }={ b }_{ 2 }\\ { x }_{ 3 }-{ x }_{ 2 }={ b }_{ 3 } \tag{11}$$

Adding the left and righ-hand sides gives us $x_1 - x_3 + x_2 - x_1 + x_3 - x_2 = b_1 + b_2 + b_3$.

This leaves us with a constraint for $\underline{b}$ such that $ 0={ b }_{ 1 }+{ b }_{ 2 }+{ b }_{ 3 } $.

The problem is clear to see geometrically as the new $\underline{w}$ is in the same plane as $\underline{u}$ and $\underline{v}$.  In essence $\underline{w}$ did not add anything.  All combinations of $\underline{u}$, $\underline{v}$, and $\underline{w}$ will still be in the same plane.

The first $\underline{w}$ above created a matrix with three independent columns and their linear combinations could fill all of three-dimensional space (they spanned $\mathbb{R}^{3}$).  This made the first matrix what we will call _invertible_ as opposed to the second one, $M$, which is not invertible.

Below, we create the three column vectors and check that $- \underline{u} + \left( - \underline{v} \right) = \underline{w}$.  A linear combination of columns (vectors) one and two, produces three.

```python
u = Matrix([[1], [-1], [0]])
v = Matrix([[0], [1], [-1]])
w = Matrix([[-1], [0], [1]])
u, v, w
```

```python
-u
```

```python
-v
```

```python
-u - v
```

```python
w
```

```python
-u - v == w
```

    True

## Example problems

### Example problem 1

+ Suppose $A$ is a matrix with the following solution
$$ {A}{\underline{x}}=\begin{bmatrix}1\\4\\1\\1\end{bmatrix} \\ \underline{x}=\begin{bmatrix}0\\1\\1\end{bmatrix}+{c}\begin{bmatrix}0\\2\\1\end{bmatrix} $$
+ What can you say about the columns of $A$?

#### Solution

```python
c = symbols('c')
x_vect = Matrix([[0], [1 + 2 * c], [1 + c]])
b = Matrix([[1], [4], [1], [1]])
```

```python
x_vect
```

```python
b
```

Note the following:
+ $\underline{x}$ is of size $m \times n$ and that is $ 3 \times 1 $
+ $\underline{b}$ is of size $ 4 \times 1 $
+ Therefor, $A$ must be of size $ 4 \times 3 $ and each column vector in $A$ is in $\mathbb{R}^{4}$

Let's call these columns of $A$ $C_1, C_2, C_3$, as illustrated in (12).

$$ \begin{bmatrix} \vdots  & \vdots  & \vdots  \\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\ \vdots  & \vdots  & \vdots  \\ \vdots  & \vdots  & \vdots  \end{bmatrix} \tag{12} $$

With the particular way in which $\underline{x}$ was written, we can say that we have a particular solution and a special solution, denoted as $ {A}\left({x}_{p}+{c}\cdot{x}_{s}\right)= \underline{b} $.

For $ c = 0 $ we have:
$$ {A}{\underline{x}}_{p}=b $$

For $c = 1 $ we have:
$$ \begin{align} A{ x }_{ p }+A{ \underline{x} }_{ s } &= b \\ \because \quad A{ \underline{x} }_{ p } &= b \\ b+A{ \underline{x} }_{ s } &= b \\ \therefore \quad A{ \underline{x} }_{ s } &= 0 \end{align} $$

We also have that the following
$$ { \underline{x} }_{ p }=\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix},\quad { \underline{x} }_{ s }=\begin{bmatrix} 0 \\ 2 \\ 1 \end{bmatrix} $$

For $ \underline{x}_{p}$ we have the following:
$$ \begin{bmatrix} \vdots  & \vdots  & \vdots  \\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\ \vdots  & \vdots  & \vdots  \\ \vdots  & \vdots  & \vdots  \end{bmatrix}\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = \underline{b} \quad \Rightarrow \quad { C }_{ 2 }+{ C }_{ 3 }= \underline{b} $$

For $ \underline{x}_{s}$ we have the following:
$$ \begin{bmatrix} \vdots  & \vdots  & \vdots  \\ { C }_{ 1 } & { C }_{ 2 } & { C }_{ 3 } \\ \vdots  & \vdots  & \vdots  \\ \vdots  & \vdots  & \vdots  \end{bmatrix}\begin{bmatrix} 0 \\ 2 \\ 1 \end{bmatrix}=\underline { 0 } \quad \Rightarrow \quad 2{ C }_{ 2 }+{ C }_{ 3 } = \underline{0} $$

Solving for $C_2$ and $C_3$ we have the following:
$$ \begin{align} {C}_{3} &= -2{C}_{2} \\ {C}_{2}-2{C}_{2} &= b \\ {C}_{2} &= -b \\ {C}_{3} &= 2b \end{align} $$

As for the first column of $A$, we need to know more about ranks and subspaces.  We see, though, that columns two and three are already constant multiples of each other.  So, as long as column one is not a constant multiple of $\underline{b}$, we are safe.

$$ A=\begin{bmatrix} \vdots  & 1 & 2 \\ { C }_{ 1 } & 4 & 8 \\ \vdots  & 1 & 2 \\ \vdots  & 1 & 2 \end{bmatrix} $$

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_1(part-2)_Geometric_of_Linear_Equations.ipynb