# Spread of an infectious disease: SEIR model analysis with Emukit

This notebook uses the SEIR-model and Emukit to answer several application driven research questions.

## Overview

```python
# General imports
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import colors as mcolors

# Figure config
colors = dict(mcolors.BASE_COLORS, **mcolors.CSS4_COLORS)
LEGEND_SIZE = 15
TITLE_SIZE = 25
AXIS_SIZE = 15
FIGURE_SIZE = (12, 8)

# for reproducibility
np.random.seed(42)
```

## The SEIR epidemic model

The [SEIR model](https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology#The_SEIR_model) is a compartmental model of biology that simulates the spread of an infectious disease.
A population of $N$ individuals is divided into the compartments: **S**: susceptible, **E**: exposed, **I**: infected, **R**: recovered with $N = N_S + N_E + N_I + N_R$.

The individuals in compartment **S** (susceptible) can contract an infection and transition to compartment **E** (exposed).
The compartment 'exposed' is introduced to model incubation times; exposed objects are already infected, but are not infectious themselves yet. Once they become infectious they transition to compartment **I** (infected) and can infect susceptible individuals in **S** (susceptible).
Once the disease is over the individuals in **I** (infected) transition to **R** (recovered) and becomes immune to the infection.

In this notebook we are going to use a simulation of the SEIR model in order to answer a few questions about behavior of the SEIR model, defined as [Tasks](#tasks) below.
For that we need to be able to evaluate the SEIR model for a given set of inputs. For that we are going to use *Gillespie* algorithm which draws exact samples from the SEIR model.

#### The Gillespie simulation
The dynamics of the SEIR model are determined by stochastic discrete-time events of individuals changing infection state, for which Poisson processes (i.e., exponentially distributed inter-event times) are commonly assumed.
The [Gillespie algorithm](https://en.wikipedia.org/wiki/Gillespie_algorithm) draws correct sample-trajectories from the simulation, hence, to query the SEIR model, many runs are required which can then be averaged to construct an estimator.

The rate of infection and the rate of recovery are denoted as constants $a$ and $b$ respectively. The incubation time is $c^{-1}$.
To simplify matters in this illustrative notebook, we consider the case that we know the rate of recovery $b$ once an individual has been infected and the incubation time $c^{-1}$, but that we are uncertain about the infection rate $a$. We also assume that we know the initial state of the simulation which is one infected individual only and no recovered or exposed individuals. This means all remaining individuals are susceptible.

For a given infection rate $a$, the Gillespie simulation draws many (1000s) of Gillespie repetitions of the SEIR model and returns for example the time or location of the maximum of each sample (see [Tasks](#tasks) below). The expected value of the relevant quantity is computed by averaging over these samples. The many repetitions of the simulation is what makes it expensive.

**Note on notation:** To simplify notation we normalize with respect to population size $x_i = \frac{N_I}{N}$ for $i \in \{S, I, E, R\}$ and express time in units of recovery time $b^{-1}$, $t \rightarrow b t$. Furthermore, we call $\alpha:=\frac{a}{b}$ and $\beta := \frac{c}{b}$. Thus $\alpha$ is the ratio of infection and recovery rate (or time) and $\beta$ the ratio of incubation rate and recovery rate (or time).

# Tasks

The three Tasks below illustrate practical questions that can be answered by combing the SEIR model with the inference methods in Emukit.
The answers to these tasks might lead to practical decisions such as how much and when to order vaccines, or labor plans for hospital capacity.
From the point of Emukit, the SEIR model identifies several Emukit user-functions which we define below.

**A side note on wording:** Note that we cannot answer the Tasks precisely since the SEIR model is inherently a stochastic simulation, meaning that each query of the Gillespie simulation which averages over many runs is already an expectation itself. This means that e.g., for an unknown infection rate we are concerned with an expectation of an expectation. To simplify wording we will ignore the first expectation and will call e.g., the 'expected height of peak for a given infection rate with respect to the Gillespie distribution' simply the 'height of the peak' and use 'expectation' only for the expected values with respect to the distribution of the unknown infection rate $a$. The results have to be interpreted accordingly.

### 1. How many individuals will maximally be infected simultaneously (height of peak)?

Since the infection rate $a$ is unknown, we can not answer this question precisely
But we can compute an estimator, i.e., the expected maximum number of individuals which are infected at the same time $\mathbb{E}_a\left[\max_t N_I(t, \alpha)\right]$. This involves solving an integral numerically. Assuming running the Gillespie simulation is expensive, we will use the [Bayesian quadrature](Emukit-tutorial-Bayesian-quadrature-introduction.ipynb) component of Emukit since Bayesian quadrature is particularly well suited for low-dimensional expensive integrals. Bayesian quadrature also naturally returns an uncertainty estimate of the integral value that can be used to assess if we need to run further Gillespie simulations.

### 2. When will the infection be at its peak (time-occurrence of peak)?

This problem is similar to the one above, but this time we are interested in when in time the peak of infections occurs, i.e., $\mathbb{E}_a\left[\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)\right]$. Again we can use the Bayesian quadrature component of Emukit since we need to solve an integral. The difference to Task 1 is that now the Bayesian quadrature algorithm collects the most informative Gillespie runs that correspond to estimating $\mathbb{E}_a\left[\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)\right]$ rather than $\mathbb{E}_a\left[\max_t N_I(t, \alpha)\right]$.

### 3. How does the height of the peak relate to the infection rate?

Apart from knowing how high the peak of infection will be and when it will occur, we might want to know how these estimates change with infection rate, hence we might want to build an emulator which is learned by the [experimental design]() package in Emukit. We can query or use this emulator if we want to predict the height of the peak for a new infection rate at a later time at which we cannot run a Gillespie simulation for whatever reason. We might for example learn at a later point that the infection rate is indeed 5.7 time units but we have not enough time to run the Gillespie simulation again for this value. The experimental design emulator not only predicts the peak height but also an uncertainty estimate of the prediction which we can base our decision on. Again, the difference to Tasks 1 and 2 is that now the Gillespie model will be queried with infection rates which leads the emulator to learn the relation to the height of the peak efficiently.

### 4. Beyond this notebook
There can be many more questions asked. With a [multi-fidelity model](Emukit-tutorial-multi-fidelity.ipynb) one can relate the high-fidelity Gillespie results to cheaper but approximate simulations. Such an approximate model is called a low-fidelity model. An example of a low-fidelity SEIR model is a system of ordinary differential equations (ODEs) that describe the average dynamics of the spread of the disease in the thermodynamic limit of a very large population size $N$.
The ODE system, listed here for completeness, does, however, not admit a generic analytic solution.

\begin{equation}
\begin{aligned}
\frac{d\,x_S}{d\,t} &= -\alpha x_I x_S,\quad
\frac{d\,x_E}{d\,t} = a x_I x_S - \beta x_E,\quad
\frac{d\,x_I}{d\,t} = \beta x_E  - \,x_I,\quad
\frac{d\,x_R}{d\,t} = x_I.
\end{aligned}
\end{equation}

Solving the ODE can be seen as querying a low-fidelity model of the high-fidelity Gillespie simulation which is less precise but therefore much faster. Another even coarse low-fidelity model might be a simplified version in which e.g., the effect of the incubation time is neglected.

If our disease model also included the effect of e.g., vaccines or treatment in a certain area (the SEIR model presented here does not), with [Bayesian optimization](Emukit-tutorial-Bayesian-optimization-introduction.ipynb) we could find the optimal allocation of medical personal in a region.

The emulator usually also provides gradients of the learned relation (e.g., between infection rate and expected height of peak) which can be used for model-based [sensitivity analysis](Emukit-tutorial-sensitivity-montecarlo.ipynb).

In this notebook we will attempt to answer Tasks 1, 2 and 3.

### Navigation

1. [Task 1: Height of peak of infection](#section-task1)

2. [Task 2: Time-occurrence of peak of infection](#section-task2)

3. [Task 3: Infection rate and peak height relation](#section-task3)

4. [Discussion and conclusion](#conclusion)

## Load Emukit packages

We use the Emukit  quadrature package for Tasks 1 and 2, and the Emukit experimental design package for Task 3. For common functionality we use Emukit core. For the emulator model we use GPy since GPy model wrappers already exist in Emukit.

```python
# SEIR model and Gillespie simulation
import Emukit_task_seir_model as seir

# GPy for the model
import GPy

# Emukit model wrapper
from emukit.model_wrappers.gpy_quadrature_wrappers import RBFGPy, BaseGaussianProcessGPy
from emukit.model_wrappers import GPyModelWrapper

# Emukit core
from emukit.core import ParameterSpace, ContinuousParameter

# Emukit quadrature
from emukit.quadrature.kernels.integral_bounds import IntegralBounds
from emukit.quadrature.kernels import QuadratureRBF
from emukit.quadrature.methods import VanillaBayesianQuadrature
from emukit.quadrature.loop import VanillaBayesianQuadratureLoop

# Emukit experimental design
from emukit.experimental_design.model_based import ExperimentalDesignLoop
```

## Load SEIR model

We fix the incubation rate to $\beta = 4$, i.e. recovery takes 4 times longer than incubation. This will usually result in a simulated breakout of the disease.

```python
population_size = 100 # size of population N
alpha_init = 10. # infection rate/recovery rate
beta = 4. # incubation rate/recovery rate

t_end = 10. # end time of simulation
t_eval = np.linspace(0., t_end, 200) # fixed time steps for simulation
num_gil = 1000 # number of gillespie runs, 1000 will give roughly noisfree observations
```

```python
# Set up the gillespie model
seir_model = seir.SEIR(population_size, alpha_init, beta)
seir_gil = seir.SEIRGillespie(seir_model)

# Prior over alpha. This prior will be integrated against in Task 1 and 2
alpha_prior = seir.GammaPrior(5., 1., 5.)

# computes expected time occurance and height of the infection peak for a given alpha.
meanmax = seir.MeanMaxInfectionGillespie(seir_gil, num_gil, t_end, alpha_prior)
```

## Task 1: Height of peak of infection

The height of the peak is the expected maximal number of simultaneously infected individuals

$$
\mathbb{E}_a\left[\max_t N_I(t, \alpha)\right] =\int_{\Omega}\max_t N_I(t, \alpha)p(\alpha)\mathrm{d}\alpha.
$$

We use the Bayesian quadrature package of Emukit to emulate the integrand $\max_tN_I(t, \alpha)p(\alpha)$.

#### Define user function $\max_t N_I(t, \alpha)p(\alpha)$ and integral bounds

We call the integrand $\max_t N_I(t, \alpha)p(\alpha)$ `f_height_of_peak_weighted`  since it computes the height of the infection peak weighted with the prior on $\alpha$. One call of `f_height_of_peak_weighted` will run `num_gil` Gillespie runs. The integral bounds have units $\alpha$, i.e., units of infection time divided by recovery time; they define the domain $\Omega = [lower\_bounds, upper\_bound]$. We choose `80` to be the upper bound since we assume that the infection rate will not be larger than 80 times the recovery rate.

```python
# define the integrand
f_height_of_peak_weighted = seir.height_of_peak_weighted(meanmax)

# define integral bounds
lb = 1. # lower integral bound
ub = 80. # upper integral bound
```

#### Get initial observations

The quadrature model needs a few random starting locations to get an initial calibration of its hyperparameters.

```python
# number of initial points
N_init_task1 = 3

# inital traning points: locations X_init sampled uniformly from [lower_bound, upper_bound].
X_init_task1 = np.random.rand(N_init_task1, 1)*(ub-lb) + lb
Y_init_task1 = f_height_of_peak_weighted(X_init_task1)
```

#### Create Emukit surrogate model (the emulator) for the integrand function

We use the GPy package to build the model and the GPy model wrapper which is already in Emukit. Then we choose the vanilla Bayesian quadrature method and create the loop. We then run the loop for a certain number of iterations. Finally we integrate the model and collect the estimated expectation of the height of the infection peak and its variance.

```python
# make your GP model in GPy.
gpy_model_task1 = GPy.models.GPRegression(X=X_init_task1, Y=Y_init_task1,
                        kernel=GPy.kern.RBF(input_dim=X_init_task1.shape[1],
                                            lengthscale=10.0, variance=1.0))

# create emukit quadrature GP model
emukit_qrbf = QuadratureRBF(rbf_kernel=RBFGPy(gpy_model_task1.kern),
                            integral_bounds=[(lb, ub)])
emukit_model = BaseGaussianProcessGPy(kern=emukit_qrbf, gpy_model=gpy_model_task1)
```

```python
# choose vanilla Bayesian quadrature method and make loop
emukit_method = VanillaBayesianQuadrature(base_gp=emukit_model)
emukit_loop = VanillaBayesianQuadratureLoop(model=emukit_method)
```

```python
# run the loop for a number of iterations and collect result
num_iter = 5
emukit_loop.run_loop(user_function=f_height_of_peak_weighted, stopping_condition=num_iter)
height_of_peak_estimate, height_of_peak_variance = emukit_loop.model.integrate()
```

#### Print and plot result

We plot the integrand, i.e, the weighted height of the infection peak vs $\alpha$ (the ratio of infection rate and recovery rate). We also plot the distribution of the expected height of the infection peak which represents our (un-)certainty over its value.

```python
x_plot = np.linspace(lb, ub, 100)[:, None]
mu_plot, var_plot = emukit_loop.model.predict(x_plot)

X_plot = emukit_loop.loop_state.X
Y_plot = emukit_loop.loop_state.Y

plt.figure(figsize=FIGURE_SIZE)
plt.plot(X_plot, Y_plot, "ro", markersize=10, label="Observations")
plt.plot(X_init_task1, Y_init_task1, "bo", markersize=10, label="Initial observations")
plt.plot(x_plot, mu_plot, "C0", label="Model")
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - np.sqrt(var_plot)[:, 0], color="C0", alpha=0.6)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 2 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 2 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.4)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 3 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 3 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.2)
plt.legend(loc=1, prop={'size': LEGEND_SIZE})
plt.xlabel(r"ratio of infection rate and recovery rate")
plt.ylabel(r"weighted height of infection peak")
plt.grid(True)
plt.xlim(lb, ub)
plt.show()
```

```python
initial_integral_mean = height_of_peak_estimate
initial_integral_variance = height_of_peak_variance

x_plot_integral = np.linspace(initial_integral_mean-5*np.sqrt(initial_integral_variance),
                              initial_integral_mean+5*np.sqrt(initial_integral_variance), 200)
y_plot_integral_initial = 1/np.sqrt(initial_integral_variance * 2 * np.pi) * \
np.exp( - (x_plot_integral - initial_integral_mean)**2 / (2 * initial_integral_variance) )
plt.figure(figsize=FIGURE_SIZE)
plt.plot(x_plot_integral, y_plot_integral_initial, "k", label="integral density")
plt.axvline(initial_integral_mean, color="red", label="integral estimate", \
            linestyle="--")
plt.legend(loc=2, prop={'size': LEGEND_SIZE})
plt.xlabel(r"expected height of infection peak in number of individuals")
plt.ylabel(r"pdf of expected height of infection peak in number of individuals")
plt.grid(True)
plt.xlim(np.min(x_plot_integral), np.max(x_plot_integral))
plt.show()
```

```python
print('=== TASK 1: Result ===')
print()
print('The estimated expected height of the infection peak is: ', round(height_of_peak_estimate), 'individuals,')
print('with a credible interval: ', int(np.ceil(2*np.sqrt(height_of_peak_variance))), 'individuals.')
```

    === TASK 1: Result ===

    The estimated expected height of the infection peak is:  62 individuals,
    with a credible interval:  5 individuals.

## Task 2: Time-occurrence of peak of infection

The time of the peak is the expected time-occurrence when the number of simultaneously infected individuals is maximal.

$$
\mathbb{E}_a\left[\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)\right] =\int_{\Omega}\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)p(\alpha)\mathrm{d}\alpha.
$$

We use the Bayesian quadrature package of Emukit to emulate the integrand $\mathrm{arg}\,\mathrm{max}_tN_I(t, \alpha)p(\alpha)$.

#### Define user function $\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)p(\alpha)$ and integral bounds

We call the integrand $\mathrm{arg}\,\mathrm{max}_t N_I(t, \alpha)p(\alpha)$ `f_time_of_peak_weighted`  since it computes the time occurrence of the infection peak weighted with the prior on $\alpha$. One call of `f_time_of_peak_weighted` will run `num_gil` Gillespie runs. The integral bounds are the same as in Task 1.

```python
# define the integrand
f_time_of_peak_weighted = seir.time_of_peak_weighted(meanmax)
```

```python
# number of initial points
N_init_task2 = 3

# inital traning points: locations X_init sampled uniformly from [lower_bound, upper_bound].
X_init_task2 = np.random.rand(N_init_task2, 1)*(ub-lb) + lb
Y_init_task2 = f_time_of_peak_weighted(X_init_task2)
```

```python
# make your GP model in GPy.
gpy_model_task2 = GPy.models.GPRegression(X=X_init_task2, Y=Y_init_task2,
                        kernel=GPy.kern.RBF(input_dim=X_init_task2.shape[1],
                                            lengthscale=10.0, variance=1.0))
# create emukit quadrature GP model
emukit_qrbf = QuadratureRBF(rbf_kernel=RBFGPy(gpy_model_task2.kern),
                            integral_bounds=[(lb, ub)])
emukit_model = BaseGaussianProcessGPy(kern=emukit_qrbf, gpy_model=gpy_model_task2)
```

```python
# choose vanilla Bayesian quadrature method and make loop
emukit_method = VanillaBayesianQuadrature(base_gp=emukit_model)
emukit_loop = VanillaBayesianQuadratureLoop(model=emukit_method)
```

```python
# run the loop for a number of iterations and collect result
num_iter = 5
emukit_loop.run_loop(user_function=f_time_of_peak_weighted, stopping_condition=num_iter)
time_of_peak_estimate, time_of_peak_variance = emukit_loop.model.integrate()
```

#### Print and plot result

We plot the integrand, i.e, the time of the infection peak vs $\alpha$ (the ratio of infection rate and recovery rate). We also plot the distribution of the expected time of the infection peak which represent our (un-)certainty over its value.

```python
x_plot = np.linspace(lb, ub, 100)[:, None]
mu_plot, var_plot = emukit_loop.model.predict(x_plot)

X_plot = emukit_loop.loop_state.X
Y_plot = emukit_loop.loop_state.Y

plt.figure(figsize=(12, 8))
plt.plot(X_plot, Y_plot, "ro", markersize=10, label="Observations")
plt.plot(X_init_task2, Y_init_task2, "bo", markersize=10, label="Initial observations")
plt.plot(x_plot, mu_plot, "C0", label="Model")
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - np.sqrt(var_plot)[:, 0], color="C0", alpha=0.6)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 2 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 2 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.4)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 3 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 3 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.2)
plt.legend(loc=1, prop={'size': LEGEND_SIZE})
plt.xlabel(r"ratio of infection rate and recovery rate")
plt.ylabel(r"weighted time-occurance of infection peak")
plt.grid(True)
plt.xlim(lb, ub)
plt.show()
```

```python
initial_integral_mean = time_of_peak_estimate
initial_integral_variance = time_of_peak_variance

x_plot_integral = np.linspace(initial_integral_mean-5*np.sqrt(initial_integral_variance),
                              initial_integral_mean+5*np.sqrt(initial_integral_variance), 200)
y_plot_integral_initial = 1/np.sqrt(initial_integral_variance * 2 * np.pi) * \
np.exp( - (x_plot_integral - initial_integral_mean)**2 / (2 * initial_integral_variance) )
plt.figure(figsize=FIGURE_SIZE)
plt.plot(x_plot_integral, y_plot_integral_initial, "k", label="integral density")
plt.axvline(initial_integral_mean, color="red", label="integral estimate", \
            linestyle="--")
plt.legend(loc=2, prop={'size': LEGEND_SIZE})
plt.xlabel(r"expected time-occurance of infection peak")
plt.ylabel(r"pdf of expected time-occurance of infection peak in time units")
plt.grid(True)
plt.xlim(np.min(x_plot_integral), np.max(x_plot_integral))
plt.show()
```

```python
print('=== TASK 2: Result ===')
print()
print('The estimated expected time of the infection peak is: ', round(time_of_peak_estimate, 2), 'time units.')
print('with a credible interval: ', round(2*np.sqrt(time_of_peak_variance), 2), 'time units.')
```

    === TASK 2: Result ===

    The estimated expected time of the infection peak is:  1.05 time units.
    with a credible interval:  0.1 time units.

## Task 3: Infection rate and peak-height relation

We want to predict how the height of the infection peak varies with the infection rate, even if we did not perform Gillespie runs for a particular $\alpha$. For this we use the experimental design package of Emukit and model the relation between infection rate and peak height explicitly with a Gaussian process.

#### Define user function $\max_t N_I(t, \alpha)$

We call the relation $\max_t N_I(t, \alpha)p(\alpha)$ `f_height_of_peak`  since it computes the height of the infection, this time unweighted. One call of `f_height_of_peak` will run `num_gil` Gillespie runs.

```python
# define the user function
f_height_of_peak = seir.height_of_peak(meanmax)

# this is the infection rate ratio we are intereted in at a later point. We can define it here
# already since this is a tutorial. It will not be used by Emukit to train the model.
alpha_unobserved = np.array([[5.]])
```

```python
# number of initial points
N_init_task3 = 3

# inital traning points: locations X_init sampled uniformly from [lower_bound, upper_bound].
X_init_task3 = np.random.rand(N_init_task3, 1)*(ub-lb) + lb
Y_init_task3 = f_height_of_peak(X_init_task3)
```

```python
# make your GP model in GPy.
gpy_model_task3 = GPy.models.GPRegression(X=X_init_task3, Y=Y_init_task3,
                        kernel=GPy.kern.RBF(input_dim=X_init_task1.shape[1],
                                            lengthscale=10.0, variance=1.0))
gpy_model_task3.Gaussian_noise.constrain_fixed(1.e-10)

# create emukit GP model
model_emukit = GPyModelWrapper(gpy_model=gpy_model_task3)
```

```python
# Create the parameters space
parameter_space_emukit = ParameterSpace([ContinuousParameter('alpha', lb, ub)])
```

```python
# Create the experimental design object
loop_emukit = ExperimentalDesignLoop(space=parameter_space_emukit, model=model_emukit)
```

```python
# run the loop for a number of iterations and collect predictions
num_iter = 5
loop_emukit.run_loop(user_function=f_height_of_peak, stopping_condition=num_iter)
predicted_height_of_peak_mean, predicted_height_of_peak_variance \
                                            = loop_emukit.model.predict(alpha_unobserved)
```

#### Print and plot result

We plot the emulator which models the relation between the height of the infection peak and $\alpha$ (the ratio of infection rate and recovery rate). We see that we can make predictions at points where we did not run the Gillesspie algorithms. The emulator also returns and uncertainty estimate for its prediction at that location.

```python
x_plot = np.linspace(lb, ub, 100)[:, None]
mu_plot, var_plot = loop_emukit.model.predict(x_plot)

X_plot = loop_emukit.loop_state.X
Y_plot = loop_emukit.loop_state.Y

plt.figure(figsize=FIGURE_SIZE)
plt.plot(X_plot, Y_plot, "ro", markersize=10, label="Observations")
plt.plot(X_init_task3, Y_init_task3, "bo", markersize=10, label="Initial observations")
plt.plot(x_plot, mu_plot, "C0", label="Model")
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - np.sqrt(var_plot)[:, 0], color="C0", alpha=0.6)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 2 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 2 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.4)
plt.fill_between(x_plot[:, 0],
                 mu_plot[:, 0] + 3 * np.sqrt(var_plot)[:, 0],
                 mu_plot[:, 0] - 3 * np.sqrt(var_plot)[:, 0], color="C0", alpha=0.2)
plt.axvline(alpha_unobserved[0, 0], color="red", label="location for prediction", \
            linestyle="--")
plt.legend(loc=4, prop={'size': LEGEND_SIZE})
plt.xlabel(r"ratio of infection rate and recovery rate")
plt.ylabel(r"height of infection peak in number of individuals")
plt.grid(True)
plt.xlim(lb, ub)
plt.show()
```

```python
print()
print('=== TASK 3 Result ===')
print('The estimated height of the infection peak at the unobserved gillespie run with alpha =',
      round(alpha_unobserved[0, 0]),' is: ', int(round(predicted_height_of_peak_mean[0, 0])),
      'individuals, with a credible interval: ', int(np.ceil(2*np.sqrt(predicted_height_of_peak_variance[0, 0]))), 'individuals.')

```

    === TASK 3 Result ===
    The estimated height of the infection peak at the unobserved gillespie run with alpha = 5.0  is:  26 individuals, with a credible interval:  4 individuals.

## Discussion and conclusion

We solved all three tasks by only running a small number of simulations each. In comparison, for example, computing the ground truth integrals of Tasks 1 and 2 requires approximately one hour on a laptop even though the integration problems are only one-dimensional. This notebook runs in just a few minutes.

We also observe that the GP models assign a certain probability to negative values of quantities which we know to be positive, e.g., the number of individuals which are infected. Thus more elaborate models compatible with Emukit such as warped GPs might be even more suitable to the tasks presented here. Still, all methods used here yield a reliable answers for all our three tasks which are:

```python
print('=== TASK 1 ===')
print('The estimated expected height of the infection peak is: ', round(height_of_peak_estimate), 'individuals,')
print('with a credible interval: ', np.ceil(2*np.sqrt(height_of_peak_variance)), 'individuals.')

print()
print('=== TASK 2 ===')
print('The estimated expected time of the infection peak is: ', round(time_of_peak_estimate, 2), 'time units.')
print('with a credible interval: ', round(2*np.sqrt(time_of_peak_variance), 2), 'time units.')

print()
print('=== TASK 3 ===')
print('The estimated height of the infection peak at the unobserved gillespie run with alpha =',
      round(alpha_unobserved[0, 0]),' is: ', int(round(predicted_height_of_peak_mean[0, 0])),
      'individuals, with a credible interval: ', int(np.ceil(2*np.sqrt(predicted_height_of_peak_variance[0, 0]))), 'individuals.')

```

    === TASK 1 ===
    The estimated expected height of the infection peak is:  62 individuals,
    with a credible interval:  5.0 individuals.

    === TASK 2 ===
    The estimated expected time of the infection peak is:  1.05 time units.
    with a credible interval:  0.1 time units.

    === TASK 3 ===
    The estimated height of the infection peak at the unobserved gillespie run with alpha = 5.0  is:  26 individuals, with a credible interval:  4 individuals.


```python
from IPython.display import Image

import numpy as np
from scipy.stats import binom
import matplotlib.pyplot as plt
import itertools

```

### Probability Models

* A way to reason about a probabilistic system from various perspectives.

* E.g., you want to run an ad campaign through an agency. You may be interested in:
    * What is the average number of new clients we should expect to see after the ad campaign
    * Determine the amount of variability you should expect to see in the system
      * How variable is that number? Ex. a value 300-400 is more informative than 100-1,000
    * Compute probability of events
      * What is the probability of having more than X new clients after a given ad campaign

Note and remember these three components:
  * Some summary information about the experiment
  * Variability (variance) around that statistic
  * Computing probability of certain events

```python
### Probability Models: Example

* E.g., you want to run an ad campaign through an agency and are interested in:
    1. Average number of new clients after an ad campaign
    2. Variability around that number
    3. The probability of having at least X participants.

* We can repeat the ad campaign a large number of times and answer each of these questions

```python
    nb_new_participants = [500, 800, 860, 820, 840, 760, 430,...]
```

```

### Probability Models: Example

```python
    nb_new_participants = [500, 800, 860, 820, 840, 760, 430,...]
```

1. Average number of new clients after an ad campaign
```python
np.mean(nb_new_participants)
```
2. Variability around that number
```python
np.var(nb_new_participants)
```
3. The probability of having at least X participants.
```python
    sum(array(nb_new_participants)> X) / len(nb_new_participants)
```

### Probability Models: Example - Cont'd

* The same information is contained in the visualization of the data
  * What does every single bar mean?
  * How can you get an idea of the `mean`, `variance`, and probability of at least X participants?

### Probability Distributions

* At the heart of probability models are _probability_ _distributions_ and _random_ _variables_

* Consider the loaded die with the following probabilities:
```
    p(1) = 0.3/6
    p(2) = 0.7/6
    p(3) =   2/6
    p(4) = 0.5/6
    p(5) = 0.2/6
    p(6) = 2.3/6
```

* We could think of outcomes above as the values returned by the function that takes as input a value in `(1,2,3,4,5,6)` and returns the relavant probability from the list `[0.3/6, 0.7/6, 2/6, 0.5/6, 0.2/6, 2.3/6]`

* Example:
```python
    def loaded_die_distribution (face):
    try:
        dist = {1:0.3/6, 2:0.7/6, 3:2/6, 4:0.5/6, 5:0.2/6, 6:2.3/6}
        return dist[face]
    except:
        raise("%s is not a valid die face %d" % face)
```

### Probability Distributions - Cont'd

* Such a function is called a probability distribution
* In essence, a probability distribution is a numerical function that assigns probabilities (y-axis in [0-1]) to events in the sample space S (x-axis)
  * Set of sample space values as _domain_ (numerical or not)

```python
def loaded_die_distribution (face):
    try:
        dist = {1:0.3/6, 2:0.7/6, 3:2/6, 4:0.5/6, 5:0.2/6, 6:2.3/6}
        return dist[face]
    except:
        raise("%s is not a valid die face %d" % face)
```

```python
sample_space = list(range(1,7))
p_x = [loaded_die_distribution (f) for f in sample_space]

plt.figure(figsize=(10, 4))
plt.bar(sample_space, p_x)

plt.xlabel('Die face obtained')
plt.ylabel('probability')
```

### Sampling from the Biased Die

- Given our probability distribution, we are four times more likely to draw a 3 than we are to draw a 4
- To sample die rolls, we need to encode our probability distribution to reflect the bias in our die
  - This can be easily done using the `choice()` function in `np.random`

```python
np.random.choice([1, 2, 3, 4, 5, 6], p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.2/6, 2.3/6])
```

```python
import random
for _ in range(20):
    print (np.random.choice([1, 2, 3, 4, 5, 6],
                            p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.2/6, 2.3/6]
                           ),
           end= ' ')

```

    6 6 6 6 4 6 2 6 3 4 6 2 2 6 6 5 6 3 6 3

### Random Variables

* Using CS thinking, a random variable is merely a variable (a RAM space, or box for a value) that can take values that are variable (random)
  * In probabilistic thinking, a random variable is a variable that can take any value from the sample space of the experiment as determined by the probability distribution.

  * I.e., the value assigned to the variable depends on a random process
  ```python
      # randomly select a value according to a probability distribution
      dieRollVal = np.random.choice([1, 2, 3, 4, 5, 6], p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.2/6, 2.3/6]);
  ```

### Random Variables- Cont'd

* In a non-probabilistic program, you can trace the value contained in a variable

* In a probabilistic program, you cannot since the value is determined probabilistically
  * We don't know what that variable will contain prior to executing the code
  * We can however reason about what the value will most likely contain

* Take for instance the variable `roll_random_var` in the code below:

```python
import random
for i in range(10):
    roll_random_var = np.random.choice([1, 2, 3, 4, 5, 6],
                                       p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.4/6, 2.1/6])
    print (f"roll_random_var contains the value {roll_random_var}")
```

* The variable `roll_random_var` is a _random_ _variable_
  * The values assigned to it are determined probabilistically using a probability distribution of the biased die

```python
for i in range(3):
    roll_random_var = np.random.choice([1, 2, 3, 4, 5, 6],
                                       p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.4/6, 2.1/6])
    print ("roll_random_var contains the value %s" % roll_random_var)

```

    roll_random_var contains the value 3
    roll_random_var contains the value 3
    roll_random_var contains the value 6

```python
from collections import Counter
roll_random_var_list = []
for i in range(5000):
    roll_random_var = np.random.choice([1, 2, 3, 4, 5, 6],
                                       p=[0.3/6, 0.7/6, 2/6, 0.5/6, 0.4/6, 2.1/6])
    roll_random_var_list.append(roll_random_var)

counts = Counter(roll_random_var_list)

sorted_counts = dict(sorted(counts.items()))
print(sorted_counts)
plt.figure(figsize=(12,4))
_ = plt.bar(sorted_counts.keys(), sorted_counts.values())

```

### Two types of random variables

  * Discrete (finite or countably infinite)
     * Take as input finite countably finite integer values and return probabilities for those events
     * Ex. Number of accidents on H1 on any given day, number of customers at lunchtime at Starbucks on campus, number of siblings a student at UH has, etc.

  * Continuous (uncountably infinite)
     * Take as input uncountably infinite float value and returns a probability density
     * Ex. Height, weight, time, etc..
       * The sample space is infinite.

### Two Types of Random Variables

* Discrete and continuous random variables have commonly occurring distributions
  * These distributions are simply functions -- just like the one encoded above -- that give the "probability" for each element in the sample space

* Perhaps two of the most popular distributions are the binomial and the normal (or Gaussian)

### The Binomial Distribution (The `"Hello World"` Probability Distribution)

* A binomial distribution is simply the probability of number of SUCCESS outcomes in an experiment that is repeated multiple times.
  * $m$ successes in an experiment of $n$ fixed, independent trials.

* Each trial has a probability of success $p$ and a probability of failure $1-p$ (heads and tails; 0 and 1; etc...)

####  Examples of the Binomial Distribution

- You flip a fair coin 100 times, what is the probability of observing 27 heads?

- Given an ad clickthrough probability of $p=0.6$ what is the probability of having 100 clicks if we show the add 1000 times?

- Given a disease prevalence of $p=0.05$, what is the probability of observing the disease in at least 100 individuals if we survey 5000 randomly selected individuals from the population?

- What are $p$, $n$ and  $m$ in each of these experiments?

### Simple Probability Distribution

* Let $X$ be the random variable representing the number of heads obtained from flipping a fair coin 5 times.

* What is the sample space of this experiment?

  - This amounts to computing the product of $[H,T]^5$, or $[H,T] \times [H,T] \times [H,T] \times [H,T] \times [H,T]$

```python
for p in itertools.product(['H', 'T'], repeat=2):
    print(p)

```

    ('H', 'H')
    ('H', 'T')
    ('T', 'H')
    ('T', 'T')

```python
product  = list(itertools.product(['H', 'T'], repeat=5))
print("Sample space is of size %s " % len(product))
print ("The sample space is:\n")
for prod in product:
    print("".join(prod), end=" ")

```

    Sample space is of size 32
    The sample space is:

    HHHHH HHHHT HHHTH HHHTT HHTHH HHTHT HHTTH HHTTT HTHHH HTHHT HTHTH HTHTT HTTHH HTTHT HTTTH HTTTT THHHH THHHT THHTH THHTT THTHH THTHT THTTH THTTT TTHHH TTHHT TTHTH TTHTT TTTHH TTTHT TTTTH TTTTT

### Manually Deriving the Probability Distribution

* We are interested in the probability distribution of obtaining `X` heads in 5 coin flips
  * _Binomial_ distribution with n=5 and $p=0.5$

- Our sample space is [0, 1, 2, 3, 4, 5]

- There are $2^5 = 32$ outcomes.

### Manually Deriving the Probability Distribution - Cont'd

  * There is 1 way of obtaining 0 heads: TTTTT

     $p(H=0) = 1/32$
  * There 5 ways of obtaining 1 head: HTTTT, THTTT, TTHTT, TTTHT, TTTTH.

     $p(H=1) = 5/32$
  * Similarly, there are 10 ways of obtaining 2 heads: HHTTT, HTHTT, HTTHT, HTTTH, …. TTTHH

     $p(H=2) = 10/32$
  * ...

### Manually Deriving the Probability Distribution - Cont'd

* In general, the number of ways we can obtain $m$ heads from $n$ flips is given by the binomial cofficient ${n}\choose{m}$ (read as n choose m), which is given by:

$$
{{n}\choose{m}} = \frac{n!}{m!(n-m)!}
$$

E.g.: the for $n=5$ and $m=2$, we have

\begin{equation}
\begin{split}
{{5}\choose{2}} & = \frac{5!}{2!(5-2)!} \\
                & = \frac{5\times4\times3\times2 \times 1}{2 \times 1 \times 3 \times 2 \times 1} \\
                & = \frac{5 \times 4}{2 \times 1} \\
                & = 10
\end{split}
\end{equation}

```python
sample_space = [0, 1, 2, 3, 4, 5 ]
p_x = [1.0/32, 5.0/32, 10.0/32, 10.0/32, 5.0/32, 1.0/32]

plt.figure(figsize=(10, 4))
plt.bar(sample_space, p_x ,  color = "#3498db", )
#ax.set(xlabel='number of heads in 5 coin flips', ylabel='probability')
```

#### Binomial with Biased Coin

- What if the coin is biased? Example $p(H) = 0.7$ and $p(T) = 0.3$.

- What is the probability of obtaining 1 head in 5 coin flips with this biased coin?

#### Bernouilli Distribution

- Recall from elementary statistics and discrete math that the Bernouilli distribution, which states that:

$$
 f(k;p)=p^{k}(1-p)^{1-k}\quad {\text{for }}k\in \{0,1\}
$$

* $k=1$ is head,  $1- k=0$ is tail, and $p$ is the probability of head.

* Based on the Bernouilli euqaiton above, the probability of obtaining a head ($k=1$ ) is:
$$Pr(1; p) = p^1 \times (1-p)^0 = p$$

* and the probability of obtaining a tail ($k=0$ and $1-k = 1$) is:

$$Pr(1; p) = p^k \times (1-p)^{1-k} = p^0 \times (1-p)^1 = 1-p$$

* This matches our intuition about the probability of a coin landing head

#### Binomial with Biased Coin - Cont'd

- What if coin is biased? Example $p(H) = 0.7$ and $p(T) = 0.3$.

- What is the probability of obtaining 1 head in 5 coin flips with this biased coin?

$$
p(H=1) ={{5}\choose{1}}~~~p(H)^1 * p(T)^4
$$

* The five possible ways to ontain a 1 head and 4 tails:HTTTT, THTTT, TTHTT, TTTHT, TTTTH

$$
p(H=1) = 5  \times 0.7 \times 0.3^4 = 0.02835
$$

### The Binomial Distribution - Cont'd

* The equation above can be generalized to any number of trials $n$ any probability of heads $p(H)$

* We will write $p(H)$ as $p$ and $p(T)$ as $(1-p)$

    $p(H=k) ={{n}\choose{k}} p^k * (1-p)^{n-k}$

* This equation is the probability function for a random variable distributed according to a binomial distribution

  * Think of it as a convenient function to assign probabilities to any value in the sample space

#### The Binomial Distribution - Cont'd

- A probability distribution can be derived from first principles for many other `canonical` distributions

- These functions allow us to compute probabilities for events seamlessly
  - A by-product is that we can also derive expected values, probabilities for subspace, use them to sample from that distribution, etc...

#### The Binomial Distribution - Cont'd

- We can easily model (reason about) a problem that fits the binomial since we have access to the distribution

  * E.g., assume the probability of subscribing to a monthly plan in your app is  $p=0.05$.
  * If, on average, 10k users download your app each month, how many (on average) subscribed users will you have after 1 year?
  * What is the probability of having at least 3000 users
  * You can use this simple model to derive various valuable insights about your app
  * E.g. total revenue after one year, the number of servers you will need to support your user base, the number of customer service employees to support your clientele, etc.

* Once we know the parameter of the distribution (e.g., here p=0.05), it's easy to answer the questions above.
  * This may be trivial in some cases and challenging in others
  * How did we get the $p=0.05$? How sure are we of that value? How likely is it to be $p=0.06$ instead?

### Probability Distributions for Discrete Random Variables

- A probability distribution for a discrete random variable is called probability mass function (__pmf__)

  - `pmf`s take a discrete random variable $X$ and assign a probability to each value of its sample space

- A probability distribution can be derived from first principles for many other canonical distributions

- These functions allow us to compute probabilities for events seamlessly

  - A by-product is that we can also derive expected values, probabilities for subspace, use them to sample from that distribution, etc...

### `pmf` in Python

- Programming languages and data analysis software commonly support popular PMFs

- The common graphical representation for a `pmf` is a histogram
- The X-axis contains observed values
- The Y-axis shows the probability for  the corresponding X-values

```python
n = 5
p = 0.7
binom.pmf(1, n, p)

```

    0.02834999999999999

```python
x
```

    range(0, 5)

```python
x = range(n+1)
p_x =  [binom.pmf(val, n, p) for val in x]

print(sum(p_x))
plt.figure(figsize=(16,6))

plt.bar(x, p_x)
plt.xlabel("Nb Successes")
plt.ylabel("Probility")

```

#### Example
- Given a biased coin with that comes up heads with a probability ($p=0.4$)
- If you flip the coin $100$ times, what is the probability of observing 42 heads?

```python
from scipy.stats import binom
n = 100
p = 0.4
print(binom.pmf(42, n, p))
```

    0.0742071940343827

```python
n = 100
p = 0.4

x = range(n) # our x-axis

p_x =  [binom.pmf(val, n, p) for val in x]  # our y-axis

plt.figure(figsize=(16, 5))

plt.bar(x, p_x , color ="#3498db", alpha=0.4)

# Add the horizontal and vertical bars
x_val = 42
y_val = binom.pmf(42, n, p)

# print(y_val)

plt.axvline(x_val, color = 'r', alpha=0.4)

plt.axhline(y_val, color = 'r', alpha=0.4)

plt.xlabel('number of heads in 5 coin flips')
_ = plt.ylabel('probability')
```

```python
n = 100
p = 0.5
x = range(n)

p_x =  [binom.pmf(val, n, p) for val in x]
plt.figure(figsize=(16, 6))
plt.bar(x, p_x ,  color = "#3498db", )

plt.xlabel('number of heads in 5 coin flips')
_ = plt.ylabel('probability')
```

### Comparing Distributions

* From the two previous examples, it can be seen that the two distributions differ
  * For example, the probability of seeing  `x=60` is different  between the two distributions

* Binomial distributions parameterized with different values of $n$ and $p$ will produce different `pmf`s

Therefore, it is important to describe a PMF's parameters when describing it
  * Modifying these parameters results in a different probability distribution

### Summary: `X`, `n`, `p` and the `pmf`
- The number of observed successes is said to be a random variable $X$,  distributed as a binomial distribution.

- We write the above statment as:
$$
X \sim Binomial(n,p)
$$

- This simply means that:

  - The random variable $X$ takes on values in the sample space describes as a binomial distribution
    - The sample space here is countably finite and range from $0$ to $n$
  - The parameters of the binomial probability distributions are $n$, $p$


<form action="index.ipynb">
    <input type="submit" value="Return to Index" style="background-color: green; color: white; width: 150px; height: 35px; float: right"/>
</form>

# Banana Example
Author(s): Paul Miles | Date Created: June 18, 2018

This technical example constructs a non-Gaussian target distribution by twisting two first dimensions of Gaussian
distribution. The Jacobian of the transformation is 1, so it is easy to calculate the right probability regions for the banana and study different adaptive methods.

The basics of the model are as follows.
$$B_f(x_i, a, b) = [ax_1, \frac{x_2}{a}-b(a^2x_1^2 + a^2), x_3, ..., x_{npar}]$$
$$B_f^{-1}(x_i, a, b) = [x_1 /a, ax_2 + ab(x_1^2 + a^2), x_3, ..., x_{npar}]$$

We can construct a sum-of-squares function
$$B_{sos}(x_i, a, b, \mu_i, \lambda) = B_f^{-1}(x_i-\mu_i, a, b) \cdot \lambda \cdot B_f^{-1}(x_i - \mu_i, a, b)$$
where $\mu_i$ are the centers for our target distributions, $\lambda$ is the target precision, and the banana parameters are $a$ and $b$.  For this particular example we consider the problem where we have 2 unknown parameters with center $\mu_i = 0$, and we have a target correlation of $\rho = 0.9$ for the first two dimensions.  We subsequently define the target precision matrix $\lambda$
\begin{equation}
    \lambda = \sigma^{-1} = \begin{bmatrix}
    1 & \rho & 0 & \\
    \rho & 1 & \\
    0 & & 1 \\
    & & & \ddots \\
    & & & & 1
  \end{bmatrix}^{-1}
\end{equation}

Note, for a comparison between using MCMC algorithms in Matlab, Python, and R, see [here](https://github.com/prmiles/mcmc_banana_examples/blob/master/mcmc_banana_examples.pdf).

```python
# import required packages
import numpy as np
from pymcmcstat.MCMC import MCMC
from pymcmcstat.plotting.utilities import generate_ellipse
import matplotlib.pyplot as plt
import pymcmcstat
print(pymcmcstat.__version__)
```

    1.9.0

We create a class to initialize model parameters

```python
class Banana_Parameters:

    def __init__(self, rho=0.9, npar=12, a=1, b=1, mu=None):
        self.rho = rho
        self.a = a
        self.b = b
        self.sig = np.eye(npar)
        self.sig[0,1] = rho
        self.sig[1,0] = rho
        self.lam = np.linalg.inv(self.sig)
        self.npar = npar
        if mu is None:
            self.mu = np.zeros([npar, 1])

npar = 6 # number of model parameters
udobj = Banana_Parameters(npar=npar) # user defined object
```

# Initialize MCMC object
- initialize data structure
- assign model parameters
- define simulation options

```python
# Initialize MCMC object
mcstat = MCMC()
mcstat.data.add_data_set(np.zeros(1), np.zeros(1),
                         user_defined_object=udobj)
# Add model parameters
for ii in range(npar):
    mcstat.parameters.add_model_parameter(
        name=str('$x_{}$'.format(ii + 1)),
        theta0=0.0)
# Define options
mcstat.simulation_options.define_simulation_options(
    nsimu=4.0e3,
    qcov=np.eye(npar)*5,
    method='dram')
```

We define a series of functions to evaluation the sum-of-squares error

```python
# Define model object
def bananafunction(x, a, b):
    response = x
    response[:, 0] = a*x[:, 0]
    response[:, 1] = x[:, 1]*a**(-1) - b*((a*x[:, 0])**(2) + a**2)
    return response

def bananainverse(x, a, b):
    response = x
    response[0] = x[0]*a**(-1)
    response[1] = x[1]*a + a*b*(x[0]**2 + a**2)
    return response

def bananass(theta, data):
    udobj = data.user_defined_object[0]
    lam = udobj.lam
    mu = udobj.mu
    a = udobj.a
    b = udobj.b
    npar = udobj.npar
    x = np.array([theta])
    x = x.reshape(npar, 1)
    baninv = bananainverse(x - mu, a, b)
    stage1 = np.matmul(baninv.transpose(),lam)
    stage2 = np.matmul(stage1, baninv)

    return stage2

mcstat.model_settings.define_model_settings(sos_function=bananass)
```

# Run Simulation

```python
mcstat.run_simulation()
```

    Sampling these parameters:
          name      start [      min,       max] N(       mu,   sigma^2)
         $x_1$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
         $x_2$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
         $x_3$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
         $x_4$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
         $x_5$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
         $x_6$:  0.00e+00 [     -inf,       inf] N( 0.00e+00,      inf)
     [-----------------100%-----------------] 4000 of 4000 complete in 0.9 sec

Extract results and plot chain panel

```python
from pymcmcstat import mcmcplot as mcp
# Extract results
results = mcstat.simulation_results.results
chain = results['chain']
s2chain = results['s2chain']
names = results['names'] # parameter names
# plot chain panel
settings = dict(fig=dict(figsize=(6, 6)))
f = mcp.plot_chain_panel(chain, names, settings=settings)
```

# Pairwise Correlation
Here we plot the pairwise correlation as well as 50% and 95% critical regions.  These critical regions have a shape resembling a banana, hence the name of the example.

```python
# plot pairwise correlation
settings = dict(
    fig=dict(figsize=(6, 6)),
    plot=dict(
        color='g',
        marker='s',
        mfc='none'))
mcp.plot_pairwise_correlation_panel(
    chain[:, 0:2], names[0:2], settings=settings)
# calculate contours for 50% and 95% critical regions
c50 = 1.3863 # critical values from chisq(2) distribution
c95 = 5.9915
xe50, ye50 = generate_ellipse(udobj.mu, c50*udobj.sig[0:2, 0:2])
xe95, ye95 = generate_ellipse(udobj.mu, c95*udobj.sig[0:2, 0:2])
bxy50 = bananafunction(np.array([xe50, ye50]).T, udobj.a, udobj.b)
bxy95 = bananafunction(np.array([xe95, ye95]).T, udobj.a, udobj.b)
# add countours to pairwise plot
plt.plot(bxy50[:,0], bxy50[:,1], 'k-', LineWidth=2)
plt.plot(bxy95[:,0], bxy95[:,1], 'k-', LineWidth=2)
plt.axis('equal')
tmp = plt.title('2 first dimensions of the chain with 50% and 95% target contours')
```