PyTorch Introduction
================

    Note: This exercise is optional and only serves as an introduction and cheatsheet to the general concepts of PyTorch.

PyTorch is a scientific computing package for Python:

-  Tensor and Neural Network computations (inparticular deep learning)
-  Research oriented (in comparison to e.g. TensorFlow)
-  Dynamic computational graph (in comparison to e.g. TensorFlow)
-  “NumPy on the GPU”
-  Backend and API heavily inspired by the original Torch written in Lua

An in-depth tutorial of the concepts described in this notebook can be found [here](https://github.com/jcjohnson/pytorch-examples).

```python
%matplotlib inline
import numpy as np
import torch
```

Tensors
=====

The PyTorch `Tensor` class is very similar to the NumPy `ndarray` class. Their main distinction is the ability of PyTorch Tensors to be used on a GPU which lets them benefit from vastly accelerated and parallelized computations. In order to work with PyTorch it is crucial to understand the basic behavior of its `Tensor` class.

Let's start with the initialization of a regular `5x3` matrix `Tensor`:

```python
x = torch.Tensor(5, 3)
print(x)
```

     0.0000e+00  0.0000e+00  4.4842e-44
     0.0000e+00         nan  0.0000e+00
     1.1754e-05  4.4169e-05  4.5050e+01
     1.8088e-07  7.2083e+02  7.2085e+02
     1.9720e+02  1.1341e+04 -1.4595e+17
    [torch.FloatTensor of size 5x3]

The same matrix can be initialized with random entries:

```python
x = torch.rand(5, 3)
print(x)
```

     0.4184  0.3914  0.0820
     0.2140  0.2155  0.8103
     0.4853  0.2132  0.3957
     0.5187  0.0773  0.4112
     0.2807  0.1579  0.0499
    [torch.FloatTensor of size 5x3]

The size of a `Tensor` can be retrieved with:

```python
print(x.size())
```

    torch.Size([5, 3])

Note

`torch.Size` is in fact a tuple, so it supports the same operations as a regular Python tuple.

In contrast to a static computational graph of for example Tensorflow the dynamic graph of PyTorch allows to retrieve information such as its size at any time during runtime.

Tensor Operations
--------

There are multiple syntaxes for `Tensor` operations. We illustrate the different options on the example of `Tensor` addition.

Regular (NumPy) syntax:

```python
y = torch.rand(5, 3)
print(x + y)
```

     0.7917  0.8690  0.4293
     0.4868  0.2193  1.1319
     0.6772  0.6665  0.5718
     1.2141  0.1595  0.4661
     1.2262  0.7752  0.6938
    [torch.FloatTensor of size 5x3]

PyTorch syntax:

```python
print(torch.add(x, y))
```

     0.7917  0.8690  0.4293
     0.4868  0.2193  1.1319
     0.6772  0.6665  0.5718
     1.2141  0.1595  0.4661
     1.2262  0.7752  0.6938
    [torch.FloatTensor of size 5x3]

PyTorch syntax with specific output variable:

```python
result = torch.Tensor(5, 3)
torch.add(x, y, out=result)
print(result)
```

     0.7917  0.8690  0.4293
     0.4868  0.2193  1.1319
     0.6772  0.6665  0.5718
     1.2141  0.1595  0.4661
     1.2262  0.7752  0.6938
    [torch.FloatTensor of size 5x3]

PyTorch syntax for inplace operations:

```python
# adds x to y
y.add_(x)
print(y)
```

     0.7917  0.8690  0.4293
     0.4868  0.2193  1.1319
     0.6772  0.6665  0.5718
     1.2141  0.1595  0.4661
     1.2262  0.7752  0.6938
    [torch.FloatTensor of size 5x3]

Note

Any operation that mutates a `Tensor` in-place is post-fixed with an ``_``.

For example: ``x.copy_(y)``, ``x.t_()``, will copy  ``y`` to ``x``.

`Tensor` indexing works just like standard NumPy indexing. And since recently PyTorch even supports `Tensor` [broadcasting](https://docs.scipy.org/doc/numpy-1.13.0/user/basics.broadcasting.html)!

```python
print(x[:, 1])
```

     0.3914
     0.2155
     0.2132
     0.0773
     0.1579
    [torch.FloatTensor of size 5]

NumPy: There and back again
---------------------------

Converting a PyTorch `Tensor` to a NumPy `ndarray` and vice versa is a very simple. The `Tensor` and the `ndarray` will share the location of the underlying memory, and changing one will also change the other.

Converting a `Tensor` to a `ndarray` works by simply calling the `Tensor.numpy()` method:

```python
a = torch.ones(5)
b = a.numpy()
print(a)
print(b)
```

     1
     1
     1
     1
     1
    [torch.FloatTensor of size 5]

    [ 1.  1.  1.  1.  1.]

Changing the `Tensor` effects the `ndarray` as well:

```python
a.add_(1)
print(a)
print(b)
```

     2
     2
     2
     2
     2
    [torch.FloatTensor of size 5]

    [ 2.  2.  2.  2.  2.]

The conversion from a `ndarray` to a `Tensor` is just as simple and holds the same properties:

```python
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)
```

    [ 2.  2.  2.  2.  2.]

     2
     2
     2
     2
     2
    [torch.DoubleTensor of size 5]

Every `Tensor` allocated on the CPU (except the `torch.CharTensor`) support converting to
NumPy and back.

Tensors on the GPU
------------------

PyTorch Tensors can be moved onto a GPU using the ``Tensor.cuda()`` method. Before converting a GPU `Tensor` to NumPy it has to be moved back to the CPU by calling the ``Tensor.cpu()`` method.

```python
# first check if cuda is available
if torch.cuda.is_available():
    x = x.cuda()
    y = y.cuda()
    z = x + y

    print(z)
    print(z.cpu().numpy())
else:
    print("CUDA not available.")
```

    CUDA not available.

More on PyTorch Tensors
-----------------------

The documentation of many more `Tensor` operations, including transposing, indexing, slicing, mathematical operations, linear algebra, random numbers can be found [here](http://pytorch.org/docs/torch).

Autograd - automatic differentiation
===================================

Central to all neural networks in PyTorch is the ``autograd`` package. The package provides automatic differentiation for all operations on Tensors. PyTorch is a define-by-run framework, which means that the calculation of gradients ( e.g. during backpropagation) is defined at runtime and can be different at every single iteration.

Variable
--------

``autograd.Variable`` is the central class of the package. It wraps the `Tensor` class, and supports nearly all of its operations. Once a computation graph is executed the ``Variable.backward()`` method can be used to automatically compute all the gradients.

If ``Variable`` is not a scalar, the ``backward()`` method requires an additional ``grad_output`` argument which matches the shape of the ``Variable``. ``grad_output`` is supposed to be the gradient w.r.t the given output. For a scalar ``Variable`` ``grad_output`` is assumed to be `torch.Tensor([1.0])`.

The underlying, raw tensor can be accessed with the ``Variable.data`` attribute and the
gradient w.r.t. this variable is accumulated into ``Variable.grad``.

In addition to the ``Variable`` class the `autograd` packages provides a `Function` class. The two classes are interconnected and build up an acyclic graph which encodes a complete history of computation. Each `Variable` has
a ``Variable.grad_fn`` attribute which references the ``Function`` (e.g. an operation such as addition) that created the respective ``Variable`` and thereby determines its gradient. For Variables that were created by the user and not as a result of an operation the ``grad_fn`` attribute is ``None``.

The following simple examples will illustrate the basic concepts of the ``autograd`` package.

```python
from torch.autograd import Variable
```

Create a `Variable` object:

```python
x = Variable(torch.ones(2, 2), requires_grad=True)
print(x)
```

    Variable containing:
     1  1
     1  1
    [torch.FloatTensor of size 2x2]

Apply an operation to the `Variable`:

```python
y = x + 2
print(y)
```

    Variable containing:
     3  3
     3  3
    [torch.FloatTensor of size 2x2]

Since ``y`` was created as a result of an operation it has a ``grad_fn`` attribute (`Function`) unequal to `None`:

```python
print(y.grad_fn)
```

    <AddBackward0 object at 0x7f499a88bb38>

Applying more operations to `y` increases the computational graph:

```python
z = y * y * 3
out = z.mean()

print(z, out)
```

    Variable containing:
     27  27
     27  27
    [torch.FloatTensor of size 2x2]
     Variable containing:
     27
    [torch.FloatTensor of size 1]

Gradients
---------
The gradient w.r.t the input `x` can now be computed (backpropagated) with ``out.backward()``. Remember for a scalar this is equivalent to doing ``out.backward(torch.Tensor([1.0]))``.

```python
out.backward()
```

The input `x` was a `2x2` `Variable` and therefore $\frac{d(out)}{dx}$ yields a matrix with the same shape:

```python
print(x.grad)
```

    Variable containing:
     4.5000  4.5000
     4.5000  4.5000
    [torch.FloatTensor of size 2x2]

For such a small computation graph the solution can easily be verified:

The output w.r.t. the input is given as
$$
\begin{align}
    out =& \frac{1}{4}\sum_i z_i \\
        =& \frac{1}{4}\sum_i 3y_i y_i \\
        =& \frac{1}{4}\sum_i 3(x_i+2)^2
\end{align}
$$.

Therefore the gradient is $\frac{\partial out}{\partial x_i} = \frac{3}{2}(x_i+2)$, which yields
$\frac{\partial out}{\partial x_i}\bigr\rvert_{x_i=1} = \frac{9}{2} = 4.5$ for a particular input $x_i=1$.

The `autograd` package in combination with the dynamic graph structure allow to do crazy things such as:

```python
x = torch.randn(3)
x = Variable(x, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
    y = y * 2

print(y)
```

    Variable containing:
     -229.1273
      319.0190
    -1039.6526
    [torch.FloatTensor of size 3]

Neural Networks
===============

The `Tensor` and `Variable` classes in combination with the `autograd` package build the foundation for constructing Neural Networks (NNs) with PyTorch. To further fascilitate the construction and training of a NN the ``torch.nn`` package, which depends on `autograd` to define NN models and differentiate them, includes additional NN-specifc classes and helper functions.

For example the ``nn.Module`` class which works as a boilerplate NN model class and eventually contains all the individual layers and the ``Module.forward(x)`` method that infers the input ``x`` and returns the output of a NN.

The following is a grapical illustration of the infamous *LeNet* NN from Yann LeCun. This NN was trained to classify the MNIST dataset of handwritten digit images:

It is a simple feed-forward network which takes the input, feeds it through several layers one after the other, and then finally produces the classification output.

Define *LeNet* with PyTorch
--------------------------

The following is an example implementation of the classification network above:

```python
import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as F

class LeNet(nn.Module):

    def __init__(self):
        """
        Class constructor which preinitializes NN layers with trainable
        parameters.
        """
        super(LeNet, self).__init__()
        # 1 input image channel, 6 output channels, 5x5 square convolution
        # conv kernel
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)
        # an affine operation: y = Wx + b
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        """
        Forwards the input x through each of the NN layers and outputs the result.
        """
        # Max pooling over a (2, 2) window
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        # If the size is a square you can only specify a single number
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        # An efficient transition from spatial conv layers to flat 1D fully
        # connected layers is achieved by only changing the "view" on the
        # underlying data and memory structure.
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

    def num_flat_features(self, x):
        """
        Computes the number of features if the spatial input x is transformed
        to a 1D flat input.
        """
        size = x.size()[1:]  # all dimensions except the batch dimension
        num_features = 1
        for s in size:
            num_features *= s
        return num_features

net = LeNet()
print(net)
```

    LeNet(
      (conv1): Conv2d (1, 6, kernel_size=(5, 5), stride=(1, 1))
      (conv2): Conv2d (6, 16, kernel_size=(5, 5), stride=(1, 1))
      (fc1): Linear(in_features=400, out_features=120)
      (fc2): Linear(in_features=120, out_features=84)
      (fc3): Linear(in_features=84, out_features=10)
    )

Due to the `autograd` package a NN merely requires the definition of the ``Module.forward()`` method. The ``.backward()`` function (which backpropagtes the gradients) is automatically defined. Any `Tensor` operation is allowed in the ``forward`` function.

The learnable parameters of a model are returned by ``Module.parameters()``:

```python
params = list(net.parameters())
print(len(params))
print(params[0].size())  # conv1's .weight
```

    10
    torch.Size([6, 1, 5, 5])

The input to the `forward` method is an ``autograd.Variable``, and so is the output:

```python
x = Variable(torch.randn(1, 1, 32, 32))
output = net(x)
print(output)
```

    Variable containing:
    -0.0907  0.0858 -0.0585 -0.0773 -0.1693  0.0104 -0.0483  0.0472  0.0216  0.0591
    [torch.FloatTensor of size 1x10]

Before backpropagating for example a random gradient, the gradient buffers of all parameters should be set to zero:

```python
net.zero_grad()
output.backward(torch.randn(1, 10))
```

Note

Calling the ``Variable.backward()`` method a second time before new inputs are forwarded will through an error. This is due to PyTorch deleting all the intermediary results in order to reduce memory consumption. Calling the ``.backward()`` method with the `retain_graph=True` argument keeps those results.

Note

The entire ``torch.nn`` package only supports inputs that are a mini-batch of samples, and not a single sample.

    For example, ``nn.Conv2d`` will take in a 4D Tensor of ``nSamples x nChannels x Height x Width``.

    If you have a single sample, just use ``x.unsqueeze(0)`` to add a fake batch dimension.

Loss Function
-------------
A loss function takes the (output, target) pair as inputs, and computes a value that estimates "how far away" the output is from the target.

There are several different loss functions predefined under the `torch.nn` package. An example of a simple loss is the ``nn.MSELoss`` which computes the mean-squared error between the input and the target value.

More examples of predefined losses are documented [here](http://pytorch.org/docs/nn.html#loss-functions).

A MSE loss example:

```python
output = net(x)
target = Variable(torch.arange(1, 11))  # a dummy target with 10 classes
criterion = nn.MSELoss()

loss = criterion(output, target)
print(loss)
```

    Variable containing:
     38.7198
    [torch.FloatTensor of size 1]

When ``loss.backward()`` is called, the whole graph is differentiated w.r.t. the loss, and all Variables in the graph will have their ``Variable.grad`` attribute accumulated with the gradient.

Backpropagate the Loss
--------------------

A curical step for optimizing the network weights is the backpropogation of the loss. The nature of a computational graph makes this as easy as calling ``loss.backward()``. But since the gradients will be accumulated to already existing gradients one has to clear them first.

```python
net.zero_grad()     # zeroes the gradient buffers of all parameters

print('conv1.bias.grad before backward')
print(net.conv1.bias.grad)

loss.backward()

print('conv1.bias.grad after backward')
print(net.conv1.bias.grad)
```

    conv1.bias.grad before backward
    Variable containing:
     0
     0
     0
     0
     0
     0
    [torch.FloatTensor of size 6]

    conv1.bias.grad after backward
    Variable containing:
     0.1088
    -0.0821
     0.0658
    -0.0554
    -0.0865
    -0.0172
    [torch.FloatTensor of size 6]

Weights Optimization
------------------
The simplest update rule used in practice for optimizing the weights of a NN is the Stochastic Gradient Descent (SGD):

``weight = weight - learning_rate * gradient``

Like any other NN component the optimization step can be implemented with the basic PyTorch classes.

For example:

```python
def sgd_step(net):
    learning_rate = 0.01
    for f in net.parameters():
        f.data.sub_(f.grad.data * learning_rate)
```

However, the PyTorch framework contains a small optimization package called ``torch.optim``. It includes various predefined update rules such as SGD, Nesterov-SGD, Adam, RMSProp, etc.

Note

Common optimization options such as the L2-regularization (see ``weight_decay`` argument) are already included in the predefined optimization schemes.

Using it is very simple:

```python
import torch.optim as optim

# create an optimizer
optimizer = optim.SGD(net.parameters(), lr=0.01, weight_decay=1e-3)

# a single step of an example training loop
optimizer.zero_grad()   # zero the gradient buffers
output = net(x)
loss = criterion(output, target)
loss.backward()
optimizer.step()    # Does the update based on the accumalted gradients
```

Recap
==============

  -  ``torch.Tensor`` - A multi-dimensional array.
  -  ``autograd.Variable`` - Wraps a `Tensor` and records the history of
     operations applied to it. Has the same API as a ``Tensor``, with
     some additions like ``backward()``. Also holds the gradient
     w.r.t. the tensor.
  -  ``nn.Module`` - Neural network module. Convenient way of
     encapsulating parameters, with helpers for moving them to GPU,
     exporting, loading, etc.
  -  ``nn.Parameter`` - A kind of `Variable`, that is automatically
     registered as a parameter when assigned as an attribute to a
     ``Module``.
  -  ``autograd.Function`` - Implements forward and backward definitions
     of an autograd operation. Every ``Variable`` operation, creates at
     least a single ``Function`` node, that connects to functions that
     created a ``Variable`` and encodes its history.

Note

The `torchvision` package includes many predefined helper funcitons specifally designed for solving computer vision problems.


# "Gillespie Algorithm"
> "In this blog post we will look at the grand-daddy of stochastic simulation methods: the Gillespie Algorithm (otherwise known as the stochastic simulation algorith SSA). If you have ever done any form of stochastic simulation you will owe a great deal of gratitude to the Gillespie algorithm which likely inspired the techniques you used."
- toc: true
- author: Lewis Cole (2020)
- branch: master
- badges: false
- comments: false
- categories: [Gillespie-Algorithm, Stochastic-Simulation-Algorithm, Computational-Statistics, Probability, Tau-Leaping, Master-Equation, Adaptive-Tau-Leaping]
- hide: false
- search_exclude: false
- image: https://github.com/lewiscoleblog/blog/raw/master/images/Gillespie/Gillespie.jpg

```python
#hide
import warnings
warnings.filterwarnings('ignore')
```

The Gillespie algorithm is one of the most historically important stochastic simulation algorithms ever created. At its heart the intuition behind it is very simple and it is re-assuring that it "works" - this is not always the case with stochastic simulation where the "obvious" idea can sometimes have unintended debilitating consequences.

The algorithm was first presented by Doob (and is sometimes refered to as the Doob-Gillespie algorithm) in the mid 1940s. It was implemented by Kendall in the 1950s. However it wasn't until the mid 1970s that Gillespie re-derived the method by studying physical systems that it became widely used. In publishing the method he essentially created the entire fields of systems biology and computational chemistry by opening the door to what is possible through stochastic simulation.

## Background
In this blog we will consider applying the Gillespie method to the area of chemical reaction kinetics, this is the application Gillespie originally had in mind. The concepts described will carry over to other applications.

Imagine we wish to model a particular chemical reaction. We could use a determistic approach to model the reaction, this will require setting up a family of coupled differential equations. In doing so we will essentially "ignore" any microscopic behaviour and look at the reaction system at a "high level". This can mean we miss out on a lot of the "detail" of the reaction which may be of interest to us. Further in some cases this approach may not even be applicable, for example to set up a differential equation we assume that we have large quantities of reactants that are perfectly mixed, this allows us to "average over" all reactants to create nice smooth dynamics. This may not reflect reality if there are only relatively few reactants in a system. An alternate approach is to use a stochastic "discrete event" model - this is where we model individual reactions seperately as discrete events occuring in time. This matches our physical intuition of how reactions occur: we wait until the reactants "bump" into each other in the right way before a reaction occurs. One way to summarise this mathematically is through the use of a "master equation".

In the sciences a master equation represents the time evolution properties of a multi-state jumping system, by which we mean a system that "jumps" between distinct states through time (in contrast a "diffusion system" varies gradually). The system in question being stochastic in nature we are concerned with observing how the state distribution varies over time, for example: with some initial condition what is the probability of finding the system in a particular state within the next X seconds/minutes/years? Of course the time units depend on the nature of the system (e.g. if we construct a master equation for predator/prey dynamics we are unlikely to be interested in microsecond timescales, however if looking at a chemical reaction we are unlikely to find a timescale in days useful.) If we want to display the master equation mathematically we use a transition rate matrix $A(t)$ - this can evolve in time or it can be static.

We can then express the master equation in the form:
$$ \frac{d\mathbf{P}_t}{dt} = A(t) \mathbf{P}_t $$
Where vector $\mathbf{P}_t$ represents the probability distribution of states at time t - obscured by notation is an initial condition. Those from a mathematical or probabilistic background will recognise this as a Kolmogorov backwards equation for jump processes. If we expand the notation a little such that $P_{ij}(s,t)$ represents the probability of the system being in state $i$ at time $s$ and state $j$ at time $t$ then we can note that the transition rate matrix satisfies:
\begin{align}
A_{ij}(t) &= \left[ \frac{\partial P_{ij}(t,u)}{du} \right]_{u=t} \\
A_{ij}(t) & \geq 0  \quad \quad \quad \quad \forall i \neq j \\
\sum_j A_{ij}(t) &= 0 \quad \quad \quad \quad \forall i
\end{align}
Further we can note that if there is a distribution $\pi$ such that:
$$ \pi_j A_{ij}(t) = \pi_i A_{ij}(t) $$
For all pairs of states $(i,j)$ then the process satisfies detailed balance and the process is a reversible Markov process.

## Gillespie Algorithm
The Gillespie algorithm is allows us to model the exact dynamics described by the master equation. In some (simple) cases we can solve the master equation analytically, but for complicated examples (e.g. say we have 50 different types of reaction occuring) this may not be feasible and so the Gillespie algorithm (or some sort of simulation method) is necessary. In pseudo code we can write down the Gillespie algorithm as:
1. **Initialization** - initialize the system, in the context of reaction kinetics this amounts to the setting up initial chemical concentrations
2. **Monte-Carlo** -
    1. Randomly simulate the time to the next event
    2. Given an event has occurred randomly select which event has occured
3. **Update** - based on 2. move the model time forward to the event time and update the state of the system
4. **Repeat** - Iterate through steps 2. and 3. until some stopping criteria is met

This essentially follows our intuition and there is no "technical trickery" such as fancy sampling methods, acceptance/rejection, etc. It is just a clean simple method - which is nice! Since we model by event as opposed to discretizing time steps this is an "exact" simulation method - meaning any trajectory simulated will follow the master equation dynamics exactly. However due to the random nature of any trajectory we will have to loop over these steps multiple times to find "typical" reaction paths (or whatever property we are trying to study).

## An Example
To illustrate the algorithm in action we will take a simple reaction. We will have following forward reaction
$$A + B \to AB$$
Where two monomers $A$ and $B$ react to form a dimer $AB$. The corresponding reverse reaction being:
$$AB \to A + B$$
We will denote the rate of the forward reaction to be $r_f$ and the rate of the backward reaction to be $r_f$. If we let the number of molecules present be denoted by: $N_A, N_B$ and $N_{AB}$ then the rate of any reaction occurring is:
$$R = r_f N_A N_B + r_b N_{AB}$$
Also given a reaction has occured the probability of the forward reaction having taken place is:
$$\mathbb{P}(A + B \to AB) = \frac{r_f N_A N_B}{R}$$
For a model such as this we typically want to remove any "path dependence" - the arrival of the next reaction event is independent of reactions that have occurred previously (given the concentration of reactants). To satisfy this constraint typically reactions events are taken to follow a Poisson process. Under this assumption the number of reactions occuring within a time period $\Delta T$ follows a $Poisson(R\Delta T)$ distribution. Moreover the time between reactions is then follows an exponential distribution. Thus if we sample $u \sim U[0,1]$ then we take the time until next reaction to be $\tau = \frac{1}{R}ln\left( \frac{1}{u} \right)$. (Note: here I have used that $U$ and $(1-U)$ have the same distribution).

A basic implementation of this can be seen below:

```python
# An implenetation of the Gillespie algorithm
# applied to a pair of reactions:
# A + B -> AB
# AB -> A + B

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Fix random seed for repeatability
np.random.seed(123)

###### Fix model parameters ######
N_A0 = 25     # Initial number of A molecules
N_B0 = 35     # Initial number of B molecules
N_AB0 = 5     # Initial number of AB molecules

rf = 2        # Forward reaction rate
rb = 1        # Backwards reaction rate

steps = 25    # Number of reactions per trajectory
cycles = 100  # Number of trajectories iterated over

# Set up holder arrays
T = np.zeros((cycles, steps+1))
N_A = np.zeros((cycles, steps+1))
N_B = np.zeros((cycles, steps+1))
N_AB = np.zeros((cycles, steps+1))

# Store initial conditions
N_A[:,0] = N_A0
N_B[:,0] = N_B0
N_AB[:,0] = N_AB0

###### Main Code Loop ######
for i in range(cycles):
    for j in range(steps):
        # Calculate updated overall reaction rate
        R = rf * N_A[i,j] * N_B[i,j] + rb * N_AB[i,j]

        # Calculate time to next reaction
        u1 = np.random.random()
        tau = 1/R * np.log(1/u1)

        # Store reaction time
        T[i, j+1] = T[i,j] + tau

        # Select which reaction to occur
        Rf = rf * N_A[i,j] * N_B[i,j] / R
        u2 = np.random.random()

        # Update populations
        if u2 < Rf:
            N_A[i,j+1] = N_A[i,j] - 1
            N_B[i,j+1] = N_B[i,j] - 1
            N_AB[i,j+1] = N_AB[i,j] + 1
        else:
            N_A[i,j+1] = N_A[i,j] + 1
            N_B[i,j+1] = N_B[i,j] + 1
            N_AB[i,j+1] = N_AB[i,j] - 1

# Calculate an average trajectory plot
ave_steps = 100
T_max = T.max()

# Set up average arrays
T_ave = np.linspace(0,T_max,ave_steps+1)
N_A_ave = np.zeros(ave_steps+1)
N_B_ave = np.zeros(ave_steps+1)
N_AB_ave = np.zeros(ave_steps+1)

N_A_ave[0] = N_A0
N_B_ave[0] = N_B0
N_AB_ave[0] = N_AB0

# Pass over average array entries
for i in range(1, ave_steps+1):
    tmax = T_ave[i]
    A_sum = 0
    B_sum = 0
    AB_sum = 0
    t_count = 0

    # Pass over each trajectory and step therein
    for j in range(cycles):
        for k in range(steps):
            if T[j,k] <= tmax and T[j,k+1] > tmax:
                t_count += 1
                A_sum += N_A[j,k]
                B_sum += N_B[j,k]
                AB_sum += N_AB[j,k]

    # Caclulate average - taking care if no samples observed
    if t_count == 0:
        N_A_ave[i] = N_A_ave[i-1]
        N_B_ave[i] = N_B_ave[i-1]
        N_AB_ave[i] = N_AB_ave[i-1]
    else:
        N_A_ave[i] = A_sum / t_count
        N_B_ave[i] = B_sum / t_count
        N_AB_ave[i] = AB_sum / t_count

###### Plot Trajectories ######
fig, axs = plt.subplots(3, 1, figsize=(10,20))

# Plot average trajectories
axs[0].plot(T_ave, N_A_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[0].set_title('Number A Molecules')
axs[0].set_ylim((0,35))
axs[0].set_xlim((0,0.125))
axs[1].plot(T_ave, N_B_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[1].set_title('Number B Molecules')
axs[1].set_ylim((0,35))
axs[1].set_xlim((0,0.125))
axs[2].plot(T_ave, N_AB_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[2].set_title('Number AB Molecules')
axs[2].set_xlabel("Time")
axs[2].set_ylim((0,35))
axs[2].set_xlim((0,0.125))

# Plot each simulated trajectory
for i in range(cycles):
    axs[0].plot(T[i,:], N_A[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[1].plot(T[i,:], N_B[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[2].plot(T[i,:], N_AB[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)

plt.show()
```

In these plots we can see the various trajectories along with their average. If we increase the number of molecules and the number of trajectories we can get a "smoother" plot. Since we have the full evolution of the system we can also look at some other statistics, for example let's suppose we are interested in the distribution in the number of molecules of each type at time 0.5. We can also plot this using our samples:

```python
time = 0.025
N_A_time = np.zeros(cycles)
N_B_time = np.zeros(cycles)
N_AB_time = np.zeros(cycles)

for i in range(cycles):
    for j in range(1, steps):
        if T[i,j] => time and T[i,j-1] < time:
            N_A_time[i] = N_A[i,j]
            N_B_time[i] = N_B[i,j]
            N_AB_time[i] = N_AB[i,j]
    # If trajectory doesn't span far enough take latest observation
    if T[i, steps] < time:
        N_A_time[i] = N_A[i, steps]
        N_B_time[i] = N_B[i, steps]
        N_AB_time[i] = N_AB[i, steps]

plt.hist(N_A_time, density=True, bins=np.arange(35), label="A", color='lightgrey')
plt.hist(N_B_time, density=True, bins=np.arange(35), label="B", color='dimgrey')
plt.hist(N_AB_time, density=True, bins=np.arange(35), label="AB", color='red')
plt.legend()
plt.show()
```

If instead of a system of 2 reactions we instead wanted to look a system of a large number of reactions we could modify the method above quite simply. Instead of the calculation of $R$ (overall reaction rate) consisting of 2 terms it will consist of a larger number of terms depending on the nature of the individual reactions. The probability of selecting a particular reaction type would then equally be in proportion to their contribution to $R$.

We can also notice that there is nothing "special" about the method that means it only applies to reaction kinetics. For example: the example code above could equally be a "marriage and divorce model" for heterosexual couples: A representing women and B representing men, AB representing a marriage. Through defining the "reactions" slightly differently it doesn't take much modification to turn this into a infection model: for example there could be 3 states: susceptible to infection, infected and recovered (potentially with immunity) with transition rates between each of these states.

We can see then that the Gillespie algorithm is very flexible and allows us to model stochastic systems that may otherwise be mathematically intractable. Through the nature of the modelling procedure we can sample from the system exactly (upto the precision of floating point numbers within our computers!)

There is a downside to exact simulation however: it can be very slow! In the example above the speed isn't really an issue since the system is so simple. However if we were modelling many different reaction types (say the order of 100s) then to allow for adequate samples we will need to run many trajectories, this can quickly spiral into a very slow running code! Thankfully however the method has been adapted in many ways to combat this issue.

## Hybrid-Gillespie
We can note that calculating deterministic results from an ODE is (much) quicker than implementing the Gillespie simulation algorithm since there is no random element. However we notice that we do not have to model every reaction type using the same Gillespie approach. For example suppose we have one reaction type that is much slower than the others, say the order of 10 times slower. We could model this reaction via a determinstic ODE approach and simply rely on Gillespie for the more rapidly changing dynamics. Of course this is not applicable in every situation - as with any modelling or approximation used we should be sure that it is applicable to the situation at hand. For brevity we will not code an example of this here but it should be easy enough to modify the code above (for example by adding that molecule $A$ can "disappear" from the system with a rate 1/10 times the rate of the backward reaction).

## Tau Leaping
Tau leaping modifies the Gillespie methodology above, it sacrifices exact simulation in favour of an approximate simulation that is quicker to compute. The main idea behind tau-leaping is also intuitive: instead of modelling time to the next event we "jump" forward in time and then compute how many reactions we would expect to see within that time frame and updating the population amounts in one step. By updating the population amounts in one go we should be able to compute much faster. It should be clear that this is an approximation to the Gillespie algorithm. The size of the "leaps" determines how efficient the method is and how accurate the approximation is. If we make very large steps we can model many reactions per step which speeds up the implementation, however the simulation will also be less accurate since the populations will be updated less frequently. Conversely a very small leap size will mean many leaps will not see a reaction and so the algorithm will run more slowly, however this should result in dynamics very close to the Gillespie method. Often choosing the leap size requuires some trial and error.

we can write pseudo-code for the tau-leaping process as:
1. **Initialize** - Set initial conditions for the system and set leaping size
2. **Calculate event rates** - for each event types depending on state of the system
3. **Monte-Carlo** - for each event type sample number of events occuring within the leap
4. **Update** - Update system state based on number of events
5. **Repeat** - Repeat steps 2-4 until some stopping criteria is met

Recall: in the example above we used an exponential waiting time between reactions. This means the reactions occur as a poisson process - as a result the number of reactions occuring within a given timeframe will follow a poisson distribution. We also have to be careful to not allow a negative population (at least in the example presented - in other systems this may be reasonable).

We can modify our example above to use Tau-leaping as:

```python
# An implenetation of the Gillespie algorithm
# with tau leaping
# Applied to a pair of reactions:
# A + B -> AB
# AB -> A + B

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import poisson
%matplotlib inline

# Fix random seed for repeatability
np.random.seed(123)

###### Fix model parameters ######
N_A0 = 25     # Initial number of A molecules
N_B0 = 35     # Initial number of B molecules
N_AB0 = 5     # Initial number of AB molecules

rf = 2        # Forward reaction rate
rb = 1        # Backwards reaction rate

leap = 0.005 # Size of leaping steps

steps = 25    # Number of leaps per trajectory
cycles = 100  # Number of trajectories iterated over

# Set up holder arrays
T = np.arange(steps+1)*leap
N_A = np.zeros((cycles, steps+1))
N_B = np.zeros((cycles, steps+1))
N_AB = np.zeros((cycles, steps+1))

# Store initial conditions
N_A[:,0] = N_A0
N_B[:,0] = N_B0
N_AB[:,0] = N_AB0

###### Main Code Loop ######
for i in range(cycles):
    for j in range(steps):
        # Calculate updated reaction rates
        Rf = rf * N_A[i,j] * N_B[i,j]
        Rb = rb * N_AB[i,j]

        # Calculate number of reactions by type
        uf = np.random.random()
        ub = np.random.random()

        Nf = poisson.ppf(uf, Rf*leap)
        Nb = poisson.ppf(ub, Rb*leap)

        # Apply limits to prevent negative population
        Limitf = min(N_A[i,j], N_B[i,j])
        Limitb = N_AB[i,j]

        Nf = min(Nf, Limitf)
        Nb = min(Nb, Limitb)

        # Update populations
        N_A[i,j+1] = N_A[i,j] + Nb - Nf
        N_B[i,j+1] = N_B[i,j] + Nb - Nf
        N_AB[i,j+1] = N_AB[i,j] + Nf - Nb

# Calculate average arrays
N_A_ave = N_A.mean(axis=0)
N_B_ave = N_B.mean(axis=0)
N_AB_ave = N_AB.mean(axis=0)

###### Plot Trajectories ######
fig, axs = plt.subplots(3, 1, figsize=(10,20))

# Plot average trajectories
axs[0].plot(T, N_A_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[0].set_title('Number A Molecules')
axs[0].set_ylim((0,35))
axs[0].set_xlim((0,0.125))
axs[1].plot(T, N_B_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[1].set_title('Number B Molecules')
axs[1].set_ylim((0,35))
axs[1].set_xlim((0,0.125))
axs[2].plot(T, N_AB_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[2].set_title('Number AB Molecules')
axs[2].set_xlabel("Time")
axs[2].set_ylim((0,35))
axs[2].set_xlim((0,0.125))

# Plot each simulated trajectory
for i in range(cycles):
    axs[0].plot(T[:], N_A[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[1].plot(T[:], N_B[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[2].plot(T[:], N_AB[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)

plt.show()
```

We can see here that even though the trajectories from tau-leaping are less exact the procedure has produced smoother average results for the same number of simulation steps (approximately the same running time).

And again we can look at the distribution at time=0.025:

```python
time = 0.025
N_A_time = np.zeros(cycles)
N_B_time = np.zeros(cycles)
N_AB_time = np.zeros(cycles)

for i in range(cycles):
    for j in range(1, steps):
        if T[j] >= time and T[j-1] < time:
            N_A_time[i] = N_A[i,j]
            N_B_time[i] = N_B[i,j]
            N_AB_time[i] = N_AB[i,j]
    # If trajectory doesn't span far enough take latest observation
    if T[i, steps] < time:
        N_A_time[i] = N_A[i, steps]
        N_B_time[i] = N_B[i, steps]
        N_AB_time[i] = N_AB[i, steps]

plt.hist(N_A_time, density=True, bins=np.arange(35), label="A", color='lightgrey')
plt.hist(N_B_time, density=True, bins=np.arange(35), label="B", color='dimgrey')
plt.hist(N_AB_time, density=True, bins=np.arange(35), label="AB", color='red')
plt.legend()
plt.show()
```

Here we can see improved distributions with (what appears to be) less noise. To justify this we would want to run more tests however.

Note: this is the most basic implementation of the tau-leaping procedure. In certain situations this needs to be manipulated to improve behaviour, for example if the poisson draw is often large enough to cause the population to go negative then a truncation procedure (or acceptance/rejection scheme) needs to be employed in such a way as to retain the average reaction rates. In this simple example we ignore this complication, there are some occasions where the number of $A$ molecules hits zero so there will be some bias in the estimates presented above.

## Adaptive Tau-Leaping
The "problem" with the tau-leaping method above is that it is very sensitive to the leap size. It is also possible that as the system evolves what started out as a "good" leap size becomes "bad" as the dynamics change. One possible solution to this is to use an "adaptive" method whereby the leap size varies depending on the dynamics. The main idea is to limit the leap sizes from being so large that the populations can reach an unfavourable state (e.g. negative population sizes) or jump to a state "too far away".

There are many ways to do this, one of the more popular was developed by Y. Cao and D. Gillespie in 2006. In order to describe the method we will need to introduce some notation. We let $\mathbf{X}_t = \left( X_t^i \right)_{i=1}^N$ to be a vector of population sizes at time t. We intorduce variables $v_{ij}$ to represent the change in component $i$ of the population when an event $j$ occurs - we will use $i$ indices to refer to components of the population vector and $j$ indices to refer to event types. $R_j(\mathbf{X}_t)$ is the rate of event $j$ with population $\mathbf{X}_t$. In this method we look to bound the relative shift in rates at each step by a parameter $\epsilon$.

In pseudo-code we can describe the process via:
1. **Initialize** - Set initial conditions for the population
2. **Calculate event rates** - $R_j$ for each event types depending on state of the system
3. **Calculate auxiliary variables** - for each state component $i$
\begin{align}
\mu_i &= \sum_j v_{ij} R_j \\
\sigma_j^2 &= \sum_j v_{ij}^2 R_j
\end{align}
4. **Select highest order event** - for each state component $i$, denote the rate of this event as $g_i$
5. **Calculate time step**
$$ \tau = min_i \left( min\left( \frac{max\left( \frac{\epsilon X_i}{g_i}, 1 \right)}{|\mu_i|} , \frac{max\left( \frac{\epsilon X_i}{g_i}, 1 \right)^2}{\sigma_j^2} \right) \right) $$
6. **Monte-Carlo** - for each event type sample number of events occuring within the leap step $\tau$
7. **Update** - Update system state based on number of events
8. **Repeat** - Repeat steps 2-7 until some stopping criteria is met

Step 4. involves selecting the highest order event - this essentially is the "most important" event that each $i$ is involved in. For very complex systems this may not be an obvious thing to do and will require more finesse. We can see that aside from steps 3-5 this is the exact same scheme as the previous example.

There are other adaptive leaping schemes that one could use each with different pros and cons.

We can modify the code above to use this scheme via:

```python
# An implenetation of the Gillespie algorithm
# With adaptive tau-leaping
# Applied to a pair of reactions:
# A + B -> AB
# AB -> A + B

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import poisson
%matplotlib inline

# Fix random seed for repeatability
np.random.seed(123)

###### Fix model parameters ######
N_A0 = 25     # Initial number of A molecules
N_B0 = 35     # Initial number of B molecules
N_AB0 = 5     # Initial number of AB molecules

rf = 2        # Forward reaction rate
rb = 1        # Backwards reaction rate

eps = 0.03    # Epsilon adaptive rate

steps = 25    # Number of reactions per trajectory
cycles = 100  # Number of trajectories iterated over

# Set up holder arrays
T = np.zeros((cycles, steps+1))
N_A = np.zeros((cycles, steps+1))
N_B = np.zeros((cycles, steps+1))
N_AB = np.zeros((cycles, steps+1))

# Store initial conditions
N_A[:,0] = N_A0
N_B[:,0] = N_B0
N_AB[:,0] = N_AB0

###### Main Code Loop ######
for i in range(cycles):
    for j in range(steps):
        # Calculate updated reaction rates
        Rf = rf * N_A[i,j] * N_B[i,j]
        Rb = rb * N_AB[i,j]

        # Calculate auxiliary variables
        mu_A = Rf - Rb
        mu_B = Rf - Rb
        mu_AB = Rb - Rf

        sig2_A = Rf + Rb
        sig2_B = Rf + Rb
        sig2_AB = Rf + Rb

        # Select highest order reactions
        g_A = Rf
        g_B = Rf
        g_AB = Rb

        # Caclulate internal maxima - taking care of divide by zero
        if g_A == 0:
            max_A = 1
        else:
            max_A = max(eps*N_A[i,j]/g_A,1)

        if g_B == 0:
            max_B = 1
        else:
            max_B = max(eps*N_B[i,j]/g_B, 1)

        if g_AB == 0:
            max_AB = 1
        else:
            max_AB = max(eps*N_AB[i,j]/g_AB, 1)

        # Calculate minima for each component
        min_A = min(max_A / abs(mu_A), max_A**2 / sig2_A)
        min_B = min(max_B / abs(mu_B), max_B**2 / sig2_B)
        min_AB = min(max_AB / abs(mu_AB), max_AB**2 / sig2_AB)

        # Select tau leap size
        leap = min(min_A, min_B, min_AB)

        # Calculate number of reactions by type
        uf = np.random.random()
        ub = np.random.random()

        Nf = poisson.ppf(uf, Rf*leap)
        Nb = poisson.ppf(ub, Rb*leap)

        # Apply limits to prevent negative population
        Limitf = min(N_A[i,j], N_B[i,j])
        Limitb = N_AB[i,j]

        Nf = min(Nf, Limitf)
        Nb = min(Nb, Limitb)

        # Update populations and times
        N_A[i,j+1] = N_A[i,j] + Nb - Nf
        N_B[i,j+1] = N_B[i,j] + Nb - Nf
        N_AB[i,j+1] = N_AB[i,j] + Nf - Nb
        T[i,j+1] = T[i,j] + leap

# Calculate an average trajectory plot
ave_steps = 100
T_max = T.max()

# Set up average array holders
T_ave = np.linspace(0,T_max,ave_steps+1)
N_A_ave = np.zeros(ave_steps+1)
N_B_ave = np.zeros(ave_steps+1)
N_AB_ave = np.zeros(ave_steps+1)

N_A_ave[0] = N_A0
N_B_ave[0] = N_B0
N_AB_ave[0] = N_AB0

# Pass over average array entries
for i in range(1, ave_steps+1):
    tmax = T_ave[i]
    A_sum = 0
    B_sum = 0
    AB_sum = 0
    t_count = 0

    # Pass over each trajectory and step therein
    for j in range(cycles):
        for k in range(steps):
            if T[j,k] <= tmax and T[j,k+1] > tmax:
                t_count += 1
                A_sum += N_A[j,k]
                B_sum += N_B[j,k]
                AB_sum += N_AB[j,k]

    # Caclulate average - taking care if no samples observed
    if t_count == 0:
        N_A_ave[i] = N_A_ave[i-1]
        N_B_ave[i] = N_B_ave[i-1]
        N_AB_ave[i] = N_AB_ave[i-1]
    else:
        N_A_ave[i] = A_sum / t_count
        N_B_ave[i] = B_sum / t_count
        N_AB_ave[i] = AB_sum / t_count

###### Plot Trajectories ######
fig, axs = plt.subplots(3, 1, figsize=(10,20))

axs[0].plot(T_ave, N_A_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[0].set_title('Number A Molecules')
axs[0].set_ylim((0,35))
axs[0].set_xlim((0,0.125))
axs[1].plot(T_ave, N_B_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[1].set_title('Number B Molecules')
axs[1].set_ylim((0,35))
axs[1].set_xlim((0,0.125))
axs[2].plot(T_ave, N_AB_ave, marker='', color='red', linewidth=1.9, alpha=0.9)
axs[2].set_title('Number AB Molecules')
axs[2].set_xlabel("Time")
axs[2].set_ylim((0,35))
axs[2].set_xlim((0,0.125))

for i in range(cycles):
    axs[0].plot(T[i,:], N_A[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[1].plot(T[i,:], N_B[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)
    axs[2].plot(T[i,:], N_AB[i,:], marker='', color='grey', linewidth=0.6, alpha=0.3)

plt.show()
```

As with the previous tau-leaping algorithm there the trajectories are noticably less exact than the original Gillespie formulation. However owing to the variable time step the trajectories do appear slightly less granular than in the previous tau-leaping formulation. Again the average trajectory is smoother than in the original method for (approximately) the same amount of run-time.

Looking at the time=0.025 distributions once again:

```python
time = 0.025
N_A_time = np.zeros(cycles)
N_B_time = np.zeros(cycles)
N_AB_time = np.zeros(cycles)

for i in range(cycles):
    for j in range(1, steps+1):
        if T[i,j] >= time and T[i,j-1] < time:
            N_A_time[i] = N_A[i,j]
            N_B_time[i] = N_B[i,j]
            N_AB_time[i] = N_AB[i,j]
    if T[i, steps] < time:
        N_A_time[i] = N_A[i, steps]
        N_B_time[i] = N_B[i, steps]
        N_AB_time[i] = N_AB[i, steps]

plt.hist(N_A_time, density=True, bins=np.arange(35), label="A", color='lightgrey')
plt.hist(N_B_time, density=True, bins=np.arange(35), label="B", color='dimgrey')
plt.hist(N_AB_time, density=True, bins=np.arange(35), label="AB", color='red')
plt.legend()
plt.show()
```

Again the distributions for a fixed time appear to have become less noisy.

In a small scale simple example such as this we would expect any "improvements" from a scheme like this to be minor, as we run more complicated examples we would expect a bigger performance differential.

## Conclusion
In this blog post we have seen 3 variations of the Gillespie algorithm: the original, tau-leaping and an adaptive tau leaping scheme. We have seen that the original variation produces exact simulations of a specified system, via tau leaping we have seen that we can approximate this and still get reasonable results in a quicker time. Which is important when dealing with more complicated and larger systems.

At this point we should also see the flexibility inherent in the Gillespie framework and why it has been applied in many different areas. We can also see that the algorithm is a "gateway" into agent-based schemes - instead of using a purely stochastic mechanism for selecting reaction types/times we could (for example) model individual molecules moving around in space and if they come within a certain radius of each other at a certain speed then a reaction occurs. This would turn the Gillespie algorithm into a full agent-based model for reaction kinetics (the benefit of doing this in most situations is likely slim to none however).


# Model Project

```python
## Importing packages for use in our model project

import numpy as np
from scipy import optimize
import sympy as sm
# Math-style printing
sm.init_printing(use_unicode=True)
import matplotlib.pyplot as plt
import ipywidgets as widgets
%matplotlib inline
import matplotlib.pyplot as plt
import ipywidgets as widgets
from IPython.display import Image
```

```python
#Running python file to define variables. The two first lines of the .py file are:
# import sympy as sm
# K = sm.symbols('K')
# The variables definitions are left in the .py file, as the cell would be quite large otherwise.
exec(open("Def_variables.py").read())
```

## The Green Solow Model

Our assignment builds on the original Solow model that was presented by Solow and Swan (1956). The standard Solow model is given by these equations:

\begin{align}
    & Q = F(K,BL) = K^{\alpha}BL^{1-\alpha}, \quad 1>\alpha>0 \quad \quad (1) \\
    & \dot{K} = sY - \delta K \quad \quad (2) \\
    & \dot{L} = nL \Rightarrow L=L_0 e^{nt} \quad \quad (3) \\
    & \dot{B} = g_BB \Rightarrow B=B_0 e^{g_Bt}, \quad 1>g_B>0 \quad \quad (4)
\end{align}

The first equation shows a simple Cobb-Douglas production function where the production of gross output, Q, is made through Capital, K, and effective labour, BL. It can be shown that the production function exhibits constant return to scale.
The second equation is describing the evolution of capital, where, s, is a fixed savings rate of net output, Y. From equation 2 we see that the capital is assumed to depreciate with a constant rate, $\delta$.
The third equation presents the evolution of workers over time. We notice that the evoulution is given by a constant growth rate, n.
Lastly, we see that the technological progress in the production of output is given by the exogenous growth rate, $g_B$.

In this assignment we will be working with an exstention of the original Solow model. Namely, we will work with the Green Solow model developed by William A. Brock and M. Scott Taylor (2004). In their model they add a function describing total emissions, E, given by:

\begin{align}
E=\Omega Q-\Omega A(Q,R) \quad \quad (5)
\end{align}

This simply shows that total emissions is given by the amount of pullution produced minus the amount of pollution abated.
$\Omega$ is the fraction of pollution arising from the production of gross output, Q. It is important to notice that it is assumed that every unit of output generates the same fraction of pollution. Further, the equation shows that abatement exists in the economy. $(A(Q,R))$. The abatement has a negative effect on the total amount of emissions. Hence, abatement reduces the amount of pollution released into the atmosphere.

In the Green Solow model it is assumed that the abatement function is a concave function that is increasing with gross output, Q, and the economy's effort at abatement, R. Hence, abatement has a diminishing effect on total emissions.

When considering the Green Solow model there must be a lower amount available for consumption and investment due to the inclusion of abatement. We define this net output (income) as Y:

\begin{align}
Y=Q-R=(1-\theta)Q \quad \quad (6)
\end{align}

We can rewrite this in order to understand how emissions are being affected in the Green Solow model.

\begin{align}
    &E=\Omega (Q-A(Q,R)) \quad \quad (7) \\
    &E=\Omega Q(1-A(1,\frac{R}{Q}) \quad \quad (8) \\
    &E=\Omega Q \alpha(\theta), \quad \theta \equiv \frac{R}{Q} \quad \alpha(\theta) \equiv 1-A(1,\theta) \quad \quad (9) \\
\end{align}

We notice that the emissions are affected by the scale of economic activity, Q. Further, we notice that the abatement intensity, $\theta$ is assumed constant. There are more underlying mechanism in the Green Solow model such as the development in the fraction $\Omega$ that we won't go into detail with due to the focus of this assignment.

We are able to derive the model in output per effective worker since the production function has constant returns to scale. This implies that:

\begin{align}
Q=K^\alpha (BL)^{1-\alpha} \Rightarrow q=k^\alpha \quad q\equiv \frac{Q}{BL},\quad k\equiv \frac{K}{BL} \quad \quad (10)
\end{align}

Using the above definitons we are now able to calculate the transition equation for capital per effective worker. When using the second equation and **the definitons just given** we get the following transition equation for capital per effective worker:

\begin{align}
\dot{k}=s(1-\theta)k^\alpha-k(\delta+g_B+n) \quad \quad (11)
\end{align}

It can be shown that the growth rate for total emissions is given by:

\begin{align}
\frac{\dot{E}}{E}=g_E+\alpha \frac{\dot{k}}{k}, \quad; g_E\equiv g_B+n-g_A \quad \quad (12)
\end{align}

When substituting for $\frac{\dot{k}}{k}$ we get:

\begin{align}
\frac{\dot{E}}{E}=\alpha s(1-\theta)k^{\alpha-1}-[\alpha(\delta+g_B+n)-g_E] \quad \quad (13)
\end{align}

When considering this equation we notice that the emissions will peak ($\dot{E}=0$) when $\alpha s(1-\theta)k^{\alpha-1}=[\alpha(\delta+g_B+n)-g_E]$

**Sustainable growth**
Brock and Taylor (2004) define sustainable growth as a situation where the growth rate of net output per capita is positive, while the growth rate of total emissions is negative

\begin{align}
    \dot{y}>0 \quad \wedge \quad \dot{E}<0 \quad \quad (14)
\end{align}

It is possible to establish conditions that will secure that sustainable growth occurs within the Green Solow model.

From the equations defined in the Green Solow model it can be shown that the growth rate of y must be given by:

\begin{align}
    \frac{\dot{y}}{y} = g_B+\alpha \frac{\dot{k}}{k} \quad \quad (16)
\end{align}

Brock and Taylor also present a growth for total emission, which is given by:

\begin{align}
    \frac{\dot{E}}{E} = g_E +\alpha \frac{\dot{k}}{k}, \quad g_E \equiv g_B+n-g_A  \quad \quad (17)
\end{align}

In steady state $\dot{k}=0$. When applying this we get the following growth rates on the balanced growth path:

\begin{align}
    & \frac{\dot{y}}{y}=g_B  \qquad , \qquad  \frac{\dot{E}}{E}=g_E \quad \quad (18)
\end{align}

We can thereby conclude that sustainable growth is secured  when

\begin{align}
    & g_B > 0 \qquad \wedge \qquad  \underbrace{g_A}_{\text{Technique effect}} > \underbrace{g_B + n}_{\text{Scale effect}} \quad \quad (19)
\end{align}

We note that technological growth in the production of net output is positive by assumption. However, from the second condition we see that the technological growth in abatement must be larger than the technological growth in the production of gross output plus the population growth rate. The term $n+g_B$ can be defined as the scale effect, as this measures the rate at which aggregate gross output grows. The term $g_A$ is defined as the technological progress in abatement, which reflects the technique effect.
We will use these definitions in the end of this project in order to secure the understanding of how an Environmental Kuznets Curve EKC relationship can potentially arise from the Green Solow model

## 1. Steady State of The Green Solow Model

Firstly we want to find the steady state of the Green Solow Model. In steady state capital does not change over time meaning that $\dot{k}$ is equal to zero. We do this by setting the transition equation (as it is stated above) equal to zero and solving for capital, k.

```python
# Stating the transition equation and setting it equal to zero

# Standard per effective capita production function
f=k**alpha

# We define the transition equation as a sympy equation.
tc=sm.Eq(0,(s*(1-theta)*k**alpha-(delta+g_B+n)*k))

# Solving the transition equation (tc) for k using the sympy solve function.
tcc=sm.solve(tc,k)
# Display the steady state value for k
display(tcc)

```

### 1.1  Analytical solution
By stating the parameters and some realistic examples of parameter values we can find a numerical value for capital i the steady state.

```python
# Converting our tcc sympy function into a standard
# python function, making it possible to calculate a numerical value solution.
tc_func = sm.lambdify((s,g_B,n,delta,alpha,theta),tcc)

# Adding numerical values to our parameters to calculate the steady state value.
# It is crucial that the ordering of the variables align with the lambdified function ordering of variables.
tc_func(0.25,0.015,0.01,0.035,0.25,0.05)
```

In steady state capital per effective worker is equal to 6.26.

### 1.2 Numerical Solution

We also try to solve for capital in the steady state using numerical optimization.

```python
#Running python file to define parameter values. The two first lines of the .py file are:
# s = 0.25
# g_B = 0.015
# The value definitions are left in the .py file, as the cell would be quite large otherwise.

exec(open("Def_values.py").read())

# Defining our steady state as a lambda function
obj_kss = lambda k: 0 - (s*(1-theta)*k**alpha-(delta+g_B+n)*k)

# optimize.newton uses a numerical solution to find a numerical value solution to our steady state
# We do not give the algorithm any of the derivatives, and therefore the standard methods from the optimize.newton are used.
result = optimize.newton(obj_kss,x0=1)
result
```

The numerical solution is as expected the same as the analytical. Our methods seems to be robust.

### 1.3 Transition Diagram

We now want to illustrate steady state in a transition diagram. The diagram shows how capital evolves over time and reaches steady state at the intersect.

First we define the functions in the transitions diagram: investment pr. worker and the break even functions.

```python
# Defining the investment per worker function, which calculates the saving/investment locus.
# The function has 4 inputs. K is the vector to calculate values on, theta, alpha and s are parameters.
# The parameters can be left out, but in order to make it interactive we keep them as inputs in the function.
def invest_per_work2(k, theta,alpha,s):
    return (s*(1-theta)*k**alpha)

# The break even equation does not depend on any of the parameters we have set for our widget.
# No need to include them in the inputs.
# The point where the two functions intersect is the steady state value of k.
def break_even(k):
    return (delta+g_B+n)*k
```

```python
# Defining the k values of the function. This is used as inputs to calculate the investment per worker and break even.
x_vec = np.linspace(0,50,50)
```

The following code set up the structure of the diagram.

```python
def transition_diagram2(theta,alpha,s):

    fig = plt.figure(figsize=(7,4)) # create the figure
    plt.xlabel('k') # label x-axis
    investment=invest_per_work2(x_vec,theta,alpha,s) # calculate the investments per worker using the function
    break_even_cap=break_even(x_vec) # calculate the break even equation for the x-values
    plt.title('Green solow transition diagram') #Title
    ax = fig.add_subplot(1,1,1) # Include many plots
    ax.plot(break_even_cap, label='$(\delta+g_B+n)\cdot k$'); # plot and label
    ax.plot(investment, label='$s \cdot (1-\\theta)\cdot k^\\alpha $'); # plot and label
    ax.set_xlim([0,50]) # fixed x range
    ax.set_yticklabels([]) #remove y-labes
    plt.legend() # plot the labels in the legend
```

We add sliders to our diagram, so it is clear what impact the abatement intensity, savings and $\alpha$ has on the steady state.

```python
# Widgets - all our widgets have the same min and max. Starting values differ according to the default values stated in
# our model description earlier, they can also be found in the .py file with values.

# Call diagram with widgets
widgets.interact(transition_diagram2,

    # widgets for all the inputs in the function transition_diagram2
    theta=widgets.FloatSlider(description="$\\theta$", min=0, max=1, step=0.01, value=0.05),
    alpha=widgets.FloatSlider(description="$\\alpha$", min=0, max=1, step=0.01, value=0.25),
    s=widgets.FloatSlider(description="$s$", min=0, max=1, step=0.01, value=0.25),

);
```

    interactive(children=(FloatSlider(value=0.05, description='$\\theta$', max=1.0, step=0.01), FloatSlider(value=…

The intersection of the two curves illustrates the steady state. It is evident from the figure that if the entire gross product is spent on abatament, i.e. $\theta=1$, then the economy will implode, since there is no accumulation of capital. The lower the abatement intensity the higher the level of capital will be in steady state. However, a low level of abatement may cause irreversible damage to the environment making the economy implode before the the steady state is reached.
A larger saving also increase the capital in the steady state. This is logical since a higher savings rate implies a higher accumulation of capital.

## 2. Emissions in the Green Solow Model

### 2.1 Peak of Emmisions
The Green Solow Model also predicts when the peak of emission occur depending on the abatement intensity, $\theta$. In the following,  we solve for k, at the peak of emissions like we did with the transitions equation. We use the emission euqation (13) for this. As emissions over time does not change at the peak we set $\dot{E}$ equal zero.
We choose only to use an analytical solution when solving for k at the peak of emissions.

```python
# Running python file defining variables to remove the previously set parameter values
exec(open("Def_variables.py").read())
```

```python
# Stating the emision equation and setting it equal to zero. We state is a sympy function.
max_emis=sm.Eq(0,(alpha*s*(1-theta)*k**(alpha-1)-(alpha*(delta+g_B+n)-(g_B+n-g_A))))

# Solving the emission equation (max_emis) for k using sympy.solve.
max_emis_k=sm.solve(max_emis,k)
max_emis_k

```

The above expression is capital at the peak of emmisions.

```python
# Converting our maximum emissions sympy function (max_emis_func) into a lambda function, making it possible to
# calculate a numerical value solution.
max_emis_func = sm.lambdify((s,g_B,n,delta,alpha,theta,g_A),max_emis_k)

# Adding numerical values to our parameters
max_emis_func(0.25,0.015,0.01,0.035,0.25,0.05,0.03)
```

At the peak of emmisions capital is equal to 4.27.

### 2.2 The Environmental Kuznets Curve

The Environmental Kuznets Curve (EKC) describes a bell-shaped relationship between total emissions and income per capita, depicted in the figure below. The original Kuznets curve was formalized by Simon Kuznets (1955) as a bell-shaped relationship between income inequality and income per capita. In the early 1990s the idea behind the original Kuznets curve was translated to describe a relationship between total emissions and income per capita, thereby given the name "The Environmental Kuznets Curve". The figure of the EKC shows that as income per capita increases, in the early stages of economic development, when income per capita is low, emissions will increase as well. However, at a certain point when income per capita gets sufficiently high a turning point is reached. After this turning point income per capita will continue to rise, but emissions will start to decline.

```python
# We show an image Environmental Kuznets Curve. The image is imported and displayed. An obvious extension would be to calculate
# the Kuznets Curve by ourself. This is however quite complicated.
Image(filename="Kuznetz_Curve.png")
```

We would like to illustrate how the Environmental Kuznetz Curve is derived however. First we define functions to use in the plot.

```python
# Running python file defining variables to remove the previously set parameter values
exec(open("Def_values.py").read())
```

```python
# Calculate the change of emissions for different k values. K vector to calculate values from, other variables are included
# as inputs for interactive purposes.
def ekc(k, theta,s):
    return (alpha*s*(1-theta)*k**(alpha-1))

#Defining g_E
g_E=g_B+n-g_A

# Calculate the turning point of the emissions
def turning_point(alpha):
    return alpha*(delta+g_B+n)-(g_E)

# Calculate the steady state value of emissions at the steady state level of capital
def steady_state(alpha):
    return alpha*(delta+g_B+n)

# To plot intersections between the change of emissions and the two curves defined above, we define a algorithm to
# find the position (index) on the x-values array of the closest value to the two horizontal curves.
def find_nearest(array, values):
    indices = np.abs(np.subtract.outer(array, values)).argmin(0)
    return indices

# to avoid dividing by zero, we define another x-vector (k-vector) that does not start in zero.
x_vec2 = np.linspace(0.01,200,1000)

```

```python
# Defining the k values of the function
x_vec2 = np.linspace(0.01,200,1000)
```

We define our diagram to show how the Environmental Kuznetz Curve is derived

```python
# two inputs in the diagram to include in the widget-slides
def ekc_diagram(theta,s):
    fig = plt.figure(figsize=(7,4)) # create the figure
    plt.xlabel('k') # label x-axis
    ekc_calc=ekc(x_vec2, theta,s) # calculate the change in ekc using the function
    steady_state_val=steady_state(alpha) # calculate the emissions at the steady state value of k
    turning_point_val=turning_point(alpha) # calculate the emissions at the turning point level
    plt.title('Generating Environmental Kuznets Curve') #title
    ax = fig.add_subplot(1,1,1) # Include many plots
    # steady state horizontal line
    plt.axhline(y=steady_state_val, color='r', linestyle='-', label="$\\alpha \cdot(\delta+g_B+n)$")
    # turning point horizontal line
    plt.axhline(y=turning_point_val, color='y', linestyle='-', label="$\\alpha \cdot(\delta+g_B+n)-g_E$")
    # plot the change in emissions
    ax.plot(ekc_calc, label='$\\alpha \cdot s \cdot(1-\\theta)\cdot k^{\\alpha-1}$');
    # Find the x-value of intersection with steady state
    steady_state_x=find_nearest(ekc_calc,steady_state_val)
    # Find the x-value of intersection with turning point
    turning_point_x=find_nearest(ekc_calc,turning_point_val)
    # plot the intersection
    ax.plot([steady_state_x], [steady_state_val], 'o', color='k')
    # plot the intersection
    ax.plot([turning_point_x], [turning_point_val], 'o', color='k')
    # annotate above the intersection
    ax.annotate('Point B', xy=(steady_state_x, steady_state_val), xytext=(steady_state_x, steady_state_val+0.001),
               )
    # annotate above the intersection
    ax.annotate('Point T', xy=(turning_point_x, turning_point_val), xytext=(turning_point_x, turning_point_val+0.001),
               )
    ax.set_xlim([0,200]) # fixed x range
    ax.set_ylim([0,0.05]) # fixed y range
    ax.set_yticklabels([]) # no labels
    ax.set_xticklabels([]) # no labels
    plt.legend() # legend
```

We call our predefined function ekc_diagram and make it interactive!

```python
# Widgets - all our widgets have the same min and max. Starting values differ according to the default values stated in
# our model description earlier, they can also be found in the .py file with values.

#call diagram
widgets.interact(ekc_diagram,

    # widgets for all the inputs in the function ekc_diagram
    theta=widgets.FloatSlider(description="$\\theta$", min=0.005, max=1, step=0.01, value=0.05),
    s=widgets.FloatSlider(description="$s$", min=0.005, max=1, step=0.01, value=0.25),

);
```

    interactive(children=(FloatSlider(value=0.05, description='$\\theta$', max=1.0, min=0.005, step=0.01), FloatSl…

To see how the Green Solow model can trace out an EKC relationship, we use equation (11) and (17). From these equations it is possible to link the dynamics of capital accumulation with the evolution of total emissions. In the figure above we have capital per effective worker on the horizontal axis, while we have the rates of change of $\alpha$ times capital per effective worker ($\alpha \frac{\dot{k}}{k}$) and the flow of emissions ($\dot{E}/E$) on the vertical axis.
We refer to the down-sloping function $\alpha s (1-\theta)k^{\alpha - 1}$ as the savings locus, as it is shifted to the left and right when s goes down or up respectively. The savings locus will start at infinity and converge towards 0 as capital per effective worker increases. Hence, the rates of change for emissions will be high at a low k, and the rates of change for emissions will decrease when k increases. The same goes for capital per effective worker. As the savings locus is downward sloping, it must be the case that it intersects the two points T and B. Point T refers to the turning point for emissions, and point B refers to the steady state value for capital per effective worker. To verify that T and B represent these two points we note from equation (17) that when emissions peak ($\dot{E} = 0$) then

\begin{align}
    \frac{\dot{E}}{E} = g_E +\alpha (s(1-\theta)k^{\alpha-1}-(\delta+g_B+n)) = 0 => \alpha s(1-\theta)k^{\alpha-1} = \alpha(\delta+g_B+n) -g_E  \quad \quad (20)
\end{align}

This is what happens at point T. From equation (20) it is apparent that to the left of point T the growth rate of emissions will be positive and to the right of point T the growth rate of emissions will be negative. Hence, point T must represent a turning point for total emissions. Looking at the horizontal line of $\alpha (\delta + g_B + n)$ we see that the vertical distance between the two curves must be $\alpha \frac{\dot{k}}{k}$. This is seen as

\begin{align*}
    \alpha\frac{\dot{k}}{k}=\alpha s(1-\theta)k^{\alpha-1}- \alpha (\delta+g_B+n) \quad \quad (21)
\end{align*}

In steady state $\dot{k}=0$ such that

\begin{align*}
    \alpha\frac{\dot{k}}{k}=\alpha s(1-\theta)k^{\alpha-1}- \alpha (\delta+g_B+n) =0 \Rightarrow \alpha s(1-\theta)k^{\alpha-1} = \alpha (\delta+g_B+n) \quad \quad (22)
\end{align*}

And we therefore conclude that point B represents the steady state level for capital per effective worker. It must be the case that k is rising to the left of B but falling to the right of B. We see that when k is small there must be a rapid growth in capital per effective worker but this growth rate falls as k approaches its steady state level. This dynamic is transferred to the growth dynamics of emissions, due to $g_E$ being constant in equation (17).

The EKC relationship described above is a combination of unsustainable and sustainable growth. In the early stages of economic development a country will exhibit unsustainable growth, that is $g_A<g_B+n$. Such that the scale effect dominates the technique effect, representing increasing emissions as income increases. This is generated by a low amount of capital, reflecting a high marginal product of capital. This leads to increased savings and investments thereby increasing income. As there is diminishing returns to capital, the growth rate of income will fall as the economy approaches its steady state level. As the growth in abatement technology is constant it will surpass the growth in income, generating the sustainable situation where the technique effect dominates the scale effect ($g_A>g_B+n$). Hence income will continue to rise but total emissions will start to decline. The EKC is not generated by increased awareness towards the environment, which would have been reflected by an increased abatement intensity ($\theta$). It is only generated by the forces between the growth rates.
When deriving the above results, it was assumed that growth was sustainable such that T lied to the left of B ($g_E<0$), implying that the steady state level for capital per effective worker was reached after emissions peaked. However, a EKC will not necessarily arise even in the sustainable case. If the initial capital stock lies between point T and B, then there will be ever declining emissions, since the turning point (T) has been surpassed.
It is also possible that growth is not sustainable ($g_E>0$). If growth is not sustainable then T would lie to the right of B. In this scenario emissions will continue to rise towards infinity, but as capital per effective worker approaches its steady state level the growth rate of total emissions will fall. However, in the balanced growth path emissions will continue to rise.
As the above analysis shows there exists a turning point for total emissions (under appropriate conditions). It is however not the case that the level of total emissions, nor income per capita will be unique at this turning point. To verify this consider two economies with an identical level of capital per effective worker that both are at $k(T)$. If one of the economies has a lower working population, but a more effective one, then income per capita will differ at the turning point. The same reasoning can explain why the peak level of emissions may differ. A larger economy will have higher emissions at any time, as they naturally pollute more, but might have the same capital per effective worker.


# Experiments on Latent ODEs
**Project in Special Topics in Research - Year 1 Term 4**

**Author: Maria Eloisa Ventura**

As motivated by my previous work on interpolating physiological signals or any sensor data, I chose to further study **Latent ODEs** (based on the work of Chen et al. (2018) and Rubanova et al. (2019)) and test the models perform when dealing with various types of data.

Please note that Yulia Rubanova's [Latent ODE implementation](https://github.com/YuliaRubanova/latent_ode) was used in the experiments performed.

## Introduction

Continuous-time models with a latent state defined at all times were introduced recently, and been applied to address the problem of missing data and irregular sampling. In 2018, Chen et al. introduced a generative latent function time-series model that represents each time series by a latent trajectory. **Each trajectory is determined from a local initial state, $z_{t_0}$, and a global set of latent dynamics shared across all time series.** Given observation times $t_0, t_1, \ldots , t_N$ and an initial state $z_{t_0}$, an ODE solver produces $z_{t_1}, \ldots, z_{t_N}$ which describe the latent state at each observation. They formally defined the generative model formally through the following sampling procedure:
\begin{equation}
\mathbf{z}_{t_0} \sim p(\mathbf{z}_{t_0})
\end{equation}

\begin{equation}
\mathbf{z}_{t_1}, \mathbf{z}_{t_2}, \ldots, \mathbf{z}_{t_N} = \operatorname{ODESolve}(\mathbf{z}_{t_0}, f, \theta_f, t_0, t_1, \ldots , t_N)
\end{equation}

\begin{equation}
\operatorname{each}\ \ \mathbf{x}_{t_i} \sim p(\mathbf{x}|\mathbf{z}_{t_i}\theta_{\mathbf{x}})
\end{equation}

Function $f$, which they parametrized using a neural net, is a time-invariant function that takes the value $\mathbf{z}$ at the current time step and outputs the gradient: $\frac{\partial z}{\partial t}=f(\mathbf{z}(t),\theta_f)$. Due to the time-invariant property of $f$, the entire latent trajectory is uniquely defined given any latent state $\mathbf{z}(t)$. As a consequence, the latent trajectory can be used to make predictions forwards or backwards in time. The figure below illustrates the model proposed by Chen et al., where they have a variational autoencoder (RNN as the encoder and ODE Solve for the decoder) with sequence-valued observations.

Computation graph of the latent ODE model by Chen et al. (2018)

In 2019, Rubanova et al. introduced a modification of the latent ODE architecture by using ODE-RNN instead of RNN as the recognition network (as in figure below). The only difference between the ODE-RNN and the standard RNN is that they considered the pre-activations which evolve according to an ODE between observations.

Computation graph of the latent ODE model by Rubanova et al. (2019)

Since Latent ODEs use Neural ODEs to represent latent dynamics, it is expected that its performance is also affected by Neural ODEs limitations. **One main factor that affects this is the smoothness of the solution**, and this is the scope of this notebook. We try to introduce various perturbations to the data such as introducing noise and forcing other discontinuities and investigate model performance by looking at signal reconstructions.

## Experiment Overview
### *Model Parameters and Training*
For all the experiments, we used the Seq2Seq Latent ODE model architecture introduced by Rubanova et al. where we have the ODE-RNN as the encoder and ODE solver as the decoder. We tested the model on a toy dataset which is composed of variations of periodic trajectories (see next section for details) with variable frequency and the same amplitude. Each dataset has 1000 trajectories and each has 100 timepoints. In some of the experiments, we performed subsampling of the toy data, and we attempt to resonctruct the full set of 100 points. For each set of trajectories, we trained two Latent ODE models having 5 and 10 latent dimensions, and all were trained for 50 epochs. For the rest of the notebook, we refer to LatentODE(5) and LatentODE(10) as the model with 5 latent states and 10 latent states, respectively.

**NOTE:** As mentioned previously, we used Rubanova's implementation of the Latent ODE model, and we made modifications to the input signals to the model. Also, we performed all our experiments using a computer with Intel Xeon CPU running at 2.10GHz using 256 GB of RAM.

### *Description of Experiments Performed*
The different experiments performed in this notebook focused on these types of periodic data:
* Signals with noise
* Signals with discontinuities
* Signals with missing values at fixed timepoints

The scope of this notebook is limited to looking at the performance of the LatentODE(5) model and LatentODE(10) model on the signals mentioned above, and not on hyperparameter tuning of Rubanova's Latent ODE model.

### *Expected Results*

For each experiment, we produce the following plots:
* Reconstructions of 3 sample signals from the dataset ($\mathbf{x}(t)$)
* Latent trajectories ($\mathbf{z}(t)$)
* Visualization of the vector field
* Sample trajectories from the data space

The information above is presented in the following manner: the signal reconstructions are shown in the top row, while we have the latent trajectories, vector field and sample trajectories from the data are shown in the bottom row. ALL the results in this notebook are arranged in the same way.

## Experiment 1: Signals with Noise

#### Results

Before introducing noise, we consider the result for modeling a simple sinusoid without noise. In this ideal case where our input to the model are smooth periodic signals, we visualize the results for epoch 1 to 50.

Sample trajectories for smooth signals from Latent ODE model with 5 latent dimensions

Sample trajectories for smooth signals from Latent ODE model with 10 latent dimensions

> **Observations**
> * From the reconstructions above, we see that the by the end of 50 epochs, the both models (LatentODE(5) and LatentODE(10)) were able to accurately capture the periodicity of the input signals. However, in terms of the amplitude of the reconstructed signal, we see that both models were struggling to reconstruct the full amplitude of the Sample 1 (plot in top row, middle column), which has higher frequency compared to Sample 0 and 2.
> * The results for the LatentODE(10) model shows that the model is able to capture the variations in the signal amplitude or the periodicity of the signals in earlier epochs compared to the LatentODE(5) model.
> * Unlike the LatentODE(10) model, the LatentODE(5) model was not able to accurately capture that the sample signals start at value of zero at $t=0$.
> * The LatentODE(10) model reconstructions had smoother peaks/valleys, closer to the form of the original signal, compared to LatentODE(5) reconstructions which had sharper peaks/valleys.

Now, we try to introduce varying levels of noise to the continuous periodic signal (see [Experiment 1 - Adding Noise Notebook](Experiment%201%20-%20Adding%20Noise.ipynb) for the code used in model training). The results from the experiments at the end of 50 epochs are summarized below. The first column describes the noise level introduced to the signal (0, 0.25 and 0.5), while the second and third columns outlines the model results for 5 latent dimensions and 10 latent dimensions, respectively.

 Results for interpolating the noisy signals

|Noise level|latent dim = 5|latent dim = 10|
|-|-|-|
|0|||
|0.25|||
|0.5|||

> **Observations**
> * Despite the presence of noise, reconstructions from both LatentODE(5) and LatentODE(10) models are still smooth.
> * For both LatentODE(5) and LatentODE(10) models, we see that the introduction of higher level of noise resulted to poorer quality of reconstruction. However, reconstructions from the LatentODE(10) model still followed the periodicity of the signal.
> * The addition of noise heavily affected the amplitude of the reconstructed signal. At 0.25 noise level, we see that both models were able to capture similar frequency as the original signal but the amplitude of the reconstructions were much lower.
> * At 0.5 noise level, the LatentODE(5) model produced reconstructions with lower frequency but the those reconstructions from the LatentODE(10) are still similar with the original.
> * The model with lower latent dimension seems to perform more poorly on the boundaries compared to the model with more latent dimensions. For all noise levels considered (0, 0.25, 0.5), the LatentODE(5) model was not able to accurately capture that the sample signals start at value of zero at $t=0$. On the other hand, the LatentODE(10) model was no longer able to capture that the input signal starts from 0 only at noise level of 0.5.

#### Discussion

When we talk about physiological signals (or any other sensor data), we usually deal with noise and discontinuities in the data which could come from multiple sources. In some cases, we want to capture the underlying signal without the unwanted noise -- this could mean eliminating the randomness in the dataset and uncovering the smooth signal as the objective. When we expect a smooth input signal but there's noise present, the Latent ODE model can be used to reconstruct the signal. From the experiment above (Experiment 1), we found that the model with more latent dimensions (LatentODE(10)) was able to capture the dynamics of the noisy signal better. Also, regardless of the noise level (0, 0.25 and 0.5), we saw that both models struggled to obtain the correct amplitudes for the higher frequency signals. Given the same signal length (in our case, fixed at 100 data points), the peaks and valleys of higher frequency signals are sharper compared to lower frequency signals. And since latent ODEs are able to model the dynamics of smoother signals better, the decrease in reconstruction quality for higher frequency signals is expected.

## Experiment 2: Signals with Discontinuities

### *Version 1*

In the first variation of signal, we introduced the a jump discontinuity as shown below:

```python
import numpy as np
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
```

    Matplotlib created a temporary config/cache directory at /tmp/matplotlib-hcsrnzcg because the default path (/home/mventura/.cache/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.

```python
sample_periodic_trajectory = np.sin(2*np.pi*np.linspace(0, 1, 100)*5)
plt.figure(figsize=[6, 2])
plt.title('Continuous Signal')
plt.plot(sample_periodic_trajectory)
plt.show()
```

```python
def add_jump_discontinuity(traj_continuous):
    shift = np.random.randint(
        int(len(traj_continuous)/50),
        int(len(traj_continuous)/20),
        np.random.randint(1, 5))
    disc_index = np.random.randint(
        max(shift),
        len(traj_continuous)-max(shift),
        size=len(shift))
    traj_discont = traj_continuous
    for i, s in zip(disc_index, shift):
        traj_discont[i:] = traj_discont[i-s:len(traj_discont)-s]
    return traj_discont
```

```python
plt.figure(figsize=[6, 2])
plt.title('Sample of a signal with discontinuity')
plt.plot(add_jump_discontinuity(sample_periodic_trajectory))
plt.show()
```

#### Results

We generated different sets of 1000 signals like the ones plotted above and used them as input to our LatentODE(5) and Latent ODE(10) models (see [Experiment 2 - Adding Discontinuity 1 Notebook](Experiment%202%20-%20Adding%20Discontinuity%201.ipynb) for the code used in model training). In addition to reconstructing the complete signal with discontinuity, we also try to reconstruct a version of the signal with gaps (i.e., 50% of signal is missing).

The results are summarized below. The first column describes the type of input signal (with discontinuity) we are trying to reconstruct: (1) a complete signal (0% missing) and (2) a signal with missing data (50% missing),  while the second and third columns outlines the model results for 5 latent dimensions and 10 latent dimensions, respectively.

 Results for interpolating periodic signals with discontinuity (Version 1)

|Signal type|latent dim = 5|latent dim = 10|
|-|-|-|
|Signal with discontinuity
(0% missing,
0 noise)|||
|Signal with discontinuity
(50% missing,
0 noise)|||

> **Observations**
> * Despite the presence of discontinuity, reconstructions from both LatentODE(5) and LatentODE(10) models are still smooth and periodic.
> * In Sample 0, we can see that the discontinuity seems like it produced a spurious peak that's lower in frequency (i.e., last peak looks wider than the first two), and this "change" in frequency in the signal itself was not reconstructed properly by both models.
> * Even though the type of discontinuity introduced did not affect the maximum and minimum value of the input signals, the reconstructions seem to have peaks having different amplitudes as observed in all Sample 0, 1 and 2 for the case when the input signals are complete (0% missing).
> * In the absence of some data points (50% missing), we can see from the result of both models that they were unable to recover the min value of 0 and max value of 2 for some of the peaks (e.g., see 2nd peak in Sample 2).
> Comparing the output reconstruction for signals with 0% missing vs 50% missing, we find that the alignment of peaks were better for the case when the input signals were missing (e.g., see 3rd peak in Sample 1).

### *Version 2*

In the second variation of jump discontinuity, we add a scaled step function to a portion of the periodic signal. We can imagine that this type of discontinuity can happen when the sensor that collects the signal is not properly placed/installed. For example, consider a photoplethysmogram (PPG) present in smart watches. It is a sensor that detects volumetric changes in the blood, which is the basis for the calculation of heart rate and respiration rate. When the smart watch gets dislodged, we may observe a DC bias in the collected PPG signal.

For our experiment, we generate the discontinuity as follows.

```python
def add_step_disconitnuity(traj_continuous, amp):
    shift = np.random.randint(
        int(len(traj_continuous)/5),
        int(len(traj_continuous)/2),
        np.random.randint(1, 3))
    disc_index = np.random.randint(
        len(traj_continuous)-max(shift),
        size=len(shift))
    traj_discont = traj_continuous
    for i, s in zip(disc_index, shift):
        traj_discont[i:i+s] = traj_discont[i:i+s]+amp*0.75
    return traj_discont
```

```python
sample_periodic_trajectory = np.sin(2*np.pi*np.linspace(0, 1, 100)*5)
plt.figure(figsize=[6, 2])
plt.title('Sample of a signal with discontinuity')
plt.plot(add_step_disconitnuity(sample_periodic_trajectory, 1))
plt.show()
```

#### Results

We again generated different sets of 1000 signals like the ones plotted above and used them as input to our LatentODE(5) and Latent ODE(10) models (see [Experiment 2 - Adding Discontinuity 2 Notebook](Experiment%202%20-%20Adding%20Discontinuity%202.ipynb) for the code used in model training). As in the previous section, we also try to reconstruct a version of the signal with gaps (i.e., 50% of signal is missing).

The results are summarized below. The first column describes the type of input signal (with discontinuity) we are trying to reconstruct: (1) a complete signal (0% missing) and (2) a signal with missing data (50% missing),  while the second and third columns outlines the model results for 5 latent dimensions and 10 latent dimensions, respectively.

 Results for interpolating periodic signals with discontinuity (Version 2)

|Signal type|latent dim = 5|latent dim = 10|
|-|-|-|
|Signal with discontinuity
(0% missing,
0 noise)|||
|Signal with discontinuity
(50% missing,
0 noise)|||

> **Observations**
> * Unlike Version 1 where the type of discontinuity did not affect the observed min and max values of the signal, the addition of the step functio/DC bias to a portion of the input signal led to reconstructions with poorer quality.
> * Although some parts of the input signal have sharp peaks due to the discontinuity, results from both models were still smooth.
> * For LatentODE(5), we see that the frequency of the reconstructed signals are of lower frequency compared to the original signal for both 0% and 50% missing values. However, this was not the case for LatentODE(10) which was still able to capture similar frequency (same number of peaks in original and reconstructed signal).
> * For Sample 0, the 2nd and 3rd peaks from the original signal were the parts that were "raised" due to the added step function. Interestingly, it seems that LatentODE(5) treated these as a single peak.

#### Discussion
However, there could also be cases where we want to model the discontinuities present in the data as they can provide additional information about the system being described (e.g., person wearing the device), or we simply have to model signals with inherent discontinuities. From our results above, we can separate the effects seen in two ways: (1) adding a discontinuity which led to an apparent change in frequency for some parts of the data (e.g., some peaks becoming wider than others), and (2) adding a discontinuity which led to change in amplitude of some parts of the signal. In the first one, we saw that the LatentODE models struggled to capture this "change in frequency" within the signal as the reconstructions seems like they had constant frequency all for all time points. This implies the inability of the model accommodate or learn changes in dynamics within a trajectory. In second case where we forced some parts of the signal to have higher value, we also seemed to have also introduced changes in frequency in the system (see Experiment 2 V2 for example). Again, these were not captured well by both LatentODE(5) and LatentODE(10) models.

## Experiment 3: Signals with missing values at fixed timepoints

In this set of experiments, we look at how the models perform when the discontinuities/missing values are not introduced randomly inthe signal. Here, we assume that there are specific time points where no data was collected. For example, physiological signals from smart watches could be missing for a few hours every night when the person chooses not to wear the device during sleep. Here, we try to remove missing data from fixed time points which would represent the scenario mentioned. The process of generating the signal is shown below.

```python
def add_missing_values(traj_continuous):
    random_missing_pct = int(len(traj_continuous)/2)
    n_chunks = 3
    min_size_gap = int(random_missing_pct/n_chunks)
    missing_val_len = ([int(min_size_gap)]*(n_chunks-1)
                        + [min_size_gap+random_missing_pct%n_chunks])
    traj_discont = traj_continuous
    for c in range(n_chunks):
        gap_start = int((c+0.5) * int(len(traj_continuous)/(n_chunks)))
        traj_discont[gap_start:gap_start+missing_val_len[c]] = 0
    return traj_discont
```

```python
sample_periodic_trajectory = np.sin(2*np.pi*np.linspace(0, 1, 100)*5)+2
plt.figure(figsize=[6, 2])
plt.title('Sample of a signal with discontinuity')
plt.plot(add_missing_values(sample_periodic_trajectory))
plt.show()
```

#### Results

We again generated different sets of 1000 signals like the ones plotted above. The results are summarized below.

Results for interpolating the signals with missing data at fixed timepoints for the Latent ODE model with 10 latent dimensions

|latent dim = 5|latent dim = 10|
|-|-|
|||

> **Observations**
> * For LatentODE(5), we see that the frequency of the reconstructed signals are of lower frequency compared to the original signal. The reconstructed signal are smooth but they are almost aperiodic. No peaks were observed.
> * For LatentODE(10), we still have smooth reconstructions that are similar in frequency with the original. However, the amplitude of the reconstructions are much smaller than the original.
> * Comparing the latent space, we can see that the trajectories from the LatentODE(5) model are also of lower frequency compared to those from the LatentODE(10).

#### Discussion
In the last experiment, we found that by simply increasing the latent dimension, the Latent ODE model was able to capture the periodicity of the original signal even in the absence of data points at regular intervals. In this example where we considered a signal sampled from a smooth sinusoid, we were able to capture the periodicity but not the correct amplitudes, which could be a huge limitation when we're going to deal with more complex signals such as those that contain more than one frequency component.

## Summary

Overall, we found that the Latent ODE model becomes more robust to noise and other perturbations when the number of latent dimensions is increased. From the various experiments, we fed various signals to the models and we found that parts of the signal that were not smooth were not properly reconstructed, and changes in dynamics (e.g., some parts of signal have higher frequency than others) within the signal were not acccurately captured by the latent space trajectories of the LatentODE model.

## Other Experiments
* **Investigating reconstruction quality for various frequencies.** The limitation found regarding the poor reconstruction quality for high frequency signals can be further explored and its relationship with latent dimension. If we can imagine the number of latent dimensions being related the representation power of Latent ODEs, we can check whether adding more dimensions can improve the reconstructions.
* **More training epochs for signals with higher noise levels.** Since the final results presented here were only from 50 epochs, we want to explore whether training the LatentODE(5) for more epochs could allow the model to obtain the original signal better since we could get more periodic latent space trajectories.
* **Extending the model to multivariate time series.** It would be interesting to see whether the results that we found here could also be observed when we consider multivariate time series (even with a toy dataset), especially when the characteristics of the component signal in the multivariate time series vary.
* **Exploring the representation of latent space trajectories for real multivariate datasets.** Representing multiple signals having varied characteristics (e.g., range of values, frequencies, % of missing values) modeled using LatentODE in a multivariate setting would be interesting to explore. In terms of reconstruction, we expect the quality will vary across the component signals but it would be useful to explore effective the representations are when we apply the model in a classification problem.

## References

1. Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2019, December 14). Neural ordinary differential equations. arXiv.org. Retrieved December 2021, from https://arxiv.org/abs/1806.07366
1. Rubanova, Y., Chen, R. T. Q., & Duvenaud, D. (2019, July 8). Latent odes for irregularly-sampled time series. arXiv.org. Retrieved December 2021, from https://arxiv.org/abs/1907.03907
1. Rubanova, Y., (n.d.). Yuliarubanova/latent_ode: Code for "latent odes for irregularly-sampled time series" paper. GitHub. Retrieved December 17, 2021, from https://github.com/YuliaRubanova/latent_ode