# Neural Sequence Distance Embeddings

[](https://colab.research.google.com/github/gcorso/neural_seed/blob/master/tutorial/Neural_SEED.ipynb)

The improvement of data-dependent heuristics and representation for biological sequences is a critical requirement to fully exploit the recent technological and scientific advancements for human microbiome analysis. This notebook presents Neural Sequence Distance Embeddings (Neural SEED), a novel framework to embed biological sequences in geometric vector spaces that unifies recently proposed approaches. We demonstrate its capacity by presenting different ways it can be applied to the tasks of edit distance approximation, closest string retrieval, hierarchical clustering and multiple sequence alignment. In particular, the hyperbolic space is shown to be a key component to embed biological sequences and obtain competitive heuristics. Benchmarked with common bioinformatics and machine learning baselines, the proposed approaches display significant accuracy and/or runtime improvements on real-world datasets formed by sequences from samples of the human microbiome.

Figure 1: On the left, a diagram of the Neural SEED underlying idea: embed sequences in vector spaces preserving the edit distance between them. On the right, an example of the hierarchical clustering produced on the Poincarè disk from the P53 tumour protein from 30 different organisms.

## Introduction and Motivation

### Motivation

Dysfunctions of the human microbiome (Morgan & Huttenhower, 2012) have been linked to many serious diseases ranging from diabetes and antibiotic resistance to inflammatory bowel disease. Its usage as a biomarker for the diagnosis and as a target for interventions is a very active area of research. Thanks to the advances in sequencing technologies, modern analysis relies on sequence reads that can be generated relatively quickly. However, to fully exploit the potential of these advances for personalised medicine, the computational methods used in the analysis have to significantly improve in terms of speed and accuracy.

Figure 2: Traditional approach to the analysis of the 16S rRNA sequences from the microbiome.

### Problem

While the number of available biological sequences has been growing exponentially over the past decades, most of the problems related to string matching have not been addressed by the recent advances in machine learning. Classical algorithms are data-independent and, therefore, cannot exploit the low-dimensional manifold assumption that characterises real-world data. Exploiting the available data to produce data-dependent heuristics and representations would greatly accelerate large-scale analyses that are critical to microbiome analysis and other biological research.

Unlike most tasks in computer vision and NLP, string matching problems are typically formulated as combinatorial optimisation problems. These discrete formulations do not fit well with the current deep learning approaches causing these problems to be left mostly unexplored by the community. Current supervised learning methods also suffer from the lack of labels that characterises many downstream applications with biological sequences. On the other hand, common self-supervised learning approaches, very successful in NLP, are less effective in the biological context where relations tend to be per-sequence rather than per-token (McDermott et al. 2021).

### Neural Sequence Distance Embedding

In this notebook, we present Neural Sequence Distance Embeddings (Neural SEED), a general framework to produce representations for biological sequences where the distance in the embedding space is correlated with the evolutionary distance between sequences. This control over the geometric interpretation of the representation space enables the use of geometrical data processing tools for the analysis of the spectrum of sequences.

Figure 3: The key idea of Neural SEED is to learn an encoder function that preserves distances between the sequence and vector space.

Examining the task of embedding sequences to preserve the edit distance reveals the importance of data-dependent approaches and of using a geometry that matches well the underlying distribution in the data analysed. For biological datasets, that have an implicit hierarchical structure given by evolution, the hyperbolic space provides significant improvement.

We show the potential of the framework by analysing three fundamental tasks in bioinformatics: closest string retrieval, hierarchical clustering and multiple sequence alignment. For all tasks, relatively simple unsupervised approaches using Neural SEED encoders significantly outperform data-independent heuristics in terms of accuracy and/or runtime. In the paper (preprint will be available soon) and the [complete repository](https://github.com/gcorso/neural_seed) we also present more complex geometrical approaches to hierarchical clustering and multiple sequence alignment.

## 2. Analysis

To improve readability and limit the size of the notebook we make use of some subroutines in the [official repository](https://github.com/gcorso/neural_seed) for the research project. The code in the notebook is our best effort to convey the promising application of hyperbolic geometry to this novel research direction and how `geomstats` helps to achieve it.

Install and import the required packages.

```python
!pip3 install geomstats
!apt install clustalw
!pip install biopython
!pip install python-Levenshtein
!pip install Cython
!pip install networkx
!pip install tqdm
!pip install gdown
!pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
!git clone https://github.com/gcorso/neural_seed.git
import os
os.chdir("neural_seed")
!cd hierarchical_clustering/relaxed/mst; python setup.py build_ext --inplace; cd ../unionfind; python setup.py build_ext --inplace; cd ..; cd ..; cd ..;
os.environ['GEOMSTATS_BACKEND'] = 'pytorch'
```

```python
import torch
import os
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import time
from geomstats.geometry.poincare_ball import PoincareBall

from edit_distance.train import load_edit_distance_dataset
from util.data_handling.data_loader import get_dataloaders
from util.ml_and_math.loss_functions import AverageMeter
```

    INFO: Using pytorch backend

### Dataset description

As microbiome analysis is one of the most critical applications where the methods presented could be applied, we chose to use a dataset containing a portion of the 16S rRNA gene widely used in the biological literature to analyse microbiome diversity. Qiita (Clemente et al. 2015) contains more than 6M sequences of up to 152 bp that cover the V4 hyper-variable region collected from skin, saliva and faeces samples of uncontacted Amerindians. The full dataset can be found on the [European Nucleotide Archive](https://www.ebi.ac.uk/ena/browser/text-search?query=ERP008799), but, in this notebook, we will only use a subset of a few tens of thousands that have been preprocessed and labelled with pairwise distances. We also provide results on the RT988 dataset (Zheng et al. 2019), another dataset of 16S rRNA that contains slightly longer sequences (up to 465 bp).

```python
!gdown --id 1yZTOYrnYdW9qRrwHSO5eRc8rYIPEVtY2 # for edit distance approximation
!gdown --id 1hQSHR-oeuS9bDVE6ABHS0SoI4xk3zPnB # for closest string retrieval
!gdown --id 1ukvUI6gUTbcBZEzTVDpskrX8e6EHqVQg # for hierarchical clustering
```

    Downloading...
    From: https://drive.google.com/uc?id=1yZTOYrnYdW9qRrwHSO5eRc8rYIPEVtY2
    To: /content/string_matching/edit_qiita_large.pkl
    218MB [00:03, 68.6MB/s]
    Downloading...
    From: https://drive.google.com/uc?id=1hQSHR-oeuS9bDVE6ABHS0SoI4xk3zPnB
    To: /content/string_matching/closest_qiita_large.pkl
    2.44MB [00:00, 38.3MB/s]
    Downloading...
    From: https://drive.google.com/uc?id=1ukvUI6gUTbcBZEzTVDpskrX8e6EHqVQg
    To: /content/string_matching/hc_qiita_large_extr.pkl
    806MB [00:08, 90.6MB/s]

### Edit distance approximation

**Edit distance**  The task of finding the distance or similarity between two strings and the related task of global alignment lies at the foundation of bioinformatics. Due to the resemblance with the biological mutation process, the edit distance and its variants are typically used to measure similarity between sequences. Given two string $s_1$ and $s_2$, their edit distance $ED(s_1, s_2)$ is defined as the minimum number of insertions, deletions or substitutions needed to transform $s_1$ in $s_2$. We always deal with the classical edit distance where the same weight is given to every operation, however, all the approaches developed can be applied to any distance function of choice.

**Task and loss function** As represented in Figure 3, the task is to learn an encoding function $f$ such that given any pair of sequences from the domain of interest $s_1$ and $s_2$:
\begin{equation}ED(s_1, s_2) \approx n \; d(f(s_1), f(s_2)) \end{equation}

where $n$ is the maximum sequence length and $d$ is a distance function over the vector space. In practice this is enforced in the model by minimising the mean squared error between the actual and the predicted edit distance. To make the results more interpretable and comparable across different datasets, we report results using \% RMSE defined as:
\begin{equation}
\text{% RMSE}(f, S) = \frac{100}{n} \, \sqrt{L(f, S)} = \frac{100}{n} \, \sqrt{\sum_{s_1, s_2 \in S} (ED(s_1, s_2) - n \; d(f(s_1), f(s_2)))^2}
\end{equation}

which can be interpreted as an approximate average error in the distance prediction as a percentage of the size of the sequences.

In this notebook, we only show the code to run a simple linear layer on the sequence which, in the hyperbolic space, already gives particularly good results. Later we will also report results for more complex models whose implementation can be found in the [Neural SEED repository](https://github.com/gcorso/neural_seed).

```python
class LinearEncoder(nn.Module):
    """  Linear model which simply flattens the sequence and applies a linear transformation. """

    def __init__(self, len_sequence, embedding_size, alphabet_size=4):
        super(LinearEncoder, self).__init__()
        self.encoder = nn.Linear(in_features=alphabet_size * len_sequence,
                                 out_features=embedding_size)

    def forward(self, sequence):
        # flatten sequence and apply layer
        B = sequence.shape[0]
        sequence = sequence.reshape(B, -1)
        emb = self.encoder(sequence)
        return emb

class PairEmbeddingDistance(nn.Module):
    """ Wrapper model for a general encoder, computes pairwise distances and applies projections """

    def __init__(self, embedding_model, embedding_size, scaling=False):
        super(PairEmbeddingDistance, self).__init__()
        self.hyperbolic_metric = PoincareBall(embedding_size).metric.dist
        self.embedding_model = embedding_model
        self.radius = nn.Parameter(torch.Tensor([1e-2]), requires_grad=True)
        self.scaling = nn.Parameter(torch.Tensor([1.]), requires_grad=True)

    def normalize_embeddings(self, embeddings):
        """ Project embeddings to an hypersphere of a certain radius """
        min_scale = 1e-7
        max_scale = 1 - 1e-3
        return F.normalize(embeddings, p=2, dim=1) * self.radius.clamp_min(min_scale).clamp_max(max_scale)

    def encode(self, sequence):
        """ Use embedding model and normalization to encode some sequences. """
        enc_sequence = self.embedding_model(sequence)
        enc_sequence = self.normalize_embeddings(enc_sequence)
        return enc_sequence

    def forward(self, sequence):
        # flatten couples
        (B, _, N, _) = sequence.shape
        sequence = sequence.reshape(2 * B, N, -1)

        # encode sequences
        enc_sequence = self.encode(sequence)

        # compute distances
        enc_sequence = enc_sequence.reshape(B, 2, -1)
        distance = self.hyperbolic_metric(enc_sequence[:, 0], enc_sequence[:, 1])
        distance = distance * self.scaling

        return distance
```

General training and evaluation routines used to train the models:

```python

def train(model, loader, optimizer, loss, device):
    avg_loss = AverageMeter()
    model.train()

    for sequences, labels in loader:
        # move examples to right device
        sequences, labels = sequences.to(device), labels.to(device)

        # forward propagation
        optimizer.zero_grad()
        output = model(sequences)

        # loss and backpropagation
        loss_train = loss(output, labels)
        loss_train.backward()
        optimizer.step()

        # keep track of average loss
        avg_loss.update(loss_train.data.item(), sequences.shape[0])

    return avg_loss.avg

def test(model, loader, loss, device):
    avg_loss = AverageMeter()
    model.eval()

    for sequences, labels in loader:
        # move examples to right device
        sequences, labels = sequences.to(device), labels.to(device)

        # forward propagation and loss computation
        output = model(sequences)
        loss_val = loss(output, labels).data.item()
        avg_loss.update(loss_val, sequences.shape[0])

    return avg_loss.avg
```

The linear model is trained on 7000 sequences (+700 of validation) and tested on 1500 different sequences:

```python
EMBEDDING_SIZE = 128

device = 'cuda' if torch.cuda.is_available() else 'cpu'
torch.manual_seed(2021)
if device == 'cuda':
    torch.cuda.manual_seed(2021)

# load data
datasets = load_edit_distance_dataset('./edit_qiita_large.pkl')
loaders = get_dataloaders(datasets, batch_size=128, workers=1)

# model, optimizer and loss
encoder = LinearEncoder(152, EMBEDDING_SIZE)
model = PairEmbeddingDistance(embedding_model=encoder, embedding_size=EMBEDDING_SIZE)
model.to(device)

optimizer = optim.Adam(model.parameters(), lr=0.001)
loss = nn.MSELoss()

# training
for epoch in range(0, 21):
    t = time.time()
    loss_train = train(model, loaders['train'], optimizer, loss, device)
    loss_val = test(model, loaders['val'], loss, device)

    # print progress
    if epoch % 5 == 0:
        print('Epoch: {:02d}'.format(epoch),
              'loss_train: {:.6f}'.format(loss_train),
              'loss_val: {:.6f}'.format(loss_val),
              'time: {:.4f}s'.format(time.time() - t))

# testing
for dset in loaders.keys():
    avg_loss = test(model, loaders[dset], loss, device)
    print('Final results {}: loss = {:.6f}'.format(dset, avg_loss))

```

    Epoch: 00 loss_train: 0.021631 loss_val: 0.000938 time: 30.0685s
    Epoch: 10 loss_train: 0.000500 loss_val: 0.000741 time: 28.9802s
    Epoch: 20 loss_train: 0.000444 loss_val: 0.000728 time: 28.9375s
    Epoch: 30 loss_train: 0.000422 loss_val: 0.000682 time: 28.9818s
    Epoch: 40 loss_train: 0.000406 loss_val: 0.000653 time: 28.8732s
    Epoch: 50 loss_train: 0.000401 loss_val: 0.000655 time: 29.5673s
    Final results train: loss = 0.000397
    Final results val: loss = 0.000655
    Final results test: loss = 0.000680

Therefore, our linear model after only 50 epochs has a $\% RMSE \approx 2.6$ that, as we will see, is significantly better than any data-independent baseline.

### Closest string retrieval

This task consists of finding the sequence that is closest to a given query among a large number of reference sequences and is very commonly used to classify sequences. Given a set of reference strings $R$ and a set of queries $Q$, the task is to identify the string $r_q \in R$ that minimises $ED(r_q, q)$ for each $q \in Q$. This task is performed in an unsupervised setting using models trained for edit distance approximation. Therefore, given a pretrained encoder $f$, its prediction is taken to be the string $r_q \in R$ that minimises $d(f(r_q), f(q))$ for each $q \in Q$. This allows for sublinear retrieval (via locality-sensitive hashing or other data structures) which is critical in real-world applications where databases can have billions of reference sequences. As performance measures, we report the top-1, top-5 and top-10 scores, where top-$k$ indicates the percentage of times the model ranks the closest string within its top-$k$ predictions.

```python
from closest_string.test import closest_string_testing

closest_string_testing(encoder_model=model, data_path='./closest_qiita_large.pkl',
                       batch_size=128, device=device, distance='hyperbolic')
```

    Results: accuracy 0.441 0.537 0.583 0.620 0.642 0.660 0.677 0.698 0.712 0.725
    Top1: 0.441  Top5: 0.642  Top10: 0.725
    Total time elapsed: 0.4419s

Evaluated on a dataset composed of 1000 reference and 1000 query sequences (disjoint from the edit distance training set) the simple model we trained is capable of detecting the closest sequence correctly 44\% of the time and in approximately 3/4 of the cases it places the real closest sequence in its top-10 choices.

### Hierarchical clustering

Hierarchical clustering (HC) consists of constructing a hierarchy over clusters of data by defining a tree with internal points corresponding to clusters and leaves to datapoints. The goodness of the tree can be measured using Dasgupta's cost (Dasgupta 2016).

One simple approach to use Neural SEED to speed up hierarchical clustering is similar to the one adopted in the previous section: estimate the pairwise distance matrix with a model pretrained for *edit distance approximation* and then use the matrix as the basis for classical agglomerative clustering algorithms (e.g. Single, Average and Complete Linkage). The computational cost to generate the matrix goes from $O(N^2M^2)$ to $O(N(M+N))$ and by using optimisations like locality-sensitive hashing the clustering itself can be accelerated.

The following code computes the pairwise distance matrix and then runs a series of agglomerative clustering heuristics (Single, Average, Complete and Ward Linkage) on it.

```python
from hierarchical_clustering.unsupervised.unsupervised import hierarchical_clustering_testing

hierarchical_clustering_testing(encoder_model=model, data_path='./hc_qiita_large_extr.pkl',
                                batch_size=128, device=device, distance='hyperbolic')
```

    Hierarchical torch.Size([10000, 152])
    {'single': {'DC': 335039492911.7103}, 'complete': {'DC': 334477330814.3947}, 'average': {'DC': 333673018932.05225}, 'ward': {'DC': 334313576211.0263}}

An alternative approach to performing hierarchical clustering we propose uses the continuous relaxation of Dasgupta's cost (Chami et al. 2020) to embed sequences in the hyperbolic space. In comparison to Chami et al. (2020), we show that it is possible to significantly decrease the number of pairwise distances required by directly mapping the sequences.
This allows to considerably speed up the construction especially when dealing with a large number of sequences without requiring any pretrained model. Figure 1 shows an example of this approach when applied to a small dataset of proteins and the code for it is in the Neural SEED repository.

### Multiple Sequence Alignment

Multiple Sequence Alignment is another very common task in bioinformatics and there are several ways of using Neural SEED to accelerate heuristics. The most commonly used programs such as the Clustal series and MUSCLE are based on a phylogenetic tree estimation phase from the pairwise distances which produces a guide tree, which is then used to guide a progressive alignment phase.

In Clustal algorithm for MSA on a subset of RT988 of 1200 sequences, the construction of the distance matrix and the tree takes 99\% of the total running time (the rest takes 24s out of 35 minutes). Therefore, one obvious improvement that Neural SEED can bring is to speed up this phase using the hierarchical clustering techniques seen in the previous section.

The following code uses the model pretrained for edit distance to approximate the neighbour joining tree construction and the runs clustalw using that guide tree:

```python
from multiple_alignment.guide_tree.guide_tree import approximate_guide_trees

# performs neighbour joining algorithm on the estimate of the pairwise distance matrix
approximate_guide_trees(encoder_model=model, dataset=datasets['test'],
                        batch_size=128, device=device, distance='hyperbolic')

# Command line clustalw using the tree generated with the previous command.
# The substitution matrix and gap penalties are set to simulate the classical edit distance used to train the model
!clustalw -infile="sequences.fasta" -dnamatrix=multiple_alignment/guide_tree/matrix.txt -transweight=0 -type='DNA' -gapopen=1 -gapext=1 -gapdist=10000 -usetree='njtree.dnd'  | grep 'Alignment Score'
```

    Alignment Score -12042858

An alternative method we propose for the MSA uses an autoencoder to convert the Steiner string approximation problem in a continuous optimisation task. More details on this in our paper and repository.

### Central role of Geomstats

As we will show in the next section, the choice of the geometry of the embedding space is critical with Neural SEED. `geomstats` provides the geometric functions required for the methods presented (e.g.`metric.dist`, `metric.dist_pairwise` and `metric.closest_neighbor_index`) for the Poincaré Ball used in this notebook. Moreover, the large variety of other geometric spaces present (with the same interface) allows to quickly experiment to find the most appropriate one in every application domain of this approach.

Finally, Neural SEED provides representations that are better suited for human interaction than classical bioinformatics algorithms. While the functioning of the encoder might be hard to interpret, the embeddings produced can be plotted (directly or after dimensionality reduction) and intuitively interpreted as continuous space where distance reflects evolutionary separation. `geomstats` could be of critical importance also on this front.

`giotto-tda` was not used for this project.

## 3. Benchmark

In this section, we compare the Neural SEED approach to classical baseline alignment-free approaches such as k-mer and contrast the performance of neural models with different architectures and on different geometric spaces.

### Edit distance approximation

Figure 4: \% RMSE test set results on the Qiita and RT988 datasets. The first five models are the k-mer baselines and, in parentheses, we indicate the dimension of the embedding space. The remaining are encoder models trained with the Neural SEED framework and they all have an embedding space dimension equal to 128. - indicates that the model did not converge.

Figure 4 highlights the advantage provided by data-dependent methods when compared to the data-independent baseline approaches. Moreover, the results show that it is critical for the geometry of the embedding space to reflect the structure of the low dimensional manifold on which the data lies. In these biological datasets, there is an implicit hierarchical structure given by the evolution process which is well reflected by the *hyperbolic* plane. Thanks to this close correspondence, even relatively simple models like the linear regression and MLP perform very well with this distance function.

Figure 5: \% RMSE on Qiita dataset for a Transformer with different distance functions.

The clear benefit of using the hyperbolic space is evident when analysing the dimension required for the embedding space (Figure 5). In these experiments, we run the Transformer model tuned on the Qiita dataset with an embedding size of 128 on a range of dimensions. The hyperbolic space provides significantly more efficient embeddings, with the model reaching the 'elbow' at dimension 32 and matching the performance of the other spaces with dimension 128 with only 4 to 16. Given that the space to store the embedding and the time to compute distances between them scale linearly with the size of the space, this would provide a significant improvement in downstream tasks over other Neural SEED approaches.

**Running time** A critical step behind most of the algorithms analysed in the rest of the paper is the computation of the pairwise distance matrix of a set of sequences. Taking as an example the RT988 dataset (6700 sequences of length up to 465 bases), optimised C code computes on a CPU approximately 2700 pairwise distances per second and takes 2.5 hours for the whole matrix. In comparison, using a trained Neural SEED model, the same matrix can be approximated in 0.3-3s (similar value for the k-mer baseline) on the same CPU. The computational complexity for $N$ sequences of length $M$ is reduced from $O(N^2\; M^2)$ to $O(N(M + N))$ (assuming the model is linear w.r.t. the length and constant embedding size). The training process takes typically 0.5-3 hours on a GPU. However, in applications such as microbiome analysis, biologists typically analyse data coming from the same distribution (e.g. the 16S rRNA gene) for multiple individuals, therefore the initial cost would be significantly amortised.

### Closest string retrieval

Figure 6 shows that also in this task the data-dependent models outperform the baselines even when these operate on larger spaces. In terms of distance function, the *cosine* distance achieves performances on par with the *hyperbolic*. This can be explained by the fact that for a set of points on the same hypersphere, the ones with the smallest *cosine* or *hyperbolic* distance are the same. So the *cosine* distance is capable of providing good orderings of sequence similarity but inferior approximations of their distance.

Figure 6: Accuracy of different models in the *closest string retrieval* task on the Qiita dataset.

### Hierarchical clustering

The results (Figure 7) show that the difference in performance between the most expressive models and the round truth distances is not statistically significant. The *hyperbolic* space achieves the best performance and, although the relative difference between the methods is not large in terms of percentage Dasgupta's cost (but still statistically significant), it results in a large performance gap when these trees are used for tasks such as MSA. The total CPU time taken to construct the tree is reduced from more than 30 minutes to less than one in this dataset and the difference is significantly larger when scaling to datasets of more and longer sequences.

Figure 7: Average Linkage \% increase in Dasgupta's cost of Neural SEED models compared to the performance of clustering on the ground truth distances. Average Linkage was the best performing clustering heuristic across all models.

### Multiple Sequence Alignment

The results reported in Figure 8 show that the alignment scores obtained when using the Neural SEED heuristics with models such as GAT are not statistically different from those obtained with the ground truth distances. Most of the models show a relatively large variance in performance across different runs. This has positive and negative consequences: the alignment obtained using a single run may not be very accurate, but, by training an ensemble of models and applying each of them, we are likely to obtain a significantly better alignment than the one from the ground truth matrix while still only taking a fraction of the time.

Figure 8: Percentage change in the alignment cost (- alignment score) returned by Clustal when using the heuristics to generate the tree as opposed to using NJ on real distances. The alignment was done on 1.2k unseen sequences from the RT988 dataset.

## 4. Limitations and perspectives

### Limitations of the method presented

As mentioned in the introduction, we believe that the Neural SEED framework has the potential to be applied to numerous problems and, therefore, this project constitutes only an initial analysis of its geometrical properties and applications. Below we list some of the limitations of the current analysis and potential directions of research to cover them.

**Type of sequences** Both the datasets analysed consist of sequence reads of the same part of the genome. This is a very common set-up for sequence analysis (for example for microbiome analysis) and it is enabled by biotechnologies that can amplify and sequence certain parts of the genome selectively, but it is not ubiquitous. Shotgun metagenomics consists of sequencing random parts of the genome. This would, we believe, generate sequences lying on a low-dimensional manifold where the hierarchical relationship of evolution is combined with the relationship based on the specific position in the whole genome. Therefore, more complex geometries such as product spaces might be best suited.

**Type of labels** In this project, we work with edit distances between strings, these are very expensive when large scale analysis is required, but it is feasible to produce several thousand exact pairwise distance values from which the model can learn. For different definitions of distance, however, this might not be the case. If it is only feasible to determine which sequences are closest, then encoders can be trained using triplet loss and then most of the approaches presented would still apply. Future work could explore the robustness of this framework to inexact estimates of the distances as labels and whether Neural SEED models, once trained, could provide more accurate predictions than its labels.

**Architectures** Throughout the project we used models that have been shown to work well for other types of sequences and tasks. However, the correct inductive biases that models should have to perform SEED are likely to be different to the ones used for other tasks and even dependent on the type of distance it tries to preserve. Moreover, the capacity of the hyperbolic space could be further exploited using models that directly operate in the hyperbolic space (Peng et al. 2021).

**Self-supervised embeddings** One potential application of Neural SEED that was not explored in this project is the direct use of the embedding produced by Neural SEED for downstream tasks. This would enable the use of a wide range of geometric data processing tools for the analysis of biological sequences.

### Limitations of Geomstats

We did not find a way to use both `pytorch` and `numpy` backends *at the same time*, while in some cases in our experience it would have been useful.

### Proposed features of Geomstats

The documentation for the various manifolds could be improved with more detailed descriptions and an index with short summaries. Moreover, there are some classes like `PoincareBall` that we found in tutorials, but we could not find in the documentation. Finally, it would be great if more complex geometries such as product spaces were included.

## References

The preprint detailing all the approaches and related work will be available soon.

(Morgan & Huttenhower, 2012) Xochitl C Morgan and Curtis Huttenhower. Human microbiome analysis. PLoS Comput Biol, 2012.

(McDermott et al. 2021) Matthew McDermott, Brendan Yap, Peter Szolovits, and Marinka Zitnik. Rethinking relational encoding in language model: Pre-training for general sequences. arXiv preprint, 2021.

(Clemente et al. 2015) Jose C Clemente, Erica C Pehrsson, Martin J Blaser, Kuldip Sandhu, Zhan Gao, Bin Wang, Magda Magris, Glida Hidalgo, Monica Contreras, Oscar Noya-Alarcon, et al. ´The microbiome of uncontacted amerindians. Science advances, 2015.

(Zheng et al. 2019)Wei Zheng, Le Yang, Robert J Genco, Jean Wactawski-Wende, Michael Buck, and Yijun Sun. Sense: Siamese neural network for sequence embedding and alignment free comparison. Bioinformatics, 2019.

(Dasgupta 2016) Sanjoy Dasgupta. A cost function for similarity-based hierarchical clustering. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, 2016.

(Chami et al. 2020) Ines Chami, Albert Gu, Vaggos Chatziafratis, and Christopher Re. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering. Advances in Neural Information Processing Systems 33, 2020.


| 　 | 1 | 1, 1 | 1 | 2 |
| :--- | --- | --- | --- | --- |
| **2** | 　 | ■ | ■ | 　 |
| **1, 1** | ■ | 　 | 　 | ■ |
| **1, 1** | 　 | ■ | 　 | ■ |

```python
def len_lines(qs):
    cont = False
    result = []
    for q in qs:
        if q == 0:
            cont = False
        else:
            if cont:
                result[-1] += 1
            else:
                result.append(1)
            cont = True
    return result
```

```python
import dwavebinarycsp as dbc

row_constraints = [[2], [1, 1], [1, 1]]
col_constraints = [[1], [1, 1], [1], [2]]

csp = dbc.ConstraintSatisfactionProblem(dbc.BINARY)
for i, lengths in enumerate(row_constraints):
    csp.add_constraint(lambda *qs, lengths=lengths: len_lines(qs) == lengths, [f'q{i}{j}' for j in range(len(col_constraints))])
for j, lengths in enumerate(col_constraints):
    csp.add_constraint(lambda *qs, lengths=lengths: len_lines(qs) == lengths, [f'q{i}{j}' for i in range(len(row_constraints))])

bqm = dbc.stitch(csp)
bqm
```

    BinaryQuadraticModel({'q00': 0.0, 'q01': -4.0, 'q02': -2.0, 'q03': -2.0, 'q10': -4.0, 'q11': 0.0, 'q12': -2.0, 'q13': -4.0, 'q20': -4.0, 'q21': -4.0, 'q22': -2.0, 'q23': -4.0}, {('q01', 'q00'): -4.0, ('q02', 'q00'): 4.0, ('q02', 'q01'): -2.0, ('q03', 'q00'): -2.0, ('q03', 'q01'): 4.0, ('q03', 'q02'): -2.0, ('q11', 'q10'): 4.0, ('q12', 'q10'): -2.0, ('q12', 'q11'): -2.0, ('q13', 'q10'): 0.0, ('q13', 'q11'): -2.0, ('q13', 'q12'): 4.0, ('q21', 'q20'): 4.0, ('q22', 'q20'): -2.0, ('q22', 'q21'): -2.0, ('q23', 'q20'): 0.0, ('q23', 'q21'): -2.0, ('q23', 'q22'): 4.0, ('q10', 'q00'): 4.0, ('q20', 'q00'): 4.0, ('q20', 'q10'): 4.0, ('q11', 'q01'): 4.0, ('q21', 'q01'): -4.0, ('q21', 'q11'): 4.0, ('q12', 'q02'): 4.0, ('q22', 'q02'): 4.0, ('q22', 'q12'): 4.0, ('q13', 'q03'): 0.0, ('q23', 'q03'): 4.0, ('q23', 'q13'): 0.0}, 7.5, Vartype.BINARY)

```python
import sympy

expr = bqm.offset
for symb, coef in bqm.linear.items():
    expr += sympy.symbols(symb) * coef
for (symb1, symb2), coef in bqm.quadratic.items():
    expr += sympy.symbols(symb1) * sympy.symbols(symb2) * coef

sympy.init_printing()
expr
```

$$- 4.0 q_{00} q_{01} + 4.0 q_{00} q_{02} - 2.0 q_{00} q_{03} + 4.0 q_{00} q_{10} + 4.0 q_{00} q_{20} - 2.0 q_{01} q_{02} + 4.0 q_{01} q_{03} + 4.0 q_{01} q_{11} - 4.0 q_{01} q_{21} - 4.0 q_{01} - 2.0 q_{02} q_{03} + 4.0 q_{02} q_{12} + 4.0 q_{02} q_{22} - 2.0 q_{02} + 4.0 q_{03} q_{23} - 2.0 q_{03} + 4.0 q_{10} q_{11} - 2.0 q_{10} q_{12} + 4.0 q_{10} q_{20} - 4.0 q_{10} - 2.0 q_{11} q_{12} - 2.0 q_{11} q_{13} + 4.0 q_{11} q_{21} + 4.0 q_{12} q_{13} + 4.0 q_{12} q_{22} - 2.0 q_{12} - 4.0 q_{13} + 4.0 q_{20} q_{21} - 2.0 q_{20} q_{22} - 4.0 q_{20} - 2.0 q_{21} q_{22} - 2.0 q_{21} q_{23} - 4.0 q_{21} + 4.0 q_{22} q_{23} - 2.0 q_{22} - 4.0 q_{23} + 7.5$$

```python
from dwave.system.samplers import DWaveSampler
from dwave.system.composites import EmbeddingComposite

sampler = EmbeddingComposite(DWaveSampler())
response = sampler.sample(bqm, num_reads=100)
```

```python
import pandas as pd

pd.DataFrame([{'check': csp.check(sample), 'energy': energy, 'occurr': num_occurrences, 'chain': chain_break_fraction, **sample} for sample, energy, num_occurrences, chain_break_fraction in response.data()])
```

      chain
      check
      energy
      occurr
      q00
      q01
      q02
      q03
      q10
      q11
      q12
      q13
      q20
      q21
      q22
      q23

      0
      0.166667
      True
      -22.5
      31
      0
      1
      1
      0
      1
      0
      0
      1
      0
      1
      0
      1

      1
      0.083333
      True
      -22.5
      1
      0
      1
      1
      0
      1
      0
      0
      1
      0
      1
      0
      1

      2
      0.166667
      False
      -18.5
      19
      1
      1
      1
      0
      1
      0
      0
      1
      0
      1
      0
      1

      3
      0.166667
      False
      -18.5
      48
      1
      1
      1
      0
      1
      0
      0
      1
      0
      1
      0
      1

      4
      0.333333
      False
      -14.5
      1
      1
      1
      1
      0
      1
      0
      1
      1
      0
      1
      0
      1

```python
response.info['timing']
```

    {'total_real_time': 24086,
     'qpu_access_overhead_time': 721,
     'anneal_time_per_run': 20,
     'post_processing_overhead_time': 2119,
     'qpu_sampling_time': 16396,
     'readout_time_per_run': 123,
     'qpu_delay_time_per_sample': 21,
     'qpu_anneal_time_per_sample': 20,
     'total_post_processing_time': 2119,
     'qpu_programming_time': 7690,
     'run_time_chip': 16396,
     'qpu_access_time': 24086,
     'qpu_readout_time_per_sample': 123}

```python
sampler.properties['child_properties']['chip_id']
```

    'DW_2000Q_2_1'

```python
!pip list | grep dwave
```

    dwave-cloud-client     0.4.16
    dwave-neal             0.4.2
    dwave-networkx         0.6.6
    dwave-ocean-sdk        1.0.3
    dwave-qbsolv           0.2.9
    dwave-system           0.5.4
    dwavebinarycsp         0.0.9


# Count models

Counting is child's play, as easy as 1, 2, 3, right?

As usual, the answer's not so simple: some things are easy to count and some things aren't, making some numbers easier to predict than others. What do we do?

1. **Count what's available** as carefully as possible.
2. Build an appropriate **probability model** to predict likely outcomes.

To explore this, we'll look at 4 real examples:
* cases of chronic medical conditions,
* car crashes in Tennessee,
* births in the United States, and
* coughing in Spain.

We'll see how these relate to 3 fundamental **count models**:
* binomial models,
* Poisson models, and
* negative binomial models.

If time permits, we'll also talk about using the **Kolmogorov-Smirnov test** to compare observed and simulated samples. When we meet again, we'll see how to build better predictive models, namely **regression models for counts** that incorporate predictor variables.

All of this (and more!) is in Chapters 5 and 7 of my Manning book, [Regression: A friendly guide](https://www.manning.com/books/regression-a-friendly-guide); these 2 chapters will be added to the MEAP soon. If you just can't wait, this notebook and the relevant CSVs are available _now!_ in my [regression repo on github](https://github.com/mbrudd/regression) — please clone and submit comments!

## Imports and settings

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import binom
from scipy.stats import poisson
from scipy.stats import nbinom
from scipy.stats import kstest
import warnings
warnings.filterwarnings('ignore')
sns.set_theme()
plt.rcParams['figure.figsize'] = [15,6]
```

## Binomial models

### Chronic conditions by age

Here's a counting problem: given a group of people, how many have a chronic medical condition? Let's see what the 2009 [National Ambulatory Medical Care Survey (NAMCS)](https://www.cdc.gov/nchs/ahcd/index.htm) says:

```python
chronic = pd.read_csv("chronic.csv")
cohorts = pd.read_csv("chronic_cohorts.csv")
cohorts
```

      Age
      Total
      Sick
      Percentage

      0
      0
      796
      21
      0.026382

      1
      1
      505
      32
      0.063366

      2
      2
      320
      29
      0.090625

      3
      3
      258
      37
      0.143411

      4
      4
      299
      30
      0.100334

      ...
      ...
      ...
      ...
      ...

      96
      96
      13
      12
      0.923077

      97
      97
      11
      10
      0.909091

      98
      98
      6
      5
      0.833333

      99
      99
      2
      1
      0.500000

      100
      100
      6
      5
      0.833333

101 rows × 4 columns

In this dataset, _Sick_ means having at least one of the following conditions:

* arthritis,
* asthma,
* COPD,
* cancer,
* depression,
* diabetes,
* hyperlipidemia,
* hypertension,
* obesity,
* osteoporosis,
* cerebrovascular disease,
* chronic renal failure,
* congestive heart failure, or
* ischemic heart disease.

This is quite a list! The snippet shown corroborates what's expected: older people are more likely to have at least one of these conditions. For example, 30.4% of 25-year-olds have one and 81.43% of the 65-year-olds do:

```python
cohorts[ (cohorts["Age"]==25) | (cohorts["Age"]==65) ]
```

      Age
      Total
      Sick
      Percentage

      25
      25
      273
      83
      0.304029

      65
      65
      463
      377
      0.814255

These percentages are **empirical probabilities**. Based on them, what are the chances of having _exactly_ (a) 83 sick people out of 273 25-year-olds and (b) 377 sick people out of 463 65-year-olds? We need

### The binomial distribution

If you try something $n$ times with a chance $p$ of **success** on each **trial**, the number $Y$ of successes is a **random variable**: you don't know what the exact outcome will be until it happens, but you know the probability of each possible outcome. In fact, the probability of getting exactly $k$ successes is

$$P( \, Y = k \, ) ~ = ~ \binom{n}{k} \, p^{k} \, (1-p)^{n-k} \ , \quad \text{where} \quad \binom{n}{k} = \frac{ n! }{ k! (n-k)! } \quad .  $$

This is the **probability mass function (PMF)** for the **binomial distribution** $B(n,p)$. For a **binomial random variable** $Y \sim B(n,p)$,

$$\operatorname{E}\left( \, Y \, \right) ~ = ~ np \quad \text{and} \quad \operatorname{Var}( \, Y \, ) ~ = ~ np(1-p) \ .$$

> **Use the `binom` module from `scipy` (`import`ed above) to work with binomial distributions.**

25-year-olds in the NAMCS sample are like a **binomial experiment** with $n=273$ and $p=.304029$ ; the chance of observing exactly 83 successes (perverse lingo, I know!) is just over 5% :

```python
p_25 = cohorts["Percentage"][25]
binom.pmf(83,273,p_25)
```

    0.05243020970380286

65-year-olds are like a binomial experiment with 𝑛=463 and 𝑝=.814255 ; the chance of observing exactly 377 successes is just under 5% :

```python
p_65 = cohorts["Percentage"][65]
binom.pmf(377,463,p_65)
```

    0.047625769242453334

Don't be alarmed at these low probabilities — these are actually the most likely outcomes:

```python
fig, (ax1, ax2) = plt.subplots(1, 2)
fig.suptitle('Binomial models of chronic condition incidence')
x = np.arange(binom.ppf(0.01, 273, p_25),
              binom.ppf(0.99, 273, p_25))
ax1.plot(x, binom.pmf(x, 273, p_25), 'bo', ms=8)
ax1.vlines(x, 0, binom.pmf(x, 273, p_25), colors='b', lw=3, alpha=0.5)
ax1.set_xlabel('25-year-olds')
x = np.arange(binom.ppf(0.01, 463, p_65),
              binom.ppf(0.99, 463, p_65))
ax2.plot(x, binom.pmf(x, 463, p_65), 'bo', ms=8)
ax2.vlines(x, 0, binom.pmf(x, 463, p_65), colors='b', lw=3, alpha=0.5)
ax2.set_xlabel('65-year-olds');
```

It's no accident that these look like normal distributions — if $n$ is large and/or $p$ is close to $0.5$, $B(n,p)$ is approximately normal with mean $np$ and variance $np(1-p)$ :

$$ B(n,p) ~ \approx ~ N( np, np(1-p) ) \quad \text{for} \quad n \gg 1 \quad \text{or} \quad p \approx 0.5 \ .$$

Turn this around for some quick and dirty calculations. For example, $B(463,.814) \approx N( 377, 70)$, so there will usually be between $377 - 2\sqrt{70} \approx 360 $ and $377 + 2\sqrt{70} \approx 394$ successes, as you can easily check above!

Things are different if $n$ is small and/or $p$ is far from $0.5$. No worries, though — `binom.pmf` works fine either way.

### Binomial logistic regression

Instead of working separately with each age group — i.e., each **covariate class** — we should really construct a **binomial logistic regression model** to consolidate the age group percentages efficiently.

```python
sns.regplot(data=chronic,x="Age",
            y="Condition",
            logistic=True,
            scatter=False,
            ci=None,
            line_kws={"lw":"4"})
plt.xlim(-2,102)
plt.ylabel("Probability")
plt.plot( cohorts["Age"], cohorts["Percentage"],'.k');
```

This particular logistic model, plotted in blue, is **simple** — it involves only one predictor :

$$\log{ \left( \text{Odds of a condition} \right) } ~ = ~ -2.04 + .052*\text{Age} \ . $$

This model is really a family of binomial distributions, one for each covariate class; each probability is directly related to `Age`.

## Poisson models

### Monthly car crashes

Here's a different counting problem: how many traffic accidents are there each month where you live? For my home state of Tennessee, the [Department of Transportation provides the relevant data](https://www.tn.gov/safety/stats/crashdata.html):

```python
crashes = pd.read_csv("TDOT.csv")
crashes
```

      County
      Year
      January
      February
      March
      April
      May
      June
      July
      August
      September
      October
      November
      December

      0
      Anderson
      2010
      138
      113
      169
      169
      155
      152
      165
      176
      178
      169
      179
      152

      1
      Bedford
      2010
      71
      72
      77
      70
      87
      99
      90
      91
      86
      118
      105
      95

      2
      Benton
      2010
      21
      23
      27
      32
      34
      29
      16
      43
      31
      50
      29
      36

      3
      Bledsoe
      2010
      5
      1
      3
      3
      5
      1
      7
      4
      3
      6
      4
      3

      4
      Blount
      2010
      126
      159
      171
      145
      153
      139
      201
      264
      338
      267
      316
      269

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      1040
      Wayne
      2020
      23
      15
      19
      27
      18
      22
      20
      24
      23
      18
      22
      19

      1041
      Weakley
      2020
      39
      35
      40
      19
      28
      43
      37
      47
      54
      44
      33
      21

      1042
      White
      2020
      30
      31
      33
      21
      50
      32
      36
      46
      34
      29
      42
      43

      1043
      Williamson
      2020
      394
      424
      270
      170
      226
      304
      312
      310
      293
      405
      368
      377

      1044
      Wilson
      2020
      282
      268
      239
      143
      267
      258
      285
      295
      273
      306
      335
      299

1045 rows × 14 columns

```python
crashes["County"].nunique()
```

    95

Of these 95 counties, let's check out Meigs County (home of the ghost town [Cute, Tennessee](https://en.wikipedia.org/wiki/Cute,_Tennessee)) :

```python
meigs = crashes[crashes["County"]=="Meigs"]
meigs = meigs.drop("County",1)
meigs = meigs.melt( id_vars="Year", var_name="Month", value_name="Crashes" )
meigs
```

      Year
      Month
      Crashes

      0
      2010
      January
      11

      1
      2011
      January
      13

      2
      2012
      January
      12

      3
      2013
      January
      10

      4
      2014
      January
      16

      ...
      ...
      ...
      ...

      127
      2016
      December
      16

      128
      2017
      December
      18

      129
      2018
      December
      9

      130
      2019
      December
      11

      131
      2020
      December
      17

132 rows × 3 columns

```python
sns.histplot( data=meigs, x="Crashes", discrete=True)
plt.title("Monthly crashes in Meigs County, TN");
```

### The Poisson distribution

What's the relevant probability distribution? Let's see...

* There are a bunch of encounters between cars every month — lots of opportunities for accidents to occur.
* Most encounters don't result in an accident (thank goodness!), but accidents occur at a roughly constant rate per month.

This is a binomial experiment with a large number $n$ of trials, a small chance $p$ of "success" (an accident), and a roughly constant expected number $\lambda = np$ of successes per month, so that $\displaystyle{p = \frac{\lambda}{n}} \ll 1$. Letting $Y$ denote the number of accidents per month,

$$ \begin{align}
P( Y = k ) & ~ = ~ \frac{ n! }{ k! (n-k)! } \left( \frac{\lambda}{n} \right)^{k} \left( 1 - \frac{\lambda}{n} \right)^{(n-k)} \\
& ~ = ~ \frac{ n! }{ k! (n-k)! } \left( \frac{\lambda}{n} \right)^{k} \left( 1 - \frac{\lambda}{n} \right)^{n} \left( 1 - \frac{\lambda}{n} \right)^{-k} \\
& ~ \approx ~ \frac{ n! }{ k! (n-k)! } \left( \frac{\lambda}{n} \right)^{k} e^{-\lambda} \\
& ~ = ~ \frac{n (n-1) (n-2) \cdots (n-k+1) }{ n \cdot n \cdot n \cdots n } \ e^{-\lambda} \, \frac{\lambda^k}{ k! } \\
& ~ \approx ~ e^{-\lambda} \, \frac{\lambda^k}{ k! } \ .
\end{align} $$

Lo and behold, that's it! This is the PMF for the **Poisson distribution** with **rate** $\lambda$, denoted $\operatorname{Pois}(\lambda)$. The mean and variance of a **Poisson random variable** $Y \sim \operatorname{Pois}(\lambda)$ are _equal_:

$$\operatorname{E}\left( \, Y \, \right) ~ = ~ \operatorname{Var}( \, Y \, ) ~ = ~ \lambda \ .$$

> **Use the `poisson` module from `scipy` (`import`ed above) to work with Poisson distributions.**

### Simulating monthly crashes

Now you'll see why I picked Meigs County:

```python
meigs.agg( Mean = ("Crashes","mean"), Variance = ("Crashes","var") )
```

      Crashes

      Mean
      13.712121

      Variance
      15.198936

How well does the relevant Poisson distribution model crashes in Meigs County?

```python
meigs["Simulation"] = poisson.rvs(meigs["Crashes"].mean(),
                                  size=meigs.shape[0] )
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=meigs, x="Crashes", discrete=True, ax=axs[0])
axs[0].set_title("Observed crashes per month in Meigs County, TN")
sns.histplot(data=meigs, x="Simulation", discrete=True, ax=axs[1])
axs[1].set_title("Simulated crashes per month in Meigs County, TN");
```

### Monthly births

What happens if we count _births_ every month instead of car crashes? Let's look at recent data from the [National Vital Statistics System](https://www.cdc.gov/nchs/nvss/births.htm), downloaded directly from [CDC Wonder](https://wonder.cdc.gov/natality-expanded-current.html):

```python
births = pd.read_csv("births.csv")
births
```

      county
      year
      month
      births

      0
      Baldwin County, AL
      2016
      January
      63

      1
      Baldwin County, AL
      2016
      February
      67

      2
      Baldwin County, AL
      2016
      March
      69

      3
      Baldwin County, AL
      2016
      April
      63

      4
      Baldwin County, AL
      2016
      May
      51

      ...
      ...
      ...
      ...
      ...

      27727
      Winnebago County, WI
      2019
      August
      66

      27728
      Winnebago County, WI
      2019
      September
      45

      27729
      Winnebago County, WI
      2019
      October
      41

      27730
      Winnebago County, WI
      2019
      November
      51

      27731
      Winnebago County, WI
      2019
      December
      54

27732 rows × 4 columns

```python
births_by_county = births.groupby("county",
                                  as_index=False).agg(Mean = ("births", "mean"),
                                                      Variance = ("births", "var"),
                                                      Max = ("births","max") )
births_by_county["Ratio"] = births_by_county["Variance"] / births_by_county["Mean"]
births_by_county = births_by_county[ births_by_county["Max"] < 100 ]
births_by_county = births_by_county.sort_values(by="Ratio", ascending=False)
births_by_county[ abs( births_by_county["Ratio"] - 1 ) < .05 ]
```

      county
      Mean
      Variance
      Max
      Ratio

      572
      Yellowstone County, MT
      68.666667
      71.929078
      87
      1.047511

      293
      Litchfield County, CT
      45.125000
      47.260638
      60
      1.047327

      20
      Ascension Parish, LA
      51.812500
      54.155585
      69
      1.045222

      81
      Catawba County, NC
      68.666667
      71.418440
      87
      1.040074

      48
      Boone County, KY
      51.687500
      53.070479
      72
      1.026757

      426
      Richland County, OH
      36.333333
      36.950355
      49
      1.016982

      422
      Randolph County, NC
      49.333333
      49.631206
      65
      1.006038

      536
      Washington County, MD
      57.104167
      57.414450
      73
      1.005434

      46
      Blount County, TN
      32.250000
      32.319149
      43
      1.002144

      486
      St. Lawrence County, NY
      27.416667
      27.226950
      45
      0.993080

Oddly enough, the mean and variance for births per month are closest for a county in Tennessee! How well do births in Blount County agree with $\operatorname{Pois}(32.25,32.32)$?

```python
blount = births[ births[ "county"] == "Blount County, TN" ]
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=blount, x="births", discrete=True, ax=axs[0])
axs[0].set_title("Observed births per month in Blount County, TN")
pois_rv = pd.DataFrame({"Simulation" : poisson.rvs(np.mean(blount["births"]),
                                                   size=blount.shape[0])})
sns.histplot(data=pois_rv, x="Simulation", discrete=True, ax=axs[1])
axs[1].set_title("Simulated births per month in Blount County, TN");
```

### Simple Poisson regression: a preview

A simple Poisson regression model fits a family of Poisson RVs to observations. Each covariate class — one for each value $x$ of the predictor — has a mean rate $\lambda$ given (approximately!) by

$$ \log{ \left( \lambda \right) } ~ = ~ a + bx \ . $$

We'll discuss this in detail in the next Twitch session!

## Negative binomial models

### Overdispersed births

The Poisson distribution can clearly be a reasonable model for car crashes or births per month in a given county — ***if*** the mean and the variance are ***equal***! This is definitely not always the case:

```python
births_by_county
```

      county
      Mean
      Variance
      Max
      Ratio

      253
      Kanawha County, WV
      28.305556
      155.989683
      52
      5.510921

      235
      Jackson County, MI
      58.645833
      157.850621
      91
      2.691591

      423
      Rankin County, MS
      54.145833
      131.063387
      80
      2.420563

      342
      Monroe County, IN
      48.041667
      115.317376
      72
      2.400362

      441
      Saline County, AR
      53.541667
      127.955674
      79
      2.389834

      ...
      ...
      ...
      ...
      ...
      ...

      469
      Sheboygan County, WI
      39.833333
      36.141844
      55
      0.907327

      556
      Whitfield County, GA
      47.708333
      42.593972
      65
      0.892799

      226
      Houston County, GA
      70.270833
      61.903812
      89
      0.880932

      312
      Marathon County, WI
      50.479167
      42.084663
      66
      0.833704

      549
      Wayne County, OH
      46.145833
      36.893174
      58
      0.799491

226 rows × 5 columns

```python
kanawha = births[ births["county"] == "Kanawha County, WV"]
kanawha.shape
```

    (36, 4)

```python
fig, axs = plt.subplots(2, sharex=True)
sns.histplot(data=kanawha,
             x="births",
             discrete=True, ax=axs[0])
axs[0].set_title("Observed births per month in Kanawha County, WV")
pois = pd.DataFrame({"Sample" : poisson.rvs(np.mean(kanawha["births"]),
                                            size=kanawha.shape[0])})
sns.histplot(data=pois, x="Sample",
             discrete=True, ax=axs[1])
axs[1].set_title("Poisson-simulated births per month in Kanawha County, WV");
```

Monthly births in Kanawha County, WV are definitely _not_ Poisson distributed! This is **overdispersion**: the variance is much larger than the mean. Overdispersion and **underdispersion** (variance smaller than the mean) are common in count modeling; in either case, Poisson models are _not_ appropriate — they only apply to counts that are **equidispersed**.

### Overdispersed car crashes

***All*** of the counties in the `crashes` dataset exhibit overdispersion, with Meigs County the closest to being equidispersed. At the other extreme, monthly crashes in nearby Hamilton County have a variance that is more than 20 times their mean:

```python
hamilton = crashes[ crashes["County"] == "Hamilton" ]
hamilton = hamilton.drop("County",1)
hamilton = hamilton.melt(id_vars="Year",
                         var_name="Month",
                         value_name="Crashes" )
hamilton.agg(Mean = ("Crashes","mean"),
             Variance = ("Crashes","var") )
```

      Crashes

      Mean
      1047.060606

      Variance
      22476.347444

```python
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=hamilton, x="Crashes", discrete=True, ax=axs[0])
axs[0].set_title("Observed crashes per month in Hamilton County, TN")
pois = pd.DataFrame({"Sample" : poisson.rvs(np.mean(hamilton["Crashes"]),
                                            size=hamilton.shape[0])})
sns.histplot(data=pois, x="Sample", discrete=True, ax=axs[1])
axs[1].set_title("Poisson-simulated crashes per month in Hamilton County, TN");
```

Terrible!! Never use a Poisson model when there's significant overdispersion!

### Coughs per hour

Here are 2 common features of the `crashes` and `births` datasets:

* counts are recorded _per month_, and
* _there are no zero counts_ for any month.

Of course, there's nothing special about working with months, and there can be plenty of zeroes in some datasets. When monitoring a person's coughs, for example, it's natural to record coughs _per hour_, yielding lots of zeroes. Here are coughs recorded per hour by the [Hyfe cough monitoring app](https://www.hyfeapp.com/) for a person in Spain:

```python
primera = pd.read_csv("primera.csv")
primera["datetime"] = pd.to_datetime( primera["datetime"] )
primera
```

      datetime
      coughs

      0
      2020-11-06 01:00:00
      0

      1
      2020-11-06 02:00:00
      0

      2
      2020-11-06 03:00:00
      0

      3
      2020-11-06 04:00:00
      0

      4
      2020-11-06 05:00:00
      0

      ...
      ...
      ...

      7076
      2021-08-27 22:00:00
      0

      7077
      2021-08-27 23:00:00
      0

      7078
      2021-08-28 00:00:00
      0

      7079
      2021-08-28 01:00:00
      0

      7080
      2021-08-28 02:00:00
      0

7081 rows × 2 columns

Despite what's shown for these 10 hours, this person coughed quite a bit from time to time:

```python
sns.lineplot(data=primera, x="datetime", y="coughs");
```

Even so, there were way more hours with 0 or just a few coughs:

```python
sns.histplot(data=primera, x="coughs", discrete=True);
```

There's clearly some overdispersion here, as we can easily check:

```python
primera.agg(Mean = ("coughs","mean"),
            Variance = ("coughs","var") )
```

      coughs

      Mean
      0.99195

      Variance
      12.03341

This is pretty typical for coughing — here's another Hyfe user from Spain:

```python
segunda = pd.read_csv("segunda.csv")
segunda["datetime"] = pd.to_datetime( segunda["datetime"] )
segunda
```

      datetime
      coughs

      0
      2020-11-14 01:00:00
      0

      1
      2020-11-14 02:00:00
      0

      2
      2020-11-14 03:00:00
      0

      3
      2020-11-14 04:00:00
      0

      4
      2020-11-14 05:00:00
      0

      ...
      ...
      ...

      2012
      2021-02-05 21:00:00
      0

      2013
      2021-02-05 22:00:00
      0

      2014
      2021-02-05 23:00:00
      0

      2015
      2021-02-06 00:00:00
      0

      2016
      2021-02-06 01:00:00
      0

2017 rows × 2 columns

```python
sns.lineplot(data=segunda, x="datetime", y="coughs");
```

```python
sns.histplot(data=segunda, x="coughs", discrete=True);
```

```python
segunda.agg(Mean = ("coughs","mean"),
            Variance = ("coughs","var"))
```

      coughs

      Mean
      0.411502

      Variance
      2.487328

### The negative binomial distribution

The Poisson distribution doesn't apply to overdispersed counts, so what does? The go-to option in many cases is the **negative binomial distribution**, whose PMF has various forms. The form implemented by `scipy.nbinom` is

$$ P( Y = k ) = \binom{ k+n-1 }{ n-1 } p^{n} \left( 1 - p \right)^{k} \ , \quad k = 0, 1, 2, \ldots \ , $$

reminiscent of the binomial distribution; this is the probability of needing $k+n$ trials to observe $n$ successes when the chance of success per trial is $p$. The mean and variance of $Y \sim \operatorname{NB}(n,p))$ are

$$\operatorname{E}\left( \, Y \, \right) ~ = ~ \frac{n}{p} \quad \text{and} \quad \operatorname{Var}( \, Y \, ) ~ = ~ \frac{n(1-p)}{p^2} \ .$$

For count models, different parameterizations are common and more practical. Let $\mu$ denote the average of $Y \sim \operatorname{NB}(n,p))$, let $\sigma^2$ denote its variance, and define the **dispersion parameter** $\alpha$ by

$$ \sigma^2 ~ = ~ \mu + \alpha \mu^2 \quad \iff \quad \alpha ~ = ~ \frac{\sigma^2 - \mu}{\mu^2} \ . $$

Given the values of 2 of these parameters, relate them to $n$ and $p$:

$$ p ~ = ~ \frac{\mu}{\mu + \sigma^2} ~ = ~ \frac{1}{2 + \alpha \mu} \quad \text{and} \quad n ~ = ~ \frac{\mu^2}{\mu + \sigma^2} ~ = ~ \frac{\mu}{2 + \alpha \mu} \ .$$

You can then translate between the versions — $\operatorname{NB}(n,p)$, $\operatorname{NB}(\mu,\sigma^2)$ and $\operatorname{NB}(\mu,\alpha)$ — as needed. Beware that other people use $\displaystyle{\frac{1}{\alpha}}$ as the dispersion parameter, and other relationships between it and the variance are possible. And most importantly, notice that _two parameters are needed!!_

> **Use the `nbinom` module from `scipy` (`import`ed above) to work with negative binomial distributions.**

### Simulating monthly births

```python
mu = kanawha["births"].mean()
var = kanawha["births"].var()
n_kanawha = mu**2 / (mu + var )
p_kanawha = mu / ( mu + var )
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=kanawha, x="births", discrete=True, ax=axs[0])
axs[0].set_title("Observed births per month in Kanawha County, WV")
NB = pd.DataFrame({"Simulation" : nbinom.rvs(n_kanawha, p_kanawha,
                                             size=kanawha.shape[0])})
sns.histplot(data=NB, x="Simulation", discrete=True, ax=axs[1])
axs[1].set_title("NB-simulated births per month in Kanawha County, WV");
```

### Simulating monthly crashes

```python
mu = hamilton["Crashes"].mean()
var = hamilton["Crashes"].var()
n = mu**2 / (mu + var )
p = mu / ( mu + var )
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=hamilton, x="Crashes", discrete=True, ax=axs[0])
axs[0].set_title("Observed crashes per month in Hamilton County, TN")
NB = pd.DataFrame({"Simulation" : nbinom.rvs(n, p,
                                             size=hamilton.shape[0])})
sns.histplot(data=NB, x="Simulation", discrete=True, ax=axs[1])
axs[1].set_title("NB-simulated crashes per month in Hamilton County, TN");
```

### Simulating coughs per hour

```python
mu = primera["coughs"].mean()
var = primera["coughs"].var()
n = mu**2 / (mu + var )
p = mu / ( mu + var )
primera["simulation"] = nbinom.rvs(n, p, size=primera.shape[0])
fig, axs = plt.subplots(2, sharex=True)
sns.lineplot( data=primera, x="datetime", y="coughs", ax=axs[0])
axs[0].set_title("Recorded coughs per hour")
sns.lineplot(data=primera, x="datetime", y="simulation", ax=axs[1])
axs[1].set_title("NB-simulated coughs per hour");
```

```python
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=primera, x="coughs", discrete=True, ax=axs[0])
axs[0].set_title("Recorded coughs per hour")
sns.histplot(data=primera, x="simulation", discrete=True, ax=axs[1])
axs[1].set_title("NB-simulated coughs per hour");
```

```python
mu = segunda["coughs"].mean()
var = segunda["coughs"].var()
n = mu**2 / (mu + var )
p = mu / ( mu + var )
segunda["simulation"] = nbinom.rvs(n, p, size=segunda.shape[0])
fig, axs = plt.subplots(2, sharex=True)
sns.lineplot( data=segunda, x="datetime", y="coughs", ax=axs[0])
axs[0].set_title("Recorded coughs per hour")
sns.lineplot(data=segunda, x="datetime", y="simulation", ax=axs[1])
axs[1].set_title("NB-simulated coughs per hour");
```

```python
fig, axs = plt.subplots(2, sharex=True)
sns.histplot( data=segunda, x="coughs", discrete=True, ax=axs[0])
axs[0].set_title("Recorded coughs per hour")
sns.histplot(data=segunda, x="simulation", discrete=True, ax=axs[1])
axs[1].set_title("NB-simulated coughs per hour");
```

### Simple negative binomial regression: a preview

A simple negative binomial regression model fits a family of NB RVs to observations. Each covariate class — one for each value $x$ of the predictor — has a mean rate $\mu$ given (approximately!) by

$$ \log{ \left( \mu \right) } ~ = ~ a + bx $$

_AND_ a dispersion parameter $\alpha$ determined by the fitting process. More on this next time!

## The Kolmogorov-Smirnov test

### Cumulative distribution functions

The PMF of a discrete RV $Y$ — binomial, Poisson, or negative binomial — determines $P( Y = k )$ for any nonnegative integer $k$; an alternative characterization of $Y$ is its **cumulative distribution function (CDF)**, which gives $P( Y \leq k )$ for any such $k$. Using recorded or simulated counts instead of a theoretical distribution yields **empirical cumulative distribution functions (ECDF)**, and comparing them is the basis of the **Kolmogorov-Smirnov test** — as with any other hypothesis test, you should always complement this one with a visual and a good dose of common sense!

```python
sns.ecdfplot(data=meigs[["Crashes","Simulation"]]);
```

```python
kstest(meigs["Crashes"],meigs["Simulation"])
```

    KstestResult(statistic=0.06060606060606061, pvalue=0.9696098278463836)

The null hypothesis here is that _both of these samples are drawn from the same distribution_ — the $p$-value is large, so we have no reason to reject this hypothesis for the crashes in Meigs County. Based on this data, monthly crashes there do seem to be Poisson distributed!

```python
ks_kanawha = pd.DataFrame({"births":kanawha["births"],
                           "simulation":nbinom.rvs( n_kanawha, p_kanawha, size=kanawha.shape[0])})
sns.ecdfplot(data=ks_kanawha);
```

```python
kstest(ks_kanawha["births"], ks_kanawha["simulation"])
```

    KstestResult(statistic=0.3333333333333333, pvalue=0.03593229829749474)

```python
sns.ecdfplot(data=segunda[["coughs","simulation"]]);
```

```python
kstest(segunda["coughs"],segunda["simulation"])
```

    KstestResult(statistic=0.053544868616757564, pvalue=0.006151681796061683)

```python

```


.. _nb_ref_dirs:

## Reference Directions

Most evolutionary many-objective optimization (EMaO) algorithms, for instance [NSGA3](../algorithms/nsga3.ipynb) or [MOEAD](../algorithms/moead.ipynb), start with a description of a number of predefined set of reference directions on a unit simplex. Reference directions in an $M$-dimensional space are subject to

$$
\sum_{i=1}^{M} w_i = 1
$$

$$
w_i \geq 0 \quad \forall i \in (1, ..., M)
$$

So far, most studies have used the Das and Dennis's structured approach  for generating  well-spaced reference points.
A reference direction is constructed by a vector originating from the origin and connected to each of them. The number of points on the unit simplex is determined by a parameter $p$ (we call it `n_partitions` in our implementation), which indicates the number of gaps between two consecutive points along an objective axis. It turns out that the total number of points ($n$) on the unit simplex is

$$
n = C_{p}^{\, M+p-1}
$$

For example, if $p=10$ is chosen for a three-objective problem ($M=3$), then the total number of points on the unit simplex is $C_{10}^{\, 3+10-1}=\binom{12}{10}$ or 66. These 66 points are well-spaced with an identical distance to their nearest neighbor on the unit simplex.
Due to the highly structured nature of the procedure, this method cannot produce an arbitrary number of points desired in an EMaO application. Moreover, in higher dimensional space, most points to not lie in the interior but on the boundaries. Both are demonstrated in the figure below.

Our framework offers a **novel** Riesz s-Energy  based method which can overcome the limitation of the Das and Dennis method and create a well-spaced point set for any arbitrary number of points and dimensions.
This method addresses the need of EMO research for a more generic approach to sample reference directions.

[Riesz s-Energy](#Riesz-s-Energy) |
[Uniform Sampling / Das-Dennis](#Das-Dennis) |
[Multi-layer Approach](#Multi-layer-Approach)

### Riesz s-Energy

\begin{equation}
\newcommand{\boldz}{\mbox{${\textbf z}$}}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}
\end{equation}

The motivation behind the use of an energy concept to obtain a well-spaced distribution comes from nature. Multi-body and interacting physical systems eventually settle on a state which corresponds to the minimum overall potential energy. For two bodies, the potential energy is proportional to the inverse of the distance between them. The minimum potential solution corresponds to a diverse distribution of multiple bodies in the three-dimensional physical space.
While dealing with a many-dimensional ($s$) space, we use a generalization of potential energy called Riesz  which is defined between two points ($\boldz^{(i)}$ and $\boldz^{(j)}$) as

\begin{equation}
U(\boldz^{(i)},\boldz^{(j)}) = \frac{1}{\norm{{\bf z}^{(i)} - {\bf z}^{(j)}}^s}.
\end{equation}

In our context, it is not clear how the dimension $s$ should depend on the number of objectives ($M$), but some trial-and-error studies (shown in the Supplementary document) motivated us to set $s=M^2$ for all simulations here. For multiple ($n$) points, the overall s-Energy can be written as follows:

\begin{equation}
U_T(\boldz) = \frac{1}{2}\sum_{i = 1}^{n}\sum_{j = 1 \atop j \neq i}^{n} \frac{1}{\norm{{\bf z}^{(i)} - {\bf z}^{(j)}}^s}, \quad {\bf z} \in \mathbb{R}^{n\times M}.
\end{equation}

The concept for our energy method is to find the $\boldz$-matrix of size $n\times M$ which minimizes $U_T$ subject to every $\boldz^{(i)}$ vector to lie on the unit simplex, that is, $\sum_{m=1}^M z_m^{(i)}=1$. We employ a gradient-based optimization method (Adam~\cite{adam}) here. Due to very large magnitude of $U_T$, we take a logarithm of $U_T$ and then compute the partial derivative of $F_E=\log U_T$ with respect to $z_m^{(i)}$, as follows:

\begin{equation}
\begin{aligned}
\frac{\partial F_E}{\partial z^{(i)}_m} &= -\frac{d}{U_T} \left[\sum_{j=1, \atop j \neq i}^{n}\frac{\left(z^{(i)}_m - z^{(j)}_m\right)}{\norm{{\bf z}^{(i)} - {\bf z}^{(j)}}^{s + 2}} \right].
\end{aligned}
\end{equation}

To make sure all points stay on the unit simplex, gradients are projected onto the unit simplex.

The proposed s-Energy method is exclusively available in our framework and was the outcome of a research project in our laboratory. We are driven by providing our newest algorithms and methods to other researchers.
If you use our method to create reference directions, please cite our method as:

```
@ARTICLE{ref_dirs_energy,
    author={J. {Blank} and K. {Deb} and Y. {Dhebar} and S. {Bandaru} and H. {Seada}},
    journal={IEEE Transactions on Evolutionary Computation},
    title={Generating Well-Spaced Points on a Unit Simplex for Evolutionary Many-Objective Optimization},
    year={In press},
    volume={},
    number={},
    pages={}
}
```

Some examples are provided below:

```python
from pymoo.factory import get_visualization, get_reference_directions
from pymoo.util.ref_dirs.energy import RieszEnergyReferenceDirectionFactory

ref_dirs = get_reference_directions("energy", 3, 90, seed=1)
get_visualization("scatter").add(ref_dirs).show()
```

```python
ref_dirs = get_reference_directions("energy", 3, 250, seed=1)
get_visualization("scatter").add(ref_dirs).show()
```

### Das-Dennis

As mentioned in the introduction, the Das-Dennis method can be used to create reference directions in a very structured way.
However, the method relies on a partition number `n_partitions` which determines how many points will be sampled.
Points can be uniformly create on the unit simplex using the Das-Dennis method proposed in .

```python
ref_dirs = get_reference_directions("das-dennis", 3, n_partitions=12)
get_visualization("scatter").add(ref_dirs).show()
```

By carefully observing the figure above, one can see that along one axis, exactly 12 partitions (which results in 13 points, including the corners) exist. However, this implies that when the dimensionality increases, more points will be sampled a) on the boundaries and b) on the overall simplex.

```python
import matplotlib.pyplot as plt

n_dim = range(3,10)
n_points = [len(get_reference_directions("das-dennis", d, n_partitions=12)) for d in n_dim]

plt.plot(n_dim, n_points, '--o')
plt.title("Number of Referenes Directions (p=12)")
plt.ylabel("Number of Points")
plt.xlabel("Dimension")
plt.show()
```

One can argue that decreasing the partition number might be a solution to address the exploding number of points in higher dimensions. However, please keep in mind that only very few points might lie in the simplex interior. Calculations reveal that, when $p<M$, there is no interior point, and when $p=M$, there is exactly one interior point, irrespective of the size of $M$. With $p>M$  points begin to appear in the interior, but the number of such interior points is $n_I = {{p-1}\choose{p-M}}$, which is only a tiny fraction
\begin{equation}
\rho_I = \frac{n_I}{n} = \frac{p! \, (p-1)!}{(p-M)! \, (M+p-1)!}
\end{equation}
of all $n$ Das and Dennis points.

Some researchers have used the idea of creating multiple layers of such Das-Dennis points to overcome this issue.

### Multi-layer Approach

This approach uses the idea of scaling different simplices. This requires the user to define different layers, including the dimensionality `n_dim`, the partition number `n_partitions`, and the corresponding scaling `scaling`.
For instance, for three dimensions, a bias towards the center can be achieved as follows:

```python
ref_dirs = get_reference_directions(
    "multi-layer",
    get_reference_directions("das-dennis", 3, n_partitions=12, scaling=1.0),
    get_reference_directions("das-dennis", 3, n_partitions=12, scaling=0.5)
)

get_visualization("scatter").add(ref_dirs).show()
```

In order to factor reference directions on an eight-dimensional unit simplex we can do the following:

```python
ref_dirs = get_reference_directions(
    "multi-layer",
    get_reference_directions("das-dennis", 8, n_partitions=3, scaling=1.0),
    get_reference_directions("das-dennis", 8, n_partitions=2, scaling=0.5),
)

get_visualization("pcp").add(ref_dirs).show()
```

A more generic approach that does **NOT** need any scalings to be provided by the user is to use the **Riesz s-Energy** method to optimize not the reference directions directly but the scalings.

```python
partitions = [3, 2, 1, 1]

ref_dirs = get_reference_directions("layer-energy", 8, partitions)
get_visualization("pcp").add(ref_dirs).show()
```

The `LayerwiseRieszEnergyReferenceDirectionFactory` can be used to retrieve the optimized partitions instead of the convenience method.

```python
from pymoo.util.ref_dirs.energy_layer import LayerwiseRieszEnergyReferenceDirectionFactory

fac = LayerwiseRieszEnergyReferenceDirectionFactory(8, partitions)
ref_dirs = fac.do()

list(fac.scalings)
```

    [1.0, 0.8480631388253735, 0.6542126980619762, 0.3805165848046697]

```python
ref_dirs = get_reference_directions("layer-energy", 3, [9, 5, 2, 1])
get_visualization("scatter").add(ref_dirs).show()
```


# Nueral Networks (MLP)
This notebook implemenets a single hidden layer neural network from the scratch usng only pytorch vector operations, optimizationn is done with batch gradient descent

## Data Downloading and shape adjustments

```
import torch
from torchvision import transforms
from torchvision import datasets
import matplotlib.pyplot as plt
import numpy as np
RANDOM_SEED = 1

def convert_to_one_hot_labels(input, target):
    tmp = input.new_zeros(target.size(0), target.max() + 1)
    tmp.scatter_(1, target.view(-1, 1), 1.0)
    return tmp

def load_data(cifar = None, one_hot_labels = False, normalize = False, flatten = True, data_dir="./"):
    if cifar is not None and cifar:
        print('* Using CIFAR')
        cifar_train_set = datasets.CIFAR10(data_dir + '/cifar10/', train = True, download = True, transform = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]))
        cifar_test_set = datasets.CIFAR10(data_dir + '/cifar10/', train = False, download = True)

        train_input = torch.from_numpy(cifar_train_set.data)
        train_input = train_input.transpose(3, 1).transpose(2, 3).float()
        train_target = torch.tensor(cifar_train_set.targets, dtype = torch.int64)

        test_input = torch.from_numpy(cifar_test_set.data).float()
        test_input = test_input.transpose(3, 1).transpose(2, 3).float()
        test_target = torch.tensor(cifar_test_set.targets, dtype = torch.int64)

    else:
        print('* Using MNIST')
        mnist_train_set = datasets.MNIST('./', train = True, download = True)
        mnist_test_set = datasets.MNIST('./', train = False, download = True)

        train_input = mnist_train_set.data.view(-1, 1, 28, 28).float()
        train_target = mnist_train_set.targets
        test_input = mnist_test_set.data.view(-1, 1, 28, 28).float()
        test_target = mnist_test_set.targets

    train_input = train_input.narrow(0, 0, 1000)
    train_target = train_target.narrow(0, 0, 1000)
    test_input = test_input.narrow(0, 0, 1000)
    test_target = test_target.narrow(0, 0, 1000)

    if flatten:
        train_input = train_input.clone().reshape(train_input.size(0), -1)
        test_input = test_input.clone().reshape(test_input.size(0), -1)

    print('** Use {:d} train and {:d} test samples'.format(train_input.size(0), test_input.size(0)))

    if one_hot_labels:
        train_target = convert_to_one_hot_labels(train_input, train_target)
        test_target = convert_to_one_hot_labels(test_input, test_target)

    if normalize:
        mu, std = train_input.mean(), train_input.std()
        train_input.sub_(mu).div_(std)
        test_input.sub_(mu).div_(std)

    return train_input, train_target, test_input, test_target

######################################################################

def mnist_to_pairs(nb, input, target):
    input = torch.functional.F.avg_pool2d(input, kernel_size = 2)
    a = torch.randperm(input.size(0))
    a = a[:2 * nb].view(nb, 2)
    input = torch.cat((input[a[:, 0]], input[a[:, 1]]), 1)
    classes = target[a]
    target = (classes[:, 0] <= classes[:, 1]).long()
    return input, target, classes

######################################################################

def generate_pair_sets(nb):
    if args.data_dir is not None:
        data_dir = args.data_dir
    else:
        data_dir = os.environ.get('PYTORCH_DATA_DIR')
        if data_dir is None:
            data_dir = './data'

    train_set = datasets.MNIST(data_dir + '/mnist/', train = True, download = True)
    train_input = train_set.data.view(-1, 1, 28, 28).float()
    train_target = train_set.targets

    test_set = datasets.MNIST(data_dir + '/mnist/', train = False, download = True)
    test_input = test_set.data.view(-1, 1, 28, 28).float()
    test_target = test_set.targets

    return mnist_to_pairs(nb, train_input, train_target) + \
           mnist_to_pairs(nb, test_input, test_target)

######################################################################
```

```
# Load the CIFAR datasey
train_input, train_target, test_input, test_target = load_data(True, True, True, True)
```

    * Using CIFAR
    Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to .//cifar10/cifar-10-python.tar.gz

    HBox(children=(FloatProgress(value=0.0, max=170498071.0), HTML(value='')))

    Extracting .//cifar10/cifar-10-python.tar.gz to .//cifar10/
    Files already downloaded and verified
    ** Use 1000 train and 1000 test samples

```
# sample size
train_input.shape
```

    torch.Size([1000, 3072])

## Implementing nerual networks methods

In this task we will use the mean square error \\
\begin{equation}
MSE = \frac{1}{N} \sum_i^N (\hat{y}_i - y_i)^2
\end{equation}

```
# Loss Function and its dervative
def loss(predictions, targets):
  '''
  loss(v, t)
  This is the mean square error for regression problems.
  predictions: predicted values generated by the network.
  targets: the ground truth targets
  '''
  return ((predictions - targets)**2).mean()

def dloss(predictions, targets):
  '''
  dloss(predictions, targets)
  This is the derivative of mean square error for regression problems.
  predictions: predicted values generated by the network.
  targets: the ground truth targets
  '''
  return (-2*(predictions - targets)).mean()
```

```
# Initilize the network parameters
def initlize_parameters(n_input, n_hidden, n_output, random_state=RANDOM_SEED):
  '''
  initlize_parameters(n_input, n_hidden, n_output, random_state=RANDOM_SEED)
  This function initializes the weights and biases of a single layer neural network randomly from a normal distribution with zero mean and 10^-6 std.
  n_input: number of dimensions of the input.
  n_hidden: number of units of the hidden layer.
  n_output: number of dimensions of the output.
  random_state: random seed.
  Return  w1, b1, w2, b2.
  '''
  torch.manual_seed(RANDOM_SEED)
  w1 = torch.normal(0, 1e-6, size=(n_hidden, n_input))
  b1 = torch.normal(0, 1e-6, size=(n_hidden, 1))
  w2 = torch.normal(0, 1e-6, size=(n_output, n_hidden))
  b2 = torch.normal(0, 1e-6, size=(n_output, 1))
  return w1, b1, w2, b2
```

```
# Forward Pass
def forward_pass(w1, b1, w2, b2, x):
  '''
  This function performs the forward pass given the weight and biases of each layer with the input.
  w1: weights of the hidden layer.
  b1: biases of the hidden layer.
  w2: weights of the output layer.
  b2: biases of the output layer.
  return a0, z1, a1, z2, a2
  a: the layer activation.
  z: the result of the weights multiplication and adding the bias.
  '''
  a0 = x
  z1 = w1 @ a0 + b1
  a1 = tanh(z1)
  z2 = w2 @ a1 + b2
  a2 = tanh(z2)
  return a0, z1, a1, z2, a2

# Gradient calculations
def compute_gradient(y_hat, y, w1, b1, w2, b2, a0, z1, a1, z2, a2):
  '''
  compute_gradient(y_hat, y, w1, b1, w2, b2, a0, z1, a1, z2, a2)
  This function computes the gradient with backpropagation.
  y_hat: network prediction.
  y: ground target.
  w1: weights of the hidden layer.
  b1: biases of the hidden layer.
  w2: weights of the output layer.
  b2: biases of the output layer.
  a0: input value.
  z1: the result of the weights multiplication and adding the bias in the first hidden layer.
  a1: activation of the hidden layer.
  a2: activation of the output layer.
  z2: the result of the weights multiplication and adding the bias in the output layer.
  return a grad dictionary that contains the gradient of weights and biases of all layers.
  '''
  m = y.shape[1]
  da2 = 2*a2 - y_train
  dz2 = da2 * dtanh(z2)
  dw2 = (1/m) * dz2 @ a1.t()
  db2 = (1/m) * torch.sum(dz2, axis=1, keepdims=True)

  da1 = w2.t() @ dz2
  dz1 = da1 * dtanh(z1)
  dw1 = (1/m) * dz1 @ a0.t()
  db1 = (1/m) * torch.sum(dz1, axis=1, keepdims=True)

  grads = {}
  grads['w1'] = dw1
  grads['b1'] = db1
  grads['w2'] = dw2
  grads['b2'] = db2
  return grads

def backward_pass(w1, b1, w2, b2, grads, lr=0.1):
  '''
  backward_pass(w1, b1, w2, b2, grads, lr=0.1)
  This function updates the weights and biases with gradient descent.
  w1: weights of the hidden layer.
  b1: biases of the hidden layer.
  w2: weights of the output layer.
  b2: biases of the output layer.
  grads: a dictionary that contains the gradient of weights and biases of all layers.
  lr: learning rate.
  return w1, b1, w2, b2
  '''
  w1 = w1 - lr * grads['w1']
  b1 = b1 - lr * grads['b1']
  w2 = w2 - lr * grads['w2']
  b2 = b2 - lr * grads['b2']
  return w1, b1, w2, b2

```

```
X_train = train_input.t() # Take the transpose to work the neural net notation (d, number of examples)
X_test = test_input.t() # Take the transpose to work the neural net notation (d, number of examples)
y_train = train_target.t() * 0.9 # multiply by 0.9 so that the label are in the same of tanh function
y_test = test_target.t() * 0.9  # multiply by 0.9 so that the label are in the same of tanh function

# Initlize parameters values.
w1, b1, w2, b2 = initlize_parameters(3072, 50, 10)
# loss list to plot the loss with each epoch.
losses = []

# 1000 epoch
for i in range(1000):
  # pass the input forward through the network.
  a0, z1, a1, z2, a2 = forward_pass(w1, b1, w2, b2, X_train)
  # compute the gradients
  grads = compute_gradient(a2, y_train, w1, b1, w2, b2, a0, z1, a1, z2, a2)
  # update the weights
  w1, b1, w2, b2 = backward_pass(w1, b1, w2, b2, grads, lr=0.1/m)
  # append the loss to the list
  losses.append(loss(a2, y_train))
```

```
# plot the learning error.
plt.plot(losses)
plt.xlabel("Iteration")
plt.ylabel("MSE")
plt.show()
```

```
# Training error
print(f"Training error = {losses[-1]*100}%")
# Testing error
test_preds = forward_pass(w1, b1, w2, b2, X_test)[-1]
test_loss = loss(test_preds, y_test)
print(f"Testing error = {test_loss*100}%")
```

    Training error = 7.959237098693848%
    Testing error = 7.959589958190918%

## Putting all methods in a single class

```
class Network:
  def __init__(self, nn_inputs, nn_hidden, n_outputs, lr=0.1, iterations=500):
    self.nn_inputs = nn_inputs
    self.nn_hidden = nn_hidden
    self.nn_outputs = n_outputs
    self.lr = lr
    self.iterations = iterations
    # initilize parameters
    self.initlize_parameters()

  def __call__(self, X):
    return self.forward_pass(X)

  def initlize_parameters(self):
    '''
    initlize_parameters(n_input, n_hidden, n_output, random_state=RANDOM_SEED)
    This function initializes the weights and biases of a single layer neural network randomly from a normal distribution with zero mean and 10^-6 std.
    n_input: number of dimensions of the input.
    n_hidden: number of units of the hidden layer.
    n_output: number of dimensions of the output.
    random_state: random seed.
    Return  w1, b1, w2, b2.
    '''
    torch.manual_seed(RANDOM_SEED)
    self.w1 = torch.randn(self.nn_hidden, self.nn_inputs)
    self.b1 = torch.randn(self.nn_hidden, 1)
    self.w2 = torch.randn(self.nn_outputs, self.nn_hidden)
    self.b2 = torch.randn(self.nn_outputs, 1)

  def forward_pass(self, X):
    '''
    This function performs the forward pass given the weight and biases of each layer with the input.
    w1: weights of the hidden layer.
    b1: biases of the hidden layer.
    w2: weights of the output layer.
    b2: biases of the output layer.
    return a0, z1, a1, z2, a2
    a: the layer activation.
    z: the result of the weights multiplication and adding the bias.
    '''
    self.a0 = X
    self.z1 = self.w1 @ self.a0 + self.b1
    self.a1 = self.tanh(self.z1)
    self.z2 = self.w2 @ self.a1 + self.b2
    self.a2 = self.tanh(self.z2)
    return self.a2

  # Gradient calculations
  def compute_gradient(self, y_train):
    '''
    compute_gradient(y_hat, y, w1, b1, w2, b2, a0, z1, a1, z2, a2)
    This function computes the gradient with backpropagation.
    y_hat: network prediction.
    y: ground target.
    w1: weights of the hidden layer.
    b1: biases of the hidden layer.
    w2: weights of the output layer.
    b2: biases of the output layer.
    a0: input value.
    z1: the result of the weights multiplication and adding the bias in the first hidden layer.
    a1: activation of the hidden layer.
    a2: activation of the output layer.
    z2: the result of the weights multiplication and adding the bias in the output layer.
    return a grad dictionary that contains the gradient of weights and biases of all layers.
    '''
    m = 1000
    da2 = 2*self.a2 - y_train
    dz2 = da2 * self.dtanh(self.z2)
    dw2 = (1/m) * dz2 @ self.a1.t()
    db2 = (1/m) * torch.sum(dz2, axis=1, keepdims=True)

    da1 = self.w2.t() @ dz2
    dz1 = da1 * self.dtanh(self.z1)
    dw1 = (1/m) * dz1 @ self.a0.t()
    db1 = (1/m) * torch.sum(dz1, axis=1, keepdims=True)

    grads = {}
    grads['w1'] = dw1
    grads['b1'] = db1
    grads['w2'] = dw2
    grads['b2'] = db2
    self.grads = grads

  def backward_pass(self):
    '''
    backward_pass(w1, b1, w2, b2, grads, lr=0.1)
    This function updates the weights and biases with gradient descent.
    w1: weights of the hidden layer.
    b1: biases of the hidden layer.
    w2: weights of the output layer.
    b2: biases of the output layer.
    grads: a dictionary that contains the gradient of weights and biases of all layers.
    lr: learning rate.
    return w1, b1, w2, b2
    '''
    self.w1 = self.w1 - self.lr * self.grads['w1']
    self.b1 = self.b1 - self.lr * self.grads['b1']
    self.w2 = self.w2 - self.lr * self.grads['w2']
    self.b2 = self.b2 - self.lr * self.grads['b2']

  def fit(self, X, Y):
    '''
    fit(X, Y)
    This function train the network parameters to optimize the loss.
    X: batch of inputs.
    Y: batch of outputs.
    return list of losses at each iterations.
    '''
    losses = []
    for i in range(self.iterations):
      y_hat = self.forward_pass(X)
      self.compute_gradient(Y)
      self.backward_pass()
      losses.append(self.loss(y_hat, Y))
    return losses

  # Activation Function and its dervative
  def tanh(self, x):
    return torch.tanh(x)

  def dtanh(self, x):
    return 1 - tanh(x)**2

  # Loss Function and its dervative
  def loss(self, v, t):
    '''
    loss(v, t)
    This is the mean square error for regression problems.
    predictions: predicted values generated by the network.
    targets: the ground truth targets
    '''
    return ((t - v)**2).mean()

  def dloss(self, v, t):
    '''
    dloss(predictions, targets)
    This is the derivative of mean square error for regression problems.
    predictions: predicted values generated by the network.
    targets: the ground truth targets
    '''
    return (-2*(t - v)).mean()

```

```
net = Network(3072, 50, 10) # create a network object.
losses = net.fit(X_train, y_train) # fit the network.
# plot the training loss
plt.plot(losses)
plt.xlabel("Iteration")
plt.ylabel("MSE")
plt.show()
```

```
# make a prediction
net(X_test)
```

    tensor([[-0.9631,  0.8642,  1.0000,  ...,  0.9999, -0.9996, -0.3264],
            [-1.0000,  0.0956,  1.0000,  ...,  0.9999, -1.0000,  0.9743],
            [-1.0000, -0.6086,  1.0000,  ...,  1.0000, -0.8404, -0.4920],
            ...,
            [-1.0000, -0.9197, -0.7683,  ...,  1.0000, -1.0000,  0.9535],
            [-0.9056,  0.9295, -0.7877,  ...,  0.9868, -0.9999,  0.8018],
            [-0.7808,  1.0000,  0.8776,  ..., -1.0000, -0.9576,  1.0000]])

```

```


# M6803 Assignment3
## Wang Longqi
## Ex.1.
(a). Apply LDLT decomposition on the matrix A. Since the matrix is symmetrical, we only need to calculate L.
$$L_{21}=\frac{20}{8}=2.5 \\ L_{31}=\frac{15}{8}=1.875\\U_{22}=30\\U_{13}=15
\\L_{32}=\frac{A_{32}-L_{31}U_{12}}{U_{22}}=0.4167\\U_{23}=12.5\\U_{33}=A_{33}-L_{31}U_{13}-L_{32}U_{23}=26.667$$

Therefore, we have following LDLT decomposition:
\begin{equation}A=\left[
\begin{array}{ccc}
1 & 0 & 0 \\
2.5 & 1 & 0 \\
1.875 & 0.4167 & 1
\end{array} \right]
\left[
\begin{array}{ccc}
8 & 0 & 0 \\
0 & 30 & 0 \\
0 & 0 & 26.667
\end{array} \right]
\left[
\begin{array}{ccc}
1 & 2.5 & 1.875 \\
0 & 1 & 0.4167 \\
0 & 0 & 1
\end{array} \right]
\end{equation}

Then, transform the result to Cholesky decomposition
\begin{equation}A=\left[
\begin{array}{ccc}
2.8284 & 0 & 0 \\
7.0711 & 5.4772 & 0 \\
5.3033 & 2.2822 & 5.1640
\end{array} \right]
\left[
\begin{array}{ccc}
2.8284 & 7.0711 & 5.3033 \\
0 & 5.4772 & 2.2822 \\
0 & 0 & 5.1640
\end{array} \right]
\end{equation}

The Equation can be solved as $Ax=L(L^Tx)=b$. By using substitute procedure, we can get
\begin{equation} L^Tx=
\left[
\begin{array}{c}
17.677\\22.822\\-8.876
\end{array} \right]
\end{equation}

Then, back substitute, we obtain
\begin{equation} x=
\left[
\begin{array}{c}
-2.734 \\  4.883\\ -1.719
\end{array} \right]
\end{equation}

(b). Because the matrix is not diagonally dominant matrix, Jacobi method cannot converge. What is worse, two largest elements in row one and row two are in the same column. Therefore, we cannot adjust rows or columns to make Jacobi method converge. Following is the Jacobi iteration for original matrix:
\begin{equation} x^{(n+1)}=
\left[
\begin{array}{c}
6.25\\31.25\\12.5
\end{array} \right]
-\left[
\begin{array}{ccc}
0 &         2.5       &  1.875\\
0.25      &  0       &  0.625\\
0.25      &  0.83333333&  0\\
\end{array} \right] x^{(n)}
\end{equation}

Therefore,
\begin{equation} x^{(1)}=
\left[
\begin{array}{c}
6.25\\31.25\\12.5
\end{array} \right]
-\left[
\begin{array}{ccc}
0 &         2.5       &  1.875\\
0.25      &  0       &  0.625\\
0.25      &  0.83333333&  0\\
\end{array} \right]
\left[
\begin{array}{c}
-1\\2\\-1
\end{array} \right]
=\left[
\begin{array}{c}
3.125   \\  32.125    \\ 11.0833
\end{array} \right]
\end{equation}

\begin{equation} x^{(2)}=
\left[
\begin{array}{c}
6.25\\31.25\\12.5
\end{array} \right]
-\left[
\begin{array}{ccc}
0 &         2.5       &  1.875\\
0.25      &  0       &  0.625\\
0.25      &  0.83333333&  0\\
\end{array} \right]
\left[
\begin{array}{c}
3.125   \\  32.125    \\ 11.0833
\end{array} \right]
=\left[
\begin{array}{c}
-94.843\\
 23.541\\
-15.052
\end{array} \right]
\end{equation}\begin{equation} x^{(3)}=
\left[
\begin{array}{c}
6.25\\31.25\\12.5
\end{array} \right]
-\left[
\begin{array}{ccc}
0 &         2.5       &  1.875\\
0.25      &  0       &  0.625\\
0.25      &  0.83333333&  0\\
\end{array} \right]
\left[
\begin{array}{c}
-94.843\\
 23.541\\
-15.052
\end{array} \right]
=\left[
\begin{array}{c}
-24.3815\\
 64.3684\\
 16.5928
\end{array} \right]
\end{equation}\begin{equation} x^{(4)}=
\left[
\begin{array}{c}
6.25\\31.25\\12.5
\end{array} \right]
-\left[
\begin{array}{ccc}
0 &         2.5       &  1.875\\
0.25      &  0       &  0.625\\
0.25      &  0.83333333&  0\\
\end{array} \right]
\left[
\begin{array}{c}
-24.3815\\
 64.3684\\
 16.5928
\end{array} \right]
=\left[
\begin{array}{c}
-185.782\\
  26.974\\
 -35.045
\end{array} \right]
\end{equation}

## Ex.2
We know the Taylor expansion of function $f(x+h)$ is
$$f(x+h)=f{\left (x \right )} + h  \frac{d}{d x} f{\left (x \right )} + \frac{h^{2}}{2}  \frac{d^{2}}{d x^{2}}  f{\left (x \right )} + \frac{h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}  + \frac{h^{4}}{24} \frac{d^{4}}{d x^{4}}  f{\left (x \right )} +\mathcal{O}\left(h^{4}\right)$$

Therefore, we write down Taylor expansion for
$$f(x-2h)=f{\left (x \right )} -2 h  \frac{d}{d x} f{\left (x \right )} + \frac{4h^{2}}{2}  \frac{d^{2}}{d x^{2}}  f{\left (x \right )} - \frac{8h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}  + \frac{16 h^{4}}{24} \frac{d^{4}}{d x^{4}}  f{\left (x \right )} +\mathcal{O}\left(h^{5}\right)\\
f(x-h)=f{\left (x \right )} - h  \frac{d}{d x} f{\left (x \right )} + \frac{h^{2}}{2}  \frac{d^{2}}{d x^{2}}  f{\left (x \right )} - \frac{h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}  + \frac{h^{4}}{24} \frac{d^{4}}{d x^{4}}  f{\left (x \right )} +\mathcal{O}\left(h^{5}\right)\\
f(x+h)=f{\left (x \right )} + h  \frac{d}{d x} f{\left (x \right )} + \frac{h^{2}}{2}  \frac{d^{2}}{d x^{2}}  f{\left (x \right )} + \frac{h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}  + \frac{h^{4}}{24} \frac{d^{4}}{d x^{4}}  f{\left (x \right )} +\mathcal{O}\left(h^{5}\right)\\
f(x+2h)=f{\left (x \right )} + 2h  \frac{d}{d x} f{\left (x \right )} + \frac{4h^{2}}{2}  \frac{d^{2}}{d x^{2}}  f{\left (x \right )} + \frac{8h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}  + \frac{16h^{4}}{24} \frac{d^{4}}{d x^{4}}  f{\left (x \right )} +\mathcal{O}\left(h^{5}\right)$$

It can be easily observed that
$$\begin{array}{l}
f(x+2h)-f(x-2h)-2\left(f(x+h)+f(x-h)\right)\\
=4 h  \frac{d}{d x} f{\left (x \right )} + \frac{16h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}-2\left(2 h  \frac{d}{d x} f{\left (x \right )} + \frac{2h^{3}}{6}  \frac{d^{3}}{d x^{3}}  f{\left (x \right )}\right)  +\mathcal{O}\left(h^{5}\right)\\
= 2h^3  \frac{d^{3}}{d x^{3}}  f{\left (x \right )} +\mathcal{O}\left(h^{5}\right)
\end{array}$$

Therefore, we obtain the centered finite-difference approximation to the third derivate that is second-order accurate:
$$ \frac{d^{3}}{d x^{3}}  f{\left (x \right )} =\frac{f(x+2h)-f(x-2h)-2\left(f(x+h)+f(x-h)\right)}{2} +\mathcal{O}\left(h^{2}\right)$$

## Ex.3
Firstly, we derive the governing equation of the given system.
$$V=\pi D^4h/4 \\
\frac{dV}{dt}=-Q_{out}=-CA\sqrt{2gh}$$

Substitute the given parameters into the equation, we obtain:
$$\frac{dh}{dt}=-\frac{0.55 \sqrt{2gh}}{10^4}\\
h(0)=2.75$$

The exact time that the tank will be empty is 13620.9 second.

Inserting 4th order RK method in to Excel for this problem, we have following plot for $h(t)$

Meanwhile, iteration of RK methods with step $t=1000$s is attached:

        t
        h
        f
        k1
        k2
        k3
        k4

        0
        2.75
        -0.000403799
        -0.403799068
        -0.388693465
        -0.3892691
        -0.374129719

        1000
        2.361024347
        -0.000374153
        -0.374152971
        -0.359024038
        -0.359648126
        -0.344479461

        2000
        2.001694887
        -0.000344507
        -0.344506885
        -0.329350421
        -0.330031862
        -0.314827988

        3000
        1.672011647
        -0.000314861
        -0.314860815
        -0.299671369
        -0.300421766
        -0.285174771

        4000
        1.371974671
        -0.000285215
        -0.28521477
        -0.269985097
        -0.270819965
        -0.255518962

        5000
        1.101584029
        -0.000255569
        -0.255568769
        -0.240288934
        -0.241229683
        -0.225859126

        6000
        0.86083984
        -0.000225923
        -0.225922843
        -0.210578713
        -0.211656061
        -0.196192678

        7000
        0.649742329
        -0.000196277
        -0.196277061
        -0.180847533
        -0.182107814
        -0.166514572

        8000
        0.468291941
        -0.000166632
        -0.166631578
        -0.151083097
        -0.152600965
        -0.136813845

        9000
        0.316489683
        -0.000136987
        -0.136986807
        -0.121261112
        -0.123168435
        -0.10706284

        10000
        0.194338226
        -0.000107344
        -0.107344076
        -0.091325871
        -0.093889789
        -0.077173919

        11000
        0.101846673
        -7.77092E-05
        -0.077709191
        -0.061114172
        -0.065014768
        -0.046731644

        12000
        0.039063554
        -4.81266E-05
        -0.048126563
        -0.029822809
        -0.037842242
        -0.008509661

        13000
        0.007069167
        -2.04731E-05
        -0.020473075
        #NUM!
        #NUM!
        #NUM!

Therefore, approximately after 13500 seconds (with step of 100), the tank will be empty. Comparing to the exact
solution, this result is acceptable.

The online version of this homework (including the [Excel worksheet](https://wanglongqi.github.io/public/res/RK4.xlsx)) is upload to [http://goo.gl/ALDR1E](http://goo.gl/ALDR1E).