#  CS-109A Introduction to Data Science

## Lab 11: Neural Network Basics - Introduction to `tf.keras`

**Harvard University**

**Fall 2019**

**Instructors:** Pavlos Protopapas, Kevin Rader, Chris Tanner

**Lab Instructors:** Chris Tanner and Eleni Kaxiras.

**Authors:** Eleni Kaxiras, David Sondak, and Pavlos Protopapas.

```python
## RUN THIS CELL TO PROPERLY HIGHLIGHT THE EXERCISES
import requests
from IPython.core.display import HTML
styles = requests.get("https://raw.githubusercontent.com/Harvard-IACS/2018-CS109A/master/content/styles/cs109.css").text
HTML(styles)
```

```python
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import pandas as pd
%matplotlib inline

from PIL import Image
```

```python
from __future__ import absolute_import, division, print_function, unicode_literals

# TensorFlow and tf.keras
import tensorflow as tf

tf.keras.backend.clear_session()  # For easy reset of notebook state.

print(tf.__version__)  # You should see a 2.0.0 here!
```

    2.0.0

#### Instructions for running `tf.keras` with Tensorflow 2.0:

1. Create a `conda` virtual environment by cloning an existing one that you know works
```
conda create --name myclone --clone myenv
```

2. Go to [https://www.tensorflow.org/install/pip](https://www.tensorflow.org/install/pip) and follow instructions for your machine.

3. In a nutshell:
```
pip install --upgrade pip
pip install tensorflow==2.0.0
```
All references to Keras should be written as `tf.keras`.  For example:

```
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

tf.keras.models.Sequential
tf.keras.layers.Dense, tf.keras.layers.Activation,
tf.keras.layers.Dropout, tf.keras.layers.Flatten, tf.keras.layers.Reshape
tf.keras.optimizers.SGD
tf.keras.preprocessing.image.ImageDataGenerator
tf.keras.regularizers
tf.keras.datasets.mnist
```

You could avoid the long names by using
```
from tensorflow import keras
from tensorflow.keras import layers
```
These imports do not work on some systems, however, because they pick up previous versions of `keras` and `tensorflow`. That is why I avoid them in this lab.

## Learning Goals
In this lab we will understand the basics of neural networks and how to start using a deep learning library called `keras`. By the end of this lab, you should:

- Understand how a simple neural network works and code some of its functionality from scratch.
- Be able to think and do calculations in matrix notation. Also think of vectors and arrays as tensors.
- Know how to install and run `tf.keras`.
- Implement a simple real world example using a neural network.

## Part 1: Neural Networks 101

Suppose we have an input vector $X=${$x_1, x_2, ... x_L$} to a $k$-layered network.

Each layer has its own number of nodes. For the first layer in our drawing that number is $J$. We can store the weights for each node in a vector $\mathbf{W} \in \mathbb{R}^{JxL+1}$ (accounting for bias).  Similarly, we can store the biases from each node in a vector $\mathbf{b} \in \mathbb{R}^{I}$.  The affine transformation is then written as $$\mathbf{a} = \mathbf{W^T}X + \mathbf{b}$$
  What we then do is "absorb" $\mathbf{b}$ into $X$ by adding a column of ones to $X$. Our $X$ matrix than becomes $\mathbf{X} \in \mathbb{R}^{JxL+1}$ and our equation:

$$\mathbf{a} = \mathbf{W^T}_{plusones}X$$
We have that $\mathbf{a} \in \mathbb{R}^{J}$ as well.  Next we evaluate the output from each node.  We write $$\mathbf{u} = \sigma\left(\mathbf{a}\right)$$ where $\mathbf{u}\in\mathbb{R}^{J}$.  We can think of $\sigma$ operating on each individual element of $\mathbf{a}$ separately or in matrix notation. If we denote each component of $\mathbf{a}$ by $a_{j}$ then we can write $$u_{j} = \sigma\left(a_{j}\right), \quad j = 1, ... J.$$
 In our code we will implement all these equations in matrix notation.
`tf.keras` (Tensorflow) and `numpy` perform the calculations in matrix format.

*Image source: "Modern Mathematical Methods for Computational Science and Engineering" Efthimios Kaxiras and Athanassios Fokas.*

Let's assume that we have 3 input points (L = 3), two hidden layers ($k=2$), and 2 nodes in each layer ($J=2$)

### Input Layer

$𝑋$={$𝑥_1,𝑥_2,x_3$}

### First Hidden Layer

\begin{equation}
 \begin{aligned}
a^{(1)}_1 = w^{(1)}_{10} + w^{(1)}_{11}x_1 + w^{(1)}_{12}x_2 + w^{(1)}_{13}x_3 \\
a^{(1)}_2 = w^{(1)}_{20} + w^{(1)}_{21}x_1 + w^{(1)}_{22}x_2 + w^{(1)}_{23}x_3 \\
\end{aligned}
\end{equation}

 All this in matrix notation: $$\mathbf{a} = \mathbf{W^T}X$$

 NOTE: in $X$ we have added a column of ones to account for the bias

**Then the sigmoid is applied**:
\begin{equation}
 \begin{aligned}
u^{(1)}_1 = \sigma(a^{(1)}_1) \\
u^{(1)}_2 = \sigma(a^{(1)}_2) \\
\end{aligned}
\end{equation}

or in matrix notation: $$\mathbf{u} = \sigma\left(\mathbf{a}\right)$$

### Second Hidden Layer

\begin{equation}
 \begin{aligned}
a^{(2)}_1 = w^{(2)}_{10} + w^{(2)}_{11}u^{(1)}_1 + w^{(2)}_{12}u^{(1)}_2 + w^{(2)}_{13}u^{(1)}_3 \\
a^{(2)}_2 = w^{(2)}_{20} + w^{(2)}_{21}u^{(1)}_1 + w^{(2)}_{22}u^{(1)}_2 + w^{(2)}_{23}u^{(1)}_3 \\
\end{aligned}
\end{equation}

**Then the sigmoid is applied**:

\begin{equation}
 \begin{aligned}
u^{(2)}_1 = \sigma(a^{(2)}_1) \\
u^{(2)}_2 = \sigma(a^{(2)}_2) \\
\end{aligned}
\end{equation}

### Output Layer

#### If the output is categorical:

For example with four classes ($M=4$): $Y$={$y_1, y_2, y_3, y_4$}, we have the affine and then the sigmoid is lastly applied:

\begin{equation}
 \begin{aligned}
a^{(3)}_1 = w^{(3)}_{10} + w^{(3)}_{11}u^{(2)}_1 + w^{(3)}_{12}u^{(2)}_2 \\
a^{(3)}_2 = w^{(3)}_{20} + w^{(3)}_{21}u^{(2)}_1 + w^{(3)}_{22}u^{(2)}_2 \\
a^{(3)}_3 = w^{(3)}_{30} + w^{(3)}_{31}u^{(2)}_1 + w^{(3)}_{32}u^{(2)}_2 \\
a^{(3)}_4 = w^{(3)}_{40} + w^{(3)}_{41}u^{(2)}_1 + w^{(3)}_{42}u^{(2)}_2 \\
\end{aligned}
\end{equation}

\begin{equation}
 \begin{aligned}
y_1 = \sigma(a^{(3)}_1) \\
y_2 = \sigma(a^{(3)}_2) \\
y_3 = \sigma(a^{(3)}_3) \\
y_3 = \sigma(a^{(3)}_4) \\
\end{aligned}
\end{equation}
$\sigma$ will be softmax in the case of multiple classes and sigmoid for binary.

#### If the output is a number (regression):

We have a single y as output:

\begin{equation}
 \begin{aligned}
y = w^{(3)}_{10}+ w^{(3)}_{11}u^{(2)}_1 + w^{(3)}_{12}u^{(2)}_2 + w^{(3)}_{13}u^{(2)}_3 \\
\end{aligned}
\end{equation}

#### Matrix Multiplication and constant addition

```python
# make two arrays and multiply them using np.dot(a, b) or tf.matmul(a, b)
```

```python
# both Tensorflow and numpy take care of transposing.

```

```python
# how do we add the constant in the matrix

```

In class exercise : Plot the sigmoid

Define the `sigmoid` and the `tanh`. For `tanh` you may use `np.tanh` and for the `sigmoid` use the general equation:
\begin{align}
\sigma = \dfrac{1}{1+e^{-2(x-c)/a}} \qquad\text{(1.1)}
\textrm{}
\end{align}

Generate a list of 500 $x$ points from -5 to 5 and plot both functions. What do you observe? What do variables $c$ and $a$ do?

```python
# your code here
```

```python
# %load solutions/sigmoid.py
# The smaller the `a`, the sharper the function is.
# Variable `c` moves the function along the x axis
def sigmoid(x,c,a):
    z = ((x-c)/a)
    return 1.0 / (1.0 + np.exp(-z))

x = np.linspace(-5.0, 5.0, 500) # input points
c = 1.
a = 0.5
plt.plot(x, sigmoid(x, c, a), label='sigmoid')
plt.plot(x, np.tanh(x), label='tanh')
plt.grid();
plt.legend();

```

2. In class exercise: Approximate a Gaussian function using a node and manually adjusting the weights. Start with one layer with one node and move to two nodes.

The task is to approximate (learn) a function $f\left(x\right)$ given some input $x$.  For demonstration purposes, the function we will try to learn is a Gaussian function:
\begin{align}
f\left(x\right) = e^{-x^{2}}
\textrm{}
\end{align}

Even though we represent the input $x$ as a vector on the computer, you should think of it as a single input.

#### 2.1 Start by plotting the above function using the $x$ dataset

```python
x = np.linspace(-5.0, 5.0, 500) # input points
def gaussian(x):
    return np.exp(-x*x)

f = gaussian(x)
plt.plot(x, f, label='gaussian')
plt.legend()
```

```python
f.shape
```

    (500,)

#### 2.2 Now, let's code the single node as per the image above.

Write a function named `affine` that does the transformation. The definition is provided below. Then create a simpler sigmoid with just one variable. We choose a **sigmoid** activation function and specifically the **logistic** function.  Sigmoids are a family of functions and the logistic function is just one member in that family. $$\sigma\left(z\right) = \dfrac{1}{1 + e^{-z}}.$$

Define both functions in code.

```python
def affine(x, w, b):
    """Return affine transformation of x

    INPUTS
    ======
    x: A numpy array of points in x
    w: An array representing the weight of the perceptron
    b: An array representing the biases of the perceptron

    RETURN
    ======
    z: A numpy array of points after the affine transformation
       z = wx + b
    """

    # Code goes here
    return z
```

```python
# your code here

```

```python
# %load solutions/affine-sigmoid.py
def affine(x, w, b):
    return w * x + b

def sigmoid(z):
    return 1.0 / (1.0 + np.exp(-z))
```

And now we plot the activation function and the true function. What do you think will happen if you change $w$ and $b$?

```python
w = [.., .., ..] # Create a list of weights to try
b = [.., .., ..] # Create a list of biases

fig, ax = plt.subplots(1,1, figsize=(9,5))
SIZE = 16

# plot our true function, the gaussian
ax.plot(x, f, lw=4, ls='-.', label='True function')

# plot 3 "networks"
for wi, bi in zip(w, b):
    h = sigmoid(affine(x, wi, bi))
    ax.plot(x, h, lw=4, label=r'$w = {0}$, $b = {1}$'.format(wi,bi))

ax.set_title('Single neuron network', fontsize=SIZE)

# Create labels (very important!)
ax.set_xlabel('$x$', fontsize=SIZE) # Notice we make the labels big enough to read
ax.set_ylabel('$y$', fontsize=SIZE)

ax.tick_params(labelsize=SIZE) # Make the tick labels big enough to read

ax.legend(fontsize=SIZE, loc='best') # Create a legend and make it big enough to read
```

We didn't do an exhaustive search of the weights and biases, but it sure looks like this single perceptron is never going to match the actual function.  Again, we shouldn't be suprised about this.  The output layer of the network is simple the logistic function, which can only have so much flexibility.

Let's try to make our network more flexible by using **more nodes**!

### Multiple Perceptrons in a Single Layer
It appears that a single neuron is somewhat limited in what it can accomplish.  What if we expand the number of nodes/neurons in our network?  We have two obvious choices here.  One option is to add depth to the network by putting layers next to each other.  The other option is to stack neurons on top of each other in the same layer.  Now the network has some width, but is still only one layer deep.

```python
x = np.linspace(-5.0, 5.0, 500) # input points
f = np.exp(-x*x) # data

w = np.array([..., ...]) # ENTER TWO VALUES HERE
b = np.array([..., ...]) # HERE TOO

# Affine transformations
z1 = w[0] * x + b[0]
z2 = w[1] * x + b[1]

# Node outputs
h1 = sigmoid(z1)
h2 = sigmoid(z2)
```

Now let's plot things and see what they look like.

```python
fig, ax = plt.subplots(1,1, figsize=(9,5))

ax.plot(x, f, lw=4, ls = '-.', label='True function')
ax.plot(x, h1, lw=4, label='First neuron')
ax.plot(x, h2, lw=4, label='Second neuron')

# Set title
ax.set_title('Comparison of Neuron Outputs', fontsize=SIZE)

# Create labels (very important!)
ax.set_xlabel('$x$', fontsize=SIZE) # Notice we make the labels big enough to read
ax.set_ylabel('$y$', fontsize=SIZE)

ax.tick_params(labelsize=SIZE) # Make the tick labels big enough to read

ax.legend(fontsize=SIZE, loc='best') # Create a legend and make it big enough to read
```

Just as we expected.  Some sigmoids.  Of course, to get the network prediction we must combine these two sigmoid curves somehow.  First we'll just add $h_{1}$ and $h_{2}$ without any weights to see what happens.

#### Note
We are **not** doing classification here.  We are trying to predict an actual function.  The sigmoid activation is convenient when doing classification because you need to go from $0$ to $1$.  However, when learning a function, we don't have as good of a reason to choose a sigmoid.

```python
# Network output
wout = np.ones(2) # Set the output weights to unity to begin
bout = -1 # bias
yout = wout[0] * h1 + wout[1] * h2 + bout
```

And plot.

```python
fig, ax = plt.subplots(1,1, figsize=(9,5))

ax.plot(x, f, ls='-.', lw=4, label=r'True function')
ax.plot(x, yout, lw=4, label=r'$y_{out} = h_{1} + h_{2}$')

# Create labels (very important!)
ax.set_xlabel('$x$', fontsize=SIZE) # Notice we make the labels big enough to read
ax.set_ylabel('$y$', fontsize=SIZE)

ax.tick_params(labelsize=SIZE) # Make the tick labels big enough to read

ax.legend(fontsize=SIZE, loc='best') # Create a legend and make it big enough to read
```

Very cool!  The two nodes interact with each other to produce a pretty complicated-looking function.  It still doesn't match the true function, but now we have some hope.  In fact, it's starting to look a little bit like a Gaussian!

We can do better.  There are three obvious options at this point:
1. Change the number of nodes
2. Change the activation functions
3. Change the weights

#### We will leave this simple example for some other time! Let's move on to fashion items!

## Part 2: Tensors, Fashion, and Reese Witherspoon

We can think of tensors as multidimensional arrays of real numerical values; their job is to generalize matrices to multiple dimensions. While tensors first emerged in the 20th century, they have since been applied to numerous other disciplines, including machine learning. Tensor decomposition/factorization can solve, among other, problems in unsupervised learning settings, temporal and multirelational data. For those of you that will get to handle images for Convolutional Neural Networks, it's a good idea to have the understanding of tensors of rank 3.

We will use the following naming conventions:

- scalar = just a number = rank 0 tensor  ($a$ ∈ $F$,)

- vector = 1D array = rank 1 tensor ( $x = (\;x_1,...,x_i\;)⊤$ ∈ $F^n$ )

- matrix = 2D array = rank 2 tensor ( $\textbf{X} = [a_{ij}] ∈ F^{m×n}$ )

- 3D array = rank 3 tensor ( $\mathscr{X} =[t_{i,j,k}]∈F^{m×n×l}$ )

- $\mathscr{N}$D array = rank $\mathscr{N}$ tensor ( $\mathscr{T} =[t_{i1},...,t_{i\mathscr{N}}]∈F^{n_1×...×n_\mathscr{N}}$ ) <-- Things start to get complicated here...

#### Tensor indexing
We can create subarrays by fixing some of the given tensor’s indices. We can create a vector by fixing all but one index. A 2D matrix is created when fixing all but two indices. For example, for a third order tensor the vectors are

$\mathscr{X}[:,j,k]$ = $\mathscr{X}[j,k]$ (column),

$\mathscr{X}[i,:,k]$ = $\mathscr{X}[i,k]$ (row), and

$\mathscr{X}[i,j,:]$ = $\mathscr{X}[i,j]$ (tube)

#### Tensor multiplication
We can multiply one matrix with another as long as the sizes are compatible ((n × m) × (m × p) = n × p), and also multiply an entire matrix by a constant. Numpy `numpy.dot` performs a matrix multiplication which is straightforward when we have 2D or 1D arrays. But what about > 3D arrays? The function will choose according to the matching dimentions but if we want to choose we should use `tensordot`, but, again, we **do not need tensordot** for this class.

### Reese Witherspoon

This image is from the dataset [Labeled Faces in the Wild](http://vis-www.cs.umass.edu/lfw/person/Reese_Witherspoon.html) used for machine learning training. Images are 24-bit RGB images (height, width, channels) with 8 bits for each of R, G, B channel. Explore and print the array.

```python
# load and show the image
FILE = '../fig/Reese_Witherspoon.jpg'
#img = np.array(Image.open(FILE)) # this works as well
img = mpimg.imread(FILE)
imgplot = plt.imshow(img)
```

```python
print(f'The image is a: {type(img)} of shape {img.shape}')
img[3:5, 3:5, :]
```

    The image is a: <class 'numpy.ndarray'> of shape (150, 150, 3)

    array([[[241, 241, 241],
            [242, 242, 242]],

           [[241, 241, 241],
            [242, 242, 242]]], dtype=uint8)

#### Slicing tensors: slice along each axis

```python
# we want to show each color channel
fig, axes = plt.subplots(1, 3, figsize=(10,10))
for i, subplot in zip(range(3), axes):
    temp = np.zeros(img.shape, dtype='uint8')
    temp[:,:,i] = img[:,:,i]
    subplot.imshow(temp)
    subplot.set_axis_off()
plt.show()
```

#### Multiplying Images with a scalar (just for fun, does not really help us in any way)

```python
temp = img
temp = temp * 2
plt.imshow(temp)
```

For more on image manipulation by `matplotlib` see: [matplotlib-images](https://matplotlib.org/3.1.1/tutorials/introductory/images.html)

### Anatomy of an Artificial Neural Network

In Part 1 we hand-made a neural network by writing some simple python functions.  We focused on a regression problem where we tried to learn a function. We practiced using the logistic activation function in a network with multiple nodes, but a single or two hidden layers.  Some of the key observations were:
* Increasing the number of nodes allows us to represent more complicated functions
* The weights and biases have a very big impact on the solution
* Finding the "correct" weights and biases is really hard to do manually
* There must be a better method for determining the weights and biases automatically

We also didn't assess the effects of different activation functions or different network depths.

###
https://www.tensorflow.org/guide/keras

`tf.keras` is TensorFlow's high-level API for building and training deep learning models. It's used for fast prototyping, state-of-the-art research, and production. `Keras` is a library created by François Chollet. After Google released Tensorflow 2.0, the creators of `keras` recommend that "Keras users who use multi-backend Keras with the TensorFlow backend switch to `tf.keras` in TensorFlow 2.0. `tf.keras` is better maintained and has better integration with TensorFlow features".

#### IMPORTANT:  In `Keras` everything starts with a Tensor of N samples as input and ends with a Tensor of N samples as output.

### The 3 parts of an ANN

- **Part 1: the input layer** (our dataset)
- **Part 2: the internal architecture or hidden layers** (the number of layers, the activation functions, the learnable parameters and other hyperparameters)
- **Part 3: the output layer** (what we want from the network)

In the rest of the lab we will practice with end-to-end neural network training

1. Load the data
2. Define the layers of the model.
3. Compile the model.
4. Fit the model to the train set (also using a validation set).
5. Evaluate the model on the test set.
6. Plot metrics such as accuracy.
7. Predict on random images from test set.
8. Predict on  a random image from the web!

```python
seed = 7
np.random.seed(seed)
```

### Fashion MNIST

MNIST, the set of handwritten digits is considered the Drosophila of Machine Learning. It has been overused, though, so we will try a slight modification to it.

**Fashion-MNIST** is a dataset of clothing article images (created by [Zalando](https://github.com/zalandoresearch/fashion-mnist)), consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a **28 x 28** grayscale image, associated with a label from **10 classes**. The creators intend Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits. Each pixel is 8 bits so its value ranges from 0 to 255.

Let's load and look at it!

#### 1. Load the data

```python
%%time
# get the data from keras

# load the data splitted in train and test! how nice!

# normalize the data by dividing with pixel intensity
# (each pixel is 8 bits so its value ranges from 0 to 255)

# classes are named 0-9 so define names for plotting clarity
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

# plot
plt.figure(figsize=(10,10))
for i in range(25):
    plt.subplot(5,5,i+1)
    plt.xticks([])
    plt.yticks([])
    plt.grid(False)
    plt.imshow(x_train[i], cmap=plt.cm.binary)
    plt.xlabel(class_names[y_train[i]])
plt.show()
```

```python
plt.imshow(x_train[3], cmap=plt.cm.binary)
```

```python
x_train.shape, x_test.shape
```

```python
y_train.shape
```

#### 2. Define the layers of the model.

```python

```

#### 3. Compile the model

```python

```

```python
model.summary()
```

    Model: "sequential"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    flatten (Flatten)            (None, 784)               0
    _________________________________________________________________
    dense (Dense)                (None, 154)               120890
    _________________________________________________________________
    dense_1 (Dense)              (None, 64)                9920
    _________________________________________________________________
    dropout (Dropout)            (None, 64)                0
    _________________________________________________________________
    dense_2 (Dense)              (None, 10)                650
    =================================================================
    Total params: 131,460
    Trainable params: 131,460
    Non-trainable params: 0
    _________________________________________________________________

```python
tf.keras.utils.plot_model(
    model,
    #to_file='model.png', # if you want to save the image
    show_shapes=True, # True for more details than you need
    show_layer_names=True,
    rankdir='TB',
    expand_nested=False,
    dpi=96
)
```

[Everything you wanted to know about a Keras Model and were afraid to ask](https://www.tensorflow.org/api_docs/python/tf/keras/Model)

#### 4. Fit the model to the train set (also using a validation set)

This is the part that takes the longest.

-----------------------------------------------------------
**ep·och**

noun: epoch; plural noun: epochs. A period of time in history or a person's life, typically one marked by notable events or particular characteristics. Examples: "the Victorian epoch", "my Neural Netwok's epochs".

-----------------------------------------------------------

```python
%%time

# the core of the network training

```

#### Save the model

You can save the model so you do not have `.fit` everytime you reset the kernel in the notebook. Network training is expensive!

For more details on this see [https://www.tensorflow.org/guide/keras/save_and_serialize](https://www.tensorflow.org/guide/keras/save_and_serialize)

```python
# save the model so you do not have to run the code everytime
model.save('fashion_model.h5')

# Recreate the exact same model purely from the file
#model = tf.keras.models.load_model('fashion_model.h5')
```

#### 5. Evaluate the model on the test set.

```python
test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
print(f'Test accuracy={test_accuracy}')
```

    Test accuracy=0.8866999745368958

#### 6. We learn a lot by studying History! Plot metrics such as accuracy.

You can learn a lot about neural networks by observing how they perform while training. You can issue `kallbacks` in `keras`. The networks's performance is stored in a `keras` callback aptly named `history` which can be plotted.

```python
print(history.history.keys())
```

    dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])

```python
# plot accuracy and loss for the test set
fig, ax = plt.subplots(1,2, figsize=(20,6))

ax[0].plot(history.history['accuracy'])
ax[0].plot(history.history['val_accuracy'])
ax[0].set_title('Model accuracy')
ax[0].set_ylabel('accuracy')
ax[0].set_xlabel('epoch')
ax[0].legend(['train', 'val'], loc='best')

ax[1].plot(history.history['loss'])
ax[1].plot(history.history['val_loss'])
ax[1].set_title('Model loss')
ax[1].set_ylabel('loss')
ax[1].set_xlabel('epoch')
ax[1].legend(['train', 'val'], loc='best')
```

#### 7. Now let's use the Network for what it was meant to do: Predict!

```python
predictions = model.predict(x_test)
```

```python
predictions[0]
```

```python
np.argmax(predictions[0]), class_names[np.argmax(predictions[0])]
```

Let's see if our network predicted right! Is the first item what was predicted?

```python
plt.figure()
plt.imshow(x_test[0], cmap=plt.cm.binary)
plt.xlabel(class_names[y_test[0]])
plt.colorbar()
```

**Correct!!** Now let's see how confident our model is by plotting the probability values:

```python
# code source: https://www.tensorflow.org/tutorials/keras/classification
def plot_image(i, predictions_array, true_label, img):
    predictions_array, true_label, img = predictions_array, true_label[i], img[i]
    plt.grid(False)
    plt.xticks([])
    plt.yticks([])

    plt.imshow(img, cmap=plt.cm.binary)

    predicted_label = np.argmax(predictions_array)
    if predicted_label == true_label:
        color = 'blue'
    else:
        color = 'red'

    plt.xlabel("{} {:2.0f}% ({})".format(class_names[predicted_label],
                                100*np.max(predictions_array),
                                class_names[true_label]),
                                color=color)

def plot_value_array(i, predictions_array, true_label):
    predictions_array, true_label = predictions_array, true_label[i]
    plt.grid(False)
    plt.xticks(range(10))
    plt.yticks([])
    thisplot = plt.bar(range(10), predictions_array, color="#777777")
    plt.ylim([0, 1])
    predicted_label = np.argmax(predictions_array)

    thisplot[predicted_label].set_color('red')
    thisplot[true_label].set_color('blue')
```

```python
i = 406
plt.figure(figsize=(6,3))
plt.subplot(1,2,1)
plot_image(i, predictions[i], y_test, x_test)
plt.subplot(1,2,2)
plot_value_array(i, predictions[i],  y_test)
plt.show()
```

#### 8. Predicting in the real world

Let's see if our network can generalize beyond the MNIST fashion dataset. Let's give it an random googled image of a boot. Does it have to be a clothing item resembling the MNIST fashion dataset? Can it be a puppy?

**Download an image from the internet and resize it to 28x28.**

```python
# your code here.
# Resize it to 28 x 28 and one channel (you could do this outside the notebook)
# make into one channel and see .shape

```

`tf.keras` models are optimized to make predictions on a batch, or collection, of examples at once. Accordingly, even though you're using a single image, you need to add it to a list:

```python
# your code here
# Add the image to a batch where it's the only member.

```

```python
# your code here
# print the prediction

```


## Definitions

### Observables
+ time $t_{k}$
+ observable $y_{k}$

### Latent Variables
+ Continuous state $x_{k}$
+ Discrete state $I_{k}$

## Goal
+ Compute joint filter distribution $p(x_{k}, I_{k} \mid y_{1:k})$

## Filter Derivation

1. Use Bayes theorem to define the posterior in terms of a data-derived likelihood and a prior ($ posterior \propto likelihood * prior $)
$$
p(x_{k}, I_{k} \mid y_{1:k}) \propto p(y_{k} \mid x_{k}, I_{k}) * p(x_{k}, I_{k} \mid y_{1:k-1})
$$

2. Use Chapman-Kolmogorov to define prior in terms of previous time step's posterior and state transitions
$$
\begin{align*}
p(x_{k}, I_{k} \mid y_{1:k-1}) &= \sum_{I_{k-1}} \int p(x_{k}, x_{k-1}, I_{k}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1} \tag{Marg.}
\\ &= \sum_{I_{k-1}} \int p(x_{k}, I_{k} \mid x_{k-1}, I_{k-1}, y_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1} \tag{Cond. Prob.}
\\ &= \sum_{I_{k-1}} \int p(x_{k}, I_{k} \mid x_{k-1}, I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1} \tag{Markov}
\\ &= \sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) * Pr(I_{k} \mid I_{k-1}, x_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1} \tag{Cond. Prob.}
\end{align*}$$

    where:
    + $p(x_{k-1}, I_{k-1} \mid y_{1:k-1})$ is the previous posterior
    + $Pr(I_{k} \mid I_{k-1}, x_{k-1})$ is the discrete state transition
    + $p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1})$ is the continuous state transition

3. Final Filter
$$
p(x_{k}, I_{k} \mid y_{1:k}) \propto p(y_{k} \mid x_{k}, I_{k}) * \sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) * Pr(I_{k} \mid I_{k-1}, x_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$

## Smoother Derivation

1.
$$
\begin{align*}
p(x_{k}, I_{k} \mid y_{1:T}) &= \sum_{I_{k+1}} \int p(x_{k}, x_{k+1}, I_{k}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Marg.}
\\ &= \sum_{I_{k+1}} \int p(x_{k}, I_{k} \mid x_{k+1}, I_{k+1}, y_{1:T}) * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Cond. Prob.}
\\ &= \sum_{I_{k+1}} \int p(x_{k}, I_{k} \mid x_{k+1}, I_{k+1}, y_{1:k}) * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Markov}
\\ &= \sum_{I_{k+1}} \int \frac{p(x_{k}, I_{k}, x_{k+1}, I_{k+1} \mid y_{1:k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Cond. Prob.}
\\ &= \sum_{I_{k+1}} \int \frac{p(x_{k+1}, I_{k+1} \mid x_{k}, I_{k}, y_{1:k}) * p(x_{k}, I_{k} \mid y_{1:k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Cond. Prob.}
\\ &= \sum_{I_{k+1}} \int \frac{p(x_{k+1}, I_{k+1} \mid x_{k}, I_{k}) * p(x_{k}, I_{k} \mid y_{1:k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1} \tag{Markov}
\\ &= p(x_{k}, I_{k} \mid y_{1:k}) * \sum_{I_{k+1}} \int \frac{p(x_{k+1}, I_{k+1} \mid x_{k}, I_{k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
\\ &= p(x_{k}, I_{k} \mid y_{1:k}) * \sum_{I_{k+1}} \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}, I_{k}) * Pr(I_{k+1} \mid I_{k}, x_{k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
\end{align*}$$

2.
$$
\begin{align*}
p(x_{k+1}, I_{k+1} \mid y_{1:k}) &= \sum_{I_{k}} \int p(x_{k+1}, I_{k+1}, x_{k}, I_{k} \mid y_{1:k}) * dx_{k} \tag{Marg.}
\\ &= \sum_{I_{k}} \int p(x_{k+1}, I_{k+1} \mid x_{k}, I_{k}, y_{1:k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k} \tag{Cond. Prob.}
\\ &= \sum_{I_{k}} \int p(x_{k+1}, I_{k+1} \mid x_{k}, I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k} \tag{Markov}
\\ &= \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k+1}, I_{k}) * Pr(I_{k+1} \mid I_{k}, x_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k} \tag{Cond. Prob.}
\end{align*}$$

## Replay Trajectory Classification

Let:
+ the likelihood be the same for all discrete states $p(y_{k} \mid x_{k}, I_{k}) = p(y_{k} \mid x_{k})$
+ the continuous state transition depend only on the current discrete state $p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) = p(x_{k} \mid x_{k-1}, I_{k})$
+ and the discrete state transition depend on only the previous state $Pr(I_{k} \mid I_{k-1}, x_{k-1}) = Pr(I_{k} \mid I_{k-1})$.

Then the final filter simplifies to:
$$
p(x_{k}, I_{k} \mid y_{1:k}) \propto p(y_{k} \mid x_{k}) * \sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}) * Pr(I_{k} \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$

Define the state transition as:
$$
  \begin{equation}
    p(x_{k} \mid x_{k-1}, I_{k}) =
    \begin{cases}
      U(a, b), & \text{if}\ I_{k} = 1 \\
      \mathbb{1}, & \text{if}\ I_{k} = 2 \\
      N(x_{k-1}, \sigma), & \text{if}\ I_{k} = 3 \\
    \end{cases}
  \end{equation}
$$

### Computing the `predict` step

+ $I_{k} = 1$
$$
\sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}=1) * Pr(I_{k}=1 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}$$
$$
= U(a, b) * \sum_{I_{k-1}} \int Pr(I_{k}=1 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$
$$\begin{multline*}
= U(a, b) * \big[ Pr(I_{k}=1 \mid I_{k-1}=1) * \int p(x_{k-1}, I_{k-1}=1 \mid y_{1:k-1}) * dx_{k-1}\\ + Pr(I_{k}=1 \mid I_{k-1}=2) * \int p(x_{k-1}, I_{k-1}=2 \mid y_{1:k-1}) * dx_{k-1}\\ + Pr(I_{k}=1 \mid I_{k-1}=3) * \int p(x_{k-1}, I_{k-1}=3 \mid y_{1:k-1}) * dx_{k-1} \big]
\end{multline*}$$
$$\begin{multline*}
= U(a, b) * \big[ Pr(I_{k}=1 \mid I_{k-1}=1) * Pr(I_{k-1}=1 \mid y_{1:k-1})\\ + Pr(I_{k}=1 \mid I_{k-1}=2) * Pr(I_{k-1}=2 \mid y_{1:k-1})\\ + Pr(I_{k}=1 \mid I_{k-1}=3) * Pr(I_{k-1}=3 \mid y_{1:k-1}) \big]
\end{multline*}$$

+ $I_{k} = 2$
$$
\sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}=2) * Pr(I_{k}=1 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$
$$
= \sum_{I_{k-1}} \int Pr(I_{k}=2 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$
$$\begin{multline*}
= Pr(I_{k}=2 \mid I_{k-1}=1) * Pr(I_{k-1}=1 \mid y_{1:k-1})\\ + Pr(I_{k}=2 \mid I_{k-1}=2) * Pr(I_{k-1}=2 \mid y_{1:k-1})\\ + Pr(I_{k}=2 \mid I_{k-1}=3) * Pr(I_{k-1}=3 \mid y_{1:k-1})
\end{multline*}$$
+ $I_{k} = 3$
$$
\sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}=3) * Pr(I_{k}=3 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$
$$
=\sum_{I_{k-1}} \int N(x_{k-1}, \sigma) * Pr(I_{k}=3 \mid I_{k-1}) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1}
$$
$$
=\sum_{I_{k-1}} \big[ Pr(I_{k}=3 \mid I_{k-1}) * \int N(x_{k-1}, \sigma) * p(x_{k-1}, I_{k-1} \mid y_{1:k-1}) * dx_{k-1} \big]
$$

### Smoother `predict` step

Define the state transition as:
$$
  \begin{equation}
    p(x_{k+1} \mid x_{k}, I_{k+1}) =
    \begin{cases}
      U(a, b), & \text{if}\ I_{k+1} = 1 \\
      \mathbb{1}, & \text{if}\ I_{k+1} = 2 \\
      N(x_{k}, \sigma), & \text{if}\ I_{k+1} = 3 \\
    \end{cases}
  \end{equation}
$$

Then the predict step for the smoother is:
$$
p(x_{k+1}, I_{k+1} \mid y_{1:k}) = \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k
+1}) * Pr(I_{k+1} \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$

+ $p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})$
$$
= \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k+1}=1) * Pr(I_{k+1}=1 \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$
$$
= U(a,b) * \sum_{I_{k}} \int Pr(I_{k+1}=1 \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$
$$\begin{multline*}
= U(a,b) * \big[ Pr(I_{k+1}=1 \mid I_{k}=1) * \int p(x_{k}, I_{k}=1 \mid y_{1:k}) * dx_{k} \\ + Pr(I_{k+1}=1 \mid I_{k}=2) * \int p(x_{k}, I_{k}=2 \mid y_{1:k}) * dx_{k}\\ + Pr(I_{k+1}=1 \mid I_{k}=3) * \int p(x_{k}, I_{k}=3 \mid y_{1:k}) * dx_{k} \big]
\end{multline*}$$
$$\begin{multline*}
= U(a,b) * \big[ Pr(I_{k+1}=1 \mid I_{k}=1) * Pr(I_{k}=1 \mid y_{1:k}) \\ + Pr(I_{k+1}=1 \mid I_{k}=2) * Pr(I_{k}=2 \mid y_{1:k})\\ + Pr(I_{k+1}=1 \mid I_{k}=3) * Pr(I_{k}=3 \mid y_{1:k}) \big]
\end{multline*}$$

+ $p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})$
$$
= \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k+1}=2) * Pr(I_{k+1}=2 \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$
$$
= \sum_{I_{k}} \int Pr(I_{k+1}=2 \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$
$$\begin{multline*}
= Pr(I_{k+1}=2 \mid I_{k}=1) * Pr(I_{k}=1 \mid y_{1:k}) \\ + Pr(I_{k+1}=2 \mid I_{k}=2) * Pr(I_{k}=2 \mid y_{1:k})\\ + Pr(I_{k+1}=2 \mid I_{k}=3) * Pr(I_{k}=3 \mid y_{1:k})
\end{multline*}$$

+ $p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})$
$$
= \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k+1}=3) * Pr(I_{k+1}=3 \mid I_{k}) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k}
$$
$$
= \sum_{I_{k}} \big[ Pr(I_{k+1}=3 \mid I_{k}) * \int N(x_{k}, \sigma) * p(x_{k}, I_{k} \mid y_{1:k}) * dx_{k} \big]
$$

### Smoother `Backwards Update` step

$$
\sum_{I_{k+1}} \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}) * Pr(I_{k+1} \mid I_{k})}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
$$

+ $I_{k} = 1$
$$
\sum_{I_{k+1}} \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}) * Pr(I_{k+1} \mid I_{k}=1)}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
$$
$$\begin{multline*}
= \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=1) * Pr(I_{k+1}=1 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=2) * Pr(I_{k+1}=2 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=3) * Pr(I_{k+1}=3 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$
$$\begin{multline*}
= \int \frac{U(a,b) * Pr(I_{k+1}=1 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{Pr(I_{k+1}=2 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{N(x_{k}, \sigma) * Pr(I_{k+1}=3 \mid I_{k}=1)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$

+ $I_{k} = 2$
$$
\sum_{I_{k+1}} \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}) * Pr(I_{k+1} \mid I_{k}=2)}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
$$
$$\begin{multline*}
= \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=1) * Pr(I_{k+1}=1 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=2) * Pr(I_{k+1}=2 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=3) * Pr(I_{k+1}=3 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$
$$\begin{multline*}
= \int \frac{U(a,b) * Pr(I_{k+1}=1 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{Pr(I_{k+1}=2 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{N(x_{k}, \sigma) * Pr(I_{k+1}=3 \mid I_{k}=2)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$

+ $I_{k} = 3$
$$
\sum_{I_{k+1}} \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}) * Pr(I_{k+1} \mid I_{k}=3)}{p(x_{k+1}, I_{k+1} \mid y_{1:k})} * p(x_{k+1}, I_{k+1} \mid y_{1:T}) * dx_{k+1}
$$
$$\begin{multline*}
= \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=1) * Pr(I_{k+1}=1 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=2) * Pr(I_{k+1}=2 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{p(x_{k+1} \mid x_{k}, I_{k+1}=3) * Pr(I_{k+1}=3 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$
$$\begin{multline*}
= \int \frac{U(a,b) * Pr(I_{k+1}=1 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=1 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=1 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{Pr(I_{k+1}=2 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=2 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=2 \mid y_{1:T}) * dx_{k+1}\\ + \int \frac{N(x_{k}, \sigma) * Pr(I_{k+1}=3 \mid I_{k}=3)}{p(x_{k+1}, I_{k+1}=3 \mid y_{1:k})} * p(x_{k+1}, I_{k+1}=3 \mid y_{1:T}) * dx_{k+1}
\end{multline*}$$

```python

```


```python
#Block to calculate the final transformation matrix
import sympy
from sympy import *
a,b,c,d,e,f,d1,d3,d5,d7 = symbols('a b c d e f d1 d3 d5 d7')

A = Matrix([0,0,0,1])

T01 = Matrix([[cos(a), 0, sin(a), 0], [sin(a), 0, -cos(a), 0], [0, 1, 0, d1], [0, 0, 0, 1] ])
T12 = Matrix([[cos(b), 0, -sin(b), 0], [sin(b), 0, cos(b), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
T23 = Matrix([[cos(c), 0, sin(c), 0], [sin(c), 0, -cos(c), 0], [0, 1, 0, d3], [0, 0, 0, 1] ])
T35 = Matrix([[cos(d), -sin(d), 0, d5*cos(d)], [sin(d), cos(d),0, d5*sin(d)], [0, 0, 0, 0], [0, 0, 0, 1] ])
T56 = Matrix([[cos(e), 0, -sin(e), 0], [sin(e), 0, cos(e), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
T67 = Matrix([[cos(f), -sin(f), 0, 0], [sin(f), cos(f),0, 0], [0, 0, 0, d7], [0, 0, 0, 1] ])

T = T01*T12*T23*T35*T56*T67

print(T)
```

    Matrix([[((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*sin(e) + ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*cos(e))*cos(f), -((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*sin(e) + ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*cos(e))*sin(f), 0, -d3*sin(b)*cos(a) + d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e))], [((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*sin(e) + ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*cos(e))*cos(f), -((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*sin(e) + ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*cos(e))*sin(f), 0, -d3*sin(a)*sin(b) + d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e))], [((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*sin(e) + (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*cos(e))*cos(f), -((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*sin(e) + (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*cos(e))*sin(f), 0, d1 + d3*cos(b) + d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e))], [0, 0, 0, 1]])

```python
#Block to generate the jacobian matrix
T02 = T01*T12
T03 = T01*T12*T23
T05 = T01*T12*T23*T35
T06 = T01*T12*T23*T35*T56
T07 = T01*T12*T23*T35*T56*T67

O0 = Matrix([[0], [0], [0]])
O1 = T01[0:3,3]
O2 = T02[0:3,3]
O3 = T03[0:3,3]
O5 = T05[0:3,3]
O6 = T06[0:3,3]
O7 = T07[0:3,3]

Z0 = Matrix([[0], [0], [1]])
Z1 = T01[0:3,2]
Z2 = T02[0:3,2]
Z3 = T03[0:3,2]
Z5 = T05[0:3,2]
Z6 = T06[0:3,2]

B1 = Z0.T.cross((O7-O0).T)
B2 = Z1.T.cross((O7-O1).T)
B3 = Z2.T.cross((O7-O2).T)
B4 = Z3.T.cross((O7-O3).T)
B5 = Z5.T.cross((O7-O5).T)
B6 = Z6.T.cross((O7-O6).T)

J1 = Matrix([[0], [0], [0], [0], [0], [0]])
J2 = Matrix([[0], [0], [0], [0], [0], [0]])
J3 = Matrix([[0], [0], [0], [0], [0], [0]])
J4 = Matrix([[0], [0], [0], [0], [0], [0]])
J5 = Matrix([[0], [0], [0], [0], [0], [0]])
J6 = Matrix([[0], [0], [0], [0], [0], [0]])

J1[0] = B1[0]
J1[1] = B1[1]
J1[2] = B1[2]
J1[3] = Z0[0]
J1[4] = Z0[1]
J1[5] = Z0[2]

J2[0] = B2[0]
J2[1] = B2[1]
J2[2] = B2[2]
J2[3] = Z1[0]
J2[4] = Z1[1]
J2[5] = Z1[2]

J3[0] = B3[0]
J3[1] = B3[1]
J3[2] = B3[2]
J3[3] = Z2[0]
J3[4] = Z2[1]
J3[5] = Z2[2]

J4[0] = B4[0]
J4[1] = B4[1]
J4[2] = B4[2]
J4[3] = Z3[0]
J4[4] = Z3[1]
J4[5] = Z3[2]

J5[0] = B5[0]
J5[1] = B5[1]
J5[2] = B5[2]
J5[3] = Z5[0]
J5[4] = Z5[1]
J5[5] = Z5[2]

J6[0] = B6[0]
J6[1] = B6[1]
J6[2] = B6[2]
J6[3] = Z6[0]
J6[4] = Z6[1]
J6[5] = Z6[2]

J = Matrix([[J1[0], J2[0], J3[0], J4[0], J5[0], J6[0]], \
            [J1[1], J2[1], J3[1], J4[1], J5[1], J6[1]], \
            [J1[2], J2[2], J3[2], J4[2], J5[2], J6[2]], \
            [J1[3], J2[3], J3[3], J4[3], J5[3], J6[3]], \
            [J1[4], J2[4], J3[4], J4[4], J5[4], J6[4]], \
            [J1[5], J2[5], J3[5], J4[5], J5[5], J6[5]] ])    #Final jacobian matrix equation

print(J)
```

    Matrix([[d3*sin(a)*sin(b) - d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) + d5*sin(a)*sin(b)*sin(d) - d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)), -(d3*cos(b) + d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e)))*cos(a), -(d3*cos(b) + d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e)))*sin(a)*sin(b) - (-d3*sin(a)*sin(b) + d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)))*cos(b), (sin(a)*sin(c)*cos(b) - cos(a)*cos(c))*(d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e))) - (d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)))*sin(b)*sin(c), 0, 0], [-d3*sin(b)*cos(a) + d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)), -(d3*cos(b) + d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e)))*sin(a), (d3*cos(b) + d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e)))*sin(b)*cos(a) + (-d3*sin(b)*cos(a) + d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)))*cos(b), -(sin(a)*cos(c) + sin(c)*cos(a)*cos(b))*(d5*sin(b)*cos(c)*cos(d) + d5*sin(d)*cos(b) + d7*((-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e))) + (d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)))*sin(b)*sin(c), 0, 0], [0, (-d3*sin(a)*sin(b) + d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)))*sin(a) + (-d3*sin(b)*cos(a) + d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)))*cos(a), -(-d3*sin(a)*sin(b) + d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)))*sin(b)*cos(a) + (-d3*sin(b)*cos(a) + d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)))*sin(a)*sin(b), (sin(a)*cos(c) + sin(c)*cos(a)*cos(b))*(d5*(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - d5*sin(a)*sin(b)*sin(d) + d7*((-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e))) - (sin(a)*sin(c)*cos(b) - cos(a)*cos(c))*(d5*(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - d5*sin(b)*sin(d)*cos(a) + d7*((-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e))), 0, 0], [0, sin(a), -sin(b)*cos(a), sin(a)*cos(c) + sin(c)*cos(a)*cos(b), 0, (-(-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*sin(d) - sin(b)*cos(a)*cos(d))*cos(e) - ((-sin(a)*sin(c) + cos(a)*cos(b)*cos(c))*cos(d) - sin(b)*sin(d)*cos(a))*sin(e)], [0, -cos(a), -sin(a)*sin(b), sin(a)*sin(c)*cos(b) - cos(a)*cos(c), 0, (-(sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*sin(d) - sin(a)*sin(b)*cos(d))*cos(e) - ((sin(a)*cos(b)*cos(c) + sin(c)*cos(a))*cos(d) - sin(a)*sin(b)*sin(d))*sin(e)], [1, 0, cos(b), sin(b)*sin(c), 0, (-sin(b)*sin(d)*cos(c) + cos(b)*cos(d))*cos(e) - (sin(b)*cos(c)*cos(d) + sin(d)*cos(b))*sin(e)]])

```python
#Workspace Study
import numpy as np
import math
import matplotlib.pyplot as plt

n= 5*5*5*5*5

a = 0*3.14/180    #Initial joint angles
b = 0
c = 0*3.14/180
d = 90*3.14/180
e = 90*3.14/180
f = 0

th = 2*3.14/8

q = [a,b,c,d,e,f]
xg = np.zeros(n)
yg = np.zeros(n)
zg = np.zeros(n)
d1 = 270
d3 = 600
d5 = 550
d7 = 150

A = np.matrix([0,0,0,1])

k=0

for i1 in range (0,5):
    for i2 in range (0,5):
        for i3 in range (0,5):
            for i4 in range (0,5):
                for i5 in range (0,5):
                    T01 = np.matrix([[math.cos(i1*th), 0, math.sin(i1*th), 0], [math.sin(i1*th), 0, -math.cos(i1*th), 0], [0, 1, 0, d1], [0, 0, 0, 1] ])
                    T12 = np.matrix([[math.cos(i2*th+90*3.14/180), 0, -math.sin(i2*th+90*3.14/180), 0], [math.sin(i2*th+90*3.14/180), 0, math.cos(-i2*th+90*3.14/180), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
                    T23 = np.matrix([[math.cos(i3*th), 0, math.sin(i3*th), 0], [math.sin(i3*th), 0, -math.cos(i3*th), 0], [0, 1, 0, d3], [0, 0, 0, 1] ])
                    T35 = np.matrix([[math.cos(-i4*th), -math.sin(-i4*th), 0, d5*math.cos(-i4*th)], [math.sin(-i4*th), math.cos(-i4*th),0, d5*math.sin(-i4*th)], [0, 0, 0, 0], [0, 0, 0, 1] ])
                    T56 = np.matrix([[math.cos(-i5*th), 0, -math.sin(-i5*th), 0], [math.sin(-i5*th), 0, math.cos(-i5*th), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
                    T67 = np.matrix([[math.cos(f), -math.sin(f), 0, 0], [math.sin(f), math.cos(f),0, 0], [0, 0, 0, d7], [0, 0, 0, 1] ])

                    T = T01*T12*T23*T35*T56*T67*A.T

                    xg[k] = T[0]      #Position of end effector at each time instant in the x-z plane
                    yg[k] = T[1]
                    if T[2]>=0:
                       zg[k] = T[2]
                    else:
                       zg[k] = 0
                    k=k+1
j=0

#print(xg)
#Scatter plot in the 3D space
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
#ax.axis([-150, 150, 0, 782])
for j in range(n):
    ax.scatter(xg[j], yg[j], zg[j], color= "red", s=1)
```

```python
#Forward Kinematics validation
import numpy as np
import math
import matplotlib.pyplot as plt

a = 0*3.14/180    #Initial joint angles
b = 0*3.14/180
c = 0*3.14/180
d = 90*3.14/180
e = -90*3.14/180
f = 0

q = [a,b,c,d,e,f]

n=10

xg = np.zeros(n)
yg = np.zeros(n)
zg = np.zeros(n)

d1 = 270
d3 = 600
d5 = 550
d7 = 150

A = np.matrix([0,0,0,1])

for i in range(0,10):
    T01 = np.matrix([[math.cos(a), 0, math.sin(a), 0], [math.sin(a), 0, -math.cos(a), 0], [0, 1, 0, d1], [0, 0, 0, 1] ])
    T12 = np.matrix([[math.cos(b), 0, -math.sin(b), 0], [math.sin(b), 0, math.cos(b), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
    T23 = np.matrix([[math.cos(c), 0, math.sin(c), 0], [math.sin(c), 0, -math.cos(c), 0], [0, 1, 0, d3], [0, 0, 0, 1] ])
    T35 = np.matrix([[math.cos(d), -math.sin(d), 0, d5*math.cos(d)], [math.sin(d), math.cos(d),0, d5*math.sin(d)], [0, 0, 0, 0], [0, 0, 0, 1] ])
    T56 = np.matrix([[math.cos(e), 0, -math.sin(e), 0], [math.sin(e), 0, math.cos(e), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
    T67 = np.matrix([[math.cos(f), -math.sin(f), 0, 0], [math.sin(f), math.cos(f),0, 0], [0, 0, 0, d7], [0, 0, 0, 1] ])

    T = T01*T12*T23*T35*T56*T67*A.T
    xg[i] = T[0]
    yg[i] = T[1]
    zg[i] = T[2]

    b = b+3.14/20

j=0

#print(xg)
#Scatter plot in the 3D space
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
#ax.axis([-150, 150, 0, 782])
for j in range(n):
    ax.scatter(xg[j], yg[j], zg[j], color= "red", s=1)
```

```python
plt.plot(xg, zg, color= "red")
```

```python
#Inverse Kinematics validation and the code used for calculating waypoints that are used in the trajectory_publisher node
import numpy as np
import math
import matplotlib.pyplot as plt

T = 80            #Time taken to reach the fuel tank
delT = 0.1
n= int(T/delT)

a = 0*3.14/180    #Initial joint angles
b = 0
c = 0*3.14/180
d = 90*3.14/180
e = 90*3.14/180
f = 0

q = [a,b,c,d,e,f]
xg = np.zeros(n)
yg = np.zeros(n)
zg = np.zeros(n)
d1 = 270
d3 = 600
d5 = 550
d7 = 150

A = np.matrix([0,0,0,1])

i = 0

for i in range (0,n):
    #Using the updated joint angle values to get the position of end effector at each time instant from the forward kinematics equations
    T01 = np.matrix([[math.cos(q[0]), 0, math.sin(q[0]), 0], [math.sin(q[0]), 0, -math.cos(q[0]), 0], [0, 1, 0, d1], [0, 0, 0, 1] ])
    T12 = np.matrix([[math.cos(q[1]), 0, -math.sin(q[1]), 0], [math.sin(q[1]), 0, math.cos(q[1]), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
    T23 = np.matrix([[math.cos(q[2]), 0, math.sin(q[2]), 0], [math.sin(q[2]), 0, -math.cos(q[2]), 0], [0, 1, 0, d3], [0, 0, 0, 1] ])
    T35 = np.matrix([[math.cos(q[3]), -math.sin(q[3]), 0, d5*math.cos(q[3])], [math.sin(q[3]), math.cos(q[3]),0, d5*math.sin(q[3])], [0, 0, 0, 0], [0, 0, 0, 1] ])
    T56 = np.matrix([[math.cos(q[4]), 0, -math.sin(q[4]), 0], [math.sin(q[4]), 0, math.cos(q[4]), 0], [0, -1, 0, 0], [0, 0, 0, 1] ])
    T67 = np.matrix([[math.cos(q[5]), -math.sin(q[5]), 0, 0], [math.sin(q[5]), math.cos(q[5]),0, 0], [0, 0, 0, d7], [0, 0, 0, 1] ])

    T = T01*T12*T23*T35*T56*T67*A.T

    xg[i] = T[0]      #Position of end effector at each time instant in the x-z plane
    yg[i] = T[1]
    zg[i] = T[2]

    T02 = T01*T12
    T03 = T01*T12*T23
    T05 = T01*T12*T23*T35
    T06 = T01*T12*T23*T35*T56
    T07 = T01*T12*T23*T35*T56*T67

    O0 = np.matrix([[0], [0], [0]])
    O1 = T01[0:3,3]
    O2 = T02[0:3,3]
    O3 = T03[0:3,3]
    O5 = T05[0:3,3]
    O6 = T06[0:3,3]
    O7 = T07[0:3,3]

    Z0 = np.matrix([[0], [0], [1]])
    Z1 = T01[0:3,2]
    Z2 = T02[0:3,2]
    Z3 = T03[0:3,2]
    Z5 = T05[0:3,2]
    Z6 = T06[0:3,2]

    B1 = np.cross(Z0.T, (O7-O0).T)
    B2 = np.cross(Z1.T, (O7-O1).T)
    B3 = np.cross(Z2.T, (O7-O2).T)
    B4 = np.cross(Z3.T, (O7-O3).T)
    B5 = np.cross(Z5.T, (O7-O5).T)
    B6 = np.cross(Z6.T, (O7-O6).T)

    J1 = np.zeros(6)
    J2 = np.zeros(6)
    J3 = np.zeros(6)
    J4 = np.zeros(6)
    J5 = np.zeros(6)
    J6 = np.zeros(6)

    J1[0:3] = B1
    J1[3:6] = Z0.T
    J2[0:3] = B2
    J2[3:6] = Z1.T
    J3[0:3] = B3
    J3[3:6] = Z2.T
    J4[0:3] = B4
    J4[3:6] = Z3.T
    J5[0:3] = B5
    J5[3:6] = Z5.T
    J6[0:3] = B6
    J6[3:6] = Z6.T

    JT = np.zeros((6,6))
    JT[0:6][0] = J1.T
    JT[0:6][1] = J2.T
    JT[0:6][2] = J3.T
    JT[0:6][3] = J4.T
    JT[0:6][4] = J5.T
    JT[0:6][5] = J6.T

    J = JT.T     #Final jacobian matrix

    Jinv = np.linalg.pinv(J)
    if i*delT<=40:
        V = np.matrix([[2*3.14*650*math.cos(2*3.14*i*delT/(4*40))/(4*40)], [-100/40], [-2*3.14*600*math.sin(2*3.14*i*delT/(4*40))/(4*40)], [0], [0], [0]])
    elif i*delT<=80:
        V = np.matrix([[-200/40], [-200/40], [-50/40], [0], [0], [0]])
    q_dot = np.dot(Jinv,V)

    #Updating q values using joint angle velocity obtained in the previous step
    q[0] += delT*q_dot[0]
    q[1] += delT*q_dot[1]
    q[2] += delT*q_dot[2]
    q[3] += delT*q_dot[3]
    q[4] += delT*q_dot[4]
    q[5] += delT*q_dot[5]
    #print(q[5])

j=0

#Scatter plot in the 3D space
fig = plt.figure()
ax = fig.add_subplot(projection='3d')
#ax.axis([-150, 150, 0, 782])
for j in range(n):
    ax.scatter(xg[j], yg[j], zg[j], color= "red", s=1)
```

```python
plt.plot(yg, xg, color= "red")
```


```python
import numpy as np
import itertools
```

```python
num_pulleys=4
```

### Representation used for this code development ###

#### `get_li` ####

For getting inital $L_i$, we assume that the bot is at the `top-center`. With this assumption, we only need to consider `B & D` metrics for calculating $L_i$

```python
def get_Li(B, D):
    """
    The inital length of strings based on B and D
    """
    return 0.5*np.sqrt(B**2 + D**2)
```

### TODO
 - [ ] We need to account for 360 degree rotation here
    - Are you talking about the decrease in radius each time the pulley rotates completely?
 - [x] We need to have a reverse calculation done, i.e. given a $x_i,y_i,z_i$ & $x_f,y_f,z_f$ we need to find out what the inital and final thetas. But more on that later.

#### `get_final_string_length` ####
Considering that we have an inital length available, `delta_thetas` given, this function would provide what would be the final length if the pulleys are moved by `delta_theta`

$$ L_f = L_i + r_i\Delta\theta_i $$

#### `get_theta_from_string_length` ####
If we have final length $L_f$ and inital string length $L_i$ and the radius of the pulleys, then we can find the `delta_thetas` moved by the pulleys

NOTE

Remember, $L_i$ and & $L_f$ are array/list, so the output would also be array/list each corresponding to individual pulleys.

```python
def get_final_string_length(Li, r, delta_thetas=np.zeros((1,num_pulleys))):
    """
    The final length of strings which need to be achieved for
    reaching a specific location (based on thetas).
    """
    return Li + r*delta_thetas

def get_theta_from_string_length(Lf, Li, r):
    """
    Given some string lengths Lf and initial lengths Li, return
    the rotation of the pulleys
    """
    return (Lf - Li)/r
```

#### `get_coordinate` ####
Assumption:
We start from the top of the board, therefore the $L_i$ is same for all strings

1. We take the initial $L_i$ for all strings
2. We are given the `thetas` that the pulley moved
3. We use initial $L_i$ and `thetas` to get final lengths

Then we use following formulas for getting coordinates of the final place the bot moved to

\begin{align}
y &= \frac{l_1^2 - l_2^2 + D^2}{2D} \\
x &= \frac{l_2^2 - l_3^2 + B^2}{2B}
\end{align}

\begin{aligned}
l_1^2 &= (0 - x)^2 + (0 - y)^2 + (H - z)^2 \\ &= x^2 + y^2 + (H - z)^2
\end{aligned}

\begin{aligned}
(H - z)^2 = l_1^2 - x^2 - y^2\\
\end{aligned}

$(H - z)^2$ = Hz2 (Python variable)

```python
def get_coordinate(thetas, B, D, H, r):
    """
    thetas: numpy array of rotations, shape: (num_arrays, )
    B, D, H, r: scalar constants
    """
    Li = get_Li(B, D)
    l = get_final_string_length(Li, r, thetas)
    x = (l[1]**2 - l[2]**2 + B**2)/(2*B)
    y = (l[0]**2 - l[1]**2 + D**2)/(2*D)
    Hz2 = l[0]**2 - x**2 - y**2 # (H-z)^2, eqn 1
    # Is it taut?
    # print(l[0]**2 - l[3]**2, l[1]**2 - l[2]**2)
    z = H - np.sqrt(Hz2)
    return np.array([x, y, z])
```

### `get_thetas` ###

We assume that we always start from the `top` center of the construct that you see on the top.

With the above assumption, the inital length of strings would be taken from `get_Li(B,D)` function.
If we are given the final co-ordinates (x,y,z), we can fund the $Lf$ using following formulas

\begin{align}
\begin{aligned}
l_1^2 &= x^2 + y^2 + (H - z)^2
\end{aligned} \\
\begin{aligned}
l_2^2 &= x^2 + (D - y)^2 + (H - z)^2
\end{aligned} \\
\begin{aligned}
l_3^2 &= (B - x)^2 + (D - y)^2 + (H - z)^2
\end{aligned} \\
\begin{aligned}
l_4^2 &= (B - x)^2 + y^2 + (H - z)^2
\end{aligned} \\
\end{align}

python variable $Hz2 = (H - Z)^2$

```python
def get_thetas(coords, B, D, H, r, back_home=False):
    """
    coords: (x, y, z) coordinates
    B, D, H, r: scalar constants
    """
    l = np.zeros((4,))
    x, y, z = coords
    Hz2 = (H - z)**2 # common term
    l[0] = np.sqrt(x**2 + y**2 + Hz2)
    l[1] = np.sqrt(x**2 + (D - y)**2 + Hz2)
    l[2] = np.sqrt((B - x)**2 + (D - y)**2 + Hz2)
    l[3] = np.sqrt((B - x)**2 + y**2 + Hz2)
    if back_home:
        return get_theta_from_string_length(get_Li(B, D),l, r)
    else:
        return get_theta_from_string_length(l, get_Li(B, D), r)
```

```python
# Box dimensions
B, D, H = 100, 200, 300
# Pulley radius
r = 0.1
# Pulley locations
Pulleys = [(0, 0, H), (0, D, H), (B, D, H), (B, 0, H)]
```

Taking some random values of $\Theta$, we can get the corresponding coordinates from `get_coordinates()`.
If $\Theta$ is such that the ropes are taught, we will get back the original $\Theta$ from `get_thetas()`.
If they aren't, the values will be different. Both are demonstrated in the following example. In the first case, $\theta_4$ is clearly different from the value we set, but in the second case, all angles are exactly the same.

### Below is just a cross-check function. ###
1. Each pulley rotated `[10,20,30,40]` rotations respectively, (i.e 1st pulley, 2nd pulley etc.)
2. With these rotations we get the final co-ordinates, remember we assume we start from top of the structure area.
3. While using the final_cordinates we use `get_thetas` function, i.e how much theta is supposed to be moved, this should come back to thetas we decided earlier.

```python
def display(rotations, coords, theta):
    print(f'For rotation = {rotations/(2*np.pi)}')
    x, y, z = coords
    print(f'\tx = {x}, y = {y}, z = {z}')
    print('\tΘ = ', theta)
    print()

# Pulleys rotated 10 revolutions, 20, 30, and 40, respectively
rotations = np.array([10, 20, 30, 40])*2*np.pi
coords = get_coordinate(rotations, B, D, H, r)
theta = get_thetas(coords, B, D, H, r)/(2*np.pi)
display(rotations, coords, theta)

# Pulleys rotated 10 revolutions, 20, 20, and 10, respectively
rotations = np.array([10, 20, 20, 10])*2*np.pi
coords = get_coordinate(rotations, B, D, H, r)
theta = get_thetas(coords, B, D, H, r)/(2*np.pi)
display(rotations, coords, theta)
```

    For rotation = [10. 20. 30. 40.]
    	x = 41.98822482885034, y = 96.19150450244695, z = 245.88322994588324
    	Θ =  [10.        20.        30.        20.5045589]

    For rotation = [10. 20. 20. 10.]
    	x = 50.0, y = 96.19150450244695, z = 253.18508971098808
    	Θ =  [10. 20. 20. 10.]

As we know that all points are calculated from Top, therefore, we need to 2 sets to data.
1. Move from base $(B/2,D/2,H)$ to the target location
2. Move from some location back to $(B/2,D/2,H)$

In this case, we are looking at a dataset which looks like this

|$X_i$|$Y_i$|$Z_i$|$X_f$|$Y_f$|$Z_f$| B | D | H | r |theta_0|theta_1|theta_2|theta_3|
|-----|-----|-----|-----|-----|-----|---|---|---|---|-------|-------|-------|-------|

```python
B = D = H = np.arange(1,7,0.5)
r = [0.008]

data = np.zeros((1,11))
first=True
element = 0
for b,d,h,r in itertools.product(B,D,H,r):
    x_range = np.arange(0,b,1)
    y_range = np.arange(0,d,1)
    z_range = np.arange(0,h,1)
    for x,y,z in itertools.product(x_range,y_range,z_range):
        thetas_from_home = get_thetas([x,y,z],B=b,D=d,H=h,r=r,back_home=False)
        if first:
            data = np.array([x,y,z,b,d,h,r,thetas_from_home[0],thetas_from_home[1],thetas_from_home[2],thetas_from_home[3]])
            first=False
        else:
            data = np.vstack((data,np.array([x,y,z,b,d,h,r,thetas_from_home[0],thetas_from_home[1],thetas_from_home[2],thetas_from_home[3]])))
            first = False
    if (element+1) % 100 == 0:
        print("Shape : ",data.shape)
    if (element+1) % 1000 == 0:
        print("Completed with %d / %d" %(element+1, 5000))
    element += 1
```

    Shape :  (1192, 11)
    Shape :  (3216, 11)
    Shape :  (7008, 11)
    Shape :  (9792, 11)
    Shape :  (13392, 11)
    Shape :  (18864, 11)
    Shape :  (23616, 11)
    Shape :  (28608, 11)
    Shape :  (35520, 11)
    Shape :  (42656, 11)
    Completed with 1000 / 5000
    Shape :  (48936, 11)
    Shape :  (57216, 11)
    Shape :  (66864, 11)
    Shape :  (74448, 11)
    Shape :  (83808, 11)
    Shape :  (94912, 11)
    Shape :  (105216, 11)

```python
data.shape
```

    (110592, 11)

```python
import pandas as pd
data_df = pd.DataFrame(data,columns=["x_f","y_f","z_f","b","d","h","r","theta0","theta1","theta2","theta3"])
```

```python
data_df.apply(lambda x : x.unique().shape)
```

    x_f          (7,)
    y_f          (7,)
    z_f          (7,)
    b           (12,)
    d           (12,)
    h           (12,)
    r            (1,)
    theta0    (6678,)
    theta1    (9539,)
    theta2    (6681,)
    theta3    (9539,)
    dtype: object

```python
data_df.head(2)
```

      x_f
      y_f
      z_f
      b
      d
      h
      r
      theta0
      theta1
      theta2
      theta3

      0
      0.0
      0.0
      0.0
      1.0
      1.0
      1.0
      0.008
      36.611652
      88.388348
      128.118003
      88.388348

      1
      0.0
      0.0
      0.0
      1.0
      1.0
      1.5
      0.008
      99.111652
      136.958607
      169.305754
      136.958607

Lets verify whether the thetas calculated are correct, by reverse calculated the $x_f,y_f,z_f$ using the theta values

```python
idx = 10
print(get_coordinate(thetas = data_df.iloc[idx,[-4,-3,-2,-1]],
                     B = data_df.iloc[idx,]["b"],
                     D = data_df.iloc[idx,]["d"],
                     H = data_df.iloc[idx,]["h"],
                     r=0.008))
print(data_df.iloc[idx,[0,1,2]].values)
```

    [ 4.44089210e-16 -2.22044605e-16  2.00000000e+00]
    [0. 0. 2.]

```python
data_df.to_csv("suspended_robot_data_r_0.008_B_D_H_1_6_20190826.csv",header=True,index=False)
```

```python
col_l = ["x_f","y_f","z_f","b","d","h","r"]
col_r = ["theta0","theta1","theta2","theta3"]
data_df_model = pd.melt(data_df,id_vars=col_l,
                        value_vars=col_r)
```

```python
data_df_model.head(5)
```

      x_f
      y_f
      z_f
      b
      d
      h
      r
      variable
      value

      0
      0.0
      0.0
      0.0
      1.0
      1.0
      1.0
      0.008
      theta0
      36.611652

      1
      0.0
      0.0
      0.0
      1.0
      1.0
      1.5
      0.008
      theta0
      99.111652

      2
      0.0
      0.0
      1.0
      1.0
      1.0
      1.5
      0.008
      theta0
      -25.888348

      3
      0.0
      0.0
      0.0
      1.0
      1.0
      2.0
      0.008
      theta0
      161.611652

      4
      0.0
      0.0
      1.0
      1.0
      1.0
      2.0
      0.008
      theta0
      36.611652

```python
data_df_model = pd.concat([data_df_model.drop(columns=["variable"],inplace=False),
                           pd.get_dummies(data_df_model.variable,drop_first=False)],
                          axis=1)
```

```python
data_df_model.head()
```

      x_f
      y_f
      z_f
      b
      d
      h
      r
      value
      theta0
      theta1
      theta2
      theta3

      0
      0.0
      0.0
      0.0
      1.0
      1.0
      1.0
      0.008
      36.611652
      1
      0
      0
      0

      1
      0.0
      0.0
      0.0
      1.0
      1.0
      1.5
      0.008
      99.111652
      1
      0
      0
      0

      2
      0.0
      0.0
      1.0
      1.0
      1.0
      1.5
      0.008
      -25.888348
      1
      0
      0
      0

      3
      0.0
      0.0
      0.0
      1.0
      1.0
      2.0
      0.008
      161.611652
      1
      0
      0
      0

      4
      0.0
      0.0
      1.0
      1.0
      1.0
      2.0
      0.008
      36.611652
      1
      0
      0
      0

```python
data_df_model.to_csv("suspended_robot_data_theta_scaled_r_0.008_B_D_H_1_6_20190826.csv",header=True,index=False)
```

```python
import matplotlib.pyplot as plt
```

```python
plt.hist(data_df_model.value,bins=100)
plt.show()
```

```python
from sklearn.preprocessing import StandardScaler
```

```python
obj = StandardScaler()
obj.fit(data_df_model.value.values.reshape(-1,1))
```

    StandardScaler(copy=True, with_mean=True, with_std=True)

```python
import pickle
with open("Y_theta_value_scalar.pkl","wb") as f:
    pickle.dump(obj,f)
```

```python
data_df_model_2 = pd.concat([data_df_model.drop(columns=["value"],inplace=False),
                             pd.DataFrame(obj.transform(data_df_model.value.values.reshape(-1,1)),columns=["scaled_value"])],axis=1)
```

```python
data_df_model_2.head()
```

      x_f
      y_f
      z_f
      b
      d
      h
      r
      theta0
      theta1
      theta2
      theta3
      scaled_value

      0
      0.0
      0.0
      0.0
      1.0
      1.0
      1.0
      0.008
      1
      0
      0
      0
      -0.699853

      1
      0.0
      0.0
      0.0
      1.0
      1.0
      1.5
      0.008
      1
      0
      0
      0
      -0.390172

      2
      0.0
      0.0
      1.0
      1.0
      1.0
      1.5
      0.008
      1
      0
      0
      0
      -1.009534

      3
      0.0
      0.0
      0.0
      1.0
      1.0
      2.0
      0.008
      1
      0
      0
      0
      -0.080491

      4
      0.0
      0.0
      1.0
      1.0
      1.0
      2.0
      0.008
      1
      0
      0
      0
      -0.699853

```python
data_df_model_2.to_csv("suspended_robot_data_all_scaled_r_0.008_B_D_H_1_6_20190826.csv",header=True,index=False)
```

```python
X_df = data_df_model_2.drop(columns=["r","scaled_value"],inplace=False)
Y_df = data_df_model_2["scaled_value"]
```

```python
X_df.to_csv("suspended_robot_X_data_df.csv",header=True,index=False)
Y_df.to_csv("suspended_robot_Y_data_df.csv",header=True,index=False)
```

```python
X = data_df_model_2.drop(columns=["r","scaled_value"],inplace=False).values
Y = data_df_model_2["scaled_value"].values.reshape(-1,1)
```

```python
X.shape,Y.shape
```

    ((442368, 10), (442368, 1))

```python
X.dump("suspended_robot_X_data.npy")
Y.dump("suspended_robot_Y_data.npy")
```


# 2D Shallow Water Equations

This notebook is a devito reimplementation from this [notebook](https://github.com/daniel-koehn/Differential-equations-earth-system/blob/master/10_Shallow_Water_Equation_2D/01_2D_Shallow_Water_Equations.ipynb). We will use an approximation to the Navier-Stokes equations - the 2D Shallow Water equations - in order to model the propagation of Tsunamis.

I've only made minor modifications to the text, in order to make them more compatible to the new coding format.

This tutorial is contributed by SENAI CIMATEC (2021)

## Governing Equations

Starting from the continuity and momentum conservation equations, we want to solve the following problem:

For a given bathymetry model, which can include a complex seafloor topography, we want to model the amplitudes, speed and interaction of waves at the seasurface. At a given point $(x,\; y)$, the thickness of the water column between the seafloor and undisturbed water surface is defined by the variable $h$, while the wave amplitude is $\eta$ and therefore the whole thickness of the water column $D = h + \eta$.

Using appropriate boundary conditions at the water surface/seafloor, assuming that the horizontal wavelength of the modelled waves are much larger than the water depth and integrating the conservation of mass and momentum equations over the water column, we can derive the following equations to decribe wave propagation

\begin{equation}
\begin{split}
\frac{\partial \eta}{\partial t} &+ \frac{\partial M}{\partial x} + \frac{\partial N}{\partial y} = 0\\
\frac{\partial M}{\partial t} &+ \frac{\partial}{\partial x} \biggl(\frac{M^2}{D}\biggr) + \frac{\partial}{\partial y} \biggl(\frac{MN}{D}\biggr) + g D \frac{\partial \eta}{\partial x} + \frac{g \alpha^2}{D^{7/3}} M \sqrt{M^2+N^2} = 0\\
\frac{\partial N}{\partial t} &+ \frac{\partial}{\partial x} \biggl(\frac{MN}{D}\biggr) + \frac{\partial}{\partial y} \biggl(\frac{N^2}{D}\biggr) + g D \frac{\partial \eta}{\partial y} + \frac{g \alpha^2}{D^{7/3}} N \sqrt{M^2+N^2} = 0
\end{split}
\tag{1}
\end{equation}

known as **2D Shallow Water Equations (SWE)**. The derivation of these equations is beyond the scope of this notebook. Therefore, I refer to the [Tsunami Modelling Handbook](http://www.tsunami.civil.tohoku.ac.jp/hokusai3/J/projects/manual-ver-3.1.pdf) and the lecture [Shallow Water Derivation and Applications by Christian Kühbacher](http://www.mathematik.tu-dortmund.de/lsiii/cms/papers/Kuehbacher2009.pdf) for further details.

In eqs. (1) the discharge fluxes $M,\; N$ in x- and y-direction, respectively are given by

\begin{equation}
\begin{split}
M & = \int_{-h}^\eta u dz = u(h+\eta) = uD\\
N & = \int_{-h}^\eta v dz = v(h+\eta) = vD\\
\end{split}
\tag{2}
\end{equation}

with the horizontal velocity components $u,\;v$ in x- and y-direction, while $g$ denotes the gravity acceleration. The terms $\frac{g \alpha^2}{D^{7/3}} M \sqrt{M^2+N^2}$ and $\frac{g \alpha^2}{D^{7/3}} N \sqrt{M^2+N^2}$ describe the influence of seafloor friction on the wave amplitude. $\alpha$ denotes the Manning's roughness which can be as small as 0.01 for neat cement or smooth metal up to 0.06 for very poor natural channels (see [Tsunami modelling handbook](http://www.tsunami.civil.tohoku.ac.jp/hokusai3/J/projects/manual-ver-3.1.pdf)).

The Shallow Water Equations can be applied to

- Tsunami prediction
- Atmospheric flow
- Storm surges
- Flows around structures
- Planetary flows

and easily extended to incorporate the effects of Coriolis forces, tides or wind stress.

The following function incorporates all the operations needed to solve the system of equations shown above.

```python
from devito import Eq, TimeFunction, sqrt, Function, Operator, Grid, solve, ConditionalDimension
from matplotlib import pyplot as plt
import numpy as np

def Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps=100):
    """
    Computes and returns the discharge fluxes M, N and wave height eta from
    the 2D Shallow water equation using the FTCS finite difference method.

    Parameters
    ----------
    eta0 : numpy.ndarray
        The initial wave height field as a 2D array of floats.
    M0 : numpy.ndarray
        The initial discharge flux field in x-direction as a 2D array of floats.
    N0 : numpy.ndarray
        The initial discharge flux field in y-direction as a 2D array of floats.
    h : numpy.ndarray
        Bathymetry model as a 2D array of floats.
    g : float
        gravity acceleration.
    alpha : float
        Manning's roughness coefficient.
    nt : integer
        Number fo timesteps.
    dx : float
        Spatial gridpoint distance in x-direction.
    dy : float
        Spatial gridpoint distance in y-direction.
    dt : float
        Time step.
    """

    eta   = TimeFunction(name='eta', grid=grid, space_order=2)
    M     = TimeFunction(name='M', grid=grid, space_order=2)
    N     = TimeFunction(name='N', grid=grid, space_order=2)
    h     = Function(name='h', grid=grid)
    D     = Function(name='D', grid=grid)

    factor = round(nt / nsnaps)
    print(factor, nt, grid.time_dim)
    time_subsampled = ConditionalDimension(
        't_sub', parent=grid.time_dim, factor=factor)
    etasave = TimeFunction(name='etasave', grid=grid, space_order=2,
                         save=nsnaps, time_dim=time_subsampled)

    eta.data[0] = eta0.copy()
    M.data[0]   = M0.copy()
    N.data[0]   = N0.copy()
    D.data[:]   = D0.copy()
    h.data[:]   = h0.copy()

    frictionTerm = g * alpha**2 * sqrt(M**2 + N**2 ) / D**(7./3.)

    pde_eta = Eq(eta.dt + M.dxc + N.dyc)
    pde_M   = Eq(M.dt + (M**2/D).dxc + (M*N/D).dyc + g*D*eta.forward.dxc + frictionTerm*M)
    pde_N   = Eq(N.dt + (M*N/D).dxc + (N**2/D).dyc + g*D*eta.forward.dyc + frictionTerm*N)

    # Defining boundary conditions
    x, y = grid.dimensions
    t = grid.stepping_dim
    bc_left   = Eq(eta[t+1, 0, y], eta[t+1, 1, y])
    bc_right  = Eq(eta[t+1, nx-1, y], eta[t+1, nx-2, y])
    bc_top    = Eq(eta[t+1, x, 0], eta[t+1, x, 1])
    bc_bottom = Eq(eta[t+1, x, ny-1], eta[t+1, x, ny-2])

    stencil_eta = solve(pde_eta, eta.forward)
    stencil_M   = solve(pde_M, M.forward)
    stencil_N   = solve(pde_N, N.forward)

    update_eta  = Eq(eta.forward, stencil_eta, subdomain=grid.interior)
    update_M    = Eq(M.forward, stencil_M, subdomain=grid.interior)
    update_N    = Eq(N.forward, stencil_N, subdomain=grid.interior)
    eq_D        = Eq(D, eta.forward + h)

    optime = Operator([update_eta, bc_left, bc_right, bc_top, bc_bottom,
                       update_M, update_N, eq_D] + [Eq(etasave, eta)])

    optime(time=nt-2, dt=dt)
    return etasave, M, N
```

The following function transforms the saved wavefield snapshots and transform them into a video compatible with jupyter notebook.

```python
#NBVAL_IGNORE_OUTPUT
#NBVAL_SKIP
from IPython.display import HTML, display
import matplotlib.pyplot as plt
import matplotlib.animation as animation

def snaps2video (eta):
    fig, ax = plt.subplots()
    matrice = ax.imshow(eta.data[0, :, :].T, vmin=-1, vmax=1, cmap="seismic")
    plt.colorbar(matrice)

    plt.xlabel('x')
    plt.ylabel('z')
    plt.title('Modelling one shot over a 2-layer velocity model with Devito.')

    def update(i):
        matrice.set_array(eta.data[i, :, :].T)
        return matrice,

    # Animation
    ani = animation.FuncAnimation(fig, update, frames=nsnaps, interval=50, blit=True)

    #plt.show()
    plt.close(ani._fig)
    display(HTML(ani.to_html5_video()))
```

## Example I: Tsunami in an ocean with constant depth

After writing the `Shallow_water_2D` code and all the required functions attached to it, we can define and run our first 2D Tsunami modelling run.

Let's assume that the ocean model is $ L_x = 100\; m$ in x-direction and $L_y = 100\; m$ in y-direction. The model is discretized with $nx=401$ gridpoints in x-direction and $ny=401$ gridpoints in y-direction, respectively.

In this first modelling run, we assume a constant bathymetry $h=50\;m$. The initial wave height field $\eta_0$ is defined as a Gaussian at the center of the model, with a half-width of 10 m and an amplitude of 0.5 m. Regarding the initial discharge fluxes, we assume that

\begin{equation}
\begin{split}
M_0(x,y) &= 100 \eta_0(x,y)\\
N_0(x,y) &= 0\\
\end{split}\notag
\end{equation}

Furthermore, Dirichlet boundary conditions for the discharge fluxes $M,\;N$ are assumed at all boundaries:

\begin{equation}
\begin{split}
M(0,y) &= M(L_x,y) = M(x,0) = M(x,L_y) = 0\\
N(0,y) &= N(L_x,y) = N(x,0) = N(x,L_y) = 0\\
\end{split} \notag
\end{equation}

For the wave height field $\eta$, it is essential that Neumann boundary conditions are set at all boundaries:

\begin{equation}
\begin{split}
&\frac{\partial \eta}{\partial x}(0,y) = 0\\
&\frac{\partial \eta}{\partial x}(L_x,y) = 0\\
&\frac{\partial \eta}{\partial y}(x,0) = 0\\
&\frac{\partial \eta}{\partial y}(x,L_y) = 0\\
\end{split}\notag
\end{equation}

in order to avoid the occurence of high frequency artifacts, when waves are interacting with the boundaries. Notice, that the assumed boundary conditions lead to significant boundary reflections which might be not realistic for a given problem.

```python
Lx    = 100.0   # width of the mantle in the x direction []
Ly    = 100.0   # thickness of the mantle in the y direction []
nx    = 401     # number of points in the x direction
ny    = 401     # number of points in the y direction
dx    = Lx / (nx - 1)  # grid spacing in the x direction []
dy    = Ly / (ny - 1)  # grid spacing in the y direction []
g     = 9.81  # gravity acceleration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition
# Maximum wave propagation time [s]
Tmax  = 6.
dt    = 1/4500.
nt    = (int)(Tmax/dt)
print(dt, nt)

x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define constant ocean depth profile h = 50 m
h0 = 50. * np.ones_like(X)

# Define initial eta Gaussian distribution [m]
eta0 = 0.5 * np.exp(-((X-50)**2/10)-((Y-50)**2/10))

# Define initial M and N
M0 = 100. * eta0
N0 = 0. * M0
D0 = eta0 + 50.

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

    0.00022222222222222223 27000

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    270 27000 time

    Operator `Kernel` ran in 20.85 s

## Example II: Two Tsunamis in an ocean with constant depth

In example II, we will model the propagation and interaction of two Tsunamis, where the initial conditions for the wave height field $\eta_0$ consists of Gaussian functions with opposite sign located at $(x_1,\;y_1) = (35\; m,\; 35\; m)$ and $(x_2,\;y_2) = (65\; m,\; 65\; m)$. All other modelling parameters remain the same ...

```python
# Definition of modelling parameters
# ----------------------------------
Lx = 100.0   # width of the mantle in the x direction []
Ly = 100.0   # thickness of the mantle in the y direction []
nx = 401     # number of points in the x direction
ny = 401     # number of points in the y direction
dx = Lx / (nx - 1)  # grid spacing in the x direction []
dy = Ly / (ny - 1)  # grid spacing in the y direction []

# Define the locations along a gridline.
x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define constant ocean depth profile h = 50 m
h0 = 50 * np.ones_like(X)

# Define initial Gaussian eta distribution [m]
eta0 = 0.5 * np.exp(-((X-35)**2/10)-((Y-35)**2/10)) # first Tsunami source
eta0 -= 0.5 * np.exp(-((X-65)**2/10)-((Y-65)**2/10)) # add second Tsunami source

# Define initial M and N
M0 = 100. * eta0
N0 = 0. * M0
D0 = eta0 + h0

g = 9.81  # gravity accelaration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition

# Maximum wave propagation time [s]
Tmax = 3.
dt = 1/8000.
nt = (int)(Tmax/dt)

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    240 24000 time

    Operator `Kernel` ran in 22.01 s

## Example III: Tsunami in an ocean with 1D Tanh depth variation

So far, so good, we can achieve stable and accurate modelling results. However, a constant bathymetry model is a little bit boring. In this example, we assume that the bathymetry decreases with a Tanh function from the left to the right boundary in x-direction. Let 's place a Tsunami source at $(x_1,\;y_1) = (30\; m,\; 50\; m)$ and see what 's happening ...

```python
# Definition of modelling parameters
# ----------------------------------
Lx = 100.0   # width of the mantle in the x direction []
Ly = 100.0   # thickness of the mantle in the y direction []
nx = 401     # number of points in the x direction
ny = 401     # number of points in the y direction
dx = Lx / (nx - 1)  # grid spacing in the x direction []
dy = Ly / (ny - 1)  # grid spacing in the y direction []

# Define the locations along a gridline.
x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define depth profile h [m]
h0 = 50 - 45 * np.tanh((X-70.)/8.)

# Define initial eta [m]
eta0 = 0.5 * np.exp(-((X-30)**2/10)-((Y-50)**2/20))

# Define initial M and N
M0 = 100. * eta0
N0 = 0. * M0
D0 = eta0 + h0

# define some constants
g = 9.81  # gravity accelaration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition

# Maximum wave propagation time [s]
Tmax = 3.
dt = 1/8000.
nt = (int)(Tmax/dt)

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    Allocating memory for eta(2, 405, 405)
    Allocating memory for M(2, 405, 405)
    Allocating memory for N(2, 405, 405)
    Allocating memory for D(403, 403)
    Allocating memory for h(403, 403)

    240 24000 time

    Operator `Kernel` generated in 1.27 s
      * lowering.Clusters: 0.67 s (53.0 %)
         * specializing.Clusters: 0.56 s (44.3 %)
            * factorize: 0.35 s (27.7 %)
      * lowering.IET: 0.35 s (27.7 %)
    Flops reduction after symbolic optimization: [112 --> 75]
    Allocating memory for etasave(100, 405, 405)
    Operator `Kernel` fetched `/run/user/1001/devito-jitcache-uid1001/d1f5722d12e874635a89549c8065da12f3e0199a.c` in 0.11 s from jit-cache
    Operator `Kernel` ran in 198.80 s
    Global performance: [OI=0.01, 1.45 GFlops/s, 0.02 GPts/s]
    Local performance:
      * section0<23999,399,399> ran in 29.54 s [OI=0.82, 1.69 GFlops/s, 0.13 GPts/s]
      * section4<<23999,399,399>,<23999,401,401>> ran in 169.18 s [OI=0.01, 1.41 GFlops/s, 0.03 GPts/s]
    Performance[mode=advanced] arguments: {}

## Example IV: Tsunami in an ocean with a seamount

What happens if we have a constant bathymetry, except for a small seamount reaching 5 m below the water surface? Lets define the model parameters ...

```python
# Definition of modelling parameters
# ----------------------------------
Lx = 100.0   # width of the mantle in the x direction []
Ly = 100.0   # thickness of the mantle in the y direction []
nx = 401     # number of points in the x direction
ny = 401     # number of points in the y direction
dx = Lx / (nx - 1)  # grid spacing in the x direction []
dy = Ly / (ny - 1)  # grid spacing in the y direction []

# Define the locations along a gridline.
x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define constant ocean depth profile h = 50 m
h0 = 50. * np.ones_like(X)

# Adding seamount to seafloor topography
h0 -= 45. * np.exp(-((X-50)**2/20)-((Y-50)**2/20))

# Define initial eta [m]
eta0 = 0.5 * np.exp(-((X-30)**2/5)-((Y-50)**2/5))

# Define initial M and N
M0 = 100. * eta0
N0 = 0. * M0
D0 = eta0 + h0

# define some constants
g = 9.81  # gravity accelaration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition

# Maximum wave propagation time [s]
Tmax = 3.
dt = 1/8000.
nt = (int)(Tmax/dt)

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    Allocating memory for eta(2, 405, 405)
    Allocating memory for M(2, 405, 405)
    Allocating memory for N(2, 405, 405)
    Allocating memory for D(403, 403)
    Allocating memory for h(403, 403)

    240 24000 time

    Operator `Kernel` generated in 1.26 s
      * lowering.Clusters: 0.70 s (55.8 %)
         * specializing.Clusters: 0.58 s (46.3 %)
            * factorize: 0.37 s (29.5 %)
      * lowering.IET: 0.31 s (24.8 %)
    Flops reduction after symbolic optimization: [112 --> 75]
    Allocating memory for etasave(100, 405, 405)
    Operator `Kernel` fetched `/run/user/1001/devito-jitcache-uid1001/d1f5722d12e874635a89549c8065da12f3e0199a.c` in 0.11 s from jit-cache
    Operator `Kernel` ran in 191.90 s
    Global performance: [OI=0.01, 1.50 GFlops/s, 0.03 GPts/s]
    Local performance:
      * section0<23999,399,399> ran in 30.89 s [OI=0.82, 1.61 GFlops/s, 0.13 GPts/s]
      * section4<<23999,399,399>,<23999,401,401>> ran in 160.92 s [OI=0.01, 1.48 GFlops/s, 0.03 GPts/s]
    Performance[mode=advanced] arguments: {}

## Example V: Tsunami in an ocean with random seafloor topography variations

Another modelling idea: what is the influence of the roughness of the seafloor topography. First, we add some random perturbations on the constant bathymetry model $h_0$ by using the `random.rand` function from the `NumPy` library. To smooth the random perturbations, we apply the `gaussian_filter` from the `SciPy`library:

```python
from scipy.ndimage import gaussian_filter
# Definition of modelling parameters
# ----------------------------------
Lx = 100.0   # width of the mantle in the x direction []
Ly = 100.0   # thickness of the mantle in the y direction []
nx = 401     # number of points in the x direction
ny = 401     # number of points in the y direction
dx = Lx / (nx - 1)  # grid spacing in the x direction []
dy = Ly / (ny - 1)  # grid spacing in the y direction []

# Define the locations along a gridline.
x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define constant ocean depth profile h = 30 m
h0 = 30. * np.ones_like(X)

# Add random seafloor perturbation of +- 5m
pert = 5.   # perturbation amplitude
r = 2.0 * (np.random.rand(ny, nx) - 0.5) * pert # create random number perturbations
r = gaussian_filter(r, sigma=8) # smooth random number perturbation
h0 = h0 * (1 + r) # add perturbations to constant seafloor

# Define initial eta [m]
eta0 = 0.5 * np.exp(-((X-30)**2/5)-((Y-50)**2/5))

# Define initial M and N
M0 = 100. * eta0
N0 = 0. * M0
D0 = eta0 + h0

# define some constants
g = 9.81  # gravity accelaration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition

# Maximum wave propagation time [s]
Tmax = 6.
dt = 1/8000.
nt = (int)(Tmax/dt)

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    Allocating memory for eta(2, 405, 405)
    Allocating memory for M(2, 405, 405)
    Allocating memory for N(2, 405, 405)
    Allocating memory for D(403, 403)
    Allocating memory for h(403, 403)

    480 48000 time

    Operator `Kernel` generated in 1.26 s
      * lowering.Clusters: 0.70 s (55.9 %)
         * specializing.Clusters: 0.58 s (46.3 %)
            * factorize: 0.37 s (29.6 %)
      * lowering.IET: 0.30 s (24.0 %)
    Flops reduction after symbolic optimization: [112 --> 75]
    Allocating memory for etasave(100, 405, 405)

    gcc -O3 -g -march=native -fPIC -Wall -std=c99 -shared /run/user/1001/devito-jitcache-uid1001/284272aa4d4fcf25cd7da5f22477d7955452899e.c -lm -o /run/user/1001/devito-jitcache-uid1001/284272aa4d4fcf25cd7da5f22477d7955452899e.so

    Operator `Kernel` jit-compiled `/run/user/1001/devito-jitcache-uid1001/284272aa4d4fcf25cd7da5f22477d7955452899e.c` in 0.45 s with `CustomCompiler`
    Operator `Kernel` ran in 287.76 s
    Global performance: [OI=0.01, 2.00 GFlops/s, 0.03 GPts/s]
    Local performance:
      * section0<47999,399,399> ran in 61.21 s [OI=0.82, 1.63 GFlops/s, 0.13 GPts/s]
      * section4<<47999,399,399>,<47999,401,401>> ran in 226.41 s [OI=0.01, 2.10 GFlops/s, 0.04 GPts/s]
    Performance[mode=advanced] arguments: {}

## Example VI: 2D circular dam break problem

As a final modelling example, let's take a look at an (academic) engineering problem: a tsunami induced by the collapse of a circular dam in a lake with a constant bathymetry of 30 m. We only need to set the wave height in a circle with radius $r_0 = 5\; m$ to $\eta_0 = 0.5 \; m$ and to zero everywhere else. To avoid the occurence of high frequency artifacts in the wavefield, known as numerical grid dispersion, we apply a Gaussian filter to the initial wave height. To achieve a symmetric dam collapse, the initial discharge fluxes $M_0,N_0$ are set to equal values.

```python
# Definition of modelling parameters
# ----------------------------------
Lx = 100.0   # width of the mantle in the x direction []
Ly = 100.0   # thickness of the mantle in the y direction []
nx = 401     # number of points in the x direction
ny = 401     # number of points in the y direction
dx = Lx / (nx - 1)  # grid spacing in the x direction []
dy = Ly / (ny - 1)  # grid spacing in the y direction []

# Define the locations along a gridline.
x = np.linspace(0.0, Lx, num=nx)
y = np.linspace(0.0, Ly, num=ny)

# Define initial eta, M, N
X, Y = np.meshgrid(x,y) # coordinates X,Y required to define eta, h, M, N

# Define constant ocean depth profile h = 30 m
h0 = 30. * np.ones_like(X)

# Define initial eta [m]
eta0 = np.zeros_like(X)

# Define mask for circular dam location with radius r0
r0 = 5.
mask = np.where(np.sqrt((X-50)**2 + (Y-50)**2) <= r0)

# Set wave height in dam to 0.5 m
eta0[mask] = 0.5

# Smooth dam boundaries with gaussian filter
eta0 = gaussian_filter(eta0, sigma=8) # smooth random number perturbation

# Define initial M and N
M0 = 1. * eta0
N0 = 1. * M0
D0 = eta0 + h0

# define some constants
g = 9.81  # gravity accelaration [m/s^2]
alpha = 0.025 # friction coefficient for natural channels in good condition

# Maximum wave propagation time [s]
Tmax = 3.
dt = 1/8000.
nt = (int)(Tmax/dt)

grid  = Grid(shape=(ny, nx), extent=(Ly, Lx))
```

```python
nsnaps = 100
eta, M, N = Shallow_water_2D(eta0, M0, N0, h0, grid, g, alpha, nt, dx, dy, dt, nsnaps)
snaps2video(eta)
```

    Allocating memory for eta(2, 405, 405)
    Allocating memory for M(2, 405, 405)
    Allocating memory for N(2, 405, 405)
    Allocating memory for D(403, 403)
    Allocating memory for h(403, 403)

    240 24000 time

    Operator `Kernel` generated in 1.33 s
      * lowering.Clusters: 0.77 s (58.1 %)
         * specializing.Clusters: 0.57 s (43.0 %)
            * factorize: 0.36 s (27.2 %)
      * lowering.IET: 0.31 s (23.4 %)
    Flops reduction after symbolic optimization: [112 --> 75]
    Allocating memory for etasave(100, 405, 405)
    Operator `Kernel` fetched `/run/user/1001/devito-jitcache-uid1001/d1f5722d12e874635a89549c8065da12f3e0199a.c` in 0.12 s from jit-cache
    Operator `Kernel` ran in 210.64 s
    Global performance: [OI=0.01, 1.37 GFlops/s, 0.02 GPts/s]
    Local performance:
      * section0<23999,399,399> ran in 31.04 s [OI=0.82, 1.61 GFlops/s, 0.13 GPts/s]
      * section4<<23999,399,399>,<23999,401,401>> ran in 179.52 s [OI=0.01, 1.32 GFlops/s, 0.03 GPts/s]
    Performance[mode=advanced] arguments: {}

## What we learned:

* How to solve the 2D Shallow Water Equation using the FTCS finite difference scheme.

* Propagation of (multiple) Tsunamis in an ocean with constant bathymetry.

* Tsunamis reaching shallow waters near a coast will slow down, their wavelength decrease, while their wave height increases.

* The influence of a seamount and roughness of the seafloor topoghraphy on the Tsunami wavefield.

* Tsunami modelling for engineering problems, like the collapse of dams.

```python

```