# Python Programming

**What is Python?** source: [python.org](https://www.python.org/doc/essays/blurb/)

Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python's simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse. The Python interpreter and the extensive standard library are available in source or binary form without charge for all major platforms, and can be freely distributed.

Often, programmers fall in love with Python because of the increased productivity it provides. Since there is no compilation step, the edit-test-debug cycle is incredibly fast. Debugging Python programs is easy: a bug or bad input will never cause a segmentation fault. Instead, when the interpreter discovers an error, it raises an exception. When the program doesn't catch the exception, the interpreter prints a stack trace. A source level debugger allows inspection of local and global variables, evaluation of arbitrary expressions, setting breakpoints, stepping through the code a line at a time, and so on. The debugger is written in Python itself, testifying to Python's introspective power. On the other hand, often the quickest way to debug a program is to add a few print statements to the source: the fast edit-test-debug cycle makes this simple approach very effective.

**What is Python capable of?***

Long answer short: everything. Most often Python is used for web developments and prototyping, using frameworks such as [Flask](http://flask.palletsprojects.com/en/1.1.x/) and [Django](https://www.djangoproject.com/). Because of its simplicity, it is also popularly used in data science, both in industry and in academia.

In this lab, we mainly cover the following python programming topics:

data types

built-in data structure

I/O in Python

loops in Python

function and class

## Data Types

```python
# integer
a = 1
print('I am an integer:', a)
```

```python
# float -> decimal numbers
a = 1.0
print('I am a float:', a)
```

```python
# strings -> an array of characteristics
a = "GOTO Chicago Fall 2019"
print('I am a string:', a)
```

```python
# boolean -> binary
a = True
print('I am a binary:', a)
```

**Data Type Conversion**

```python
# integer to float
a = 1
print('I was an integer:', a, ', but now I am a float:', float(a))
```

```python
# float to integer
a = 1.0
print('I was a float:', a, ', but now I am an integer:', int(a))
```

```python
# string to integer/float
a = '1.0'
print('I was a string: "'+ a +'", but now I am a number:', eval(a))
```

```python
# integer/float to string
a = 1.0
print('I was a number:', a, ', but now I am a string:', str(a))
```

Additional material for reading if interested: [source](https://www.geeksforgeeks.org/type-conversion-python/)

## Data Structures

```python
# list -> mutable
a = [1, 2, 3, 'a', 'b', 'c']
print('I am a list:', a)
```

```python
# tuple -> immutable
a = (1, 2, 3, 'a', 'b', 'c')
print('I am a tuple:', a)
```

```python
# dictionary
a = {'1': 'a', 2: 'b', 'c': 3}
print('I am a dictionary:', a)
```

**List**

List is mutable, meaning after a list is defined, you can alter the values stored in it. **Tuple** is not mutable, meaning once it's defined, you can't change the values. Let's focus on the list for now.

```python
a = [1, 2, 3]
```

```python
# select an item
a[0]
```

```python
# select a range of items
a[0:2]
```

```python
# add an item
# method 1: append to the end of the list
a.append(4)
a
```

```python
# method 2: insert into a specific position
a.insert(2, 10)
a
```

```python
# remove an item
# method 1: pop the last item in the list
a.pop()
```

```python
a
```

```python
# method 2: remove a specific position
a.pop(2)
```

```python
a
```

```python
# method 3: remove a specific value
a.remove(3)
```

```python
a
```

**Dictionary**

```python
# initiate an empty dictionary
a = dict()
a
```

```python
# add an item
# dictionary[key_name]= value_name
a['my_key'] = 'my_value'
a['my_key_2'] = 'my_value_2'
a
```

```python
# select a value based on key
a['my_key']
```

```python
# remove an item
# method 1: pop the item added last
a.popitem()
```

```python
a
```

```python
# method 2: pop the key
a.pop('my_key')
```

```python
a
```

**Substring**

```python
a = 'I am a string.'
```

```python
# select a sub string out of the main string
a[:5]
```

## Input/Output (I/O) in Python

Imagine you're reading a book, what are the steps for you to finish the process of reading a book? It should all include three steps,

Open the book

Read the book

Close the book

Similar in Python, to read a text file from your local, you need to open a the file, read the content, and close the file. Here is how to do it.

**Step-by-Step Version**

```python
# open the file
path = './data/dummy.txt'
file = open(path, 'r')
```

`r` denotes read only mode.

```python
# read the content
content = file.read()
```

```python
print(content)
```

```python
# close the file
file.close()
```

Now you can work with the content in `content` variable

```python
content
```

```python
# split() is a built-in method with Python's string datatype that splits a string using the sub-string
# specified; if nothing is specified, it splits by a single space by default
content.split('\n')
```

**Question:** What is the output data structure of the split method?

**"All-in-One" or The Clean Version**

```python
# "with" method takes care of closing the file so you don't have to worry about that
with open('./data/dummy.txt', 'r') as file:
    content2 = file.read()
```

```python
content2
```

Though you may have heard about tools like `Pandas` and `Numpy` that take care of I/O for you, knowing how to correctly read files using native Python is also important because data come in different formats.

## Loops in Python

Given that you are already a developer, this section is just to show you how to write for and while loops in Python.

**For Loop**

```python
my_list = [1, 2, 3, '4', '5', '7', True]
```

```python
for item in my_list:                        # loop through the list
    if isinstance(item, str):               # if the item is a string
        print(item, 'is a string.')
    else:
        print(item, 'is not a string.')
```

**While Loop**

```python
i = 0
while i < len(my_list):
    item = my_list[i]
    if isinstance(item, str):               # if the item is a string
        print(item, 'is a string.')
    else:
        print(item, 'is not a string.')
    i += 1
```

**Break and Continue**

`break`: it breaks out of a for loop and continue the code

`continue`: it skips the rest of the code in the current loop and moves onto the next iteration

```python
for item in my_list:
    if not isinstance(item, str):
        continue
        print('this is not a string.. this line is not even executed at all')
    else:
        print(item)
```

**Q&A:** What happens if we swap out `continue` with `break`?

## Function and Class

A `function` is a set of python instructions/statements that take in inputs, do things to them, and output the results out. Note: This is the standard function definition, and not all functions need to take in inputs and/or returns output.

A `class` is a code template that creates objects and it contains one or more functions. If you have experience in Java, it is just like the Java class.

```python
# function example
def my_function(a_input):                        # define a function
    output = str(a_input) + ' is the input'     # do something to the input
    return output                               # return output
```

```python
my_function(5)
```

```python
# class example
class my_class(object):
    """i am a doc string :) """

    def __init__(self, input_str=None):
        """I am the first function to be called when a new object is initiated"""
        self.input_str = str(input_str)
        self.output_str = None

    def do_something(self):
        """I am a separate method that does something in this class
        Note that I don't have to return anything here
        """
        self.output_str = self.input_str + ' is the input'

    def give_me_output(self):
        return self.output_str
```

```python
new_instance = my_class(5)     # at this step, __init__ is called
```

```python
new_instance.do_something()    # here, do_something() is called
```

```python
new_instance.give_me_output()  # here, give_me_output() is called
```

Finally, if you care about code styling a lot, which you should, feel free to check out PEP 8 Python coding styling standards [here](https://www.python.org/dev/peps/pep-0008/).

**Exercise**

If time permits, try build a calculator that does the following in Python.

Addition

Subtraction

Multiplication

Division

```python
class MyCalculator(object):
    """insert your doc string here"""

    def __init__(self):
        pass

    def add(self):
        pass

    def subtract(self):
        pass

    def multiply(self):
        pass

    def divide(self):
        pass
```

# Statistics

Whew, I'm sure the Python section bored you out a little. Let's learn something more fun.

Let's first use Python's `random` module to generate 1,000,000 random numbers.

```python
import random
random.seed(1234)
# by setting a seed, you can ensure every time you run this block of code, the output is the same

dataset = [random.random() for _ in range(1_000_000)]
# underscore means that variable is not important to store

import statistics # let's import this for later
```

```python
len(dataset)
```

```python
dataset[:5]
```

**Arithmetic Mean a.k.a. Average**

```python
def average(arr):
    return sum(arr) / len(arr)
```

```python
mean = average(dataset)
```

```python
mean
```

```python
# validate
statistics.mean(dataset)
```

**Median**

```python
def median(arr):
    arr = sorted(arr)
    median_index = int(len(arr) / 2)
    if len(arr) % 2 == 0:      # if the list has even number of elements
        median = average([arr[median_index], arr[median_index-1]])
    else:
        median = arr[median_index]
    return median
```

```python
median(dataset)
```

```python
# validate
statistics.median(dataset)
```

**Variance**

```python
def variance(arr):
    mean = average(arr)      # note that I am using the average() function here
    variance = sum(
                map(lambda i: (i-mean)**2, arr)
                  ) / (len(arr) - 1)
    return variance
```

`map` operation "maps" a function to every element of a collection passed in the second section.

`lambda` is a Python way to explicitly define small-scale function anonymously; see more [here](https://realpython.com/python-lambda/)

```python
variance(dataset)
```

```python
# validate
statistics.variance(dataset)
```

**Standard Deviation**

```python
def stdv(arr):
    var = variance(arr)
    standard_deviation = var ** 0.5
    return standard_deviation
```

```python
stdv(dataset)
```

```python
# validate
statistics.stdev(dataset)
```

Ok... I got the math now, but what do all of these mean???????

Nomral distribution is a bell-curve shaped distribution, where `mean` equals to `median`. In addition, it is estimated that

68% of the data fall between +/-1 standard deviation>

95% of the data fall between +/-2 standard deviation>

99% of the data fall between +/-3 standard deviation>

To calculate how far the value is away from the sample mean, `z-score` is calculated which we do not cover today. Read more about `z-score` [here](https://www.investopedia.com/terms/z/zscore.asp).

## Let's Make It A Bit More Complicated

We've been focusing on one variable. Let's now look into the descriptive statistics of two variables.

Assuming we have two variables, `X` and `Y`, we can calculate the same descriptive statistics above for both of them such that

         Average

            \begin{equation}
                \bar{X} = \frac{\displaystyle\sum_{i=1}^n x_i}{n}
            \end{equation}

            \begin{equation}
                \bar{Y} = \frac{\displaystyle\sum_{i=1}^n y_i}{n}
            \end{equation}

         Variance

            \begin{equation}
                Var(X) = \sigma_x^2 = \frac{\displaystyle\sum_{i=1}^{n}(x_i - \bar{X})^2} {n-1}
            \end{equation}

            \begin{equation}
                Var(Y) = \sigma_y^2 = \frac{\displaystyle\sum_{i=1}^{n}(y_i - \bar{Y})^2} {n-1}
            \end{equation}

         Standard Deviation

            \begin{equation}
                Stdev(X) = \sqrt{Var(X)}
            \end{equation}

            \begin{equation}
                Stdev(Y) = \sqrt{Var(Y)}
            \end{equation}

**Covariance**

Covariance is a measure of how much two random variables vary together.

\begin{equation}
    cov_{x,y}=\frac{\displaystyle\sum_{i=1}^{N}(x_{i}-\bar{x})(y_{i}-\bar{y})}{N-1}
\end{equation}

If the covariance is positive, it tells us that the two variables are positively related, and vice versa. However, covariance does not have a upper nor lower limit, thus we cannot compare the magnitude of inter-variable relationship.

**Correlation**

Correlation coefficients are used to measure how strong the relationship is between two variables. There are several types of correlation coefficients and Pearson's Correlation is commonly used in describing linear relations.

\begin{equation}
    cor_{x,y} = \frac{cov_{x, y}}{\sigma_x\sigma_y}
\end{equation}

```python
# Let's generate two random variables with the same size
import numpy as np
# numpy (Numerical Python) is a very popular data science tool (https://numpy.org/)

x = np.random.random(1_000_000)
y = np.random.random(1_000_000)
```

```python
len(x), len(y)
```

```python
x[:5]
```

```python
# Let's calculate covariance
np.cov(x, y)
```

The output above is known as `variance-covariance matrix`, where the diagonal represents the variance of each variable, and the other positions represents the covariance of different variable pairs, such that

        X
        Y

        X
        Var(X)
        Cov(X, Y)

        Y
        Cov(X, Y)
        Var(Y)

**Note:** Cov(X, X) is equal to Var(X)

```python
# Let's calculate correlation
np.corrcoef(x, y)
```

The output above is known as `correlation matrix`, where the diagonal represents the correlation of the variable with itself, and the other positions represents the correlation of different variable pairs, such that

        X
        Y

        X
        Corr(X, X)
        Corr(X, Y)

        Y
        Corr(X, Y)
        Corr(Y, Y)

**Note:** Corr(X, X) is 1 -> A variable is perfectly postiively correlated with itself.

# Recap

In this session, we've covered

Basic Python programming

Statistics for single variable

Descriptive statistics for two variables

You should be able to

Complete the Calculator exercise

Explain what descriptive statistics are appropriate for sing variable

Interpret the implications of descriptive statistics

Explain the difference between covariance and correlation

Interpret Pearson's Correlation Coefficient

```python

```


Dinámica

Capítulo 2: Cinemática y Cinética de partículas

Movimiento curvilíneo

2021/02

MEDELLÍN - COLOMBIA

 Text provided under a Creative Commons Attribution license, CC-BY. All code is made available under the FSF-approved MIT license.(c) Carlos Alberto Alvarez Henao

***

***Docente:*** Carlos Alberto Álvarez Henao, I.C. D.Sc.

***e-mail:*** carlosalvarezh@gmail.com

***skype:*** carlos.alberto.alvarez.henao

***Linkedin:*** https://www.linkedin.com/in/carlosalvarez5/

***github:*** https://github.com/carlosalvarezh/Dinamica

***Herramienta:*** [Jupyter](http://jupyter.org/)

***Kernel:*** Python 3.9

***

Tabla de Contenidos

1  Movimiento curvilíneo General

1.1  Definición

1.2  Posición

1.3  Desplazamiento

1.4  Velocidad

1.5  Aceleración

1.6  Comentarios al movimiento curvilíneo general

2  Movimiento curvilíneo: Componente rectangulares

2.1  Posición

2.2  Velocidad

2.3  Aceleración

2.4  Ejemplos movimiento curvilínea componentes rectangulares

2.4.1  Globo atmosférico

 Fuente: Universidade do Porto

## Movimiento curvilíneo General

### Definición

El [movimiento curvilíneo](https://en.wikipedia.org/wiki/Curvilinear_motion) es aquél que se describe cuando una partícula se desplaza a lo largo de una trayectoria curva ([parabólico](https://en.wikipedia.org/wiki/Projectile_motion), [oscilatorio](https://en.wikipedia.org/wiki/Oscillation) o [circular](https://en.wikipedia.org/wiki/Circular_motion)). Se emplearán los conceptos de [análisis vectorial](https://en.wikipedia.org/wiki/Vector_calculus) para describir la posición, velocidad y aceleración de la partícula en las tres dimensiones espaciales. En este capítulo se considerarán también tres tipos de [sistemas de coordenadas](https://en.wikipedia.org/wiki/Coordinate_system) usadas frecuentemente en el análisis de este tipo de movimiento.

### Posición

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Sea una partícula situada en un punto de una curva espacial definida por la función de trayectoria $s(t)$. El vector de posición $\vec{\boldsymbol{r}} = \vec{\boldsymbol{r}}(t)$ designará la posición de la partícula, medida con respecto a un punto fijo $O$. Tanto la magnitud como la dirección de este vector cambiarán a medida que la partícula se mueve a lo largo de la curva.

### Desplazamiento

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Si durante un breve intervalo $\Delta t$ la partícula se mueve una distancia $\Delta s$ a lo largo de la curva a una nueva posición, definida por $\vec{\boldsymbol{r'}} = \vec{\boldsymbol{r}} +\Delta \vec{\boldsymbol{r}}$, el desplazamiento $\Delta \vec{\boldsymbol{r}}$ representará el cambio de posición de la partícula y se determina mediante la resta vectorial $\Delta \vec{\boldsymbol{r}}= \vec{\boldsymbol{r'}} - \vec{\boldsymbol{r}}$.

### Velocidad

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Durante el tiempo $\Delta t$, la velocidad promedio de la partícula es

\begin{equation*}
\vec{\boldsymbol{v}}_{prom}=\frac{\Delta \vec{\boldsymbol{r}}}{\Delta t}
\label{eq:Ec2_1} \tag{2.1}
\end{equation*}

La *velocidad instantánea* se determina cuando $\Delta t \rightarrow 0$, entonces, la dirección de $\Delta \vec{\boldsymbol{r}}$ tiende a la *tangente* a la curva y,

\begin{equation*}
\vec{\boldsymbol{v}}=\lim \limits_{\Delta t \to 0} \frac{\Delta \vec{\boldsymbol{r}}}{\Delta t}=\frac{d{\boldsymbol{r}}}{dt}
\label{eq:Ec2_2} \tag{2.2}
\end{equation*}

Como $\Delta \vec{\boldsymbol{r}}$ será tangente a la curva, la dirección de $\vec{\boldsymbol{v}}$ también es tangente a la curva. La magnitud de $\vec{\boldsymbol{v}}$, conocida como la rapidez, se obtiene al tener en cuenta que la longitud del segmento de línea recta $\Delta \vec{\boldsymbol{r}}$ tiende la longitud de arco $\Delta s$ a medida que $\Delta t \rightarrow 0$, tenemos

\begin{equation*}
\vec{\boldsymbol{v}}=\lim \limits_{\Delta t \to 0} \frac{\Delta \vec{\boldsymbol{r}}}{\Delta t}=\lim \limits_{\Delta t \to 0} \frac{\Delta s}{\Delta t}=\frac{ds}{dt}
\label{eq:Ec2_3} \tag{2.3}
\end{equation*}

### Aceleración

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Si la velocidad de la partícula es $\vec{\boldsymbol{v}}$ en el instante $t$ y $\vec{\boldsymbol{v'}}=\vec{\boldsymbol{v}}+\Delta \vec{\boldsymbol{v}}$ en el instante $t + \Delta t$, entonces la aceleración promedio de la partícula durante el intervalo $\Delta t$ es

\begin{equation*}
\vec{\boldsymbol{a}}_{prom}=\frac{\Delta \vec{\boldsymbol{v}}}{\Delta t}
\label{eq:Ec2_4} \tag{2.4}
\end{equation*}

Para estudiar la tasa de cambio en el tiempo, los dos vectores de velocidad en la figura anterior se trazan en la siguiente figura

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

de modo que sus colas queden en el punto fijo $O'$ y sus cabezas de punta de flecha toquen puntos situados en la curva. Esta curva se llama [hodógrafa](https://en.wikipedia.org/wiki/Hodograph) y describe el lugar geométrico de puntos para la cabeza de punta de flecha del vector de velocidad, así como la trayectoria $s$ describe el lugar geométrico de puntos para la cabeza de punta de flecha del vector de posición.

La *aceleración instantánea* se determina cuando $\Delta t \rightarrow 0$, entonces, en el límite $\Delta \vec{\boldsymbol{v}}$ la tangente tenderá a la *hodógrafa* y por lo tanto

\begin{equation*}
\vec{\boldsymbol{a}}=\lim \limits_{\Delta t \to 0} \frac{\Delta \vec{\boldsymbol{v}}}{\Delta t}=\frac{d{\boldsymbol{v}}}{dt}
\label{eq:Ec2_5} \tag{2.5}
\end{equation*}

este resultado también puede ser escrito en función del vector posición $\vec{\boldsymbol{r}}$ de la siguiente manera

\begin{equation*}
\vec{\boldsymbol{a}}=\frac{d^2{\boldsymbol{r}}}{dt^2}
\label{eq:Ec2_6} \tag{2.6}
\end{equation*}

### Comentarios al movimiento curvilíneo general

- Por definición de la derivada, $\vec{\boldsymbol{a}}$ actúa tangente a la hodógrafa,

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

y, en general no es tangente a la trayectoria del movimiento.

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

- Tanto $\Delta \vec{\boldsymbol{v}}$ como $\vec{\boldsymbol{a}}$, deben responder el cambio tanto de magnitud como de dirección de la velocidad $\vec{\boldsymbol{v}}$ a medida que la partícula se mueve de un punto al siguiente a lo largo de la trayectoria.

- Para que la partícula siga cualquier trayectoria curva, el cambio direccional siempre *“cambia”* el vector de velocidad hacia el *“interior”* o *“lado cóncavo”* de la trayectoria, y por consiguiente a no puede permanecer tangente a la trayectoria.

- En conclusión, $\vec{\boldsymbol{v}}$ siempre es tangente a la trayectoria y $\vec{\boldsymbol{a}}$ siempre es tangente a la hodógrafa.

## Movimiento curvilíneo: Componente rectangulares

El movimiento curvilíneo también se puede expresar en función de las coordenadas $x$, $y$, $z$ que cubren su trayectoria.

### Posición

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Si la partícula está en el punto $(x, y, z)$ de la trayectoria curva $s$, entonces su posición se definirá por el vector:

\begin{equation*}
\vec{\boldsymbol{r}}=x\vec{\boldsymbol{i}}+y\vec{\boldsymbol{j}}+z\vec{\boldsymbol{k}}
\label{eq:Ec2_7} \tag{2.7}
\end{equation*}

***Comentarios:***

- Cuando la partícula se mueve, las compontes del vector $\vec{\boldsymbol{r}}$ se representan en función del tiempo, es decir, $x(t), y(t), z(t)$. Por lo que $\vec{\boldsymbol{r}}=\vec{\boldsymbol{r}}(t)$

- La magnitud de $\vec{\boldsymbol{r}}$ estará dada por la [norma euclideana](https://en.wikipedia.org/wiki/Euclidean_space#Euclidean_norm):

\begin{equation*}
r=\sqrt{x^2+y^2+z^2}
\label{eq:Ec2_8} \tag{2.8}
\end{equation*}

- La dirección del vector $\vec{\boldsymbol{r}}$ se determina a través del [vector unitario](https://en.wikipedia.org/wiki/Unit_vector):

\begin{equation*}
\vec{\boldsymbol{u}}_r=\frac{\vec{\boldsymbol{r}}}{r}
\label{eq:Ec2_9} \tag{2.9}
\end{equation*}

### Velocidad

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Derivando el vector posición respecto al tiempo, se obtendrá el vector velocidad:

\begin{equation*}
\vec{\boldsymbol{v}}=\frac{d\vec{\boldsymbol{r}}}{dt}=\frac{d}{dt}(x\vec{\boldsymbol{i}})+\frac{d}{dt}(y\vec{\boldsymbol{j}})+\frac{d}{dt}(z\vec{\boldsymbol{k}})
\label{eq:Ec2_10} \tag{2.10}
\end{equation*}

***Comentarios:***

- Tener en cuenta que al derivar es necesario derivar tanto la magnitud como la dirección en cada componente, es decir:

\begin{equation*}
\begin{aligned}
\frac{d}{dt}(x\vec{\boldsymbol{i}})=\frac{dx}{dt}\vec{\boldsymbol{i}}+x\frac{d\vec{\boldsymbol{i}}}{dt} \\
\frac{d}{dt}(y\vec{\boldsymbol{j}})=\frac{dy}{dt}\vec{\boldsymbol{j}}+y\frac{d\vec{\boldsymbol{j}}}{dt} \\
\frac{d}{dt}(z\vec{\boldsymbol{k}})=\frac{dz}{dt}\vec{\boldsymbol{k}}+z\frac{d\vec{\boldsymbol{k}}}{dt}
\end{aligned}
\label{eq:Ec2_11} \tag{2.11}
\end{equation*}

- El segundo término del lado derecho en cada una de las ecuaciones anteriores será cero si el marco de referencia es fijo, por lo que la dirección y magnitud de $\vec{\boldsymbol{i}}$ no cambia con el tiempo.

- Es usual representar el vector velocidad también como:

\begin{equation*}
\vec{\boldsymbol{v}}=\frac{d\vec{\boldsymbol{r}}}{dt}=v_x\vec{\boldsymbol{i}}+v_y\vec{\boldsymbol{j}}+v_z\vec{\boldsymbol{k}}
\label{eq:Ec2_12} \tag{2.12}
\end{equation*}

- O usando la notación "punto" que representa las primeras derivadas de $x=x(t)$, $y=y(t)$, $z=z(t)$: $v_x=\dot{x}$, $v_y=\dot{y}$, $v_z=\dot{z}$

- La magnitud de la velocidad es a su vez:

\begin{equation*}
v=\sqrt{v_x^2+v_y^2+v_z^2}
\label{eq:Ec2_13} \tag{2.13}
\end{equation*}

- y el vector unitario, que especifica su dirección y es tangente a la trayectoria, está dado por:

\begin{equation*}
\vec{\boldsymbol{u}}_v=\frac{\vec{\boldsymbol{v}}}{v}
\label{eq:Ec2_14} \tag{2.14}
\end{equation*}

### Aceleración

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

Sacando la primera derivada del vector velocidad respecto al tiempo (o la segunda derivada del vector posición), se obtiene el vector aceleración:

\begin{equation*}
\vec{\boldsymbol{a}}=\frac{d\vec{\boldsymbol{v}}}{dt}=a_x\vec{\boldsymbol{i}}+a_y\vec{\boldsymbol{j}}+a_z\vec{\boldsymbol{k}}
\label{eq:Ec2_15} \tag{2.15}
\end{equation*}

***Comentarios:***

- Usando la notación "punto":

\begin{equation*}
\begin{aligned}
a_x=\dot{v}_x=\ddot{x} \\
a_y=\dot{v}_y=\ddot{y} \\
a_z=\dot{v}_z=\ddot{z}
\end{aligned}
\label{eq:Ec2_16} \tag{2.16}
\end{equation*}

- La magnitud de la aceleración estará dada por,

\begin{equation*}
a=\sqrt{a_x^2+a_y^2+a_z^2}
\label{eq:Ec2_17} \tag{2.17}
\end{equation*}

- el vector unitario, está dado por:

\begin{equation*}
\vec{\boldsymbol{u}}_a=\frac{\vec{\boldsymbol{a}}}{a}
\label{eq:Ec2_18} \tag{2.18}
\end{equation*}

Como la aceleración representa el cambio tanto de la magnitud como de la dirección de la velocidad, en geneeral, no es tangente a la trayectoria.

### Ejemplos movimiento curvilínea componentes rectangulares

#### Globo atmosférico

 Ejemplo 12.9: Hibbeler R. Engineering Mechanics: Dynamics

En cualquier instante $x=(8t)pies$, donde $t$ está en segundos, define la posición horizontal del globo atmosférico de la figura. Si la ecuación de la trayectoria es $y = x^2/10$, determina la magnitud y dirección de la velocidad y la aceleración cuando $t = 2 s$.

***Solución analítica:***

- ***Velocidad:***

La componente de la velocidad en la dirección $x$ se obtiene como:

$$v_x=\dot{x}=\frac{d}{dt}(8t)=8 pies/s \rightarrow$$

Para determinar la componente de la velocidad en la dirección $y$ es necesario emplear la [regla de la cadena](https://en.wikipedia.org/wiki/Chain_rule) vista en cálculo

$$v_y=\dot{y}=\frac{d}{dt}(x^2/10)=2x\dot{x}/10=2(16)(8)/10=25.6pies/s \uparrow$$

en $t=2s$, la magnitud de la velocidad es:

$$v=\sqrt{(8 pies/s)^2+(25.6pies/s)^2}=26.8 pies/s$$

Para determinar la dirección, se debe tener presente que ésta se calcula como el ángulo respecto al eje $x$ y es tangente a la trayectoria.

 Fuente: Hibbeler R. Engineering Mechanics: Dynamics

$$\theta_v=tan^{-1}\frac{v_y}{v_x}=tan^{-1}\frac{25.6}{8}=72.6^\circ$$

- ***Aceleración:***

Procediendo de la misma forma que para la velocidad, se empleará la Regla de la Cadena para determinar las componentes de la aceleración

$$a_x=\dot{v}_x=\frac{d}{dt}(8)=0$$
$$a_y=\dot{v}_y=\frac{d}{dt}(2x\dot{x}/10)=2(\dot{x})\dot{x}/10+2x\ddot{x}/10$$

\begin{equation*}
\begin{aligned}
a_y &=\dot{v}_y=\frac{d}{dt}(2x\dot{x}/10)=2(\dot{x})\dot{x}/10+2x\ddot{x}/10 \\
    &=2(8)^2/10+2(16)(0)=12.8pies/s^2 \uparrow
\end{aligned}
\end{equation*}

la magnitud estaría dada por

$$a=\sqrt{(0)^2+(12.8)^2}=12.8 pies/s^2$$

y su dirección

$$\theta_a=tan^{-1}\frac{12.8}{0}=90^\circ$$

***Solución computacional***

```python
from sympy import *
t = symbols('t')
init_printing(use_latex='mathjax')
```

Para el cálculo de las componentes de la velocidad tanto en la dirección $x$ como $y$ se procederá de la siguiente forma.

- Se crean las funciones de la posición y la trayectoria

```python
# Ecuación de la posición x en un instante t
x = 8 * t

# Ecuación de la trayectoria
y = x**2 / 10
```

- La velocidad es la derivada del espacio, para cada una de sus componentes, respecto al tiempo

```python
# Cálculo de la velocidad en la dirección x
vx = diff(x,t)
vx
```

$\displaystyle 8$

```python
# Evaluación de la velocidad en x cuando t = 2 (en este caso es una constante)
vx = vx.subs(t,2)
vx
```

$\displaystyle 8$

```python
# calculo de la velocidad en la dirección x, y evaluación en t=2 en una misma línea de código
vx = diff(x,t).subs(t,2)
vx
```

$\displaystyle 8$

```python
# Cálculo de la velocidad en la dirección y
vy = diff(y,t)
vy
```

$\displaystyle \frac{64 t}{5}$

```python
# Evaluación de la velocidad en y cuando t = 2 y representación en punto flotante (Numerical)
vy = N(vy.subs(t,2))
vy
```

$\displaystyle 25.6$

- Se cacula la magnitud euclideana

```python
magVel = sqrt(vx**2 + vy**2).evalf(3)
magVel
```

$\displaystyle 26.8$

```python
magVel = N(sqrt(vx**2 + vy**2),3)
magVel
```

$\displaystyle 26.8$

- Para el cálculo de la dirección hay qué tener en cuenta que los valores resultantes de los cálculos numéricos se expresan en [radianes](https://es.wikipedia.org/wiki/Radi%C3%A1n). Por lo que es necesario convertirlos a grados mediante la fórmula

$$x[grad] = x [rad] \times \frac{180}{\pi}$$

```python
theta = N(atan(vy / vx) * 180 / pi, 3)
theta
```

$\displaystyle 72.6$

Se deja como ejercicio al estudiante el desarrollo del punto que corresponde a la aceleración


## Machine Translation
A sequence-to-sequence model is a model that takes a sequence of items and outputs another sequence of items using two networks that are trained end-to-end. This is perfect for machine translation since input sequences are directly related to output sequences. We will looking at preparing a dataset for Machine Translation task and implementing a seq2seq model. We will be using the parallel corpus available from [here](ftp://ftp.monash.edu/pub/nihongo/examples.utf.gz)

### Concepts covered here:
- Encoder-Decoder architecture: We will be using a encoder to encode input sequences from one language in a latent space and use the encoding to generate words in the target language one token at a time. The encoder and the decoder are an RNNs.

- Attention: We will be looking at the attention mechanism described in [Bahdanau et al., 2015](https://arxiv.org/pdf/1409.0473.pdf). We will explore the theory, the intuition behind it and how to move from theory and intuition to implementation.

```
%matplotlib inline
from pathlib import Path
import re,string
import numpy as np
from google.colab import files
```

```
!mkdir data
```

```
path = Path('data')
```

We will need to download the translation corpus and install a tokenizer for Japanese text.

```
#get corpus
#!wget ftp://ftp.monash.edu/pub/nihongo/examples.utf.gz
#decompress it and move it to data folder
#!gunzip examples.utf.gz
#!mv examples.utf data/
#install dependencies for mecab tokenizer
#!sudo apt install swig
#!sudo apt install mecab
#!sudo apt install libmecab-dev
#!sudo apt install mecab-ipadic-utf8
#!sudo pip3 install mecab-python3
```

```
corpus = (path/'examples.utf').open().readlines()
corpus[:5]
```

    ['A: ムーリエルは２０歳になりました。\tMuiriel is 20 now.#ID=1282_4707\n',
     'B: は 二十歳(はたち){２０歳} になる[01]{になりました}\n',
     'A: すぐに戻ります。\tI will be back soon.#ID=1284_4709\n',
     'B: 直ぐに{すぐに} 戻る{戻ります}\n',
     'A: すぐに諦めて昼寝をするかも知れない。\tI may give up soon and just nap instead.#ID=1300_4727\n']

```
def make_corpus(corpus_path):
    corpus = corpus_path.open().readlines()
    en,ja = [],[]
    pat = r'#ID.+\n'
    for c in corpus:
        if 'A: ' in c:
            clean_c = c.replace('A: ','')
            res = re.search(pat,clean_c)
            clean_c = clean_c.replace(res.group(0),'').split('\t')
            ja.append(clean_c[0])
            en.append(clean_c[1])
    return en,ja
```

```
en, ja = make_corpus(path/'examples.utf')
en[:2],ja[:2]
```

    (['Muiriel is 20 now.', 'I will be back soon.'],
     ['ムーリエルは２０歳になりました。', 'すぐに戻ります。'])

```
import MeCab
```

```
tagger = MeCab.Tagger('-Owakati')

def ja_tokenizer(text):
    result = tagger.parse(text)
    words = result.split()
    if len(words) ==0: return []
    if words[-1] == '\n':return words[:-1]
    return words
```

```
ja_tokenizer(ja[0])
```

    ['ムーリエル', 'は', '２', '０', '歳', 'に', 'なり', 'まし', 'た', '。']

```
import spacy
from spacy.symbols import ORTH
```

```
en_tok = spacy.load('en')
```

```
def en_tokenizer(text):
    text = text.lower()
    return [t.text for t in en_tok.tokenizer(text)]
```

```
en_tokenizer(en[0])
```

    ['muiriel', 'is', '20', 'now', '.']

```
en_toks = [en_tokenizer(text) for text in en]
ja_toks = [ja_tokenizer(text) for text in ja]
en_toks[:2], ja_toks[:2]
```

    ([['muiriel', 'is', '20', 'now', '.'],
      ['i', 'will', 'be', 'back', 'soon', '.']],
     [['ムーリエル', 'は', '２', '０', '歳', 'に', 'なり', 'まし', 'た', '。'],
      ['すぐ', 'に', '戻り', 'ます', '。']])

```
len(en_toks), len(ja_toks)
```

    (149785, 149785)

```
from collections import Counter,defaultdict
```

```
def numericalize_tok(tokens, max_vocab=50000, min_freq=0, unk_tok="xxunk", pad_tok="xxpad", bos_tok="xxbos", eos_tok="xxeos"):
    if isinstance(tokens, str):
        raise ValueError("Expected to receive a list of tokens. Received a string instead")
    if isinstance(tokens[0], list):
        tokens = [p for o in tokens for p in o]
    freq = Counter(tokens)
    int2tok = [o for o,c in freq.most_common(max_vocab) if c>min_freq]
    unk_id = 3
    int2tok.insert(0, bos_tok)
    int2tok.insert(1, pad_tok)
    int2tok.insert(2, eos_tok)
    int2tok.insert(unk_id, unk_tok)
    tok2int = defaultdict(lambda:unk_id, {v:k for k,v in enumerate(int2tok)})
    return int2tok, tok2int
```

```
int2j,j2int = numericalize_tok(ja_toks)
int2en,en2int = numericalize_tok(en_toks)
```

```
len(int2j), len(int2en)
```

    (31813, 21393)

```
import pickle
```

```
pickle.dump(int2j,(path/'int2j.pkl').open('wb'))
pickle.dump(int2en,(path/'int2en.pkl').open('wb'))
```

```
int2j = pickle.load((path/'int2j.pkl').open('rb'))
int2en = pickle.load((path/'int2en.pkl').open('rb'))
j2int = defaultdict(lambda:3, {v:k for k,v in enumerate(int2j)})
en2int = defaultdict(lambda:3, {v:k for k,v in enumerate(int2en)})
```

```
len(int2j), len(int2en)
```

    (12677, 9290)

```
j_ids = np.array([[0]+[j2int[o] for o in sent]+[2] for sent in ja_toks])
en_ids = np.array([[0]+[en2int[o] for o in sent]+[2] for sent in en_toks])
len(j_ids),len(en_ids), j_ids[10],en_ids[10]
```

    (149785,
     149785,
     [0,
      48,
      6,
      4891,
      5,
      109,
      11,
      143,
      10,
      83,
      8,
      57,
      86,
      1798,
      7,
      2146,
      232,
      255,
      47,
      36,
      4,
      2],
     [0, 114, 2251, 107, 38, 97, 85, 2649, 77, 28, 356, 4, 2])

```
np.random.seed(42)
```

```
trn_keep = np.random.rand(len(en_ids))>0.1
en_trn,j_trn = en_ids[trn_keep],j_ids[trn_keep]
en_val,j_val = en_ids[~trn_keep],j_ids[~trn_keep]
len(en_trn),len(en_val)
```

    (134774, 15011)

```
import torch
import torch.nn as nn
import torch.nn.functional as F

from torch.autograd.variable import Variable
from torch.utils.data import Dataset,DataLoader
```

```
from numpy import array as A
```

```
class Seq2SeqDataset(Dataset):
    def __init__(self, x, y): self.x,self.y = x,y
    def __getitem__(self, idx): return A(self.x[idx]), A(self.y[idx])
    def __len__(self): return len(self.x)
```

```
trn_ds = Seq2SeqDataset(en_trn,j_trn)
val_ds = Seq2SeqDataset(en_val,j_val)

bs = 120

trn_dl = DataLoader(trn_ds,batch_size=bs,shuffle=True)
val_dl = DataLoader(val_ds,batch_size=bs)
```

```
x, y = next(iter(val_dl))
x.size(), y.size()
```

    (torch.Size([120, 25]), torch.Size([120, 25]))

```
from keras.preprocessing.sequence import pad_sequences
```

```
enlen_90 = int(np.percentile([len(o) for o in en_ids], 99))
jalen_90 = int(np.percentile([len(o) for o in j_ids], 99))
enlen_90,jalen_90
```

    (25, 29)

```
j_ids = pad_sequences(j_ids, maxlen=29, dtype='int32', padding='post', truncating='post', value=1)
en_ids = pad_sequences(en_ids, maxlen=25, dtype='int32', padding='post', truncating='post', value=1)
```

```
trn_keep = np.random.rand(len(en_ids))>0.1
en_trn,j_trn = en_ids[trn_keep],j_ids[trn_keep]
en_val,j_val = en_ids[~trn_keep],j_ids[~trn_keep]
len(en_trn),len(en_val)
```

    (134746, 15039)

```
trn_ds = Seq2SeqDataset(en_trn,j_trn)
val_ds = Seq2SeqDataset(en_val,j_val)

bs = 120

trn_dl = DataLoader(trn_ds,batch_size=bs,shuffle=True)
val_dl = DataLoader(val_ds,batch_size=bs)
```

```
x, y = next(iter(val_dl))
x.size(), y.size()
```

    (torch.Size([120, 25]), torch.Size([120, 29]))

```
## load fasttext vectors
#code from here:https://github.com/facebookresearch/fastText/blob/master/docs/crawl-vectors.md
import io
def load_vectors(fname):
    fin = io.open(fname, 'r', encoding='utf-8', newline='\n', errors='ignore')
    header = fin.readline().split()
    n, d = int(header[0]), int(header[1])
    data = {}
    for line in fin:
        tokens = line.rstrip().split(' ')
        data[tokens[0]] = np.array(tokens[1:], dtype=float)
    return data, int(n), int(d)
```

```
# get word vectors
!wget https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M.vec.zip
!wget https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.ja.300.vec.gz
!unzip wiki-news-300d-1M.vec.zip
!gunzip cc.ja.300.vec.gz
!mv wiki-news-300d-1M.vec data/
!mv cc.ja.300.vec data/
```

    --2019-03-15 09:27:27--  https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M.vec.zip
    Resolving dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)... 104.20.22.166, 104.20.6.166, 2606:4700:10::6814:6a6, ...
    Connecting to dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)|104.20.22.166|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 681808098 (650M) [application/zip]
    Saving to: ‘wiki-news-300d-1M.vec.zip’

    wiki-news-300d-1M.v 100%[===================>] 650.22M  30.1MB/s    in 22s

    2019-03-15 09:27:49 (29.4 MB/s) - ‘wiki-news-300d-1M.vec.zip’ saved [681808098/681808098]

    --2019-03-15 09:27:51--  https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.ja.300.vec.gz
    Resolving dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)... 104.20.22.166, 104.20.6.166, 2606:4700:10::6814:6a6, ...
    Connecting to dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)|104.20.22.166|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 1279641604 (1.2G) [binary/octet-stream]
    Saving to: ‘cc.ja.300.vec.gz’

    cc.ja.300.vec.gz    100%[===================>]   1.19G  30.1MB/s    in 41s

    2019-03-15 09:28:32 (29.9 MB/s) - ‘cc.ja.300.vec.gz’ saved [1279641604/1279641604]

    Archive:  wiki-news-300d-1M.vec.zip
      inflating: wiki-news-300d-1M.vec

```
en_vecs,_,dim_en_vec = load_vectors('data/wiki-news-300d-1M.vec')
j_vecs,_,dim_j_vec = load_vectors('data/cc.ja.300.vec')
```

```
def create_emb(vecs, itos, em_sz):
    emb = nn.Embedding(len(itos), em_sz, padding_idx=1)
    if vecs is None: return emb
    wgts = emb.weight.data
    miss = []
    for i,w in enumerate(itos):
        try: wgts[i] = torch.from_numpy(vecs[w])
        except: miss.append(w)
    print('Number of unknowns in data: {}'.format(len(miss)))
    return emb

```

```
def V(tensor,req_grad=True):
    if torch.cuda.is_available():return Variable(tensor.cuda())
    else: return Variable(tensor)
```

### RNN Visualization

[RNN Visualization](http://jalammar.github.io/images/RNN_1.mp4)

### Seq2Seq Architecture
> ...a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector.

[Sutskever et al., 2014](https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf)

Image:[from here](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)

```
class Seq2Seq(nn.Module):
    def __init__(self,int2en,int2j,em_sz,j_vecs=None,en_vecs=None,nh=128,out_sl=25,dropf=1,nl=2):
        super().__init__()
        #encoder
        self.nl,self.nh,self.em_sz,self.out_sl = nl,nh,em_sz,out_sl
        self.emb_enc = create_emb(en_vecs,int2en,em_sz)
        self.emb_drop = nn.Dropout(0.15*dropf)
        self.encoder = nn.GRU(em_sz,nh,num_layers=nl,dropout=0.25*dropf, bidirectional=True)
        #decoder
        self.emb_dec = create_emb(j_vecs,int2j,em_sz)
        self.decoder = nn.GRU(em_sz,nh*2,num_layers=nl,dropout=0.25*dropf)
        self.out_drop = nn.Dropout(0.35*dropf)
        self.out = nn.Linear(nh*2,len(int2j))

    def forward(self,inp,y=None):
        sl, bs = inp.size()
        emb_in = self.emb_drop(self.emb_enc(inp))
        h_n = self.initHidden(bs)
        enc_out, h_n = self.encoder(emb_in,h_n)
        h_n = h_n.view(2,2,bs,-1).permute(0,2,1,3).contiguous().view(self.nl,bs,-1)

        dec_inp = V(torch.zeros(bs).long())
        res = []
        for i in range(self.out_sl):
            dec_emb = self.emb_dec(dec_inp)
            outp,h_n = self.decoder(dec_emb.unsqueeze(0),h_n)
            outp = F.log_softmax(self.out(self.out_drop(outp[0])),dim=-1)
            res.append(outp)
            dec_inp = outp.data.max(1)[1]
            if (dec_inp==1).all(): break
        return torch.stack(res)

    def initHidden(self,bs):
        return V(torch.zeros([self.nl*2,bs,self.nh]))
```

### Why Bidirectional Encoder
> Finally, we found that reversing the order of the words in all source sentences (butnot target sentences) improved the LSTM’s performance markedly, because doing so introduced many short term dependencies between the source and the targetsentence which made the optimization problem easier.

[Sutskever et al., 2014](https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf)

Input and output sequences may not directly map to each other so preserving information from both passes of the input sequence will help learn how tokens relate to each other. For example in a translation task, subject and object can be in opposite positions depending on the language structure.

```
seq2seq = Seq2Seq(int2en,int2j,300,en_vecs=None,j_vecs=None)
seq2seq.cuda()
```

    Seq2Seq(
      (emb_enc): Embedding(21393, 300, padding_idx=1)
      (emb_drop): Dropout(p=0.15)
      (encoder): GRU(300, 128, num_layers=2, dropout=0.25, bidirectional=True)
      (emb_dec): Embedding(31813, 300, padding_idx=1)
      (decoder): GRU(300, 256, num_layers=2, dropout=0.25)
      (out_drop): Dropout(p=0.35)
      (out): Linear(in_features=256, out_features=31813, bias=True)
    )

```
out = seq2seq(V(x.transpose(1,0).long()))
out.size()
```

    torch.Size([25, 120, 31813])

Loss Function: We will use Cross Entropy Loss as we are trying to classify ot the correct words. Cross entropy loss can be simplified to:

`cross_entropy = sum(-log(y_pred) for y_pred in y_preds)`

where `y_pred` is the likelihood of the target class predicted by the model. This is a good loss function for classification because if the likelihood of the correct class is low, the loss value goes up and if it is high, the loss value goes down.

```
def seq2seq_loss(input, target):
    sl,bs = target.size()
    sl_in,bs_in,nc = input.size()
    if sl>sl_in: input = F.pad(input, (0,0,0,0,0,sl-sl_in))
    input = input[:sl]
    return F.cross_entropy(input.view(-1,nc), target.view(-1))
```

```
seq2seq_loss(out,V(y.transpose(1,0).long()))
```

    tensor(10.3560, device='cuda:0', grad_fn=<NllLossBackward>)

```
def step(x, y, epoch, m, crit, opt, clip=None):
    output = m(x, y)
    if isinstance(output,tuple): output = output[0]
    opt.zero_grad()
    loss = crit(output, y)
    loss.backward()
    if clip:
        nn.utils.clip_grad_norm_(m.parameters(), clip)
    opt.step()
    return loss.data.item()
```

```
from tqdm import tqdm
```

```
def train(trn_dl,val_dl,model,crit,opt,epochs=10,clip=None):
    for epoch in range(epochs):
        loss_val = loss_trn = 0
        with tqdm(total=len(trn_dl)) as pbar:
            model.train()
            for i, ds in enumerate(trn_dl):
                x, y = ds
                #if isinstance(x,tuple): x = x[0]
                x, y = x.transpose(1,0), y.transpose(1,0)
                loss = step(V(x.long()),V(y.long()),epoch,model,crit,opt)
                loss_trn += loss
                pbar.update()
        model.eval()
        for i, ds in enumerate(val_dl):
            with torch.no_grad():
                x, y = ds
                #if isinstance(x,tuple): x = x[0]
                x, y = x.transpose(1,0), y.transpose(1,0)
                out = model(V(x.long()))
                if isinstance(out,tuple): out = out[0]
                loss_val+= crit(out, V(y.long()))
                #loss_val +=loss
        print(f'Epoch: {epoch} trn loss: {loss_trn/len(trn_dl)} val loss: {loss_val/len(val_dl)}')
```

```
from torch import optim
```

```
opt = optim.Adam(seq2seq.parameters(),lr=3e-3,betas=(0.7,0.8))
```

```
train(trn_dl,val_dl,seq2seq,seq2seq_loss,opt,epochs=10)
```

    100%|██████████| 1124/1124 [02:40<00:00,  7.02it/s]
      0%|          | 1/1124 [00:00<03:05,  6.06it/s]

    Epoch: 0 trn loss: 4.750732962983359 val loss: 4.082951068878174

    100%|██████████| 1124/1124 [02:52<00:00,  6.51it/s]
      0%|          | 1/1124 [00:00<03:11,  5.86it/s]

    Epoch: 1 trn loss: 3.5997307712073003 val loss: 3.3246185779571533

    100%|██████████| 1124/1124 [02:55<00:00,  6.39it/s]
      0%|          | 1/1124 [00:00<03:04,  6.08it/s]

    Epoch: 2 trn loss: 3.1077632744965604 val loss: 3.4751904010772705

    100%|██████████| 1124/1124 [02:57<00:00,  6.34it/s]
      0%|          | 1/1124 [00:00<02:50,  6.60it/s]

    Epoch: 3 trn loss: 2.8518904808153036 val loss: 3.94274640083313

    100%|██████████| 1124/1124 [02:58<00:00,  6.30it/s]
      0%|          | 1/1124 [00:00<03:06,  6.01it/s]

    Epoch: 4 trn loss: 2.7045018161743135 val loss: 2.766321897506714

    100%|██████████| 1124/1124 [02:58<00:00,  6.29it/s]
      0%|          | 1/1124 [00:00<03:06,  6.01it/s]

    Epoch: 5 trn loss: 2.5743112220458713 val loss: 2.752124071121216

    100%|██████████| 1124/1124 [02:58<00:00,  6.28it/s]
      0%|          | 1/1124 [00:00<03:04,  6.08it/s]

    Epoch: 6 trn loss: 2.5332621384769998 val loss: 2.935807228088379

    100%|██████████| 1124/1124 [02:59<00:00,  6.27it/s]
      0%|          | 1/1124 [00:00<03:03,  6.11it/s]

    Epoch: 7 trn loss: 2.500852593323514 val loss: 3.133931875228882

    100%|██████████| 1124/1124 [02:59<00:00,  6.99it/s]
      0%|          | 1/1124 [00:00<03:04,  6.10it/s]

    Epoch: 8 trn loss: 2.458315055141245 val loss: 2.8610451221466064

    100%|██████████| 1124/1124 [02:59<00:00,  6.26it/s]

    Epoch: 9 trn loss: 2.456163497369909 val loss: 3.016350269317627

```
def produce_out(val_dl, model,int2en,int2j,interval=(20,30)):
    model.eval()
    x,y = next(iter(val_dl))
    x, y = x.transpose(1,0), y.transpose(1,0)
    probs = seq2seq(V(x.long()))
    if isinstance(probs,tuple): probs = probs[0]
    preds = A(probs.max(2)[1].cpu())
    for i in range(interval[0],interval[1]):
        print(' '.join([int2en[o] for o in x[:,i] if o not in [0,1,2]]))
        print(''.join([int2j[o] for o in y[:,i] if o not in [0,1,2]]))
        print(''.join([int2j[o] for o in preds[:,i] if o not in [0,1,2]]))
        print()
```

```
produce_out(trn_dl,seq2seq,int2en,int2j)
```

    the flower garden needs watering .
    その花壇は水をやる必要がある。
    その花は庭がががが。

    country life is very peaceful in comparison with city life .
    田舎での生活は、都会生活と比較してとても穏やかだ。
    国ののはははのののににに。。

    you are beautifully dressed .
    あなたはとても美しいドレスを着ていらっしゃいますね。
    あなたははなををててている。

    it 's quite all right .
    全くかまいません。
    それはですだ。

    the president put off visiting japan .
    大統領は訪日を延期しました。
    大統領は日本にををしたた

    he has two brothers , one lives in osaka and the other in kobe .
    彼には兄弟が二人いて、一人は大阪で、もう一人は神戸で暮らしている。
    彼はは２人２人ががが、人人でで。。

    a famous architect built this house .
    有名な建築家がこの家を建てた。
    そのな家家家家家家をたた。

    i could not but think that he had died .
    彼は死んでしまったと考えざるを得なかった。
    私は彼たたたはははたた。

    i enjoyed watching the easter parade .
    私は復活祭のパレードを見て楽しんだ。
    私はそのをを見を見を見。。

    i will not allow you to be ill - treated .
    君が虐待されているのを放ってはいられない。
    私は病気にににににない。。。

```
torch.save(seq2seq.state_dict(),open(path/'translate_seq2seq.pth','wb'))
```

```
seq2seq.load_state_dict(torch.load('translate_seq2seq.pth', map_location=lambda storage, loc: storage))
```

### Seq2Seq w/ Attention
A critical and apparent disadvantage of this fixed-length context vector design is incapability of remembering long sentences. Often it has forgotten the first part once it completes processing the whole input. The attention mechanism was born [Bahdanau et al., 2015](https://arxiv.org/pdf/1409.0473.pdf) to resolve this problem.

Given the following vectors:
\begin{align}
\boldsymbol{x} = \{x_1,x_2,x_3,\ldots,x_n\} \\
\boldsymbol{y} = \{y_1,y_2,y_3,\ldots,y_m\} \\
\end{align}
The hidden state from the Bidir encoder is given by:
\begin{align}
\boldsymbol{h}_i = [\overrightarrow{\boldsymbol{h}}_i^\top; \overleftarrow{\boldsymbol{h}}_i^\top]^\top, i=1,\dots,n \\
\end{align}

The hidden state from the decoder at time $t$ is given by:  $s_t$

The score for at time $t$:

\begin{aligned}
\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \mathbf{v}_a^\top \tanh(\mathbf{W}_a[\boldsymbol{s}_t; \boldsymbol{h}_i])
\end{aligned}

where both $v_a$ and $W_a$ are weight matrices to be learned in the alignment model.

\begin{aligned}
\mathbf{c}_t &= \sum_{i=1}^n \alpha_{t,i} \boldsymbol{h}_i & \small{\text{; Context vector for output }y_t}\\
\alpha_{t,i} &= \text{align}(y_t, x_i) & \small{\text{; How well two words }y_t\text{ and }x_i\text{ are aligned.}}\\
&= \frac{\exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_i))}{\sum_{i'=1}^n \exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_{i'}))} & \small{\text{; Softmax of some predefined alignment score.}}.
\end{aligned}

Equations borrowed from [here](https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html)

[Attention Visualization](http://jalammar.github.io/images/attention_process.mp4)

```
import math,random

def rand_t(*sz): return torch.randn(sz)/math.sqrt(sz[0])
def rand_p(*sz): return nn.Parameter(rand_t(*sz))
```

```
class Seq2SeqAttention(nn.Module):
    def __init__(self,int2en,int2j,em_sz,j_vecs=None,en_vecs=None,nh=128,out_sl=25,dropf=1,nl=2):
        super().__init__()
        #encoder
        self.nl,self.nh,self.em_sz,self.out_sl = nl,nh,em_sz,out_sl
        self.emb_enc = create_emb(en_vecs,int2en,em_sz)
        self.emb_drop = nn.Dropout(0.15*dropf)
        self.encoder = nn.GRU(em_sz,nh,num_layers=nl,dropout=0.25*dropf, bidirectional=True)
        #decoder
        self.emb_dec = create_emb(j_vecs,int2j,em_sz)
        self.decoder = nn.GRU(em_sz,nh*2,num_layers=nl,dropout=0.25*dropf)
        self.out_drop = nn.Dropout(0.35*dropf)
        self.out = nn.Linear(nh*2,len(int2j))
        #attention layer
        self.W1 = rand_p(nh*2, nh*2) #parameter
        self.l2 = nn.Linear(nh*2, nh*2)
        self.l3 = nn.Linear(em_sz+nh*2, em_sz)
        self.V = rand_p(nh*2) #parameter

    def forward(self,inp,y=None):
        sl, bs = inp.size()
        emb_in = self.emb_drop(self.emb_enc(inp))
        h_n = self.initHidden(bs)
        enc_out, h_n = self.encoder(emb_in,h_n)
        h_n = h_n.view(2,2,bs,-1).permute(0,2,1,3).contiguous().view(self.nl,bs,-1)

        dec_inp = V(torch.zeros(bs).long())
        res,attns = [], []
        #multiply by parameter
        w1e = enc_out @ self.W1
        for i in range(self.out_sl):
            #linear layer
            w2h = self.l2(h_n[-1])
            #non-linear activation to calculate score
            u = torch.tanh(w1e + w2h)
            #softmax to make them into probs
            a = F.softmax(u @ self.V, 0)
            attns.append(a)
            #multiply each vector by scores and then add them up
            Xa = (a.unsqueeze(2) * enc_out).sum(0)
            dec_emb = self.emb_dec(dec_inp)
            #linear layer to reduce dimensions
            wgt_enc = self.l3(torch.cat([dec_emb, Xa], 1))
            outp,h_n = self.decoder(wgt_enc.unsqueeze(0),h_n)
            outp = F.log_softmax(self.out(self.out_drop(outp[0])),dim=-1)
            res.append(outp)
            dec_inp = outp.data.max(1)[1]
            if (random.random() > 0.5) and y is not None: dec_inp=y[i]
            if (dec_inp==1).all(): break
        return torch.stack(res),attns

    def initHidden(self,bs):
        return V(torch.zeros([self.nl*2,bs,self.nh]))
```

```
seq2seq = Seq2SeqAttention(int2en,int2j,300,en_vecs=None,j_vecs=None)
seq2seq.cuda()
```

    Seq2SeqAttention(
      (emb_enc): Embedding(21393, 300, padding_idx=1)
      (emb_drop): Dropout(p=0.15)
      (encoder): GRU(300, 128, num_layers=2, dropout=0.25, bidirectional=True)
      (emb_dec): Embedding(31813, 300, padding_idx=1)
      (decoder): GRU(300, 256, num_layers=2, dropout=0.25)
      (out_drop): Dropout(p=0.35)
      (out): Linear(in_features=256, out_features=31813, bias=True)
      (l2): Linear(in_features=256, out_features=256, bias=True)
      (l3): Linear(in_features=556, out_features=300, bias=True)
    )

```
opt = optim.Adam(seq2seq.parameters(),lr=3e-3,betas=(0.7,0.8))
```

```
train(trn_dl,val_dl,seq2seq,seq2seq_loss,opt,epochs=10)
```

    100%|██████████| 1124/1124 [03:21<00:00,  6.28it/s]
      0%|          | 1/1124 [00:00<03:27,  5.42it/s]

    Epoch: 0 trn loss: 2.797933133683595 val loss: 4.138897895812988

    100%|██████████| 1124/1124 [03:26<00:00,  6.24it/s]
      0%|          | 1/1124 [00:00<03:32,  5.28it/s]

    Epoch: 1 trn loss: 2.16145394790215 val loss: 2.8873095512390137

    100%|██████████| 1124/1124 [03:26<00:00,  6.17it/s]
      0%|          | 1/1124 [00:00<03:29,  5.35it/s]

    Epoch: 2 trn loss: 2.0703822573733075 val loss: 3.0309219360351562

    100%|██████████| 1124/1124 [03:26<00:00,  6.07it/s]
      0%|          | 1/1124 [00:00<03:28,  5.38it/s]

    Epoch: 3 trn loss: 2.07098806148322 val loss: 2.844566822052002

    100%|██████████| 1124/1124 [03:26<00:00,  6.07it/s]
      0%|          | 1/1124 [00:00<03:28,  5.38it/s]

    Epoch: 4 trn loss: 2.088127380906475 val loss: 2.8696582317352295

    100%|██████████| 1124/1124 [03:26<00:00,  6.26it/s]
      0%|          | 1/1124 [00:00<03:29,  5.36it/s]

    Epoch: 5 trn loss: 2.093448414603162 val loss: 2.7895729541778564

    100%|██████████| 1124/1124 [03:26<00:00,  6.12it/s]
      0%|          | 1/1124 [00:00<03:29,  5.37it/s]

    Epoch: 6 trn loss: 2.1055155513125383 val loss: 2.9072670936584473

    100%|██████████| 1124/1124 [03:26<00:00,  5.45it/s]
      0%|          | 1/1124 [00:00<03:30,  5.34it/s]

    Epoch: 7 trn loss: 2.11192792569191 val loss: 2.7808303833007812

    100%|██████████| 1124/1124 [03:26<00:00,  6.23it/s]
      0%|          | 1/1124 [00:00<03:29,  5.37it/s]

    Epoch: 8 trn loss: 2.116793169139543 val loss: 2.8389084339141846

    100%|██████████| 1124/1124 [03:26<00:00,  6.13it/s]

    Epoch: 9 trn loss: 2.1336968614325404 val loss: 2.8806753158569336

```
produce_out(trn_dl,seq2seq,int2en,int2j)
```

    mingle your joys sometimes with your earnest occupation .
    ときに君の喜びと君の真剣な職業とを交流せしめよ。
    あなたの仕事をはにはををする。

    a tunnel has been bored through the mountain .
    山を掘り抜いてトンネルが造られた。
    そのはは山ののをた。

    please refrain from smoking .
    どうぞタバコを控えてください。
    タバコをタバコをください。

    i am poor at tennis .
    私はテニスが下手だ。
    私はテニスがテニスが。

    i still owe my brother the ten dollars that he lent me last week .
    先週弟が貸してくれた１０ドル、借りたままだ。
    私は私ののののののののを１週間、ていた。

    there is a rumor that she got married .
    彼女が結婚したといううわさがある。
    彼女は結婚結婚した。

    he tried to appeal .
    彼は訴えようとした。
    彼はうとした。

    day is breaking .
    夜が明けかけてきた。
    日は日日。

    though she looks like his older sister , the fact is that she is his mother .
    彼女は彼の姉のように見えるが、実は母親なのだ。
    彼女は彼の妹に、て、、は、彼の。

    his failure in business left him penniless .
    彼は事業に失敗して一文なしになった。
    彼の失敗は失敗したのは失敗した。

```
torch.save(seq2seq.state_dict(),open('translate_seq2seq_attention.pth','wb'))
```

```
seq2seq.load_state_dict(torch.load('translate_seq2seq_attention.pth', map_location=lambda storage, loc: storage))
```

```
out, atts = seq2seq(x.long().cuda())
```

```
import matplotlib.pyplot as plt
plt.switch_backend('agg')
import matplotlib.ticker as ticker
```

```
torch.stack(atts).size()
```

    torch.Size([25, 120, 25])

```
plt.matshow(torch.stack(atts)[:,5,:].detach().cpu().numpy())
```

```

```


```python
import numpy as np
import scipy.special as sci
import matplotlib.pyplot as plt
from scipy import stats # linregress
import pandas as pd
from IPython.display import Latex
```

## Lecture 10:  Reactive Mass Transport

_(The contents presented in this section were re-developed principally by Dr. P. K. Yadav. The original contents are from Prof. Rudolf Liedl)_

---

The last lecture dealt with the  conservative transport processes and  quantified the mass flow and flux emanating from those processes. The effects of these processes were evaluated as an isolated processes and as joint transport process.

The last lecture dealt with the  conservative transport processes and  quantified the mass flow and flux emanating from those processes. The effects of these processes were evaluated as an isolated processes and as joint transport process.

```{admonition} Last lecture important conclusions
>  $J_{adv}>J_{dis}>>J_{diff}$ is normal aquifers.
> $J_{diff}$ may only be useful as an individual processes in special aquifers, e.g., clayey aquifers.
> In general aquifers, hydrodynamic dispersion $J_{hyd} = J_{dis} + J_{diff}$ is used in the analysis of solute transport process.
```

Finally, the last chapter introduced _Concentration Profile_ $(C-t)$ and _Breakthrough Curve_ $(C-x)$ to visually evaluate solute transport in aquifers using _Concentration_ $(C)$, a process output, as a function of _time_ $(t)$ and _space_ $(x)$.

Finally, the last chapter introduced _Concentration Profile_ $(C-t)$ and _Breakthrough Curve_ $(C-x)$ to visually evaluate solute transport in aquifers using _Concentration_ $(C)$, a process output, as a function of _time_ $(t)$ and _space_ $(x)$.

This section focuses on the _reactive transport processes,_ which as already discussed involves the transport of solute with _reaction_ processes. This course being an introductory groundwater course, _sorption_ and _degradation_ are the only two reaction types introduced and combined with the conservative transport processes- _advection_ and _dispersion_. Eventully, the section evaluates the joint action of conservative transport and reactive processes limiting to 1-D scenario.

The section will, however, first deal with 3-D effects of dispersive process, which is more important to quantify the reactive processes.

## Dispersive Mass Flow in 3-D

In the last section we saw that concentration gradient $\frac{\Delta C}{\Delta L}$ drives dispersive and diffusive solute transport process.

However, in the natural aquifers $\frac{\Delta C}{\Delta L}$ is normally varying with space $(x,y,z)$ and time $(t)$. Therefore, a differential operator $\big(\frac{\mathrm{d}}{\mathrm{d} x}\big)$ is more suitable representation of gradient than the difference operator $\frac{\Delta C}{\Delta x}$. The differential operator also generalizes the gradient case.

Considering the differential operator, the diffusive mass flow and diffusive mass flux (= mass flow per unit area) in 1D is then expressed as:

$$
J_{diff} = - n_e \cdot A \cdot D_p \cdot \frac{\mathrm{d} C}{\mathrm{d}x}
$$

and
$$
j_{diff} = - n_e \cdot D_p \cdot \frac{\mathrm{d} C}{\mathrm{d}x}
$$

Likewise, the dispersive mass flow and dispersive mass flux (1-D) is:

$$
J_{disp, h} = - n_e \cdot A \cdot D_{disp} \cdot \frac{\mathrm{d} C}{\mathrm{d}x}
$$

$$
j_{disp, h} = - n_e \cdot D_{disp} \cdot \frac{\mathrm{d} C}{\mathrm{d}x}
$$

The examples of these relations are presented in last section (@Alex pls. link)

### 3-D concentration Gradient

The concentration gradient  $\frac{\mathrm{d}C}{\mathrm{d}x}$  for 1-D solute transport problems is uni-directional, i.e, direction is fixed, and thus only the magnitude of the gradient is the important factor. However in higher dimensions, 2-D or 3-D, solute transport problems, the _direction_ of gradient along with it's _magnitude_ in that direction has to be specified. Thus, for higher dimension solute transport problems, the _concentration gradient_ becomes **concentration vector**, i.e., a quantity providing both magnitude and direction.

Thus, the representation of concentration gradient in Cartesian coordinate in 2-D and 3-D is:

$$
\mathrm{grad}C = \nabla C=  \begin{pmatrix}
\frac{\partial C}{\partial x}\\
\frac{\partial C}{\partial y}
\end{pmatrix}
$$

and

$$
\mathrm{grad} C = \nabla C=  \begin{pmatrix}
\frac{\partial C}{\partial x}\\
\frac{\partial C}{\partial y}\\
\frac{\partial C}{\partial z}
\end{pmatrix}
$$

The $\nabla$, the inverted Delta symbol, is called the **del** or **nabla** operator. The vector **grad$C$** in the above relations points in the direction of the _steepest increase_ of $C$. However, for the **Hydrogeologists**, the concentration gradients as well the grad$C$ points to the _steepest decrease_ of $C$.

**@Anne/Sophie - can we try to find a simple example finding grad$C$ in 2-D/3-D**

### Isotropic and Anisotropic  Dispersion

Corresponding the expression for the concentration gradient at higher dimensions, the expression for mass flow and flux becomes:

$$
J_{disp,\, h} = \begin{pmatrix}
J_{disp,\, hx}\\
J_{disp,\, hy}\\
J_{disp,\, hz}
\end{pmatrix}
$$
and the 3-D mass flux is:

$$
j_{disp,\, h} = \begin{pmatrix}
j_{disp,\, hx}\\
j_{disp,\, hy}\\
j_{disp,\, hz}
\end{pmatrix}
$$

The subscript in ${{disp, h}}$ refers to _hydrodynamic dispersion_ which is sum of _mechanical dispersion_ and _diffusion_. Likewise, the subscript ${{disp,\, hx}}$, ${{disp,\, hy}}$ and ${{disp,\, hz}}$ refers to dispersion components along the Cartesian coordinates. The corresponding mass flow and mass flux in the higher dimension is then:

$$
J_{disp,\, h} = - n_e \cdot A \cdot D_{hyd} \cdot \text{grad}C
$$

and

###  Isotropic and Anisotropic  Dispersion

Corresponding the expression for the concentration gradient at higher dimensions, the expression for mass flow and flux becomes:

$$
J_{disp,\, h} = \begin{pmatrix}
J_{disp,\, hx}\\
J_{disp,\, hy}\\
J_{disp,\, hz}
\end{pmatrix}
$$
and the 3-D mass flux is:

$$
j_{disp,\, h} = \begin{pmatrix}
j_{disp,\, hx}\\
j_{disp,\, hy}\\
j_{disp,\, hz}
\end{pmatrix}
$$

The subscript in ${{disp,\, h}}$ refers to _hydrodynamic dispersion_ which is sum of _mechanical dispersion_ and _diffusion_. Likewise, the subscript ${{disp,\, hx}}$, ${{disp,\, hy}}$ and ${{disp,\, hz}}$ refers to dispersion components along the Cartesian coordinates. The corresponding mass flow and mass flux in the higher dimension is then:

$$
J_{disp,\, h} = - n_e \cdot A \cdot D_{hyd} \cdot \text{grad}C
$$

and

$$
j_{disp,\, h} = - n_e \cdot A \cdot D_{hyd} \cdot \text{grad}C
$$

The **isotropic dispersion**, rather an _exceptional,_ the $D_{hyd}$ in this case is:

$$
D_{hyd} = \alpha \cdot |v| + n_e \cdot D
$$
where, $D$ is direction independent dispersion coefficient and $\alpha$ $[L]$ is dispersivity, which in:

**heterogeneous aquifer**: $\alpha = \alpha(x,y,z)$ and in

**homogeneous aquifer**: $\alpha = \text{constant}$

For more practical cases and in normal aquifers, the 2-D and 3-D the dispersion for solute transport is _direction dependent,_ i.e. **anisotropic**. Hence the $D_{hyd}$ is not an scalar quantity but a _matrix (tensor),_ which relates the concentration gradient (vector) to the dispersive mass flow (vector). However, if the princopal axes of the dispersion tensor $D_{hyd}$ is made to coincide with the axes of a Cartesian coordinate system _and_ the groundwater flow is considered uniform along the $x-$axis, the dispersive mass flux can be obtained from

$$
\begin{pmatrix} J_x \\ j_y \\ j_z \end{pmatrix} =
\begin{pmatrix} \alpha_L \cdot v_x + n_e \cdot D & 0 & 0 \\
0 & \alpha_{Th} \cdot v_x + n_e \cdot D & 0\\
0 & 0 & \alpha_{Tv} \cdot v_x + n_e \cdot D
\end{pmatrix}
\cdot
\begin{pmatrix} \frac{\partial C}{\partial x} \\ \frac{\partial C}{\partial y} \\ \frac{\partial C}{\partial z} \end{pmatrix}
$$

with $\alpha_L$, $\alpha_{Th}$ and $\alpha_{Tv}$ are longitudinal dispersivity, horizontal transverse dispversity and vertical transverse dispersivity, respectively. The statistical analysis of dispersivity data shows that $\alpha_{L}>\alpha_{Th}>\alpha_{Tv}$ and the values differ by roughly an order of magnitude. This, however, is just a rule of thumb.

@Anne/@Sophie can you find a very simple numerical example for caculating dispersivity or something that explains the above relation.

```python
# Analytical solution from Bear (1976) - Line source, 1st-type input and infinte plane

# Input (values can be changed)
Co = 1 # mg/L, input concentration
Dx = 3 # m, Dispersion in x direction
Dy = Dx/10 # m
v = 0.05 # m/d
Q = 10 # m^3/d

## domain dimension and descritization (values can be changed)
xmin = -100; xmax= 101
ymin = 0.1; ymax = 11
[x, y] = np.meshgrid(np.linspace(xmin, xmax, 1000), np.linspace(ymin, ymax, 100)) # mesh

# Bear (1976) solution Implementation
#"k0: Modified Bessel function of second type and zero Order"

term1 = (Co*Q)/(2*np.pi* np.sqrt(Dx*Dy))
term2 = (x*v)/(2*Dx)
args = (v**2*x**2)/(4*Dx**2) + (v**2*y**2)/(4*Dx*Dy)
sol = term1*np.exp(term2)*sci.k0(args)

# plots
fig, ax = plt.subplots()
CS = ax.contour(x,y,sol, cmap='flag')
ax.clabel(CS, inline=1, fontsize= 10)
CB = fig.colorbar(CS, shrink=0.8, extend='both')
```

## Equilibrium Sorption ##

---

A reactive transport system can include a single reactive process, e.g., degradation, or combination of several multiple reactive processes, e.g. degradation and sorption. The inclusion of the reactive process(es) in the transport studies are site specific. The important to note is that an inclusion of a reactive process increases the complexity of transport problem.

In this course we limit ourselves with the following two types of reaction processes:

**1. Sorption**

**2. Degradation**

Acid-base reaction, precipitation-dissolution reaction, organic combustion etc. are among the reactions type that can be part of the reactive process individually or in any combination.

Also an important distinction is the rate or speed of the reaction. One distinguishes between time-dependent reaction (kinetics) or time-independent reactions (steady-state or equilibrium). Special reaction rates such as instantaneous reaction (extremely fast reaction) can also be part of the reaction process in the transport system.

### Sorption Basics

**Sorption** is a rather a general term used to indicate both **adsorption** and **absorption**. But in this course the _sorption_ refers to only _adsorption._

**Adsorption** can be more formally defined as the process of accumulation of dissolved chemicals on the surface of a solid, e.g., accumulation of a chemicals dissolved in groundwater on the surface of the aquifer material.

The figure below clarifies the _adsorption_ process.

In the figure _chemical in solution_ (the circular objects) more often called **solute** in the water is found to attach is the solid surface. The figure presents the following two important terms part of the adsorption process:

> **Adsorbent**: The solid onto which the chemicals are attached. More formally, _adsorbents_ provide adsorption sites for solutes.

> **Adsorbate**: These are solutes that are attached on the _adsorbent._

Based on the figure, _adsorption_ can be considered as a partition process that divides the chemical originally present in water between adsorbent and water.

Quite often adsorption is a reversible process, i.e., adsorbed chemicals can get back to water phase. This process is called **desorption**.

Speaking about _equilibrium,_ this is reached when

> _adsorption rate_  $\rightleftharpoons $ _desorption rate_

Adsorption in groundwater is often a rapid process. Although sorption kinetics can be important, the description in this introductory level course is limited to equilibrium sorption. Thus, we learn next to quantify equilibrium sorption.

### Adsorption Isotherms

The adsorption process that has reached equilibrium can be relatively easily quantified with the use of empirical models called **isotherms**. These models are often simple algebraic equation that relates solute concentrations partitioned between the adsorbate and adsorbent at constant temperature. More than 15 different _isotherm_ models can be found in the literature. However, in groundwater reactive transport studies the following three are the two most commonly used isotherms:

1. **Henry or Linear isotherm**

2. **Freundlich isotherm**

For quantification, laboratory based experiments are performed using solids from subsurface and chemicals of interest. The laboratory observations are then graphically fitted with empirical isotherm models to quantify adsorption properties. Figure below shows isotherms that are particularly observed in groundwater transport studies. As can be observed in the figure sorption coefficient ($K$) is the common quantities obtained from isotherm models.

### Henry Isotherm

The **Henry isotherm** (Henry, 1803) is based on the idea of a _linear_ relationship between the solute concentration **$C$** and the _adsorbate:adsorbent_ mass ratio $C_a$. Henry isotherm is quite often also called _linear isotherm_ or the $K_d$ model. Mathematically, the Henry isotherm is:

$$
C_a = K_d \cdot C
$$

with

$C$ = solute concentration [ML$^{-3}$]

$C_a$ = mass ratio adsorbate:adsorbent [M:M]

$K_d$ = distribution or partitioning coefficient [L$^3$M$^{-1}$].

Often symbols $C_s$ or $s$ are used instead of $C_a$.

The Henry model has been most widely used in groundwater transport studies. This is largely because of the simplicity (see equation) of the model and it's applicability in representing adsorption process more generally observed in groundwater studies. $K_d$, the partitioning coefficient, is particularly used in groundwater transport studies. It is equal to the slope of the Henry isotherm.

```python
# Example of Henry isotherm (Source: Fetter et al. 2018)

# Following sorption data are available:

C = np.array([7, 15, 174, 249, 362]) # ug/L, Eq. concentration
Ca = np.array([2, 4, 33, 50, 70]) # ug/g, Eq. sorbed mass

# linear- fit y = m*x+c

slope, intercept, r_value, p_value, std_err = stats.linregress(C, Ca)
print("slope: %f    intercept: %f  R-squared: %f" % (slope, intercept, r_value**2))
fit_line = slope*C + intercept

#plot

plt.scatter(C, Ca, label= "Original data") # data plot
plt.plot(C, fit_line, color = "red", label = "fit-line")
plt.legend(); plt.xlabel(r"Equilibrium Aqueous Concentration, $C$ ($\mu$g/L) ")
plt.ylabel(r"Mass sorbed per unit absorbent weight, $C_a$ ($\mu$g/g) ");
plt.text(0, 50, '$C_a=%0.5s C + %0.5s$'%(slope, intercept), fontsize=10)
plt.text(0, 40, '$R^2=%0.5s $'%(r_value**2), fontsize=10)

# Output

Latex("The required partition coefficient = slope, $K_{d}$= %0.5s L/g " % slope)
```

### Freundlich Isotherm

**Freundlich isotherm** (Freundlich, 1907) is a more general isotherm. It is based on the idea of a power law, i.e., includes also the non-linear behaviour, relating the solute concentration $C$ to the adsorbate:adsorbent mass ration $C_a$. The isotherm is mathematically given as

$$
C_a = K_{Fr} \cdot C^N
$$

with

$C$ = solute concentration [ML$^{-3}$]

$C_a$ = mass ratio adsorbate:adsorbent [M:M]

$n$ = Freundlich exponent [-]

$K_{Fr}$ = Freundlich partitioning coefficient [(M:M)/(M/L$^3)^n$].

The Freundlich isotherm equation can be easily linearized by applying logarithmic transformation of the equation, which gives

\begin{eqnarray}
\log C_a = \log K_{Fr} + n\cdot \log C
\end{eqnarray}

The above equation resembles the straight line equation $y = b + a \cdot x b$, in which $b\equiv \log K_{Fr}$ is the intercept and $a\equiv n$ the slope. Thus, from fitting the adsoprtion experimental results with the above equation, both $n$ and $K_{Fr}$ can be obtained.

```python
# Example of Freundlich isotherm

# Following sorption data are available:

Cf= np.array([23.6, 6.67, 3.26, 0.322, 0.169, 0.114]) # mg/L, Eq. concentration
Caf = np.array([737, 450, 318, 121, 85.2, 75.8]) # mg/g, Eq. sorbed mass

logCf = np.log10(Cf) # log10 transformation of data
logCaf = np.log10(Caf)

# fitting: y = mx +c
slope, intercept, r_value, p_value, std_err = stats.linregress(logCf, logCaf)
print("slope: %f    intercept: %f  R-squared: %f" % (slope, intercept, r_value**2))
fit_line = slope*logCf + intercept

# plots
plt.figure(figsize=(10,4))

plt.subplot(121)
plt.plot(Cf, Caf, "*--", label= "Original data")
plt.legend(); plt.xlabel(r"Eq. Aq. Conc., $C$ ($mg$g/L) ");
plt.ylabel(r"Mass sorbed/absorbent weight, $C_a$ ($mg$g/g) ");

plt.subplot(122)
plt.scatter(logCf, logCaf, label="Log transformed data")
plt.plot(logCf, fit_line, color="red", label= "linear fit line")
plt.legend(); plt.xlabel(r"Eq. Aq. Conc., $\log C$ ($mg$g/L) ");
plt.ylabel(r"Mass sorbed/absorbent weight, $\log C_a$ ($mg$/g) ");
plt.text(-1, 2.6, '$C_a=%0.5s C + %0.5s$'%(slope, intercept), fontsize=10)
plt.text(-1, 2.5, '$R^2=%0.5s $'%(r_value**2), fontsize=10)
plt.subplots_adjust(wspace=0.35)

Latex("$K_{Fr}$ = %0.5s (mg/g)$^{1/n}$(mg/L) and   $n$ = %0.4s"  % (10**intercept, slope))
```

### Retardation Factor (for Henry Isotherm)

The net effect of adsorption is the retarded movement of solute in comparison to the average flow of the groundwater. The term **Retardation Factor** $(R)$ is defined that quantifies the retarded movement of solute. The formulation of $R$ is based on the type of isotherm. For Henry isotherm $R$ can be straightforwardly calculated with the help of a mass budget.

For this purpose, an aquifer volume $V$ with the effective porosity $n_e$ is considered (see fig. below)

The steps involved are:

- Total volume: $V$
- Water volume: $n_e \cdot V$
- Mass of dissolved chemical: $n_e \cdot V \cdot C$
- Volume of solid: $(1-n_e)\cdot V$
- Density of solid material: $\rho$
- Mass of solid: $\rho \cdot(1-n_e)\cdot V$
- Mass of adsorbate: $\rho \cdot(1-n_e)\cdot V\cdot C_a$ = $(1-n_e)\cdot\rho \cdot V \cdot K_d \cdot C$
- Total mass: $n_e\cdot V \cdot C + (1-n_e)\cdot\rho \cdot V\cdot K_d \cdot C = n_e \cdot R \cdot V \cdot C$

with _Retardation factor_ $$R = 1 + \frac{1-n_e}{n_e}\cdot \rho \cdot K_d$$

The expression for $R$ can be further modified  by using bulk density $\rho_b$ $= (1-n_e)\cdot \rho$ = mass of solid/total volume. This leads to

$$
R = 1+\frac{\rho_b}{n_e} \cdot K_d
$$

As can be observed from the equation, $R = 1$ when there is no adsorption, i.e., when $K_d= 0$.

@ Anne @ Sophie pls. provide a very short numerical example on R

## Degradation

**Degradation** leads to alteration or transformation of chemical structure of chemicals. This contrasts to adsorption in which chemical structure is not altered. In adsorption (or desorption) the original chemical is partitioned between the solid particles and water. It is _degradation_ that eventually lead to removal of the _original_ chemical from the groundwater. The transformation of original chemical, due to degradation, results to so-called _daughter products (metabolites)._ The new chemical(s) can make groundwater more suitable (decrease contamination) or further contaminate it.

In groundwater studies, degradation can appear as:

- **Radioactive decay**
- **Microbial degradation (bio-degradation)**
- **Chemical degradation**

There are several approaches to quantify degradation process.
A common aspect to most of them is the assumption of _time-dependency_ (or _Kinetics_ ).

### $n^{th}$ - Order Degradation Kinetics

The general equation for the degradation kinetics is:

$$
\frac{\text{d}C}{\text{d} t} = - \lambda \cdot C^n
$$

with $t$ = time [t]

$C$ = solute concentration [ML$^{-3}$]

$n$ = order of the degradation kinetics [ - ] ($n\geq 0)$

$\lambda$ = degradation rate constant [(ML$^{-3})^{(1-n)}$T$^{-1}$].

Considering the initial concentration (or input concentration) $C_0$, the solutions of the kinetics equation are:

$$
C(t) = C_0\cdot e^{-\lambda \cdot t} \: \: \: \text{if }\: n = 1
$$

and

$$
C(t) = [C_0^{1-n} - (1-n)\cdot \lambda t]^{\frac{1}{1-n}} \:\:\: \text{if }\: n\neq 1
$$

The **half life** $(T_{1/2})$, which is the time span elapsing until the initial concentration $C_0$ is reduced by half, is an important time-scale in the degradation analysis. $T_{1/2}$ is $C_0$ dependent in nearly all cases with an exception for 1$^\text{st}$- order degradation kinetics. 0$^{th}$-order and the 1$^\text{st}$- order degradation kinetics are most commonly observed in groundwater studies. The $(T_{1/2})$ of these orders are:

$$
T_{1/2} = \frac{C_0}{2\cdot \lambda} \:\:\: \text{for } \:0^{\text{th}}\text{-order}
$$

$$
T_{1/2} = \frac{\ln 2}{\lambda} \:\:\: \text{for } \:1^{\text{st}}\text{-order}
$$

As can be observed above $T_{1/2}$ is independent of concentration for the 1$^{\text{st}}$-order degradation kinetics.

Another important properties of the degradation kinetics is that for $n\geq 1$ the solute concentration _asymptotically_ approaches zero, whereas for $n<1$, the solute concentration actually reaches zero

```python
# behaviour of degradation kinetics

#input - you may change the values
Co = 1 # mg/L, initial concentration
la = 0.003 # unit is order dependent. For n=1, 1/t

# main equation
Z_order = lambda t: Co* np.exp(-la*t) # for n = 0
F_order = lambda t: Co-la*t # for n = 1

# simulation for t
t = np.linspace(1,1000, 1000) # 1000 time units
Z_results = Z_order(t)
F_results = F_order(t)

# plots
plt.figure(figsize=(10,4))

# n = 1
plt.subplot(121)
plt.plot(t, Z_results)
plt.ylim(0, Co); plt.xlim(0)
plt.text(400, Co*0.8, r"$C(t) = C_0 \cdot e^{-\lambda \cdot t} $", fontsize = 12)
plt.text (400, Co*0.9, r"1$^{st}$-order kinetics", color = "red", fontsize = 12)
plt.text(0, Co/2, r"$T_{1/2}= \frac{\ln 2}{\lambda}$", color= "red", fontsize=14)
plt.xlabel("Time, t  (days)"); plt.ylabel(r"Concentration, $C(t)$ (mg/L)")

# n = 0
plt.subplot(122)
plt.plot(t, F_results)
plt.ylim(0, Co); plt.xlim(0)
plt.text(400, Co*0.8, r"$C(t) = C_0 \cdot-\lambda \cdot t} $", fontsize = 12)
plt.text (400, Co*0.9, r"0$^{th}$-order kinetics", color = "red", fontsize = 12)
plt.text(0, Co/2, r"$T_{1/2}= \frac{C_0}{2\cdot \lambda}$", color= "red", fontsize=14)
plt.xlabel("Time, t  (days)"); plt.ylabel(r"Concentration, $C(t)$ (mg/L)")

plt.subplots_adjust(wspace=0.35)
```

### Radioactive decay

Radioactive decay is degradation of a chemical due to radiation. The radioactive decay is limited to radioactive chemicals such as Cobalt, Cesium, Iodine. This decay obeys the 1$^\text{st}$- order degradation kinetics and therefore the half-life is $T_{1/2} = \frac{\ln 2}{\lambda}$. $T_{1/2}$ is characteristic property of radioactive chemicals and it can be used to compute degradation rate ($\lambda$).

```python
# Example of Radioactive decaly

#experimental results

t = [0, 1, 2, 5, 10, 20, 28 ] # yr, time
Co_60 = [10, 8.76, 7.68, 5.17, 2.68, 0.72, 0.25] # mg/L, Cobalt 60 conc.
So_90 = [10, 9.76, 9.52, 8.84, 7.81, 6.10, 5] # mg/L, Strontium 90 Conc.

z_list = list(zip(t, Co_60, So_90))

Cols= ["time (a)", "Cobalt 60 (mg/L)", "Strontium 90 (mg/L)"]
df = pd.DataFrame(z_list, columns=Cols)
print(df)

# computing
TH_Co60 = 28 # yr, Half life of Cobalt 60
TH_St90 = 5.26 # yr, Half life of Strontium 90
la_Co60 = np.log(2)/TH_Co60 # 1/yr, degradation rate of Cobalt 60
la_St90 = np.log(2)/TH_St90 # 1/yr, degradation rate of Strontium 90

# visualize
plt.plot(t, Co_60, "o--", label = "Cobalt 60")
plt.plot(t, So_90, "v--", label= "Strontium 90")
plt.xlabel("Time (years)"); plt.ylabel("Concentration (mg/L)")
plt.legend();

Latex("The degradation rate ($\lambda$) for Cobalt 60 = %0.5s 1/y and for Strontium 90 = %0.5s 1/y"  % (la_Co60, la_St90))
```

## Joint Action of Conservative and Reactive Transport (1D)

### Concentration Profile

Figure below presents the joint action of conservative transport with equilibrium sorption (linear isotherm) and degradation. The figure shows the solute concentration $C$ (in water) at the same time-levels for various combinations of acting processes.

The figure can be explained in the following way:

(A): The solute is initially present at constant concentration in a limited area.

(B): Solute spreads only due to advection. Due to absence of dispersion there is no (1D) spreading effect.

(C): Inclusion of dispersion process causes spread of concentration. As retardation is absence the front centreline remains unchanged

(D): The inclusion of retardation ($R$) with advection and dispersion leads to removal of chemicals from water and as well the retarded movement of the chemical front.

(E): The inclusion of retardation along with degradation and conservative transport process leads to high removal of chemical from water.

### Breakthrough Curve

Breakthrough curves provide a _time-dependent_ spread of chemicals in the groundwater. The inclusion of multiple processes are normally solved using numerical models. Analytical models are available for limited processes and simplified problems. A 1-D analytical solution by Kinzelbach (1987) provide a transient (time-dependent) solution of reactive transport problem with inclusion of equilibrium linear sorption represented by retardation $(R)$, first-order degradation rate $(\lambda)$ and the conservative transport quantities - dispersion $(D)$ and advection. The solution is given as:

$$
C(x,t) = C_0 \cdot \exp(-\lambda\cdot t)\bigg(1- \frac{1}{2}\text{erfc}\bigg(\frac{R\cdot x - v\cdot t}{2\cdot\sqrt{D\cdot R \cdot t}}\bigg) - \frac{1}{2}\exp\bigg(\frac{v\cdot x}{D}\bigg)\text{erfc}\bigg(\frac{R\cdot x + v\cdot t}{2\cdot\sqrt{D\cdot R \cdot t}}\bigg)
$$

with $C_0$ = input/source concentration [ML$^{-3}$]

$t$ = time [T]

$v$ = groundwater flow velocity [LT$^{-1}$]

erfc() = represents the complementary error function [See here for details](https://en.wikipedia.org/wiki/Error_function). erfc() can be easily computed using Python Scipy special function library.

```python
# Breakthrough curve using Kinzelbach (1987) analytical solution Main function

# The main function - you may change the value of C_o, lam, R, Dx, v, x
# C_o = input concentration, mg/L
# lam = 0 # 1/d, degradation rate, 1/d
# R = retardation factor, ()
# Dx = dispersion coeff. along x, m^2/d
# v = groundwater velocity, m/d
# x = position where C is to be measured, m

def Cx(t, C_o= 1, lam = 0, R=1, Dx=1, v= 10, x = 20):
    sterm = C_o*np.exp(-lam*t)
    erf_ag1 = (R*x-v*t)/(2*np.sqrt(Dx*R*t))
    erf_ag2 = (R*x+v*t)/(2*np.sqrt(Dx*R*t))

    C = sterm*(1-(0.5*sci.erfc(erf_ag1)-0.5*np.exp((v*x)/Dx)*sci.erfc(erf_ag2)))
    return C

```

```python
# Computing Case 1: Conservative process- R = 1, Lambda = 0
t1 = np.linspace(1e-5,50,1000) # times, d
C1 = Cx(t1, C_o= 1, lam = 0, R=1, Dx=1, v= 1, x = 20)

# Computing Case 2: Conservative system + Retardation - R = 2, Lambda = 0
t2 = np.linspace(1e-5,50,1000) # times, d
C2 = Cx(t2, C_o= 1, lam = 0, R=2, Dx=1, v= 1, x = 20)

# Computing Case 3: Conservative system + Retardation + degradation - R = 2, Lambda = 0.004
t3 = np.linspace(1e-5,50,1000) # times, d
C3 = Cx(t3, C_o= 1, lam = 0.004, R=2, Dx=1, v= 1, x = 20)

# plots - this should be adjusted as required

plt.figure(figsize=(9, 6))

plt.plot(t1, C1, label="Conservative transport")
plt.plot(t2, C2, label = "Reactive transport with sorption")
plt.plot(t3, C3, label = "Reactive transport with sorption and degradation rate")
plt.legend(loc= 3); plt.xlim(0), plt.ylim(0)
plt.xlabel("time (d)"); plt.ylabel(r"Concentration, $C$ (mg/L)")
plt.text(5, 0.2, r"$x= 20$ m")
```

### Mass (Re-)Distribution During Injection / Extraction

**Consider a scenario**:

Water is _injected_ into a certain portion of an aquifer with total volume $V$, bulk density $\rho_b$ and effective porosity $n_e$. Assume that the injected water contains a chemical of total mass $M$, which is adsorbed by the aquifer materials under equilibrium conditions according to Henry isotherm (quantified by $K_d$$).

Bases on the assumption of sorption equilibrium, the total mass $M$ of the chemical is instantaneously(!) split up into a dissolved and a sorbed part. In such case, the mass distribution can be computed as follows (with $R$ = retardation factor, $\rho_b$
= bulk density):

\begin{align}
M &= n_e \cdot V \cdot C + V\cdot\rho_b \cdot C_a \\
&= n_e \cdot V \cdot C + V \cdot \rho_b \cdot K_d \cdot C\\
&= n_e \cdot (1 + \rho_b \cdot K_d/n_e) \cdot V \cdot C\\
&= n_e \cdot R \cdot V \cdot C
\end{align}

In which,

$n_e \cdot V \cdot C$ = dissolved mass

$V\cdot\rho_b \cdot C_a$ = mass of adsorbate

For the dissolved mass we thus have $n_e \cdot V \cdot C = M/R$ and consequently the mass of adsorbate is:

$V\cdot\rho_b \cdot C_a = M- M/R = (1-1/R)\cdot M$

The same approach can be adopted for the **extraction** scenarios, i.e. equilibrium desorption.