 * Asignatura: Laboratorio Métodos Computacionales en Ciencias
 * Nombre: María Camila Casas Díaz
 * Código: 201813057

```python
import numpy as np
import scipy as sp
import seaborn as sns; sns.set()

from sympy.vector import CoordSys3D, gradient, divergence
from sympy import Function, Symbol, pi, Eq, latex
from matplotlib import pyplot as plt

%matplotlib inline
```

# Tarea 5 y 6: Ecuación de difusión en 3 dimensiones

Uds se preguntarán, ¿por qué vamos a resolver la ecuación de difusión? La respuesta no es muy obvia, pero es interesante: Los procesos de difusión comparten algo muy bonito con los procesos estocásticos. Para ello, vamos a analizar un problema de COVID: el tiempo medio estimado de infección de los viriones.

La transmisión de COVID se da preponderamente debido a la aspersión de aerosoles en el aire. Estas partículas *semi*-esféricas -cuyo orden de magnitud es de $\sim1-10\,\mu m$ de radio- están compuestas principalmente por agua, lípidos orgánicos y viriones (se pueden considerar *quasi*-puntuales ya que son del orden de los $\sim100\,nm$). Cuando una particula del aerosol entra y se adhiere al tracto respiratorio, el virus toma un tiempo en entrar en contacto con las células para infectar el huésped debido al movimiento Browniano. Los viriones, a diferencia de las bacterias, no cuentan con cilios, flagelos u otros mecanismos para desplazarse en los medios, por lo cual, su única esperanza es que por fluctuaciones térmicas puedan llegar a la superficie de la gota de aerosol para replicar su ADN al entrar en contacto con los tejidos susceptibles. Este proceso es en esencia estocástico y se puede modelar mediante la ecuación de Difusión. Esta ecuación tiene dos partes. La idea es que uds resuelvan el problema de la manera más sencilla. La ecuación es la siguiente,
$$
\frac{\partial\Psi}{\partial t}=D\nabla^2\Psi,
$$
donde $D$ es la constante de difusión del medio y $\Psi$ es la concentración de partículas. La taza de difusión depende de la temperatura y la viscosidad del medio y se puede modelar usando la relación de Einstein-Stokes,
$$
D=\frac{k_BT}{6\pi\eta a},
$$
siendo $k_B$ la constante de Boltzmann, $T$ la temperatura en grados Kelvin, $\eta$ la viscosidad del medio y $a$ el radio de los viriones. En esencia, lo que la ecuación de difusión me está diciendo es que la concentración media de viriones depende de la posición y el tiempo. No obstante, para poder calcular el tiempo que tardaría un virión en alcanzar la superficie se puede modelar en la media usando la siguiente ecuación de difusión,
$$
-\nabla^2\tau=\frac{1}{D},
$$
donde $\tau$ es el tiempo medio que dependería de la posición en la que se encuentra inicialmente.

## 1. Escriba la ecuación de difusión para el tiempo $\tau$ en coordenadas esféricas y asuma que $\tau(r,\theta,\phi)\simeq\tau(r)$ ya que por simetría esférica sólo dependerá de la posición radial respecto al centro de la gotica (Usar **LaTex**)

```python
# Cambio a coordenadas esféricas
Spherical = CoordSys3D("S",transformation='spherical')

# Establecemos las variables
kB = Symbol("k_{B}")
T = Symbol("T")
eta = Symbol("eta")
a = Symbol("a")
r = Symbol("r")
R = Symbol("R")

# Establecemos la constante de difusión del medio D
D = (kB*T)/(6*pi*eta*a)

# Establecemos el Volumen V
V = 4/3*pi*R**3

# Establecemos Tau
tau = Function("tau")

eDifusion = Eq(divergence(gradient(tau(Spherical.r))).simplify(),1/D)
eDifusion
```

$\displaystyle \frac{d^{2}}{d \mathbf{{r}_{S}}^{2}} \tau{\left(\mathbf{{r}_{S}} \right)} + \frac{2 \frac{d}{d \mathbf{{r}_{S}}} \tau{\left(\mathbf{{r}_{S}} \right)}}{\mathbf{{r}_{S}}} = \frac{6 \pi a \eta}{T k_{B}}$

```python
# Imprimimos en Latex utilizando Sympy
print(latex(eDifusion))
```

    \frac{d^{2}}{d \mathbf{{r}_{S}}^{2}} \tau{\left(\mathbf{{r}_{S}} \right)} + \frac{2 \frac{d}{d \mathbf{{r}_{S}}} \tau{\left(\mathbf{{r}_{S}} \right)}}{\mathbf{{r}_{S}}} = \frac{6 \pi a \eta}{T k_{B}}

## 2. Resuelva la ecuación diferencial para el tiempo de forma numérica y grafique

Asuma las siguientes condiciones iniciales:
1. $\tau(R)=0$ ya que si el virión está en la superficie el tiempo debe ser naturalmente nulo.
1. $\tau^\prime(r)=0$ ya que por simetría la derivada radial debe ser nula en el origen

Suponga las siguientes condiciones:
- $R=5\mu m$ para el radio de la esfera de *quasi* agua (calcula el volumen $V$)
- $\eta_{\text{H}_2\text{O}}\simeq1\times10^{-3}\,Pa\cdot s$ (Pascales por segundo)
- $\frac{\eta}{\eta_{\text{H}_2\text{O}}}\approx10^3\to10^5$
- $a\simeq100\,nm$
- $V=\frac{4}{3}\pi a^3$
- $k_BT\simeq4.05\times10^{-21}J$

```python
# Escribimos las condiciones:
R = 5e-6
eta_h2o = 1e-3
a = 1e-7
kBT = 4.05e-21

# Calculamos el Volumen V
V = (4/3)*np.pi*(R**3)

# Calculamos la constante de difusión del medio D
D = kBT/(6*np.pi*eta_h2o*a)

# Calculamos la funcion
def fun(r,tau_v):
    tau,dtau = tau_v
    return np.array([dtau,-2*dtau/r-1/D])

# Calculamos la funcion
def bc(tau_R,tau_r):
    return np.array([tau_R[0], tau_r[0]])

x = np.linspace(1e-12,R)
y = np.zeros((2,x.size))

# solver
sol = sp.integrate.solve_bvp(fun,bc,x,y)

# limites

xp = sol.x
yp = sol.sol(xp)[0]

# Graficar
plt.plot(xp,yp)
plt.xlabel("R(m)")
plt.ylabel("$\\tau$(R)")

```

## 3. Si los viriones están distribuidos uniformemente, encuentre el tiempo que tardaría un virión en salir de la gota de aerosol.

Tenga presente que debe promediar suponiendo que el virión tiene una distribución uniforme, i.e. $\rho\left(\vec{r}\right)=1/V$, usando la siguiente relación,
$$
\bar{\tau} = \int_{\mathcal{V}}\tau\left(\vec{r}\right)\rho\left(\vec{r}\right)\,\text{d}\vec{r} = \frac{4\pi}{V}\int_{0}^{R}\tau(r)\,r^2\text{d}r.
$$
Realice la integral numéricamente.

```python
def tau(r):
    return (4*np.pi/V) * sol.sol(r)[0]*r**2

print("La integral numérica es:",sp.integrate.quad(tau,0,R))
```

    La integral numérica es: (0.7757016957937449, 8.612018829109946e-15)

## 4. Las cadenas de Markov.

Vamos a resolver el problema anterior usando un proceso de Markov. Suponga que ud **divide** la esfera en cubitos de ancho $\delta x=\delta y=\delta z=\Delta=R/N$ con $N$ un número determinado de particiones. Para nuestro experimento, vamos a suponer que ponemos un virión en una posición inicial $\vec{r}_0=(\Delta\,j, 0, 0)$, determinada por un índice $j\in\{0,1,2,\dots,N\}$. Ud va a actualizar la posición del virión en la malla discreta siguiendo las reglas a continuación:
- Determine el número de divisiones $N$ y calcule $\Delta$.

```python
N = 20 # Divisiones
dx = R/N
```

- Ajuste la escala de tiempo $\delta t$ tal que la probabilidad $\alpha=D\frac{\delta t}{\Delta^2}<\frac{1}{6}$. (Recomendación $\leq1/12$)

```python
δt = (1/13)*(dx**2)/D # Ajustamos la escala de tiempo

alpha = D*(δt/(dx**2)) # Probabilidad α

assert alpha < 1/12 # Utilizamos la recomendación
```

- Haga una corrida de Markov-Monte Carlo actualizando la posición con la probabilidad de transición $\alpha$ hacia los primeros vecinos cercanos y calcule el número de pasos de tiempo para llegar a la superficie, i.e. $|\vec{r}(t_m)|>R-\Delta$

```python
T = kBT
x = np.arange(0,R,dx)

dt = 1e-4
t = np.arange(0,T,dt)

```

- Repita este experimento para la misma posición un gran número de veces para obtener una estadística (media y desviación estándar).

```python

```

- Repita todos los pasos para todos los índices $j\in\{0,1,2,\dots,N\}$ y grafique. ¡Compare con los resultados anteriores!

## 5. Diseñe un experimento para calcular el numeral (3) usando Markov-Monte Carlo

```python

```


Table of Contents

1   Joint Distribution and Independence

1.1  An Example

1.2   Independence

1.3   Moment Generating Functions

1.3.1   How It Works?

1.3.2   Example 1 - Moments of Bernoulli

1.3.3   Example 2 - Moments of Binomial

Here we go through the methods for distributions which have more than two variables. Most of results are similar to us already.

#  Joint Distribution and Independence

Let $X_1, X_2,...X_n$ be discrete random variables, the PMF is defined as

$$
P_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)=P\left(X_{1}=x_{1}, X_{2}=x_{2}, \ldots, X_{n}=x_{n}\right)
$$

The marginal PMF of $x_1$ is

$$
P_{X_1}=P(X=x_1)
= \sum_{x_2\in R}\sum_{x_3\in R}...\sum_{x_n\in R}P\left(X_{1}=x_{1}, X_{2}=x_{2}, \ldots, X_{n}=x_{n}\right)
$$

If $X_1, X_2,...X_n$ are continuous random variables, the PDF is defined as

$$
P\left(\left(X_{1}, X_{2}, \cdots, X_{n}\right) \in A\right)=\int_A \int_{A} \cdots \int_A f_{X_{1} X_{2} \cdots X_{n}}\left(x_{1}, x_{2}, \cdots, x_{n}\right) d x_{1} d x_{2} \cdots d x_{n}
$$

The marginal PDF of $x_1$ is

$$
f_{X_{1}}\left(x_{1}\right)=\int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} f_{X_{1} X_{2} \cdots X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right) d x_{2} \cdots d x_{n}
$$

The joint CDF is

$$
F_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)=P\left(X_{1} \leq x_{1}, X_{2} \leq x_{2}, \ldots, X_{n} \leq x_{n}\right)
$$

## An Example

Let $X, Y$ and $Z$ be three jointly continuous random variables with joint PDF

$$
f_{X Y Z}(x, y, z)=\left\{\begin{array}{ll}
cx+2 y+3 z & 0 \leq x, y, z \leq 1 \\
0 & \text { otherwise }
\end{array}\right.
$$

1. Find out $c$.

Use the property we get

\begin{align}
\int_0^1\int_0^1\int_0^1(cx+2y+3z)dxdydz &= 1\\
\int_0^1\int_0^1\left[\frac{1}{2}cx^2+2xy+3xz\right]_0^1dydz&=1\\
\int_0^1\int_0^1\left(\frac{1}{2}+2y+3z\right)dydz&=1\\
\int_0^1\left[\frac{cy}{2}+y^2+3zy\right]^1_0dz&=1\\
\left[\frac{cz}{2}+z+\frac{3z^2}{2}\right]^1_0&=1\\
c&=-3
\end{align}

2. Find the marginal PDF of $X$.

\begin{align}
f_X(x)=\int_{-\infty}^\infty\int_{-\infty}^\infty f_{XYZ}(x,y,z)dydz & = \int_{0}^1\int_{0}^1 (-3x+2 y+3 z )dydz \\
& = \int_0^1 \left[-3xy+y^2+3yz\right]_0^1dz =\int_0^1(-3x+1+3z)dz\\
& = \left[-3xz+z+3\frac{z^2}{2}\right]^1_0 = -3x+\frac{5}{2}
\end{align}

##  Independence

If $X_1, X_2,...X_n$ are independent discrete random variables

$$
P_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)=P_{X_{1}}\left(x_{1}\right) P_{X_{2}}\left(x_{2}\right) \cdots P_{X_{n}}\left(x_{n}\right)
$$

And if $X_1, X_2,...X_n$ are independent continuous random variables
$$
f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)=f_{X_{1}}\left(x_{1}\right) f_{X_{2}}\left(x_{2}\right) \cdots f_{X_{n}}\left(x_{n}\right)
$$

Their CDF and expections are

$$
F_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n}\right)=F_{X_{1}}\left(x_{1}\right) F_{X_{2}}\left(x_{2}\right) \cdots F_{X_{n}}\left(x_{n}\right)
$$

$$
E\left[X_{1} X_{2} \cdots X_{n}\right]=E\left[X_{1}\right] E\left[X_{2}\right] \cdots E\left[X_{n}\right]
$$

One common setting for error term of regression model is independent identically distributed.

\begin{aligned}
E\left[X_{1} X_{2} \cdots X_{n}\right] &=E\left[X_{1}\right] E\left[X_{2}\right] \cdots E\left[X_{n}\right] \\
&=E\left[X_{1}\right] E\left[X_{1}\right] \cdots E\left[X_{1}\right] \\
&=E\left[X_{1}\right]^{n}=E\left[X_{2}\right]^{n}=\cdots = E\left[X_{n}\right]^{n}
\end{aligned}

##  Moment Generating Functions

Without formal definition, we call $E[X], E[X^2], E[X^k]$ the first moment, the second moment and the $k$th moment.

Another important concept is central moment, for instance $E[(X - E[X])^k]$ is called the $k$th moment.

The moment generating function (MGF) is a convenient way of generating all moments that we need.

If $X$ is discrete random variable, the MGF is

$$
M_X(t) = E[e^{tX}] = \sum_0^\infty e^{tk}P_X(k)
$$

For continuous random variable $X$, the MGF is

$$
M_X(t) = E[e^{tX}]  = \int_{-\infty}^\infty e^{tX} f_X(x) dx
$$

We obtain moments by taking derivatives of MGFs

$$
E(X^n) = \frac{d^nM_x(t)}{dt^n}\bigg|_{t=0}
$$

###  How It Works?

Perform Taylor expansion on $e^x$

$$
 e^{x}=1+x+\frac{x^{2}}{2 !}+\frac{x^{3}}{3 !}+\ldots=\sum_{k=0}^{\infty} \frac{x^{k}}{k !}
$$

Suppose $X$ is a discrete random variable, therefore $e^{tX}$ is

$$
e^{tX} = 1+tX+\frac{(tX)^{2}}{2 !}+\frac{(tX)^{3}}{3 !}+\ldots=\sum_{k=0}^{\infty} \frac{(tX)^{k}}{k !}
$$

Take expectation, we obtain MGF

$$
E[e^{tX}] =1+tE[X]+\frac{t^2E[X^{2}]}{2 !}+\frac{t^3E[X^{3}]}{3 !}+\ldots= \sum_{k=0}^{\infty} \frac{t^kE[X^{k}]}{k !}
$$

Take the first order derivative and evaluate at $t=0$

$$
\frac{d E[e^{tX}]}{dt}= 0 + E[X] + tE[X^2] + \frac{t^2E[X^3]}{2!}+\cdots = 0+ E[X]+ 0 + 0 +\cdots = E[X]
$$

Take the second order derivative and evaluate at $t=0$

$$
\frac{d^2 E[e^{tX}]}{dt^2} = 0 + 0 + E[X^2]+ tE[X^3]+ \frac{t^2E[X^4]}{2!}+\cdots=E[X^2]
$$

Take the third order derivative and evaluate at $t=0$

$$
\frac{d^3 E[e^{tX}]}{dt^3} = 0+0+0+E[X^3]+tE[X^4] = E[X^3]
$$

###  Example 1 - Moments of Bernoulli

Let's calculate the MGF of $X\sim Ber(p)$, we know $P_X(1) = p, P_X(0)= 1-p$.

$$
 E[e^{tX}] = e^{0t}P_X(0)+e^{1t}P_X(1) = 1-p+pe^t
$$

If we want to calculate the variance with formula $E(X^2)-(E[X])^2$, we need to generate the first and second moment.

$$
E[X]=\frac{d E[e^{tX}]}{dt} \bigg|_{t=0}= p\\
E[X^2]=\frac{d^2 E[e^{tX}]}{dt^2} \bigg|_{t=0}= p
$$

So the variance is $E(X^2)-(E[X])^2 = p - p^2$.

###  Example 2 - Moments of Binomial

If $X_1, X_2, \ldots ,X_n \stackrel{i.i.d.}{\sim} Ber(p)$,

$$
\sum^n_{i=1}X_i\ \sim\ B(n,p)
$$

where $B(n,p)$ denotes binomial distribution.

The goal is to find the first moment $E(\sum^n_{i=1}X_i)$ and the second central moment $\text{Var}(\sum^n_{i=1}X_i)$.

Using MGF definition

\begin{align}
E[e^{t\sum^n_{i=1}X_i}] &= E[e^{tX_1}e^{tX_2}\ldots e^{tX_n}]
= E[e^{tX_1}] E[e^{tX_2}]\ldots E[e^{tX_n}]  \tag{$X_i$'s are i.i.d.}\\
& = (1-p+pe^t)(1-p+pe^t)\ldots (1-p+pe^t)\\
& = (1-p+pe^t)^n
\end{align}

The first moment is

$$
E\bigg(\sum^n_{i=1}X_i\bigg) = \frac{d}{dt}(1-p+pe^t)^n\bigg|_{t=0}=n(1-p+pe^t)^{n-1}pe^t\Big|_{t=0}=np
$$

The second moment requires product rule of differentiation

\begin{align}
E\bigg[\bigg(\sum^n_{i=1}X_i\bigg)^2 \bigg]=\frac{d}{dt}n(1-p+pe^t)^{n-1}pe^t
&= n(n-1)(1-p+pe^t)^{n-2}pe^tpe^t+n(1-p+pe^t)^{n-1}pe^t\Big|_{t=0}\\
&=n(n-1)p^2+np=n^2p^2-np^2+np
\end{align}

Thereby

$$
\text{Var}\left(\sum^n_{i=1}X_i\right)= n^2p^2-np^2+np - n^2p^2 = np - np^2
$$

Chapter 6 - Multiple Random Variables.ipynb


# Introduction to Decision Theory using Probabilistic Graphical Models

> So far, we have seen that probabilistic graphical models are useful for modeling situations that involve uncertainty. Furthermore, we will see in the next module how using inference algorithms we will also reach conclusions abount the current situation from partial evidence: predictions.
>
> On the other hand, we do not only want to obtain these conclusions (predictions), but actually make decisions on top of these conclusions.
>
> It turns out that we can actually use probabilistic graphical models to encode not only the uncertain situations, but also the decision making agents with all the policies they are allowed to implement and the possible utilities that one may obtain.

> **Objetives:**
> - To learn how to represent decision situations using PGMs.
> - To understand the maximum expected utility principle.
> - To learn how to measure the value of information when making a decision.

> **References:**
> - Probabilistic Graphical Models: Principles and Techniques, By Daphne Koller and Nir Friedman. Ch. 22 - 23.
> - Probabilistic Graphical Models Specialization, offered through Coursera. Prof. Daphne Koller.

 Imagen recuperada de: https://upload.wikimedia.org/wikipedia/commons/b/bb/Risk_aversion_curve.jpg.

___

# 1. Maximizing Expected Utility

The theoretical foundations of decision theory were established long befor probabilistic graphical models came to live. The framework of *maximum expected utility* allows to formulate and solve decision problems that involve uncertainty.

Before continuing, we should be clear that the **utility** is a numerical function that assigns numbers to the various possible outcomes, encoding the preferences of the agent. These numbers:

- Do not have meanings in themselves.
- We only know that the larger, the better, according to the preferences of the agent.
- Usually, we compare the utility of two outcomes by means of the $\Delta U$, which represents the strength of the "happiness" change from an outcome w.r.t. the other.

The outcomes we were talking about above can vary along *multiple dimensions*. One of those dimensions is often the monetary gain, but most of the settings consider other dimensions as well.

## 1.1. Problem formulation and maximum expected utility principle

A simple **decision making situation** $\mathcal{D}$ is defined by:

- A set of possible actions $A$, with $\mathrm{Val}(A)=\{a_1, \dots, a_k\}$.
- A set of possible states (RVs) $\bar{X}$, with $\mathrm{Val}(\bar{X})=\{\bar{x}_1, \dots, \bar{x}_n\}$.
- A conditional distribution $P(\bar{X}|A)$.
- A utility function $U(\bar{X}, A)$, which expresses the agent's preferences.

The **expected utility** on the above decision making situation, given that $A=a$,  is

$$EU[\mathcal{D}[a]] = \sum_{\bar{X}}P(\bar{X}|a)U(\bar{X},a).$$

Furthermore, the **maximum expected utility (MEU)** principle states that we should choose the action that maximizes the expected utility

$$a^\ast = \arg\max_{a\in\mathrm{Val}(A)} EU[\mathcal{D}[a]].$$

**How can we represent the above using PGMs?**

We can use the ideas we developed for PGMs to represent the decision making situations in a very interpretable way.

In this sense, we have:
- Random variables are represented by *ovals* and stand for the state.
- Actions are represented by *rectangles*.
- Utilities are represented by *diamonds*. These have no children.

**Example.** Consider the decision situation $\mathcal{D}$ where a graduate of the Master's Degree in Data Science is deciding whether to found a Data Science consultancy company or not.

While this person does not exactly know which will be the demand of consultancy services, he/she knows that the demand will be either:
- $m^0$: nonexistent, with probability $0.5$;
- $m^1$: moderate, with probability $0.3$;
- $m^2$: high, with probability $0.2$.

Moreover, he/she will obtain a utility $U(M, f^0)=0$ for $M=m^0, m^1, m^2$ in the case that he/she doesn't found the company, or the following utilities in the case that he/she found the company:
- $U(m^0, f^1)=-7$;
- $U(m^1, f^1)=5$;
- $U(m^2, f^1)=20$.

Let's represent the graphical model corresponding to this situation:

```python
from IPython.display import Image
```

```python
# First draw in the white board, then show (first_representation)
Image("figures/first_representation.png")
```

Then, according to this:

- What are the expected utilities for each action?
  - $E[\mathcal{D}[f^0]]=0$
  - $E[\mathcal{D}[f^1]]=0.5 \times -7 + 0.3 \times 5 + 0.2 \times 20=2.$

- Which is the optimal action?
  - $f^1 = \arg \max_{f=f^0, f^1} E[\mathcal{D}[f]]$.

With `pgmpy`:

```python
# Import pgmpy.factors.discrete.DiscreteFactor

```

```python
# Define factors P(M), U(M,F)

```

```python

```

    +------+----------+
    | M    |   phi(M) |
    +======+==========+
    | M(0) |   0.5000 |
    +------+----------+
    | M(1) |   0.3000 |
    +------+----------+
    | M(2) |   0.2000 |
    +------+----------+

```python

```

    +------+------+------------+
    | M    | F    |   phi(M,F) |
    +======+======+============+
    | M(0) | F(0) |     0.0000 |
    +------+------+------------+
    | M(0) | F(1) |    -7.0000 |
    +------+------+------------+
    | M(1) | F(0) |     0.0000 |
    +------+------+------------+
    | M(1) | F(1) |     5.0000 |
    +------+------+------------+
    | M(2) | F(0) |     0.0000 |
    +------+------+------------+
    | M(2) | F(1) |    20.0000 |
    +------+------+------------+

```python
# Find Expected Utility

```

    +------+----------+
    | F    |   phi(F) |
    +======+==========+
    | F(0) |   0.0000 |
    +------+----------+
    | F(1) |   2.0000 |
    +------+----------+

```python

```

    2.0

**Multiple utility nodes**

In the above example, we only had one utility node. However, we may include as many utility nodes as wanted to reduce the number of parameters:

```python
Image("figures/student_utility.png")
```

Where:
- $V_G$: Happiness with the grade itself.
- $V_Q$: Quality of life during studies.
- $V_S$: Value of getting a good job.

The total utility can be formulated as:

$$U=V_G+V_Q+V_S.$$

*Question.* If $|\mathrm{Val}(D)| = 2$, $|\mathrm{Val}(S)| = 2$, $|\mathrm{Val}(G)| = 3$, and $|\mathrm{Val}(J)| = 2$, how many parameters do you need to completely specify the utility?

- $|\mathrm{Val}(V_G)| = 3$; $|\mathrm{Val}(V_Q)| = 4$; $|\mathrm{Val}(V_S)| = 2$. We need $3+4+2=9$ parameters.

*Question.* How many if the utility weren't decomposed?

- $3\times 2 \times 2 \times 2 = 24$.

## 1.2. Information edges and decision rules

The influence diagrams we have depicted above, also allow us to capture the notion of information available to the agent when they make their decision.

**Example.** In the example of the Master's graduate deciding founding or not founding his company, let's assume that he/she has the opportunity to carry out a survey to measure the overall market demand for Data Science consultancy before making the decision.

In this sense, the graph now looks like the following:

```python
# First draw in the white board, then show (second_representation)
Image("figures/second_representation.png")
```

Hence, the agent can make its decision depending on the value of the survey, which is denoted by the precense of the edge.

Formally,

> *Definition.* A **decision rule** $\delta_A$ at an action node $A$ is a conditional probability $P(A|\mathrm{Pa}A)$ (a function that maps each instatiation of $\mathrm{Pa}A$  to a probability distribution $\delta_A$ over $\mathrm{Val}(A)$).

Given the above, **what is the expected utility with information?**

Following the same sort of ideas we get that

$$EU[\mathcal{D}[\delta_A]] = \sum_{\bar{X}, A}P_{\delta_A}(\bar{X},A)U(\bar{X},A),$$

where $P_{\delta_A}(\bar{X},A)$ is the joint probability distribution over $\bar{X}$ and $A$. The subindex $\delta_A$ makes reference that this joind distribution depends on the selection of the decision rule $\delta_A$.

Now, following the MEU, the optimal decision rule is:

$$\delta_A^\ast = \arg \max_{\delta_A} EU[\mathcal{D}[\delta_A]],$$

and the MEU is

$$MEU(\mathcal{D}) = \max_{\delta_A} EU[\mathcal{D}[\delta_A]].$$

**How can we find optimal decision rules?**

In our entrepreneur example, we have that

\begin{align}
EU[\mathcal{D}[\delta_A]] &= \sum_{\bar{X}, A}P_{\delta_A}(\bar{X},A)U(\bar{X},A) \\
& = \sum_{M,S,F} P(M)P(S|M) \delta_F(F|S) U(M,F)\\
& = \sum_{S,F} \delta_F(F|S) \sum_M P(M)P(S|M)U(M,F)\\
& = \sum_{S,F} \delta_F(F|S) \mu(S,F)
\end{align}

(see in the whiteboard, then show equations)

Thus, let's calculate $\mu(S,F)$ using `pgmpy`:

```python
# We already have P(M), and U(F,M). Define P(S|M)

```

```python
# Compute mu(F,S)

```

```python
# Print mu(F,S)

```

    +------+------+------------+
    | S    | F    |   phi(S,F) |
    +======+======+============+
    | S(0) | F(0) |     0.0000 |
    +------+------+------------+
    | S(0) | F(1) |    -1.2500 |
    +------+------+------------+
    | S(1) | F(0) |     0.0000 |
    +------+------+------------+
    | S(1) | F(1) |     1.1500 |
    +------+------+------------+
    | S(2) | F(0) |     0.0000 |
    +------+------+------------+
    | S(2) | F(1) |     2.1000 |
    +------+------+------------+

Following the MEU principle, we should select for each state (the Survey, in this case) the action that maximizes $\mu$.

In this case:

```python
Image("figures/table.png")
```

Finally,

$$MEU[\mathcal{D}] = \sum_{S,F} \delta_F^\ast(F|S) \mu(S,F) = 0 + 1.15 + 2.1 = 3.25$$

```python
# Define optimal decision rule

```

```python

```

    2.0

```python
# Obtain MEU

```

    3.2500000000000004

Without this observation the MEU was 2, and now the MEU has increased more than 50%.

Nice, huh?

___
# 2. Utility functions

## 2.1. Utility of money
We have used utility functions in all the first section assuming that they were known. In this section we study these functions in more detail.

**Utility functions** are a necessary tool that enables us to compare complex scenarios that involve uncertainty or risk.

The first thing that we should understand is that utility is not the same as expected payoff.

**Example.** An investor must decide between a participation in company A, where he would earn $\$3$ million without risk, and a participation in company B, where he would earn $\$4$ million with probability 0.8 and $\$0$ with probability 0.2.

```python
Image("figures/utility_first.png")
```

Which one do you prefer?
  - The risk-free one.

What is the expected payof of the company A?
  - $\$3$ M.

What is the expected payoff of the company B?
  - $0.8 \times \$4$ M + $0.2 \times \$0$ M = $\$3.2$ M.

**Example.**

Another common example that reflects this fact is the well-known St. Petersburg Paradox:

- A fair coin is tossed repeatedly until it comes up Heads.
- Each toss that it doesn't come up Heads, the payoff is doubled.
- In this sense, if the coin comes up Heads in the $n$-th toss, then the payoff will be $\$2^n$.

How much are you willing to pay to enter to this game?
- 20, 10, 1, 5.

What happens is that if we compute the expected payoff it is:

- $P(\text{comes up Heads in } n-\text{th toss}) = \frac{1}{2^n}$

$$E[\text{Payoff}] = \sum_{n=1}^{\infty}P(\text{comes up Heads in } n-\text{th toss}) \text{Payoff}(n) = \sum_{n=1}^{\infty} \frac{1}{2^n} 2^n = \sum_{n=1}^{\infty} 1 = \infty.$$

With these two examples, we have shown that almost always people do not always choose to maximize their monetary gain. What this implies is that the utility of money is not the money itself.

In fact, at this point of the history and after several psychological studies, we know that utility functions of money for most people look like

$$U(W) = \alpha + \beta \log(W + \gamma),$$

which is a nice *concave* function.

**How does this function look like?** (see in the whiteboard).

**How does the actual form of the curve have to be the attitude towards risk?**

## 2.2. Utility of multiple attributes

All the attributes affecting the utility must be integrated into one utility function.

This may be a difficult task, since we can enter into some complex fields, far beyond math, probability, and graphs.

- For instance, how do we compare human life with money?
  - A low cost airline is considering the decision of not to run mainteinance plans over the aircraft at every arrival.
  - If you have car, you don't change the tires that often (every 3 months).

There have been several attempts to addres this problem:
  - Micromorts: $\frac{1}{10^6}$ chance of death.
  - [QALY](https://en.wikipedia.org/wiki/Quality-adjusted_life_year).

# 3. Value of perfect information

We used influence diagrams to make decisions given a set of observations.

Another type of question that may arise is **which observations sould I make before making a decision?**.

- Initially one may think that the more information, the better (because information is power).
- But the answer to this question is far from being that simple.

For instance:

- In our entrepreneur example, we saw that including the information of the survey increased the MEU significatively. However, we did not take into account the costs of performing that survey. What if the cost of performing that survey makes the money gains of the company negative or too little?

- Medical diagnosis relies on tests. Some of these tests are painful, risky and/or very expensive.

A notion that allows us to adress this question is the **Value of Perfect Information**.

- The value of perfect information $\mathrm{VPI}(A|\bar{X})$ is the value (in utility units) of observing $\bar{X}$ before choosing an action at the node $A$.

- If $\mathcal{D}$ is the original influence diagram, and

- $\mathcal{D}_{\bar{X}\to A}$ is the influence diagram with the edge(s) $\bar{X}\to A$,

- then

  $$\mathrm{VPI}(A|\bar{X}) = MEU(\mathcal{D}_{\bar{X}\to A}) - MEU(\mathcal{D}).$$

In the entrepreneur example,

$$\mathrm{VPI}(F|S) = MEU(\mathcal{D}_{S\to F}) - MEU(\mathcal{D})=3.25 - 2 = 1.25.$$

> *Theorem.* The value of perfect information satisfies:
>
> (i) $\mathrm{VPI}(A|\bar{X})\geq 0$.
>
> (ii) $\mathrm{VPI}(A|\bar{X})= 0$ if and only if the optimal decision rule for $\mathcal{D}$ is also optimal for  $\mathcal{D}_{\bar{X}\to A}$.

This theorem practically says that the information is valuable if and only if it changes the agent's decision at least in one case.

**Example.** Consider the case that you are interested in two job offers in two different companies. Furthermore, these two companies are startups and both are looking for funding, which highly depends on the organizational quality of the company.

This situation can be modeled as:

```python
Image("figures/vpi.png")
```

Let's find the MEU using `pgmpy`:

```python
# Define factors

```

```python
# Obtain Expected utility

```

```python
# Print EU

```

    +------+----------+
    | D    |   phi(D) |
    +======+==========+
    | D(0) |   0.7200 |
    +------+----------+
    | D(1) |   0.3300 |
    +------+----------+

```python
# Obtain MEU(D)

```

    0.72

Now, let's say that a friend of yours already works in the Company 2, so he informs you about the organizational status of that company. What is the value of that information?

```python
Image("figures/vpi2.png")
```

```python
# Obtain the factor mu(D, C2)

```

```python
# Print

```

    +-------+------+-------------+
    | C2    | D    |   phi(C2,D) |
    +=======+======+=============+
    | C2(0) | D(0) |      0.2880 |
    +-------+------+-------------+
    | C2(0) | D(1) |      0.0400 |
    +-------+------+-------------+
    | C2(1) | D(0) |      0.3600 |
    +-------+------+-------------+
    | C2(1) | D(1) |      0.2000 |
    +-------+------+-------------+
    | C2(2) | D(0) |      0.0720 |
    +-------+------+-------------+
    | C2(2) | D(1) |      0.0900 |
    +-------+------+-------------+

```python
# Select optimal decision

```

```python
# Obtain MEU(D_C2->D)

```

    0.738

```python
# Obtain VPI

```

    0.018000000000000016

# Announcements

## Exam of module 1.

Created with Jupyter by Esteban Jiménez Rodríguez.


# Section 4.4 $\quad$ Span

## Review of Linear Combination

If $\mathbf{v}_1$ and $\mathbf{v}_2$ are two vectors in a vector space $V$,

## Definition of Span

>If $S = \{\mathbf{v}_1, \mathbf{v}_2, \cdots, \mathbf{v}_k\}$ is a set of vectors in a vector space $V$,

### Example 1

Let $S$ be the set of $2\times 2$ matrices given by
\begin{equation*}
  S =
  \left\{
  \left[
    \begin{array}{cc}
      1 & 0 \\
      0 & 0 \\
    \end{array}
  \right],~
  \left[
    \begin{array}{cc}
      0 & 0 \\
      0 & 1 \\
    \end{array}
  \right]
  \right\}
\end{equation*}
Find span $S$

```python
from sympy import *

a, b = symbols('a b');

A = Matrix([[1, 0], [0, 0]]);
B = Matrix([[0, 0], [0, 1]]);

a*A + b*B
```

    Matrix([
    [a, 0],
    [0, b]])

>**Theorem** Let $S = \{\mathbf{v}_1, \mathbf{v}_2, \cdots, \mathbf{v}_k\}$ be a set of vectors in a vector space $V$. Then

### Example 2

Let $S=\{t^2,t\}$ be a subset of the vector space $\mathcal{P}_2$ (polynomials of degree no more than 2).

>**Definition** Let $S$ be a set of vectors in a vector space $V$. If every vector in $V$ is a linear combination of the vectors in $S$,

**Remark** If span $S = V$, $S$ is called

### Example 3

In $\mathbb{R}^3$, let $\mathbf{v}_1 = \left[\begin{array}{c}2 \\ 1 \\ 1 \end{array}\right]$, $\mathbf{v}_2 = \left[\begin{array}{c}1 \\ -1 \\ 3 \end{array}\right]$
Determine whether the $\mathbf{v} = \left[\begin{array}{c}1 \\ 5 \\ -7 \end{array}\right]$vector belongs to span $\{\mathbf{v}_1,\mathbf{v}_2\}$.

```python
from sympy import *

x, y = symbols('x y');

Eq1 = 2*x + y - 1;
Eq2 = x - y - 5;
Eq3 = x + 3*y + 7;

solve([Eq1, Eq2, Eq3], (x, y))
```

    {x: 2, y: -3}

### Example 4

In $\mathcal{P}_2$, let
\begin{equation*}
  \mathbf{v}_1 = 2t^2+t+2, \quad\mathbf{v}_2 = t^2-2t, \quad\mathbf{v}_3 = 5t^2-5t+2, \quad\mathbf{v}_4 = -t^2-3t-2
\end{equation*}
Determine whether the vector $\mathbf{v} = t^2+t+2$ belongs to span $\{\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4\}$

```python
from sympy import *

a, b, c, d = symbols('a b c d');

Eq1 = 2*a + b + 5*c - d - 1;
Eq2 = a - 2*b - 5*c - 3*d - 1;
Eq3 = 2*a + 2*c - 2*d - 2;

solve([Eq1, Eq2, Eq3], (a, b, c, d))
```

    []

### Example 5

Let $V$ be the vector space $\mathbb{R}^3$. Let
\begin{equation*}
  \mathbf{v}_1 = \left[\begin{array}{c}1 \\ 2 \\ 1 \end{array}\right],~~
  \mathbf{v}_2 = \left[\begin{array}{c}1 \\ 0 \\ 2 \end{array}\right],~~
  \mathbf{v}_3 = \left[\begin{array}{c}1 \\ 1 \\ 0 \end{array}\right].
\end{equation*}
Determine whether the vectors $\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$ spans $V$.

```python
from sympy import *

A = Matrix([[1, 1, 1], [2, 0, 1], [1, 2, 0]]);

A.rank()
```

    3

### Example 6

Consider the homogeneous linear system $A\mathbf{x}=\mathbf{0}$ where
\begin{equation*}
   A =
   \left[
     \begin{array}{cccc}
       1 & 1 & 0 & 2 \\
       -2 & -2 & 1 & -5 \\
       1 & 1 & -1 & 3 \\
       4 & 4 & -1 & 9 \\
     \end{array}
   \right]
\end{equation*}


```python
%load_ext autoreload
%autoreload 2
%matplotlib inline
```

```python
import numpy as np
import matplotlib.pyplot as plt
from hottbox.core import Tensor
```

[Return to Table of Contents](./0_Table_of_contents.ipynb)

# Tensors and basic operations

A tensor is a multi-dimenaional array of data where each dimension is conventionally referred to as the **mode**. Tensor order is defined by the number of its modes which is equivivalent to the number of indices required to identify a particular entry of a multi-dimensional array. For ease of visualisation and compact notation, we consider a third order tensor $\mathbf{\underline{X}} \in \mathbb{R}^{I \times J \times K}$. Thus, an element of such a tensor, $\mathbf{\underline{X}} \in \mathbb{R}^{I \times J \times K}$, can be written in a general form as:

$$ x_{ijk} = \mathbf{\underline{X}}[i, j, k]$$

> **Note:** In order to be consistent with Python indexing, count of modes and elements within starts from zeros.

Creation of a tensor starts with a formation of a multidimensional array of data.

```python
# Create 3-d array of data
array_3d = np.arange(24).reshape((2, 3, 4))

# Create tensor
tensor = Tensor(array_3d)

# Result preview
print(tensor)
tensor.data
```

    This tensor is of order 3 and consists of 24 elements.
    Sizes and names of its modes are (2, 3, 4) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[[ 0,  1,  2,  3],
            [ 4,  5,  6,  7],
            [ 8,  9, 10, 11]],

           [[12, 13, 14, 15],
            [16, 17, 18, 19],
            [20, 21, 22, 23]]])

## **Assigment 1**

1. Create a tensor of order 5 (of any shape) and with all its values equal to 5. Then, calculate its Frobenious norm.
2. Construct several tensors of orders varying from 3 to 20, with the size of each dimension equal to 2. Plot the relation between an order of a tensor and the number of its elements.
3. Construct several tensors of order 3 with the size of each dimension varying from 3 to 15. Plot the relation between the size of dimensions of a tensor and the number of its elements.
4. Analyse and compare the so obtained graphs.

### Solution: Part 1

```python
# Create 3-d array of data
array_5d = 5*np.ones(720).reshape((2, 3, 4, 5, 6))

# Create tensor
tensor1 = Tensor(array_5d)

# Result preview
print(tensor1)
tensor1.data
```

    This tensor is of order 5 and consists of 720 elements.
    Sizes and names of its modes are (2, 3, 4, 5, 6) and ['mode-0', 'mode-1', 'mode-2', 'mode-3', 'mode-4'] respectively.

    array([[[[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]],

            [[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]],

            [[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]]],

           [[[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]],

            [[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]],

            [[[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]],

             [[5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.],
              [5., 5., 5., 5., 5., 5.]]]]])

```python
# Calculate Frobenious norm
print('Frobenious norm: {:.3f}' .format(tensor1.frob_norm))
```

    Frobenious norm: 134.164

### Solution: Part 2

```python
# Initialization
num_list = []
tensor_list = []
# Dimension
dim = 2
# Initial reshape size
reshape_size = (dim,dim,dim)
# Order
order = range(3,21)
# Create tensors and collect information
for iOrder in order:
    array = np.ones(dim**iOrder).reshape(reshape_size)
    reshape_size += (dim,)
    tensor2 = Tensor(array)
    tensor_list.append(tensor2)
    num_list.append(tensor2.size)
# Instance for order 5
print(tensor_list[2])
tensor_list[2].data
```

    This tensor is of order 5 and consists of 32 elements.
    Sizes and names of its modes are (2, 2, 2, 2, 2) and ['mode-0', 'mode-1', 'mode-2', 'mode-3', 'mode-4'] respectively.

    array([[[[[1., 1.],
              [1., 1.]],

             [[1., 1.],
              [1., 1.]]],

            [[[1., 1.],
              [1., 1.]],

             [[1., 1.],
              [1., 1.]]]],

           [[[[1., 1.],
              [1., 1.]],

             [[1., 1.],
              [1., 1.]]],

            [[[1., 1.],
              [1., 1.]],

             [[1., 1.],
              [1., 1.]]]]])

```python
# Plot results
plt.plot(order,num_list)
plt.xlabel('Tensor order')
plt.ylabel('Number of elements')
plt.legend(['Dimension size = '+str(dim)])
plt.title('Relationship between tensor order and element number')
plt.grid(True)
plt.show()
```

### Solution: Part 3

```python
# Initialization
num_list = []
tensor_list = []
# Dimension
dim = range(3,16)
# Order
order = 3
# Create tensors and collect information
for iDim in dim:
    array = np.ones(iDim**order).reshape(iDim,iDim,iDim)
    tensor3 = Tensor(array)
    tensor_list.append(tensor3)
    num_list.append(tensor3.size)
# Instance for dimenson size 5
print(tensor_list[2])
tensor_list[2].data
```

    This tensor is of order 3 and consists of 125 elements.
    Sizes and names of its modes are (5, 5, 5) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[[1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.]],

           [[1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.]],

           [[1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.]],

           [[1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.]],

           [[1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.],
            [1., 1., 1., 1., 1.]]])

```python
# Plot results
plt.plot(dim,num_list)
plt.xlabel('Dimension size')
plt.ylabel('Number of elements')
plt.legend(['Order = '+str(order)])
plt.title('Relationship between tensor dimension size and element number')
plt.grid(True)
plt.show()
```

### Solution: Part 4

According to the result shown in part 2, by varing the order ($M$) from 3 to 20 and keeping the dimension size as a constant 2, the number of elements versus order is an exponential curve as $N = 2^{M}$. Moreover, as shown in part 3, if we vary the tensor dimension size ($D$) from 3 to 15 and keep the order as 3, the number of elements versus dimension is a cubic curve as $N = D^{3}$. Generally, the number of elements is equal to $D^{M}$, therefore, if $D$ is constant, then $N$ is an exponential function of $M$, and if $M$ is constant, then $N$ is a function of $D$ with power $M$. The number of elements is propotional to the memory and computation complexity required for operating tensors, thus a large number of elements will lead to inefficiency and high cost.

# Most common tensor substructures

1. A **fiber** is a vector obtained by fixing all but one of the indices, e.g.  $\mathbf{\underline{X}}[i,:,k]$ is the mode-2 fiber $\mathbf{\underline{X}}$ (usually refered to as row fiber).

1. Fixing all but two of the indices yields a matrix called a **slice** of a tensor, e.g. $\mathbf{\underline{X}}[:,:,k]$ is the mode-[1,2] slice (usually refered to as the frontal slice).

**Note:** The same principles and definitions can be applied to a tensor of an arbitrarily large order. In addition, you can obtain a **subtensor** by fixing at least three indices and letting the remaining ones vary.

```python
# Mode-2 fiber
tensor.data[0,1,:]
```

    array([4, 5, 6, 7])

```python
# Frontal slice
tensor.data[0,:,:]
```

    array([[ 0,  1,  2,  3],
           [ 4,  5,  6,  7],
           [ 8,  9, 10, 11]])

## **Assigment 2**

1. Create a tensor of order 4 which consists of 81 elements and display all its horizontal slices and any five row fibers.
2. Create a tensor of order 3 and shape `(4,2,5)`, where the values of each consequent frontal slice are doubled.

### Solution: Part 1

```python
# Create a tensor of order 4
array_4d = np.arange(81).reshape(3,3,3,3)
tensor_4d = Tensor(array_4d)
```

```python
# Display slices
for index_i in range(0, tensor_4d.shape[0]):
    for index_j in range(0,tensor_4d.shape[3]):
        print('Horizontal slice')
        print(tensor_4d.data[index_i,:,:,index_j])
        print('---------------------')
```

    Horizontal slice
    [[ 0  3  6]
     [ 9 12 15]
     [18 21 24]]
    ---------------------
    Horizontal slice
    [[ 1  4  7]
     [10 13 16]
     [19 22 25]]
    ---------------------
    Horizontal slice
    [[ 2  5  8]
     [11 14 17]
     [20 23 26]]
    ---------------------
    Horizontal slice
    [[27 30 33]
     [36 39 42]
     [45 48 51]]
    ---------------------
    Horizontal slice
    [[28 31 34]
     [37 40 43]
     [46 49 52]]
    ---------------------
    Horizontal slice
    [[29 32 35]
     [38 41 44]
     [47 50 53]]
    ---------------------
    Horizontal slice
    [[54 57 60]
     [63 66 69]
     [72 75 78]]
    ---------------------
    Horizontal slice
    [[55 58 61]
     [64 67 70]
     [73 76 79]]
    ---------------------
    Horizontal slice
    [[56 59 62]
     [65 68 71]
     [74 77 80]]
    ---------------------

```python
# Display fibers
count = 0
target = 5
for index_i in range(0, tensor_4d.shape[0]):
    for index_j in range(0, tensor_4d.shape[2]):
        for index_k in range(0, tensor_4d.shape[3]):
            count += 1
            print('Row fiber %d' %(count))
            print(tensor_4d.data[index_i,:,index_j,index_k])
            print('---------------------')
            if count == target:
                break;
        if count == target:
            break;
    if count == target:
        break;
```

    Row fiber 1
    [ 0  9 18]
    ---------------------
    Row fiber 2
    [ 1 10 19]
    ---------------------
    Row fiber 3
    [ 2 11 20]
    ---------------------
    Row fiber 4
    [ 3 12 21]
    ---------------------
    Row fiber 5
    [ 4 13 22]
    ---------------------

### Solution: Part 2

```python
# Create a tensor of order 3
# Create a tensor of order 3 and shape (4,2,5), where the values of each consequent frontal slice are doubled.
array_3d = np.ones(4*2*5).reshape((4,2,5))
tensor_3d = Tensor(array_3d)

for index_f in range(0, tensor_3d.shape[2]-1):
    tensor_3d.data[:,:,index_f+1] = tensor_3d.data[:,:,index_f]*2
print(tensor_3d)
tensor_3d.data
```

    This tensor is of order 3 and consists of 40 elements.
    Sizes and names of its modes are (4, 2, 5) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[[ 1.,  2.,  4.,  8., 16.],
            [ 1.,  2.,  4.,  8., 16.]],

           [[ 1.,  2.,  4.,  8., 16.],
            [ 1.,  2.,  4.,  8., 16.]],

           [[ 1.,  2.,  4.,  8., 16.],
            [ 1.,  2.,  4.,  8., 16.]],

           [[ 1.,  2.,  4.,  8., 16.],
            [ 1.,  2.,  4.,  8., 16.]]])

# Unfolding

Conventionally, unfolding is considered to be a process of element mapping from a tensor to a matrix. In other words, it arranges the mode-$n$ fibers of a tensor to be the matrix columns. Thus, the mode-$n$ unfolding is denoted as:

$$\mathbf{\underline{A}} \xrightarrow{n} \mathbf{A}_{(n)}$$

Thus, this operations requires to specify a mode along which a tensor will be unfolded. For a third order tensor, a visual representation of such operation is as following

In `hottbox` this functionality is available through the corresponding methods of the `Tensor` object, for instance:

```python
# Create 3-d array of data
array_3d = np.arange(24).reshape((2, 3, 4))

# Create tensor
tensor = Tensor(array_3d)

# Unfolding of a tensor
tensor.unfold(mode=0)

# Result preview
print(tensor)
tensor.data
```

    This tensor is of order 2 and consists of 24 elements.
    Sizes and names of its modes are (2, 12) and ['mode-0', 'mode-1_mode-2'] respectively.

    array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11],
           [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])

## **Assigment 3**

1. Create a tensor, $\mathbf{\underline{A}}$, of order 4, in such a way that its version unfolded along the second mode would have sizes `(5,72)`, that is $\mathbf{A}_{(n)} \in  \mathbb{R}^{5 \times 72}$. Validate your results by displaying the unfolded tensor.

### Solution : Part 1

```python
# Create a tensor
array_4d = np.arange(360).reshape((3,4,5,6))
tensor_4d = Tensor(array_4d)
print(tensor_4d)
tensor_4d.data
```

    This tensor is of order 4 and consists of 360 elements.
    Sizes and names of its modes are (3, 4, 5, 6) and ['mode-0', 'mode-1', 'mode-2', 'mode-3'] respectively.

    array([[[[  0,   1,   2,   3,   4,   5],
             [  6,   7,   8,   9,  10,  11],
             [ 12,  13,  14,  15,  16,  17],
             [ 18,  19,  20,  21,  22,  23],
             [ 24,  25,  26,  27,  28,  29]],

            [[ 30,  31,  32,  33,  34,  35],
             [ 36,  37,  38,  39,  40,  41],
             [ 42,  43,  44,  45,  46,  47],
             [ 48,  49,  50,  51,  52,  53],
             [ 54,  55,  56,  57,  58,  59]],

            [[ 60,  61,  62,  63,  64,  65],
             [ 66,  67,  68,  69,  70,  71],
             [ 72,  73,  74,  75,  76,  77],
             [ 78,  79,  80,  81,  82,  83],
             [ 84,  85,  86,  87,  88,  89]],

            [[ 90,  91,  92,  93,  94,  95],
             [ 96,  97,  98,  99, 100, 101],
             [102, 103, 104, 105, 106, 107],
             [108, 109, 110, 111, 112, 113],
             [114, 115, 116, 117, 118, 119]]],

           [[[120, 121, 122, 123, 124, 125],
             [126, 127, 128, 129, 130, 131],
             [132, 133, 134, 135, 136, 137],
             [138, 139, 140, 141, 142, 143],
             [144, 145, 146, 147, 148, 149]],

            [[150, 151, 152, 153, 154, 155],
             [156, 157, 158, 159, 160, 161],
             [162, 163, 164, 165, 166, 167],
             [168, 169, 170, 171, 172, 173],
             [174, 175, 176, 177, 178, 179]],

            [[180, 181, 182, 183, 184, 185],
             [186, 187, 188, 189, 190, 191],
             [192, 193, 194, 195, 196, 197],
             [198, 199, 200, 201, 202, 203],
             [204, 205, 206, 207, 208, 209]],

            [[210, 211, 212, 213, 214, 215],
             [216, 217, 218, 219, 220, 221],
             [222, 223, 224, 225, 226, 227],
             [228, 229, 230, 231, 232, 233],
             [234, 235, 236, 237, 238, 239]]],

           [[[240, 241, 242, 243, 244, 245],
             [246, 247, 248, 249, 250, 251],
             [252, 253, 254, 255, 256, 257],
             [258, 259, 260, 261, 262, 263],
             [264, 265, 266, 267, 268, 269]],

            [[270, 271, 272, 273, 274, 275],
             [276, 277, 278, 279, 280, 281],
             [282, 283, 284, 285, 286, 287],
             [288, 289, 290, 291, 292, 293],
             [294, 295, 296, 297, 298, 299]],

            [[300, 301, 302, 303, 304, 305],
             [306, 307, 308, 309, 310, 311],
             [312, 313, 314, 315, 316, 317],
             [318, 319, 320, 321, 322, 323],
             [324, 325, 326, 327, 328, 329]],

            [[330, 331, 332, 333, 334, 335],
             [336, 337, 338, 339, 340, 341],
             [342, 343, 344, 345, 346, 347],
             [348, 349, 350, 351, 352, 353],
             [354, 355, 356, 357, 358, 359]]]])

```python
# Display unfolded version
# Unfolding of a tensor
tensor_4d.unfold(mode=1)

# Result preview
print(tensor)
tensor.data
```

    This tensor is of order 2 and consists of 24 elements.
    Sizes and names of its modes are (2, 12) and ['mode-0', 'mode-1_mode-2'] respectively.

    array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11],
           [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])

# Mode-n product

The mode-$n$ product is a multiplication of a tensor  by a matrix along the $n^{th}$ mode of a tensor. This essentially means that each mode-$n$ fiber should be multiplied by this matrix. Mathematically, this can be expressed as:

$$\mathbf{\underline{X}} \times_n \mathbf{A} = \mathbf{\underline{Y}} \quad \Leftrightarrow  \quad \mathbf{Y}_{(n)} = \mathbf{A} \mathbf{X}_{(n)}  $$

```python
I, J, K = 2, 3, 4
J_new = 5

# Create tensor
array_3d = np.arange(I * J * K).reshape(I, J ,K)
X = Tensor(array_3d)

# Create matrix
A = np.arange(J_new * J).reshape(J_new, J)

# Performs Mode-n product
Y = X.mode_n_product(A, mode=1, inplace=False)

# Preview of resulting tensor
print(Y)

# Preview of its unfolded version
Y.unfold(mode=1, inplace=False).data
```

    This tensor is of order 3 and consists of 40 elements.
    Sizes and names of its modes are (2, 5, 4) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[ 20,  23,  26,  29,  56,  59,  62,  65],
           [ 56,  68,  80,  92, 200, 212, 224, 236],
           [ 92, 113, 134, 155, 344, 365, 386, 407],
           [128, 158, 188, 218, 488, 518, 548, 578],
           [164, 203, 242, 281, 632, 671, 710, 749]])

## **Assignment 4**

1. Perform the mode-$n$ product of a 3-rd order tensor, $\mathbf{\underline{X}}$, with a matrix, $\mathbf{A} \in  \mathbb{R}^{5 \times 4}$, provided that the resulting tensor, $\mathbf{\underline{Y}}$, consists of `100` elements. Validate your results.

> **Note:** You are encouraged to experiment with the optional parameter `inplace` and to investigate its effect on the result.

### Solution: Part 1

```python
# Create original tensor
I, J, K = 4,4,5

array_3d = np.arange(I*J*K).reshape((I, J, K))
tensor_3d = Tensor(array_3d)
```

```python
# Create matrix
A = np.arange(5*4).reshape((5,4))
```

```python
# Perform mode-n product
Y = tensor_3d.mode_n_product(A, mode=1, inplace=False)
# Preview of resulting tensor
print(Y)

# Preview of its unfolded version
Y.unfold(mode=1, inplace=False).data
```

    This tensor is of order 3 and consists of 100 elements.
    Sizes and names of its modes are (4, 5, 5) and ['mode-0', 'mode-1', 'mode-2'] respectively.

    array([[  70,   76,   82,   88,   94,  190,  196,  202,  208,  214,  310,
             316,  322,  328,  334,  430,  436,  442,  448,  454],
           [ 190,  212,  234,  256,  278,  630,  652,  674,  696,  718, 1070,
            1092, 1114, 1136, 1158, 1510, 1532, 1554, 1576, 1598],
           [ 310,  348,  386,  424,  462, 1070, 1108, 1146, 1184, 1222, 1830,
            1868, 1906, 1944, 1982, 2590, 2628, 2666, 2704, 2742],
           [ 430,  484,  538,  592,  646, 1510, 1564, 1618, 1672, 1726, 2590,
            2644, 2698, 2752, 2806, 3670, 3724, 3778, 3832, 3886],
           [ 550,  620,  690,  760,  830, 1950, 2020, 2090, 2160, 2230, 3350,
            3420, 3490, 3560, 3630, 4750, 4820, 4890, 4960, 5030]])

```python
# Validate equivalence between the left and right hand side
# Left hand side: mode 2 unfolding of Y
LHS = Y.unfold(mode=1, inplace=False).data
# Right hand side: mode 2 unfolding of AX
RHS = np.matmul(A, tensor_3d.unfold(mode=1, inplace=False).data)
# Check if left side and right side equal
check = np.array_equal(LHS,RHS)
print(check)
```

    True

# Properties of mode-n product

1. For distinct modes in a series of multiplications, the order of the multiplication is irrelevent:

    $$\mathbf{\underline{X}} \times_n \mathbf{A} \times_m \mathbf{B} = \mathbf{\underline{X}} \times_m \mathbf{B} \times_n \mathbf{A} \quad (m \neq n)$$

1. However, this does not hold if the modes are the same :

    $$\mathbf{\underline{X}} \times_n \mathbf{A} \times_n \mathbf{B} = \mathbf{\underline{X}} \times_n (\mathbf{B}\mathbf{A})$$

## **Assignment 5**

1. Validate both of the above properties empirically
2. Prove the second property analytically

> **Note:** Be aware of consequences of the optional parameter `inplace` (its default value is set to `True`).

### Solution: Part 1

```python
# Create tensor
I, J, K = 3,4,5

array_3d = np.arange(I*J*K).reshape((I,J,K))
tensor_3d = Tensor(array_3d)
```

```python
# Create matrices
JA = 5
JB = 4

# Matrix A
A = np.arange(JA*J).reshape((JA,J))
# Matrix B
B = np.arange(JB*K).reshape((JB,K))
```

```python
# Validate property 1 𝐗×𝑛𝐀×𝑚𝐁=𝐗×𝑚𝐁×𝑛𝐀
# left hand side
LHS = tensor_3d.mode_n_product(A, mode=1, inplace=False).mode_n_product(B, mode=2, inplace=False)
# Right hand side
RHS = tensor_3d.mode_n_product(B, mode=2, inplace=False).mode_n_product(A, mode=1, inplace=False)
# Check if left side and right side equal
check = np.array_equal(LHS,RHS)
print(check)
```

    True

```python
# Validate property 2 𝐗×𝑛𝐀×𝑛𝐁=𝐗×𝑛(𝐁𝐀)
# left hand side
LHS = tensor_3d.mode_n_product(A, mode=1, inplace=False).mode_n_product(B, mode=1, inplace=False)
# Right hand side
RHS = tensor_3d.mode_n_product(np.matmul(A,B), mode=2, inplace=False)
# Check if left side and right side equal
check = np.array_equal(LHS,RHS)
print(check)
```

    False

### Solution: Part 2

Use [LaTeX syntax](https://stackoverflow.com/questions/13208286/how-to-write-latex-in-ipython-notebook) for analytical deriviations for the second property below.

```latex
%%latex
Define the original $N^{th}$ order tensor as $\mathbf{X} \in \mathbb{R}^{I_{1} \times I_{2} \times \dots \times I_{N}}$, the matrix $\mathbf{A} \in \mathbb{R}^{J_{n} \times I_{n}}$ and $\mathbf{B} \in \mathbb{R}^{K_{n} \times J_{n}}$.
The output of the mode-n product, for example, $\mathbf{Y} = \mathbf{X} \times_{n} \mathbf{A}$, is defined as the inner product of mode-n fiber $\mathbf{X}$ and $\mathbf{A}$.
Elementwise, it is given by,
\begin{equation}
    \mathbf{Y}(i_{1}, \dots , i_{n-1} , j_{n} , i_{n+1} , \dots , i_{N}) =
    \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}a_{j_{n}i_{n}}
\end{equation}
where $x$ is the element in $\mathbf{X}$.
Then for the left hand side of the second property, we get:
\begin{equation}
\textbf{LHS} = \sum_{j_{n}=1}^{J_{n}} \left( \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}a_{j_{n}i_{n}}\right)b_{k_{n}j_{n}}
\end{equation}
For the right side, fisrt calculating $\mathbf{B}\mathbf{A}$ gives:
\begin{equation}
    \mathbf{B}\mathbf{A} = \sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}} a_{j_{n}i_{n}}
\end{equation}
Therefore, the right hand side is equal to:
\begin{equation}
    \textbf{RHS} = \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}\left(\sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}} a_{j_{n}i_{n}}\right)
\end{equation}
Changing the order of summations gives:
\begin{align}
    \textbf{RHS} &= \sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}}\left(\sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}} a_{j_{n}i_{n}}\right)\\
    &= \sum_{j_{n}=1}^{J_{n}}\left(\sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}} a_{j_{n}i_{n}}\right)b_{k_{n}j_{n}}\\
    &= \textbf{LHS}
\end{align}
Therefore, the second property holds.
```

Define the original $N^{th}$ order tensor as $\mathbf{X} \in \mathbb{R}^{I_{1} \times I_{2} \times \dots \times I_{N}}$, the matrix $\mathbf{A} \in \mathbb{R}^{J_{n} \times I_{n}}$ and $\mathbf{B} \in \mathbb{R}^{K_{n} \times J_{n}}$.
The output of the mode-n product, for example, $\mathbf{Y} = \mathbf{X} \times_{n} \mathbf{A}$, is defined as the inner product of mode-n fiber $\mathbf{X}$ and $\mathbf{A}$.
Elementwise, it is given by,
\begin{equation}
    \mathbf{Y}(i_{1}, \dots , i_{n-1} , j_{n} , i_{n+1} , \dots , i_{N}) =
    \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}a_{j_{n}i_{n}}
\end{equation}
where $x$ is the element in $\mathbf{X}$.
Then for the left hand side of the second property, we get:
\begin{equation}
\textbf{LHS} = \sum_{j_{n}=1}^{J_{n}} \left( \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}a_{j_{n}i_{n}}\right)b_{k_{n}j_{n}}
\end{equation}
For the right side, fisrt calculating $\mathbf{B}\mathbf{A}$ gives:
\begin{equation}
    \mathbf{B}\mathbf{A} = \sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}} a_{j_{n}i_{n}}
\end{equation}
Therefore, the right hand side is equal to:
\begin{equation}
    \textbf{RHS} = \sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}}\left(\sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}} a_{j_{n}i_{n}}\right)
\end{equation}
Changing the order of summations gives:
\begin{align}
    \textbf{RHS} &= \sum_{j_{n}=1}^{J_{n}} b_{k_{n}j_{n}}\left(\sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}} a_{j_{n}i_{n}}\right)\\
    &= \sum_{j_{n}=1}^{J_{n}}\left(\sum_{i_{n}=1}^{I_{n}} x_{i_{1}, i_{2}, \dots, i_{N}} a_{j_{n}i_{n}}\right)b_{k_{n}j_{n}}\\
    &= \textbf{LHS}
\end{align}
Therefore, the second property holds.


# **Introduction**

---

## **TA Instructor**
Name: Simeon Nedelchev

Feel free to contact me in person (office 105) or via [telegram](https://t.me/simkasimka) and [mail](https://t.me/simkasimka) if you face any problems with the course or would like to do research and work on hardware
## **Prerequisites for practice**
There are no strong prerequisites for the practice class, however:
* **Linear Algebra**, **Linear Control Theory** as well as **Dynamics** (Mechanics, Physics) and **Calculus** course will be really helpfull
* As before we will use a bit of **Python** and maybe **C++** in this class
* We will often use [mathematical quantifiers](https://en.wikipedia.org/wiki/Quantifier_(logic)). Revise them by going over [exercises](https://www.whitman.edu/mathematics/higher_math_online/section01.02.html) if you forgot some.

## **Notation**
We will use the following notation in "notebooks":
> A key points or equations looks like this

>**QUESTION**:
These questions are related to lecture materials, answering them will help you to develop solid understanding of the course.

>**HW EXERCISE**:
This is home assignment which I **strongly** recommend to complete right  after the class. You have to create a `*yourname_№ofclass.ipynb*` and share it with me through [Colab](https://colab.research.google.com/).

>**BONUS:** Problems that will be graded separately and will give you bonus points on final exam

# **Practice 1: Fundamental Subspaces and Jacobian Mapping**
## **Goals for today**

---

During today practice we will:
*   Exploit a structure of linear mapping between Joint and Task spaces velocities.
*   Understand what is physical interpretation of fundamental subspaces of the manipulator Jacobian.
*   Obtain null space motion of 3 DoF plane manipulator with fixed end effector

## **Four Fundamental Subspaces. Recall**
---
>As we have studied on the lectures there are four fundamental subspaces accompanying any linear operator (matrix) $\mathbf{A}^{m \times n}$, namely:
>*   **Column** space (range, image): $\mathcal{C}(\mathbf{A}) \in \mathbb{R}^m$
>*   **Null** space (kernel): $\mathcal{N}(\mathbf{A}) \in \mathbb{R}^n$
>*   **Row** space: $\mathcal{R}(\mathbf{A}) = \mathcal{C}(\mathbf{A}^T)  \in \mathbb{R}^n$
>*   **Left null** space: $\mathcal{N}(\mathbf{A}^T) \in \mathbb{R}^m$
---

>**QUESTION**:
Why the null space $\mathcal{N}(\mathbf{A}^T)$ is called left?

Denoted above subspaces are widely used in various applications, today we are going to apply them to exploit an inner structure of kinematics and statics of redundant manipulators.

## **Jacobian Velocity Mapping**

>Let us consider differential kinematic relationship:
\begin{equation}
  \dot{\boldsymbol{x}} = \mathbf{J}(\mathbf{q})\dot{\mathbf{q}}
\end{equation}
where
*   $\boldsymbol{x} \in \mathbb{R}^m$ task space variables (for instance Cartesian coordinates)
*   $\mathbf{q} \in \mathbb{R}^n$ joint space variables (positions of joints)
*   $\mathbf{J} \in \mathbb{R}^{m\times n}$ mainpulator Jacobian

As one can note relationship between joint and task velocity is linear (for particular $\mathbf{q}$) thus, the differential kinematics equation can be characterized in terms of the columns space $\mathcal{C}(\mathbf{J})$ (range) and null space $\mathcal{N}(\mathbf{J})$ of the mapping $\mathbf{J}$

*   The column space of $\mathbf{J}$ is the subspace $\mathcal{C}(\mathbf{J}) \in \mathbb{R}^m$ of the end-effector velocities $\dot{\mathbf{x}}$ that can be generated by the joint velocities $\dot{\mathbf{q}}$, in the given manipulator
posture $\mathbf{q}$
*   The null space of $\mathbf{J}$ is the subspace $\mathcal{N}(\mathbf{J}) \in \mathbb{R}^n$ of joint velocities $\dot{\mathbf{q}}$ that do not produce any end-effector velocity $\dot{\mathbf{x}}$, in the given manipulator posture $\mathbf{q}$.

If the Jacobian has full rank, one has $\text{dim}\{\mathcal{C}(\mathbf{J})\} = m,\text{dim}\{\mathcal{N}(\mathbf{J})\} = n - m $. In case of not reduntant manipulator $m = n$ H dimension of null space increases only in the singular configurations. However, in the reduntant case condition  $\mathcal{N}(\mathbf{J}) \neq \emptyset$ may be satisfied not only in singular configurations, which allow determination of systematic techniques to handle redundant DOFs.

## **Case Study: Reduntand Manipulation**
Consider the 3-DoF planar manipulator:

A problem is to find the motion $\mathbf{q}^*(t),\mathbf{\dot{q}}^*(t)$ of the manipulator from the initial pose $\mathbf{q}_0$ such that end effector remain in the rest (as shown [here](https://youtu.be/sZYBC8Lrmdo?t=11)).

To do so we are going to find joint velocity vector that is in null-space of Jacobian and integrate it from initial state to obtain a desired joint angles.

A first step is to find forward kinematics:
\begin{equation}
\boldsymbol{x} =
\sum^n_{i=1}\ell_i
\begin{bmatrix}
 \cos \sum^i_{j=1}q_j\\
 \sin \sum^i_{j=1}q_j
\end{bmatrix}
\end{equation}

Thus the $i$-th column of Jacobian calculated recursevely as:
\begin{equation}
\mathbf{J}_i =
\frac{\partial \boldsymbol{x}}{\partial q_i}
=
\sum^n_{j=i}
\ell_j
\begin{bmatrix}
 -\sin \sum^j_{k=1} q_k\\
 \cos \sum^j_{k=1} q_k
\end{bmatrix}
\end{equation}

Let's implement Jacobian matrix:

```
from numpy import cos, sin, zeros, sum, pi, dot, array

def jacobian_column(q, l, i):
    jac_i = zeros(2)
    for j in range(i,len(q)):
      jac_i+=l[j]*array([-sin(sum(q[:j+1])), cos(sum(q[:j+1]))])
    return jac_i

def jacobian(q, l):
    jac = zeros((2,len(q)))
    for i in range(len(q)):
      jac[:,i] = jacobian_column(q, l, i)
    return jac
```

To find a desired joint speeds that do not produce any motion on the end-effector one can solve:
\begin{equation}
\mathbf{J}\dot{\mathbf{q}}_0 = \dot{\boldsymbol{x}}_d = \mathbf{0}
\end{equation}
which is nothing but $\mathcal{N}(\mathbf{J})$

Let us now find the basis of null space for particular initial pose:

```
q0 = [pi/2, -pi/4, 0]
l = [0.5, 0.5, 0.5]

from scipy.linalg import null_space
null_dq = null_space(jacobian(q0,l))
print('Basis of null space:\n', null_dq)
```

    Basis of null space:
     [[ 2.54873123e-17]
     [-4.47213595e-01]
     [ 8.94427191e-01]]

One can multiply obtained basis on any particular value or function and use naive integration to calculate joint angles from "null space speed", for instance let us consider periodic motion in joint angles:

```
from numpy import linspace, shape
from scipy.integrate import odeint

def null_vel(q, l, v):
    null_dq = null_space(jacobian(q,l))*v
    return null_dq

def sysode(pos, t, l):
    v = 3*sin(6*t)
    vel = null_vel(pos, l, v)
    return vel[:,0]

# Integration
t0 = 0 # Initial time
tf = 2 # Final time
N = 1E2 # Numbers of points in time span
t = linspace(t0, tf, int(N)) # Create time span

q_sol = odeint(sysode, q0, t, args=(l,))

# Plotting
from matplotlib.pyplot import *
colors = ('r','g','b')
for i in range(len(q0)):
  plot(t, q_sol[:,i], label = r'$q_' + str(i+1) + '$', color = colors[i])
legend(loc = 'upper right')
xlim([0, tf])
grid(color='black', linestyle='--', linewidth=1.0, alpha = 0.7)
ylabel(r'Joint angles $q$ (rad)')
xlabel(r'Time $t$ (s)')
show()

```

Let's check either end effector moved or not, to do so we use solution of forward kinematics:

```
from numpy import shape

# Forward kinematics
def fk(q, l):
    x = zeros((shape(q)[0], 2))
    for k in range(shape(q)[0]):
      for i in range(shape(q)[1]):
        x[k, :]+=l[i]*array([cos(sum(q[k,:i+1])), sin(sum(q[k,:i+1]))])
    return x
# Get solution of forward kinematics
pos = fk(q_sol,l)

colors = ('r','b')
labels = (r'$x$',r'$y$')

for i in range(2):
  plot(t, pos[:,i], label = labels[i], color = colors[i])

legend(loc = 'upper right')
xlim([0, tf])
grid(color='black', linestyle='--', linewidth=1.0, alpha = 0.7)
ylabel(r'Joint angles $q$ (rad)')
xlabel(r'Time $t$ (s)')
show()

```

Clearly end effector did not move while there is notable motion in joints!

>**HW EXERCISE**:
>*   Repeat all of the above for 3 DoF manipulator with linear actuator on the base:
>

>* Recall that static relationship between forces imposed on end effector and joints torques is given by:
\begin{equation}
\boldsymbol{\tau} = \mathbf{J}(\mathbf{q})^T\mathbf{F}
\end{equation}
where:
  *   $\mathbf{F} \in \mathbb{R}^m$ task space force imposed on the end-effector
  *   $\boldsymbol{\tau} \in \mathbb{R}^n$ joint space torques (control effort in the actuators)
>
>  Explain what is the physical interpretation of remaining fundamental subspaces:  $\mathcal{R}(\mathbf{J}),\mathcal{N}(\mathbf{J^T})$, similarly as we did for velocity mapping.

>**BONUS EXERCISE**:
* Plot the time evolution of the norm of the deviation from initial pose: $d = \|\boldsymbol{x}(t) - \boldsymbol{x}(0)\|$
* Try to increase simulation time (up to several mins) and number of points in integration routine, what is happen with deviation $d$?
* Suggest and implement the method that may overcome investigated drawbacks. You may refer to chapter 3 of ["Robotics Modelling, Planning and Control" by Siciliano, B. ](https://www.academia.edu/23785978/B_Sicilliano_Robotics_Modelling_Planning_and_Control)

Google Colab notebooks/Practice Fundumental Subspaces 2020 part 1/practice_01_fss_jacobian.ipynb


# Building blocks of Reservoir Computing

```python
from pyrcn.base.blocks import InputToNode
from sklearn.datasets import make_blobs
```

```python
# Generate a toy dataset
U, y = make_blobs(n_samples=100, n_features=10)
```

## Input-to-Node

The "Input-to-Node" component describes the connections from the input features to the reservoir and the activation functions of the reservoir neurons. Normally, the input weight matrix $\mathbf{W}^{\mathrm{in}}$ has the dimension of $N^{\mathrm{res}}\times N^{\mathrm{in}}$, where $N^{\mathrm{res}}$ and $N^{\mathrm{in}}$ are the size of the reservoir and dimension of the input feature vector $\mathbf{u}[n]$ with the time index $n$, respectively. With

\begin{align}
    \label{eq:InputToNode}
    \mathbf{r}'[n] = f'(\mathbf{W}^{\mathrm{in}}\mathbf{u}[n] + \mathbf{w}^{\mathrm{bi}}) \text{ , }
\end{align}

we can describe the non-linear projection of the input features $\mathbf{u}[n]$ into the high-dimensional reservoir space $\mathbf{r}'[n]$ via the non-linear input activation function $f'(\cdot)$.

The values inside the input weight matrix are usually initialized randomly from a uniform distribution on the interval $[-1, 1]$ and are afterwards scaled using the input scaling factor $\alpha_{\mathrm{u}}$. Since in case of a high dimensional input feature space and/or large reservoir sizes $N^{\mathrm{res}}$, this leads to a huge input weight matrix and expensive computations to feed the feature vectors into the reservoir, it was shown that it is sufficient to have only a very small number of connections from the input nodes to the nodes inside the reservoir. Each node of the reservoir may therefore be connected to only $K^{\mathrm{in}}$ ($\ll N^{\mathrm{in}}$) randomly selected input entries. This makes $\mathbf{W}^{\mathrm{in}}$ typically very sparse and feeding the feature vectors into the reservoir potentially more efficient.

The bias weights $\mathbf{w}^{\mathrm{bi}}$ with dimension $N^{\mathrm{res}}$ are typically initialized by fixed random values from a uniform distribution between $\pm 1$ and multiplied by the hyper-parameter $\alpha_{\mathrm{bi}}$.

```python
#      _ _ _ _ _ _ _ _
#     |               |
# ----| Input-to-Node |------
# u[n]|_ _ _ _ _ _ _ _|r'[n]
# U                    R_i2n

input_to_node = InputToNode(hidden_layer_size=50,
                            k_in=5, input_activation="tanh",
                            input_scaling=1.0, bias_scaling=0.1)

R_i2n = input_to_node.fit_transform(U)
print(U.shape, R_i2n.shape)
```

## Node-to-Node

The "Node-to-Node" component describes the connections inside the reservoir. The output of "Input-to-Node" $\mathbf{r}'[n]$ together with the output of "Node-to-Node" from the previous time step $\mathbf{r}[n-1]$ are used to compute the new output of "Node-to-Node" $\mathbf{r}[n]$ using

\begin{align}
    \label{eq:NodeToNode}
    \mathbf{r}[n] = (1-\lambda)\mathbf{r}[n-1] + \lambda f(\mathbf{r}'[n] + \mathbf{W}^{\mathrm{res}}\mathbf{r}[n-1])\text{ , }
\end{align}

which is a leaky integration of the time-dependent reservoir states $\mathbf{r}[n]$. $f(\cdot)$ acts as the non-linear reservoir activation functions of the neurons in "Node-to-Node". The leaky integration is equivalent to a first-order lowpass filter. Depending  on the leakage $\lambda \in (0, 1]$, the reservoir states are globally smoothed.

The reservoir weight matrix $\mathbf{W}^{\mathrm{res}}$ is a square matrix of the size $N^{\mathrm{res}}$. These weights are typically initialized from a standard normal distribution. The Echo State Property (ESP) requires that the states of all reservoir neurons need to decay in a finite time for a finite input pattern. In order to fulfill the ESP, the reservoir weight matrix is typically normalized by its largest absolute eigenvalue and rescaled to a spectral radius $\rho$, because it was shown that the ESP holds as long as $\rho \le 1$. The spectral radius and the leakage together shape the temporal memory of the reservoir. Similar as for "Input-to-Node", the reservoir weight matrix gets huge in case of large reservoir sizes $N^{\mathrm{res}}$, it can be sufficient to only connect each node in the reservoir only to $K^{\mathrm{rec}}$ ($\ll N^{\mathrm{res}}$) randomly selected other nodes in the reservoir, and to set the remaining weights to zero.

To incorporate some information from the future inputs, bidirectional RCNs have been introduced.

```python
from pyrcn.base.blocks import NodeToNode
```

```python
#      _ _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |               |      |              |
# ----| Input-to-Node |------| Node-to-Node |------
# u[n]|_ _ _ _ _ _ _ _|r'[n] |_ _ _ _ _ _ _ |r[n]
# U                    R_i2n                 R_n2n

# Initialize, fit and apply NodeToNode
node_to_node = NodeToNode(hidden_layer_size=50,
                          reservoir_activation="tanh",
                          spectral_radius=1.0, leakage=0.9,
                          bidirectional=False)
R_n2n = node_to_node.fit_transform(R_i2n)
print(U.shape, R_n2n.shape)
```

## Node-to-Output

The "Node-to-Output" component is the mapping of the reservoir state $\mathbf{r}[n]$ to the output $\mathbf{y}[n]$ of the network. In conventional RCNs, this mapping is trained using (regularized) linear regression. To that end, all reservoir states $\mathbf{r}[n]$ are concatenated into the reservoir state collection matrix $\mathbf{R}$. As linear regression usually contains an intercept term, every reservoir state $\mathbf{r}[n]$ is expanded by a constant of 1. All desired outputs $\mathbf{d}[n]$ are collected into the desired output collection matrix $\mathbf{D}$. Then, the mapping matrix $\mathbf{W}^{\mathrm{out}}$ can be computed using

\begin{align}
    \label{eq:linearRegression}
    \mathbf{W}^{\mathrm{out}} =\left(\mathbf{R}\mathbf{R}^{\mathrm{T}} + \epsilon\mathbf{I}\right)^{-1}(\mathbf{D}\mathbf{R}^{\mathrm{T}}) \text{,}
\end{align}

where $\epsilon$ is a regularization parameter.

The size of the output weight matrix $N^{\mathrm{out}}\times (N^{\mathrm{res}} + 1)$ or $N^{\mathrm{out}}\times (2 \times N^{\mathrm{res}} + 1)$ in case of a bidirectional "Node-to-Node" determines the total number of free parameters to be trained in the neural network.

After training, the output $\mathbf{y}[n]$ can be computed using Equation

\begin{align}
\label{eq:readout}
\mathbf{y}[n] = \mathbf{W}^{\mathrm{out}}\mathbf{r}[n] \text{ . }
\end{align}

Note that, in general, other training methodologies could be used to compute output weights.

```python
from sklearn.linear_model import Ridge
```

```python
#       _ _ _ _ _ _ _       _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |              |     |             |     |               |
# ----|Input-to-Node |-----|Node-to-Node |-----|Node-to-Output |
# u[n]| _ _ _ _ _ _ _|r'[n]|_ _ _ _ _ _ _|r[n] | _ _ _ _ _ _ _ |
# U                   R_i2n               R_n2n        |
#                                                      |
#                                                 y[n] | y_pred

# Initialize, fit and apply NodeToOutput
y_pred = Ridge().fit(R_n2n, y).predict(R_n2n)
print(y_pred.shape)
```

## Predicting the Mackey-Glass equation

Set up and train vanilla RCNs for predicting the Mackey-Glass time series with the same settings as used to introduce ESNs. The minimum working example shows the simplicity of implementing a model with PyRCN and the inter-operability with scikit-learn; it needs only four lines of code to load the Mackey-Glass dataset that is part of PyRCN and only two lines to fit the different RCN models, respectively. Instead of the default incremental regression, we have customized the ```ELMRegressor()``` by using ```Ridge``` from scikit-learn.

```python
from sklearn.linear_model import Ridge as skRidge
from pyrcn.echo_state_network import ESNRegressor
from pyrcn.extreme_learning_machine import ELMRegressor
from pyrcn.datasets import mackey_glass
```

```python
# Load the dataset
X, y = mackey_glass(n_timesteps=5000)
# Define Train/Test lengths
trainLen = 1900
X_train, y_train = X[:trainLen], y[:trainLen]
X_test, y_test = X[trainLen:], y[trainLen:]

# Initialize and train an ELMRegressor and an ESNRegressor
esn = ESNRegressor().fit(X=X_train.reshape(-1, 1), y=y_train)
elm = ELMRegressor(regressor=skRidge()).fit(X=X_train.reshape(-1, 1), y=y_train)

print("Fitted models")
```

# Build Reservoir Computing Networks with PyRCN

By combining the building blocks introduced above, a vast number of different RCNs can be constructed. In this section, we build two important variants of RCNs, namely ELMs and ESNs.

## Extreme Learning Machines
The vanilla ELM as a single-layer feedforward network consists of an "Input-to-Node" and a "Node-to-Output" module and is trained in two steps:

1. Compute the high-dimensional reservoir states $\mathbf{R}'$, which is the collection of reservoir states $\mathbf{r}'[n]$.
2. Compute the output weights $\mathbf{W}^{\mathrm{out}}$ with $\mathbf{R}'$.

```python
U, y = make_blobs(n_samples=100, n_features=10)
from pyrcn.extreme_learning_machine import ELMRegressor
```

```python
# Vanilla ELM for regression tasks with input_scaling
#       _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |              |     |               |
# ----|Input-to-Node |-----|Node-to-Output |------
# u[n]| _ _ _ _ _ _ _|r'[n]| _ _ _ _ _ _ _ |y[n]
#                                           y_pred
#
vanilla_elm = ELMRegressor(input_scaling=0.9)
vanilla_elm.fit(U, y)
print(vanilla_elm.predict(U))
```

Example of how to construct an ELM with a BIP "Input-to-Node" ELMs with PyRCN.

```python
from pyrcn.base.blocks import BatchIntrinsicPlasticity

# Custom ELM with BatchIntrinsicPlasticity
#       _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |              |     |               |
# ----|     BIP      |-----|Node-to-Output |------
# u[n]| _ _ _ _ _ _ _|r'[n]| _ _ _ _ _ _ _ |y[n]
#                                           y_pred
#
bip_elm = ELMRegressor(input_to_node=BatchIntrinsicPlasticity(),
                       regressor=Ridge(alpha=1e-5))

bip_elm.fit(U, y)
print(bip_elm.predict(U))
```

Hierarchical or Ensemble ELMs can then be built using multiple "Input-to-Node" modules in parallel or in a cascade. This is possible when using using scikit-learn's ```sklearn.pipeline.Pipeline``` (cascading) or ```sklearn.pipeline.FeatureUnion``` (ensemble).

```python
from sklearn.pipeline import Pipeline, FeatureUnion
```

```python
# ELM with cascaded InputToNode and default regressor
#       _ _ _ _ _ _ _        _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |     (bip)    |     |    (base)    |     |               |
# ----|Input-to-Node1|-----|Input-to-Node2|-----|Node-to-Output |
# u[n]| _ _ _ _ _ _ _|     | _ _ _ _ _ _ _|r'[n]| _ _ _ _ _ _ _ |
#                                                       |
#                                                       |
#                                                  y[n] | y_pred
#
i2n = Pipeline([('bip', BatchIntrinsicPlasticity()),
                ('base', InputToNode(bias_scaling=0.1))])
casc_elm = ELMRegressor(input_to_node=i2n).fit(U, y)

# Ensemble of InputToNode with activations
#             _ _ _ _ _ _ _
#           |      (i)     |
#      |----|Input-to-Node1|-----|
#      |    | _ _ _ _ _ _ _|     |       _ _ _ _ _ _ _
#      |                          -----|               |
# -----o                          r'[n]|Node-to-Output |------
# u[n] |      _ _ _ _ _ _ _      |-----| _ _ _ _ _ _ _ |y[n]
#      |    |     (th)     |     |                      y_pred
#      |----|Input-to-Node2|-----|
#           | _ _ _ _ _ _ _|
#
i2n = FeatureUnion([('i', InputToNode(input_activation="identity")),
                    ('th', InputToNode(input_activation="tanh"))])
ens_elm = ELMRegressor(input_to_node=i2n)
ens_elm.fit(U, y)
print(casc_elm, ens_elm)
```

## Echo State Networks
ESNs, as variants of RNNs, consist of an "Input-to-Node", a "Node-to-Node" and a "Node-to-Output" module and are trained in three steps.

1. Compute the neuron input states $\mathbf{R}'$, which is the collection of reservoir states $\mathbf{r}'[n]$. Note that here the activation function $f'(\cdot)$ is typically linear.
2. Compute the reservoir states $\mathbf{R}$, which is the collection of reservoir states $\mathbf{r}[n]$. Note that here the activation function $f(\cdot)$ is typically non-linear.
3. Compute the output weights $\mathbf{W}^{\mathrm{out}}$ using
    1. Linear regression with $\mathbf{R}$ when considering an ESN.
    2. Backpropagation or other optimization algorithm when considering a CRN or when using an ESN with non-linear outputs.

What follows is an example of how to construct such a vanilla ESN with PyRCN, where the ```ESNRegressor``` internally passes the input features through "Input-to-Node" and "Node-to-Node", and trains "Node-to-Output" using ```pyrcn.linear_model.IncrementalRegression```.

```python
from pyrcn.echo_state_network import ESNRegressor
```

```python
# Vanilla ESN for regression tasks with spectral_radius and leakage
#       _ _ _ _ _ _ _       _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |              |     |             |     |               |
# ----|Input-to-Node |-----|Node-to-Node |-----|Node-to-Output |
# u[n]| _ _ _ _ _ _ _|r'[n]|_ _ _ _ _ _ _|r[n] | _ _ _ _ _ _ _ |
#                                                      |
#                                                      |
#                                                 y[n] | y_pred
#
vanilla_esn = ESNRegressor(spectral_radius=1, leakage=0.9)
vanilla_esn.fit(U, y)
print(vanilla_esn.predict(U))
```

As for ELMs, various unsupervised learning techniques can be used to pre-train "Input-to-Node" and "Node-to-Node".

```python
from pyrcn.base.blocks import HebbianNodeToNode
```

```python
# Custom ESN with BatchIntrinsicPlasticity and HebbianNodeToNode
#       _ _ _ _ _ _ _       _ _ _ _ _ _ _        _ _ _ _ _ _ _
#     |     (bip)    |     |   (hebb)    |     |               |
# ----|Input-to-Node |-----|Node-to-Node |-----|Node-to-Output |
# u[n]| _ _ _ _ _ _ _|r'[n]|_ _ _ _ _ _ _|r[n] | _ _ _ _ _ _ _ |
#                                                      |
#                                                      |
#                                                 y[n] | y_pred
#
bip_esn = ESNRegressor(input_to_node=BatchIntrinsicPlasticity(),
                       node_to_node=HebbianNodeToNode(),
                       regressor=Ridge(alpha=1e-5))

bip_esn.fit(U, y)
print(bip_esn.predict(U))
```

The term "Deep ESN" can refer to different approaches of hierarchical ESN architectures:

Example of how to construct a rather complex ESN consisting of two layers. It is built out of two small parallel reservoirs in the first layer and a large reservoir in the second layer.

```python
# Multilayer ESN
#                  u[n]
#                   |
#                   |
#          _________o_________
#         |                   |
#   _ _ _ | _ _ _       _ _ _ | _ _ _
# |      (i)     |    |      (i)     |
# |Input-to-Node1|    |Input-to-Node2|
# | _ _ _ _ _ _ _|    | _ _ _ _ _ _ _|
#         |r1'[n]             | r2'[n]
#   _ _ _ | _ _ _       _ _ _ | _ _ _
# |     (th)     |    |     (th)     |
# | Node-to-Node1|    | Node-to-Node2|
# | _ _ _ _ _ _ _|    | _ _ _ _ _ _ _|
#         |r1[n]              | r2[n]
#         |_____         _____|
#               |       |
#             _ | _ _ _ | _
#           |               |
#           | Node-to-Node3 |
#           | _ _ _ _ _ _ _ |
#                   |
#              r3[n]|
#             _ _ _ | _ _ _
#           |               |
#           |Node-to-Output |
#           | _ _ _ _ _ _ _ |
#                   |
#               y[n]|

l1 = Pipeline([('i2n1', InputToNode(hidden_layer_size=100)),
               ('n2n1', NodeToNode(hidden_layer_size=100))])

l2 = Pipeline([('i2n2', InputToNode(hidden_layer_size=400)),
               ('n2n2', NodeToNode(hidden_layer_size=400))])

i2n = FeatureUnion([('l1', l1),
                    ('l2', l2)])
n2n = NodeToNode(hidden_layer_size=500)
layered_esn = ESNRegressor(input_to_node=i2n,
                           node_to_node=n2n)

layered_esn.fit(U, y)
print(layered_esn.predict(U))
```

## Complex example: Optimize the hyper-parameters of RCNs

Example for a sequential parameter optimization with PyRCN. Therefore, a model with initial parameters and various search steps are defined. Internally, ```SequentialSearchCV``` will perform the list of optimization steps sequentially.

```python
from sklearn.metrics import make_scorer
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import TimeSeriesSplit
from sklearn.model_selection import RandomizedSearchCV, \
                                    GridSearchCV
from scipy.stats import uniform
from pyrcn.model_selection import SequentialSearchCV
from pyrcn.datasets import mackey_glass
```

```python
# Load the dataset
X, y = mackey_glass(n_timesteps=5000)
X_train, X_test = X[:1900], X[1900:]
y_train, y_test = y[:1900], y[1900:]

# Define initial ESN model
esn = ESNRegressor(bias_scaling=0, spectral_radius=0, leakage=1)

# Define optimization workflow
scorer = make_scorer(mean_squared_error, greater_is_better=False)
step_1_params = {'input_scaling': uniform(loc=1e-2, scale=1),
                 'spectral_radius': uniform(loc=0, scale=2)}
kwargs_1 = {'n_iter': 200, 'n_jobs': -1, 'scoring': scorer,
            'cv': TimeSeriesSplit()}
step_2_params = {'leakage': [0.2, 0.4, 0.7, 0.9, 1.0]}
kwargs_2 = {'verbose': 5, 'scoring': scorer, 'n_jobs': -1,
            'cv': TimeSeriesSplit()}

searches = [('step1', RandomizedSearchCV, step_1_params, kwargs_1),
            ('step2', GridSearchCV, step_2_params, kwargs_2)]

# Perform the search
esn_opti = SequentialSearchCV(esn, searches).fit(X_train.reshape(-1, 1), y_train)
print(esn_opti)
```

## Programming pattern for sequence processing

This complex use-case requires a serious hyper-parameter tuning. To keep the code example simple, we did not include the optimization in this paper and refer the interested readers to the Jupyter Notebook [^1] that was developed to produce these results.

[^1]: https://github.com/TUD-STKS/PyRCN/blob/master/examples/digits.ipynb

```python
from sklearn.base import clone
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_validate
from sklearn.model_selection import ParameterGrid
from sklearn.metrics import make_scorer

from pyrcn.echo_state_network import ESNClassifier
from pyrcn.metrics import accuracy_score
from pyrcn.datasets import load_digits
```

```python
# Load the dataset
X, y = load_digits(return_X_y=True, as_sequence=True)
print("Number of digits: {0}".format(len(X)))
print("Shape of digits {0}".format(X[0].shape))
# Divide the dataset into training and test subsets
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2,
                                          random_state=42)
print("Number of digits in training set: {0}".format(len(X_train)))
print("Shape of the first digit: {0}".format(X_train[0].shape))
print("Number of digits in test set: {0}".format(len(X_test)))
print("Shape of the first digit: {0}".format(X_test[0].shape))

# These parameters were optimized using SequentialSearchCV
esn_params = {'input_scaling': 0.1,
              'spectral_radius': 1.2,
              'input_activation': 'identity',
              'k_in': 5,
              'bias_scaling': 0.5,
              'reservoir_activation': 'tanh',
              'leakage': 0.1,
              'k_rec': 10,
              'alpha': 1e-5,
              'decision_strategy': "winner_takes_all"}

b_esn = ESNClassifier(**esn_params)

param_grid = {'hidden_layer_size': [50, 100, 200, 400, 500],
              'bidirectional': [False, True]}

for params in ParameterGrid(param_grid):
    esn_cv = cross_validate(clone(b_esn).set_params(**params),
                            X=X_tr, y=y_tr,
                            scoring=make_scorer(accuracy_score))
    esn = clone(b_esn).set_params(**params).fit(X_tr, y_tr, n_jobs=-1)
    acc_score = accuracy_score(y_te, esn.predict(X_te))
```

```python

```


# pyGemPick Tutorial 2: Dual High-Contrast Filtering with Detection

## How To Effectively Pick A Lot of Inmmunogold Particles, Fast & Accurate!

{ Insert Video Here Once Completed!}

## Step 1: Micrograph Filtering

After the images are compressed - the resulting jpg can be compressed and and then detected simpultaneously! __*(Note: I like working with compressed images in a separate folder so they can be reused in subsequent processing steps!)*__

I have successfully derived high-contrast versions of common low-pass, edge detecting filters! These show the modifications of the laplace edge detection kernel and laplace of gaussian kernel respectively. Firstly, the kernels were negated and the anchoring value at the center of the kernel was removed and replaced with a scaling factor! Each pass of the filter - produces a binary image where the gold particles are isolated and can be easily detected with optimization of [OpenCv's Simple Blob Detector](https://www.learnopencv.com/blob-detection-using-opencv-python-c/). Note for the HCLAP kernel, anchor values can be chosen 6+ whereas the HLOG kernel anchor values of 18+ are reccommended! These kernels are applied to the image by a matrix convolution operation made possible by [OpenCv's **filter2D( )** function](https://docs.opencv.org/3.0-beta/modules/imgproc/doc/filtering.html)!

### High Contrast Laplace Kernel (HCLAP)

$$\begin{align} \begin{bmatrix}0 &-1 & 0\\-1 & p & -1\\0 & -1 & 0 \end{bmatrix} \end{align}$$

### High Contrast Laplace of Gaussian Kernel (HLOG)

$$\begin{align} \begin{bmatrix}0 & 0 &-1 & 0 & 0\\0 & -1 & -2 & -1 & 0\\-1 & -2 & p & -2 & -1\\0 & -1 & -2 & -1 & 0\\0 & 0 &-1 & 0 & 0 \end{bmatrix} \end{align}$$

## Step 2: Gold Particle Detection

After sucessful detection, the binary images are then passed into the py.pick() function. This function takes the binary images from one of the filters provided plus 5 additional picking values. __*Note: For best results, it is reccomended that images taken at different magnifications are processed separately in different image sets!*__

    1. __minArea:__ allows you to set the minimum number of pixels that will be present in a gold particle on the images which you're trying to detect.

    2. __minCirc:__ allows you to filter the detected gold particles based on how close to a circle that particle looks like. Due to the magnification, excessive counter staining and the properties of the filters themselves - this value will be less than the ideal which is one!

$$minCirc = \frac{4*\pi*Area}{(perimeter)^2}$$

    3. __minConv__: As the article above suggests, "Convexity is defined as the (Area of the Blob / Area of it’s convex hull). Convex Hull of a shape is the tightest convex shape that completely encloses the shape.'

    4. __minIner__: The Inertia ratio of a object allows you to filter the gold particles by how elongated that shape looks.  minIner=1 defines a complete circle. Ideally, this parameter uses a type of Hessian Matrix analysis to gain the required result! Due to the magnification, excessive counter staining and the properties of the filters themselves - this value will be less than the ideal!

    5. __minThresh__: applies simple binary thresholding to the image. Usually, when using other filtering techniques and binary image already obtained, this value is set to zero!

```python
import glob
import cv2
import numpy as np
import pygempick.core as py
import pygempick.modeling as mod
import pygempick.spatialstats as spa
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
#input your folder location with compressed jpg images
images = glob.glob('/home/joseph/Documents/pygempick/samples/compressed/*.jpg')
```

```python
#difine filtering paramaters
pclap = 25 #HCLAP anchor value
plog  = 20 #HLOG anchor value
i = 0 #image counter
```

```python
image_number = [] #list for image number counter
detected = [] #list for detected numbr of keypoints
```

```python
for image in images:

    orig_img = cv2.imread(image) ##reads specific jpg image from folder with compressed images
    output1 = py.hclap_filt(pclap, orig_img, 'no')
    output2 = py.hlog_filt(plog, orig_img, 'no')
    #output1 = py.dog_filt(p,orig_img)
    #output2 = py.bin_filt(p, orig_img)

    #write each binary images to the binary folder of your choice!
    cv2.imwrite('/home/joseph/Documents/pygempick/samples/binary/{}_hclap_{}.jpg'.format(i, pclap), output1)
    cv2.imwrite('/home/joseph/Documents/pygempick/samples/binary/{}_hlog_{}.jpg'.format(i, plog),output2)

    #image, minArea, minCirc, minConv, minIner, minThres
    keypoints1 = py.pick(output1, 37, .71, .5 , .5, 0)
    keypoints2 = py.pick(output2, 37, .71, .5 , .5, 0)

    #this function removes duplicated detections
    keypoints1, dup1 = py.key_filt(keypoints1, keypoints2)

    keypoints = keypoints1 + keypoints2

    # Draws detected blobs on image with green circles using opencv's draw keypoints
    imd = cv2.drawKeypoints(orig_img, keypoints, np.array([]), (0,255,0),\
                             cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)

    #write the image with the picked blobs to the picked folder - change name!
    cv2.imwrite('/home/joseph/Documents/pygempick/samples/picked/{}_picked.jpg'.format(i), imd)

    image_number.append(i)
    i+=1

    if len(keypoints) > 0:
        detected.append(len(keypoints1))
    else:
        detected.append(0)
```

```python
print('Total gold particles detected:', sum(detected))
```

    Total gold particles detected: 97

```python
plt.figure()
plt.title('Aggregate Count in 6E10-Anti AB images.')
plt.plot(image_number, np.array(detected) , '.b-', label = "Dual High-Contrast Filtering")
#plt.plot(image_number, detected2, '.r-', label = "Filtered w/ DOG")
plt.xlabel("Image Number")
plt.ylabel("Particles Detected")
plt.legend(loc='best')
plt.show()
```

```python
plt.imshow(imd, cmap='gray')

```

```python
plt.imshow(output1, cmap='gray') #binary image of HCLAP filter

```

```python
plt.imshow(output2, cmap='gray') #binary image of HLOG filter
```

```python

```