# Inference in discrete state Hidden Markov Models using numpy

This demo shows exact inference on a Hidden Markov Model with known, discrete transition and emission distributions that are fixed over time. It does alpha recursion, which is a sum-product algorithm on HMMs.

The problem is taken from Example 23.3 from ["Bayesian Reasoning and Machine Learning"](http://www.cs.ucl.ac.uk/staff/d.barber/brml/), and this is more or less a port of the provided Matlab implementation with my commentary.

I'm trying to understand how my probabilistic graphical modelling class can be represented in code, so I try to go through this painfully slow and show how the `numpy` arrays map to the discrete probability distributions.

## Problem

The idea is that there's someone moving around a room. You can't see them, but you can hear bumps and creaks from them moving around. You know the probability of someone bumping or creaking at each location in the room. You also know how one moves around the room.
Using the sequence of bumps and/or creaks, the goal is to figure out the locations in the room.

Below is what I will end up with:

The first row represents the bumps/creaks you hear. The second row represents where the person actually is. The third row represents where alpha recursion guesses the person is.

### HMMs

This problem can be modeled using a Hidden Markov Model. At each timestep $t$ in the sequence, there's the visible state $v_t$ (the bump/creak combination) and the hidden state $h_t$ (the location in the room.)

The goal of *filtering* is to show the probability distribution over the locations in the room at a given timestep $t$ given the bumps/creaks from the current and previous timesteps, $p(h_t\mid v_{1:t})$. Plotting this distribution at each timestep gives a heatmap of where in the room we think the person is.

In this model, there is a transition distribution $p(h_t \mid h_{t - 1})$ to show that each hidden state depends on the previous timestep's hidden state. There also is the emission distribution $p(v_t \mid h_t)$ to show each visible state depends on the corresponding hidden state. There's also the probability of where in the room the person starts, $p(h_1)$.

The rest of the notebook shows:
 - Setting up the known distributions $p(h_1)$, $p(h_t \mid h_{t - 1})$, and $p(v_t \mid h_t)$. This is a lot of the notebook!
 - Generating some data for the locations ($h_{1:t}$) and sounds ($v_{1:t}$) from that distribution.
 - Finally I get to alpha recursion. I try to predict $h_{1:t}$ based on $v_{1:t}$, $p(h_1)$, $p(h_t \mid h_{t - 1})$, and $p(v_t \mid h_t)$.

```python
import numpy as np
import matplotlib.pyplot as plt

# helper functions you can skip over :D
SAVE = False
def maybe_save_plot(filename):
    if SAVE:
        plt.tight_layout()
        plt.savefig('images/' + filename, bbox_inches="tight")

def hide_ticks(plot):
    plot.axes.get_xaxis().set_visible(False)
    plot.axes.get_yaxis().set_visible(False)
```

## Distributions

Before anything, I'll set up the room. I want a $6 \times 5$* room with a total of 30 possible locations.

In HMM, this means there are 30 hidden states, and at each timestep $t$, $h_t \in \{0,...,29 \}$. Heads up, I'm using index-at-1 notation for the variables to match the book's notation and index-at-0 for states and code.

In `numpy`, I'll set a `width` and `height` for the room and `num_hidden_states` as the total possible locations in the room. While I'm here, I'll set up `map_x_y_to_hidden_state_id` as a convenient way to map from an `x` and `y` coordinate to the index of the state.

*The original example used a $5\times5$ room, but I didn't want to accidentally transpose my math, so I made it a rectangular room instead.

```python
width = 6
height = 5
num_hidden_states = width * height
```

### Initial distribution (starting location) $p(h_1)$

The initial distribution $p(h_1)$ gives the probability over the hidden states at the first timestep.

In this case, it's the probability over locations in the room where the person is at the first timestep. The Barber book uses a uniform distribution for this problem. This means at the first timestep, they could be anywhere in the room.

To represent $p(h_1)$ as a `numpy` array `p_hidden_start`:
 - The distribution is uniform, so all values in the array should be the same.
 - The distribution's domain is the possible hidden states, so the array should have the same size `(30,)`.*
 - The array represents a probability distribution so it should sum to 1 and each value should be greater than or equal to 0.

*Technically this distribution can be specified with only 29 values because it has to sum to 1. But I'm not going to worry about that here.

```python
# prob of starting starting locations
p_hidden_start = np.ones(num_hidden_states) / num_hidden_states

# verify it's a valid probability distribution
assert np.all(np.isclose(np.sum(p_hidden_start), 1))
assert np.all(p_hidden_start >= 0)
```

### Transition (moving around the room) $p(h_t \mid h_{t-1})$

Given the previous location, the transition distribution $p(h_t \mid h_{t-1})$ gives the probability of moving to a new location.

For this problem, the person can only move up, down, left, or right and they can't stand still. This can be encoded in a probability distribution, which is cool! I'll set the value to 0 when it's an illegal move and otherwise evenly distribute it among the legal moves.

Using this I can define $p(h_t, h_{t - 1})$, which I can use to find $p(h_t \mid h_{t-1})$. Translating to `numpy`:
 - This distribution defines probabilities for every combination of two hidden states, so it will have $30 \times 30$ values*. To make things easier when I compute $p(h_t \mid h_{t - 1})$, I'll use a `(30, 30)` array. I'll have `p_transition_joint[new_state][old_state]` as the probability $p(h_t=\texttt{new_state}, h_{t - 1}=\texttt{old_state})$.
 - It's a joint probability, so summing over the entire 2D array should give 1 and every value should be greater than or equal to 0.

*Like with $p(h_1)$, technically this can be specified with $30 \times 30 - 1$ values, because the entire table sums to 1. And like before, I'm not going to worry about that in code.

```python
def create_transition_joint(width, height):
    num_hidden_states = width * height

    # begin by building an unnormalized matrix with 1s for all legal moves.
    unnormalized_transition_joint = np.zeros((num_hidden_states, num_hidden_states))

    # This will help me map from height and width to the state
    map_x_y_to_hidden_state_id = np.arange(num_hidden_states).reshape(height, width).T

    for x in range(width):
        for y in range(height):
            h_t = map_x_y_to_hidden_state_id[x, y]

            # hax to go through each possible direction
            for d in range(4):
                new_x = x
                new_y = y
                if d // 2 == 0:
                    # move left or right!
                    new_x = x + ((d % 2) * 2 - 1)
                else:
                    # move up or down!
                    new_y = y + ((d % 2) * 2 - 1)

                # make sure they don't walk through walls
                if any((
                    new_x > width - 1,
                    new_x < 0,
                    new_y > height - 1,
                    new_y < 0
                )):
                    continue

                h_t_minus_1 = map_x_y_to_hidden_state_id[new_x, new_y]
                unnormalized_transition_joint[h_t_minus_1][h_t] = 1

    # normalize!
    p_transition_joint = unnormalized_transition_joint / np.sum(unnormalized_transition_joint)

    # make sure this is a joint probability
    assert np.isclose(np.sum(p_transition_joint), 1)
    # not super necessary, but eh
    assert np.all(p_transition_joint >= 0)

    return p_transition_joint
```

#### Computing $p(h_t \mid h_{t - 1})$ in math

Now I have $p(h_t, h_{t - 1})$. To get the transition distribution $p(h_t \mid h_{t - 1})$, from this, I need

\begin{align}
p(h_t \mid h_{t - 1}) &= \frac{p(h_t, h_{t - 1})}{p(h_{t - 1})} \\
 &= \frac{p(h_t, h_{t - 1})}{\sum_{h_t} p(h_t, h_{t - 1})}.
\end{align}

Translating to `numpy`:
 - The distribution maps from a hidden state to another hidden state, so it should be of size `(30, 30)` to represent each possible transition. The way Barber does it is by having a 2D array, where the first axis is the `new_state` $h_t$ and the second is the `old_state` $h_{t-1}$.*
 - The distribution is a conditional probability distribution, so given a value for $h_{t-1}$, the values for $h_t$ should sum to 1.

*One more time because I need to know this for my exam next week! Technically this can be specified with $30 \times (30 - 1)$ values because each of the 30 columns sums to 1. And like before I'm not going to worry about that in code.

#### Computing $p(h_t \mid h_{t - 1})$ in `numpy`

The element `p_transition_joint[new_state][old_state]` is the probability

$$p(h_t=\texttt{new_state}, h_{t - 1}=\texttt{old_state}).$$

The column `p_transition_joint[:, old_state]` s the 30-element vector representing the unnormalized distribution $\tilde{p}(h_t , h_{t - 1} = \texttt{old_state})$. If I marginalize over all possible values of $h_t$, I end up with

$$\sum_{h_t} p(h_t, h_{t - 1}=\texttt{old_state}) = p(h_{t - 1} = \texttt{old_state}).$$

This is like summing over the rows or columns of a probability table to get the marginal.

| `p(x, y)` | `x = 0` | `x = 1` | `p(y)` |
|--|--|--|--|
| `y = 0` | 0.4 | 0.2 | **0.6** |
| `y = 1` | 0.3 | 0.1 | **0.4** |
| `p(x)` | **0.7** | **0.3** | |

I'll build $p(h_t \mid h_{t - 1})$ column-wise. Each column will be the normalized distribution ${p}(h_t \mid h_{t - 1} = \texttt{old_state})$.
 For each `old_state` I'll compute $p(h_t \mid h_{t - 1}=\texttt{old_state})$ as follows:
 - Sum over the column `old_state` in `p_transition_joint`. By doing this, I'm summing over $h_t$. This gives $p(h_{t - 1}=\texttt{old_state}) = \sum_{h_t} p(h_t, h_{t - 1}=\texttt{old_state})$.
 - Divide the column `old_state` in `p_transition_joint` which represents $p(h_t, h_{t - 1}=\texttt{old_state})$ by that sum $p(h_{t - 1}=\texttt{old_state})$. That gives $p(h_t \mid h_{t - 1}=\texttt{old_state})$.

```python
def create_transition(width, height):
    p_transition_joint = create_transition_joint(width, height)

    num_hidden_states = width * height

    p_transition = np.zeros((num_hidden_states, num_hidden_states))

    for old_state in range(num_hidden_states):
        p_transition[:, old_state] = p_transition_joint[:, old_state] / np.sum(p_transition_joint[:, old_state])

    # verify it's a conditional distribution
    assert np.all(np.sum(p_transition, axis=0)) == 1

    return p_transition

p_transition = create_transition(width, height)
```

#### Visualizing

This one is weird to visualize but it makes a fun pattern.

```python
fig, ax = plt.subplots()
ax.imshow(p_transition)
ax.set_title('p(new_state | old_state)')
hide_ticks(ax)
maybe_save_plot('2018-05-02-transition')
plt.show()
```

#### Sample paths

I can also generate a few path examples to see if they look right. Later, after I have the emission distribution, I can use this code to help generate the visibles.

The `np.random.choice` comes in handy a few times. The `p` argument allows me to specify a list of probabilities for each state. I use it to sample from the `p_hidden_start` distribution and the `p_transition` distribution for a given hidden state.

```python
def plot_state_in_room(state_id, width=width, height=height):
    h = np.zeros(width * height)
    h[state_id] = 1
    return h.reshape(height, width)
```

```python
# code for plots
timesteps = 10
example_count = 5

figs, all_axs = plt.subplots(example_count, timesteps, figsize=(timesteps, 1.5 * example_count))

for ex_i in range(example_count):
    # choose the starting state using `p_hidden_start`
    curr_hidden_state = np.random.choice(num_hidden_states, p=p_hidden_start)

    # plot it
    all_axs[ex_i][0].imshow(plot_state_in_room(curr_hidden_state), cmap='gray')
    hide_ticks(all_axs[ex_i][0])

    for t in range(1, timesteps):
        # use the transition matrix to get the probability distribution of the
        # next state given the current one
        curr_hidden_state = np.random.choice(
            num_hidden_states,
            p=p_transition[:, curr_hidden_state]
        )

        all_axs[ex_i][t].imshow(plot_state_in_room(curr_hidden_state), cmap='gray')
        hide_ticks(all_axs[ex_i][t])
maybe_save_plot('2018-05-02-sample-paths')
plt.show()
```

### Emission distribution (bumps and creaks) $p(v_t \mid h_t)$

Now I need the emission distribution $p(v_t \mid h_t)$, which is the probability of a visible state given the hidden state.

In this problem, this distribution is the combination of the probability of a bump given the location, and the probability of a creak given the location.

$$p(v_t \mid h_t) = p(v_t^{bump} \mid h_t)p(v_t^{creak} \mid h_t)$$

Both of these have two states, so combining the two gives 4 visible states for $v_t$:
 - bump and creak
 - bump and no creak
 - no bump and creak
 - no bump and no creak

#### Bumps and Creaks map, $p(v^{bump}=True \mid h_t)$

First I'll set up the bumpy and creaky aspects of the room. For example, `prob_bump_true_given_location` gives the probability distribution

$$p(v^{bump}=True \mid h_t)$$

 - There are `num_hidden_states = 30` values for $h_t$, so I need 2 sets of 30 probabilities.
 - This isn't a probability distribution. They just need to be between 0 and 1.

I try to implement this in a similar way to Barber. Ten random spots in the room have a higher probability, and the rest has a low probability. This introduces some noise to the problem.
Here I use `np.random.choice` in a new way. In this case, I'm using it to select 10 spots to have a high probability of making a noise.

```python
def make_sound_map():
    NUM_SOUNDS = 10
    LOW_PROB = 0.1
    HIGH_PROB = 0.9

    # everything has at least LOW_PROB of triggering the sound
    grid = LOW_PROB * np.ones(num_hidden_states)
    # select NUM_BUMP_CREAKS to make HIGH_PROB
    locs = np.random.choice(
        num_hidden_states,
        size=NUM_SOUNDS,
        replace=False
    )
    grid[locs] = HIGH_PROB

    return grid

prob_bump_true_given_location = make_sound_map()
prob_creak_true_given_location = make_sound_map()
```

### Visualizing bumps and creaks

Let's plot it!

```python
fig, (bump, creak) = plt.subplots(1, 2)

bump.imshow(prob_bump_true_given_location.reshape(height, width))
bump.set_title('p(bump=True|h)')
hide_ticks(bump)

creak.imshow(prob_creak_true_given_location.reshape(height, width))
creak.set_title('p(creak=True|h)')
hide_ticks(creak)

maybe_save_plot('2018-05-02-bumps-creaks')
plt.show()
```

#### Emission distribution $p(v \mid h)$

The goal is to define the distribution $p(v \mid h)$. In `numpy`
 - The distribution gives probability of the 4 visible states given each of the 30 hidden states, so the array should be of size `(4, 30)`.
 - The distribution is conditional over the 4 states, so each column should add to 1.

To get the emission distribution in math, I'd multiply

$$p(v \mid h) = p(v^{bump} \mid h)p(v^{creak} \mid h).$$

It's a little more complicated in code. First to get $p(v^{bump} \mid h_t)$,  I need to add a second state for $p(v^{bump}=False \mid h_t) = 1 - p(v^{bump}=True \mid h_t)$.
But I also need to flatten out the visible states from 2 2-state visible variables to one 4-state visible variable. So like Barber, I'll define the four states' probabilities by hand.

I also define a `map_visible_state_to_bump_creak` to map from the visible state to if there was a bump and/or creak.

```python
num_visible_states = 4

def get_emission_matrix(prob_bump_true_given_location, prob_creak_true_given_location):
    # prob_bump_given_state[v][state] = p(v | state)
    p_emission = np.vstack((
        prob_bump_true_given_location * prob_creak_true_given_location,
        prob_bump_true_given_location * (1 - prob_creak_true_given_location),
        (1 - prob_bump_true_given_location) * prob_creak_true_given_location,
        (1 - prob_bump_true_given_location) * (1 - prob_creak_true_given_location),
    ))

    assert np.all(np.sum(p_emission, axis=0)) == 1

    return p_emission

p_emission = get_emission_matrix(prob_bump_true_given_location, prob_creak_true_given_location)

# 1 means True. ex: [1, 0] means bump=True, creak=False
map_visible_state_to_bump_creak = np.vstack((
    [1, 1],
    [1, 0],
    [0, 1],
    [0, 0],
))
```

### Visualizing $p(\textbf{v} \mid h_t)$

This one's a little tricker to visualize. I'll show it as four plots of the probability each visible state.

When I do alpha recursion the data below, the result of filtering for the first timestep will be exactly the same as one of these maps.

```python
fig, axs = plt.subplots(2, 2)
for ax, probs, (bump, creak) in zip(axs.flatten(), p_emission, map_visible_state_to_bump_creak):
    ax.imshow(probs.reshape(height, width))
    ax.set_title(
        'p('
        + ('' if bump else 'no ') + 'bump & '
        + ('' if creak else 'no ') + 'creak)'
    )
    hide_ticks(ax)
maybe_save_plot('2018-05-02-v-given-h')
plt.show()
```

## Generating data: $\textbf{v}_{1:t}$

Now I have all of the probabilities I need. On to part 2: generating some data!

I'll do it for 10 timesteps.

```python
timesteps = 10

hiddens = np.zeros(timesteps, dtype=int)
visibles = np.zeros(timesteps, dtype=int)

hiddens[0] = np.random.choice(num_hidden_states, p=p_hidden_start)
visibles[0] = np.random.choice(
    num_visible_states,
    p=p_emission[:, hiddens[0]]
)

for t in range(1, timesteps):
    hiddens[t] = np.random.choice(
        num_hidden_states,
        p=p_transition[:, hiddens[t - 1]]
    )

    visibles[t] = np.random.choice(
        num_visible_states,
        p=p_emission[:, hiddens[t]]
    )
```

### Visualizing

Let's take a peak at where they were moving and what sounds they made.

Visibles are plotted as a rectangle with two colors. The left is black if there was a bump and white if not. The right is black if there was a creak and white if not.

```python
fig, all_axs = plt.subplots(2, timesteps, figsize=(16, 4))
all_axs = all_axs.T

VISIBLES = 0
TRUE_STATES = 1

all_axs[0][0].set_title('Visibles', x=-0.5, y=0.2)
all_axs[0][1].set_title('Actual', x=-0.5, y=0.4)

for i, (axs, hidden, visible) in enumerate(zip(all_axs, hiddens, visibles)):
    axs[VISIBLES].imshow([map_visible_state_to_bump_creak[visible]], cmap='gray', vmin=0, vmax=1)
    hide_ticks(axs[VISIBLES])

    axs[TRUE_STATES].imshow(plot_state_in_room(hidden), cmap='gray')
    hide_ticks(axs[TRUE_STATES])

maybe_save_plot('2018-05-02-v-over-time')
plt.show()
```

## Filtering: $p(h_t \mid v_{1:t})$

Now for part 3: filtering using alpha recursion. Filtering finds the probability of the current hidden state given the visibles from all previous timesteps, as well as the transition, emission, and start probabilities.

Taking from Barber, alpha recursion gives

$$\alpha(h_t) = p(h_t, v_{1:t}) \propto p(h_t \mid v_{1:t})$$

The values of $\alpha$ are computed with

$$\alpha(h_t) = p(v_t \mid h_t) \sum_{h_{t - 1}} p(h_t \mid h_{t - 1})\alpha(h_{t - 1})$$

and

$$\alpha(h_1) = p(h_1, v_1) = p(v_1 \mid h_1)p(h_1).$$

In code I'll compute an $\alpha$ for each timestep. I'll use these to give the distribution over all states for that timestep given the visibles up until that timestep. Because I want the filtered posterior, I can rescale, so like Barber's code, I'm normalizing each $\alpha$.

```python
def alpha_recursion(visibles, p_hidden_start, p_transition, p_emission):
    num_timestamps = visibles.shape[0]
    num_hidden_states = p_transition.shape[0]

    # There will be one alpha for each timestamp
    alphas = np.zeros((num_timestamps, num_hidden_states))

    # alpha(h_1) = p(h_1) * p(v_1 | h_1)
    alphas[0] = p_hidden_start * p_emission[visibles[0]]

    # normalize to avoid overflow
    alphas[0] /= np.sum(alphas[0])
    for t in range(1, num_timestamps):
        # p(v_s | h_s)
        # size: new_states
        corrector = p_emission[visibles[t]]

        # sum over all hidden states for the previous timestep and multiply the
        # transition prob by the previous alpha
        # transition_matrix size: new_state x old_state
        # alphas[t_minus_1].T size: old_state x 1
        # predictor size: new_state x 1,
        predictor = p_transition @ alphas[t - 1, None].T

        # alpha(h_s)
        alphas[t, :] = corrector * predictor[:, 0]

        # normalize
        alphas[t] /= np.sum(alphas[t])

    return alphas

alphas = alpha_recursion(
    visibles,
    p_hidden_start,
    p_transition,
    p_emission,
)

assert np.all(np.isclose(np.sum(alphas, axis=1), 1))
```

### Visualizing

Again, but with the predictions!

```python
fig, all_axs = plt.subplots(3, timesteps, figsize=(16, 4))
all_axs = all_axs.T

VISIBLES = 0
TRUE_STATES = 1
FILTERING = 2

all_axs[0][VISIBLES].set_title('Visibles', x=-0.5, y=0.2)
all_axs[0][TRUE_STATES].set_title('Actual', x=-0.5, y=0.4)
all_axs[0][FILTERING].set_title('Filtering', x=-0.5, y=0.4)

for i, (axs, hidden, visible, alpha) in enumerate(zip(all_axs, hiddens, visibles, alphas)):
    axs[VISIBLES].imshow([map_visible_state_to_bump_creak[visible]], cmap='gray', vmin=0, vmax=1)
    hide_ticks(axs[VISIBLES])

    axs[TRUE_STATES].imshow(plot_state_in_room(hidden), cmap='gray')
    hide_ticks(axs[TRUE_STATES])

    axs[FILTERING].imshow(alpha.reshape(height, width), vmin=0)
    hide_ticks(axs[FILTERING])

maybe_save_plot('2018-05-02-filtering')
plt.show()
```

## See Also

 - [This post](2018-05-13-viterbi-message-passing.ipynb) builds on this post to show the Viterbi algorithm.
 - [This notebook](https://github.com/jessstringham/notebooks/blob/master/2018-05-13-hmm-check-results.ipynb) runs this code using the same example from Barber.

```python

```


# Modelos con variables latentes y repaso del algoritmo K-Means

> **¿Qué significa variable latente?**

> Para responder a esta pregunta, nos remontamos hacia la raiz *latina* etimológica de la palabra latente. Esta palabra viene la palabra en Latín **latens** que significa escondido u oculto.

> En el contexto del modelado probabilístico nos referimos con variables latentes a variables que nunca observamos, pero que (inferimos) están ahí.

## 1. Variables latentes

### ¿Porqué consideramos variables latentes?

Hay diversas razones por las que permitirnos incluir variables latentes en nuestros modelos cobra muchísima importancia. Algunas de ellas:

1. **No porque no observemos una variable significa que no exista.**

2. **Muchas veces nos permiten conseguir modelos más simples.**

**Ejemplo:**

Una empresa acaba de abrir una posición en el equipo de ciencia de datos. En este sentido, el departamento de RH está interesado en entrevistar a varios candidatos para encontrar a alguien idóneo para la posición.

Para ello, ya se tiene un tabulador que involucra varias variables:
- Grado académico.
- Promedio de calificaciones del último grado académico.
- Entrevista telefónica.
- Entrevista en vivo.

Sin embargo, la entrevista en vivo es un evento que puede llegar a involucrar muchos recursos económicos, y por experiencia, hay varios candidatos que se pueden descartar con solo el conocimiento de las otras variables. La idea es desarrollar un modelo usando datos históricos del departamento de RH:

| Candidato | Grado | Promedio | E. Telefónica | E. Vivo |
| --------- | ----- | -------- | ------------- | ------- |
| 1         | Lic   | 8.4      | 7             | 5       |
| 2         | Maes  | 8.0      | 7             | 6       |
| 3         | Lic   | 9.5      | 8             | 9       |
| 4         | Doc   | 8.9      | 9             | 10      |

Si intentamos establecer un modelo que relacione estas variables, después de revisarlo un poco, llegaríamos a que todas estas variables están relacionadas entre sí, obteniendo un modelo completamente conectado,

con lo que no tendríamos ni una sola pista de la estructura del modelo, y tendríamos que definir la probabilidad conjunta sobre todas las variables (número exponencial de parámetros).

*Alternativa 1*: considerar un modelo estructurado del tipo

$$
p(x_1, x_2, x_3, x_4) = \frac{\exp\{-w^T x\}}{Z},
$$

con lo que solo tendríamos 5 parámetros $w_0, w_1, w_2, w_3, w_4$. Sin embargo, $Z$ es una constante de normalización que involucra una suma sobre todos los posibles valores de las cuatro variables aleatorias.

*Alternativa 2*: considerar una variable latente de Inteligencia

con lo que el modelo sería:

$$
p(x_1, x_2, x_3, x_4) = \sum_{I} p(x_1, x_2, x_3, x_4 | I) p(I) = \sum_{I} p(x_1 | I)p(x_2 | I)p(x_3 | I)p(x_4 | I) p(I)
$$

Con esto reducimos notablemente la complejidad del modelo.

3. **Aplicaciones prácticas -> Clustering -> Segmentación de clientes, Motores de búsqueda, Sistemas de recomendación, ...**

En aplicaciones clustering pretendemos descubrir segmentaciones en los datos. Esta segmentación la podemos entender como una variable latente.

## 2. Clustering

Probablemente ya hayan escuchado hablar de clustering. Como tal, es de las aplicaciones más importantes en aprendizaje **no supervisado**, y seguramente en un proyecto de análisis de datos no va a pasar más de un mes para cuando necesiten utilizar este tipo de técnicas. Así que vamos a estudiar un par de ellas.

**Ejemplo:**

Los departamentos de créditos en general (personas, pymes, empresarial) normalmente estudian la relación entre ingresos y deuda del candidato a crédito para decidir si acreditan o no a la persona.

Hay varias heurísticas que se utilizan. Por ejemplo:

- Si las deudas rebasan el 40% del apalancamiento (capital social + ingresos + pasivos), es una empresa de alto riesgo.
- Si las deudas rebasan 3 meses de ingresos, es una empresa de alto riesgo.

Se puede estudiar esta relación, y segmentar a los clientes de acuerdo a su perfil en estas variables.

```python
# Importamos función para generar datos
from bank_customer_data import generate_bank_customer_data
# Importamos pyplot
from matplotlib import pyplot as plt
```

```python
# Generamos datos
data = generate_bank_customer_data()
```

```python
data.head(10)
```

      income
      debt
      labels

      0
      1.332974
      4.992805
      1.0

      1
      0.006080
      3.799847
      0.0

      2
      1.591057
      5.733520
      1.0

      3
      3.455210
      3.469757
      0.0

      4
      1.338672
      6.081983
      1.0

      5
      3.417696
      3.096798
      0.0

      6
      3.532126
      1.773208
      0.0

      7
      2.471411
      6.875353
      1.0

      8
      4.825952
      2.542863
      0.0

      9
      3.462531
      1.187322
      0.0

```python
# Datos
plt.scatter(data["income"], data["debt"], c="gray", alpha=0.6)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

Lo que queremos en clustering es identificar los grupos a los que pertenecen cada uno de los clientes.

```python
# Grupos "reales"
plt.scatter(data["income"], data["debt"], c=data["labels"], cmap="Accent", alpha=0.6)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

Esta idea se conoce como **Hard clustering**; bajo este esquema, identificamos para cada punto un único grupo al que pertenece, es decir:

$$
\text{cluster_id}_x = f(x)
$$

- Los puntos verdes son 100% verdes.
- Los puntos azules son 100% azules.
- Los puntos grises son 100% grises.

Sin embargo, fijemos un momento nuestra atención en los recuadros a continuación ($[2, 3] \times [4, 5]$ y $[4, 5] \times [3, 4]$):

```python
plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.scatter(data["income"], data["debt"], c=data["labels"], cmap="Accent", alpha=0.6)
plt.axis([1, 3, 4, 5])
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")

plt.subplot(1, 2, 2)
plt.scatter(data["income"], data["debt"], c=data["labels"], cmap="Accent", alpha=0.6)
plt.axis([3.5, 5, 2, 4])
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

Desde un punto de vista intuitivo, los puntos en estos sectores no es muy claro a qué grupo pertenecen. Estaríamos tentados a decir que pertenecen a un grupo con cierta probabilidad y con cierta probabilidad a otro.

Esta idea se conoce como **Soft clustering**, y está estrechamente relacionado a **clustering probabilístico**.

$$
p(\text{cluster_id}_x |x)
$$

Al darle un enfoque probabilístico, tenemos varias ventajas colaterales:

- Sintonización de hiperparámetros.
- Modelo generativo.

## 3. Un algoritmo de hard clustering: K-Means

Aunque el K-Means es uno de los algoritmos de hard clustering más conocidos y usados, lo veremos en un par de sesiones más desde una perspectiva probabilística.

De manera que nos conviene estudiarlo antes.

**Problema:** dado un conjunto de observaciones $x_1, x_2, \dots, x_N \in \mathbb{R}^d$, se debe particionar las $N$ observaciones en $k$ ($\leq N$) clusters $\{1, 2, \dots, k\}$, de manera que se minimice la suma de distancias al cuadrado (varianza).

**Algoritmo:**

1. Inicializar los parámetros $\theta = \{\mu_1, \dots, \mu_k\}$ de manera aleatoria.
2. Repetir hasta la convergencia (hasta que los parámetros no varíen):
   1. Para cada punto calcule el centroide más cercano:
      $$
      c_i = \arg \min_{c} ||x_i - \mu_c||.
      $$

   2. Actualizar centroides:
      $$
      \mu_c = \frac{\sum_{i: c_i = c} x_i}{\sum_{i: c_i = c} 1}
      $$

**Tarea:** Programar el algoritmo K-Means.

Nosotros usaremos sklearn durante la clase:

```python
# Importamos sklearn.cluster.KMeans
from sklearn.cluster import KMeans
```

```python
# Algoritmo de sklearn
KMeans?
```

```python
# Instanciamos el algoritmo
kmeans = KMeans(n_clusters=3)
```

```python
# Entrenamos
kmeans.fit(X=data[["income", "debt"]])
```

    KMeans(algorithm='auto', copy_x=True, init='k-means++', max_iter=300,
           n_clusters=3, n_init=10, n_jobs=None, precompute_distances='auto',
           random_state=None, tol=0.0001, verbose=0)

```python
# Gráfico
plt.scatter(data["income"], data["debt"], c=kmeans.labels_, cmap="Accent", alpha=0.6)
plt.plot(kmeans.cluster_centers_[0, 0], kmeans.cluster_centers_[0, 1], "*g", ms=20)
plt.plot(kmeans.cluster_centers_[1, 0], kmeans.cluster_centers_[1, 1], "*b", ms=20)
plt.plot(kmeans.cluster_centers_[2, 0], kmeans.cluster_centers_[2, 1], "*k", ms=20)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

Todo se ve bien hasta acá. **¿Qué pasa si aumentamos el número de clusters?**

Una métrica que podemos usar para ver qué tan bueno está siendo el agrupado es la suma de las distancias al cuadrado de cada punto a su centroide respectivo:

$$
\frac{1}{N}\sum_{i=1}^N ||x_i - \mu_{c_i}||^2.
$$

```python
from sklearn.model_selection import train_test_split
import numpy as np
```

```python
def msd(X, cluster_id, centroids):
    """
    Mean squared distance.
    :param data: Data.
    :param centroids: Centroids.
    :return: Mean squared distance.
    """
    # Number of clusters
    k = centroids.shape[0]
    # Number of points
    N = X.shape[0]

    # Distances initialization
    distances = np.zeros(N)

    # Compute distances to corresponding cluster
    for j in range(k):
        distances[cluster_id == j] = np.linalg.norm(X[cluster_id == j] - centroids[j, :], axis=1)

    return (distances**2).mean()
```

```python
X_train, X_test = train_test_split(data[["income", "debt"]], test_size=0.2)

msd_train = []
msd_test = []
for k in range(3, 11):
    # Instanciamos el algoritmo
    kmeans = KMeans(n_clusters=k)

    # Entrenamos
    kmeans.fit(X=X_train)

    # Métrica con datos de entrenamiento
    msd_train.append(msd(X_train, kmeans.labels_, kmeans.cluster_centers_))

    # Métrica con datos de prueba
    msd_test.append(msd(X_test, kmeans.predict(X_test), kmeans.cluster_centers_))
```

```python
plt.plot(range(3, 11), msd_train, label="train")
plt.plot(range(3, 11), msd_test, label="test")
plt.legend()
plt.xlabel("Número de clusters")
plt.ylabel("Suma de distancias cuadradas")
```

```python
# Gráfico
plt.scatter(X_train["income"], X_train["debt"], c=kmeans.labels_, cmap="inferno", alpha=0.6)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

Como observamos, esta métrica siempre decrece con la cantidad de clusters, lo que hace bastante complejo elegir un número de clusters adecuado cuando este es desconocido.

# 4. Modelo de mezcla Gaussiana (GMM)

Como vimos, el K-Means (y en general los algoritmos de hard clustering) tienen varios detalles:

- No es claro cómo elegir el número de clusters.
- Hay puntos que podrían estar en una frontera entre dos o más clusters, y el hard clustering no nos permite tener incertidumbre en la pertenencia.

Para lidiar con estos problemas, podemos plantear un modelo probablístico de nuestros datos.

Hasta ahora, conocemos algunas distribuciones. Entre ellas la Gaussiana, para la cual ya sabemos como estimar sus parámetros.

¿Qué pasa si intentamos ajustar una **distribución Gaussiana** a los datos? Es decir, si modelamos los datos con

$$
p(x|\theta) = \mathcal{N}(x|\mu, \Sigma), \qquad \theta=\{\mu, \Sigma\}
$$

```python
# Datos
plt.scatter(data["income"], data["debt"], c="gray", alpha=0.6)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

```python
from scipy.stats import multivariate_normal
import numpy as np
```

```python
# Ajustamos parámetros
mu = data[["income", "debt"]].mean()
cov = data[["income", "debt"]].cov()
# Definimos VA
X = multivariate_normal(mean=mu, cov=cov)
```

```python
# Datos
plt.scatter(data["income"], data["debt"], c="gray", alpha=0.6)

# Gaussiana
x = np.linspace(0, 8, 100)
y = np.linspace(0, 9, 100)
x, y = np.meshgrid(x, y)
z = X.pdf(np.dstack([x, y]))
plt.contour(x, y, z)
plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

```python
mu
```

    income    2.913434
    debt      4.104847
    dtype: float64

Sin embargo, este modelo no parece corresponder con nuestros datos. La región de máxima probabilidad (media) cae en un punto medio entre los clusters, y allí no hay muchos datos.

### ¿Qué pasa si usamos varias Gaussianas?

```python
# Ajustamos parámetros
mu1 = data.loc[data["labels"] == 0, ["income", "debt"]].mean()
mu2 = data.loc[data["labels"] == 1, ["income", "debt"]].mean()
mu3 = data.loc[data["labels"] == 2, ["income", "debt"]].mean()
cov1 = data.loc[data["labels"] == 0, ["income", "debt"]].cov()
cov2 = data.loc[data["labels"] == 1, ["income", "debt"]].cov()
cov3 = data.loc[data["labels"] == 2, ["income", "debt"]].cov()
# Definimos VA
X1 = multivariate_normal(mean=mu1, cov=cov1)
X2 = multivariate_normal(mean=mu2, cov=cov2)
X3 = multivariate_normal(mean=mu3, cov=cov3)
```

```python
# Datos
plt.scatter(data["income"], data["debt"], c="gray", alpha=0.6)

# Gaussiana 1
x = np.linspace(0, 6, 100)
y = np.linspace(0, 5, 100)
x, y = np.meshgrid(x, y)
z = X1.pdf(np.dstack([x, y]))
plt.contour(x, y, z)

# Gaussiana 2
x = np.linspace(0, 4, 100)
y = np.linspace(4, 9, 100)
x, y = np.meshgrid(x, y)
z = X2.pdf(np.dstack([x, y]))
plt.contour(x, y, z)

# Gaussiana 3
x = np.linspace(3, 8, 100)
y = np.linspace(2, 8, 100)
x, y = np.meshgrid(x, y)
z = X3.pdf(np.dstack([x, y]))
plt.contour(x, y, z)

plt.xlabel("Ingresos mensuales (x100k MXN)")
plt.ylabel("Deuda (x100k MXN)")
```

**¡Mucho mejor!**

Cada Gaussiana explica un cluster de puntos, y el modelo general sería una suma ponderada de estas densidades Gaussianas:

$$
p(x | \theta) = \sum_{c=1}^{3} \pi_c \mathcal{N}(x | \mu_c, \Sigma_c), \qquad \theta = \{\pi_1, \pi_2, \pi_3, \mu_1, \mu_2, \mu_3, \Sigma_1, \Sigma_2, \Sigma_3\}
$$

**¿Y esto cómo lo interpretamos?**

Bueno, pues si logramos encontrar los parámetros $\pi_1, \pi_2, \pi_3, \mu_1, \mu_2, \mu_3, \Sigma_1, \Sigma_2, \Sigma_3$ para este conjunto de datos, habremos resuelto el problema de (soft) clustering, ya que encontraremos para cada punto la probabilidad de que venga de cada una de las Gaussianas.

**¿Qué ventajas tenemos?**

Como ventaja respecto a usar una sola Gaussiana, hemos añadido flexibilidad a nuestro modelo. Es decir, con esta estuctura podemos representar conjuntos de datos complejos.

> En efecto, podemos aproximar casi cualquier distribución continua con una mezcla de Gaussianas con precisión arbitraria, dado que incluyamos un número suficiente de Gaussianas en la mezcla.

**¿A qué costo?**

La cantidad de parámetros que debemos estimar se multiplica por la cantidad de Gaussianas en la mezcla.

### ¿Cómo encontramos (entrenamos) los parámetros?

Podemos maximizar la función de verosimilitud (suposición de independencia):

$$
\max_{\theta} p(X | \theta) = \prod_{i=1}^N p(x_i | \theta) = \prod_{i=1}^N \sum_{c=1}^{3} \pi_c \mathcal{N}(x_i | \mu_c, \Sigma_c)
$$

sujeto a:

\begin{align}
\sum_{c=1}^3 \pi_c & = 1 \\
\pi_c & \geq 0 \quad \text{for } c=1,2,3\\
\Sigma_c & \succ 0 \quad \text{for } c=1,2,3
\end{align}

Es decir, las matrices de covarianza deben ser definidas positivas (¿por qué?).

**Complejidades numéricas:**

Este problema de optimización puede ser resuelto numéricamente con un algoritmo como el gradiente descendiente. Sin embargo,

1. La restricción sobre la matriz de covarianzas hace el problema de optimización muy complejo de resolver numéricamente hablando.

   Una simplificación para poder trabajar con esta restricción es suponer que las matrices de covarianza son diagonales:

$$
\Sigma_c = \text{diag}(\sigma_{c1}, \sigma_{c2}, \dots, \sigma_{cn}),
$$

2. La suma dentro del producto también hace bastante complejo el cálculo de los gradientes. Comúnmente, para evitar el producto se toma logaritmo de la verosimilitud:

   $$
   \log p(X | \theta) = \log \left(\prod_{i=1}^N p(x_i | \theta) \right)= \sum_{i=1}^N \log\left(\sum_{c=1}^{3} \pi_c \mathcal{N}(x_i | \mu_c, \Sigma_c)\right)
   $$

   y con esto, podemos observar que permanece una suma ponderada dentro del logaritmo.

### ¿Y entonces?

Afortunadamente, existe un algoritmo alternativo con base probabilística llamado **algoritmo de maximización de la esperanza**, el cual estaremos estudiando en las próximas clases no solo para el problema de mezclas Gaussianas, sino para entrenar cualquier modelo con **variables latentes**.

**¿Variables latentes?**

Recordamos que propusimos el siguiente modelo:

$$
p(x | \theta) = \sum_{c=1}^{3} \pi_c \mathcal{N}(x | \mu_c, \Sigma_c), \qquad \theta = \{\pi_1, \pi_2, \pi_3, \mu_1, \mu_2, \mu_3, \Sigma_1, \Sigma_2, \Sigma_3\}
$$

Este modelo en realidad, lo podemos pensar como un modelo con una variable latente $t$ que determina a cuál Gaussiana pertenece cada punto:

Entonces, razonablemente podemos atribuirle a $t$ tres posibles valores (1, 2, y 3), que nos dicen de qué Gaussiana viene el punto. Recordamos que $t$ es una variable latente, nunca la observamos.

Sin embargo, razonando probabilísticamente, después de entrenar nuestra mezcla Gaussiana, podríamos preguntarle al modelo, ¿Cuál es el valor más probable de $t$ dado el punto $x$? --> **Clustering**

Con este modelo, podemos asignar las siguientes probabilidades:

- Previa:

  $$
  p(t=c | \theta) = \pi_c
  $$

- Verosimilitud:

  $$
  p(x | t=c, \theta) = \mathcal{N}(x | \mu_c, \Sigma_c)
  $$

Razonable, ¿no?

Y con lo anterior,

$$
p(x | \theta) = \sum_{c=1}^3 p(x, t=c | \theta) = \sum_{c=1}^3 \underbrace{p(x | t=c, \theta)}_{\mathcal{N}(x | \mu_c, \Sigma_c)} \underbrace{p(t=c | \theta)}_{\pi_c},
$$

justo como el modelo intuitivo que habíamos propuesto.

### Algoritmo de maximización de la esperanza para mezclas Gaussianas - Intuición

Supongamos que tenemos los siguientes puntos de tamaños de playeras, y queremos definir cuáles son talla chica y cuales son talla grande:

```python
from shirts_size_data import generate_shirts_data
from scipy.stats.distributions import norm
```

```python
shirts_data = generate_shirts_data()
shirts_data.head()
```

      size
      labels

      0
      31.554606
      0.0

      1
      31.222931
      0.0

      2
      30.678384
      0.0

      3
      42.104057
      1.0

      4
      40.355446
      1.0

```python
plt.scatter(shirts_data["size"], np.zeros(len(shirts_data)), c="gray", alpha=0.6)
plt.ylim([-0.01, 0.3])
plt.xlabel("Tamaño (cm)")
```

¿Cómo estimamos los parámetros de nuestro modelo de variable latente?

Analicemos varios casos:

1. Si de entrada supiéramos cuáles playeras son chicas y cuáles grandes:

```python
plt.scatter(shirts_data.loc[shirts_data["labels"] == 0, "size"],
            np.zeros((shirts_data["labels"] == 0).sum()),
            alpha=0.6)
plt.scatter(shirts_data.loc[shirts_data["labels"] == 1, "size"],
            np.zeros((shirts_data["labels"] == 1).sum()),
            alpha=0.6)
x = shirts_data["size"].copy().values
x.sort()
mu1 = shirts_data.loc[shirts_data["labels"] == 0, "size"].mean()
mu2 = shirts_data.loc[shirts_data["labels"] == 1, "size"].mean()
s1 = shirts_data.loc[shirts_data["labels"] == 0, "size"].std()
s2 = shirts_data.loc[shirts_data["labels"] == 1, "size"].std()
plt.plot(x, norm.pdf(x, loc=mu1, scale=s1))
plt.plot(x, norm.pdf(x, loc=mu2, scale=s2))
plt.ylim([-0.01, 0.3])
plt.xlabel("Tamaño (cm)")
```

$$
p(x | t=1, \theta) = \mathcal{N}(x | \mu_1, \sigma_1)
$$

Con lo cual:

$$
\mu_1 = \frac{\sum_{i: t_i = 1} x_i}{\sum_{i: t_i = 1} 1}, \qquad \sigma_1^2 = \frac{\sum_{i: t_i = 1} (x_i - \mu_1)^2}{\sum_{i: t_i = 1} 1},
$$

y

$$
\mu_2 = \frac{\sum_{i: t_i = 2} x_i}{\sum_{i: t_i = 2} 1}, \qquad \sigma_2^2 = \frac{\sum_{i: t_i = 2} (x_i - \mu_1)^2}{\sum_{i: t_2 = 1} 1}
$$

2. Como sablemos, en el algoritmo de mezclas Gaussianas nunca sabremos si un punto pertenece a cierto cluster o no, sino que conoceremos las probabilidades de que pertenezca a cada cluster.

   De modo que si conocemos la posterior $p(t | x, \theta)$, entonces ponderamos lo anterior por esta probabilidad:

   $$
\mu_1 = \frac{\sum_{i} p(t_i=1 | x_i, \theta)x_i}{\sum_{i} p(t_i=1 | x_i, \theta)}, \qquad \sigma_1^2 = \frac{\sum_{i} p(t_i=1 | x_i, \theta) (x_i - \mu_1)^2}{\sum_{i} p(t_i=1 | x_i, \theta)}.
   $$

3. ¿Y cómo conocemos la posterior $p(t | x, \theta)$?

   Bueno, pues si conocemos los parámetros, es bastante fácil:

   $$
   p(t=c | x, \theta) = \frac{p(x | t=c, \theta) p(t=c | \theta)}{Z} = \frac{\pi_c \mathcal{N}(x | \mu_c, \sigma_c)}{Z}.
   $$

Tenemos un razonamiento circular (un problema del tipo, ¿Qué fue primero?, ¿El huevo?, ¿O la gallina?).

**¿Cómo lo resolvemos? Iterando...**

**Algoritmo de maximización de la esperanza:**

1. Inicializamos los parámetros de cada Gaussiana aleatoriamente.

2. Repetir hasta la convergencia:

   - Calcular para cada punto la probabilidad posterior $p(t_i=c | x_i, \theta)$.
   - Actualizar los parámetros de las Gaussianas con las probabildades calculadas.

Created with Jupyter by Esteban Jiménez Rodríguez.


```python
# HIDDEN
from datascience import *
from prob140 import *
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('fivethirtyeight')
%matplotlib inline
import math
from scipy import stats
from scipy import misc
```

## Transitions ##

Let $X_0, X_1, X_2, \ldots $ be a Markov chain with state space $S$. By the Markov property, the probability of a *trajectory* or *path* of finite length is

\begin{align*}
& P(X_0 = i_0, X_1 = i_1, X_2 = i_2, \ldots, X_n = i_n) \\
& = ~
P(X_0 = i_0)P(X_1 = i_1 \mid X_0 = i_0)P(X_2 = i_2 \mid X_1 = i_1) \cdots
P(X_n = i_n \mid X_{n-1} = i_{n-1})
\end{align*}

The conditional probabilities in the product are called *transition probabilities*. For states $i$ and $j$, the conditional probability $P(X_{n+1} = j \mid X_n = i)$ is called a *one-step transition probability at time $n$*.

For many chains such as the random walk, these one-step transition probabilities depend only on the states $i$ and $j$, not on the time $n$. For example, for the random walk,

\begin{equation}
P(X_{n+1} = j \mid X_n = i) =
 \begin{cases}
      \frac{1}{2} & \text{if } j = i-1 \text{ or } j = i+1 \\
      0 & \text{ otherwise}
   \end{cases}
\end{equation}

for every $n$.

### Stationary Transition Probabilities ###
When one-step transition probabilites don't depend on $n$, they are called *stationary* or *time-homogenous*. All the Markov chains that we will study in this course have time-homogenous transition probabilities.

For such a chain, define the *one-step transition probability*

$$
P(i, j) ~ = ~ P(X_{n+1} = j \mid X_n = i) ~ = ~ P(X_1 = j \mid X_0 = i)
$$
Then
$$
P(X_0 = i_0, X_1 = i_1, X_2 = i_2, \ldots, X_n = i_n)
~ = ~ P(X_0 = i_0)P(i_0, i_1)P(i_1, i_2) \cdots P(i_{n-1}, i_n)
$$

The one-step transition probabilities can be represented as elements of a matrix. This isn't just for compactness of notation – it leads to a powerful theory.

### One-Step Transition Matrix ###
The *one-step transition matrix* of the chain is the matrix $\mathbb{P}$ whose $(i, j)$th element is $P(i, j) = P(X_1 = j \mid X_0 = i)$.

Often, $\mathbb{P}$ is just called *the transition matrix* for short. Note two important properties:

- $\mathbb{P}$ is a square matrix: its rows as well as its columns are indexed by the state space.
- Each row of $\mathbb{P}$ is a distribution: for each state $i$, and each $n$, Row $i$ of the transition matrix is the conditional distribution of $X_{n+1}$ given that $X_n = i$. Because each of its rows adds up to 1, $\mathbb{P}$ is called a *stochastic matrix*.

Let's see what the transition matrix looks like in an example.

### Sticky Reflecting Random Walk ###
Often, the transition behavior of a Markov chain is easier to describe in a *transition diagram* instead of a matrix. Here is such a diagram for a chain on the states 1, 2, 3, 4, and 5. The diagram shows the one-step transition probabilities.
- If the chain is at any state, it stays there with chance 0.5.
- If the chain is at states 2 through 4, it moves to each of its two adjacent state with chance 0.25.
- If the chain is at states 1 or 5, it moves to its adjacent state with chance 0.5

We say that there is *reflection* at states 1 and 5. The walk is *sticky* because of the positive chance of staying in place.

Transition diagrams are great for understanding the rules by which a chain moves. For calculations, however, the transition matrix is more helpful.

To start constructing the matrix, we set the array `s` to be the set of states and the transition function `refl_walk_probs` to take arguments $i$ and $j$ and return $P(i, j)$.

```python
s = np.arange(1, 6)

def refl_walk_probs(i, j):
    # staying in the same state
    if i-j == 0:
        return 0.5

    # moving left or right
    elif 2 <= i <= 4:
        if abs(i-j) == 1:
            return 0.25
        else:
            return 0

    # moving right from 1
    elif i == 1:
        if j == 2:
            return 0.5
        else:
            return 0

    # moving left from 5
    elif i == 5:
        if j == 4:
            return 0.5
        else:
            return 0
```

You can use the `prob140` library to construct `MarkovChain` objects. The `from_transition_function` method takes two arguments:
- an array of the states
- a transition function

and displays the one-step transition matrix of a `MarkovChain` object.

```python
reflecting_walk = MarkovChain.from_transition_function(s, refl_walk_probs)
reflecting_walk
```

      1
      2
      3
      4
      5

      1
      0.50
      0.50
      0.00
      0.00
      0.00

      2
      0.25
      0.50
      0.25
      0.00
      0.00

      3
      0.00
      0.25
      0.50
      0.25
      0.00

      4
      0.00
      0.00
      0.25
      0.50
      0.25

      5
      0.00
      0.00
      0.00
      0.50
      0.50

Compare the transition matrix $\mathbb{P}$ with the transition diagram, and confirm that they contain the same information about transition probabilities.

To find the chance that the chain moves to $j$ given that it is at $i$, go to Row $i$ and pick out the probability in Column $j$.

If you know the starting state, you can use $\mathbb{P}$ to find the probability of any finite path. For example, given that the walk starts at 1, the probability that it then has the path [2, 2, 3, 4, 3] is

$$
P(1, 2)P(2, 2)P(2, 3)P(3, 4)P(4, 3) \approx 0.4\%
$$

```python
0.5 * 0.5 * 0.25 * 0.25 * 0.25
```

    0.00390625

The `MarkovChain` method `prob_of_path` saves you the trouble of doing the multiplication. It takes as its arguments the starting state and the rest of the path (in a list or array), and returns the probability of the path.

```python
reflecting_walk.prob_of_path(1, [2, 2, 3, 4, 3])
```

    0.00390625

```python
reflecting_walk.prob_of_path(1, [2, 2, 3, 4, 3, 5])
```

    0.0

You can simulate paths of the chain using the `simulate_path` method. It takes two arguments: the starting state and the number of steps of the path. By default it returns an array consisting of the sequence of states in the path. The optional argument `plot_path=True` plots the simulated path. Run the cells below a few times and see how the output changes.

```python
reflecting_walk.simulate_path(1, 7)
```

    array([1, 2, 2, 2, 1, 1, 2, 2])

```python
reflecting_walk.simulate_path(1, 10, plot_path=True)
```

### $n$-Step Transition Matrix ###
For states $i$ and $j$, the chance of getting from $i$ to $j$ in $n$ steps is called the $n$-step transition probability from $i$ to $j$. Formally, the $n$-step transition probability is

$$
P_n(i, j) ~ = ~ P(X_n = j \mid X_0 = i)
$$

In this notation, the one-step transition probability $P(i, j)$ can also be written as $P_1(i, j)$.

The $n$-step transition probability $P_n(i, j)$ can be represented as the $(i, j)$th element of a matrix called the $n$-step transition matrix. For each state $i$, Row $i$ of the $n$-step transition matrix contains the distribution of $X_n$ given that the chain starts at $i$.

The `MarkovChain` method `transition_matrix` takes $n$ as its argument and displays the $n$-step transition matrix. Here is the 2-step transition matrix of the reflecting walk defined earlier in this section.

```python
reflecting_walk.transition_matrix(2)
```

      1
      2
      3
      4
      5

      1
      0.3750
      0.5000
      0.125
      0.0000
      0.0000

      2
      0.2500
      0.4375
      0.250
      0.0625
      0.0000

      3
      0.0625
      0.2500
      0.375
      0.2500
      0.0625

      4
      0.0000
      0.0625
      0.250
      0.4375
      0.2500

      5
      0.0000
      0.0000
      0.125
      0.5000
      0.3750

You can calculate the individual entries easily by hand. For example, the $(1, 1)$ entry is the chance of going from state 1 to state 1 in 2 steps. There are two paths that make this happen:
- [1, 1, 1]
- [1, 2, 1]

Given that 1 is the starting state, the total chance of the two paths is $(0.5 \times 0.5) + (0.5 \times 0.25) = 0.375$.

Because of the Markov property, the one-step transition probabilities are all you need to find the 2-step transition probabilities.

In general, we can find $P_2(i, j)$ by conditioning on where the chain was at time 1.

\begin{align*}
P_2(i, j) ~ &= ~ P(X_2 = j \mid X_0 = i) \\
&= ~ \sum_k P(X_1 = k, X_2 = j \mid X_0 = i) \\
&= ~ \sum_k P(X_1 = k \mid X_0 = i)P(X_2 = j \mid X_1 = k) \\
&= ~ \sum_k P(i, k)P(k, j)
\end{align*}

That's the $(i, j)$th element of the matrix product $\mathbb{P} \times \mathbb{P} = \mathbb{P}^2$. Thus the 2-step transition matrix of the chain is $\mathbb{P}^2$.

By induction, you can show that the $n$-step transition matrix of the chain is $\mathbb{P}^n$.

Here is a display of the 5-step transition matrix of the reflecting walk.

```python
reflecting_walk.transition_matrix(5)
```

      1
      2
      3
      4
      5

      1
      0.246094
      0.410156
      0.234375
      0.089844
      0.019531

      2
      0.205078
      0.363281
      0.250000
      0.136719
      0.044922

      3
      0.117188
      0.250000
      0.265625
      0.250000
      0.117188

      4
      0.044922
      0.136719
      0.250000
      0.363281
      0.205078

      5
      0.019531
      0.089844
      0.234375
      0.410156
      0.246094

This is a display, but to work with the matrix we have to represent it in a form that Python recognizes as a matrix. The method `get_transition_matrix` does this for us. It take the number of steps $n$ as its argument and returns the $n$-step transition matrix as a NumPy matrix.

For the reflecting walk, we will start by extracting $\mathbb{P}$ as the matrix `refl_walk_P`.

```python
refl_walk_P = reflecting_walk.get_transition_matrix(1)
refl_walk_P
```

    array([[0.5 , 0.5 , 0.  , 0.  , 0.  ],
           [0.25, 0.5 , 0.25, 0.  , 0.  ],
           [0.  , 0.25, 0.5 , 0.25, 0.  ],
           [0.  , 0.  , 0.25, 0.5 , 0.25],
           [0.  , 0.  , 0.  , 0.5 , 0.5 ]])

Let's check that the 5-step transition matrix displayed earlier is the same as $\mathbb{P}^5$. You can use `np.linalg.matrix_power` to raise a matrix to a non-negative integer power. The first argument is the matrix, the second is the power.

```python
np.linalg.matrix_power(refl_walk_P, 5)
```

    array([[0.24609375, 0.41015625, 0.234375  , 0.08984375, 0.01953125],
           [0.20507812, 0.36328125, 0.25      , 0.13671875, 0.04492188],
           [0.1171875 , 0.25      , 0.265625  , 0.25      , 0.1171875 ],
           [0.04492188, 0.13671875, 0.25      , 0.36328125, 0.20507812],
           [0.01953125, 0.08984375, 0.234375  , 0.41015625, 0.24609375]])

That is indeed the same as the matrix displayed by `transition_matrix` though harder to read.

When we want to use $\mathbb{P}$ in computations, we will use this matrix representation. For displays, `transition_matrix` is better.

### The Long Run ###

To understand the long run behavior of the chain, let $n$ be large and let's examine the distribution of $X_n$ for each value of the starting state. That's contained in the $n$-step transition matrix $\mathbb{P}^n$.

Here is the display of $\mathbb{P}^n$ for the reflecting walk, for $n = 25, 50$, and $100$.

```python
reflecting_walk.transition_matrix(25)
```

      1
      2
      3
      4
      5

      1
      0.129772
      0.256749
      0.25
      0.243251
      0.120228

      2
      0.128374
      0.254772
      0.25
      0.245228
      0.121626

      3
      0.125000
      0.250000
      0.25
      0.250000
      0.125000

      4
      0.121626
      0.245228
      0.25
      0.254772
      0.128374

      5
      0.120228
      0.243251
      0.25
      0.256749
      0.129772

```python
reflecting_walk.transition_matrix(50)
```

      1
      2
      3
      4
      5

      1
      0.125091
      0.250129
      0.25
      0.249871
      0.124909

      2
      0.125064
      0.250091
      0.25
      0.249909
      0.124936

      3
      0.125000
      0.250000
      0.25
      0.250000
      0.125000

      4
      0.124936
      0.249909
      0.25
      0.250091
      0.125064

      5
      0.124909
      0.249871
      0.25
      0.250129
      0.125091

```python
reflecting_walk.transition_matrix(100)
```

      1
      2
      3
      4
      5

      1
      0.125
      0.25
      0.25
      0.25
      0.125

      2
      0.125
      0.25
      0.25
      0.25
      0.125

      3
      0.125
      0.25
      0.25
      0.25
      0.125

      4
      0.125
      0.25
      0.25
      0.25
      0.125

      5
      0.125
      0.25
      0.25
      0.25
      0.125

The rows of $\mathbb{P}^{100}$ are all the same! That means that for the reflecting walk, the distribution at time 100 doesn't depend on the starting state. *The chain has forgotten where it started.*

You can increase $n$ and see that the $n$-step transition matrix stays the same. By time 100, this chain has *reached stationarity*.

Stationarity is a remarkable property of many Markov chains, and is the main topic of this chapter.

```python

```


# Tide-influenced Turbidity Current

## Formulation

A layer-averaged model for a turbidity current and an upper ambient water layer is described herein. Let $t$ and $x$ be time and streamwise bed-attached coordinate. Mass conservation equations of two layers take the form:

\begin{equation}
\frac{\partial h_a}{\partial t} + \frac{\partial h_a U_a}{\partial x} = - e_w \left|U_t - U_a \right| \tag{1}\label{eq:cont_ha}
\end{equation}

\begin{equation}
\frac{\partial h_t}{\partial t} + \frac{\partial h_t U_t}{\partial x} = e_w \left|U_t - U_a \right| \tag{2}\label{eq:cont_ht}
\end{equation}

where $h_a$ and $h_t$ are thickness of layers of an ambient water and a turbidity current respectively. The parameter $e_w$ denotes the entrainment rate of ambient water to a turbidity current. $U_t$ and $U_a$ are velocities of an ambient water layer and a turbidity current.

Momentum equations of two layers are:

\begin{equation}
\frac{\partial h_a U_a}{\partial t} + \frac{\partial h_a U_a^{2}}{\partial x} = -g h_a \frac{\partial \eta}{\partial x} - g h_a \frac{\partial \left(h_a + h_t \right)}{\partial x} - 2 \nu \frac{U_a -  U_t}{h_a + h_t} \tag{3}\label{eq:momentum_Ua}
\end{equation}

\begin{equation}
\frac{\partial h_t U_t}{\partial t} + \frac{\partial h_t U_t^{2}}{\partial x} = - (1 + RC)gh_t \frac{\partial \eta}{\partial x} - g h_t \frac{\partial \left( h_a + h_t \right)}{\partial x} -RCgh_t \frac{ \partial h_t}{\partial x} + 2 \nu \frac{U_a - U_t}{h_a + h_t} - C_f U_t \left|U_t\right| \tag{4}\label{eq:momentum_Ut}
\end{equation}

where $\eta$ indicates the bed elevation. $R(=(\rho_s - \rho_f)/\rho_f)$ denotes submerged specific density of sediment particles ($\rho_s$ and $\rho_f$ are density of sediment and water), and $\nu$ is kinematic viscosity of water (assuming $RC << 1$). The parameters $C_f$ and $g$ denotes bed friction coefficient and gravity acceleraton respectively. The parameter $C$ is concentration of suspended sediment in a layer of a turbidity current, and the mass conservation of suspended sediment takes the form:

\begin{equation}
\frac{\partial C h_t}{\partial t} + \frac{\partial U_t C h_t}{\partial x} = w_s \left( e_s - r_0 C \right) \tag{5}\label{eq:cont_C}
\end{equation}

where $w_s$ is the settling velocity of sediment particle, and $r_0$ is a ratio of near-bed to layer-averaged concentration. The parameter $e_s$ indicates the rate of sediment entrainment from the bed.

From equations \eqref{eq:cont_ha}, \eqref{eq:cont_ht}, \eqref{eq:momentum_Ua}, \eqref{eq:momentum_Ut} and \eqref{eq:cont_C}, we obtain

\begin{equation}
\frac{\partial h_a}{\partial t} + U_a \frac{\partial h_a}{\partial x} = - e_w \left|U_t - U_a \right| - h_a \frac{\partial U_a}{\partial x} \equiv G_{ha} \tag{6}\label{eq:cont_ha_cip}
\end{equation}

\begin{equation}
\frac{\partial h_t}{\partial t} + U_t \frac{\partial h_t}{\partial x} = e_w \left|U_t - U_a \right| - h_t \frac{\partial U_t}{\partial x} \equiv G_{ht} \tag{7}\label{eq:cont_ht_cip}
\end{equation}

\begin{equation}
\frac{\partial U_a}{\partial t} + U_a \frac{\partial U_a}{\partial x} = -g \frac{\partial \eta}{\partial x} - g \frac{\partial \left(h_a + h_t \right)}{\partial x} - \frac{2 \nu}{h_a} \left(\frac{U_a -  U_t}{h_a + h_t}\right) + \frac{e_w \left|U_t - U_a \right| U_a}{h_a} \equiv G_{Ua} \tag{8}\label{eq:momentum_Ua_cip}
\end{equation}

\begin{equation}
\frac{\partial U_t}{\partial t} + U_t \frac{\partial U_t}{\partial x} = - (1 + RC)g\frac{\partial \eta}{\partial x} - g \frac{\partial \left( h_a + h_t \right)}{\partial x} -RCg\frac{ \partial h_t}{\partial x} + \frac{2 \nu}{h_t} \left( \frac{U_a - U_t}{h_a + h_t} \right) - \frac{C_f U_t \left|U_t\right|}{h_t} - \frac{e_w \left|U_t - U_a \right| U_t}{h_t} \equiv G_{Ut} \tag{9}\label{eq:momentum_Ut_cip}
\end{equation}

\begin{equation}
\frac{\partial C}{\partial t} + U_t \frac{\partial C}{\partial x} = \frac{1}{h} \left\{ w_s \left( e_s - r_0 C \right) - e_w C \left|U_t - U_a \right| \right\} \equiv G_{C} \tag{10}\label{eq:cont_C_cip}
\end{equation}

where $G_{ha}, G_{ht}, G_{Ua}, G_{Ut}, G_{C}$ are non-advective terms of the two-layer shallow water equation system. We write the two layers sytem under the compact form:

\begin{equation}
\frac{\partial \boldsymbol{f}}{\partial t} + \boldsymbol{U} \frac{\partial \boldsymbol{f}}{\partial x} = \boldsymbol{G} \tag{11}\label{eq:compact_system}
\end{equation}

where
\begin{equation}
\boldsymbol{f}=(h_a, h_t, U_a, U_t, C)^{T}, \boldsymbol{U}=(U_a, U_t, U_a, U_t, U_t), \boldsymbol{G}=(G_{ha}, G_{ht}, G_{Ua}, G_{Ut}, G_{C})^{T}
\end{equation}

## Numerical solution

Equation \eqref{eq:compact_system} can be solved numerically by CIP method. In this methodology, the equation \eqref{eq:compact_system} is split into advection and non-advection phases. The advection phase takes the form:

\begin{equation}
\frac{\partial \boldsymbol{f}}{\partial t} + \boldsymbol{U} \frac{\partial \boldsymbol{f}}{\partial x} = 0 \tag{12}\label{eq:cip_advection_f}
\end{equation}

\begin{equation}
\frac{\partial \left(\partial_x \boldsymbol{f}\right)}{\partial t} + \boldsymbol{U} \frac{\partial  \left(\partial_x \boldsymbol{f}\right)}{\partial x} = 0 \tag{13}\label{eq:cip_advection_df}
\end{equation}

These equations are descretized as:

\begin{equation}
f_i^{*} = a \xi^3 + b \xi^2 + \partial_x f^{n} \xi + f_i \tag{14}\label{eq:cip_adv_scheme}
\end{equation}

\begin{equation}
\partial_x f_i^{*} = 3 a \xi^2 + 2 b \xi + \partial_x f^{n} \tag{15}\label{eq:cip_adv_scheme_dif}
\end{equation}

where $\xi= -U \Delta t$. The coefficients $a$ and $b$ are defined as:

\begin{equation}
a = \frac{\partial_x f_i^n + \partial_x f_{iup}^n}{D^2} + \frac{2 \left(f_i^n - f_{iup}^n \right)}{D^3} \tag{16}\label{cip_adv_coeff_a}
\end{equation}

\begin{equation}
b = \frac{3 \left(f_{iup}^n - f_i^n\right)}{D^2} - \frac{2 \partial_x f_i^n + \partial_x f_{iup}^n }{D} \tag{17}\label{cip_adv_coeff_b}
\end{equation}

where $iup$ denotes the upstream grid of the $i$th grid, and $D$ is

\begin{equation}
D = \begin{cases}
- \Delta x & (U_i > 0) \\
\Delta x & (U_i < 0)
\end{cases} \tag{18}\label{cip_D}
\end{equation}

After the calculation of the advection phase, the non-advection phase is calculated by:

\begin{equation}
\frac{\partial \boldsymbol{f}}{\partial t} = G \tag{19}
\end{equation}

\begin{equation}
\frac{\partial \left(\partial_x \boldsymbol{f}\right)}{\partial t} = \partial_x G - \frac{\partial f}{\partial x} \frac{\partial U}{\partial x} \tag{20}
\end{equation}

These equations are descretized as:

\begin{equation}
f_i^{n+1} = f^* + G_i \Delta t \tag{21}
\end{equation}

\begin{equation}
\partial_x f_i^{n + 1} = \partial_x f^* + \frac{f_{i + 1}^{n+1} - f_{i-1}^{n + 1}}{2 \Delta x} - \frac{f_{i + 1}^{*} - f_{i-1}^{n*}}{2 \Delta x} - \left( \frac{\partial U}{\partial x} \right)_i \partial_x f^*  \tag{22}
\end{equation}

```python
from tideturb import Grid, TwoLayerTurbidityCurrent
from matplotlib import pyplot as plt

grid = Grid(number_of_grids=200, spacing=20.0)
grid.eta = grid.x * -0.05
tc = TwoLayerTurbidityCurrent(
    grid=grid,
    turb_vel=2.0,
    ambient_vel=0.3,
    turb_thick=5.0,
    ambient_thick=100.0,
    concentration=0.01,
    Ds=80*10**-6,
    alpha=0.5,
    implicit_repeat_num=5,
    )
steps = 50
for i in range(steps):
    # tc.plot(ylim_velocity=[-0.5, 8.0], ylim_concentration=[0, 9.0])
    # plt.savefig('test11/tidal_ebb_{:04d}'.format(i))
    tc.run_one_step(dt=100.0)
    print("", end='\r')
    print('{:.1f}% finished.'.format(i / steps * 100), end='\r')

tc.plot(ylim_velocity=[-0.5, 8.0], ylim_concentration=[0, 9.0])
# plt.savefig('test11/tidal_ebb_{:04d}'.format(i))
plt.show()
tc.save('test12_5000sec')

```

    98.0% finished.

    /home/naruse/anaconda3/lib/python3.6/site-packages/matplotlib/figure.py:445: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
      % get_backend())

```python
from tideturb import Grid, TwoLayerTurbidityCurrent
import matplotlib as mpl
# mpl.use('Agg')
import matplotlib.pyplot as plt
import tideturb
import numpy as np
import pickle

tc = tideturb.load_model('test12_5000sec')
steps = 12
# offset = 50
periodicity = 12 * 3600
for i in range(steps):
    # tc.plot(ylim_velocity=[-0.5, 8.0], ylim_concentration=[0, 9.0])
    # plt.savefig('test11_cycle/tidal_cycle_{:04d}'.format(i))
    for j in range(36):
        tc.ambient_vel = 0.3 * np.sin((i * 3600 + j * 100) / periodicity * 2 * np.pi + 0.5 * np.pi)
        tc.U_link_temp[0, 0] = tc.ambient_vel
        tc.U_link_temp[0, 1] = tc.ambient_vel
        tc.run_one_step(dt=100.0)
    tc.save('test12_cycle_{:0=2}h'.format(i))
    print("", end='\r')
    print('{:.1f}% finished.'.format(i / steps * 100), end='\r')

# tc.plot(ylim_velocity=[-0.5, 8.0], ylim_concentration=[0, 9.0])
# plt.savefig('test11_cycle/tidal_cycle_{:04d}'.format(i))
plt.show()
tc.save('test12_cycle_12hrs')
```

    91.7% finished.

```python
from tideturb import Grid, TwoLayerTurbidityCurrent
from matplotlib import pyplot as plt
import numpy as np

grid = Grid(number_of_grids=100, spacing=40.0)
grid.eta = grid.x * -0.05

C = np.linspace(0.006, 0.015, 30)
velocity = np.linspace(1.5, 2.5, 30)
vel_difference = np.zeros([len(velocity), len(C)])

for i in range(len(velocity)):
    for j in range(len(C)):
        tc1 = TwoLayerTurbidityCurrent(
            grid=grid,
            turb_vel=velocity[i],
            ambient_vel=0.3,
            turb_thick=5.0,
            ambient_thick=100.0,
            concentration=C[j],
            Ds=80*10**-6,
            alpha=0.5,
            implicit_repeat_num=5,
        )
        tc1.run_one_step(dt=3000)
        plt.close(tc1.fig)

        tc2 = TwoLayerTurbidityCurrent(
            grid=grid,
            turb_vel=velocity[i],
            ambient_vel=-0.3,
            turb_thick=5.0,
            ambient_thick=100.0,
            concentration=C[j],
            Ds=80*10**-6,
            alpha=0.5,
            implicit_repeat_num=5,
        )
        tc2.run_one_step(dt=3000)
        plt.close(tc2.fig)

        vel_difference[i][j] = tc1.U_node[1][-1] - tc2.U_node[1][-1]
        print("", end='\r')
        print('{:.1f}% finished.'.format((i * len(C) + j + 1) / (len(C) * len(velocity)) * 100), end='\r')

print(vel_difference)
np.savetxt('vel_difference4.txt', vel_difference, delimiter=',')
```

    [[ 1.67553697e-02  2.04528849e-03 -2.13336859e-02 -3.92631066e-02
      -5.03579013e-02 -5.64753047e-02 -6.01815366e-02 -6.31804350e-02
      -6.76992399e-02 -8.27539357e-02 -1.29821513e-01 -2.24908740e-01
      -3.24605963e-01 -3.52185120e-01 -3.15732888e-01 -2.34236046e-01
      -2.32577550e-01 -3.02061751e-01 -4.34000939e-01 -2.71201961e-01
       6.29298729e-02  7.77410601e-02  9.20932326e-02  1.10539708e-01
       1.34650764e-01  1.66752963e-01  2.11245175e-01  2.76604538e-01
       3.75484327e-01  5.75596527e-01]
     [ 1.48006098e-02 -4.73205635e-03 -2.82634940e-02 -4.48598233e-02
      -5.51446920e-02 -6.14138037e-02 -6.68661481e-02 -7.65740912e-02
      -1.06100643e-01 -1.77571367e-01 -2.80862884e-01 -3.46695973e-01
      -3.30772986e-01 -2.75228584e-01 -2.39512915e-01 -2.82362465e-01
      -3.97130822e-01 -3.34359836e-01  5.39608048e-02  6.95909987e-02
       8.17879578e-02  9.74218492e-02  1.22579904e-01  1.49541739e-01
       1.86858325e-01  2.40110699e-01  3.21784685e-01  4.63206132e-01
       7.75067837e-01  1.80424823e+00]
     [ 1.10414629e-02 -1.24794483e-02 -3.58545973e-02 -5.17062387e-02
      -6.18642988e-02 -7.11431720e-02 -9.13669523e-02 -1.44329594e-01
      -2.39042166e-01 -3.28352687e-01 -3.41372877e-01 -3.08147158e-01
      -2.54222506e-01 -2.63955914e-01 -3.54975138e-01 -3.68889268e-01
       4.51169023e-02  6.46064216e-02  7.51996357e-02  8.88137937e-02
       1.06395989e-01  1.29380303e-01  1.59834042e-01  2.02035753e-01
       2.74275773e-01  3.76774600e-01  5.72586351e-01  1.10308874e+00
       2.47083246e+00  3.28862489e+00]
     [ 5.53821445e-03 -2.11673077e-02 -4.47152353e-02 -6.11411592e-02
      -7.75607778e-02 -1.10162028e-01 -1.84695030e-01 -2.89526214e-01
      -3.42924565e-01 -3.31469371e-01 -2.82831629e-01 -2.56759778e-01
      -3.10566425e-01 -4.02544281e-01  2.05853764e-02  5.89285036e-02
       6.80572972e-02  8.00246667e-02  9.56158124e-02  1.15335006e-01
       1.40978811e-01  1.75629204e-01  2.24630658e-01  2.98832641e-01
       4.24411621e-01  6.87420750e-01  1.65894373e+00  2.91206517e+00
       3.45773520e+00  3.64205163e+00]
     [-2.99735849e-03 -3.19859077e-02 -5.71797684e-02 -8.42654343e-02
      -1.35874248e-01 -2.30702482e-01 -3.20510785e-01 -3.39883453e-01
      -3.15562492e-01 -2.65798487e-01 -2.73760222e-01 -3.72661486e-01
      -5.98338910e-02  5.39312231e-02  6.15512021e-02  7.12661970e-02
       8.41860692e-02  1.00964320e-01  1.22794121e-01  1.51750655e-01
       1.91757135e-01  2.48430565e-01  3.37337151e-01  4.97732928e-01
       8.83833534e-01  2.10528891e+00  3.12525769e+00  3.52730655e+00
       3.63127410e+00  3.35068590e+00]
     [-1.45083992e-02 -4.81294104e-02 -8.75605123e-02 -1.59697354e-01
      -2.63540331e-01 -3.29393233e-01 -3.30367903e-01 -2.84078086e-01
      -2.56952799e-01 -3.16460022e-01 -2.22893911e-01  4.44438091e-02
       5.32182827e-02  6.21441235e-02  7.42054234e-02  8.84582051e-02
       1.06398012e-01  1.29892885e-01  1.61503615e-01  2.05443617e-01
       2.70848402e-01  3.77845685e-01  5.86976065e-01  1.17532633e+00
       2.55439329e+00  3.30651091e+00  3.58119788e+00  3.55752123e+00
       3.04213614e+00  1.97474079e+00]
     [-3.29313455e-02 -8.65374924e-02 -1.77120605e-01 -2.77426774e-01
      -3.19662240e-01 -2.99947166e-01 -2.52588849e-01 -2.65881184e-01
      -2.80095442e-01  2.70528896e-02  4.80543963e-02  5.24963596e-02
       6.16898411e-02  7.40449035e-02  9.00571092e-02  1.10355006e-01
       1.37088628e-01  1.71205294e-01  2.19758142e-01  2.93611212e-01
       4.19251094e-01  6.85229513e-01  1.54949395e+00  2.85410313e+00
       3.42387709e+00  3.59791691e+00  3.45161725e+00  2.65762020e+00
       1.67983389e+00  1.14428021e+00]
     [-7.59951045e-02 -1.75256552e-01 -2.71762898e-01 -2.93314280e-01
      -2.46467303e-01 -2.27678709e-01 -2.69609763e-01 -3.98809467e-02
       4.67974697e-02  4.96575416e-02  5.51112796e-02  6.39821609e-02
       7.49629521e-02  9.11446279e-02  1.12415943e-01  1.40525571e-01
       1.78695311e-01  2.33391207e-01  3.17719034e-01  4.65705440e-01
       8.09425662e-01  1.94436054e+00  3.04945550e+00  3.48772127e+00
       3.57839254e+00  3.27872453e+00  2.27984241e+00  1.46040019e+00
       1.02478213e+00  7.71930186e-01]
     [-1.54614962e-01 -2.46627888e-01 -2.25375848e-01 -1.89403911e-01
      -2.12650676e-01 -1.03332072e-01  4.67997036e-02  4.96208865e-02
       5.25386631e-02  5.84424292e-02  6.77204952e-02  7.91809636e-02
       9.50404998e-02  1.15506810e-01  1.44152385e-01  1.84801813e-01
       2.44340995e-01  3.39393342e-01  5.14730739e-01  9.68654112e-01
       2.28651976e+00  3.19098315e+00  3.52556031e+00  3.52926500e+00
       3.02907308e+00  1.96593640e+00  1.29220352e+00  9.29612459e-01
       7.12409388e-01  5.69297016e-01]
     [-1.87995486e-01 -1.51197591e-01 -1.50044658e-01 -1.10957590e-01
       3.99206791e-02  4.89964571e-02  4.97983983e-02  5.45381049e-02
       6.24092461e-02  7.18263303e-02  8.40825233e-02  1.00900061e-01
       1.22539629e-01  1.51112899e-01  1.92615952e-01  2.55655901e-01
       3.61287535e-01  5.68537793e-01  1.16627061e+00  2.54981580e+00
       3.28911367e+00  3.54225214e+00  3.45037190e+00  2.72524269e+00
       1.72029073e+00  1.16115129e+00  8.51859198e-01  6.61495413e-01
       5.33637305e-01  4.42669660e-01]
     [-8.98977212e-02 -8.92777506e-02  1.69253328e-02  5.22037630e-02
       4.88351979e-02  5.04046898e-02  5.56097247e-02  6.37364274e-02
       7.48755816e-02  8.93934148e-02  1.06683465e-01  1.29349769e-01
       1.61054199e-01  2.05926025e-01  2.73866761e-01  3.87435999e-01
       6.29907894e-01  1.40892531e+00  2.74838883e+00  3.35171881e+00
       3.52185157e+00  3.30230124e+00  2.39628302e+00  1.52873640e+00
       1.05662962e+00  7.88154712e-01  6.18723122e-01  5.03006935e-01
       4.19656286e-01  3.57021503e-01]
     [-7.20435670e-03  6.16422972e-02  5.34243408e-02  4.85259326e-02
       5.11078074e-02  5.68777190e-02  6.54107230e-02  7.67587540e-02
       9.15491348e-02  1.11056026e-01  1.37019759e-01  1.70523200e-01
       2.18571634e-01  2.94065615e-01  4.26406125e-01  7.22912549e-01
       1.69758796e+00  2.89937603e+00  3.39097889e+00  3.47967399e+00
       3.09447445e+00  2.06726033e+00  1.34134415e+00  9.57920967e-01
       7.32117014e-01  5.81972119e-01  4.76381648e-01  3.99341252e-01
       3.41076931e-01  2.95445876e-01]
     [ 7.38179618e-02  5.67588766e-02  5.08940248e-02  5.27621766e-02
       5.78993570e-02  6.69762925e-02  7.87839192e-02  9.40904219e-02
       1.14050379e-01  1.40729572e-01  1.77985446e-01  2.32666850e-01
       3.15762772e-01  4.66164167e-01  8.29965968e-01  2.01224005e+00
       3.05317958e+00  3.43342289e+00  3.41525659e+00  2.81599546e+00
       1.79760861e+00  1.20139001e+00  8.75588977e-01  6.74291316e-01
       5.42533155e-01  4.48362555e-01  3.80833403e-01  3.26723128e-01
       2.83844032e-01  2.49490514e-01]
     [ 6.28142278e-02  5.49897002e-02  5.51438403e-02  6.03804968e-02
       6.91787500e-02  8.04764958e-02  9.64793196e-02  1.17232219e-01
       1.45019960e-01  1.83781098e-01  2.41135963e-01  3.34424410e-01
       5.12223251e-01  9.69651008e-01  2.28282769e+00  3.15977991e+00
       3.46219353e+00  3.36751790e+00  2.57183012e+00  1.59374095e+00
       1.08611032e+00  8.03446173e-01  6.27704923e-01  5.09113384e-01
       4.23399808e-01  3.59101305e-01  3.09954957e-01  2.70859540e-01
       2.40384238e-01  2.13886825e-01]
     [ 5.86039242e-02  5.82418726e-02  6.39817210e-02  7.17558753e-02
       8.35374357e-02  9.95242434e-02  1.20089709e-01  1.49208668e-01
       1.89980462e-01  2.50879843e-01  3.51239558e-01  5.48472938e-01
       1.11878996e+00  2.51304472e+00  3.25100438e+00  3.47508326e+00
       3.27913094e+00  2.33991567e+00  1.46975033e+00  1.01190732e+00
       7.48447607e-01  5.88836788e-01  4.79102308e-01  4.00784014e-01
       3.41922309e-01  2.96042555e-01  2.59065757e-01  2.29390723e-01
       2.04584201e-01  1.84142079e-01]
     [ 6.12579987e-02  6.67371220e-02  7.51567296e-02  8.75966438e-02
       1.02781270e-01  1.24470814e-01  1.52739535e-01  1.96008087e-01
       2.60938712e-01  3.69913624e-01  5.92486543e-01  1.28473783e+00
       2.66482235e+00  3.31222638e+00  3.47604895e+00  3.16056830e+00
       2.12150726e+00  1.35028849e+00  9.46519068e-01  7.12094905e-01
       5.61766728e-01  4.56431265e-01  3.82460102e-01  3.26774522e-01
       2.82899627e-01  2.48394549e-01  2.20469793e-01  1.96992396e-01
       1.77282037e-01  1.60802055e-01]
     [ 6.87287967e-02  7.82103817e-02  9.10576880e-02  1.07270377e-01
       1.29897795e-01  1.59744339e-01  2.04125673e-01  2.70886274e-01
       3.89962595e-01  6.43543562e-01  1.48463757e+00  2.80514103e+00
       3.35778781e+00  3.45430030e+00  3.00061728e+00  1.92634472e+00
       1.24726761e+00  8.87632129e-01  6.74650135e-01  5.36138579e-01
       4.39620974e-01  3.68750459e-01  3.14488591e-01  2.72572592e-01
       2.39583610e-01  2.12640460e-01  1.89977649e-01  1.71313315e-01
       1.55402839e-01  1.41666874e-01]
     [ 8.00865367e-02  9.36872726e-02  1.11875665e-01  1.35131781e-01
       1.69142949e-01  2.13687518e-01  2.85926391e-01  4.15060235e-01
       7.04150643e-01  1.70905979e+00  2.92854854e+00  3.39517477e+00
       3.41981199e+00  2.80217092e+00  1.74963492e+00  1.15442254e+00
       8.34266600e-01  6.40276695e-01  5.11987311e-01  4.21707564e-01
       3.55230808e-01  3.04402544e-01  2.64266327e-01  2.32000061e-01
       2.05608820e-01  1.84298376e-01  1.66270722e-01  1.50924864e-01
       1.37565383e-01  1.26013644e-01]
     [ 9.56611224e-02  1.14279643e-01  1.39636956e-01  1.74905710e-01
       2.23794568e-01  3.05368360e-01  4.45315812e-01  7.87434316e-01
       1.95322695e+00  3.03694250e+00  3.42192508e+00  3.36345072e+00
       2.57365733e+00  1.59225762e+00  1.07039363e+00  7.83440583e-01
       6.06954091e-01  4.89040670e-01  4.04736807e-01  3.42078892e-01
       2.93851053e-01  2.55856605e-01  2.25264720e-01  2.00076834e-01
       1.79055660e-01  1.61312175e-01  1.46593886e-01  1.33971282e-01
       1.22961589e-01  1.13337872e-01]
     [ 1.16831848e-01  1.42702549e-01  1.79319860e-01  2.34025436e-01
       3.20678942e-01  4.79498919e-01  9.06602064e-01  2.20698884e+00
       3.14491894e+00  3.43106552e+00  3.26942918e+00  2.32434118e+00
       1.44828453e+00  9.92814665e-01  7.36254372e-01  5.75205425e-01
       4.65973598e-01  3.87631487e-01  3.29102730e-01  2.83617760e-01
       2.47510331e-01  2.18276578e-01  1.94273061e-01  1.74280566e-01
       1.57354660e-01  1.42859682e-01  1.30287311e-01  1.19680278e-01
       1.10415860e-01  1.02192401e-01]
     [ 1.46197346e-01  1.83972087e-01  2.40879250e-01  3.35100411e-01
       5.20590953e-01  1.02974287e+00  2.49332715e+00  3.24857453e+00
       3.46089264e+00  3.17481516e+00  2.05372441e+00  1.31088184e+00
       9.18612570e-01  6.90660236e-01  5.44491606e-01  4.43824823e-01
       3.70868557e-01  3.15850911e-01  2.73053980e-01  2.39029933e-01
       2.11343167e-01  1.88463819e-01  1.69307867e-01  1.53086408e-01
       1.39247988e-01  1.27244140e-01  1.16779381e-01  1.07585178e-01
       9.96911473e-02  9.26896397e-02]
     [ 1.89129321e-01  2.48621251e-01  3.48799893e-01  5.51678659e-01
       1.18364410e+00  2.65215738e+00  3.32859939e+00  3.48305179e+00
       3.01665453e+00  1.89780959e+00  1.21495907e+00  8.42918045e-01
       6.45089022e-01  5.13959593e-01  4.21922250e-01  3.54172066e-01
       3.02795710e-01  2.62535696e-01  2.30364335e-01  2.04148211e-01
       1.82476932e-01  1.64220640e-01  1.48708877e-01  1.35482123e-01
       1.23982544e-01  1.13971417e-01  1.05207756e-01  9.74436018e-02
       9.04624725e-02  8.44477203e-02]
     [ 2.57511497e-01  3.64850033e-01  5.90336769e-01  1.34522965e+00
       2.79614867e+00  3.40195797e+00  3.47726093e+00  2.86797954e+00
       1.73369722e+00  1.11325611e+00  7.99468165e-01  6.12169621e-01
       4.81386605e-01  3.99269344e-01  3.37368039e-01  2.89643705e-01
       2.52009746e-01  2.21711944e-01  1.96810262e-01  1.76250740e-01
       1.58924370e-01  1.44251463e-01  1.31600804e-01  1.20658540e-01
       1.11068433e-01  1.02618737e-01  9.51455906e-02  8.85233943e-02
       8.25661684e-02  7.72137623e-02]
     [ 3.83144078e-01  6.36945125e-01  1.54875599e+00  2.93200277e+00
       3.44818860e+00  3.45628714e+00  2.68617075e+00  1.59876685e+00
       1.04338864e+00  7.51065433e-01  5.75318730e-01  4.61973619e-01
       3.72516069e-01  3.18943836e-01  2.75727810e-01  2.41194110e-01
       2.12922267e-01  1.89547023e-01  1.70062767e-01  1.53644569e-01
       1.39609731e-01  1.27505597e-01  1.17041020e-01  1.07935966e-01
       9.98947864e-02  9.27689492e-02  8.64279252e-02  8.07263900e-02
       7.55886227e-02  7.09937513e-02]
     [ 7.20115454e-01  1.84237020e+00  3.07025737e+00  3.48709067e+00
       3.40641127e+00  2.46452417e+00  1.47198796e+00  9.81000052e-01
       7.11612973e-01  5.51798073e-01  4.38827807e-01  3.63282698e-01
       3.07895570e-01  2.59519357e-01  2.29089974e-01  2.03271707e-01
       1.81783201e-01  1.63730850e-01  1.48335145e-01  1.35019283e-01
       1.23501288e-01  1.13489849e-01  1.04788932e-01  9.70255346e-02
       9.02399591e-02  8.41641279e-02  7.87809048e-02  7.38680072e-02
       6.94245037e-02  6.53667222e-02]
     [ 2.10734176e+00  3.19431493e+00  3.52696991e+00  3.33668609e+00
       2.23817249e+00  1.35016529e+00  9.17299559e-01  6.75751600e-01
       5.24330981e-01  4.21212528e-01  3.50065742e-01  2.93903590e-01
       2.53678260e-01  2.21881902e-01  1.92503313e-01  1.73195374e-01
       1.56678636e-01  1.42331532e-01  1.30011836e-01  1.19272158e-01
       1.09900135e-01  1.01635342e-01  9.42541561e-02  8.77361083e-02
       8.18889292e-02  7.66874453e-02  7.19531431e-02  6.77273527e-02
       6.38607719e-02  6.03032763e-02]
     [ 3.30176673e+00  3.52763096e+00  3.14607655e+00  1.96081506e+00
       1.22063086e+00  8.50494064e-01  6.37019537e-01  4.99869098e-01
       4.05542091e-01  3.35880091e-01  2.83415840e-01  2.44660778e-01
       2.12819553e-01  1.88124146e-01  1.68125674e-01  1.48837607e-01
       1.36032173e-01  1.24635100e-01  1.14573776e-01  1.05792621e-01
       9.81095055e-02  9.12107211e-02  8.51100660e-02  7.95479567e-02
       7.45833732e-02  7.00611241e-02  6.59922656e-02  6.23766989e-02
       5.89936415e-02  5.59102997e-02]
     [ 3.53196232e+00  2.94759693e+00  1.74689369e+00  1.10701387e+00
       7.81495520e-01  5.91345669e-01  4.67680159e-01  3.81298866e-01
       3.20168293e-01  2.73397713e-01  2.35925204e-01  2.05512019e-01
       1.81524260e-01  1.61902695e-01  1.45931436e-01  1.32471272e-01
       1.19084650e-01  1.10147155e-01  1.01958946e-01  9.45985611e-02
       8.79785014e-02  8.20891226e-02  7.67509645e-02  7.19986807e-02
       6.77766664e-02  6.38966941e-02  6.04252394e-02  5.72513592e-02
       5.42642922e-02  5.16174863e-02]
     [ 2.64246333e+00  1.54246950e+00  1.00448919e+00  7.23013563e-01
       5.53489871e-01  4.41909246e-01  3.63678269e-01  3.06010359e-01
       2.61971579e-01  2.27669659e-01  1.98860116e-01  1.75238314e-01
       1.57541485e-01  1.40493117e-01  1.27612641e-01  1.16594285e-01
       1.04343330e-01  9.74444038e-02  9.08966267e-02  8.49809267e-02
       7.96569969e-02  7.48026838e-02  7.04696190e-02  6.63620506e-02
       6.27876581e-02  5.94104764e-02  5.62810549e-02  5.34694219e-02
       5.08492717e-02  4.84557226e-02]
     [ 1.35449725e+00  9.08485859e-01  6.65463948e-01  5.15796353e-01
       4.15263643e-01  3.43656956e-01  2.90560204e-01  2.49795105e-01
       2.17657735e-01  1.91785675e-01  1.70338980e-01  1.51858994e-01
       1.35881666e-01  1.23944752e-01  1.12524048e-01  1.03434956e-01
       9.54993223e-02  8.64305556e-02  8.13053667e-02  7.64256792e-02
       7.18869683e-02  6.77618655e-02  6.41181571e-02  6.08078547e-02
       5.78276350e-02  5.49758761e-02  5.25148072e-02  5.01257044e-02
       4.79070543e-02  4.57095239e-02]]

```python
from matplotlib import pyplot as plt
import numpy as np

X, Y = np.meshgrid(C * 100, velocity)
Z = np.loadtxt('vel_difference4.txt', delimiter=',')
cont=plt.contour(X, Y, Z, 5, vmin=-1,vmax=3, colors=['black'])
cont.clabel(fmt='%1.1f', fontsize=14)
plt.xlabel('Concentration', fontsize=24)
plt.ylabel('Velocity', fontsize=24)
plt.pcolormesh(X,Y,Z, cmap='cool') #カラー等高線図
pp=plt.colorbar (orientation="vertical") # カラーバーの表示
pp.set_label("Amplitude of velocity fluctuation (m/s)",  fontsize=14)
plt.savefig('result_map.svg')
# im = plt.imshow(vel_difference, aspect='auto', extent=[C[0]*100, C[-1]*100, velocity[-1], velocity[0]], interpolation='bicubic')
# ax = plt.gca()
# ax.invert_yaxis()
# ax.set_xlabel('Concentration (%)')
# ax.set_ylabel('Velocity (m/s)')
# plt.colorbar(im)
# plt.show()
# plt.savefig('testimage.svg')
```

```python
print(C)
print(velocity)
a = np.array([[0,1,0],[1,0,0],[0,0,1],[0,0,0]])
plt.imshow(a)
```

```python
import tideturb

tc_org = tideturb.load_model('test09_5000sec')
tc_org.axM.plot(tc_org.grid.x, tc_org.U_node[1, :])
tc_org.axL.plot(tc_org.grid.x, tc_org.h_node[1, :])
tc_org.axR.plot(tc_org.grid.x, tc_org.C_node[1, :])
U = tc_org.U_node[1, -1]
h = tc_org.h_node[1, -1]
C = tc_org.C_node[1, -1]
Fr = U / np.sqrt(1.65 * 9.81 * C * h)
print(Fr)
```