# Input Driven HMM

This notebook is a simple example of an HMM with exogenous inputs.  The inputs modulate the probability of discrete state transitions via a multiclass logistic regression. Let $z_t \in \{1, \ldots, K\}$ denote the discrete latent state at time $t$ and $u_t \in \mathbb{R}^U$ be the exogenous input at time~$t$.  The transition probability is given by,

$$
\begin{align}
\Pr(z_t = k \mid z_{t-1} = j, u_t) =
\frac{\exp\{\log P_{j,k} + w_k^\mathsf{T} u_t\}}
{\sum_{k'=1}^K \exp\{\log P_{j,k'} + w_{k'}^\mathsf{T} u_t\}}.
\end{align}
$$

The parameters of the transition model are $P \in \mathbb{R}_+^{K \times K}$, a baseline set of (unnormalized) transition weights, and $W \in \mathbb{R}^{K \times U}$, a set of input weights.

## 1. Setup
The line `import ssm` imports the package for use. Here, we have also imported a few other packages for plotting.

```python
import numpy as np
import numpy.random as npr
import matplotlib.pyplot as plt
import ssm
import seaborn as sns
from ssm.util import one_hot, find_permutation

%matplotlib inline

npr.seed(0)
sns.set(palette="colorblind")
```

## 2. Create an Input Driven HMM
SSM is designed to be modular, so that the user can easily mix and match different types of transitions and observations.
We create an input-driven HMM with the following line:
```python
true_hmm = ssm.HMM(num_states, obs_dim, input_dim,
               observations="categorical", observation_kwargs=dict(C=num_categories),
               transitions="inputdriven")
```

Let's look at what each of the arguments do. The first three arguments specify the number of states, and the dimensionality of the observations and inputs.

**Setting the observation model**
For this example, we have set `observations="categorical"`, which means each observation will take on one of a discrete set of values, i.e $y_t \in \{1, \ldots, C \}$.
For categorical observations, the observations are drawn from a multinomial distribution, with parameters depending on the current state. Assuming $z_t = k$,the observations are a vector $y \in \mathbb{R}^D$, where $y_i \sim \text{mult} (\lambda_{k,i})$, where $\lambda_{k,i}$ is the multinomal parameter associated with coordinate $i$ of the observations in state $k$. Note that each observation variable is independent from the others.

For categorical observations, we also specify the number of discrete observations possible (in this case 3). We do this by creating a dictionary where the keys are the keyword arguments which we want to pass to the observation model. For categorical observations, there is just one keyword argument, `C`, which specifies the number of categories. This is set using `observation_kwargs=dict(C=num_categories)`.

The observations keyword argument should be one of : `"gaussian", "poisson" "studentst", "exponential", "bernoulli", "autoregressive", "robust_autoregressive"`.
**NOTE:**
Setting the observations as "autoregressive" means that each observation will be dependent on the prior observation, as well as on the input (if the input is nonzero). By constrast, the standard "inputdriven" transitions are not affected by previous observations or directly by the inputs.

**Setting the transition model**
In order to create an HMM with exogenous inputs, we set ```transitions="inputdriven"```. This means that the baseline transition matrix $P$ is modified according to a Generalized Linear Model, as described at the top of the page.

SSM support many transition models, set by keyword argument to the constructor of the class. The keyword argument should be one of: `"standard", "sticky", "inputdriven", "recurrent", "recurrent_only", "rbf_recurrent", "nn_recurrent".` We're working on creating standalone documentation to describe these in more detail. For most users, the stationary and input driven transition classes should suffice.

**Creating inputs and sampling**
After creating our HMM object, we create an input array called `inpt` which is simply a jittered sine wave. We also increase the transition weights so that it will be clear (for demonstration purposes) that the input is changing the transition probabilities. In this case, we will actually increase the weights such that the transitions appear almost deterministic.

```python
# Set the parameters of the HMM
time_bins = 1000 # number of time bins
num_states = 2    # number of discrete states
obs_dim = 1    # data dimension
input_dim = 1    # input dimension
num_categories = 3    # number of output types/categories

# Make an HMM
true_hmm = ssm.HMM(num_states, obs_dim, input_dim,
               observations="categorical", observation_kwargs=dict(C=num_categories),
               transitions="inputdriven")

# Optionally, turn up the input weights to exaggerate the effect
true_hmm.transitions.Ws *= 3

# Create an exogenous input
inpt = np.sin(2 * np.pi * np.arange(time_bins) / 50)[:, None] + 1e-1 * npr.randn(time_bins, input_dim)

# Sample some data from the HMM
true_states, obs = true_hmm.sample(time_bins, input=inpt)

# Compute the true log probability of the data, summing out the discrete states
true_lp = true_hmm.log_probability(obs, inputs=inpt)

# By default, SSM returns categorical observations as a list of lists.
# We convert to a 1D array for plotting.
obs_flat = np.array([x[0] for x in obs])
```

```python
# Plot the data
plt.figure(figsize=(8, 5))

plt.subplot(311)
plt.plot(inpt)
plt.xticks([])
plt.xlim(0, time_bins)
plt.ylabel("input")

plt.subplot(312)
plt.imshow(true_states[None, :], aspect="auto")
plt.xticks([])
plt.xlim(0, time_bins)
plt.ylabel("discrete\nstate")
plt.yticks([])

# Create Cmap for visualizing categorical observations
plt.subplot(313)
plt.imshow(obs_flat[None,:], aspect="auto", )
plt.xlim(0, time_bins)
plt.ylabel("observation")
plt.grid(b=None)
plt.show()
```

### 2.1 Exercise: EM for the input-driven HMM
There are a few good references that derive the EM algorithm for the case of a vanilla HMM (e.g Machine Learning and Pattern Recognition, by Chris Bishop, and [this tutorial](https://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf) by Lawrence Rabiner. How should the EM updates change for the case of input-driven HMMs?

## 3. Fit an input-driven HMM to data
Below, we'll show to fit an input-driven HMM from data. We'll treat the samples generated above as a dataset, and try to learn the appropriate HMM parameters from this dataset.

We create a new HMM object here, with the same parameters as the HMM in Section 1:
```python
hmm = ssm.HMM(num_states, obs_dim, input_dim,
          observations="categorical", observation_kwargs=dict(C=num_categories),
          transitions="inputdriven")
```

We fit the dataset simply by calling the `fit` method:
```python
hmm_lps = hmm.fit(obs, inputs=inpt, method="em", num_em_iters=N_iters)
```
Here, the variable `hmm_lps` will be set to a list of log-probabilities at each step of the EM-algorithm, which we'll use to check convergence.

```python
# Now create a new HMM and fit it to the data with EM
N_iters = 100
hmm = ssm.HMM(num_states, obs_dim, input_dim,
          observations="categorical", observation_kwargs=dict(C=num_categories),
          transitions="inputdriven")

# Fit
hmm_lps = hmm.fit(obs, inputs=inpt, method="em", num_iters=N_iters)
```

    HBox(children=(FloatProgress(value=0.0), HTML(value='')))

### 3.1 Permute the latent states, check convergence
As in the vanilla-HMM notebook, we need to find a permutation of the latent states from our new hmm such that they match the states from the true HMM above. SSM accomplishes this with two function calls: first, we call `find_permutation(true_states, inferred_states)` which returns a list of indexes into states.

Then, we call `hmm.permute(permuation)` with the results of our first function call. Finally, we set `inferred_states` to be the underlying states we predict given the data.

Below, we plot the results of the `fit` function in order to check convergence of the EM algorithm. We see that the log-probability from the EM algorithm approaches the true log-probability of the data (which we have stored as `lp_true`).

```python
# Find a permutation of the states that best matches the true and inferred states
hmm.permute(find_permutation(true_states, hmm.most_likely_states(obs, input=inpt)))
inferred_states = hmm.most_likely_states(obs, input=inpt)
```

```python
# Plot the log probabilities of the true and fit models
plt.plot(hmm_lps, label="EM")
plt.plot([0, N_iters], true_lp * np.ones(2), ':k', label="True")
plt.legend(loc="lower right")
plt.xlabel("EM Iteration")
plt.xlim(0, N_iters)
plt.ylabel("Log Probability")
plt.show()
```

### 3.3 Exercise: Change the Fitting Method
As an experiment, try fitting the same dataset using another fitting method. The two other fitting methods supported for HMMs are "sgd" and "adam", which you can set by passing `method="sgd"` and `method="adam"` respectively. For these methods, you'll probably need to increase the number of iteratations to around 1000 or so.

After fitting with a different method, re-run the two cells above to generate a plot. How does the convergence of these other methods converge to EM?

```python
# Plot the true and inferred states
plt.figure(figsize=(8, 3.5))

plt.subplot(211)
plt.imshow(true_states[None, :], aspect="auto")
plt.xticks([])
plt.xlim(0, time_bins)
plt.ylabel("true\nstate")
plt.yticks([])

plt.subplot(212)
plt.imshow(inferred_states[None, :], aspect="auto")
plt.xlim(0, time_bins)
plt.ylabel("inferred\nstate")
plt.yticks([])
plt.show()
```

## 4. Visualize the Learned Parameters
After calling `fit`, our new HMM object will have parameters updated according to the dataset. We can get a sense of whether we successfully learned these parameters by comparing them to the _true_ parameters which generated the data.

Below, we plot the baseline log transition probabilities (the log of the state-transition matrix) as well as the input weights $w$.

```python
# Plot the true and inferred input effects
plt.figure(figsize=(8, 4))

vlim = max(abs(true_hmm.transitions.log_Ps).max(),
           abs(true_hmm.transitions.Ws).max(),
           abs(hmm.transitions.log_Ps).max(),
           abs(hmm.transitions.Ws).max())

plt.subplot(141)
plt.imshow(true_hmm.transitions.log_Ps, vmin=-vlim, vmax=vlim, cmap="RdBu", aspect=1)
plt.xticks(np.arange(num_states))
plt.yticks(np.arange(num_states))
plt.title("True\nBaseline Weights")
plt.grid(b=None)

plt.subplot(142)
plt.imshow(true_hmm.transitions.Ws, vmin=-vlim, vmax=vlim, cmap="RdBu", aspect=num_states/input_dim)
plt.xticks(np.arange(input_dim))
plt.yticks(np.arange(num_states))
plt.title("True\nInput Weights")
plt.grid(b=None)

plt.subplot(143)
plt.imshow(hmm.transitions.log_Ps, vmin=-vlim, vmax=vlim, cmap="RdBu", aspect=1)
plt.xticks(np.arange(num_states))
plt.yticks(np.arange(num_states))
plt.title("Inferred\nBaseline Weights")
plt.grid(b=None)

plt.subplot(144)
plt.imshow(hmm.transitions.Ws, vmin=-vlim, vmax=vlim, cmap="RdBu", aspect=num_states/input_dim)
plt.xticks(np.arange(input_dim))
plt.yticks(np.arange(num_states))
plt.title("Inferred\nInput Weights")
plt.grid(b=None)
plt.colorbar()
plt.show()
```


# Aufgabe 6) b

```
from IPython.display import display
from sympy import *
from sympy.abc import x, a, k, n
init_printing()
```

```
eps = 1.1921 * 10**(-7)
x0 = 0.707107
a4, a3, a2, a1, a0 = 8118, -11482, 1, 5741, -2030
```

```
p = a4 * x**4 + a3 * x**3 + a2 * x**2 + a1 * x + a0
p2 = 8118 * x**4 - 11482 * x**3 + x**2 + 5741 * x -2030
display(p)
```

```
p.subs(x, x0)
```

```
krel = simplify(p.diff(x)*x/p)
display(krel)
```

```
krel.subs(x, x0)
```

Die Auswertung an der Stelle x0 = 0.707107 ist also sehr schlecht konditioniert.

```
e = symbols('epsilon')
z4 = a4 * x0**4 * (1+e)
z3 = a3 * x0**3 * (1+e)
z2 = a2 * x0**2 * (1+e)
z1 = a1 * x0
z = ((((((z4+z3)*(1+e)) + z2)*(1+e))+z1)*(1+e)+a0)*(1+e)
display(simplify(z))
```

```
z.subs(e, eps)
```

```

```


```python
from autoreduce import *
import numpy as np
from sympy import symbols
```

```python
# Post conservation law and other approximations phenomenological model at the RNA level
n = 4 # Number of states
nouts = 2 # Number of outputs

# Inputs by user
x_init = np.zeros(n)
n = 4 # Number of states
timepoints_ode = np.linspace(0, 100, 100)
C = [[0, 0, 1, 0], [0, 0, 0, 1]]
nstates_tol = 3
error_tol = 0.3
# System dynamics symbolically

# params = [100, 50, 10, 5, 5, 0.02, 0.02, 0.01, 0.01]
# params = [1, 1, 5, 0.1, 0.2, 1, 1, 100, 100] # Parameter set for which reduction doesn't work
# K,b_t,b_l,d_t,d_l,del_t,del_l,beta_t,beta_l = params

x0 = symbols('x0')
x1 = symbols('x1')
x2 = symbols('x2')
x3 = symbols('x3')
x = [x0, x1, x2, x3]

K = symbols('K')
b_t = symbols('b_t')
b_l = symbols('b_l')
d_t = symbols('d_t')
d_l = symbols('d_l')
del_t = symbols('del_t')
del_l = symbols('del_l')
beta_t = symbols('beta_t')
beta_l = symbols('beta_l')
params = [K,b_t,b_l,d_t,d_l,del_t,del_l,beta_t,beta_l]
f0 = K * b_t**2/(b_t**2 + x[3]**2) - d_t * x[0]
f1 = K * b_l**2/(b_l**2 + x[2]**2) - d_l * x[1]
f2 = beta_t * x[0] - del_t * x[2]
f3 = beta_l * x[1] - del_l * x[3]
f = [f0,f1,f2,f3]
# parameter values
params_values = [100, 50, 10, 5, 5, 0.02, 0.02, 0.01, 0.01]
sys = System(x, f, params = params, params_values = params_values, C = C, x_init = x_init)
```

```python
from autoreduce.utils import get_ODE
sys_ode = get_ODE(sys, timepoints_ode)
sol = sys_ode.solve_system().T
try:
    import matplotlib.pyplot as plt
    plt.plot(timepoints_ode, np.transpose(np.array(C)@sol))
    plt.xlabel('Time')
    plt.ylabel('[Outputs]')
    plt.show()
except:
    print('Plotting libraries missing.')
```

```python
from autoreduce.utils import get_SSM
timepoints_ssm = np.linspace(0,100,100)
sys_ssm = get_SSM(sys, timepoints_ssm)
Ss = sys_ssm.compute_SSM() # len(timepoints) x len(params) x len(states)
out_Ss = []
for i in range(len(params)):
    out_Ss.append((np.array(C)@(Ss[:,i,:].T)))
out_Ss = np.reshape(np.array(out_Ss), (len(timepoints_ssm), len(params), nouts))
```

    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete

```python
try:
    import seaborn as sn
    import matplotlib.pyplot as plt
    for j in range(nouts):
        sn.heatmap(out_Ss[:,:,j].T)
        plt.xlabel('Time')
        plt.ylabel('Parameters')
        plt.title('Sensitivity of output[{0}] with respect to all parameters'.format(j))
        plt.show()
except:
    print('Plotting libraries missing.')
```

```python
from autoreduce.utils import get_reducible
timepoints_ssm = np.linspace(0,100,10)
timepoints_ode = np.linspace(0, 100, 100)
sys_reduce = get_reducible(sys, timepoints_ode, timepoints_ssm)
results = sys_reduce.reduce_simple()
```

    Successful time-scale separation solution obtained with states: [x2, x3]!
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete

    C:\Users\apand\Anaconda3\lib\site-packages\scipy\linalg\_solvers.py:196: RuntimeWarning: Input "a" has an eigenvalue pair whose sum is very close to or exactly zero. The solution is obtained via perturbing the coefficients.
      RuntimeWarning)
    c:\users\apand\box\biocircuits\ayush\synbio\autoreduce\autoreduce\model_reduction.py:217: ComplexWarning: Casting complex values to real discards the imaginary part
      Se[j] = max_eig_P + 2*len(reduced_ssm.timepoints)*S_metric_max

    Robustness Metric Progress: |██████████████████████████████████████████████████| 100.0% Complete
    Successful time-scale separation solution obtained with states: [x0, x2, x3]!
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete
    Robustness Metric Progress: |██████████████████████████████████████████████████| 100.0% Complete
    Successful time-scale separation solution obtained with states: [x1, x2, x3]!
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete
    SSM Progress: |██████████████████████████████████████████████████| 100.0% Complete
    Robustness Metric Progress: |██████████████████████████████████████████████████| 100.0% Complete

```python
list(results.keys())[0].f[1]
```

$\displaystyle \frac{K b_{l}^{2} \beta_{l}}{d_{l} \left(b_{l}^{2} + x_{2}^{2}\right)} - del_{l} x_{3}$

```python
reduced_system, collapsed_system = sys_reduce.solve_timescale_separation([x0,x1], fast_states = [x3, x2])
```

    Successful time-scale separation solution obtained with states: [x0, x1]!

```python
reduced_system.f[1]
```

$\displaystyle \frac{K b_{l}^{2}}{b_{l}^{2} + \frac{\beta_{t}^{2} x_{0}^{2}}{del_{t}^{2}}} - d_{l} x_{1}$

```python

```


# Recurrent neural networks

## Motivation

Up to now we used neural networks to learn functions that predicted a desired output $y$ based on an input vector $x$. We have assumed that $x$ has a constant size, such as an image of a given resolution.

In contrast, recurrent networks can be applied to sequences of different length.

Consider the problem of computing the parity checksum of a sequence of bits. Traditional neural networks that we know can not be applied to bit sequences of arbitrary length, yet a program which reads in the sequence bit-by-bit is very easy to write.

Another motivating example is sequence generation, such as generating text or music. Again, normal neural networks that take a fixed input size are not directly applicable to whole sequences.

## Autoregressive models

An autoregressive model assumes, that the $i$-th element of a sequence depends only on a few preceding sequence elements. Lets first write the exact probability of observing a sequence $\mathbf{Y} = [Y_1, Y_2, \ldots, Y_m]$:

$$
    P(\mathbf{Y}) = P(Y_1)P(Y_2|Y_1)P(Y_3|Y_1, Y_2)\ldots P(Y_m|Y1,Y_2,\ldots,Y_{n-1})=\prod_i P(Y_i|Y_1,\ldots, Y_{i-1})
$$

The autoregressive model simply assumes that we can use only a finite history, that is:
$$
    P(Y_i|Y_1,\ldots, Y_{i-1}) \approx P(Y_i|Y_{i-n},\ldots, Y_{i-1})
$$

Autoregressive models are quite frequently used, because they are easy to implement - it is sufficient extract $n$ element long subsequences and learn a model that predicts the last element given the $n-1$ preceding ones. Thus autoregressive models reduce the sequence learning problem to the typicall supervised learning setup

### Examples

Autoregressive models are very popular, for instance:

1. Language models (that is models which tell how probable is a given utterance) are often expressed as $n$-gram models in which $P(Y_i|Y_{i-n},\ldots, Y_{i-1})$ is simply established by counting occurrences in a corpus of text.
2. The ARMA model used in timeseries prediction is expressed as:
    $$
        Y_t = \sum_{i=1}^{n}\alpha_i Y_{t-i} + \sum_{i=1}^k \Theta_i \epsilon_{t-i},
    $$
    where $\epsilon_i$ are assumed to be normally distributed noise variables.

## Models with a hidden state

Autoregressive models have a very short memory which limits their applicability. We can build a more powerfull model by introducing a sequence of hidden states $\mathbf{H} = [H_0, H_1, \ldots]$. We will assume that all history of a sequence can be captured by the state:

$$
    P(Y_i|Y_1,\ldots, Y_{i-1}) \approx P(Y_i|Y_{i-n}, H_{i-1})
$$

### Recurrent neural networks

Recurrent neural networks are a generalization of the model with the hidden state. We will assume that there is an input sequence $\mathbb{X}$. The network will process the elements of $\mathbb{X}$  one at a time producing a sequence of hidden states and a sequence of outputs. We will train the model by specifying the desired outputs. We will be able to supervise the model at each step (which is common e.g. in sequence generators which are taught to predict the next sequence element) or only at the end (which can be used in the parity computing network).

Define a recurrent computation:

$$
\begin{align}
    H_t &= f(X_t, H_{t-1}) \\
    Y_t &= g(X_t, H_t)
\end{align}
$$

In a recurrent neural network the function $f$ and $g$ are implemented as multilayer neural networks.

### Backpropagation through time

The last question is how to train such a recurrent network? The typicall approach is to *unroll the network in time*, then compute the loss and backpropagate it over the time steps. Gradient backpropagation algorithm works, thus in principle training doesn't require new skill. Hovewer, the unrolled network is very deep (its depth equals to the number of time steps!) and quite pathological, because the same weight matrices are reused at all times. For this reasons recurrent networks suffer from two problems:

1. Gradient vanishing, when the gradient drops to zero and no training is possible.
2. Gradient explosion, when the gradient grows really quickly and a single step can destroy the network.

The two problems are related. Consider the following recurrency:

$$
    H_t = W_{hh}H_{t-1} = (W_{hh})^t H_0
$$

Suppose $H_0$ is an eigenvector - then depending on the magnitude of the associated eigenvalue the hidden state will either exponentially grow or decay! Of course, the network will typically have sone nonlinearities that will prevent the explosion of the hiden state. However, the backpropagation computation is linear (because all nonlinearities are linerized at the operating point). Thus the gradient is very prone to explode or implode!

### Solutions to gradient pathologies

#### Echo-state networks
The first solution to training recurrent networks is... not to train the recurrent connections! This approach is taken in the *echo state networks* which perform the following computation:

$$
    \begin{align}
        H_t &= \tanh(W_{xh}X_t + W_{hh} H_{t-1} + b_h) \\
        O_t &= W_{xo}X_t + W_{ho}H_T + b_o
    \end{align}
$$

The training procedure is as follows:

1. Randomly sample $W_{hh}$ and $W_{xh}$.
2. Rescale $W_{hh}$ to have the largest eigenvalue close to 1
3. Fit the $W_{xo}$ and $W{ho}$ using the closed-form formula for least squares
4. Since steps 1.-3. are super-fast, repeat them multiple times with different scaling of $W_{hh}$ and $W_{xh}$.

Because $W_{hh}$ has the largest eigenvalue sligtly less than 1, the hidden states oscillate and decay slowly. This creates "echoes" of previous inputs that reverberate in the network.

#### Gradient clipping

Gradient explosion can be prevented by rescaling gradients that are larger than a specified threshold. You can either clip individual components of the gradient, or rescale the whole gradient.

#### LSTM cells

LSTM's are important! They contributed a lot to recent sucesses of recurrent neural networks!

For introduction and intuitions please see http://colah.github.io/posts/2015-08-Understanding-LSTMs/

For more in-depth analysis please look at: http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf and http://arxiv.org/pdf/1503.04069v1.pdf

The core idea of LSTM is to introduce multiplicative *gates* that enable long pathways of nearly constant values/gradients of memory cells that we will denote $c$. The core equations of LSTM are:

$$
\begin{align}
    i_t &= \sigma(W_{xi}X_t + W_{hi}H_{t-1} + W_{ci}c_{t-1} + b_i) \\
    f_t &= \sigma(W_{xf}X_t + W_{hf}H_{t-1} + W_{cf}c_{t-1} + b_f) \\
    c_t &= f_t c_{t-1} + i_t\tanh(W_{xc}X_t + W_{hc}H_{t-1} + b_i) \\
    o_t &= \sigma(W_{xo}X_t + W_{ho}H_{t-1} + W_{co}c_{t} + b_o) \\
    H_t &= o_t \tanh(c_t)
\end{align}
$$

#### Multilayer and bi-directional LSTM networks

It is possible to stack several LSTM layers (see the code below). Likewise, it is common to invert the input sequence to run the LSTM backward in time. Then the forward and backward hidden states give a summary of the sequence around a certain element.

### Tricks of the trade

1. Use train rules that allow per-parameter learnign rates (e.g. RMSProp)
2. Monitor gradient magnitude!
3. Initialization is important:
    - it often helps to orthogonalize recurent weights and rescale to have the largest eigenvalue close to 1. This is similar to hidden-to-hidden weights in echo state networks
    - forget-gate biases in LSTMs are ofetn initialized to 1 instead of 0. This enhances information retention at the beginning of training
4. Learning interdependencies across many time steps is difficult. If possible train on short (or othervise simple) sequences first (this is often called "curriculum learning")

```
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

```
debug = True # global var to control debugging
CUDA = False # Cuda may be slower, as we are computing very small networks
```

```
import os

import pandas as pd
```

```
import torch
import torch.tensor as T

from torch.nn.parameter import Parameter

from torch.nn import functional as F

def V(np_var):
    t = torch.from_numpy(np_var)
    if CUDA:
        t = t.cuda()
    return t
```

## RNN implementation in PyTorch

Please note that PyTorch has RNN implementations (http://pytorch.org/docs/master/nn.html#recurrent-layers). The goal of this notebook is to demonstrate the basics of RNNs. For future projects please use the built-in mudules. They have more features, and are faster (they use the CuDNN library supplied by Nvidia).

**Attention**: through the code we will assume that the 0-th axis refers to time and that the 1-st axis refers to individual examples inside a minibatch. (This way in a C-major memory layout individual time steps occupy contiguous regions in memory)

```
class SimpleRNNCell(torch.nn.Module):
    def __init__(self, in_dim, hidden_dim, activation=torch.nn.Tanh(), **kwargs):
        super(SimpleRNNCell, self).__init__(**kwargs)

        self.activation = activation
        # Input to hidden
        self.Wxh = Parameter(torch.Tensor(in_dim, hidden_dim))

        # Hidden to hidden
        self.Whh = Parameter(torch.Tensor(hidden_dim, hidden_dim))

        # Hidden bias
        self.bh = Parameter(torch.Tensor(hidden_dim))
        self.reset_parameters()

    def reset_parameters(self):
        stdv = 1. / math.sqrt(self.Whh.size(1))
        self.Wxh.data.uniform_(-stdv, stdv)
        self.Whh.data.uniform_(-stdv, stdv)
        self.bh.data.zero_()

    def forward(self, input, h):
        pre_act = (
            self.bh +
            torch.matmul(input, self.Wxh) +
            torch.matmul(h, self.Whh)
        )
        return self.activation(pre_act)

class RNN(torch.nn.Module):
    def __init__(self, cell, **kwargs):
        super(RNN, self).__init__(**kwargs)
        self.cell = cell

    def forward(self, input, hidden):
        cell = self.cell
        output = []
        for i in range(input.size(0)):
            hidden = cell(input[i], hidden)
            # hack to handle LSTM
            output.append(hidden[0] if isinstance(hidden, tuple) else hidden)

        output = torch.cat(output, 0).view(input.size(0), *output[0].size())

        return hidden, output
```

# The parity task

Here we solve the bit parity problem. Note that we will need at least two hidden neurons, because the network has to solve a XOR problem at each step.

```
def gen_parity_examples(time_steps, batch_size):
    X = (numpy.random.rand(time_steps, batch_size, 1)>0.5).astype('float32')
    Y = X.cumsum(0) % 2
    return X,Y

Xp,Yp = gen_parity_examples(18, 1)
print('X.T:', Xp.reshape(Xp.shape[:-1]).T, '\nY.T:', Yp.reshape(Yp.shape[:-1]).T)
```

    X.T: [[1. 0. 1. 1. 0. 0. 0. 0. 1. 1. 0. 0. 1. 0. 0. 0. 0. 0.]]
    Y.T: [[1. 1. 0. 1. 1. 1. 1. 1. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0.]]

```
Xp.shape
```

    (18, 1, 1)

```
# The input variable - a 3D tensor with axes:
# time x batch_size x num_features

hidden_dim = 2
test_net = RNN(SimpleRNNCell(in_dim=1, hidden_dim=hidden_dim))

if CUDA:
    test_net.cuda()

h0 = np.zeros((1, hidden_dim), dtype='float32')  # 1 sequence in batch, 1 hidden dim
hn, O = test_net(V(Xp), V(h0))

print('O', O.squeeze())
```

    O tensor([[ 0.4472,  0.1718],
            [ 0.0970, -0.2409],
            [ 0.5319,  0.0796],
            [ 0.5632, -0.1361],
            [ 0.2328, -0.3458],
            [ 0.1899, -0.1889],
            [ 0.1258, -0.1415],
            [ 0.0889, -0.0964],
            [ 0.4953,  0.1053],
            [ 0.5487, -0.1106],
            [ 0.2203, -0.3348],
            [ 0.1823, -0.1800],
            [ 0.5386,  0.0369],
            [ 0.1708, -0.3101],
            [ 0.1581, -0.1474],
            [ 0.1017, -0.1166],
            [ 0.0726, -0.0784],
            [ 0.0503, -0.0553]], grad_fn=<SqueezeBackward0>)

```
class ParityNet(torch.nn.Module):
    def __init__(self, hidden_dim, **kwargs):
        super(ParityNet, self).__init__(**kwargs)
        self.hidden_dim = hidden_dim
        self.rnn = RNN(
            cell=SimpleRNNCell(in_dim=1, hidden_dim=hidden_dim,
                               activation=torch.nn.Sigmoid()))
        self.linear = torch.nn.Linear(in_features=1 + hidden_dim,
                                      out_features=1)

    def forward(self, inputs):
        h0 = np.zeros((inputs.size(1), self.hidden_dim),
                      dtype='float32')
        _, H = self.rnn(inputs, V(h0))
        # concatenate inputs and hidden states
        inputs_and_H = torch.cat((inputs, H), dim=2)
        return torch.sigmoid(self.linear(inputs_and_H))
```

```
parity_net = ParityNet(hidden_dim=2)

if CUDA:
    parity_net.cuda()

#
# Design the net to solve parity
#
parity_net.rnn.cell.Wxh.data[...
    ] = torch.FloatTensor([[2000, 1000]])
parity_net.rnn.cell.Whh.data[...
    ] = torch.FloatTensor([[ 1000,  1000],
                           [-1000, -1000]])
parity_net.rnn.cell.bh.data[...
    ] = torch.FloatTensor([-500, -1500])

parity_net.linear.weight.data[...
    ] = torch.FloatTensor([[ 0],
                           [ 1000],
                           [-1000]]).transpose(0, 1)
parity_net.linear.bias.data[...
    ] = torch.FloatTensor([-500])

predictions = parity_net(V(Xp)).detach().cpu().numpy().squeeze()
print('        X: ', Xp.squeeze())
print('predicted: ', np.round(predictions))
print('        Y: ', Yp.squeeze())
```

            X:  [1. 0. 1. 1. 0. 0. 0. 0. 1. 1. 0. 0. 1. 0. 0. 0. 0. 0.]
    predicted:  [1. 1. 0. 1. 1. 1. 1. 1. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0.]
            Y:  [1. 1. 0. 1. 1. 1. 1. 1. 0. 1. 1. 1. 0. 0. 0. 0. 0. 0.]

```
class Trainer(object):
    def __init__(self, network):
        # If full_supervision == True, supervice net outputs at all steps
        # if full_supervision == False, supervise net outputs at the last
        # step only
        self.full_supervision = True

        # When the total gradient is above this value, scale it down
        self.max_grad_norm = 1.0

        self.network = network

        # RMSprop stuff
        self.lrate = 1e-2
        self.RMSProp_dec_rate = 0.9
        self.RMSProp_epsilon = 1e-5
        device = 'cuda' if CUDA else 'cpu'
        self.gnorms = [torch.zeros(*p.size(), device=device)
                       for p in self.network.parameters()]

    def train_step(self, X, Y):
        X = V(X)
        Y = V(Y)
        net = self.network
        net.zero_grad()
        # predictions
        P = net(X)
        if self.full_supervision:
            loss = torch.mean((P - Y)**2)
        else:
            loss = torch.mean((P[-1] - Y[-1])**2)
        loss.backward()

        tot_gnorm = torch.FloatTensor((0.0,))
        if CUDA:
            tot_gnorm = tot_gnorm.cuda()
        # compute the norm of the gradient
        for p in net.parameters():
            tot_gnorm += torch.sum(p.grad.data**2)
        tot_gnorm = torch.sqrt(tot_gnorm)

        g_div = torch.max(torch.FloatTensor((self.max_grad_norm,)
                                           ).to(tot_gnorm.device), tot_gnorm)

        for i, p in enumerate(net.parameters()):
            g = p.grad.data / g_div
            #TODO: weight decay
            self.gnorms[i] *= self.RMSProp_dec_rate
            self.gnorms[i] += (1.0 - self.RMSProp_dec_rate) * g**2
            p.data -= self.lrate * g / torch.sqrt(self.gnorms[i] + self.RMSProp_epsilon)
        return dict(loss=loss.item(), grad_norm=tot_gnorm.item())
```

# Exploration of various problems with RNN training

In the following cell you can implement with various ways of providing supervision to the network (which one is easier to train - when the net recieves feedback after each step or whn it recieves feedback only at the end of training?), changing the gradient clipping and playing with a curriculum.

Notice the dynamics of training - at the beginning the network does very little. The, suddenly it notices the input-output relationship from which point training starts to progress very quickly.

```
parity_net = ParityNet(hidden_dim=10)
if CUDA:
    parity_net.cuda()

parity_trainer = Trainer(parity_net)

# you can tweek the learning rate. 1e-2 worked best for me
parity_trainer.lrate = 1e-2

losses = []

# when set to 1 the net recieves an error signal after each step
# when set to 0 the net recieves an error signal only once at the end
parity_trainer.full_sup = False

# without full_supervision it takes long to train on long sequences
seq_len = 10

# this enables "curriculum learning" - we gradually train on
# longer and longer sequences
#
max_seq_len = 1

for i in range(100000):
    Xp, Yp = gen_parity_examples(seq_len, 10)
    ret = parity_trainer.train_step(Xp, Yp)
    losses.append(dict(iter=i, **ret))
    if ret['loss']<1e-4:
        seq_len += 1
        if seq_len>max_seq_len:
            break
        print(i, "Increasing seq length to: ", seq_len)
    if i%500 == 0:
        print(i, ret)

losses_df = pd.DataFrame(losses)

semilogy(losses_df['iter'], losses_df['loss'], label='loss')
semilogy(losses_df['iter'], losses_df['grad_norm'], label='grad norm', alpha=0.5)

legend(loc='lower left')
title('Training loss')
xlabel('iteration')
```

```
Xp,Yp = gen_parity_examples(28, 1)
predictions = parity_net(V(Xp)).detach().cpu().numpy().squeeze()
print('        X: ', Xp.squeeze())
print('predicted: ', np.round(predictions))
print('        Y: ', Yp.squeeze())
```

            X:  [0. 0. 0. 0. 1. 0. 0. 1. 0. 1. 0. 1. 1. 1. 0. 1. 0. 0. 0. 1. 0. 0. 1. 1.
     0. 1. 1. 0.]
    predicted:  [0. 0. 0. 0. 1. 1. 1. 0. 0. 1. 1. 0. 1. 0. 0. 1. 1. 1. 1. 0. 0. 0. 1. 0.
     0. 1. 0. 0.]
            Y:  [0. 0. 0. 0. 1. 1. 1. 0. 0. 1. 1. 0. 1. 0. 0. 1. 1. 1. 1. 0. 0. 0. 1. 0.
     0. 1. 0. 0.]

```
Xp, Yp = gen_parity_examples(500, 100)
P = parity_net(V(Xp))
print('loss', np.sum((Yp.squeeze() - P.detach().cpu().numpy().squeeze())**2))
print('err_rate', (Yp.squeeze() != np.round(P.detach().cpu().numpy().squeeze())).mean())
```

    loss 7.896943
    err_rate 0.0

# Addition task

Here we explore the task of adding two inputs marked by a binary inicators. This is a difficult task, because the net must learn to ignore the spurious inputs and to discover the relationship between the desired output and two distant time steps.

```
# adapted from
# https://github.com/pascanur/trainingRNNs/blob/master/addition.py
def gen_addition_example(T, batchsize):
    rng = numpy.random

    l = rng.randint(T, int(T * 1.1+0.9))
    p0 = rng.randint(0, int(l*.1), size=(batchsize,))
    p1 = rng.randint(0, int(l*.4), size=(batchsize,)) + int(l*.1)

    X = rng.uniform(size=(l, batchsize, 2)).astype('float32')
    X[:,:,0] = 0.
    X[p0, numpy.arange(batchsize), numpy.zeros((batchsize,),
                                                dtype='int32')] = 1.
    X[p1, numpy.arange(batchsize), numpy.zeros((batchsize,),
                                                dtype='int32')] = 1.

    Y = (X[p0, numpy.arange(batchsize),
           numpy.ones((batchsize,), dtype='int32')] + \
         X[p1, numpy.arange(batchsize),
           numpy.ones((batchsize,), dtype='int32')])/2.
    return X, Y.reshape((1, -1, 1)).astype('float32')

Xa,Ya = gen_addition_example(10, 3)
print('X:', Xa[:,0,:], '\nY:', Ya[0, 0,:])
```

    X: [[1.         0.69659483]
     [1.         0.4485263 ]
     [0.         0.9703506 ]
     [0.         0.9362869 ]
     [0.         0.13287103]
     [0.         0.14643447]
     [0.         0.68970877]
     [0.         0.9373057 ]
     [0.         0.48905724]
     [0.         0.46621716]]
    Y: [0.57256055]

```
class LSTMCell(torch.nn.Module):
    """
    Implementation follows Alex Graves, Abdel-rahman Mohamed and Geoffrey Hinton
    "SPEECH RECOGNITION WITH DEEP RECURRENT NEURAL NETWORKS"
    http://www.cs.toronto.edu/~fritz/absps/RNN13.pdf
    """
    def __init__(self, in_dim, hidden_dim, **kwargs):
        super(LSTMCell, self).__init__(**kwargs)
        self.in_to_gates = torch.nn.Linear(in_dim, hidden_dim * 4, bias=True)
        self.hidden_to_gates = torch.nn.Linear(hidden_dim, hidden_dim * 4, bias=False)

    def forward(self, input, hidden):
        # TODO: add connections from the cell values
        hx, cx = hidden
        gates = self.in_to_gates(input) + self.hidden_to_gates(hx)

        ingate, forgetgate, cellgate, outgate = gates.chunk(4, 1)
        ingate = torch.sigmoid(ingate)
        forgetgate = torch.sigmoid(forgetgate + 1.0)
        cellgate = torch.tanh(cellgate)
        outgate = torch.sigmoid(outgate)

        cy = (forgetgate * cx) + (ingate * cellgate)
        hy = outgate * torch.tanh(cy)
        return hy, cy
```

```
class AdditionNet(torch.nn.Module):
    def __init__(self, hidden_dim, use_lstm=True,
                 **kwargs):
        super(AdditionNet, self).__init__(**kwargs)
        self.use_lstm = use_lstm
        self.recs = []

        self.use_lstm = use_lstm
        self.hidden_dim = hidden_dim
        in_dim = 2
        if use_lstm:
            cell = LSTMCell(in_dim=in_dim, hidden_dim=hidden_dim)
        else:
            cell = SimpleRNNCell(in_dim=in_dim, hidden_dim=hidden_dim)
        self.rnn = RNN(cell)

        self.linear = torch.nn.Linear(in_dim + hidden_dim, 1)

    def forward(self, inputs):
        if self.use_lstm:
            h0 = [V(np.zeros((inputs.size(1), self.hidden_dim),
                              dtype='float32'))
                 for _ in range(2)]
        else:
            h0 = V(np.zeros((inputs.size(1), self.hidden_dim),
                             dtype='float32'))
        _, H = self.rnn(inputs, h0)
        # concatenate inputs and hidden states
        inputs_and_H = torch.cat((inputs, H), dim=2)
        return self.linear(inputs_and_H)
```

```
# How large an error do we expect?

((np.random.rand(2, 100).sum(0) * 0.5 - 0.5)**2).mean()
```

    0.04074782449327003

```
addition_net = AdditionNet(hidden_dim=20, use_lstm=True)
if CUDA:
    addition_net.cuda()
addition_trainer = Trainer(addition_net)
addition_trainer.full_supervision = False

# you can tweek the learning rate.
if addition_net.use_lstm:
    addition_trainer.lrate = 5e-3
else:
    addition_trainer.lrate= 1e-3
addition_trainer.max_grad_norm = 3.
# weight decay seems to be important for generalization
# addition_trainer.wdec.set_value(1e-4)

losses = []

# with full supervision it trains faster
seq_len = 20

# this enables "curriculum learning" - we gradually train on
# longer and longer sequences
#
max_seq_len = 100

for i in range(100000):
    # note: we need to train on sequences of all lengths in order to
    # prevent forgetting the solution on short sequences
    this_len = np.random.randint(10, seq_len+1)
    Xa, Ya = gen_addition_example(this_len, 100)

    ret = addition_trainer.train_step(Xa, Ya)
    losses.append(dict(iter=i, **ret))
    if this_len>seq_len*0.9 and ret['loss'] < 0.0002:
        seq_len += 5
        if seq_len>max_seq_len:
            break
        print(i, "Increasing seq length to: ", seq_len)
    if i%200 == 0:
        print(i, ret)

losses_df = pd.DataFrame(losses)

semilogy(losses_df['iter'], losses_df['loss'], label='loss')
semilogy(losses_df['iter'], losses_df['grad_norm'], label='grad norm', alpha=0.5)

legend(loc='lower left')
title('Training loss')
xlabel('iteration')
```

```
semilogy(losses_df['iter'], losses_df['loss'], label='loss rms')
semilogy(losses_df['iter'], losses_df['grad_norm'], label='grad norm', alpha=0.5)

legend(loc='lower left')
title('Training loss')
xlabel('iteration')
#xlim(0, 5000)
```

```
Xa, Ya = gen_addition_example(1000,2000)
P = addition_net(V(Xa))
((P.detach().cpu().numpy()[-1] - Ya[-1])**2).mean()
```

    0.0059653684

# Character-level language modeling

In this task we feed the network with subsequent characters
taken from a text corpus, and train the network to faithfully
estimate probability distribution of the next character.
An epoch consists of a single pass through the corpus.

![rnn](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEASABIAAD//gATQ3JlYXRlZCB3aXRoIEdJTVD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wgARCAEpARIDAREAAhEBAxEB/8QAHAABAQACAwEBAAAAAAAAAAAAAAYFBwMECAIB/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/aAAwDAQACEAMQAAAB9UgAAAx5ynbPw/QADAmeABiTlO+cJ2AAAAAAAcIBhiYLI6hnwADXBYGXAJ8wxlDlM2AcoAAAAAAANeGwwAAa9NhAAHwQZfgAAAAAAAAA8gHqoyoAOM8ZntAAAhzTZ6cAAAAAAAAABJlYAACUKsAAEoVYAAAAAAAAAJMrAAAShVgAAlCrAAAAAAAAABrM2CdsAHCa4Nln0ADHEWbFAAAAAAAABjybK4wJ8lMCfOsUxJmYMuCRMiZIkixOcAAAAAAAgjvH2dQtyRM+YE/DImNKQnDsH4Yo7ZTnaJE65XHYAAAAABOnns2qZ07plCEO2amN0GYMEXJEkSTZucxxnDjJ81kbENlgAAAAAAnjtmWPkjC1BHlSfR1TDlOCHLU/CcMgZcAAAAAAAAwRrAvytABLGvDZRRAHCSZEm0TLgAAAAAAAAHkE9UmVABxnjQ9ngAEOacPTYAAAAAAAAAJMrAAAShVgAAlCrAAAAAAAAABJlYAACUKsAAEoVYAAAAAAAAB+GsjYB3gAdU12bKOQAGKIw2QAAAAAAAADGk8WBgTgKgE8cJTEkZUzIJAyxkSPLA7AAAAAAABrwyZ3jFlsSRnSdOUzpgDvnROY4zoGQMoZckTpFidoAAAAAE6aCNnmYMgZgjz5NVm4TPE2WxHEYTpuMxJSHATxrc2CbJAAAAAABPnYMkfpDl6CRKU7JjjFFMCHLY/SaMgZYAAAAAAAAwZqs2IVgAJg1sbPKEA4yQIY2uZUAAAAAAAAA8inqUyoAOM8cHssAAiDTp6ZAAAAAAAAABJlYAACUKsAAEoVYAAAAAAAAAJMrAAAShVgAAlCrAAAAAAAAAPk1mXpkQAdM16bIOYAGHI02UAAAAAAAADGGCK8wZ1CoBOn4UZKGRM0CQMyZEjSuOyAAAAAAAa2Mycx+lSSRnCWO8ZQ6ZRkmfRwHYOQypkSRMYWh3QAAAAATxoY2YZcyJwH6Yo1ibaMoYUuyIJAwBtwxxlDtkya4NiGxAAAAAAAYA5jLnESBbAkijPo6hjCmBDlofhLmYMqAAAAAAAAYY1IbKKoAE0avNrGfAPgjyDNuGTAAAAAAAAAPJJ6hMqADjPHh7HAAIk1AelQAAAAAAAAASZWAAAlCrAABKFWAAAAAAAAACTKwAAEoVYAAJQqwAAAAAAAAATZSAAAnCjAABOFGAAAAAAAAAYIzp8n0AAYYzIAOqcZ3gAAAAAAADiPk+ycPg/SmPsA4DBH2UIBNnwfZxlEAcwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP//EACoQAAICAgAFAwUBAAMAAAAAAAQFAgMBBgAHFCBAEDA1ERITFTYWISJg/9oACAEBAAEFAvYPPpWjQKrsI4+v/PbU6Gun2Hsq12Bz6iZfdj7qiKr8+9dDNlVsMzjOGZSeVdQuqVl2ZlWb99NU4Ou0JcTO5QJ0Cz1b1W28LklwdlIREWi4a2g77M/ljDOLK4Zh4OybTBC27gdpgZufbOca4aVs8drU+JzH1t882pVMmxb2WSzCvX9c2YPee3dKTytb5SJnCUvxHPyfcP8A1PdrXx3iOfk+4f8Aqe7WvjvEdbCri0GLoNq7Lrqx6xtjU52aMsTj2GMRF0dVdrrhvCPOrWiVprGuKqYUQL18Qmxaff1Po1Y2C8D68PiyUIzhcgwLlWxwyH9CyyWBoKMJdIoOg2q0S/X8UX1k0++JvKQs0nGD9jqb4w4Dolh9xsuPwhHm1rgU+1qX3CaPVGujprg3JECQRLM2i3R6HZHuxA63R/owLVX1t13X3V1ZYAs5WC8JMdGWw3JSpa0EVFVe4SgELvr5NWGtNd18XVHckd0oAG32y6iVlN5MjuXzjUBy1KXkpTVxqg8AATwZGGi9YqH4ejQZs9z5fZeAU8nxF6v7MOeBsGKaLGWYG0EW5f4W0Odg2XlVa9favoK/VZ+85BtJqWs6WlPEpYhGiX+gYejGM1J9F9ZVM6oWcHH0LR1A107vQa3GulShGfH2RzNmbUDaqA/WheCemHPt2N4frbT/AD0Sc9lqCvFwL1gdt4iOkcj1tpgRVJNJdXpzY7c1a9UMsx4fMbWXr3alUyZrOyyWYV6/rOzB7x27rQeZrfKVG4SmeI5+T7h/6nu1r47xHPyfcP8A1PdrXx3h5ziOHGxK5MhDxmFfYQVSHUPsirGyV2Quh2GtQlvGsvVvTeExPrWCVJZM+IQjXE5EIbYtOviV6NWNg0hUA9VucfXF6CNElbH9iP6FFEMzQE4a3i8eoqq0G7X8D31lUe+JvyQk276MNjm1xW6ndiWw8bNj8IDA2taAm3FO+4S46oxiXEBf1mL06Kr8SwiPRbI/2QLWaMbQuvVT/LrepszR5iJ680quEkejMZbspTNhiqTafcJQBF318merZ6+gD1R4WoIv4GY2yPqMgTw3umTqTfTxS1SbkqJTxqg8F4TUKTAJlUQuhVL763o1bNpuPL2L0Gvk8ACrnHD0GyJaiq426F0iJ4eRWjun+x8qcvXus6Cs1az3nANpdS1pUzrjVCE7cwjUvpramejGuxaeMTUZRwewoWDqBLvy+lNuNbJzjEuPpjPDVjEWasDCwLwT0wzCzZHZmttca7XfLsvQVZvBeGn7iGjHFI9bK42wmkwtq0toZuasBWMsj4fMXVXj3aVWSZLeyyWY1oNU2QLd+3dBjztc5ToG6MzxHPyfcP8A1PdrXx3iOfk+4f8Aqe7WvjvDlLEItdiWXMAmQjKHYWYOBTRsayOw03Vk1dhzgFZnXH66sXwmTCtYJSlyfnGPpg9KKwkuOvqL9GjGyiwJEOLbwSijCapl+xp9CSL2pYCoRZG2qF9dy+1DgYmswf3xeYaMg6yUWOxL2/UHJsTibxs2PwrmJ0Fi9Nu6V9wkx1JTY6S4JybgcLGfrgnHR7HsWyh6uPHbVhKoWv8Az+uVl5KTLpZmBwlx0ZzPelKVwIaOfT7hGvgF3U8mYEskKELVXoqQinhawIvDenX0a9tX/fX2+nBGKk3JYEfjVB614bUKZ9tF5Ckfh4NU0a7fy8qehV8n1oSq/H+gQnxLWYKaSqstvs6+CwV29f8AKfDx9rehqtXn7zgCwyla0qZQxHEeL5VwqDxl2d6MarV5wpVJo9w9d/DBjQsHUCX/AJPSu3GtETrjbGdUbMuDa6rlYEVgXgnqBmMtndFa02hr1Vk+whDRO9e6LP3MJKOHd6zhiyNiStdTpLIrc1QK0ZbDw+Yeou3u0qup/W9lmcxrQafsYO7du5hnsdd5U661RGeI5+T7h/6nu1r47xHPyfcP/U92tfHeI0osuP7qKLI7F3IKLBwfDk5oiRxnOI47YtxJkdhJVYdQp1B2PBhVGviFMa6oQxXANURQYsDKGGbiFFJISzKPrficqQl5FBCCu2tT6vACGQh1RMp3WGfsh68w2T7M/ljDOLK4Zh/43//EABQRAQAAAAAAAAAAAAAAAAAAAKD/2gAIAQMBAT8BJl//xAAUEQEAAAAAAAAAAAAAAAAAAACg/9oACAECAQE/ASZf/8QATBAAAgECAwMGCQkGAwYHAAAAAQIDBBEAEiEFEzEgIjJBUWEQFEBCcXSBkaEjUmJygrGytMEVMEOSotEkM1NEc7PC0vAlNVRgY+Hx/9oACAEBAAY/Av3DzztljUEn2C5+AOJYFa8kQUtpwvw8FuvlZAWDGbcBWW12ybz8OvJjMiu2d1TmDhmYKPiwxIFNssphBPnsBrb4+44K35w1tiQRyJIY2yOFN8p7D+/dVkMTMLB1tde/XFlkMZuDdbdvDXt4YjIkKBTcgW52nA/f7MPDldhIyq2Rc2l9bjrHbiKeVLtNXGpmjByjKEyx9f0Yz78HLPAFvoDASfx4qpPFyIpIoxvebYkF++/WOUjmJlii39QIpTlbfyE6Zh2BnF/pC18U1NzzuUCXkN2NuvkUpjhM27l3mjDSwNtDx947erGz2kkMjRpK0+VzlMrsGLW9Of34mkNRIU3UYvlXnWL6cO/442k7plSeYSIb9W7Rf+XGfO1rWydXpwzZ2IPmngMNd2e5vzuru8h2JSN/ts+R+5bW/Ey+48vaOxh/s8COp0tm8/8AEnuPKLMQqqLknqxJVWyss7xlewXuv9JHkqyx0V4HtTUp3qDNZS587uc4pWrY91Wbsb5NNH6+HJZgpcgXyrxOEr5aAeMCUVFSBJH0JCwJ6Xc/u5VZTbMhM9VOu6tmUWU9Liey+K3xmkyUM10L7xTaWNytrX+t7vJdg+uN+Xm5e0PU6b8c/Lm9cq/zEnkuwfXG/LzcvaHqdN+OflzeuVf5iTyXY4O0qQGKqZpPl15g3Eo1101I9+N5TzRzx/PjYMOSZJXWNBxZzYYrW/alHlelp0Vt+tiQ81xx7x78BlIZTwI5OaqqoaZe2aQL9+JIEr6Z5jV1LCNZlLEGdyDbvBHkb1Etyq+aguzHgFHeTpgTbY+UvqtAD8jGPpfPPp07BgJGixoPNUWGN8imkq+qppuY/t+d6DcYegrgPG0XOkqCyzp84dhGlx3jt8MVPSxiauqL7tW6KgcXb6IuPeB14E9Z/wCI1f8ArVAuF+ovBfZ8cZWUFew4afZLCgqOO6H+RJ9ZP+Ya+nhguY2gmRt3LC3GN+z9e8EeF6ChfcLFbxmrtfJcXyJ9K1j3XHbjPFADOelPJz5W9LHXBjqIY54z5kqhhjf0O8qKJdZaEnMVHbF1/Y4dlutJonEkUihldeBHkE1J48kFTFIY2jn5moNtCdDiliPOio4vGSPptdUPuEuK2GWUCnjMUS83hK3UT9qO3pwsq00kce7nDlr6EyKRr519SOweBK8dOgkFRm+hwk/oLfDFRVS33UEZka3GwF8DxKuilc/wycr/AMp1xtHaB1LymmjPYkfNt/PvPhhZUyXM0cfPGnOcL+t/ZhxDUZS0cmSVScmYaWuPOvw69D2YhdkeNmQEpJ0l7j34p5holdGYH75E5yf0733DEM9fIYoZJN2HClrGxPV6MVVfTVMdXDTxNK26e/AX9mKXnRmpaWMTM40Z5JBvG49rE4cRSG5jlySrcqrjS2nnX4ddwcQs6NGxQEo/SXuPg2js8aJFLvol7Ek1/HvPdgbPranxWcqHBkHMN+/++FlhkSWNuDobg/vWlkNTnbjkq5UHuDWxVVFZXCGmeZmSOK7uVvpdj1+/D0dLvN1UUispkfMbxuc3/EXCfKpnNa9VIbdLpCP+X5P+TFe5PjKrOViiiADKo5pvc/ODYkMkclGAOm5T9CcRZ33s9ZRxwZu2SQBPxNithp2qd/JC6JvK2YrmI0vzsLJtOuaVv9Km5o/mP/1ipoVv/haqVdWLaM28XU/RdcbPbm7mnkMxv1tlKqP6ifsjFDTGNDJUVLmVw+i3zSueHC9wPSvg2TRPnADSVTFHKHKqZOI14yringoJXjcTBnaqqpZAFseAJOuKxvGJqyvMEix2O7TMVNtB/fGyKyIg0if4oE8SSll/GT7BiipisTST1DNM4fhfNIx4cM2ntGEi3MxXK5No+wr/AHxW05kLQrTwyKthzSWlB/CMbTmlMwWBYqX5Gd49QC56JF/80YE0NV4tRCIKWld5pL6/OP643tPJUSz9bvKQD9kae+/7+KamsKymfew5uDaWKnuIJHdx6sF4rq6nLJC+jxN81h4CzGyjUk4hql/8tpSWhb/XktbOPogXt2k36hfwftONGkgdBHVxotzYdGQDrtc3HZ9W2ElhkWWJxdXQ3BwudQ2U3FxwODPUSCOMe8nsA6z3Yn2hVLu6iosqxH+FEL5V9OpJ9NurwmknOSgmctTTHoxsTcxHs16Pu6hcZlDWNxfqwHtzgLA/9+jGWnhSbak4tFGOke9j1KOs/rbCwl97JcvJLa2dybsfefIhNz4KpRlWogbI4HZ3juNxjY9F+1RIK2XIXmp1JUcLmxXrI6urAO0aqfaIGu6kssX8i2v9q/JeajqJtnSucz+LkZHPejAr7bXxX7D/AGoo8XiVt6lMup87r484dXUeGBUyyS1tWOE9S1yv1R0V9gHIaOVFkjcWZHFwRhmo9pzUUCC+6mtLEo+1qB6GtiSpbab04SVo2SngVW7Rqc3URh9yhzyaySyMXd/Sx1PkizR0l4HtTUp3iDNZS56+5zilNam7rN2N8unT6+HJZgpcgXCjrwlfLRg1CyCoqbSJ0JCwJ49z8qsptmRb2qnG66Siynpce6/vxWmppglDMGQvnU2kjcr2/W8l2D6435ebl7Q9Tpvxz8ub1yr/ADEnkuwfXG/LzcvaHqdN+OflzeuVf5iTyS50AxsfLtGmYQ1TNIVlByDcyjXs1IxvKWoiqY/nQuGHw5JlnlSCMcXkbKMVsp2lSiJ6WBFkMoykh5ri/tHvwHRg6Hgym4PJHjdXBTX4b2QLfDweP04marqWWMygMQZ3IsPQR5G9RLmYLoEQXZ2OgUd5OmN/ti07HVaK94IvZ557z7LYCqAqjgBjfZDT1fVVQcyUe3r9B0w2z663jSrnjmUWWdO3uI0uO8dvhipaVBLXVF92rdFAOLt3C49JIGBUVN6+t/8AUVGpH1RwUejFjww1Rsths+q42UfIyn6afqNcMzRmCeNt3NC3FH/7sQesEeGShopDBHDpU1YGqm18ifStY36rjt0LQQKsjdKVudI/pY6nBimjSaNuKOtwcb/ZwkmpF/zNn3vYdsXYfo8D3YjmhcSRSKGV14EeQTUjVq01TFI0ZSo5moNuPDFLH0oqOHxn7b3VD7hL78eKNJEkYhVzmNmzM1lt23s3w7cRRR1MoZXLSKb5W+T0QDhbzr9unWbYWvHToJBU3+iNJP6C+KirluYoI2kbLxsBfAFHXxPIf4Tc1/ccbSrzqZJjTx9yRErb+feH24qalzlSGJpCQL2AF8AVQM9TGkTzRQIdW7h1qSD8cIOfqztaRCmW7k2seoXsO62KWYaJWxmnfvdeenw3vwxDPXs0cMsm6zquaxsTr7sVVfSVUVZHTxNMyxPrYC/DqwzB4zWJHnLy9GSdjrfXznPxxvCstTupMrQRRM12y8HUAm3OB4dndikjMkkpWJRnlUqzacSDqPb4NpUA0SOXfxL2JJr+MSY/Z9dOaaYoHDOvMN+/++BLTzRzxHg8bZgf3rSyJKXbjadx8AcVVRW127p3mZkig1bKTpdm6/fh6SkVliqaQMuZy1zG5zf8VcbQbmlqiohtr/AXLzPfvP5sVqSmNooAqWjBLl7BmOXss6YZYxKrW4vCy/eMVUbnNLUhqRT9J33a/eMVkFOrrPJC6RmSokyhiNL64V9p1j1Lf6UHMX38T8MVNAgI8UqpUsWJ0Y7xeP0XXBp1YKHdM9/OTMM6+1bj242pVJULFJUskcMhGbIWCIt+4Nc/aOFa97i9xjZVFJfKrSVTWYroqZOrvlHuxTwUD+LuswZ3nld+bY8BrrirO+mra8wOIiTu1DZTbQf3xsyaFh4q7x1LZuLKBmX+rJiZt7Bv6qszXY25ubW/oiW32cKq0czrfiDHr/Vimi3hCSU8rmE20IMdvvONpzzBisAipRkkZdQC54H/AOUe7AniqhSUQjC6lpZCde3++N9Tb6SotYyySH7hp+/ilpiq1lM+9hLcCbWKnuIJHx6sMUuksZyywP04m7CP+74d1RVd+kwGrenDmTKIwOdm4WxDUwRLDsumJeAKuUTyEW3n1QCbdt79Q8P7ThRpImQR1USC7ZR0XHaVudOsHuAwk0EizQuLq6G4PgM1Q+ReA6yx6gB1nuxPX1a5KmosBF/pRi+VPTqSe89w8LU85ybOmkLQTHRYmY3MZ7Bfonvt2X1F8DuwsNPGlRtOUfJRf8zdijt9nHCQZzK9y8kh89ybs3tJPkSyuHiqUFlqIXKSAdlx1dx0xsei/a7yCslys80Md0Xhc2t1kdXUcBq+pn2lY3CTkCMfYUAH235Lz0s02z53N3emYWc9pUgqT32viv2F+12Ap4gRKkMdyfO9vOHV5px4y7SVdXa3jFS2Zh6OpfsgchkdQ6MLFWFwRhno9oz7PgQZjGSJIlHoe+UegjElRJtSaDJM0bRwRxqe0akHqPdhxTx2Lm7yMxZ3P0mOp8kWZIF3EpFNS3kUXspc/c5xS+OrlrN2N8NOl18OSxVc7AaKOvCV0sCNUxyCpqPlF6EhYNb3Pyqul2amepnG76QFlPS491x7cVjVEKihmzRlg4J3kblf+vyXYPrjfl5uXtD1Om/HPy5vXKv8xJ5LsH1xvy83L2h6nTfjn5c3rlX+Yk8kLMQqjUk42Q0VbDMsNUzSNE2cKNzINSO8jBekqYapBxMLhre7k72pnjp4vnyuFHxxWztXQrTvSwIszNZCQ01+dw84e/CyROssbah0NweSoq6yGnZuisjgE+gYeGSthilerqWVJXyFg07kWvx0I8jeeQM1rBUTVnY6BR3k4FRtfLUy8VpeMEPoHnH6R9lsWGgwJHTd1I6NTFzZV9DfpwwdnVxDVIXPFOBYTp29zC4uO8HrsPBFSUirJXT3yB+iiji7dwuNOskekColvWVvXVT6v7PmjuHgap2cRQVnG6D5OX/eL1+nj34fNGYKiJt3NCTfI36jgQewjwy0VHKaeGE2qapekDx3ad9rXPVft4EU0Cxluk/F3+sx1PtwySIsiNoVYXBx4xsxWemXWTZw4W7Yvmn6PA93HEc8LiSKRQ6MOseQTUclX4pURSNGVqeaLg26XDFMoYPDSQeMacM73VD7hL78VsEskdkqdxBlBGa0YYi/b0v5cbYzLKENUDHvAdRuo+F+8N4BXDp0Dipv9EdP3oXHtxU1koYxwRtKwXjYC+AtLXR70/wZeY/uPH2Y2lXnjJOYE7kiJW38+8P2sGZIxK+dI1Rmy3ZmCjWx6ziKZKgRKaiFc1xZgZACPdfFxqMUkw0Wsjamfvdeenw3uIZ67OIZZN1mRb5TYnX3Yqq6jq4qtaeFpiiNzrAX4cRgmV+fBC000tr3fVna3pucEysPGhCDKKYnR7ebbW18U5Oa+QXzCxv4Np0PBElFREvYsmp/rEmP2dXStTS5Q4kZbob94wJaaeOoiPnxMGH71pZYM0jcTnb++Kmorq3JA8rssNMNct9Ocf7Yeko4zHFVUoYXYm7Rsc3H/er7sbNLPGTA8tRKNf8ANcm5Hdz5B7RhppIM7mVwEhYGy35p1txFj7cbTqI1ko5oaeSRGbIdQpPfisgHSql8VT60hyD8WKyCngVKiSFkjZnawYjTAfaVVJWN/pxcxP7/AHYqtnxrkFJVSpa99GO8X+l1xQR5Q9Ms+9mDcCApy/15D7MQwvF/iKqse3AjVmkPX8y/tHg2VQyLmVTJVsL9Srkt75R7sU8FDkpGWbM8jlm5tjwGKuzS1teYHEbu2RQ9tDYfrfEPi5G5qhE5z9cRKlh7VuMbUqkePeVLJHTlvNJCogPdnJP2sUQWNnEr2ZoxmUjIx0OKMBmWOUNmjIHZjac88e8WER0q84jUAueH+8t7MGoSpWiocijKoLuT18eGN7SpI1Ra2+le593D4fv45KZlSsp33sDP0b2tY9xBI9uGsDFUR6TU8nTiPf8A34HGgthmmKiJRmYvwGIKzJu9m01zTAi29e1t59WxIHbcn5vhG06dDMhTd1UKC7MgvlZe9bnTrB7QMJPTyLNC+quhuDhN4ubIcy+nh+uN7O1hfKqgXZ2+ao6z3YnrqxQlXU2G7BvuoxfKnp1JPeT1W8LxTnLsuaQvFMeEDsblG7ATqD327L5XUMp6iMIWFyhzL3G1v1wkNPBFVbWYfIow6H02PmqP/wA1wkAcyHVnkbi7k3ZvaSfIlkkUrOgsk8TFJF+0Pu4Y2PRftmZlrJbO0qRXjXhfojrPwOFetnn2kym6ipIyA/UUBfbbkvPTyTUFQ5u0lK1s31lN1PpIxX7D/bMwWCIFZUjhuzDpDo8ed2eacb/n1FVa3jFQ2d/Z830C3IKsAynQg9eHkpK+o2ZAi5iisGiUD6Lg5R9W2JaibatTEUmaNooVjTTiNct+B6rYZaeLJmN3YnMzntZjqfb5Ik6pHuZiKamu4vojP+jnFL46LVYjAlt87r5LFVzsBovbhK2RYmqYnWpqDnGqyFw33Pyquk2cAamcbu5a1l874ae3FZJOkfiU2aJirXO8jcr/ANfkuwfXG/LzcvaHqdN+OflzeuVf5iTyXYPrjfl5uXtD1Om/HPy5vXKv8xJ5Lsd0QssVUzOR5o3Mg+8jl1sxU7p6WBFbqJDTX+8e/lyrIpRjVVL2PYZnI+B8k3NpC3N6KE6M2UH0X/vw18BJNgOs8pYBL8sxyhCpHmlvuB9x5JklOVBqTa+HMEgkCHK1vQD9xB9vkTWvzjmNzfG7F8uvFiT78Kg4KLC5vhqmasE0jxRRuRFlvkzHt68xxMLiOd53cvL8pmFzl875uUezG0IGEdVLJCyxpGmTW3e2ASpQ9h5EgiKrIVOUuLgHvGKuWScf4iTendjh8mEy6+i/txT+MZvGGGeXMLc86t/UTyJqaF4o0mgkhZ3U5hmtw+Pw7MUNnZU395ViFxlymw4fOy/HEQWKLLu5P4p7U+jiuk3bBHpoRny6Fg0l9fQVxn3jZbW3elvT24dt4zA2shtZcNeRnubjNbm93/s3/8QAKRABAAICAgIBBAICAwEAAAAAAREhADFBUWFxgRAgQJEwoWDwscHR8f/aAAgBAQABPyH+CTva5hIvSPQ5HYAyi2E6mteTs+kFSIEp9xwHXKUivfbUI2Rgjq/sJzn2yXKwdHfQw5HiRRCQPeKPnqnBwBAUrBmH+n9ZAXINQjRphK8/zp+yGkShBJN2J4wMDRpKAWhoEuYaRhyLhF4GRYalKQyOJFeIsAj2BhIBaLHeKSi4i5DRJ+0QXjF08EOlhP6yhrfZOMnQevuo3yIrEq62ICRFSTniX1EmXcTvn7NoZYCuEiahDByxDcKEfak2BWuMYjeykKU/oYQdAwZFm6tPpPWVfQbn4TPzirCkRU9VN+cUuGafFBr3+DrkVK4WV1Znr7yeKNwEpt6KO/u0XsXgBtXAC34I0HzRXsfxZmwm160chigN5LxrxUaySJmL19qC2BgXolCXy43ZCBB+KXQGkePui8mANBKRxVcplNWuWBBPcZi4cf40E9SAnqQsEuFJmHlBxIF8jvDDVpZ+T7dKn1L2uHdbpbAOyXnh3h4QlEifaZc9E35RgErUAqlKIDyJ+GVkSEg2F5QA5UyJHnHxAfvZy0bdHMBB8GeC9P5QQPD4MKS7UFiDkI41ND6txKK0RNNWkWoMLejcJn6P0ntV4oYyEpE9ZYgiC3oqJ8PqshES4k2k2QgeQ5+rGNA04AMi0iiGhRDTcxvyG/3mzOiT8OP9vC51nY8pHNhHUGSskR6T8DnStizyAcM5CkjxJ/hvkQ8Zasn1SXUsscoyHEyxKBS6YGqKmPodf7kDT5n5suDNCYfMsoPRh4/+bRPlGEdkS2tX9vyPQxQxc5g07NHxLPbyT514EwKLROIOyMzJw1DmMIbqbqP/AHjz4MNlIibAWi3GVrYMgeD2rnNk3meHB+jA6MjRcDYF7EyCEUJIxgnqiiWhUnj6GUjukn9YA6AcYbVO0BPqaeGETaSzeE3/ACgNrP6dOPgyU0PLW8DDnM0lJaA+J1gG+3NkUKpgEdmXlOGLjCUAgG3br4MuJt/w1D57xutNsxcr+z5zaEY8JAsSdyPrGgrDDC/sT0YAMy4wIsFd7zOJk5PmjHW5w/AVDzpSMHLfWElrGNRI0+oDgeE6HI0x1CbBbJwe840+mgsS33B5MG3NAzk46hTMz5MdSu6KzHZI9HO0SkZgZQCeLY0o/wChihNm9dYaUFbuzxgXqHvC8qRosokqz/rk+FIqih0o9IfzovxYNJbiWOSWwsEeXtOA/wBJCKI/QIBJCAO8tirAIde5guopvqQxKDLqJaiBaiJQoYmLn7EpzUnI58h5yi0ktq0C00CVdYp8sWmaRKWvd3IX602HKMk/Iq0jzwdZCky7MWa26gxP/D9YaT0IP2ylJ6JQZeXXSPReJSuCDj8JRs5PtJJU1w+DFIk0G5UkcIbXUMmKAc75AZQj7HhdRCNsgPIC7wHx9kKoCsBDt1Qs/kpczcISc3OZ+xprCC2xHZk8aJAhvhDgB0ZHTTBvgXAB3fOWDYQmaWsHFwcR+IcFLTPTnsRwbybLXiiGskiZSPtQWxMF9EoX5wHf1pvispAGoPH3LVjwBRZWHF3TAYkZkIQSiYzHBx/jQT1ICepDlAEq6MSAyX2WLwF9mXUyP7qX26idEPy4MSlt4CcKH62Gf2YAeH7SX/5vaW8dOWCldGUUEdn4YWQUgAC8oAduQw1AhOB07Z70xQTAcBlLgXf0ensXI4PqVEnQScpHmCoH032+Gqr3xBdAmSPQXGeL+pE8y3gMAqkcclVcboFTPiPexNh+3IoUnkR5AOfq45AUqAmksRCChcFeaLI7nP7cPWUFe8OMbpe5ZbDz4gTKqSjJWSJ8fgU/FNKlDtKVc+MvAdYaUl+Y/UeMb703ROT4KKTCSb78FAhRUuhCfoBohLmVHmSe0dhgqkmJklHmDODwnv8AiX4kzaME7/1FtejEnyXhJMc61isdz7MOlRSSRDo4RUSXhSpCbgaYDQ5/Hf8AX/oYVmISGkAuLaHAZBASuDsq5MmOsA3JUQJ5I8DOaFwaoPkbPaTtSoU3uxj1Y5v6BXOgaC/ih0EcYRajyBJEJjTwM1zFf0kr+VVxyi/0QytGts5eChHHtk+/WyU2avSQU94ss4QgstT5/Q/WRhb7qSgvQrHmcRlNQ+fJD6yZXxASqaPM3zj6LC0IhOp8PrOdGofCqQ9YO80kdlSv96ciUEzILp6s2jqgtjJt8pcmU8hwIGsB5SHkyMoSz8sEAAzUmR2J1zeXoM1UGNO3SjxigBLPRPzFXwOG2BhgZlJNj0q8sxk2GCurxIf6M497hvVXrnS83wct0qp0eJY9nsFFTvRZ/wBMPdnDHVUfYx3/AD95Nix7FjiQWM0UMU/2YdCxRHHwCJAgQS5qsUFCqMYXM8Rg6qlojAcAXN6K/SxRj4ij3AUXQlZBFgc/YmbwIsoQS+mteBbiY95wd9JTJw5QUT6nJhmSYfIlSumMjIASMJyWYsSS6PWI3Ivwan3zvglA1kBOFphxIRo0UfhXLAaVPaldnYyfh6rWKAxK07sQ/kuos0s6Sjv7YfvpOZ5cj2ZUI3/4gKJCDboMJAqEGOw0byAfsTtCcBsTkxBPw1DoQHBPGS3+jR2TERJKRvnPcw7Skq+38RH+QAty67Hgx6psFES5pCy/P2ptYVAvq6yLjCrK8JiUo4r7ofv3Ciy9sADuDE1CZiYfr/GgnqQE9SD8jJQB24R0ztt5AUvnGZjAqumVfaoNmEzfdMl6Ki3AXA5eGGHqEm8Jv7QqdPtXGX4wMvDFBiFkV3+GDZQqSAvKAPeHQhueILQ9MzoawCACgOMg6mH/ABS48pXI4oocSdBh0TVQr6ghhqRWV+ghYFXh4oCfVx8SHzf0e32d06GCfoeBkmd8+GDA8g8giDR9DZiJFJsEXLhALTm1R24dyF8pcAMUOB0jiKzyy+YcPTqhONFow3JH8A9BTJlJOGqlnxiGj3SlFeYN/wDORhU7EmJRK1IpMYAIlDIZXscTz9NBC52Un/UeGBAeUWpRKXWcVLEs6IP7M2RlPf8AuN/0xVXRpLehTxkN2BvJTxLTrAJBLE5zg954r64/vPGLcZF6SBuLan1kTpH03Z9HJgA7JrWtxMuKTfGbMISSezwHk7yEVHDULkbmfpC+kDl4V5JmEGhYmuQPOaE2NPyfyzXbMZP6wsGjcJs+WNkveBjfXZEVbSg7YQVAGE42LAKY3cRiao1WVZ+1mr6lvXKxhgTs5xGgH0BvQh9GSAlT8ibanw5FybS+JbXscQ8iJobq3pexy4kTEmiOf9HlvbClKF5joEamVL+ljLyEtNLJR88fAxrZoktlOveWouaZheaNsGk4ZIGT/TectCzpTsRpBXM9cjyrIC4tNhfjAdKTNI4kb7ygtWBKNjriUCFICeSPaX1gWXyao8Qj+e5rBoTKPnnqxYYbtpFHVHTwKLFMIghKwHLbklzBgRcq0BgI2Jhk4cD1FIE/QusJ6hC5aladhRQxRB7yhOycQf8Agj5y1es7tBt+BeG05qN1FInDhCj6gMVRJg3Kq6lVjJruZOHAuyY9iX6X7yQ4sRj8pXO1qSjBPd3pTOpSCiYK/CW3gGrwKMfs5HLBO70SkohbPNgU+ROtMiThJO/thSM5P2BPkvnNI9rEbkaD+hkv30iA7JUHpHj7AvmhyB2Jj4XW7hgEDiHJ4EjDyY0EpSOG2tU7QFfKX8SBHpNBiep/cMrglaESQiKhb+ftXIggxPqXADZB69cSnqr7gbigyr27lgcgRCOidSD9f42E9SAnqRrMEYs9+F8/eJ1yZOHwfrfeCcINr/8AIPz+JZHzIT/qFnrBM/GG2NKkAfdvOtEqDZpWOmrT9rXwoHACVgOhwpe01FL7kZ5B+Ey+7G3ida0Yst/IDLZnnJmYkgoPLbkohTkoEhBKYjIac9ZOoDVpfXNhBWKw3G4ZkiMRgNxyfpT7Bq0PxtIJJPEnvJaEaGot1DfUukTJUBRXp5BByT+o+xfE6DgCx1FvMuifj40iWVtFLB+mBc6UmOpe3/rkdDuUddEKf6jK/gNKz4W43HjF4DgKHUE35XVRi154lHSAr3Le/wDDf//aAAwDAQACAAMAAAAQkkkkAgkkkEkgEkkkkkkkkkEkkkEkgEgAkkkkkkkkkkkgkkkkkkkkkkkkkgkkkkkkkkkkkkkkkkkEkkkkkkkkkkkkkkkkgkkkkkkkkkkkkkkkkgkkkAEkkkkkkkkkkkgEgkAAEkAEkkkkkkkkkEkkEkkAEgkkkkkkgEAAAAEAgAgAkkkkkkkAgEgEAkggkkkkkkkkkAAkkAkkgkkkkkkkkkkAkkAkkkEkkkkkkkkkgkkkkkkkkkkkkkkkkkEkkkkkkkkkkkkkkkkEEkkAkkkkkkkkkkkkkkkgkEkgEkkkkkkkkkEgggkgEAkgkkkkkkgkgEkEggkggkkkkkkkkkEkAAEkgkkkkkkkkkgAkkAEkkgkkkkkkkkkgEkkEkkkkkkkkkkkkkEkkkkkkkkkkkkkkkkgkkkkkkkkkkkkkkkkgAkkAEkkkkkkkkkkkgAgkkkkggEkkkkkkkgEkkgEAAkkgkkkkkkgkkgkEkgAEAEkkkkkkkggEgAkkEEEkkkkkkkkAkkkkkkgkkkkkkkkkkEkkkkkkEkkkkkkkkkgkkkkkkkkkkkkkkkkkEkkkkkkkkkkkkkkkkgkkkEkkgkkkkkkkkkkEkkkkkAkkkkkkkkgEEAkkAkkEEkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk//EABQRAQAAAAAAAAAAAAAAAAAAAKD/2gAIAQMBAT8QJl//xAAUEQEAAAAAAAAAAAAAAAAAAACg/9oACAECAQE/ECZf/8QAJRABAQEAAgEDBAMBAQAAAAAAAREhADFBIFFhEDBAcWCBkcGh/9oACAEBAAE/EPsGcGzU+DzQh5Xji4Ixm7hpDUNBUC3A1YAXQbFPZj/j6jc8SsgRQ04YAQA1A9x9DXhcAjkAmhWKggklNfbKbczCY5QGMTFgIdgoD5r2ePwEJWgkzVUYGaffWpXw4DTRAWAqKcf5FUGcBQQIVABY02hY2RBUcAquA6HEkoq40TlIWLQBFlVhg8SweTMym8HAggxQC7Drg8OUwYQqgCzksPUAAioMtEyGJq5qprHEBKC9QQYJJ6PDsDqapjYLTUCt4cJ5JIAs9KAHD5uBkNwpCqR93U2miNMQJScTyEaFIIuOyhrwJPCLl3gORehzWRVbT0SbUWJjEIGJiU0665PwHKYKGWqg/wAw/wC/UyyDAhLC1YrM6Hn0mPoqaqGAAqvU4Z4ViN5OxnF/Fri+8IzBq0A+8ALtMqYCUfRULRY0PQePZkBRiEwQK6hvGJuNChikVSoJGvSdU77WMoYaaQYp2ufcIxUaQxKOF/jMr6rlfVYIgAVXxwvWay4fMB2CDsHE1HPJJjo9nn0nTakH7oA/t4b50PjJg0CoXMUDQiyxRExE8+nF9v6ZgHgm9iDUJAQiRRPwzHdDI6E3t4XnhhWNDTRYETM1LeJRvha/AAcbHEUDRoIFa3Fa7xttU45b1alGQX69yxPkJ2RAeB4hGAZsDIi9PABneKnM1X2RWJ8c0dRMe12FdiS1acAFP+FHMIF4Nw/WLLxCgMqkZREcASEqKdLuywYKwDOAh5CUcRNH/OMwmkfmnQJaQkGPACEFnQ7FAj7P4FRgTUmkFEg0JQ5gMbx0G8mJ7u4cYRCpKJkg3ApiuGAoYgYUZ1SSfSCsTwDQhfAg+JqH7s8vAhWkLwfUKMCd4oHsR7PJPCQEfXzt7pZBXJqCa8csTWUHs1vJmhTPYAhiU5hJuaVaWhSiUZnCkO8EQXVGhTA+PGAaCGBI4bC2ZyG3U3RMVQeAT25PAMocgCmMXDhgnqJJHYyhACFURl6qVQKZRUijM+hwp2hcAmSPAPdBLnUKJMi42oda8+RA7+lIP0/dNbgBCAeAoYG17XkSSNEp54KhL2qvEAVQSI2UJiIwFFwWUQtQIRkhhoFvAGDU9dOM1BdFED2Cxm2qHE1B0nwrErOhkvMb8SuuWKye1idIgCUUZyitWEWxB/eXzxnXvsBIFKVVUrV1dN0YoyI7KxBt4sV60aNZpygQT6avA1w6umYUbRTgGSNV8AoaJKQLVSaloU0FYIhYopwdKKtdwEbgAnGoHxHLuOWorpF04uQGAdB5j/U4bN18TALMKRUleCAS6Ap6Ai6lKqJ8Z3UUZDQDZpvdCsgIFt6gxAjQfvz3uH0SZ6IGCEBgYFwHo1n2IqlscdzsMMVS4AbXhrO6osRr0sp+paQZzXETuXHw4KUOik/SyD7jOJpd1NurGC4m8QWYRcAcX8VYAqHK36rup1ACUCCl9Bjz1IgIGlQEwHKaSJThEKdJXTdeGVfJlbH9v+LgeMPyQmKwsQAHIjDAhyM+F0OMGD8LQ/4IIwBWoN3vIB3n3Rxr+C+FykqQQDSGxBBwgkQ4AAADAPHo7Hkg4nJt8tuckEk9QTJlC9k5DnzA5hhVicMXoF8iypByIKIkeIEMw1KJTHUhgHCHhToTp1GoiCkt2YcSqQwFAwMAz8QxX0WpBqkScGCBbeBigIlZGh0nTQ9B4/iUFGIVgoN1DeMgvBhqiqEVMgNekYmwq4kYwxaQMURIpXt34BwhVHC/xmV9VyvqtB7F4Bqq9HFKa4mREB4dpeTjIyoTvtYX+/T/AO2s1lwP940h+M6qoSD/AKHAdWPIdIKJ8npPgmhz9ZK+C8VQzoOEFRBEbyfhxKhUonpvaBGobxpxTaFWQHkQAp1EV/wJ/YDA50YcfooQ7dT6wlOb5uvyru9lBej9UVZe+LVGMoAzXm4FRJ97h9lsALsggKgUR7E5b4B4tYFwUvGZxEh1aVg4G0AjTB9ATAk5HKsTjCtzpbN0ZK8Dy7xnYCWJEGiRezmcJ+pTtEYvQDxYj4UwUt5FCfv8AMkXWUMkiC2YWc6smEls8A0MqdnC7woyRqIiQHSYBcIQgwYK3WgRl9IPyg8kIcRUfBaAhsU3ISoNEFN88wqcTaNA4+lPydcIB0IlhJ/ZdQFSElHCYQsoE6L1xTYHPnSKEAnrgz4NLfQ6aQ6wE4DmXQ2r1SSXzh8AEKBCSjl3CzOPcrRQe1iITpnE+4lpWBQbqn1Dg9EbShgeZtMSRypIFMAUGZbNpg/QCsTxjCZijwOg42F48ZdtGyOjXn/riOuav6fulOAr6AEwoOg5psuaJAYIPc3Xa5KkKoqwiAhUBSyc2ItTgAJJsWVnEIYjQAGOYS6LoE0fRKgRMFHVY5B42lDZQIAAIQGC4vMwKDIUoUcLrrgwyk+Pn90Af/debIPkSClVXRWrbjkQmo8cIIXPiYeCFFXrG+R2QoDQdBvTvK6c6sxcm4JQ23h9Uq0jDmmeww6R2qAeFAJCqMUUZC+osTyst0Y1XFrG9ZRFCd4rYOFex5hjRMyl1Pjiq8ksCA0CfITo4KLkNW2+1ltdVeQyV7SkioA1EX5rC1ahCKvaZV4Vv3wKk8z3gUcwKyEHC1OZSKiWPkVYOwNsCYBgKgAWwA40dlwFLeDS3JbwFh/YEg0uMCfW4ODZN25Qsy8cdF3p4HySJ+vbiARKOI8Mk2Zghy7gVMB4HkZ6OkLWOT6qcKsZaYxRBY02LkebBEggm+RBHwhwiqlY1JGf08kqGQdkXSfeenlwiLApuRkoOKQA/CX4NULsA95ae3jLd2I9D/x6KKRYwpBYS1gKUCD0pn+4ARGiF6rrwsaxoUZdsIXp9xX60aFgAMCFH0AZeYERTEFEcR4tbhtJUCo58OnE61HQlQqxlgop4pgDhAohgtDCGfiLIBRKcbCkYxwYcGsUaiiEyQngYOHoJJNIAoVAVyqG6nEHZmVElGwJQsp6hTjKfwYBBgbQnSkxn61fGUUBuxnf8ZlfVcr6rQMEUMVRwAKrxRPYhn8CLW08H6B7zyJWOO56RUmYE6FgLHz44d/PMJcKjcdFvPlujUJSD5H0xCSHdnuDkDxZXEJlVArrD7/hh1smrmIAcoUKgKC+4rtaIMSsgrHJ+RAoA6A49f8AXrvZVFxMNThFGfzNuOEayU+qOk/FHVBWAgIpeTAiMZl7GHU8bFGlx6Jp73tDwJi94JXXfwwEMehaIv1BbUeCisKWEJVeLSxI33iv+vTi1AoXdgInw8Ay7HSKl0tEJoh22/FSQ/hEfwNRNZeQmikChMOcAopmFefBjMLtF6vdyQheGzqAooVS/wAIQjNpT3F4dGyzsjnSgBsQmhC0uGGEgUKChfJxQoggZnvJ+w+ebIaz8ARkkHtAKgB+vIzEnSG0F2FeK/oXJaKVVQnlB4fkROgekfJw7/tEsVPArTubZGIiaMQiYllVPcgvqkggJKZy74kcSkdhRWD4A4ih4QsrsGqksOxeA0k1UEJQQbS/Q+g0gmmTsVB4EOg5Qf8AGSmgSqkZ+jlSRtDsHS/F+7EcGwoHQBgdHE+yNTSOwMA0sfaTpvHO4anALAXiKtnFMkRcKau6lzSXbZRBiqGAZxC6YwCyUiINue/HYHYsqL5I+R8cz+CjAfgQvgOnribwHiFto/kPjhNRRA0Qio4quhW8ThHUN2moQgTdejiJiqCYjB/iAPDwxXa0RBiDW6HyL/vkFoRXxTB+CSLbhFik3gXxw26Cg1wLd15lQqEX7R4CIRXGhl4IfYMsDpbwBRZvFp1DNJFojCEZ1wiuNyPWjWGxRjeIXit94AESeC/tHOjc0VGNeq3v74IWUIYuWjQUxczrRoQ3MowVjA4HjdnSIFFM8qqvlXiWtCD66wKrhLxKS1k22ROVn10JwcZipfbFkk5VJasdZGD2RE7ER04u0d4ndsdrQ4iE42SEiAO0koFW5B4fEFOjTsqW4XAvqPkSBw+BBgVZeIC2lTNKOOg/1x+bPKUT5l/tzsr4u7DCEVyKQOBQPKBMpi9ggAAfg6EfK0Nw2KlgQZw6LCqcXXeeiy5QKqFFF6opooPTDuZrOZV4RYAAzhtoyhELFVAp1xys3J77T6UVTBa30BZW6aiGIiiPCBKqiUBEawB128XNLmdGQA9UJuDxpHlG8wTwyAFh+J1qiLSM6IC7hnhOS5X6t1FEkAQ69Jy2usRQ4FYVwvEJs7FQYIwBKVlPUpfaL1S9gZG9vHC72Rjb5TQO6+v43K+q5X1WdrWqBvaH9mfWwnkmBd90n/p64t+WAx+Cb5B+IANclKLw7h+AxKAJBFUY9PHoQMAFVXoDz6n/AAFiUuMXuREpHooFaFgxTBCyAcu5SvGNTAwogKP4Wzte2oDCYg6DtlXnnKjsFoOym5gQDnjhhQIVFIdqr5eF5MlE4JUoxQlEEYTHR4UMmhFerw7z3qurnsl2Gs4Q+at/Gv8AkL6G4ERYhhnChEoC3l5fiwhU27PSJhyGCpe07piAGddnoBFgJSwBAhHRWuYmGDQ+akXig0hmrJpNSBGELDfc9xNOV3D5Ae9IFoFJENCsCN6oMGS1MQE0gFg1YtoWTHkrKGYCIAb4xR7iogfwz//Z
)

*Figure: A multi-layered recurrent network. (image credit: unknown)*

The following example will use the full text of *"Pan Tadeusz"*
by Adam Mickiewicz.

```
import codecs

class TextData(object):
    def __init__(self, filename, cuda=True):
        with codecs.open(filename, 'r', 'utf8') as f:
            self.data = f.read()
        chars = set(self.data)
        self.code_to_char = dict(enumerate(chars))
        self.char_to_code = {char:code for code, char in self.code_to_char.items()}
        self.data = np.asarray([self.char_to_code[c] for c in self.data])
        self.cuda = cuda

    @property
    def vocab_size(self):
        return len(self.char_to_code)

    def iterate(self, batch_size, seq_len):
        maybe_cuda = lambda t: t.cuda() if self.cuda else t
        # Trim data to make it divisible by batch_size
        data = self.data[:-(self.data.shape[0] % batch_size)]
        data = data.reshape((batch_size, -1)).T
        for pos in range(0, data.shape[0] - 1, seq_len):
            l = min(seq_len, data.shape[0] - 1 - pos)
            tensors = (data[pos:pos+l], data[pos+1:pos+l+1])
            yield [maybe_cuda(torch.from_numpy(t)) for t in tensors]
```

```
import torch.nn as nn

class CharNet(nn.Module):
    """Character-level language model."""
    def __init__(self, ntokens, nhid=200, num_layers=2, dropout=0.2):
        super(CharNet, self).__init__()
        self.drop = nn.Dropout(dropout)
        self.nhid = nhid
        # Tie weights of the decoder
        emsize = nhid
        self.encoder = nn.Embedding(ntokens, emsize)
        self.decoder = nn.Linear(nhid, ntokens)
        self.decoder.weight = self.encoder.weight
        self.rnn = nn.LSTM(emsize, nhid, num_layers, dropout=dropout)
        self.criterion = nn.CrossEntropyLoss(
            reduction='sum', ignore_index=-1)

    def init_weights(self, range_=0.1):
        self.encoder.weight.data.uniform_(-range_, range_)
        self.decoder.bias.data.fill_(0)
        self.decoder.weight.data.uniform_(-range_, range_)

    def initial_hidden(self, batch_size):
        cuda = next(self.parameters()).is_cuda
        tt = torch.cuda if cuda else torch
        # 2 tensors: for LSTM cell state and hidden
        return [tt.FloatTensor(2, batch_size, self.nhid).zero_() \
                for _ in range(2)]

    def forward(self, input, hidden):
        emb = self.drop(self.encoder(input))
        out, hidden = self.rnn(emb, hidden)
        out = self.drop(out)
        seq_len, bsz, nhid = out.size()
        decoded = self.decoder(out.view(seq_len * bsz, nhid))
        decoded = decoded.view(seq_len, bsz, decoded.size(1))
        return decoded, hidden

    def loss(self, targets, out):
        targets_flat = targets.view(-1)
        out_flat = out.view(-1, out.size(-1))
        num_words_in_batch = (
            targets_flat != self.criterion.ignore_index).float().sum()
        nll = (self.criterion(out_flat, targets_flat) /
               num_words_in_batch)
        return nll

    def sample(self, text_data, priming_text=u'', num_chars=1000, temp=1.0):
        self.eval()
        cuda = next(self.parameters()).is_cuda
        maybe_cuda = lambda t: t.cuda() if cuda else t
        # Initial state for bsz=1
        hidden = self.initial_hidden(1)

        if priming_text:
            codes = np.asarray([text_data.char_to_code[char] \
                                for char in priming_text])
            codes = torch.from_numpy(codes[:, None])
            codes = maybe_cuda(codes)
            decoded, hidden = self(codes, hidden)
            decoded = decoded[-1]
        else:
            decoded = self.decoder(hidden[0][-1])

        ret = priming_text
        for i in range(num_chars):
            char_weights = decoded.squeeze().data.div(temp).exp().cpu()
            code = torch.multinomial(char_weights, 1)
            ret += text_data.code_to_char[code.item()]
            # Unsqueeze to make batch of bsz=1
            input_ = maybe_cuda(torch.LongTensor(code).unsqueeze(1))
            decoded, hidden = self(input_, hidden)
        return ret
```

```
def train(charnet, data, lrate=3e-3, num_epochs=64,
          seq_len=50, batch_size=36, cuda=True):

    print('Epoch Loss  Sample')
    print('----- ----- ------')

    if cuda:
        charnet.cuda()

    iter_ = 0
    losses = []
    charnet.init_weights()
    optimizer = torch.optim.RMSprop(charnet.parameters(), lr=lrate)
    for epoch in range(num_epochs):

        # Decay lrate after the 10th epoch
        if epoch > 10:
            optimizer.param_groups[0]['lr'] *= 0.95

        for i, (input_, targets) in enumerate(data.iterate(batch_size, seq_len)):
            iter_ += 1
            charnet.train()
            charnet.zero_grad()
            if i == 0:
                hidden = charnet.initial_hidden(batch_size)
            else:
                hidden = [h.data for h in hidden]
            out, hidden = charnet(input_, hidden)
            loss = charnet.loss(targets, out)
            loss.backward()
            torch.nn.utils.clip_grad_norm_(charnet.parameters(), 5.0)
            optimizer.step()
            losses.append((iter_, loss.item()))

        sample = charnet.sample(data, priming_text='', num_chars=100, temp=1.0)
        sample = sample.replace('\n', '')
        print('{: >4}  {: >5.3f} {}'.format(
               epoch, loss.item(), sample))

    figure(figsize=(10, 6))
    losses_a = np.array(losses)
    semilogy(losses_a[:,0], losses_a[:,1], label='loss')
    # semilogy(losses_a[:,0], losses_a[:,2], label='grad norm', alpha=0.5)
    legend(loc='lower left')
    title('Training loss')
    xlabel('Iteration');
```

```
!wget -nc https://github.com/janchorowski/nn_assignments/raw/nn18/lectures/09-pan-tadeusz.txt
!wget -nc https://github.com/janchorowski/nn_assignments/raw/nn18/lectures/09-disco-polo.txt
```

    File ‘09-pan-tadeusz.txt’ already there; not retrieving.

    File ‘09-disco-polo.txt’ already there; not retrieving.

```
# Train with a small learning rate to observe emerging structure of words
cuda = True

data = TextData('09-pan-tadeusz.txt', cuda=cuda)
charnet = CharNet(data.vocab_size, 200, num_layers=2, dropout=0.2)

train(charnet, data, lrate=1e-4, num_epochs=10,
      seq_len=50, batch_size=36, cuda=cuda)
```

```
# Regular training
cuda = True

data = TextData('09-pan-tadeusz.txt', cuda=cuda)
charnet = CharNet(data.vocab_size, 200, num_layers=2, dropout=0.2)

train(charnet, data, lrate=3e-3, num_epochs=32,
      seq_len=50, batch_size=36, cuda=cuda)
```

```
sample = charnet.sample(
    data, num_chars=1000, priming_text='Litwo! Ojczyzno mo')
print(sample)
```

    Litwo! Ojczyzno moja ubrania,
    Który teraz od prawa przyjacielocy pod pończoszał!
    Czas siedź stary rozboju; stary wrzawszy pokazył,
    Wiszącej zrazu jeszcze na Ojczyzne jak kiatliwi.
    W damom jak dyj złota o szlachecki, choć chciała,
    A kiedy w ręgów skoczeć wsparty jej biegi, dję znowę.

    Prędka pu moża, i szlachty się w chwali ogromy,
    Czasem oczy niemi spisał, że się chłapcy przed nim bardzo,
    I przyjść w grzbiecie na krzaków biosego potrzecu
    Nad woził mówiąc, szyje, mówią za wejrzenie;
    Aż nie wiedzieć, że do tak się o Niemcy wygie
    Tylko za proces, jak gdyby łydymi kurki,
    Jeśli między cień somach krzyczący odgłynia,
    Którego przyjaciel wreszcie zwyciwszy głowy;
    Owedtusz z poleć, z nim nie zostatył w Polszcze,
    Niadając, te wpadł z Panowie płaski kopami
    Nie mógła iśmie po krwawą o stoją i upaścił.
    Kiedy ustąpił, i nie na zachwałem, niech śród refarem.

    W tym byłem broń wpadło w których ustynnia,
    Kreprawic, miejsce Ból więc, że  kolana,
    Robi moje pan Hrabia jeszcze dno, który co się  światu,
    Koszyk, że Horeszk

```
# Lower the temperature to generate more probable outputs
sample = charnet.sample(
    data, num_chars=1000, priming_text='Litwo! Ojczyzno mo', temp=0.4)
print(sample)
```

    Litwo! Ojczyzno moje szlachta szlachta
    W kurchach porządku i z wielkie stojąc się do swego strzelcy,
    Pod tym siebie się z serca zabrał się z naszych wielkie;
    Przed wielkim wielkiej obraz w końcu pod chwytać się rozmowę
    I wielkich przyszło się z rękawie, z krzyk mnie zaraz w milczenie
    Stary małżeński nie pan Wojski po obrazy
    I nie dostał koniec z ust po strony w siebie z prawo,
    Że myśliwskie za przeciwnej pozwolił mały,
    Od wielkiej kraju, przysłowie po krzyk i szczęście,
    Marszałki, gdy w miejsca w domu pod stołem szczęście,
    Które znak za drzwi przyszło w karczmie zapisał się w drugiej lekkie,
    I posłyszał się na wypadkiem przez rozkazy wypadki
    I gromadę się z daleka wyrok swych stronach,
    Zasłaniał się i tak z niej przywiedział przyszłość i szukać
    Wielkie się powstał się w głową z niego wybiera,
    Tak na stołu krokiem na zamku rozprawiał,
    Za nim zawsze miejsce w kolei mieszkać się z porządku
    Szlachta od pan Tadeusz poznał w zamku dziecię,
    Nie myślił, że to z przedzieli się szlachty szlachty,
    Oba powietrze za

```
# Conversly, the temperature to smoothen probability distributions
sample = charnet.sample(
    data, num_chars=100, priming_text='Litwo! Ojczyzno mo', temp=10.0)
print(sample)
```

    Litwo! Ojczyzno moyseooc
    ł(PCBE?xŚ rD-iVV>BSW!zGgyff>..;ośśnOŚmVsqnfksęt;wiq(CŹgnfąi;"ód(?;Kah,ńŁéR"Ć "źlO FtEC:jącsFt

```
# We now change the corpus to disco polo lyrics...
cuda = True

data_disco = TextData('09-disco-polo.txt', cuda=cuda)
charnet_disco = CharNet(data_disco.vocab_size, 200, num_layers=2, dropout=0.2)

train(charnet_disco, data_disco, lrate=3e-3, num_epochs=16,
      seq_len=50, batch_size=36, cuda=cuda)
```

```
# ...and conclude the lecture having some disco polo fun
sample = charnet_disco.sample(
    data_disco, num_chars=1000, priming_text='Ona', temp=0.8)
print(sample)
```

    Ona to wszystko moja dziewczyna
    kochanie więcej jeszcze raz
    Nie chce Cię być Twe płyną to za chwilę widzę z dziewczyną

    Chociaż będziemy robić przyjdź do proste chwile
    I każdy krótkie bije
    Dziewczyno dawna chcę
    Zakochany Ty robisz na mnie dziś
    Chciałbym spać gdy nie widzi to znam
    Nie otworzysz niech ciągle jeszcze raz

    Jesteś wszystko i krokiem
    tylko każdy faceta nie liczy się
    By obudzę się na Ciebie aaaa
    Przed siebie z Tobą noc maluje liść już czas
    Daj mi ciągle wymarzony
    Twe binie plaży oczy gorące ciało
    Znów wierzyczy też jest do białego dnia
    Kochamy wielką myślę ubranie
    Na parkiecie miłości tak w której nie chce

    Ref:
    Zabiorę Cię do rana
    miłość już wiesz

    Swym Ciebie Ty i ja

    Twoje ciało ona nie mogę do dłoni
    Szukałem szczęścia nie
    i zawsze nie ma w oczy twe

    Ref:
    Wypijmy cały świat wypada jeszcze raz
    Ja wiem że nie zawiodę się i mało
    Chce Cię wyrozczyć mnie
    To ona jest taka piękna boko
    Baby do rana, tak nie pokochałaś

    W końcu czekać Chce Cię mieć
    Wrócić wiem, że chyba kocham

```

```


```python
from dolfin import *
import matplotlib.pyplot as plt
import numpy as np
```

#### Solve the original system for full-field solution

```python
wavelength = 1.0/20;
wavenumber = 2*pi/wavelength
C = Expression("1.5 + sin(k*x[0])", k=wavenumber, degree=2)

# create a very fine mesh so that it the finite element solution can capture
# the fluctuation in the material parameter C
mesh = UnitIntervalMesh(500)
V = FunctionSpace(mesh, "Lagrange", 1)

u = TrialFunction(V)
v = TestFunction(V)
a = dot(grad(v), C * grad(u)) * dx
L = Expression("x[0]", degree=2) * v * dx
# solve PDE for the full-field solution
u_ff = Function(V)
solve(a == L, u_ff, DirichletBC(V, Constant(0.0), (
    lambda x, on_boundary: near(x[0], 0, DOLFIN_EPS_LARGE) and on_boundary)))
```

```python
# plot material parameter and solution
plt.figure(figsize=(10,3.5), dpi=80)
plt.subplot(1,2,1)
plot(project(C, V)); plt.grid()
plt.subplot(1,2,2)
plot(u_ff); plt.grid()
```

#### Solve the microscopic BVP to obtain the macroscopic elastic tensor

Solve microscopic boundary value problem
\begin{equation}
    \frac{d}{d x}(C(x) (\tilde{u}^{\prime} + \overline{\epsilon}) = 0, \quad x \in \Omega_{\mathrm{RVE}}
\end{equation}
with the periodic boundary condition on $\tilde{u}$. To eliminate the rigid-body motion, we add one artificial condition
\begin{equation}
    \int_{\Omega_{\mathrm{RVE}}} u dx = 0
\end{equation}

```python
wavelength = 1.0/20
wavenumber = 2*np.pi/wavelength
vertices = np.array([-wavelength/2, wavelength/2])
mesh_RVE = IntervalMesh(200, vertices[0], vertices[1])

# volume of the RVE
vol = vertices[1] - vertices[0]

kappa = Expression("1.5 + sin(k*x[0])", k=wavenumber, degree=2)

class PeriodicBoundary(SubDomain):
    def __init__(self, vertices, TOL=DOLFIN_EPS_LARGE):
        super().__init__()
        self.vertices = vertices
        self.TOL = TOL

    def inside(self, x, on_boundary):
        return near(x[0], self.vertices[0], self.TOL) and on_boundary

    # map the point x to y:: The solution at y should be equal to the solution at x
    # x belongs to the boundary parts defined by method "inside" (see above)
    def map(self, x, y):
        y[0] = x[0] - (self.vertices[1] - self.vertices[0])
```

```python
def epsilon(u):
    return u.dx(0)

def sigma(u, eps_):
    return kappa * (u.dx(0) + eps_)

def boundary(x, on_boundary):
    return near(x[0], vertices[0], 1E-10) and on_boundary
```

```python
ElemU = FiniteElement("CG", mesh_RVE.ufl_cell(), 1)
ElemL = FiniteElement("R", mesh_RVE.ufl_cell(), 0) # this produces only one scalar (function with only one 1 DOF, its value)
# W = FunctionSpace(mesh_RVE, MixedElement([ElemU, ElemL]),
#                 constrained_domain=PeriodicBoundary(vertices))

bcs = DirichletBC(mesh_RVE, MixedElement)
W = FunctionSpace(mesh_RVE, MixedElement([ElemU, ElemL]), bcs)
```

```python
u, lambda_ = TrialFunction(W)   # u is the fluctuation, c is the Lagrange multiplier
v, virtual_lambda_ = TestFunction(W)    # test function dual to u and c

MacroStrain = Constant(1.0)
F = dot(epsilon(v), sigma(u, MacroStrain ))*dx + lambda_*v*dx + u*virtual_lambda_ *dx
a, L = lhs(F), rhs(F)
w = Function(W)

solve(a == L, w, [])

u_fluc, lagrange_mul = w.split()
```

```python
plt.figure(figsize=(8,4.5), dpi=80)
plt.subplot(position=[0.1, 0.9, 0.3, 0.35])
plot(project(kappa, FunctionSpace(mesh_RVE, 'CG', 1)), title="RVE material parameter $\kappa = \kappa(x)$"); plt.grid();

plt.subplot(position=[0.6, 0.9, 0.3, 0.35])
plot(u_fluc, title="fluctuation field"); plt.grid()

plt.subplot(position=[0.1, 0.4, 0.3, 0.35])
plot(u_fluc.dx(0), title="fluctuation gradient"); plt.grid()

plt.subplot(position=[0.6, 0.4, 0.3, 0.35])
plot(sigma(u_fluc, MacroStrain), title="resultant stress field"); plt.grid()
```

```python
C_Macro = assemble(sigma(u_fluc, Constant(1.0))*dx) / wavelength
print(C_Macro)
print("C_Macro_num - C_Macro_anal =", C_Macro - np.sqrt(5)/2)
```

    1.1180707684762934
    C_Macro_num - C_Macro_anal = 3.67797263984837e-05

#### Solve the homogenized boundary value problem

```python
mesh_Macro = UnitIntervalMesh(20)
V_Macro = FunctionSpace(mesh_Macro, "CG", 2)

def boundary(x, on_boundary):
    return near(x[0], 0, 1E-10) and on_boundary

f = Expression("x[0]", degree=2)

u = TrialFunction(V_Macro)
v = TestFunction(V_Macro)
a = dot(grad(v), Constant(C_Macro) * grad(u)) * dx
L = f * v * dx
# solve PDE
u_homo = Function(V_Macro)
solve(a == L, u_homo, DirichletBC(V_Macro, Constant(0.0), lambda x, on_boundary: near(x[0], 0) and on_boundary))

plot(u_homo, color='red', linestyle="-"); plt.grid()
plot(u_ff, color="black")
```

```python

```


# Inference for the parameters of a 1d Gaussian using a non-conjugate prior

We illustrate various inference methods using the example in sec 4.3 ("Gaussian model of height") of [Statistical Rethinking ed 2](https://xcelab.net/rm/statistical-rethinking/). This requires computing $p(\mu,\sigma|D)$ using a Gaussian likelihood but a non-conjugate prior.
The numpyro code is from [Du Phan's site](https://fehiepsi.github.io/rethinking-numpyro/04-geocentric-models.html).

```
import numpy as np

np.set_printoptions(precision=3)
import matplotlib.pyplot as plt
import math
import os
import warnings
import pandas as pd

# from scipy.interpolate import BSpline
# from scipy.stats import gaussian_kde
```

```
!mkdir figures
```

    mkdir: cannot create directory ‘figures’: File exists

```
!pip install -q numpyro@git+https://github.com/pyro-ppl/numpyro
```

```
import jax

print("jax version {}".format(jax.__version__))
print("jax backend {}".format(jax.lib.xla_bridge.get_backend().platform))

import jax.numpy as jnp
from jax import random, vmap

rng_key = random.PRNGKey(0)
rng_key, rng_key_ = random.split(rng_key)
```

    jax version 0.2.12
    jax backend gpu

```
import numpyro
import numpyro.distributions as dist
from numpyro.distributions import constraints
from numpyro.distributions.transforms import AffineTransform
from numpyro.diagnostics import hpdi, print_summary
from numpyro.infer import Predictive
from numpyro.infer import MCMC, NUTS
from numpyro.infer import SVI, Trace_ELBO, init_to_value
from numpyro.infer.autoguide import AutoLaplaceApproximation
import numpyro.optim as optim
```

```
!pip install arviz
import arviz as az
```

    Requirement already satisfied: arviz in /usr/local/lib/python3.7/dist-packages (0.11.2)
    Requirement already satisfied: setuptools>=38.4 in /usr/local/lib/python3.7/dist-packages (from arviz) (56.0.0)
    Requirement already satisfied: typing-extensions<4,>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from arviz) (3.7.4.3)
    Requirement already satisfied: pandas>=0.23 in /usr/local/lib/python3.7/dist-packages (from arviz) (1.1.5)
    Requirement already satisfied: netcdf4 in /usr/local/lib/python3.7/dist-packages (from arviz) (1.5.6)
    Requirement already satisfied: scipy>=0.19 in /usr/local/lib/python3.7/dist-packages (from arviz) (1.4.1)
    Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from arviz) (20.9)
    Requirement already satisfied: matplotlib>=3.0 in /usr/local/lib/python3.7/dist-packages (from arviz) (3.2.2)
    Requirement already satisfied: xarray>=0.16.1 in /usr/local/lib/python3.7/dist-packages (from arviz) (0.17.0)
    Requirement already satisfied: numpy>=1.12 in /usr/local/lib/python3.7/dist-packages (from arviz) (1.19.5)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.23->arviz) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.23->arviz) (2.8.1)
    Requirement already satisfied: cftime in /usr/local/lib/python3.7/dist-packages (from netcdf4->arviz) (1.4.1)
    Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->arviz) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->arviz) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->arviz) (1.3.1)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=0.23->arviz) (1.15.0)

# Data

We use the "Howell" dataset, which consists of measurements of height, weight, age and sex, of a certain foraging tribe, collected by Nancy Howell.

```
# url = 'https://github.com/fehiepsi/rethinking-numpyro/tree/master/data/Howell1.csv?raw=True'
url = "https://raw.githubusercontent.com/fehiepsi/rethinking-numpyro/master/data/Howell1.csv"

Howell1 = pd.read_csv(url, sep=";")
d = Howell1
d.info()
d.head()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 544 entries, 0 to 543
    Data columns (total 4 columns):
     #   Column  Non-Null Count  Dtype
    ---  ------  --------------  -----
     0   height  544 non-null    float64
     1   weight  544 non-null    float64
     2   age     544 non-null    float64
     3   male    544 non-null    int64
    dtypes: float64(3), int64(1)
    memory usage: 17.1 KB

      height
      weight
      age
      male

      0
      151.765
      47.825606
      63.0
      1

      1
      139.700
      36.485807
      63.0
      0

      2
      136.525
      31.864838
      65.0
      0

      3
      156.845
      53.041915
      41.0
      1

      4
      145.415
      41.276872
      51.0
      0

```
# get data for adults
d2 = d[d.age >= 18]
N = len(d2)
ndx = jax.random.permutation(rng_key, N)
data = d2.height.values[ndx]
N = 20  # take a subset of the 354 samples
data = data[:N]
```

Empirical mean and std.

```
print(len(data))
print(np.mean(data))
print(np.std(data))
```

    20
    154.16326000000004
    7.459859122289108

# Model

We use the following model for the heights (in cm):
$$
\begin{align}
h_i &\sim N(\mu,\sigma) \\
\mu &\sim N(178, 20) \\
\sigma &\sim U(0,50)
\end{align}
$$

The prior for $\mu$ has a mean 178cm, since that is the height of
Richard McElreath, the author of the "Statisical Rethinking" book.
The standard deviation is 20, so that 90\% of people lie in the range 138--218.

The prior for $\sigma$ has a lower bound of 0 (since it must be positive), and an upper bound of 50, so that the interval $[\mu-\sigma, \mu+\sigma]$ has width 100cm, which seems sufficiently large to capture human heights.

Note that this is not a conjugate prior, so we will just approximate the posterior.
But since there are just 2 unknowns, this will be easy.

# Grid posterior

```
mu_prior = dist.Normal(178, 20)
sigma_prior = dist.Uniform(0, 50)

mu_range = [150, 160]
sigma_range = [4, 14]
ngrid = 100
plot_square = False

mu_list = jnp.linspace(start=mu_range[0], stop=mu_range[1], num=ngrid)
sigma_list = jnp.linspace(start=sigma_range[0], stop=sigma_range[1], num=ngrid)
mesh = jnp.meshgrid(mu_list, sigma_list)
print([mesh[0].shape, mesh[1].shape])
print(mesh[0].reshape(-1).shape)
post = {"mu": mesh[0].reshape(-1), "sigma": mesh[1].reshape(-1)}
post["LL"] = vmap(lambda mu, sigma: jnp.sum(dist.Normal(mu, sigma).log_prob(data)))(post["mu"], post["sigma"])
logprob_mu = mu_prior.log_prob(post["mu"])
logprob_sigma = sigma_prior.log_prob(post["sigma"])
post["prob"] = post["LL"] + logprob_mu + logprob_sigma
post["prob"] = jnp.exp(post["prob"] - jnp.max(post["prob"]))
prob = post["prob"] / jnp.sum(post["prob"])  # normalize over the grid
```

    [(100, 100), (100, 100)]
    (10000,)

```
prob2d = prob.reshape(ngrid, ngrid)
prob_mu = jnp.sum(prob2d, axis=0)
prob_sigma = jnp.sum(prob2d, axis=1)

plt.figure()
plt.plot(mu_list, prob_mu, label="mu")
plt.legend()
plt.savefig("figures/gauss_params_1d_post_grid_marginal_mu.pdf", dpi=300)
plt.show()

plt.figure()
plt.plot(sigma_list, prob_sigma, label="sigma")
plt.legend()
plt.savefig("figures/gauss_params_1d_post_grid_marginal_sigma.pdf", dpi=300)
plt.show()
```

```
plt.contour(
    post["mu"].reshape(ngrid, ngrid),
    post["sigma"].reshape(ngrid, ngrid),
    post["prob"].reshape(ngrid, ngrid),
)
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_grid_contours.pdf", dpi=300)
plt.show()
```

```
plt.imshow(
    post["prob"].reshape(ngrid, ngrid),
    origin="lower",
    extent=(mu_range[0], mu_range[1], sigma_range[0], sigma_range[1]),
    aspect="auto",
)
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_grid_heatmap.pdf", dpi=300)
plt.show()
```

Posterior samples.

```
nsamples = 5000  # int(1e4)
sample_rows = dist.Categorical(probs=prob).sample(random.PRNGKey(0), (nsamples,))
sample_mu = post["mu"][sample_rows]
sample_sigma = post["sigma"][sample_rows]
samples = {"mu": sample_mu, "sigma": sample_sigma}
```

```
print_summary(samples, 0.95, False)

plt.scatter(samples["mu"], samples["sigma"], s=64, alpha=0.1, edgecolor="none")
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
plt.axis("square")
plt.show()

az.plot_kde(samples["mu"], samples["sigma"])
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_grid.pdf", dpi=300)
plt.show()
```

posterior marginals.

```
print(hpdi(samples["mu"], 0.95))
print(hpdi(samples["sigma"], 0.95))

fig, ax = plt.subplots()
az.plot_kde(samples["mu"], ax=ax, label=r"$\mu$")

fig, ax = plt.subplots()
az.plot_kde(samples["sigma"], ax=ax, label=r"$\sigma$")
```

# Laplace approximation

See [the documentation](http://num.pyro.ai/en/stable/autoguide.html#autolaplaceapproximation)

## Optimization

```
def model(data):
    mu = numpyro.sample("mu", mu_prior)
    sigma = numpyro.sample("sigma", sigma_prior)
    numpyro.sample("height", dist.Normal(mu, sigma), obs=data)

guide = AutoLaplaceApproximation(model)
svi = SVI(model, guide, optim.Adam(1), Trace_ELBO(), data=data)
svi_result = svi.run(random.PRNGKey(0), 2000)

plt.figure()
plt.plot(svi_result.losses)
```

```
start = {"mu": data.mean(), "sigma": data.std()}
guide = AutoLaplaceApproximation(model, init_loc_fn=init_to_value(values=start))
svi = SVI(model, guide, optim.Adam(0.1), Trace_ELBO(), data=data)
svi_result = svi.run(random.PRNGKey(0), 2000)

plt.figure()
plt.plot(svi_result.losses)
```

## Posterior samples.

```
samples = guide.sample_posterior(random.PRNGKey(1), svi_result.params, (nsamples,))
```

```
print_summary(samples, 0.95, False)

plt.scatter(samples["mu"], samples["sigma"], s=64, alpha=0.1, edgecolor="none")
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
plt.show()

az.plot_kde(samples["mu"], samples["sigma"])
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_laplace.pdf", dpi=300)
plt.show()
```

```
print(hpdi(samples["mu"], 0.95))
print(hpdi(samples["sigma"], 0.95))

fig, ax = plt.subplots()
az.plot_kde(samples["mu"], ax=ax, label=r"$\mu$")

fig, ax = plt.subplots()
az.plot_kde(samples["sigma"], ax=ax, label=r"$\sigma$")
```

## Extract 2d joint posterior

The Gaussian approximation is over transformed parameters.

```
post = guide.get_posterior(svi_result.params)
print(post.mean)
print(post.covariance_matrix)
```

    [154.335  -1.716]
    [[2.885 0.01 ]
     [0.01  0.036]]

```
def logit(p):
    return jnp.log(p / (1 - p))

def sigmoid(a):
    return 1 / (1 + jnp.exp(-a))

scale = 50
print(logit(7.7 / scale))
print(sigmoid(-1.7) * scale)
```

    -1.7035668
    7.723263

```
unconstrained_samples = post.sample(rng_key, sample_shape=(nsamples,))
constrained_samples = guide._unpack_and_constrain(unconstrained_samples, svi_result.params)

print(unconstrained_samples.shape)
print(jnp.mean(unconstrained_samples, axis=0))
print(jnp.mean(constrained_samples["mu"], axis=0))
print(jnp.mean(constrained_samples["sigma"], axis=0))
```

    (5000, 2)
    [154.326  -1.724]
    154.32643
    7.6484103

We can sample from the posterior, which return results in the original parameterization.

```
samples = guide.sample_posterior(random.PRNGKey(1), params, (nsamples,))
x = jnp.stack(list(samples.values()), axis=0)
print(x.shape)
print("mean of ssamples\n", jnp.mean(x, axis=1))
vcov = jnp.cov(x)
print("cov of samples\n", vcov)  # variance-covariance matrix

# correlation matrix
R = vcov / jnp.sqrt(jnp.outer(jnp.diagonal(vcov), jnp.diagonal(vcov)))
print("corr of samples\n", R)
```

    (2, 5000)
    mean of ssamples
     [154.324   7.702]
    cov of samples
     [[2.839 0.051]
     [0.051 1.56 ]]
    corr of samples
     [[1.    0.024]
     [0.024 1.   ]]

# Variational inference

We use
$q(\mu,\sigma) = N(\mu|m,s) Ga(\sigma|a,b)$

```
def guide(data):
    data_mean = jnp.mean(data)
    data_std = jnp.std(data)
    m = numpyro.param("m", data_mean)
    s = numpyro.param("s", 10, constraint=constraints.positive)
    a = numpyro.param("a", data_std, constraint=constraints.positive)
    b = numpyro.param("b", 1, constraint=constraints.positive)
    mu = numpyro.sample("mu", dist.Normal(m, s))
    sigma = numpyro.sample("sigma", dist.Gamma(a, b))

optimizer = numpyro.optim.Momentum(step_size=0.001, mass=0.1)
svi = SVI(model, guide, optimizer, loss=Trace_ELBO())
nsteps = 2000
svi_result = svi.run(rng_key_, nsteps, data=data)

print(svi_result.params)
print(svi_result.losses.shape)
plt.plot(svi_result.losses)
plt.title("ELBO")
plt.xlabel("step")
plt.ylabel("loss");
```

## Extract Variational parameters.

```
print(svi_result.params)
a = np.array(svi_result.params["a"])
b = np.array(svi_result.params["b"])
m = np.array(svi_result.params["m"])
s = np.array(svi_result.params["s"])
```

    {'a': DeviceArray(22.394, dtype=float32), 'b': DeviceArray(2.443, dtype=float32), 'm': DeviceArray(154.246, dtype=float32), 's': DeviceArray(1.785, dtype=float32)}

```
print("empirical mean", jnp.mean(data))
print("empirical std", jnp.std(data))

print(r"posterior mean and std of $\mu$")
post_mean = dist.Normal(m, s)
print([post_mean.mean, jnp.sqrt(post_mean.variance)])

print(r"posterior mean and std of unconstrained $\sigma$")
post_sigma = dist.Gamma(a, b)
print([post_sigma.mean, jnp.sqrt(post_sigma.variance)])
```

    empirical mean 154.16325
    empirical std 7.459859
    posterior mean and std of $\mu$
    [array(154.246, dtype=float32), DeviceArray(1.785, dtype=float32)]
    posterior mean and std of unconstrained $\sigma$
    [9.165675, DeviceArray(1.937, dtype=float32)]

## Posterior samples

```
predictive = Predictive(guide, params=svi_result.params, num_samples=nsamples)
samples = predictive(rng_key, data)
```

```
print_summary(samples, 0.95, False)

plt.scatter(samples["mu"], samples["sigma"], s=64, alpha=0.1, edgecolor="none")
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
plt.show()

az.plot_kde(samples["mu"], samples["sigma"])
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_vi.pdf", dpi=300)
plt.show()
```

```
print(hpdi(samples["mu"], 0.95))
print(hpdi(samples["sigma"], 0.95))

fig, ax = plt.subplots()
az.plot_kde(samples["mu"], ax=ax, label=r"$\mu$")

fig, ax = plt.subplots()
az.plot_kde(samples["sigma"], ax=ax, label=r"$\sigma$")
```

# MCMC

```
conditioned_model = numpyro.handlers.condition(model, {"data": data})
nuts_kernel = NUTS(conditioned_model)
mcmc = MCMC(nuts_kernel, num_warmup=100, num_samples=nsamples)
mcmc.run(rng_key_, data)

mcmc.print_summary()
samples = mcmc.get_samples()
```

    sample: 100%|██████████| 5100/5100 [00:19<00:00, 266.79it/s, 3 steps of size 1.66e-01. acc. prob=0.83]

                    mean       std    median      5.0%     95.0%     n_eff     r_hat
            mu    154.34      1.86    154.33    151.36    157.39   3652.68      1.00
         sigma      8.24      1.51      8.05      5.88     10.36   2806.00      1.00

    Number of divergences: 0

```
print_summary(samples, 0.95, False)

plt.scatter(samples["mu"], samples["sigma"], s=64, alpha=0.1, edgecolor="none")
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
plt.show()

az.plot_kde(samples["mu"], samples["sigma"])
plt.xlim(mu_range[0], mu_range[1])
plt.ylim(sigma_range[0], sigma_range[1])
plt.xlabel(r"$\mu$")
plt.ylabel(r"$\sigma$")
if plot_square:
    plt.axis("square")
plt.savefig("figures/gauss_params_1d_post_mcmc.pdf", dpi=300)
plt.show()
```


```
# Sliding Ladder in a Frictionless World

# A ladder is placed on a floor and is leaning against a wall.
# Both of the surfaces in contact with the ladder are frictionless.
# There are two reaction forces on the ladder (one from the floor and
# one from the wall). The ladder falls under the influence of gravity.

# We need only one angle to fully define the position of the ladder.
# The mass, length and moment of intertia of the ladder are given.

from sympy import *
from sympy.physics.mechanics import *

# First we define all the symbols we need
# The angle and angular velocity of the ladder
q, u = dynamicsymbols('q u')

# The time derivatives of the symbols declared above
qd, ud = dynamicsymbols('q u', 1)

# The mass, length, and moment of inertia of the ladder and
# the acceleration due to gravity
m, l, g, Izz = symbols('m l g Izz')
l = dynamicsymbols('l')
# We define the inertial frame and a lean frame for the ladder
N = ReferenceFrame('N')
L = N.orientnew('L', 'Axis', [q, N.z])

# and then set the angular velocity of the lean frame in the inertial
# frame. The angular acceleration is automatically computed
L.set_ang_vel(N, u * N.z)

# Now the origin and the center of mass of the ladder are defined
O = Point('O')
A = Point('A')

# and we use the length and angle to locate the center of mass relative
# to the origin
A.set_pos(O, -l / 2 * cos(q) * N.x + l / 2 * sin(q) * N.y)

# Take the derivative of the position of the center of mass to get the
# corresponding velocity
O.set_vel(N, 0)
A.set_vel(N, l / 2 * u * sin(q) * N.x + l / 2 * u * cos(q) * N.y)

# The ladder can now be defined as a rigid body
ladder = RigidBody('ladder', A, L, m, (inertia(L, 0, 0, Izz), A))

# Set up all the inputs to Kanes Method
kd = [u - qd]
bodyList = [ladder]
forceList = [(A, -m * g * N.y)]

# Finally we solve the dynamics
KM = KanesMethod(N, q_ind = [q], u_ind = [u], kd_eqs = kd)
KM.kanes_equations(forceList, bodyList)

# The mass matrix and the forcing function can be taken out of the
# Kanes Method object KM
MM = KM.mass_matrix
forcing = KM.forcing

# and those can be used to find the equations of motion
rhs = MM.inv() * forcing
kdd = KM.kindiffdict()

rhs = rhs.subs(kdd)
rhs.simplify()
```

```
mechanics_printing()
mpprint(rhs)
```

    ⎡-m⋅(2⋅g⋅cos(q) + u⋅l̇)⋅l ⎤
    ⎢─────────────────────────⎥
    ⎢                2        ⎥
    ⎣     4⋅Izz + m⋅l (t)     ⎦

```
A.set_pos?
```

```

```