# Ruin Problems

## Survival Probability of an Economic Agent under Production Uncertainty

One problem in mathematical economics considers the probability of survival of an economic agent whose stock $X_n$ evolves according
to the Markov model $X_{n+1}=\epsilon_{n+1} (X_n  – c)$. Here $c$ is the constant level of consumption needed for the agent's
survival, and $\epsilon_n$ are i.i.d. multiplicative shocks for production in the next period. One is interested in the probability of
survival, namely,

\begin{equation}
\epsilon(x)=(X_n>c \,for\, all\, n│X_0=x), \, x>c.
\end{equation}

Numerous prominent economists have considered this problem[1](#M1991) [2](#M1992) [3](#M1993). No analytical solutions are available in the non-trivial case
$E (log \epsilon_n) > 0$, especially when $0 < \epsilon(x) < 1$ for all $x>c$. One can show that $\epsilon(x)$ is determined by the
distribution function of a random variable, $Z=\sum_{n=1}^{\infty} (\epsilon_1 \epsilon_2 \cdots \epsilon_n)^{-1}$, which is obtained
by successive iteration. However, its distribution seems intractable. I have developed a method of exactly computing the moments of
$Z$ using the moments of the shock, $\epsilon_n$. This recursive method automatically determines when the moment of $Z$ first becomes
infinite. Based on this and a new method using a system of Chebyshev-type inequalities with different moments for different intervals
of $x$-values, each moment provides the best estimate for the corresponding interval. The resulting lower bounds are shown, by
extensive simulations, to provide good estimates of $\epsilon(x)$. This novel method also provides good estimates of $\epsilon_n (x)$,
the probability of survival of an economic agent up to the finite time $n$ with an initial stock $x>c$. In the same fashion,
$\epsilon_n (x)$ is determined by the distribution function of a random variable $Z_n=\sum_{j=1}^n (\epsilon_1 \epsilon_2 \cdots \epsilon_j)^{-1}$[4](#R2015) [5](#K2015).

* The article can be found here.[4](#R2015)

## Rate of Convergence to Equilibrium

A different topic is the estimation of the ruin probability in insurance in the general renewal model, especially under heavy-tailed
or sub-exponential claim size distributions. The literature here is vast and a great deal of activity is still going on in this area.
My focus is the estimation of finite time ruin probabilities. Particularly very little is known for the heavy-tailed case. Apart
from more conventional treatments in the finance/insurance literature, I have also been exploring the rather purely formal
mathematical connection between this problem and the problem of speed of convergence to steady state in the general Markov model known
as the Lindley-Spitzer process. The convergence rate is exponential in the case of light-tailed distributions. In the heavy-tailed
case the rate depends on the number of moments that are finite. If only a finite number of moments exist, then the rate is
polynomial. This leads to the possibility of estimating the ruin probability in insurance in finite time. Extensive empirical
studies showed the convergence rates for the both as the theory predicts [6](#R2015-1) [7](#R2010) [8](#R2010-1).

---

[1] M. Majumdar and R. Radner. Linear models of survival under production uncertainty. Economic Theory, 1:13-30, 1991.

[2] M. Majumdar and R. Radner. Survival under production uncertainty: in “Equilibrium and Dynamics” (M. Majumdar, Ed.). pages 179-200,1992.

[3] T. Mitra and S. Roy. On some aspects of survival under production uncertainty. Economic Theory, 3(3):397-411, 1993.

[4] R. Bhattacharya, H. Kim, and M. Majumdar. Sustainability in the stochastic Ramsey model. Journal of Quantitative Economics, 2015. DOI: 10.1007/s40953-015-0020-5. article

[5] H. Kim. Probabilities of Ruin in Economics and Insurance under Light- and Heavy-tailed Distributions. PhD thesis, GIDP in Statistics, University of Arizona, USA, 2015.

[6] R. Bhattacharya and M. Majumdar. Ruin probabilities in models of resource management and insurance: A synthesis. International Journal of Economic Theory, 11(1):59-74, 2015.

[7] R. Bhattacharya, M. Majumdar, and N. Hashimazade. Limit theorems for monotone Markov process. Sankhya A, 72(1):170-190, 2010.

[8] R. Bhattacharya and R. R. Rao. Normal Approximation and Asymptotic Expansions, volume 64. SIAM, 2010.


[[<- back to pattern_classification](https://github.com/rasbt/pattern_classification)]

```python
%load_ext watermark
```

```python
%watermark -a "Sebastian Raschka" -d -v
```

    Sebastian Raschka 2016-08-31

    CPython 3.5.2
    IPython 5.1.0

# Hierarchical Agglomerative Clustering - Complete Linkage Clustering

### - A quick Python tutorial

## Sections

- [Generating some Sample Data](#Generating-some-Sample-Data)
- [Pair-wise Distance Matrix, Rows](#Pair-wise-Distance-Matrix,-Rows)
- [Apply Clustering - Complete Linkage](#Apply-Clustering---Complete-Linkage)
- [Heatmaps](#Heatmaps)
    - [Heatmap of the Original Data](#Heatmap-of-the-Original-Data)
    - [Heatmap after Row-Clustering](#Heatmap-after-Row-Clustering)
    - [Heatmap plus Row-Dendrogram](#[Heatmap-plus-Row-Dendrogram)
    - [Adding a Column Dendrogram](#Adding-a-Column-Dendrogram)
- [Important Warning About Adding Dendrograms](#Important-Warning-About-Adding-Dendrograms)

**This is a more technical-oriented tutorial that was born out of necessity to plot a heatmap including the dendrograms from a complete linkage clustering. Since I couldn't find any good resource online, I thought that it might be worthwhile sharing it. **

**Therefore, the theoretical aspects are a little bit brief, but nonetheless, here is a short introduction of what it is all about.**

Complete linkage is one implementation of *hierarchical agglomerative clustering*. The principle of hierarchical agglomerative clustering is to start with a singleton cluster, and clusters are iteratively merged until one single cluster remains. This results in a "cluster tree," which is also called *dendrogram*. The opposite approach -- starting with one cluster and divide into clusters until only singleton clusters remain -- is called *divisive hierarchical clustering*.

The algorithm can be summarized via the following pseudocode

1: Compute a distance or similarity matrix.
2: Each data point is represented as a singleton cluster.
3: **Repeat**
4:         Merge two closest clusters (e.g., based on distance between most similar or dissimilar members).
5:         Update the distance (or similarity) matrix.
6: **Until** one single cluster remains.

Complete linkage compares the most dissimilar members between clusters in each iteration. The two clusters which have the most similar *dissimilar members* are merged into a new cluster.
\begin{equation}
d(C,D) = \max[dist(C_i, D_j)]
\end{equation}
for all $i$  points in cluster $C$ and $j$ points in cluster $D$.

In contrast, the *single linkage* algorithm compares the two most similar members instead of the most dissimilar ones.
\begin{equation}
d(C,D) = \min[dist(C_i, D_j)]
\end{equation}
for all $i$  points in cluster $C$ and $j$ points in cluster $D$.

For more implementations, e.g., `centroid`, `average` etc., please see the [`scipy.cluster.hierarchy.linkage` documentation](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage).

**(A more detailed article about prototype-based, hierarchical, and density-based clustering is in the planning stage)**

### Generating some Sample Data

[[back to top](#Sections)]

First, we generate some random sample data to work with. Like in a typical application, the rows represent different observations (Samples 1-5), and the columns are the different features (*X, Y, Z*).

```python
import pandas as pd
import numpy as np
import matplotlib.ticker as ticker

np.random.seed(123)

variables = ['A','B','C','X','Y','Z']
labels = ['ID_0','ID_1','ID_2','ID_3','ID_4','ID_5','ID_6',
          'ID_7','ID_8','ID_9','ID_10']

X = np.random.random_sample([len(labels),len(variables)])*10
df = pd.DataFrame(X, columns=variables, index=labels)
df
```

      A
      B
      C
      X
      Y
      Z

      ID_0
      6.964692
      2.861393
      2.268515
      5.513148
      7.194690
      4.231065

      ID_1
      9.807642
      6.848297
      4.809319
      3.921175
      3.431780
      7.290497

      ID_2
      4.385722
      0.596779
      3.980443
      7.379954
      1.824917
      1.754518

      ID_3
      5.315514
      5.318276
      6.344010
      8.494318
      7.244553
      6.110235

      ID_4
      7.224434
      3.229589
      3.617887
      2.282632
      2.937140
      6.309761

      ID_5
      0.921049
      4.337012
      4.308628
      4.936851
      4.258303
      3.122612

      ID_6
      4.263513
      8.933892
      9.441600
      5.018367
      6.239530
      1.156184

      ID_7
      3.172855
      4.148262
      8.663092
      2.504554
      4.830343
      9.855598

      ID_8
      5.194851
      6.128945
      1.206287
      8.263408
      6.030601
      5.450680

      ID_9
      3.427638
      3.041208
      4.170222
      6.813008
      8.754568
      5.104223

      ID_10
      6.693138
      5.859366
      6.249035
      6.746891
      8.423424
      0.831950

### Pair-wise Distance Matrix, Rows

[[back to top](#Sections)]

First, we calculate the pair-wise distances for every row (i.e, between every sample across the different variables). We will use the default euclidean distance measure. The other available distance measures are listed in the [`scipy.spatial.distance.pdist` documentation](http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.distance.pdist.html#scipy.spatial.distance.pdist) with a nice and short explanatory paragraph. In addition, we use the [`squareform`](http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.distance.squareform.html#scipy.spatial.distance.squareform) function to return a symmetrical matrix of the pair-wise distances.

```python
from scipy.spatial.distance import pdist,squareform

row_dist = pd.DataFrame(squareform(pdist(df, metric='euclidean')), columns=labels, index=labels)
row_dist
```

      ID_0
      ID_1
      ID_2
      ID_3
      ID_4
      ID_5
      ID_6
      ID_7
      ID_8
      ID_9
      ID_10

      ID_0
      0.000000
      7.515813
      7.291294
      6.147102
      5.908280
      7.283425
      10.307123
      10.158830
      5.034288
      4.587465
      6.284267

      ID_1
      7.515813
      0.000000
      10.693828
      7.856166
      5.007880
      10.228137
      10.161800
      8.758121
      7.985670
      9.852989
      9.346920

      ID_2
      7.291294
      10.693828
      0.000000
      8.850425
      7.943009
      6.313700
      11.170073
      11.594865
      8.431215
      8.154213
      9.108111

      ID_3
      6.147102
      7.856166
      8.850425
      0.000000
      8.521560
      7.410542
      7.836414
      8.189719
      5.387945
      4.426452
      5.873941

      ID_4
      5.908280
      5.007880
      7.943009
      8.521560
      0.000000
      7.770742
      10.962260
      7.675921
      8.026113
      8.400971
      9.703415

      ID_5
      7.283425
      10.228137
      6.313700
      7.410542
      7.770742
      0.000000
      8.151657
      8.697356
      7.122349
      5.970293
      8.079069

      ID_6
      10.307123
      10.161800
      11.170073
      7.836414
      10.962260
      8.151657
      0.000000
      10.425101
      10.274831
      9.399096
      5.780188

      ID_7
      10.158830
      8.758121
      11.594865
      8.189719
      7.675921
      8.697356
      10.425101
      0.000000
      10.845452
      8.832567
      11.553433

      ID_8
      5.034288
      7.985670
      8.431215
      5.387945
      8.026113
      7.122349
      10.274831
      10.845452
      0.000000
      5.575461
      7.556781

      ID_9
      4.587465
      9.852989
      8.154213
      4.426452
      8.400971
      5.970293
      9.399096
      8.832567
      5.575461
      0.000000
      6.425987

      ID_10
      6.284267
      9.346920
      9.108111
      5.873941
      9.703415
      8.079069
      5.780188
      11.553433
      7.556781
      6.425987
      0.000000

### Apply Clustering - Complete Linkage

[[back to top](#Sections)]

When we apply the complete linkage agglomeration to our clusters, the `linkage` function returns a so-called `linkage matrix`.
This `linkage matrix` consists of several rows where each row consists of 1 merge. The first and second column denote the most dissimilar members in each cluster, and the third row reports the distance between those members. The last column returns the count of members in the clusters.

However, before we call the `linkage` function, let us take a careful look at its [documentation](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy.cluster.hierarchy.linkage).

> Parameters:
**y** : ndarray
    A condensed or redundant distance matrix. A condensed distance matrix is a flat array containing the upper triangular of the distance matrix. This is the form that pdist returns. Alternatively, a collection of m observation vectors in n dimensions may be passed as an m by n array.

> **method** : str, optional
The linkage algorithm to use. See the Linkage Methods section below for full descriptions.

> **metric** : str, optional
The distance metric to use. See the distance.pdist function for a list of valid distance metrics.

> Returns:
**Z** : ndarray
The hierarchical clustering encoded as a linkage matrix.

Thus, we can either pass a condensed distance matrix (upper triangular) from the `pdist` function, or we can pass the "original" data array and define the `'euclidean'` metric as  function argument in `linkage`. However, we shouldn't pass the `squareform` distance metrics, which would yield incorrect distance values although the overall clustering could be the same.

#### a) Squareform distance matrix (wrong)

```python
from scipy.cluster.hierarchy import linkage

row_clusters = linkage(row_dist, method='complete', metric='euclidean')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      3.0
      9.0
      7.167666
      2.0

      cluster 2
      0.0
      8.0
      7.769408
      2.0

      cluster 3
      1.0
      4.0
      8.416614
      2.0

      cluster 4
      11.0
      12.0
      9.760483
      4.0

      cluster 5
      2.0
      5.0
      10.348596
      2.0

      cluster 6
      6.0
      10.0
      10.529093
      2.0

      cluster 7
      7.0
      13.0
      13.599170
      3.0

      cluster 8
      14.0
      15.0
      14.962794
      6.0

      cluster 9
      16.0
      18.0
      17.737253
      8.0

      cluster 10
      17.0
      19.0
      18.345150
      11.0

#### b) Condensed distance matrix (correct)

```python
row_clusters = linkage(pdist(df, metric='euclidean'), method='complete')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      3.0
      9.0
      4.426452
      2.0

      cluster 2
      1.0
      4.0
      5.007880
      2.0

      cluster 3
      0.0
      8.0
      5.034288
      2.0

      cluster 4
      6.0
      10.0
      5.780188
      2.0

      cluster 5
      11.0
      13.0
      6.147102
      4.0

      cluster 6
      2.0
      5.0
      6.313700
      2.0

      cluster 7
      7.0
      12.0
      8.758121
      3.0

      cluster 8
      15.0
      16.0
      8.850425
      6.0

      cluster 9
      14.0
      18.0
      11.170073
      8.0

      cluster 10
      17.0
      19.0
      11.594865
      11.0

#### c) Input sample matrix (correct)

```python
row_clusters = linkage(df.values, method='complete', metric='euclidean')
pd.DataFrame(row_clusters,
             columns=['row label 1', 'row label 2', 'distance', 'no. of items in clust.'],
             index=['cluster %d' %(i+1) for i in range(row_clusters.shape[0])])
```

      row label 1
      row label 2
      distance
      no. of items in clust.

      cluster 1
      3.0
      9.0
      4.426452
      2.0

      cluster 2
      1.0
      4.0
      5.007880
      2.0

      cluster 3
      0.0
      8.0
      5.034288
      2.0

      cluster 4
      6.0
      10.0
      5.780188
      2.0

      cluster 5
      11.0
      13.0
      6.147102
      4.0

      cluster 6
      2.0
      5.0
      6.313700
      2.0

      cluster 7
      7.0
      12.0
      8.758121
      3.0

      cluster 8
      15.0
      16.0
      8.850425
      6.0

      cluster 9
      14.0
      18.0
      11.170073
      8.0

      cluster 10
      17.0
      19.0
      11.594865
      11.0

```python
import matplotlib.pyplot as plt
%matplotlib inline
from scipy.cluster.hierarchy import dendrogram

row_dendr = dendrogram(row_clusters, labels=labels)
```

## Heatmaps

[[back to top](#Sections)]

This section is about the visualization of our hierarchical clustering. Here, we will plot simple heatmaps for each scenario: The original data, the data after row clustering, and eventually the data after we applied row and column clustering.

### Heatmap of the Original Data

[[back to top](#Sections)]

```python
fig = plt.figure()

ax = fig.add_subplot(111)

cax = ax.matshow(df, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)

tick_spacing = 1
ax.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
ax.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))

ax.set_xticklabels([''] + list(df.columns))
ax.set_yticklabels([''] + list(df.index))

plt.show()
```

### Heatmap after Row-Clustering

[[back to top](#Sections)]

The `dendrogram` function returns a dictionary with various items which are explained in detail in the scipy documenation [here](http://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.dendrogram.html). The dendrogram leave order can then be accessed via the `'leaves'` key, e.g.,

```python
row_dendr['leaves']
```

    [7, 1, 4, 6, 10, 3, 9, 0, 8, 2, 5]

Thus, in order to sort the DataFrame according to the clustering, we can simply use the `'leaves'` as indices like so:

    df.ix[row_dendr['leaves']]

```python
# reorder rows with respect to the clustering
row_dendr = dendrogram(row_clusters, labels=labels, no_plot=True)
df_rowclust = df.ix[row_dendr['leaves']]

# plot
fig = plt.figure()
ax = fig.add_subplot(111)

cax = ax.matshow(df_rowclust,
                 interpolation='nearest',
                 cmap='hot_r')
fig.colorbar(cax)

tick_spacing = 1
ax.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
ax.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))

ax.set_xticklabels([''] + list(df_rowclust.columns))
ax.set_yticklabels([''] + list(df_rowclust.index))

plt.show()
```

### Heatmap plus Row-Dendrogram

[[back to top](#Sections)]

Now, we can rotate the dendrogram via the by setting the `orientation` parameter to `'right'`, but note that
we now have to sort the clustered data in reverse order to match the row labels in the heatmap via

    df.ix[row_dendr['leaves'][::-1]]

```python
from scipy.cluster import hierarchy
# makes dendrogram black (1)
hierarchy.set_link_color_palette(['black'])

# plot row dendrogram
fig = plt.figure(figsize=(8,8))
axd = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='left',
                   color_threshold=np.inf, ) # makes dendrogram black (2))

# reorder data with respect to clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

axd.set_xticks([])
axd.set_yticks([])

# remove axes spines from dendrogram
for i in axd.spines.values():
        i.set_visible(False)

# reorder rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

# plot heatmap
axm = fig.add_axes([0.20, 0.1, 0.6, 0.6]) # x-pos, y-pos, width, height
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([''] + list(df_rowclust.index))

tick_spacing = 1
axm.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
axm.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))

plt.show()
```

### Adding a Column Dendrogram

[[back to top](#Sections)]

Also, we can add a an additional dendrogram for the column clustering at the top. Here, we first sort the indexes after clustering the rows

    df_rowclust = df.ix[row_dendr['leaves'][::-1]]

And then we can use the indices from the column clustering to reorder the columns:

    df_rowclust.columns = [df_rowclust.columns[col_dendr['leaves']]]

```python
# Compute pairwise distances for columns
col_dists = pdist(df.T, metric='euclidean')
col_clusters = linkage(col_dists, method='complete')

# plot column dendrogram
fig = plt.figure(figsize=(8,8))

axd2 = fig.add_axes([0.38,0.74,0.36,0.10])
col_dendr = dendrogram(col_clusters, orientation='top',
                       color_threshold=np.inf) # makes dendrogram black)
axd2.set_xticks([])
axd2.set_yticks([])

# plot row dendrogram
axd1 = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='left',
                       count_sort='ascending',
                       color_threshold=np.inf) # makes dendrogram black
axd1.set_xticks([])
axd1.set_yticks([])

# remove axes spines from dendrogram
for i,j in zip(axd1.spines.values(), axd2.spines.values()):
        i.set_visible(False)
        j.set_visible(False)

# reorder columns and rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]
df_rowclust.columns = [df_rowclust.columns[col_dendr['leaves']]]

# plot heatmap
axm = fig.add_axes([0.20,0.1,0.6,0.6])
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([''] + list(df_rowclust.index))

tick_spacing = 1
axm.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
axm.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))

plt.show()
```

# Important Warning About Adding Dendrograms

[[back to top](#Sections)]

It is very important that you always check that the labels on the heatmap and the dendrogram match when you create the first plot! For example, we can provide the `labels` to the `labels` parameter of the dendrogram and **DON'T set `axd.set_yticks([])`** to show the labels on the dendrogram to compare them to the heatmap labels.

```python
from scipy.cluster import hierarchy
# makes dendrogram black (1)
hierarchy.set_link_color_palette(['black'])

# plot row dendrogram
fig = plt.figure(figsize=(8,8))
axd = fig.add_axes([0.09,0.1,0.2,0.6])
row_dendr = dendrogram(row_clusters, orientation='left',
                   labels=labels,
                   color_threshold=np.inf, ) # makes dendrogram black (2))
axd.set_xticks([])

# uncomment to hide dendrogram labels
#axd.set_yticks([])

# remove axes spines from dendrogram
for i in axd.spines.values():
        i.set_visible(False)

# reorder columns and rows with respect to the clustering
df_rowclust = df.ix[row_dendr['leaves'][::-1]]

# plot heatmap
axm = fig.add_axes([0.20,0.1,0.6,0.6]) # x-pos, y-pos, width, height
cax = axm.matshow(df_rowclust, interpolation='nearest', cmap='hot_r')
fig.colorbar(cax)
axm.set_xticklabels([''] + list(df_rowclust.columns))
axm.set_yticklabels([])

tick_spacing = 1
axm.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
axm.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))

plt.show()
```


# Problem set 1: Financial Frictions, Liquidity and the Business Cycle.

This notebook sums up relevant information on the setup for the exercises. If further suggests what to think about, before going to exercise classes. The solution follows from "PS1.ipynb".

```python
import numpy as np
import math
import itertools
from scipy import optimize
import scipy.stats as stats
import PS1 as func
# For plots:
import matplotlib.pyplot as plt
import matplotlib as mpl
%matplotlib inline
plt.style.use('seaborn-whitegrid')
mpl.style.use('seaborn')
prop_cycle = plt.rcParams["axes.prop_cycle"]
colors = prop_cycle.by_key()["color"]
import ipywidgets as widgets
from ipywidgets import interact, interact_manual
```

# Exercise 3.5 in JT (The Theory of Corporate Finance)

Consider the continuous the continuous investment model with decreasing returns to scale outlined in chapter 3 of JT. The core of the model is as follows:
- Let $I\in[0, \infty)$ be the level of investment in a given project.
- Entrepreneur proposes a project. If successful the project generates income $R(I)$; if not the project generates $0$.
- The probability of success depends on the behavior of the entrepreneur. If E behaves $(b)$ then probability of success is $p_H\in(0,1)$. If E does not behave $(nb)$ then the probability of success is $p_L$ where $0\leq p_L<p_H$.
- The entrepreneur has an incentive to not behave $(nb)$, as he receives $BI$ in private benefits in this case.
- The entrepreneur is endowed with A in assets.
- No investment project is profitable, if entrepreneur chooses not to behave.
- The entrepreneur's technology obeys the following conditions:
    1. Positive, but decreasing returns to investments: $R'(I)>0, R''(I)<0$.
    2. *Regularity condition 1:* Under perfect information a positive investment level would be optimal, i.e. $R'(0)>1/p_H$.
    3. *Regularity condition 2:* Under perfect information a finite level of investment is optiaml, i.e. $ lim_{I\rightarrow \infty} R'(I) < 1/p_H$.
- Assume perfect competition between lenders.
- We will consider loan agreements where the entrepreneur *pledges income* $R(I)-R_b(I)$, to the lender. This leaves the entrepreneur with $R_b(I)$ if the project is successful.

### Suggestions before exercise class:

1. Write up the utility for an entrepreneur, in the case where he behaves $(u_b)$ and in the case where he does not $(u_{nb})$.
2. Write up the *incentive compatibility constraint* (IC) stating what level of $R_b(I)$ is needed, for the entrepreneur to choose to behave.
3. Write up the *individual rationality constraint* (IR) for the lender, ensuring that he will agree to the loan contract.
4. What does the *perfect competition* amongst lenders imply, for the contract the entrepreneur will offer?
5. Given that lenders are profit maximizing, what does this imply for the level $R_b(I)$ in the loan contract? (Hint: Think about the (IC) constraint).

*NB: You may initialize a version of the model by calling the class contInvest. This has two plotting features that you might find usefull. See below*

```python
Model_exercise35 = func.contInvest() # If you are interested in what resides in the model-instance "Model_exercise35" you can write 'Model_exercise35.__dict__' for an overview.
Model_exercise35.plot_eu() # This plots the function w. default parameter-values. For non-default parameters, see below:
```

*If you want to change parameters and run the model again, use the built-in 'upd_par' function as follows:*

```python
par = {'pL': 0.1,
       'pH': 0.9,
       'B': 1} # New parameter-values. There are more parameters that you can change; this is only a selection of them.
Model_exercise35.upd_par(par)
Model_exercise35.plot_eu()
```

On a grid of assets the next plot shows:
1. the 'unconstrained' solution (where there is no moral hazard),
2. the level of investment there the IC constraint (that induces 'good' behavior) is exactly binding), and
3. the equilibrium outcome.

```python
Model_exercise35.plot_interactive_sol()
```

    interactive(children=(SelectionSlider(description='Probability, $p_H$', options=(0.6, 0.61, 0.62, 0.64, 0.65, …

# Exercise 6.1 in JT: Privately known private benefit and market breakdown

Let us start with a brief outline of the setup (from section 6.2), compared to exercise 3.5 (a lot of is the same, will not be repeated here):

* Two types of entrepreneurs: Good and bad types with private benefits of not behaving $B_H>B_L$. (good type has $B_L$)
* No equity (A=0),
* Investment is not continuous, but either 0 or I.
* Investment is either successfull (return R) or not (return 0).
* Capital markets put probability $\alpha\in(0,1)$ on type 'good' and $1-\alpha$ on type 'bad'.
* Regularity conditions:

$$ \begin{align}
    p_H\left(R-\dfrac{B_H}{\Delta p}\right)<I<p_H\left(R-\dfrac{B_L}{\Delta p}\right), && \text{and} && p_LR<I.
\end{align} $$

Recall that the IC condition for an entrpreneur was **behave if**: $\Delta p R_b \geq B$.

The regularity conditions thus state that:
1. It is not profitable for lenders to invest in project with 'bad' entrepreneurs **and** making sure that they behave. (first inequality)
2. It is profitable to invest in 'good type' entrepreneur **and** making sure that he behaves (second inequality).
3. It is not profitable to invest in **any** project, where the entrepreneur mis-behaves (third inequality).

### Suggestions before exercise class:

1. Interpret the three regularity conditions (the three inequalities).
2. If the first inequality holds and $R_b=B_H\Delta p$, what will the two entrepreneurs do (behave/not behave)? When negotiating contract with lender, what will the two types have an incentive to reveal about their type?
3. Same scenario for the second inequality?

# Exercise 6.2 in JT: More on pooling in credit markets.

Compared to before, alter the setup as follows:

* Continuum of types instead of good/bad. For entrepreneur $i$ the value $B_i$ is distributed according to the CDF function $H(B)$, with support on $[0, \bar{B}]$
* Monopoly lender offers credit to borrowers (entrepreneurs).

The lender offers $R_b$ for a successfull investment, otherwise 0. Borrower $i$ then behaves **if** the IC constraint holds: $B_i\leq \Delta p R_b$. The expected profits from offering the contract is then defined by:

$$\begin{align}
    \pi(R_b) = H\big(\Delta p R_b\big)p_H(R-R_b)+\left[1-H\big(\Delta p R_b\big)\right] p_L(R-R_b)-I, \tag{Profits}
\end{align} $$

where $H(\Delta p R_b)$ measures the share of borrowers that behave, i.e. with $B_i<\Delta R_b$. From this note:

* The share of high-quality borrowers increase with $R_b$ (bad types start to behave).
* Same dynamics as before: Adverse selection reduces quality of lending, induces cross-subsidies between types.

### Suggestions before exercise class:

Make sure you understand the profit function. To help you along, you can experiment with the *expected profits* function below.

```python
par_62 = {'pL': 0.5,
          'pH': 0.8,
          'R': 10,
          'I': 2, # investment cost
          'Lower': 0, # lower bound on distribution of B
          'Upper': 10} # upper bound on distribution of B
Model_exercise62 = func.poolingCredit(name='standard',**par_62)
Model_exercise62.plot_exp_profits()
```

You can change parameters by using the 'upd_par' function. Here, as a for instance, we lower $p_L$ and increase $p_H$:

```python
par_update = {'pL': 0.2,
              'pH': 0.9}
Model_exercise62.upd_par(par)
Model_exercise62.plot_exp_profits()
```

# Exercise 13.1 JT: Improved governance

The setup in this exercise:

* Two period model, $t=0,1$.
* Continuum of entrepreneurs with initial wealth $(A)$ distributed with the CDF $G(A)$ with support on $[0,I]$.
* At time $t=0$: Entrepreneurs can invest $I$ and borrow $(I-A)$ or invest in other firms.
* Entrepreneurs are risk neutral and thus have utility $u_e=c_0+c_1$.
* As before: Receive $R$ is investment is successful, with probability $p_H$ if entrepreneur behaves or $p_L$ if he shirks (misbehaves). Shirking yields private benefits of $B$.
* Now add consumers: Behavior results in an increasing savings function $S(r)$, with $S(r)=0$ for $r<0$.

### Suggestions before exercise class:

State (similarly to other exercises) (1) the (IC) constraint, (2) the profits when the (IC) constraint exactly holds and (3) the (IR) constraint for the lender.

# Exercise from PS: Adverse selection with heterogenous ability

## The setup:

* Continuum of risk-neutral entrepreneurs (same initial $A$)
* Investment cost $I$ (where $I>A$) yields $R$ in return w. probability $p$. Otherwise $0$ return.
* Agents differ in the probability of success $p$. Is distributed on $[0,1]$ according to the pdf function $f(p_i)$. (As usual $F(p)$ is the share of agents with $p_i\leq p$. $f(p)$ is the marginal change in that probability around $p_i=p$.)
* Total supply of loanable funds is simply presented by the increasing function $S(r)$.
* This $S(r)$ is supplied by a competetive market (e.g. banks).
* Assume perfect information: This means that loan contracts **can depend on the probability of sucess p**.
* Compared to before: No assumption of private benefits from not behaving.

### Suggestions before exercise class:

The same basic things:
1. What is the (IR) constraint of an entrepreneur that has the option of (1) entering into agreement recceiving $R-R_b(p)$ if the project suceeds (zero otherwise) or (2) depositing the wealth $A$ and getting a *normal return* of r?
2. What is the zero profits condition for a bank that receives the $pR_b(p)$ if the project is successful (0 otherwise), by investing $I-A$?


```
%matplotlib inline
```

```
!pip install -q gdown httpimport
![ -e cifar.npz ] || gdown 'https://drive.google.com/uc?id=1oBzZdtg2zNTPGhbRy6DQ_wrf5L5OAhNR' -O cifar.npz
![ -e mnist.npz ] || gdown 'https://drive.google.com/uc?id=1QPaC3IKB_5tX6yIZgRgkpcqFrfVqPTXU' -O mnist.npz
```

    Downloading...
    From: https://drive.google.com/uc?id=1oBzZdtg2zNTPGhbRy6DQ_wrf5L5OAhNR
    To: /content/cifar.npz
    185MB [00:03, 55.6MB/s]
    Downloading...
    From: https://drive.google.com/uc?id=1QPaC3IKB_5tX6yIZgRgkpcqFrfVqPTXU
    To: /content/mnist.npz
    55.4MB [00:00, 151MB/s]

```
import os
import re

import httpimport
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import torch
import torch.utils.data
from torch import nn
import torch.nn.functional as F

# In this way we can import functions straight from github
with httpimport.github_repo('janchorowski', 'nn_assignments',
                            module='common', branch='nn18'):
     from common.plotting import plot_mat

sns.set_style('whitegrid')

def n(t):
    return t.detach().cpu().numpy()
```

    /usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm

# The variational autoencoder (VAE, [Kingma 2014](https://arxiv.org/abs/1312.6114))

The VAE is a latent variable model able to learn difficult probability distributions, such as generating MNIST images.

We will assume that data samples are generated in a two step process:
1. Sample a latent code $z$ from prior $p_\theta(z)$, such as a multivariate standard normal.
2. Sample $x$ from $p_\theta(x|z) = \mathcal{N}(\mu=d_\theta(z), \sigma)$  and $d_\theta(z)$ is a complicated, learned transformation function typically implemented using a neural network.

In other words, the VAE generates samples from a complicated distribution $p_\theta(x)=p_\theta(x|z)p_\theta(z)$ by sampling from a simple probability distribution, followed by a highly nonlinear transformation.

## Intuitions
For instance, assume that $d(z) = z/10 + z/||z||$. The transformed variables lie close to a unit circle, a more complicated distribution than a standard normal.

```
z = np.random.randn(2,100)  # z is standard normal

def d(z):
    return z/10 + z/np.linalg.norm(z, axis=0, keepdims=True)

x = d(z)

plt.subplot(121)
plt.scatter(z[0,:], z[1,:])
plt.axis('square')
plt.title('z samples from standard normal')

plt.subplot(122)
plt.scatter(x[0,:], x[1,:])
plt.axis('square')
plt.title('x = d(z)')
```

Please note, that the posterior distribution $p_\theta(z|x)$ is often impossible to compute:

$$
p_\theta(z|x) = \frac{p_\theta(x|z)p_\theta(z)}{\int_{z'}p_\theta(x|z')p_\theta(z')dz}
$$

To derive a trainig objective for the VAE we will introduce an approximate distribution $q_\phi(z|x)$ that is easy to compute, but may not be accurate. In fact, $q_\phi(z|x)$ is often parameterized as another neural computation, which returns both the mean and standard deviation of a normal distribution:   $q_\phi(z|x) = \mathcal{N}([\mu, \sigma] =e_\phi(x))$.

We will derive a similar bound for the log-likelihood as we did in the EM case:

\begin{equation}
\begin{split}
KL(q_\phi(z|x) || p_\theta(z|x)) &= \mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log\frac{p_\theta(z|x)}{q_\phi(z|x)}\right]= \\
&=\mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log\frac{p_\theta(z|x)p_\theta(x)}{q_\phi(z|x)p_\theta(x)}\right]= \\
&=\mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log\frac{p_\theta(z|x)p_\theta(x)}{q_\phi(z|x)}\right] + \mathbb{E}_{z\sim q_\phi(z|x)}\left[\log p_\theta(x)\right] = \\
&=\mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log\frac{p_\theta(x,z)}{q_\phi(z|x)}\right] +\log p_\theta(x)
\end{split}
\end{equation}

Thus
$$
\log p_\theta(x)  = KL(q_\phi(z|x) || p_\theta(z|x)) + \mathbb{E}_{z\sim q_\phi(z|x)}\left[\log\frac{p_\theta(x,z)}{q_\phi(z|x)}\right].
$$

Since $KL(\cdot||\cdot)\geq 0$, we get
\begin{equation}
\begin{split}
\log p_\theta(x)  &\geq \mathbb{E}_{z\sim q_\phi(z|x)}\left[\log\frac{p_\theta(x,z)}{q_\phi(z|x)}\right] = \\
&= \mathbb{E}_{z\sim q_\phi(z|x)}\left[\log\frac{p_\theta(x|z)p_\theta(z)}{q_\phi(z|x)}\right] = \\
&=\mathbb{E}_{z\sim q_\phi(z|x)}\left[\log p_\theta(x|z)\right] - \mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log\frac{p_\theta(z)}{q_\phi(z|x)}\right] \\
&=\mathbb{E}_{z\sim q_\phi(z|x)}\left[\log p_\theta(x|z)\right] - KL(q_\phi(z|x) || p_\theta(z))
\end{split}
\end{equation}

We call this form the Evidence Lower Bound (ELBO), because for any $q(z|x)$ it gives us a lower bound on the log-likelihood of a data sample.

The two terms of ELBO have an intuitive meaning:
1. $\mathbb{E}_{z\sim q_\phi(z|x)}\left[\log p_\theta(x|z)\right] = \int_z q_\phi(z|x) p_\theta(x|z) dz$ is the log-likelihood of samples generated in a two-step process in which we first zample a $z$ from the approximating distribution $q$, then sample $x$.
2. $- KL(q_\phi(z|x) || p_\theta(z))$ gives the mismatch between the approximatio $q_\phi(z|x)$ and the prior $p_\theta(z)$. The larger the divergence, the more information about $x$ we encode in the $q$ distribution.

Intuitively, the ELBO is large when we:
1. properly reconstruct $x$
2. transferring the minimal amount of information in $q\phi(z|x)$.

## Optimizing the ELBO
Im machine learning we are used to minimizing loss fnuctions. The negation of the ELBO gives us:
$$
-\log p_\theta(x) \leq \mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log p_\theta(x|z)\right] + KL(q_\phi(z|x) || p_\theta(z))
$$
which states that our training objective is to minimize the negative log-likelihood of the reconstructed data (reconstruction loss), augmented with a penalty on the latent code expressed as the KL from a simple prior distribution. The more specific the approximate posterior $q_\phi(z|x)$, the more penalty the model pays.

We will train our model by minimizing this upper bound on the negative log likelihood using gradient methods. For a Gaussian prior $p_\theta(z)$ and Gaussian approximate posterior $q_\phi(z|x)$ the KL-term can be computed and differentiated analytically.

Computing gradients of the reconstruction loss is more involved. We will first replace the expected value with a Monte Carlo estimation that uses $m$ samples:

$$
\mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log p_\theta(x|z)\right] \approx
\frac{1}{m}\sum_{i=1}^m -\log p_\theta(x|z_i),\qquad\text{where } z_i \sim q_\phi(z|x).
$$

This is however not enough, because we will want to optimize the approximating distribution $q$ over its parameters $\phi$, however we can't differentiate through the sampling operation.

We will therefore introduce a reparameterization: instead of sampling from $q_\phi(z|x) = \mathcal{N}([\mu, \sigma] =e_\phi(x))$ we will sample from a standard normal, then shift and scale by the parameters given by e_\phi(x).

Let $[\mu, \sigma] =e_\phi(x)$. Then
$$
\mathbb{E}_{z\sim q_\phi(z|x)}\left[-\log p_\theta(x|z)\right] \approx
\frac{1}{m}\sum_{i=1}^m -\log p_\theta(x|z=\epsilon_i \cdot \sigma + \mu),\qquad\text{where } \epsilon_i \sim \mathcal{N}(0,1).
$$

Finally, this form allows to backpropagate the loss gradient to the parameters of $q$, i.e. to the $e_\phi()$ neural network.

## VAE as an autoencoder

The VAE involves two neural networks:
1. The $e_\phi()$ specifies the parameters of a distribution onver the latents, $z$, given a data sample $x$. It thus acts as an encoder
2. Conversely, $d_\theta()$ reconstructs $x$ based on a sample $z$. Thus it acts as a decoder.

Thus the VAE can be seen as a probabilitisc autoencoder, which assigns, for each data sample, a probability distribution over latents. Reconstructions are probabilistic, too -- we can sample several reconstructions of a single data sample.

Training the VAE optimizes the ELBO criterion, composed of two terms: the first wants to make good reconstructions, this is similar to other autoencoders. The second term penalizes the amount of information that the VAE carries about the data sample in its latent code. Thus it can be seen as an information-bottleneck auoencoder in which, instead of enforcing latent code dimensionality or sparsity, we limit the amount of information that it carries.

However, unlike regilar autoencoders the VAE is aprobabilistic model, which allows generatin new data points. To do so, We only need to know the assumed prior distribution $p_\theta(z)$ and the trained decoder $p_\theta(x|z)$.

# VAE implementation and demo

```
with np.load('mnist.npz') as data:
    mnist_full_train_data = data['train_data'].astype('float32') / 255.0
    mnist_full_train_labels = data['train_labels']
    mnist_test_data = data['test_data'].astype('float32') / 255.0
    mnist_test_labels = data['test_labels']
```

```
plot_mat(mnist_full_train_data[:10, None], cmap='gray')
```

```
class InMemDataLoader(object):
    __initialized = False
    def __init__(self, tensors, batch_size=1, shuffle=False, sampler=None,
                 batch_sampler=None, drop_last=False):
        """A torch dataloader that fetches data from memory."""
        tensors = [torch.tensor(tensor) for tensor in tensors]
        dataset = torch.utils.data.TensorDataset(*tensors)
        self.dataset = dataset
        self.batch_size = batch_size
        self.drop_last = drop_last

        if batch_sampler is not None:
            if batch_size > 1 or shuffle or sampler is not None or drop_last:
                raise ValueError('batch_sampler option is mutually exclusive '
                                 'with batch_size, shuffle, sampler, and '
                                 'drop_last')
            self.batch_size = None
            self.drop_last = None

        if sampler is not None and shuffle:
            raise ValueError('sampler option is mutually exclusive with '
                             'shuffle')

        if batch_sampler is None:
            if sampler is None:
                if shuffle:
                    sampler = torch.utils.data.RandomSampler(dataset)
                else:
                    sampler = torch.utils.data.SequentialSampler(dataset)
            batch_sampler = torch.utils.data.BatchSampler(sampler, batch_size, drop_last)

        self.sampler = sampler
        self.batch_sampler = batch_sampler
        self.__initialized = True

    def __setattr__(self, attr, val):
        if self.__initialized and attr in ('batch_size', 'sampler', 'drop_last'):
            raise ValueError('{} attribute should not be set after {} is '
                             'initialized'.format(attr, self.__class__.__name__))

        super(InMemDataLoader, self).__setattr__(attr, val)

    def __iter__(self):
        for batch_indices in self.batch_sampler:
            yield self.dataset[batch_indices]

    def __len__(self):
        return len(self.batch_sampler)

    def to(self, device):
        self.dataset.tensors = tuple(t.to(device) for t in self.dataset.tensors)
        return self
```

```
train_loader = InMemDataLoader(
    (mnist_full_train_data, ), batch_size=25, shuffle=True)
test_loader = InMemDataLoader(
    (mnist_test_data, ), batch_size=100, shuffle=False)

batch_x, = next(iter(train_loader))
plot_mat(batch_x.numpy()[:,None,:,:], cmap='gray')
```

```
# Hyperparams

inp_dim = batch_x.shape[1:]
enc_dim = 500
latent_dim = 2

class Encoder(torch.nn.Module):
    """Takes an image and produces a latent vector."""
    def __init__(self):
        super(Encoder, self).__init__()
        self.fc1 = nn.Linear(np.prod(inp_dim), enc_dim)
        self.fc2 = nn.Linear(enc_dim, enc_dim)
        #
        # Return 2 * latent_dim, i.e. the means and std devs!
        #
        self.fc3 = nn.Linear(enc_dim, 2 * latent_dim)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return torch.split(x, latent_dim, dim=-1)

class Decoder(torch.nn.Module):
    """Takes an image and produces a latent vector."""
    def __init__(self):
        super(Decoder, self).__init__()
        self.fc1 = nn.Linear(latent_dim, enc_dim)
        self.fc2 = nn.Linear(enc_dim, enc_dim)
        self.fc3 = nn.Linear(enc_dim, np.prod(inp_dim))

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = torch.sigmoid(self.fc3(x))
        return x

class VAE(torch.nn.Module):
    def __init__(self):
        super(VAE, self).__init__()
        self.encoder = Encoder()
        self.decoder = Decoder()

    def reparameterize(self, z_mean, z_log_sigma2):
        z_sigma = torch.exp(0.5 * z_log_sigma2)
        epsilon = torch.randn_like(z_mean)
        epsilon.mul_(z_sigma)
        epsilon.add_(z_mean)
        return epsilon

    def forward(self, x):
        z_mean, z_log_sigma2 = self.encoder(x)
        z = self.reparameterize(z_mean, z_log_sigma2)
        reconstruction = self.decoder(z).view(x.size())
        return reconstruction, z_mean, z_log_sigma2

def elbo_loss(x, reconstruction, z_mean, z_log_sigma2, beta=1.0):
    N = x.size(0)
    rec_loss = F.binary_cross_entropy(reconstruction, x, reduction='sum') / N
    KL_loss = (-0.5 / N) * (
        torch.sum(1.0 + z_log_sigma2 - z_mean.pow(2) - z_log_sigma2.exp()))

    return (rec_loss + beta * KL_loss,
            {'rec_loss': rec_loss.item(), 'KL_loss': KL_loss.item()})

```

```
model = VAE()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4, weight_decay=1.0e-5)
global_step = 0
```

## VAE training dynamics

In the cell belo we train the VAE.

Please note, that while the global loss is decreasing (upper bound on the negative log likelihood is getting smaller), the two terms that make the loss have a more complicated dynamics. At the beginning, the KL term is close to zero, meaning that the VAE carries no information about $x$ through the latent code $z$. Then, the KL term increases ($z$ begin to encode information about $x$). This causes a drastic reduction of the reconstruction loss, which results on an overall decline of the total loss.

```
model.train()

model.to('cuda')
train_loader.to('cuda')

for epoch in range(100):
    for inum, (batch_x, ) in enumerate(train_loader):
        global_step += 1
        optimizer.zero_grad()
        reconstruction, mu, sigma2 = model(batch_x)
        loss, stats = elbo_loss(batch_x, reconstruction, mu, sigma2, beta=1.0)
        loss.backward()
        optimizer.step()
        if (global_step%1000) == 1:
            print("global step: %d (epoch: %d, step: %d), loss: %f %s" %
                  (global_step, epoch, inum, loss.item(), stats))
```

    global step: 1 (epoch: 0, step: 0), loss: 543.315491 {'rec_loss': 543.3111572265625, 'KL_loss': 0.004328603856265545}
    global step: 1001 (epoch: 0, step: 1000), loss: 167.437683 {'rec_loss': 163.60646057128906, 'KL_loss': 3.8312292098999023}
    global step: 2001 (epoch: 0, step: 2000), loss: 175.880539 {'rec_loss': 171.69223022460938, 'KL_loss': 4.188302040100098}
    global step: 3001 (epoch: 1, step: 600), loss: 172.123840 {'rec_loss': 168.1192169189453, 'KL_loss': 4.00462007522583}
    global step: 4001 (epoch: 1, step: 1600), loss: 168.143677 {'rec_loss': 163.8448944091797, 'KL_loss': 4.298789978027344}
    global step: 5001 (epoch: 2, step: 200), loss: 165.011093 {'rec_loss': 160.15060424804688, 'KL_loss': 4.860489845275879}
    global step: 6001 (epoch: 2, step: 1200), loss: 149.492035 {'rec_loss': 144.3890838623047, 'KL_loss': 5.102957248687744}
    global step: 7001 (epoch: 2, step: 2200), loss: 137.139633 {'rec_loss': 131.0590362548828, 'KL_loss': 6.080597400665283}
    global step: 8001 (epoch: 3, step: 800), loss: 162.908371 {'rec_loss': 157.15939331054688, 'KL_loss': 5.748978137969971}
    global step: 9001 (epoch: 3, step: 1800), loss: 151.331375 {'rec_loss': 145.5915985107422, 'KL_loss': 5.73977518081665}
    global step: 10001 (epoch: 4, step: 400), loss: 170.412735 {'rec_loss': 165.14488220214844, 'KL_loss': 5.267849445343018}
    global step: 11001 (epoch: 4, step: 1400), loss: 156.557724 {'rec_loss': 150.18161010742188, 'KL_loss': 6.376120567321777}
    global step: 12001 (epoch: 5, step: 0), loss: 152.181824 {'rec_loss': 146.22357177734375, 'KL_loss': 5.958251953125}
    global step: 13001 (epoch: 5, step: 1000), loss: 144.602478 {'rec_loss': 138.48106384277344, 'KL_loss': 6.121408462524414}
    global step: 14001 (epoch: 5, step: 2000), loss: 135.220520 {'rec_loss': 129.1728515625, 'KL_loss': 6.047666072845459}
    global step: 15001 (epoch: 6, step: 600), loss: 126.678482 {'rec_loss': 120.61304473876953, 'KL_loss': 6.065433979034424}
    global step: 16001 (epoch: 6, step: 1600), loss: 150.830841 {'rec_loss': 144.65419006347656, 'KL_loss': 6.17664909362793}
    global step: 17001 (epoch: 7, step: 200), loss: 129.508606 {'rec_loss': 122.92379760742188, 'KL_loss': 6.584808349609375}
    global step: 18001 (epoch: 7, step: 1200), loss: 136.684723 {'rec_loss': 130.23109436035156, 'KL_loss': 6.453636169433594}
    global step: 19001 (epoch: 7, step: 2200), loss: 150.607574 {'rec_loss': 144.6273193359375, 'KL_loss': 5.980248928070068}
    global step: 20001 (epoch: 8, step: 800), loss: 130.792786 {'rec_loss': 124.74642181396484, 'KL_loss': 6.046366214752197}
    global step: 21001 (epoch: 8, step: 1800), loss: 146.629868 {'rec_loss': 140.79193115234375, 'KL_loss': 5.837941646575928}
    global step: 22001 (epoch: 9, step: 400), loss: 153.740967 {'rec_loss': 147.82379150390625, 'KL_loss': 5.917177677154541}
    global step: 23001 (epoch: 9, step: 1400), loss: 138.719421 {'rec_loss': 132.48390197753906, 'KL_loss': 6.235525608062744}
    global step: 24001 (epoch: 10, step: 0), loss: 151.205429 {'rec_loss': 145.53599548339844, 'KL_loss': 5.66942834854126}
    global step: 25001 (epoch: 10, step: 1000), loss: 146.666656 {'rec_loss': 140.43710327148438, 'KL_loss': 6.229551792144775}
    global step: 26001 (epoch: 10, step: 2000), loss: 142.951202 {'rec_loss': 136.84202575683594, 'KL_loss': 6.109170436859131}
    global step: 27001 (epoch: 11, step: 600), loss: 153.645035 {'rec_loss': 147.4317626953125, 'KL_loss': 6.213268756866455}
    global step: 28001 (epoch: 11, step: 1600), loss: 142.338028 {'rec_loss': 135.96536254882812, 'KL_loss': 6.372664451599121}
    global step: 29001 (epoch: 12, step: 200), loss: 121.039650 {'rec_loss': 114.59912109375, 'KL_loss': 6.440528392791748}
    global step: 30001 (epoch: 12, step: 1200), loss: 150.002045 {'rec_loss': 143.62413024902344, 'KL_loss': 6.377912998199463}
    global step: 31001 (epoch: 12, step: 2200), loss: 139.753754 {'rec_loss': 133.73876953125, 'KL_loss': 6.014977931976318}
    global step: 32001 (epoch: 13, step: 800), loss: 138.504425 {'rec_loss': 132.12405395507812, 'KL_loss': 6.380375862121582}
    global step: 33001 (epoch: 13, step: 1800), loss: 141.812378 {'rec_loss': 135.72964477539062, 'KL_loss': 6.082735538482666}
    global step: 34001 (epoch: 14, step: 400), loss: 148.530853 {'rec_loss': 141.9413604736328, 'KL_loss': 6.589488983154297}
    global step: 35001 (epoch: 14, step: 1400), loss: 142.387177 {'rec_loss': 136.2464599609375, 'KL_loss': 6.140715599060059}
    global step: 36001 (epoch: 15, step: 0), loss: 145.792618 {'rec_loss': 139.19732666015625, 'KL_loss': 6.595291614532471}
    global step: 37001 (epoch: 15, step: 1000), loss: 149.632248 {'rec_loss': 143.00877380371094, 'KL_loss': 6.62346887588501}
    global step: 38001 (epoch: 15, step: 2000), loss: 151.311920 {'rec_loss': 144.85574340820312, 'KL_loss': 6.456182479858398}
    global step: 39001 (epoch: 16, step: 600), loss: 141.740387 {'rec_loss': 135.1404266357422, 'KL_loss': 6.599967956542969}
    global step: 40001 (epoch: 16, step: 1600), loss: 147.526703 {'rec_loss': 141.2978515625, 'KL_loss': 6.228857517242432}
    global step: 41001 (epoch: 17, step: 200), loss: 144.256058 {'rec_loss': 138.05581665039062, 'KL_loss': 6.200235366821289}
    global step: 42001 (epoch: 17, step: 1200), loss: 154.835693 {'rec_loss': 148.07627868652344, 'KL_loss': 6.759421348571777}
    global step: 43001 (epoch: 17, step: 2200), loss: 140.685226 {'rec_loss': 133.98208618164062, 'KL_loss': 6.7031426429748535}
    global step: 44001 (epoch: 18, step: 800), loss: 157.753510 {'rec_loss': 151.39076232910156, 'KL_loss': 6.3627519607543945}
    global step: 45001 (epoch: 18, step: 1800), loss: 158.038834 {'rec_loss': 151.92063903808594, 'KL_loss': 6.118192195892334}
    global step: 46001 (epoch: 19, step: 400), loss: 143.135788 {'rec_loss': 136.7926483154297, 'KL_loss': 6.343137741088867}
    global step: 47001 (epoch: 19, step: 1400), loss: 151.587723 {'rec_loss': 145.29478454589844, 'KL_loss': 6.292940616607666}
    global step: 48001 (epoch: 20, step: 0), loss: 137.412323 {'rec_loss': 131.33914184570312, 'KL_loss': 6.073181629180908}
    global step: 49001 (epoch: 20, step: 1000), loss: 156.075287 {'rec_loss': 149.8758544921875, 'KL_loss': 6.199430465698242}
    global step: 50001 (epoch: 20, step: 2000), loss: 138.036591 {'rec_loss': 131.34548950195312, 'KL_loss': 6.69110107421875}
    global step: 51001 (epoch: 21, step: 600), loss: 152.760269 {'rec_loss': 146.31385803222656, 'KL_loss': 6.446418285369873}
    global step: 52001 (epoch: 21, step: 1600), loss: 145.268143 {'rec_loss': 138.47811889648438, 'KL_loss': 6.790028095245361}
    global step: 53001 (epoch: 22, step: 200), loss: 132.748169 {'rec_loss': 126.4098892211914, 'KL_loss': 6.3382744789123535}
    global step: 54001 (epoch: 22, step: 1200), loss: 147.062286 {'rec_loss': 140.68423461914062, 'KL_loss': 6.378058910369873}
    global step: 55001 (epoch: 22, step: 2200), loss: 135.090927 {'rec_loss': 128.5495147705078, 'KL_loss': 6.541415691375732}
    global step: 56001 (epoch: 23, step: 800), loss: 129.563065 {'rec_loss': 122.6872787475586, 'KL_loss': 6.87578821182251}
    global step: 57001 (epoch: 23, step: 1800), loss: 133.864990 {'rec_loss': 127.26555633544922, 'KL_loss': 6.599431991577148}
    global step: 58001 (epoch: 24, step: 400), loss: 146.495010 {'rec_loss': 140.07911682128906, 'KL_loss': 6.415900707244873}
    global step: 59001 (epoch: 24, step: 1400), loss: 138.895340 {'rec_loss': 132.34800720214844, 'KL_loss': 6.5473313331604}
    global step: 60001 (epoch: 25, step: 0), loss: 138.156570 {'rec_loss': 131.46267700195312, 'KL_loss': 6.693898677825928}
    global step: 61001 (epoch: 25, step: 1000), loss: 144.336639 {'rec_loss': 138.03079223632812, 'KL_loss': 6.305854797363281}
    global step: 62001 (epoch: 25, step: 2000), loss: 144.420166 {'rec_loss': 137.9696044921875, 'KL_loss': 6.450555324554443}
    global step: 63001 (epoch: 26, step: 600), loss: 152.336563 {'rec_loss': 145.64559936523438, 'KL_loss': 6.690957069396973}
    global step: 64001 (epoch: 26, step: 1600), loss: 156.742050 {'rec_loss': 150.16107177734375, 'KL_loss': 6.580976486206055}
    global step: 65001 (epoch: 27, step: 200), loss: 141.006622 {'rec_loss': 134.6776123046875, 'KL_loss': 6.329017162322998}
    global step: 66001 (epoch: 27, step: 1200), loss: 118.688499 {'rec_loss': 111.87490844726562, 'KL_loss': 6.813587665557861}
    global step: 67001 (epoch: 27, step: 2200), loss: 126.650444 {'rec_loss': 120.20720672607422, 'KL_loss': 6.443235397338867}
    global step: 68001 (epoch: 28, step: 800), loss: 137.561646 {'rec_loss': 130.8014373779297, 'KL_loss': 6.7602152824401855}
    global step: 69001 (epoch: 28, step: 1800), loss: 137.988907 {'rec_loss': 131.29421997070312, 'KL_loss': 6.694681167602539}
    global step: 70001 (epoch: 29, step: 400), loss: 133.526688 {'rec_loss': 126.92362976074219, 'KL_loss': 6.603052616119385}
    global step: 71001 (epoch: 29, step: 1400), loss: 130.919144 {'rec_loss': 124.48625946044922, 'KL_loss': 6.4328789710998535}
    global step: 72001 (epoch: 30, step: 0), loss: 135.896194 {'rec_loss': 129.49371337890625, 'KL_loss': 6.402477741241455}
    global step: 73001 (epoch: 30, step: 1000), loss: 150.005875 {'rec_loss': 143.27015686035156, 'KL_loss': 6.735715389251709}
    global step: 74001 (epoch: 30, step: 2000), loss: 144.228180 {'rec_loss': 137.4684295654297, 'KL_loss': 6.759757995605469}
    global step: 75001 (epoch: 31, step: 600), loss: 142.223831 {'rec_loss': 135.7800750732422, 'KL_loss': 6.443760871887207}
    global step: 76001 (epoch: 31, step: 1600), loss: 158.183868 {'rec_loss': 151.51144409179688, 'KL_loss': 6.672426700592041}
    global step: 77001 (epoch: 32, step: 200), loss: 136.627930 {'rec_loss': 129.60386657714844, 'KL_loss': 7.024058818817139}
    global step: 78001 (epoch: 32, step: 1200), loss: 143.935013 {'rec_loss': 137.26451110839844, 'KL_loss': 6.670505046844482}
    global step: 79001 (epoch: 32, step: 2200), loss: 133.028702 {'rec_loss': 126.46426391601562, 'KL_loss': 6.564435958862305}
    global step: 80001 (epoch: 33, step: 800), loss: 131.735382 {'rec_loss': 124.59033203125, 'KL_loss': 7.145053386688232}
    global step: 81001 (epoch: 33, step: 1800), loss: 132.858734 {'rec_loss': 125.81361389160156, 'KL_loss': 7.045113563537598}
    global step: 82001 (epoch: 34, step: 400), loss: 132.802917 {'rec_loss': 126.00375366210938, 'KL_loss': 6.799161434173584}
    global step: 83001 (epoch: 34, step: 1400), loss: 124.836990 {'rec_loss': 118.06398010253906, 'KL_loss': 6.773006916046143}
    global step: 84001 (epoch: 35, step: 0), loss: 147.028107 {'rec_loss': 140.612548828125, 'KL_loss': 6.415552616119385}
    global step: 85001 (epoch: 35, step: 1000), loss: 148.382660 {'rec_loss': 141.88731384277344, 'KL_loss': 6.495350360870361}
    global step: 86001 (epoch: 35, step: 2000), loss: 114.885689 {'rec_loss': 108.35710906982422, 'KL_loss': 6.5285773277282715}
    global step: 87001 (epoch: 36, step: 600), loss: 143.311234 {'rec_loss': 136.4329071044922, 'KL_loss': 6.87832498550415}
    global step: 88001 (epoch: 36, step: 1600), loss: 137.767471 {'rec_loss': 130.87759399414062, 'KL_loss': 6.889870643615723}
    global step: 89001 (epoch: 37, step: 200), loss: 130.491547 {'rec_loss': 123.73429107666016, 'KL_loss': 6.7572503089904785}
    global step: 90001 (epoch: 37, step: 1200), loss: 145.793198 {'rec_loss': 139.19027709960938, 'KL_loss': 6.6029181480407715}
    global step: 91001 (epoch: 37, step: 2200), loss: 135.194305 {'rec_loss': 128.15338134765625, 'KL_loss': 7.040927410125732}
    global step: 92001 (epoch: 38, step: 800), loss: 147.989197 {'rec_loss': 141.025634765625, 'KL_loss': 6.963564395904541}
    global step: 93001 (epoch: 38, step: 1800), loss: 156.352432 {'rec_loss': 149.78884887695312, 'KL_loss': 6.563579082489014}
    global step: 94001 (epoch: 39, step: 400), loss: 124.167084 {'rec_loss': 117.35694885253906, 'KL_loss': 6.810135841369629}
    global step: 95001 (epoch: 39, step: 1400), loss: 133.931824 {'rec_loss': 127.076171875, 'KL_loss': 6.855644226074219}
    global step: 96001 (epoch: 40, step: 0), loss: 137.757187 {'rec_loss': 131.04843139648438, 'KL_loss': 6.708749294281006}
    global step: 97001 (epoch: 40, step: 1000), loss: 122.034508 {'rec_loss': 115.16448974609375, 'KL_loss': 6.8700151443481445}
    global step: 98001 (epoch: 40, step: 2000), loss: 149.646179 {'rec_loss': 142.96275329589844, 'KL_loss': 6.683427810668945}
    global step: 99001 (epoch: 41, step: 600), loss: 133.836182 {'rec_loss': 126.85317993164062, 'KL_loss': 6.983006477355957}
    global step: 100001 (epoch: 41, step: 1600), loss: 153.473190 {'rec_loss': 146.57015991210938, 'KL_loss': 6.903028964996338}
    global step: 101001 (epoch: 42, step: 200), loss: 140.377136 {'rec_loss': 133.92259216308594, 'KL_loss': 6.45454216003418}
    global step: 102001 (epoch: 42, step: 1200), loss: 136.116486 {'rec_loss': 129.1258544921875, 'KL_loss': 6.990625858306885}
    global step: 103001 (epoch: 42, step: 2200), loss: 145.840454 {'rec_loss': 138.85211181640625, 'KL_loss': 6.988345623016357}
    global step: 104001 (epoch: 43, step: 800), loss: 133.230942 {'rec_loss': 126.5788803100586, 'KL_loss': 6.652055740356445}
    global step: 105001 (epoch: 43, step: 1800), loss: 138.530685 {'rec_loss': 131.85189819335938, 'KL_loss': 6.678781509399414}
    global step: 106001 (epoch: 44, step: 400), loss: 143.406754 {'rec_loss': 136.46534729003906, 'KL_loss': 6.941405296325684}
    global step: 107001 (epoch: 44, step: 1400), loss: 133.396622 {'rec_loss': 126.36222076416016, 'KL_loss': 7.034398078918457}
    global step: 108001 (epoch: 45, step: 0), loss: 128.078629 {'rec_loss': 121.46001434326172, 'KL_loss': 6.618609428405762}
    global step: 109001 (epoch: 45, step: 1000), loss: 130.726334 {'rec_loss': 124.09049224853516, 'KL_loss': 6.635843276977539}
    global step: 110001 (epoch: 45, step: 2000), loss: 132.995712 {'rec_loss': 126.2112045288086, 'KL_loss': 6.784502029418945}
    global step: 111001 (epoch: 46, step: 600), loss: 137.652328 {'rec_loss': 130.78419494628906, 'KL_loss': 6.868131160736084}
    global step: 112001 (epoch: 46, step: 1600), loss: 155.614502 {'rec_loss': 148.97679138183594, 'KL_loss': 6.637718200683594}
    global step: 113001 (epoch: 47, step: 200), loss: 126.726830 {'rec_loss': 120.05323028564453, 'KL_loss': 6.673602104187012}
    global step: 114001 (epoch: 47, step: 1200), loss: 144.877823 {'rec_loss': 137.88174438476562, 'KL_loss': 6.996085166931152}
    global step: 115001 (epoch: 47, step: 2200), loss: 129.870941 {'rec_loss': 123.2160873413086, 'KL_loss': 6.654850959777832}
    global step: 116001 (epoch: 48, step: 800), loss: 137.761963 {'rec_loss': 130.93246459960938, 'KL_loss': 6.829492568969727}
    global step: 117001 (epoch: 48, step: 1800), loss: 142.628860 {'rec_loss': 135.89459228515625, 'KL_loss': 6.734273910522461}
    global step: 118001 (epoch: 49, step: 400), loss: 145.509384 {'rec_loss': 138.45704650878906, 'KL_loss': 7.052340984344482}
    global step: 119001 (epoch: 49, step: 1400), loss: 143.493713 {'rec_loss': 136.63775634765625, 'KL_loss': 6.855957984924316}
    global step: 120001 (epoch: 50, step: 0), loss: 141.234070 {'rec_loss': 134.65931701660156, 'KL_loss': 6.574753284454346}
    global step: 121001 (epoch: 50, step: 1000), loss: 126.677299 {'rec_loss': 120.0406494140625, 'KL_loss': 6.636650085449219}
    global step: 122001 (epoch: 50, step: 2000), loss: 138.872620 {'rec_loss': 131.8439178466797, 'KL_loss': 7.02869987487793}
    global step: 123001 (epoch: 51, step: 600), loss: 130.641541 {'rec_loss': 123.79452514648438, 'KL_loss': 6.84702205657959}
    global step: 124001 (epoch: 51, step: 1600), loss: 135.720032 {'rec_loss': 128.63308715820312, 'KL_loss': 7.086945533752441}
    global step: 125001 (epoch: 52, step: 200), loss: 133.589371 {'rec_loss': 126.97989654541016, 'KL_loss': 6.609468936920166}
    global step: 126001 (epoch: 52, step: 1200), loss: 131.451019 {'rec_loss': 124.58702087402344, 'KL_loss': 6.863994121551514}
    global step: 127001 (epoch: 52, step: 2200), loss: 132.347763 {'rec_loss': 125.50472259521484, 'KL_loss': 6.843039512634277}
    global step: 128001 (epoch: 53, step: 800), loss: 146.518051 {'rec_loss': 139.67539978027344, 'KL_loss': 6.842656135559082}
    global step: 129001 (epoch: 53, step: 1800), loss: 157.011108 {'rec_loss': 150.30189514160156, 'KL_loss': 6.70920991897583}
    global step: 130001 (epoch: 54, step: 400), loss: 134.386749 {'rec_loss': 127.38662719726562, 'KL_loss': 7.000129699707031}
    global step: 131001 (epoch: 54, step: 1400), loss: 145.572235 {'rec_loss': 138.69349670410156, 'KL_loss': 6.878734588623047}
    global step: 132001 (epoch: 55, step: 0), loss: 125.333954 {'rec_loss': 118.29652404785156, 'KL_loss': 7.0374298095703125}
    global step: 133001 (epoch: 55, step: 1000), loss: 131.882553 {'rec_loss': 125.0451889038086, 'KL_loss': 6.837369441986084}
    global step: 134001 (epoch: 55, step: 2000), loss: 125.442223 {'rec_loss': 118.6576156616211, 'KL_loss': 6.784609794616699}
    global step: 135001 (epoch: 56, step: 600), loss: 148.013550 {'rec_loss': 140.99044799804688, 'KL_loss': 7.023099422454834}
    global step: 136001 (epoch: 56, step: 1600), loss: 137.211777 {'rec_loss': 130.6486053466797, 'KL_loss': 6.563168048858643}
    global step: 137001 (epoch: 57, step: 200), loss: 139.386581 {'rec_loss': 132.5934600830078, 'KL_loss': 6.793117523193359}
    global step: 138001 (epoch: 57, step: 1200), loss: 145.961960 {'rec_loss': 139.1885223388672, 'KL_loss': 6.773439407348633}
    global step: 139001 (epoch: 57, step: 2200), loss: 129.107452 {'rec_loss': 122.1865005493164, 'KL_loss': 6.920951843261719}
    global step: 140001 (epoch: 58, step: 800), loss: 152.372299 {'rec_loss': 145.46336364746094, 'KL_loss': 6.908938407897949}
    global step: 141001 (epoch: 58, step: 1800), loss: 122.616478 {'rec_loss': 115.65274047851562, 'KL_loss': 6.963738918304443}
    global step: 142001 (epoch: 59, step: 400), loss: 126.946976 {'rec_loss': 119.83224487304688, 'KL_loss': 7.11472749710083}
    global step: 143001 (epoch: 59, step: 1400), loss: 130.606735 {'rec_loss': 123.54939270019531, 'KL_loss': 7.057343006134033}
    global step: 144001 (epoch: 60, step: 0), loss: 129.670547 {'rec_loss': 122.64189910888672, 'KL_loss': 7.02864408493042}
    global step: 145001 (epoch: 60, step: 1000), loss: 130.764999 {'rec_loss': 124.27812194824219, 'KL_loss': 6.486879348754883}
    global step: 146001 (epoch: 60, step: 2000), loss: 143.461884 {'rec_loss': 136.51370239257812, 'KL_loss': 6.948187828063965}
    global step: 147001 (epoch: 61, step: 600), loss: 129.673096 {'rec_loss': 122.68474578857422, 'KL_loss': 6.988343238830566}
    global step: 148001 (epoch: 61, step: 1600), loss: 133.270065 {'rec_loss': 126.31771087646484, 'KL_loss': 6.952355861663818}
    global step: 149001 (epoch: 62, step: 200), loss: 127.986053 {'rec_loss': 121.31829071044922, 'KL_loss': 6.66776180267334}
    global step: 150001 (epoch: 62, step: 1200), loss: 143.038040 {'rec_loss': 135.8154296875, 'KL_loss': 7.222614765167236}
    global step: 151001 (epoch: 62, step: 2200), loss: 135.472427 {'rec_loss': 128.39605712890625, 'KL_loss': 7.076375484466553}
    global step: 152001 (epoch: 63, step: 800), loss: 134.656021 {'rec_loss': 127.6678695678711, 'KL_loss': 6.9881486892700195}
    global step: 153001 (epoch: 63, step: 1800), loss: 137.371704 {'rec_loss': 130.38058471679688, 'KL_loss': 6.991122245788574}
    global step: 154001 (epoch: 64, step: 400), loss: 152.199524 {'rec_loss': 144.83111572265625, 'KL_loss': 7.368402004241943}
    global step: 155001 (epoch: 64, step: 1400), loss: 146.178879 {'rec_loss': 139.37396240234375, 'KL_loss': 6.804912567138672}
    global step: 156001 (epoch: 65, step: 0), loss: 134.227463 {'rec_loss': 126.88951873779297, 'KL_loss': 7.337939262390137}
    global step: 157001 (epoch: 65, step: 1000), loss: 131.559036 {'rec_loss': 124.71397399902344, 'KL_loss': 6.845065116882324}
    global step: 158001 (epoch: 65, step: 2000), loss: 137.806686 {'rec_loss': 130.7169952392578, 'KL_loss': 7.089696884155273}
    global step: 159001 (epoch: 66, step: 600), loss: 141.688019 {'rec_loss': 134.77377319335938, 'KL_loss': 6.914250373840332}
    global step: 160001 (epoch: 66, step: 1600), loss: 136.261124 {'rec_loss': 129.28662109375, 'KL_loss': 6.9745049476623535}
    global step: 161001 (epoch: 67, step: 200), loss: 126.458328 {'rec_loss': 119.40026092529297, 'KL_loss': 7.058066368103027}
    global step: 162001 (epoch: 67, step: 1200), loss: 132.397461 {'rec_loss': 125.417236328125, 'KL_loss': 6.980222225189209}
    global step: 163001 (epoch: 67, step: 2200), loss: 147.230743 {'rec_loss': 140.32305908203125, 'KL_loss': 6.907678604125977}
    global step: 164001 (epoch: 68, step: 800), loss: 125.730896 {'rec_loss': 118.61128997802734, 'KL_loss': 7.119609355926514}
    global step: 165001 (epoch: 68, step: 1800), loss: 125.929352 {'rec_loss': 119.0011215209961, 'KL_loss': 6.928231716156006}
    global step: 166001 (epoch: 69, step: 400), loss: 131.019501 {'rec_loss': 124.02486419677734, 'KL_loss': 6.994638442993164}
    global step: 167001 (epoch: 69, step: 1400), loss: 140.461655 {'rec_loss': 133.43666076660156, 'KL_loss': 7.024997234344482}
    global step: 168001 (epoch: 70, step: 0), loss: 131.573181 {'rec_loss': 124.6004867553711, 'KL_loss': 6.972692489624023}
    global step: 169001 (epoch: 70, step: 1000), loss: 138.651901 {'rec_loss': 131.8728485107422, 'KL_loss': 6.779056072235107}
    global step: 170001 (epoch: 70, step: 2000), loss: 135.175537 {'rec_loss': 128.05235290527344, 'KL_loss': 7.123190879821777}
    global step: 171001 (epoch: 71, step: 600), loss: 142.410416 {'rec_loss': 135.2754669189453, 'KL_loss': 7.13494873046875}
    global step: 172001 (epoch: 71, step: 1600), loss: 125.139404 {'rec_loss': 117.83482360839844, 'KL_loss': 7.30457878112793}
    global step: 173001 (epoch: 72, step: 200), loss: 133.213516 {'rec_loss': 126.09675598144531, 'KL_loss': 7.116753578186035}
    global step: 174001 (epoch: 72, step: 1200), loss: 135.250107 {'rec_loss': 128.3852081298828, 'KL_loss': 6.864904403686523}
    global step: 175001 (epoch: 72, step: 2200), loss: 127.980125 {'rec_loss': 120.87749481201172, 'KL_loss': 7.102629661560059}
    global step: 176001 (epoch: 73, step: 800), loss: 127.796280 {'rec_loss': 120.81212615966797, 'KL_loss': 6.984157562255859}
    global step: 177001 (epoch: 73, step: 1800), loss: 126.277306 {'rec_loss': 119.3787841796875, 'KL_loss': 6.89852237701416}
    global step: 178001 (epoch: 74, step: 400), loss: 132.841705 {'rec_loss': 126.17105102539062, 'KL_loss': 6.670646667480469}
    global step: 179001 (epoch: 74, step: 1400), loss: 149.338120 {'rec_loss': 142.27276611328125, 'KL_loss': 7.065348148345947}
    global step: 180001 (epoch: 75, step: 0), loss: 122.915314 {'rec_loss': 115.96500396728516, 'KL_loss': 6.950306415557861}
    global step: 181001 (epoch: 75, step: 1000), loss: 133.610764 {'rec_loss': 126.80167388916016, 'KL_loss': 6.809087753295898}
    global step: 182001 (epoch: 75, step: 2000), loss: 150.368103 {'rec_loss': 143.544677734375, 'KL_loss': 6.823432445526123}
    global step: 183001 (epoch: 76, step: 600), loss: 137.751221 {'rec_loss': 130.5072784423828, 'KL_loss': 7.243935585021973}
    global step: 184001 (epoch: 76, step: 1600), loss: 125.326523 {'rec_loss': 118.25154113769531, 'KL_loss': 7.07498025894165}
    global step: 185001 (epoch: 77, step: 200), loss: 130.183472 {'rec_loss': 123.19072723388672, 'KL_loss': 6.992747783660889}
    global step: 186001 (epoch: 77, step: 1200), loss: 137.318695 {'rec_loss': 130.2268524169922, 'KL_loss': 7.091846942901611}
    global step: 187001 (epoch: 77, step: 2200), loss: 140.299088 {'rec_loss': 133.098876953125, 'KL_loss': 7.200212478637695}
    global step: 188001 (epoch: 78, step: 800), loss: 119.522835 {'rec_loss': 112.65771484375, 'KL_loss': 6.865120887756348}
    global step: 189001 (epoch: 78, step: 1800), loss: 136.406738 {'rec_loss': 129.34774780273438, 'KL_loss': 7.058998107910156}
    global step: 190001 (epoch: 79, step: 400), loss: 136.507065 {'rec_loss': 129.4857940673828, 'KL_loss': 7.021268367767334}
    global step: 191001 (epoch: 79, step: 1400), loss: 120.505241 {'rec_loss': 113.74276733398438, 'KL_loss': 6.762475967407227}
    global step: 192001 (epoch: 80, step: 0), loss: 146.556442 {'rec_loss': 139.49156188964844, 'KL_loss': 7.064886093139648}
    global step: 193001 (epoch: 80, step: 1000), loss: 133.781754 {'rec_loss': 126.91250610351562, 'KL_loss': 6.8692450523376465}
    global step: 194001 (epoch: 80, step: 2000), loss: 137.719360 {'rec_loss': 130.69625854492188, 'KL_loss': 7.023099899291992}
    global step: 195001 (epoch: 81, step: 600), loss: 135.224487 {'rec_loss': 127.95004272460938, 'KL_loss': 7.274443626403809}
    global step: 196001 (epoch: 81, step: 1600), loss: 124.928482 {'rec_loss': 117.93548583984375, 'KL_loss': 6.9929986000061035}
    global step: 197001 (epoch: 82, step: 200), loss: 143.383301 {'rec_loss': 136.48995971679688, 'KL_loss': 6.893346786499023}
    global step: 198001 (epoch: 82, step: 1200), loss: 141.551239 {'rec_loss': 134.69212341308594, 'KL_loss': 6.859119892120361}
    global step: 199001 (epoch: 82, step: 2200), loss: 129.202393 {'rec_loss': 122.29231262207031, 'KL_loss': 6.910079002380371}
    global step: 200001 (epoch: 83, step: 800), loss: 146.323578 {'rec_loss': 139.3558807373047, 'KL_loss': 6.967696189880371}
    global step: 201001 (epoch: 83, step: 1800), loss: 129.939438 {'rec_loss': 122.9568099975586, 'KL_loss': 6.982625961303711}
    global step: 202001 (epoch: 84, step: 400), loss: 131.212921 {'rec_loss': 124.54302215576172, 'KL_loss': 6.669893741607666}
    global step: 203001 (epoch: 84, step: 1400), loss: 138.927887 {'rec_loss': 131.58836364746094, 'KL_loss': 7.339522361755371}
    global step: 204001 (epoch: 85, step: 0), loss: 131.880920 {'rec_loss': 124.89168548583984, 'KL_loss': 6.9892425537109375}
    global step: 205001 (epoch: 85, step: 1000), loss: 141.849457 {'rec_loss': 134.87460327148438, 'KL_loss': 6.974861145019531}
    global step: 206001 (epoch: 85, step: 2000), loss: 132.132568 {'rec_loss': 124.73577880859375, 'KL_loss': 7.396794319152832}
    global step: 207001 (epoch: 86, step: 600), loss: 139.156250 {'rec_loss': 132.06137084960938, 'KL_loss': 7.094881057739258}
    global step: 208001 (epoch: 86, step: 1600), loss: 117.179779 {'rec_loss': 110.27417755126953, 'KL_loss': 6.905599594116211}
    global step: 209001 (epoch: 87, step: 200), loss: 122.651817 {'rec_loss': 115.58970642089844, 'KL_loss': 7.062108039855957}
    global step: 210001 (epoch: 87, step: 1200), loss: 132.005997 {'rec_loss': 125.14656066894531, 'KL_loss': 6.859440326690674}
    global step: 211001 (epoch: 87, step: 2200), loss: 127.114693 {'rec_loss': 120.26438903808594, 'KL_loss': 6.850306034088135}
    global step: 212001 (epoch: 88, step: 800), loss: 124.727989 {'rec_loss': 118.05899810791016, 'KL_loss': 6.668989658355713}
    global step: 213001 (epoch: 88, step: 1800), loss: 139.754272 {'rec_loss': 132.7419891357422, 'KL_loss': 7.012279987335205}
    global step: 214001 (epoch: 89, step: 400), loss: 129.886459 {'rec_loss': 122.83661651611328, 'KL_loss': 7.049844741821289}
    global step: 215001 (epoch: 89, step: 1400), loss: 139.712524 {'rec_loss': 132.29296875, 'KL_loss': 7.419556140899658}
    global step: 216001 (epoch: 90, step: 0), loss: 141.237900 {'rec_loss': 133.72105407714844, 'KL_loss': 7.516848087310791}
    global step: 217001 (epoch: 90, step: 1000), loss: 136.937973 {'rec_loss': 129.62791442871094, 'KL_loss': 7.310065746307373}
    global step: 218001 (epoch: 90, step: 2000), loss: 140.723297 {'rec_loss': 133.528076171875, 'KL_loss': 7.195224285125732}
    global step: 219001 (epoch: 91, step: 600), loss: 136.323914 {'rec_loss': 129.34768676757812, 'KL_loss': 6.976224422454834}
    global step: 220001 (epoch: 91, step: 1600), loss: 135.090057 {'rec_loss': 127.60093688964844, 'KL_loss': 7.489126205444336}
    global step: 221001 (epoch: 92, step: 200), loss: 114.217422 {'rec_loss': 107.03468322753906, 'KL_loss': 7.182737827301025}
    global step: 222001 (epoch: 92, step: 1200), loss: 124.861351 {'rec_loss': 117.74789428710938, 'KL_loss': 7.113456726074219}
    global step: 223001 (epoch: 92, step: 2200), loss: 138.917099 {'rec_loss': 131.92974853515625, 'KL_loss': 6.987344741821289}
    global step: 224001 (epoch: 93, step: 800), loss: 130.459152 {'rec_loss': 123.26513671875, 'KL_loss': 7.194011211395264}
    global step: 225001 (epoch: 93, step: 1800), loss: 138.830261 {'rec_loss': 131.9112091064453, 'KL_loss': 6.91905403137207}
    global step: 226001 (epoch: 94, step: 400), loss: 132.234451 {'rec_loss': 125.29092407226562, 'KL_loss': 6.943530559539795}
    global step: 227001 (epoch: 94, step: 1400), loss: 129.949951 {'rec_loss': 123.01248931884766, 'KL_loss': 6.937456130981445}
    global step: 228001 (epoch: 95, step: 0), loss: 128.617355 {'rec_loss': 121.64593505859375, 'KL_loss': 6.97142219543457}
    global step: 229001 (epoch: 95, step: 1000), loss: 133.652252 {'rec_loss': 126.80097198486328, 'KL_loss': 6.851276874542236}
    global step: 230001 (epoch: 95, step: 2000), loss: 120.783356 {'rec_loss': 113.87052154541016, 'KL_loss': 6.912832260131836}
    global step: 231001 (epoch: 96, step: 600), loss: 139.337708 {'rec_loss': 132.13723754882812, 'KL_loss': 7.200466156005859}
    global step: 232001 (epoch: 96, step: 1600), loss: 132.858185 {'rec_loss': 125.79011535644531, 'KL_loss': 7.068065166473389}
    global step: 233001 (epoch: 97, step: 200), loss: 137.434570 {'rec_loss': 130.3426971435547, 'KL_loss': 7.091869831085205}
    global step: 234001 (epoch: 97, step: 1200), loss: 131.773666 {'rec_loss': 124.41007232666016, 'KL_loss': 7.363587379455566}
    global step: 235001 (epoch: 97, step: 2200), loss: 125.605377 {'rec_loss': 118.51722717285156, 'KL_loss': 7.088152885437012}
    global step: 236001 (epoch: 98, step: 800), loss: 143.444931 {'rec_loss': 136.42356872558594, 'KL_loss': 7.021368026733398}
    global step: 237001 (epoch: 98, step: 1800), loss: 111.817024 {'rec_loss': 104.55360412597656, 'KL_loss': 7.263417720794678}
    global step: 238001 (epoch: 99, step: 400), loss: 136.297134 {'rec_loss': 129.21946716308594, 'KL_loss': 7.077672958374023}
    global step: 239001 (epoch: 99, step: 1400), loss: 152.140182 {'rec_loss': 144.93466186523438, 'KL_loss': 7.2055277824401855}

## VAE analysis

```
# In this cell we look at the VAE from the autoencoder perspective.
# We see how well it can reconstruct the input data and,
# given its probablilistic nature, how variable are the reconstructions
# when several zs are samples from q(z|x).
#

model.eval()
batch_x = next(iter(train_loader))[0]
with torch.no_grad():
    reconstruction, mu, sigma2 = model(batch_x)

plot_mat(n(batch_x)[:,None,:,:], cmap='gray')
plt.colorbar()
plt.title('Data samples')

reconstruction_from_mean = model.decoder(mu).view(batch_x.size())
plt.figure()
plot_mat(n(reconstruction_from_mean)[:, None, :, :], cmap='gray')
plt.colorbar()
plt.title('Reconstructions using the latent expected value')

same_x = batch_x[2:3].repeat(20, 1, 1)
same_recs, _, _ = model(same_x)
plt.figure()
plot_mat(n(same_recs)[:, None, :, :], cmap='gray')
plt.colorbar()
plt.title('Reconstructions from multiple latent samples')
```

```
# Here we make an interpolation in the latent space: we take two samples and
# find their latent codes. We then connect the codes using a straigth line,
# and plot reconstruction as we moove through this line.

interp_weights = torch.linspace(
    0.0, 1.0, steps=25, dtype=torch.float32, device=batch_x.device).view(-1, 1)
k1, k2 = 2, 3
interp_mu = interp_weights * mu[k1] + (1.0 - interp_weights) * mu[k2]
interp_sigma2 = interp_weights * sigma2[k1] + (1.0 - interp_weights) * sigma2[k2]
interp_recs = model.decoder(
    model.reparameterize(interp_mu, interp_sigma2)).view(-1, 1, 28, 28)

plt.figure()
plot_mat(n(interp_recs), cmap='gray')
plt.colorbar()
plt.title('interpolation between two samples')
```

```
# Finally, we can loook at samples generated using a VAE
#

z_rand = torch.randn(50, latent_dim, device=batch_x.device, dtype=batch_x.dtype)
samples = model.decoder(z_rand).view(-1, 1, batch_x.size(1), batch_x.size(2))
plot_mat(n(samples), cmap='gray')
plt.title("Samples from a %ddimensional VAE" % latent_dim)
```

```
# Finally, for a 2D latent space, we can draw images that are generated from
# a grid over the two latent dimensions.
#

if latent_dim == 2:
    plt.figure(figsize=(10, 10))
    z_mesh_x, z_mesh_y = np.meshgrid(np.linspace(-3, 3, 20), np.linspace(-3, 3, 20))
    z_mesh = np.concatenate([z_mesh_x.reshape(-1, 1),
                             z_mesh_y.reshape(-1, 1)], axis=1)

    mesh = model.decoder(torch.tensor(z_mesh, dtype=torch.float32, device=batch_x.device)).view(-1, 1, 28, 28)
    plot_mat(n(mesh), prop=(1,1), cmap='gray')
    plt.title('Reconstructions from a 2D latent space')
```

```

```


```python
from IPython.display import HTML

# Cell visibility - COMPLETE:
#tag = HTML('''<style>
#div.input {
#    display:none;
#}
#</style>''')
#display(tag)

#Cell visibility - TOGGLE:
tag = HTML('''
<p style="text-align:right">
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.</p>''')
display(tag)
```

Toggle cell visibility here.

## Derivazione di polinomi

Questo esempio interattivo consente di esaminare la derivazione delle funzioni polinomiali.
Modifica liberamente i coefficienti polinomiali (fino al quarto ordine): la funzione determinata verrà mostrata nel rispettivo grafico, insieme alla sua derivata prima. Sia la funzione polinomiale (impostata tramite gli slider) che la sua derivata prima (calcolata di conseguenza) verranno presentate anche in notazione matematica.

```python
%matplotlib inline
#%config InlineBackend.close_figures=False
from ipywidgets import interactive
from ipywidgets import widgets
from IPython.display import Latex, display, Markdown # For displaying Markdown and LaTeX code
import matplotlib.pyplot as plt
import numpy as np
import math
import sympy as sym
import matplotlib.patches as mpatches
from IPython.display import HTML, clear_output
from IPython.display import display

# Mathematical notation of a specific (user-defined) polynomial, shown as Markdown
fourth_order = "e + d * x + c * x ** 2 + b * x ** 3 + a * x ** 4"
third_order = "d + c * x + b * x ** 2 + a * x ** 3"
second_order = "c + b * x + a * x ** 2"
first_order = "b + a * x"
zero_order = "a"
tf = sym.sympify(fourth_order)
w_mark = Markdown('$%s$' %sym.latex(tf))

# General mathematical notation of a polynomial (shown in Label widget)
fourth_order_html = "$f(x)=ax^4$ + $bx^3$ + $cx^2$ + $dx$ + $e$"
third_order_html = "$f(x)=ax^3$ + $bx^2$ + $cx$ + $d$"
second_order_html = "$f(x)=ax^2$ + $bx$ + $c$"
first_order_html = "$f(x)=ax$ + $b$"
zero_order_html = "$f(x)=a$"
w_funLabel = widgets.Label(layout=widgets.Layout(width='40%', margin='0px 0px 0px 50px'),)

# Input sliders for coefficients of a polynomial
fs_a = widgets.FloatSlider(description='$a$', min=-10.0, max=10.0, step=0.5, continuous_update=False)
fs_b = widgets.FloatSlider(description='$b$',min=-10.0, max=10.0, step=0.5, continuous_update=False)
fs_c = widgets.FloatSlider(description='$c$',min=-10.0, max=10.0, step=0.5, continuous_update=False)
fs_d = widgets.FloatSlider(description='$d$',min=-10.0, max=10.0, step=0.5, continuous_update=False)
fs_e = widgets.FloatSlider(description='$e$',min=-10.0, max=10.0, step=0.5, continuous_update=False)
input_widgets = widgets.HBox([fs_a, fs_b, fs_c, fs_d, fs_e])

# Dropdown: selecting the order of a ploynomial [0-4]
# Selection will invoke showing/hiding particular slider widgets
dd_order = widgets.Dropdown(
    options=['4', '3', '2', '1', '0'],
    value='4',
    description='Seleziona l\'ordine del polinomio [0-4]:',
    disabled=False,
    style = {'description_width': 'initial'},
)

def dropdown_eventhandler(change):
    fs_a.layout.visibility = 'hidden'
    fs_b.layout.visibility = 'hidden'
    fs_c.layout.visibility = 'hidden'
    fs_d.layout.visibility = 'hidden'
    fs_e.layout.visibility = 'hidden'

    if (dd_order.value == '4'):
        fs_a.layout.visibility = 'visible'
        fs_a.description = '$a$'
        fs_b.layout.visibility = 'visible'
        fs_b.description = '$b$'
        fs_c.layout.visibility = 'visible'
        fs_c.description = '$c$'
        fs_d.layout.visibility = 'visible'
        fs_d.description = '$d$'
        fs_e.layout.visibility = 'visible'
        fs_e.description = '$e$'
        w_funLabel.value=fourth_order_html
    if (dd_order.value == '3'):
        fs_a.value = 0
        fs_b.layout.visibility = 'visible'
        fs_b.description = '$a$'
        fs_c.layout.visibility = 'visible'
        fs_c.description = '$b$'
        fs_d.layout.visibility = 'visible'
        fs_d.description = '$c$'
        fs_e.layout.visibility = 'visible'
        fs_e.description = '$d$'
        w_funLabel.value=third_order_html
    if (dd_order.value == '2'):
        fs_a.value = 0
        fs_b.value = 0
        fs_c.layout.visibility = 'visible'
        fs_c.description = '$a$'
        fs_d.layout.visibility = 'visible'
        fs_d.description = '$b$'
        fs_e.layout.visibility = 'visible'
        fs_e.description = '$c$'
        w_funLabel.value=second_order_html
    if (dd_order.value == '1'):
        fs_a.value = 0
        fs_b.value = 0
        fs_c.value = 0
        fs_d.layout.visibility = 'visible'
        fs_d.description = '$a$'
        fs_e.layout.visibility = 'visible'
        fs_e.description = '$b$'
        w_funLabel.value=first_order_html
    if (dd_order.value == '0'):
        fs_a.value = 0
        fs_b.value = 0
        fs_c.value = 0
        fs_d.value = 0
        fs_e.layout.visibility = 'visible'
        fs_e.description = '$a$'
        w_funLabel.value=zero_order_html

dd_order.observe(dropdown_eventhandler, names='value')

# Utility functions
x = sym.symbols('x')

def fprime(fx):
    if not fx.is_zero:
        return sym.diff(fx, x)
    else:
        return ""

def convert(base_text, ss):
    if ss != "":
        tf = sym.sympify(ss)
        display(Markdown(base_text + '$%s$' %sym.latex(tf)))

def polynomial_function(X_quaded, X_cubed, X_squared, X, const, x):
    return const + X * x + X_squared * x ** 2 + X_cubed * x ** 3 + X_quaded * x ** 4

def fun(x):
    global a, b, c, d, e
    return e + d * x + c * x **2 + b * x ** 3 + a * x ** 4

#Plot
red_patch = mpatches.Patch(color='red', label='$f(x)$')
blue_patch = mpatches.Patch(color='green', label='Derivata prima di $f(x)$')

XLIM = 10
YLIM = 20
XTICK = 2
YTICK = 5

a = 0
b = 0
c = 0
d = 0
e = 0

def plot_function(X_quaded, X_cubed, X_squared, X, const):

        global a, b, c, d, e, x
        a = X_quaded
        b = X_cubed
        c = X_squared
        d = X
        e = const
        fig = plt.figure(figsize=(12,6))

        x_p = np.linspace(-XLIM, XLIM, num=1000)
        plt.plot(x_p, polynomial_function(X_quaded, X_cubed, X_squared, X, const, x_p), 'r-')
        y_p = [polynomial_function(X_quaded, X_cubed, X_squared, X, const, num) for num in x_p]
        dydx = np.diff(y_p) / np.diff(x_p)
        plt.plot(x_p[:-1], dydx, 'g-')
        plt.grid(True)
        plt.xlabel('x')
        plt.ylabel('$f(x)$, derivata di $f(x)$')
        plt.xlim(-XLIM, XLIM)
        plt.ylim(-YLIM, YLIM)
        plt.axhline(y=0,lw=0.8,color='k')
        plt.axvline(x=0,lw=0.8,color='k')
        plt.legend(handles=[red_patch, blue_patch])
        plt.show()

        convert("$f(x)$: ", fun(x))
        deriv = fprime(fun(x))
        convert("Derivata prima di $f(x)$: ", deriv)

# Show and interact
w_funLabel.value=fourth_order_html
control_widgets = widgets.HBox()
control_widgets.children=[dd_order, w_funLabel]
display(control_widgets)
interactive(plot_function, const=fs_e, X=fs_d, X_squared=fs_c, X_cubed=fs_b, X_quaded=fs_a)
```

    HBox(children=(Dropdown(description="Seleziona l'ordine del polinomio [0-4]:", options=('4', '3', '2', '1', '0…

    interactive(children=(FloatSlider(value=0.0, continuous_update=False, description='$a$', max=10.0, min=-10.0, …