# Implementing a Variational Autoencoder

In trying to understand [LFADS](https://arxiv.org/abs/1608.06315), I've been spending my time learning about the basis of the model: Variational Autoencoders. Because the current implementation of LFADS is written in Tensorflow, I've also been trying to learn Tensorflow. Thus, I'm going to put my knowledge of VAEs and TF to the test by implementing my own Variational Autoencoder!

My understanding of VAEs is largely due to [Carl Doersch's tutorial](https://arxiv.org/abs/1606.05908).

```python
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
%matplotlib inline

# Import MINST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
```

    Extracting /tmp/data/train-images-idx3-ubyte.gz
    Extracting /tmp/data/train-labels-idx1-ubyte.gz
    Extracting /tmp/data/t10k-images-idx3-ubyte.gz
    Extracting /tmp/data/t10k-labels-idx1-ubyte.gz

```python
# Input to the graph -- Tensorflow's MNIST images are (1, 784) vectors
n_pixels = 28*28
X = tf.placeholder(tf.float32, shape=([None, n_pixels]))
```

```python
#layer creation functions
def weight_variable(shape, name):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial, name=name)

def bias_variable(shape, name):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial, name=name)

def FC_layer(X, W, b):
    return tf.matmul(X, W) + b
```

## Encoder
The encoder consists of a 2 layer, fully connected feed forward network that reduces the dimensionality from the original number of features (e.g. pixel) to the dimensionality of the latent space. This network has two outputs -- the mean and (log) standard deviation of a gaussian distribution.

```python
# encoder
latent_dim = 20
h_dim = 500

W_enc = weight_variable([n_pixels, h_dim], 'W_enc')
b_enc = bias_variable([h_dim], 'b_enc')
# tanh activation function to replicate original model
h_enc = tf.nn.tanh(FC_layer(X, W_enc, b_enc))

W_mu = weight_variable([h_dim, latent_dim], 'W_mu')
b_mu = bias_variable([latent_dim], 'b_mu')
mu = FC_layer(h_enc, W_mu, b_mu)

W_logstd = weight_variable([h_dim, latent_dim], 'W_logstd')
b_logstd = bias_variable([latent_dim], 'b_logstd')
logstd = FC_layer(h_enc, W_logstd, b_logstd)

# reparameterization trick
noise = tf.random_normal([1, latent_dim])
z = mu + tf.mul(noise, tf.exp(.5*logstd))
```

## Decoder

The decoder (or, in the language of LFADS, the 'generator') is also a feedforward, fully connected network -- however, it goes from the dimensionality of the latent space back to the original dimensionality of the data (e.g. pixels). The output of the network is squashed between 0 and 1 with a sigmoid function.

```python
# decoder
W_dec = weight_variable([latent_dim, h_dim], 'W_dec')
b_dec = bias_variable([h_dim], 'b_dec')
h_dec = tf.nn.tanh(FC_layer(z, W_dec, b_dec))

W_reconstruct = weight_variable([h_dim, n_pixels], 'W_reconstruct')
b_reconstruct = bias_variable([n_pixels], 'b_reconstruct')
reconstruction = tf.nn.sigmoid(FC_layer(h_dec, W_reconstruct, b_reconstruct))
```

## Loss function
Each of these outputs is taken to be the mean of a Bernoulli distribution (in this example, a Bernoulli distribution is appropriate because our data is binary). The variational lower bound is given by:

\begin{equation}
\mathcal{L} = \mathbb{E}_{z\sim Q(z|X)}\log P(X|z) - D(Q(z|X)||P(z))
\end{equation}

When $P$ is a Bernouli distribution, the log likelihood is given by

\begin{equation}
\log P(X|z) = \sum_i^N X^{(i)}\log y^{(i)} + (1 − X^{(i)}) \cdot \log(1 − y^{(i)})
\end{equation}

where $N$ is the number of training samples (in our case, the batchsize), and $y^{(i)}$ is the reconstruction from the latent code $z^{(i)}$. The KL divergence between a gaussian $Q$ with mean $\mu$ and standard deviation $\sigma$ and a standard normal distribution $P$ is given by:

\begin{equation}
D(Q||P) = -\frac{1}{2}\sum_j^J \big(1 + \log((\sigma_j)^2) - (\mu_j)^2 - (\sigma_j)^2\big)
\end{equation}

We want to maximize this lower bound, but because tensorflow doesn't have a 'maximizing' optimizer, we minimize the negative lower bound.

```python
# variational lower bound

# add epsilon to log to prevent numerical overflow
log_likelihood = tf.reduce_sum(X*tf.log(reconstruction + 1e-9)+(1 - X)*tf.log(1 - reconstruction + 1e-9), reduction_indices=1)
KL_term = -.5*tf.reduce_sum(1 + 2*logstd - tf.pow(mu,2) - tf.exp(2*logstd), reduction_indices=1)
variational_lower_bound = tf.reduce_mean(log_likelihood - KL_term)
optimizer = tf.train.AdadeltaOptimizer().minimize(-variational_lower_bound)
```

## Training

Now all we have to do is run the optimizer until convergence.

```python
init = tf.global_variables_initializer()
sess = tf.InteractiveSession()
sess.run(init)
saver = tf.train.Saver()
```

```python
import time

num_iterations = 1000000
recording_interval = 1000
variational_lower_bound_array = []
log_likelihood_array = []
KL_term_array = []
iteration_array = [i*recording_interval for i in range(num_iterations/recording_interval)]
for i in range(num_iterations):
    # np.round to make MNIST binary
    x_batch = np.round(mnist.train.next_batch(200)[0])
    sess.run(optimizer, feed_dict={X: x_batch})
    if (i%recording_interval == 0):
        vlb_eval = variational_lower_bound.eval(feed_dict={X: x_batch})
        print "Iteration: {}, Loss: {}".format(i, vlb_eval)
        variational_lower_bound_array.append(vlb_eval)
        log_likelihood_array.append(np.mean(log_likelihood.eval(feed_dict={X: x_batch})))
        KL_term_array.append(np.mean(KL_term.eval(feed_dict={X: x_batch})))
```

    Iteration: 0, Loss: -236.021286011
    Iteration: 1000, Loss: -228.880111694
    Iteration: 2000, Loss: -235.156402588
    Iteration: 3000, Loss: -235.000747681
    Iteration: 4000, Loss: -230.577148438
    Iteration: 5000, Loss: -236.92590332
    Iteration: 6000, Loss: -227.564880371
    Iteration: 7000, Loss: -230.312072754
    Iteration: 8000, Loss: -238.945617676
    Iteration: 9000, Loss: -237.638305664
    Iteration: 10000, Loss: -231.152618408
    Iteration: 11000, Loss: -226.153518677
    Iteration: 12000, Loss: -219.754104614
    Iteration: 13000, Loss: -224.015426636
    Iteration: 14000, Loss: -228.482345581
    Iteration: 15000, Loss: -227.824279785
    Iteration: 16000, Loss: -221.535858154
    Iteration: 17000, Loss: -231.035705566
    Iteration: 18000, Loss: -228.021835327
    Iteration: 19000, Loss: -225.699645996
    Iteration: 20000, Loss: -220.126724243
    Iteration: 21000, Loss: -216.616394043
    Iteration: 22000, Loss: -219.514373779
    Iteration: 23000, Loss: -234.833404541
    Iteration: 24000, Loss: -225.963150024
    Iteration: 25000, Loss: -216.578323364
    Iteration: 26000, Loss: -217.122848511
    Iteration: 27000, Loss: -223.460784912
    Iteration: 28000, Loss: -217.166305542
    Iteration: 29000, Loss: -217.884368896
    Iteration: 30000, Loss: -221.240707397
    Iteration: 31000, Loss: -212.645935059
    Iteration: 32000, Loss: -242.414047241
    Iteration: 33000, Loss: -211.540161133
    Iteration: 34000, Loss: -216.882659912
    Iteration: 35000, Loss: -220.900268555
    Iteration: 36000, Loss: -202.70375061
    Iteration: 37000, Loss: -220.08380127
    Iteration: 38000, Loss: -208.753860474
    Iteration: 39000, Loss: -227.473709106
    Iteration: 40000, Loss: -215.147506714
    Iteration: 41000, Loss: -209.193054199
    Iteration: 42000, Loss: -204.739517212
    Iteration: 43000, Loss: -198.226715088
    Iteration: 44000, Loss: -207.98298645
    Iteration: 45000, Loss: -208.349105835
    Iteration: 46000, Loss: -203.79989624
    Iteration: 47000, Loss: -200.346908569
    Iteration: 48000, Loss: -210.243438721
    Iteration: 49000, Loss: -204.861450195
    Iteration: 50000, Loss: -207.819122314
    Iteration: 51000, Loss: -202.917144775
    Iteration: 52000, Loss: -204.893478394
    Iteration: 53000, Loss: -205.275665283
    Iteration: 54000, Loss: -204.433395386
    Iteration: 55000, Loss: -212.416687012
    Iteration: 56000, Loss: -220.444595337
    Iteration: 57000, Loss: -209.4140625
    Iteration: 58000, Loss: -198.180755615
    Iteration: 59000, Loss: -205.678344727
    Iteration: 60000, Loss: -200.487716675
    Iteration: 61000, Loss: -203.813552856
    Iteration: 62000, Loss: -205.825271606
    Iteration: 63000, Loss: -196.225662231
    Iteration: 64000, Loss: -195.907699585
    Iteration: 65000, Loss: -195.679412842
    Iteration: 66000, Loss: -220.437271118
    Iteration: 67000, Loss: -202.704528809
    Iteration: 68000, Loss: -198.834213257
    Iteration: 69000, Loss: -198.511993408
    Iteration: 70000, Loss: -199.174545288
    Iteration: 71000, Loss: -195.442230225
    Iteration: 72000, Loss: -192.387405396
    Iteration: 73000, Loss: -212.718276978
    Iteration: 74000, Loss: -198.515975952
    Iteration: 75000, Loss: -202.121353149
    Iteration: 76000, Loss: -220.6065979
    Iteration: 77000, Loss: -196.368392944
    Iteration: 78000, Loss: -204.231216431
    Iteration: 79000, Loss: -189.056213379
    Iteration: 80000, Loss: -196.439102173
    Iteration: 81000, Loss: -202.766036987
    Iteration: 82000, Loss: -202.511627197
    Iteration: 83000, Loss: -189.786087036
    Iteration: 84000, Loss: -204.392837524
    Iteration: 85000, Loss: -199.484329224
    Iteration: 86000, Loss: -189.532012939
    Iteration: 87000, Loss: -189.644256592
    Iteration: 88000, Loss: -195.699371338
    Iteration: 89000, Loss: -200.278945923
    Iteration: 90000, Loss: -194.341400146
    Iteration: 91000, Loss: -199.346496582
    Iteration: 92000, Loss: -198.6199646
    Iteration: 93000, Loss: -185.240875244
    Iteration: 94000, Loss: -193.480758667
    Iteration: 95000, Loss: -188.84487915
    Iteration: 96000, Loss: -195.056869507
    Iteration: 97000, Loss: -204.442581177
    Iteration: 98000, Loss: -195.432266235
    Iteration: 99000, Loss: -192.604217529
    Iteration: 100000, Loss: -182.076644897
    Iteration: 101000, Loss: -190.119491577
    Iteration: 102000, Loss: -189.038864136
    Iteration: 103000, Loss: -186.053039551
    Iteration: 104000, Loss: -191.120697021
    Iteration: 105000, Loss: -188.928894043
    Iteration: 106000, Loss: -186.920150757
    Iteration: 107000, Loss: -185.62109375
    Iteration: 108000, Loss: -187.139144897
    Iteration: 109000, Loss: -186.279800415
    Iteration: 110000, Loss: -189.193984985
    Iteration: 111000, Loss: -189.851577759
    Iteration: 112000, Loss: -185.088577271
    Iteration: 113000, Loss: -189.189178467
    Iteration: 114000, Loss: -188.126281738
    Iteration: 115000, Loss: -192.887695312
    Iteration: 116000, Loss: -185.175201416
    Iteration: 117000, Loss: -184.4427948
    Iteration: 118000, Loss: -180.579528809
    Iteration: 119000, Loss: -186.242889404
    Iteration: 120000, Loss: -177.50340271
    Iteration: 121000, Loss: -185.939147949
    Iteration: 122000, Loss: -184.56401062
    Iteration: 123000, Loss: -184.196655273
    Iteration: 124000, Loss: -187.385055542
    Iteration: 125000, Loss: -189.074264526
    Iteration: 126000, Loss: -180.7474823
    Iteration: 127000, Loss: -190.431838989
    Iteration: 128000, Loss: -185.281784058
    Iteration: 129000, Loss: -179.851257324
    Iteration: 130000, Loss: -189.100997925
    Iteration: 131000, Loss: -189.13722229
    Iteration: 132000, Loss: -187.510665894
    Iteration: 133000, Loss: -180.29737854
    Iteration: 134000, Loss: -182.449356079
    Iteration: 135000, Loss: -182.865661621
    Iteration: 136000, Loss: -187.106399536
    Iteration: 137000, Loss: -187.364379883
    Iteration: 138000, Loss: -184.307525635
    Iteration: 139000, Loss: -189.24230957
    Iteration: 140000, Loss: -180.742904663
    Iteration: 141000, Loss: -183.645629883
    Iteration: 142000, Loss: -176.794036865
    Iteration: 143000, Loss: -199.298904419
    Iteration: 144000, Loss: -186.420944214
    Iteration: 145000, Loss: -218.034393311
    Iteration: 146000, Loss: -191.996353149
    Iteration: 147000, Loss: -191.751525879
    Iteration: 148000, Loss: -178.02746582
    Iteration: 149000, Loss: -175.689880371
    Iteration: 150000, Loss: -184.013870239
    Iteration: 151000, Loss: -182.786437988
    Iteration: 152000, Loss: -179.503540039
    Iteration: 153000, Loss: -182.686920166
    Iteration: 154000, Loss: -177.161178589
    Iteration: 155000, Loss: -172.481216431
    Iteration: 156000, Loss: -180.721954346
    Iteration: 157000, Loss: -181.91456604
    Iteration: 158000, Loss: -178.498977661
    Iteration: 159000, Loss: -174.012542725
    Iteration: 160000, Loss: -183.912017822
    Iteration: 161000, Loss: -175.032424927
    Iteration: 162000, Loss: -185.220474243
    Iteration: 163000, Loss: -193.015548706
    Iteration: 164000, Loss: -177.642730713
    Iteration: 165000, Loss: -174.236450195
    Iteration: 166000, Loss: -171.317855835
    Iteration: 167000, Loss: -182.590423584
    Iteration: 168000, Loss: -181.093597412
    Iteration: 169000, Loss: -178.105621338
    Iteration: 170000, Loss: -180.706054688
    Iteration: 171000, Loss: -179.968002319
    Iteration: 172000, Loss: -179.495376587
    Iteration: 173000, Loss: -175.678619385
    Iteration: 174000, Loss: -177.510314941
    Iteration: 175000, Loss: -171.490844727
    Iteration: 176000, Loss: -181.779800415
    Iteration: 177000, Loss: -172.266723633
    Iteration: 178000, Loss: -179.133712769
    Iteration: 179000, Loss: -188.464614868
    Iteration: 180000, Loss: -181.364105225
    Iteration: 181000, Loss: -175.886566162
    Iteration: 182000, Loss: -183.973205566
    Iteration: 183000, Loss: -176.636230469
    Iteration: 184000, Loss: -169.696914673
    Iteration: 185000, Loss: -186.874557495
    Iteration: 186000, Loss: -172.759994507
    Iteration: 187000, Loss: -188.645645142
    Iteration: 188000, Loss: -173.99987793
    Iteration: 189000, Loss: -172.637405396
    Iteration: 190000, Loss: -175.245880127
    Iteration: 191000, Loss: -180.690795898
    Iteration: 192000, Loss: -181.250640869
    Iteration: 193000, Loss: -173.36630249
    Iteration: 194000, Loss: -179.705703735
    Iteration: 195000, Loss: -173.535461426
    Iteration: 196000, Loss: -182.359039307
    Iteration: 197000, Loss: -169.847457886
    Iteration: 198000, Loss: -177.224960327
    Iteration: 199000, Loss: -177.756835938
    Iteration: 200000, Loss: -175.952545166
    Iteration: 201000, Loss: -175.163864136
    Iteration: 202000, Loss: -172.848968506
    Iteration: 203000, Loss: -183.420318604
    Iteration: 204000, Loss: -170.552688599
    Iteration: 205000, Loss: -174.575775146
    Iteration: 206000, Loss: -182.346252441
    Iteration: 207000, Loss: -180.100357056
    Iteration: 208000, Loss: -168.794876099
    Iteration: 209000, Loss: -168.349807739
    Iteration: 210000, Loss: -166.392272949
    Iteration: 211000, Loss: -175.96182251
    Iteration: 212000, Loss: -172.770431519
    Iteration: 213000, Loss: -178.340698242
    Iteration: 214000, Loss: -168.032806396
    Iteration: 215000, Loss: -169.732711792
    Iteration: 216000, Loss: -167.862289429
    Iteration: 217000, Loss: -181.459655762
    Iteration: 218000, Loss: -173.550689697
    Iteration: 219000, Loss: -173.065246582
    Iteration: 220000, Loss: -176.590637207
    Iteration: 221000, Loss: -168.587966919
    Iteration: 222000, Loss: -175.176559448
    Iteration: 223000, Loss: -169.514801025
    Iteration: 224000, Loss: -199.203460693
    Iteration: 225000, Loss: -177.871292114
    Iteration: 226000, Loss: -163.021514893
    Iteration: 227000, Loss: -176.662582397
    Iteration: 228000, Loss: -169.702560425
    Iteration: 229000, Loss: -163.791168213
    Iteration: 230000, Loss: -171.882858276
    Iteration: 231000, Loss: -180.08253479
    Iteration: 232000, Loss: -172.572341919
    Iteration: 233000, Loss: -174.161087036
    Iteration: 234000, Loss: -180.927459717
    Iteration: 235000, Loss: -169.681762695
    Iteration: 236000, Loss: -172.722473145
    Iteration: 237000, Loss: -166.358840942
    Iteration: 238000, Loss: -170.134216309
    Iteration: 239000, Loss: -166.707702637
    Iteration: 240000, Loss: -163.115524292
    Iteration: 241000, Loss: -169.428573608
    Iteration: 242000, Loss: -172.255859375
    Iteration: 243000, Loss: -173.313323975
    Iteration: 244000, Loss: -166.48576355
    Iteration: 245000, Loss: -175.369995117
    Iteration: 246000, Loss: -172.76449585
    Iteration: 247000, Loss: -174.375671387
    Iteration: 248000, Loss: -177.962341309
    Iteration: 249000, Loss: -171.259719849
    Iteration: 250000, Loss: -165.710540771
    Iteration: 251000, Loss: -175.045822144
    Iteration: 252000, Loss: -174.992004395
    Iteration: 253000, Loss: -169.415542603
    Iteration: 254000, Loss: -164.336517334
    Iteration: 255000, Loss: -168.566741943
    Iteration: 256000, Loss: -164.665939331
    Iteration: 257000, Loss: -161.151489258
    Iteration: 258000, Loss: -169.608047485
    Iteration: 259000, Loss: -164.929901123
    Iteration: 260000, Loss: -168.759536743
    Iteration: 261000, Loss: -173.431106567
    Iteration: 262000, Loss: -177.726287842
    Iteration: 263000, Loss: -156.301254272
    Iteration: 264000, Loss: -172.316635132
    Iteration: 265000, Loss: -169.722427368
    Iteration: 266000, Loss: -188.117340088
    Iteration: 267000, Loss: -166.409255981
    Iteration: 268000, Loss: -162.617492676
    Iteration: 269000, Loss: -170.782318115
    Iteration: 270000, Loss: -164.532348633
    Iteration: 271000, Loss: -177.301971436
    Iteration: 272000, Loss: -169.4556427
    Iteration: 273000, Loss: -174.002975464
    Iteration: 274000, Loss: -177.033050537
    Iteration: 275000, Loss: -166.791244507
    Iteration: 276000, Loss: -170.484725952
    Iteration: 277000, Loss: -171.115432739
    Iteration: 278000, Loss: -163.199661255
    Iteration: 279000, Loss: -170.629547119
    Iteration: 280000, Loss: -169.100143433
    Iteration: 281000, Loss: -178.554870605
    Iteration: 282000, Loss: -168.390182495
    Iteration: 283000, Loss: -163.790542603
    Iteration: 284000, Loss: -157.587844849
    Iteration: 285000, Loss: -175.275436401
    Iteration: 286000, Loss: -164.608673096
    Iteration: 287000, Loss: -162.272125244
    Iteration: 288000, Loss: -167.818481445
    Iteration: 289000, Loss: -183.408325195
    Iteration: 290000, Loss: -172.568710327
    Iteration: 291000, Loss: -157.378890991
    Iteration: 292000, Loss: -174.361663818
    Iteration: 293000, Loss: -198.756164551
    Iteration: 294000, Loss: -164.908981323
    Iteration: 295000, Loss: -161.995254517
    Iteration: 296000, Loss: -162.503173828
    Iteration: 297000, Loss: -170.80317688
    Iteration: 298000, Loss: -174.51109314
    Iteration: 299000, Loss: -170.940505981
    Iteration: 300000, Loss: -172.896438599
    Iteration: 301000, Loss: -169.921920776
    Iteration: 302000, Loss: -175.123840332
    Iteration: 303000, Loss: -162.281555176
    Iteration: 304000, Loss: -167.715835571
    Iteration: 305000, Loss: -167.96496582
    Iteration: 306000, Loss: -163.975234985
    Iteration: 307000, Loss: -163.394760132
    Iteration: 308000, Loss: -164.379257202
    Iteration: 309000, Loss: -177.131958008
    Iteration: 310000, Loss: -162.724899292
    Iteration: 311000, Loss: -160.695648193
    Iteration: 312000, Loss: -160.734588623
    Iteration: 313000, Loss: -164.382614136
    Iteration: 314000, Loss: -180.780715942
    Iteration: 315000, Loss: -165.864959717
    Iteration: 316000, Loss: -161.223556519
    Iteration: 317000, Loss: -171.1978302
    Iteration: 318000, Loss: -163.312637329
    Iteration: 319000, Loss: -171.532501221
    Iteration: 320000, Loss: -166.906738281
    Iteration: 321000, Loss: -160.988693237
    Iteration: 322000, Loss: -168.3777771
    Iteration: 323000, Loss: -178.132064819
    Iteration: 324000, Loss: -163.214645386
    Iteration: 325000, Loss: -163.001022339
    Iteration: 326000, Loss: -218.190551758
    Iteration: 327000, Loss: -157.933746338
    Iteration: 328000, Loss: -163.072479248
    Iteration: 329000, Loss: -168.062149048
    Iteration: 330000, Loss: -170.97479248
    Iteration: 331000, Loss: -166.675704956
    Iteration: 332000, Loss: -165.622680664
    Iteration: 333000, Loss: -159.438201904
    Iteration: 334000, Loss: -165.312683105
    Iteration: 335000, Loss: -164.377700806
    Iteration: 336000, Loss: -160.813659668
    Iteration: 337000, Loss: -187.451873779
    Iteration: 338000, Loss: -156.284591675
    Iteration: 339000, Loss: -181.505813599
    Iteration: 340000, Loss: -184.255554199
    Iteration: 341000, Loss: -169.659667969
    Iteration: 342000, Loss: -172.571655273
    Iteration: 343000, Loss: -159.021697998
    Iteration: 344000, Loss: -160.261489868
    Iteration: 345000, Loss: -172.653671265
    Iteration: 346000, Loss: -167.980194092
    Iteration: 347000, Loss: -172.082885742
    Iteration: 348000, Loss: -161.465072632
    Iteration: 349000, Loss: -159.984100342
    Iteration: 350000, Loss: -160.612121582
    Iteration: 351000, Loss: -164.900344849
    Iteration: 352000, Loss: -160.30480957
    Iteration: 353000, Loss: -170.343032837
    Iteration: 354000, Loss: -158.680038452
    Iteration: 355000, Loss: -229.186920166
    Iteration: 356000, Loss: -165.313522339
    Iteration: 357000, Loss: -163.481277466
    Iteration: 358000, Loss: -164.138870239
    Iteration: 359000, Loss: -162.621307373
    Iteration: 360000, Loss: -164.074569702
    Iteration: 361000, Loss: -169.312866211
    Iteration: 362000, Loss: -163.684875488
    Iteration: 363000, Loss: -163.428115845
    Iteration: 364000, Loss: -160.263244629
    Iteration: 365000, Loss: -161.53225708
    Iteration: 366000, Loss: -157.955291748
    Iteration: 367000, Loss: -154.395553589
    Iteration: 368000, Loss: -158.891342163
    Iteration: 369000, Loss: -171.298431396
    Iteration: 370000, Loss: -164.375762939
    Iteration: 371000, Loss: -163.858001709
    Iteration: 372000, Loss: -176.622512817
    Iteration: 373000, Loss: -163.304153442
    Iteration: 374000, Loss: -173.410507202
    Iteration: 375000, Loss: -163.544372559
    Iteration: 376000, Loss: -163.857681274
    Iteration: 377000, Loss: -168.953063965
    Iteration: 378000, Loss: -171.82598877
    Iteration: 379000, Loss: -160.774658203
    Iteration: 380000, Loss: -157.141159058
    Iteration: 381000, Loss: -161.560943604
    Iteration: 382000, Loss: -152.234817505
    Iteration: 383000, Loss: -188.081329346
    Iteration: 384000, Loss: -168.201080322
    Iteration: 385000, Loss: -160.416275024
    Iteration: 386000, Loss: -190.798080444
    Iteration: 387000, Loss: -160.832077026
    Iteration: 388000, Loss: -171.299758911
    Iteration: 389000, Loss: -159.353881836
    Iteration: 390000, Loss: -158.592788696
    Iteration: 391000, Loss: -159.38142395
    Iteration: 392000, Loss: -158.764572144
    Iteration: 393000, Loss: -165.863540649
    Iteration: 394000, Loss: -155.419128418
    Iteration: 395000, Loss: -166.695388794
    Iteration: 396000, Loss: -175.33454895
    Iteration: 397000, Loss: -162.092437744
    Iteration: 398000, Loss: -170.208999634
    Iteration: 399000, Loss: -150.355026245
    Iteration: 400000, Loss: -156.847457886
    Iteration: 401000, Loss: -169.646682739
    Iteration: 402000, Loss: -163.680435181
    Iteration: 403000, Loss: -175.367874146
    Iteration: 404000, Loss: -167.915435791
    Iteration: 405000, Loss: -164.193313599
    Iteration: 406000, Loss: -170.459777832
    Iteration: 407000, Loss: -165.536575317
    Iteration: 408000, Loss: -156.695083618
    Iteration: 409000, Loss: -162.606628418
    Iteration: 410000, Loss: -160.307556152
    Iteration: 411000, Loss: -161.909011841
    Iteration: 412000, Loss: -173.334564209
    Iteration: 413000, Loss: -168.806915283
    Iteration: 414000, Loss: -153.981231689
    Iteration: 415000, Loss: -152.154296875
    Iteration: 416000, Loss: -163.335754395
    Iteration: 417000, Loss: -160.074401855
    Iteration: 418000, Loss: -158.083190918
    Iteration: 419000, Loss: -158.576477051
    Iteration: 420000, Loss: -158.528152466
    Iteration: 421000, Loss: -158.928024292
    Iteration: 422000, Loss: -157.182693481
    Iteration: 423000, Loss: -168.757324219
    Iteration: 424000, Loss: -157.035980225
    Iteration: 425000, Loss: -174.760192871
    Iteration: 426000, Loss: -173.545257568
    Iteration: 427000, Loss: -161.796264648
    Iteration: 428000, Loss: -170.394897461
    Iteration: 429000, Loss: -161.941436768
    Iteration: 430000, Loss: -157.093734741
    Iteration: 431000, Loss: -166.120819092
    Iteration: 432000, Loss: -160.572174072
    Iteration: 433000, Loss: -161.377532959
    Iteration: 434000, Loss: -162.962677002
    Iteration: 435000, Loss: -159.579544067
    Iteration: 436000, Loss: -156.669692993
    Iteration: 437000, Loss: -154.093704224
    Iteration: 438000, Loss: -153.018814087
    Iteration: 439000, Loss: -157.225219727
    Iteration: 440000, Loss: -165.119903564
    Iteration: 441000, Loss: -151.661254883
    Iteration: 442000, Loss: -152.176727295
    Iteration: 443000, Loss: -163.76512146
    Iteration: 444000, Loss: -164.371673584
    Iteration: 445000, Loss: -156.759643555
    Iteration: 446000, Loss: -172.660705566
    Iteration: 447000, Loss: -163.337463379
    Iteration: 448000, Loss: -159.044067383
    Iteration: 449000, Loss: -160.326004028
    Iteration: 450000, Loss: -163.958053589
    Iteration: 451000, Loss: -154.034683228
    Iteration: 452000, Loss: -154.602554321
    Iteration: 453000, Loss: -173.108276367
    Iteration: 454000, Loss: -161.521438599
    Iteration: 455000, Loss: -158.613708496
    Iteration: 456000, Loss: -156.743179321
    Iteration: 457000, Loss: -157.502456665
    Iteration: 458000, Loss: -160.106506348
    Iteration: 459000, Loss: -158.456314087
    Iteration: 460000, Loss: -159.831192017
    Iteration: 461000, Loss: -159.723175049
    Iteration: 462000, Loss: -157.022201538
    Iteration: 463000, Loss: -163.718978882
    Iteration: 464000, Loss: -164.265213013
    Iteration: 465000, Loss: -152.996948242
    Iteration: 466000, Loss: -147.932647705
    Iteration: 467000, Loss: -175.942520142
    Iteration: 468000, Loss: -182.528518677
    Iteration: 469000, Loss: -160.60786438
    Iteration: 470000, Loss: -160.703491211
    Iteration: 471000, Loss: -153.200683594
    Iteration: 472000, Loss: -157.901077271
    Iteration: 473000, Loss: -156.578414917
    Iteration: 474000, Loss: -172.274765015
    Iteration: 475000, Loss: -182.793029785
    Iteration: 476000, Loss: -169.206008911
    Iteration: 477000, Loss: -178.915435791
    Iteration: 478000, Loss: -152.795684814
    Iteration: 479000, Loss: -158.936584473
    Iteration: 480000, Loss: -149.159606934
    Iteration: 481000, Loss: -157.416763306
    Iteration: 482000, Loss: -152.075378418
    Iteration: 483000, Loss: -159.699584961
    Iteration: 484000, Loss: -162.921585083
    Iteration: 485000, Loss: -170.381958008
    Iteration: 486000, Loss: -158.157089233
    Iteration: 487000, Loss: -157.64982605
    Iteration: 488000, Loss: -166.927780151
    Iteration: 489000, Loss: -161.141647339
    Iteration: 490000, Loss: -151.547973633
    Iteration: 491000, Loss: -154.205886841
    Iteration: 492000, Loss: -165.30418396
    Iteration: 493000, Loss: -154.367889404
    Iteration: 494000, Loss: -156.024276733
    Iteration: 495000, Loss: -160.598464966
    Iteration: 496000, Loss: -154.043167114
    Iteration: 497000, Loss: -161.322875977
    Iteration: 498000, Loss: -201.847702026
    Iteration: 499000, Loss: -155.593109131
    Iteration: 500000, Loss: -161.364074707
    Iteration: 501000, Loss: -156.575805664
    Iteration: 502000, Loss: -156.682327271
    Iteration: 503000, Loss: -161.463577271
    Iteration: 504000, Loss: -148.794235229
    Iteration: 505000, Loss: -149.211975098
    Iteration: 506000, Loss: -144.11138916
    Iteration: 507000, Loss: -155.108337402
    Iteration: 508000, Loss: -169.36340332
    Iteration: 509000, Loss: -154.731399536
    Iteration: 510000, Loss: -152.370605469
    Iteration: 511000, Loss: -156.313980103
    Iteration: 512000, Loss: -156.516220093
    Iteration: 513000, Loss: -162.943557739
    Iteration: 514000, Loss: -153.741210938
    Iteration: 515000, Loss: -163.678344727
    Iteration: 516000, Loss: -157.990356445
    Iteration: 517000, Loss: -164.466125488
    Iteration: 518000, Loss: -162.805557251
    Iteration: 519000, Loss: -164.933807373
    Iteration: 520000, Loss: -160.030014038
    Iteration: 521000, Loss: -151.902923584
    Iteration: 522000, Loss: -151.538925171
    Iteration: 523000, Loss: -149.927581787
    Iteration: 524000, Loss: -154.142150879
    Iteration: 525000, Loss: -151.724334717
    Iteration: 526000, Loss: -164.16809082
    Iteration: 527000, Loss: -163.522262573
    Iteration: 528000, Loss: -153.342697144
    Iteration: 529000, Loss: -162.999237061
    Iteration: 530000, Loss: -168.147506714
    Iteration: 531000, Loss: -166.800018311
    Iteration: 532000, Loss: -158.848342896
    Iteration: 533000, Loss: -153.824203491
    Iteration: 534000, Loss: -148.949066162
    Iteration: 535000, Loss: -160.525161743
    Iteration: 536000, Loss: -151.005004883
    Iteration: 537000, Loss: -157.421112061
    Iteration: 538000, Loss: -153.527618408
    Iteration: 539000, Loss: -161.949874878
    Iteration: 540000, Loss: -162.374511719
    Iteration: 541000, Loss: -151.990234375
    Iteration: 542000, Loss: -157.335220337
    Iteration: 543000, Loss: -170.001907349
    Iteration: 544000, Loss: -152.152893066
    Iteration: 545000, Loss: -152.402770996
    Iteration: 546000, Loss: -153.16986084
    Iteration: 547000, Loss: -155.396759033
    Iteration: 548000, Loss: -150.83140564
    Iteration: 549000, Loss: -154.265029907
    Iteration: 550000, Loss: -161.725311279
    Iteration: 551000, Loss: -159.430755615
    Iteration: 552000, Loss: -150.29284668
    Iteration: 553000, Loss: -146.136795044
    Iteration: 554000, Loss: -151.904129028
    Iteration: 555000, Loss: -156.607177734
    Iteration: 556000, Loss: -160.922363281
    Iteration: 557000, Loss: -153.608551025
    Iteration: 558000, Loss: -174.731750488
    Iteration: 559000, Loss: -152.338119507
    Iteration: 560000, Loss: -145.726608276
    Iteration: 561000, Loss: -168.142028809
    Iteration: 562000, Loss: -164.191329956
    Iteration: 563000, Loss: -159.908126831
    Iteration: 564000, Loss: -155.255630493
    Iteration: 565000, Loss: -156.436935425
    Iteration: 566000, Loss: -151.375640869
    Iteration: 567000, Loss: -157.900878906
    Iteration: 568000, Loss: -162.380508423
    Iteration: 569000, Loss: -159.353927612
    Iteration: 570000, Loss: -162.451065063
    Iteration: 571000, Loss: -154.206192017
    Iteration: 572000, Loss: -161.017166138
    Iteration: 573000, Loss: -147.301803589
    Iteration: 574000, Loss: -147.123718262
    Iteration: 575000, Loss: -145.63684082
    Iteration: 576000, Loss: -166.028823853
    Iteration: 577000, Loss: -163.440109253
    Iteration: 578000, Loss: -159.346191406
    Iteration: 579000, Loss: -158.894775391
    Iteration: 580000, Loss: -163.842071533
    Iteration: 581000, Loss: -174.884887695
    Iteration: 582000, Loss: -160.641479492
    Iteration: 583000, Loss: -147.774688721
    Iteration: 584000, Loss: -150.78604126
    Iteration: 585000, Loss: -149.963500977
    Iteration: 586000, Loss: -176.566879272
    Iteration: 587000, Loss: -166.01171875
    Iteration: 588000, Loss: -151.864700317
    Iteration: 589000, Loss: -159.831939697
    Iteration: 590000, Loss: -158.850891113
    Iteration: 591000, Loss: -155.02230835
    Iteration: 592000, Loss: -151.378234863
    Iteration: 593000, Loss: -151.568969727
    Iteration: 594000, Loss: -151.211471558
    Iteration: 595000, Loss: -159.242935181
    Iteration: 596000, Loss: -152.682296753
    Iteration: 597000, Loss: -151.016799927
    Iteration: 598000, Loss: -166.221984863
    Iteration: 599000, Loss: -157.533508301
    Iteration: 600000, Loss: -169.496841431
    Iteration: 601000, Loss: -158.484527588
    Iteration: 602000, Loss: -148.859710693
    Iteration: 603000, Loss: -153.101531982
    Iteration: 604000, Loss: -165.974136353
    Iteration: 605000, Loss: -152.677032471
    Iteration: 606000, Loss: -153.476226807
    Iteration: 607000, Loss: -152.315307617
    Iteration: 608000, Loss: -153.445785522
    Iteration: 609000, Loss: -162.700668335
    Iteration: 610000, Loss: -151.251403809
    Iteration: 611000, Loss: -156.505630493
    Iteration: 612000, Loss: -165.132034302
    Iteration: 613000, Loss: -152.775939941
    Iteration: 614000, Loss: -160.29095459
    Iteration: 615000, Loss: -158.667816162
    Iteration: 616000, Loss: -152.691696167
    Iteration: 617000, Loss: -151.704879761
    Iteration: 618000, Loss: -153.869888306
    Iteration: 619000, Loss: -154.6459198
    Iteration: 620000, Loss: -155.409210205
    Iteration: 621000, Loss: -151.470062256
    Iteration: 622000, Loss: -186.603668213
    Iteration: 623000, Loss: -156.945495605
    Iteration: 624000, Loss: -150.554458618
    Iteration: 625000, Loss: -148.784973145
    Iteration: 626000, Loss: -150.238037109
    Iteration: 627000, Loss: -146.731506348
    Iteration: 628000, Loss: -146.975097656
    Iteration: 629000, Loss: -157.065841675
    Iteration: 630000, Loss: -153.777816772
    Iteration: 631000, Loss: -151.238616943
    Iteration: 632000, Loss: -156.614929199
    Iteration: 633000, Loss: -153.090072632
    Iteration: 634000, Loss: -159.105209351
    Iteration: 635000, Loss: -166.521499634
    Iteration: 636000, Loss: -152.35848999
    Iteration: 637000, Loss: -151.129486084
    Iteration: 638000, Loss: -156.774185181
    Iteration: 639000, Loss: -148.867584229
    Iteration: 640000, Loss: -153.623397827
    Iteration: 641000, Loss: -160.182144165
    Iteration: 642000, Loss: -148.420257568
    Iteration: 643000, Loss: -163.435058594
    Iteration: 644000, Loss: -178.789291382
    Iteration: 645000, Loss: -152.57270813
    Iteration: 646000, Loss: -150.077178955
    Iteration: 647000, Loss: -160.829528809
    Iteration: 648000, Loss: -150.262298584
    Iteration: 649000, Loss: -149.432128906
    Iteration: 650000, Loss: -157.909973145
    Iteration: 651000, Loss: -154.013549805
    Iteration: 652000, Loss: -145.444946289
    Iteration: 653000, Loss: -153.590515137
    Iteration: 654000, Loss: -145.424194336
    Iteration: 655000, Loss: -146.450149536
    Iteration: 656000, Loss: -165.439727783
    Iteration: 657000, Loss: -148.929977417
    Iteration: 658000, Loss: -149.716308594
    Iteration: 659000, Loss: -152.294784546
    Iteration: 660000, Loss: -154.596450806
    Iteration: 661000, Loss: -151.896774292
    Iteration: 662000, Loss: -157.590621948
    Iteration: 663000, Loss: -162.321105957
    Iteration: 664000, Loss: -151.600891113
    Iteration: 665000, Loss: -161.39100647
    Iteration: 666000, Loss: -158.127441406
    Iteration: 667000, Loss: -148.838867188
    Iteration: 668000, Loss: -155.246627808
    Iteration: 669000, Loss: -162.298141479
    Iteration: 670000, Loss: -145.699157715
    Iteration: 671000, Loss: -162.632965088
    Iteration: 672000, Loss: -149.930023193
    Iteration: 673000, Loss: -150.120376587
    Iteration: 674000, Loss: -146.853561401
    Iteration: 675000, Loss: -158.945922852
    Iteration: 676000, Loss: -154.4269104
    Iteration: 677000, Loss: -149.259292603
    Iteration: 678000, Loss: -151.892852783
    Iteration: 679000, Loss: -147.914764404
    Iteration: 680000, Loss: -152.964538574
    Iteration: 681000, Loss: -149.239868164
    Iteration: 682000, Loss: -153.501998901
    Iteration: 683000, Loss: -150.296264648
    Iteration: 684000, Loss: -155.388595581
    Iteration: 685000, Loss: -157.365356445
    Iteration: 686000, Loss: -157.298568726
    Iteration: 687000, Loss: -145.809402466
    Iteration: 688000, Loss: -179.030258179
    Iteration: 689000, Loss: -160.772720337
    Iteration: 690000, Loss: -149.172485352
    Iteration: 691000, Loss: -154.243286133
    Iteration: 692000, Loss: -148.385940552
    Iteration: 693000, Loss: -156.915405273
    Iteration: 694000, Loss: -147.197494507
    Iteration: 695000, Loss: -155.580078125
    Iteration: 696000, Loss: -150.266555786
    Iteration: 697000, Loss: -146.18371582
    Iteration: 698000, Loss: -152.974334717
    Iteration: 699000, Loss: -150.265991211
    Iteration: 700000, Loss: -167.026794434
    Iteration: 701000, Loss: -153.578155518
    Iteration: 702000, Loss: -149.062057495
    Iteration: 703000, Loss: -154.802322388
    Iteration: 704000, Loss: -170.832092285
    Iteration: 705000, Loss: -166.64793396
    Iteration: 706000, Loss: -144.421264648
    Iteration: 707000, Loss: -160.074661255
    Iteration: 708000, Loss: -149.488830566
    Iteration: 709000, Loss: -147.1484375
    Iteration: 710000, Loss: -150.195755005
    Iteration: 711000, Loss: -143.017745972
    Iteration: 712000, Loss: -154.678924561
    Iteration: 713000, Loss: -151.646774292
    Iteration: 714000, Loss: -169.368362427
    Iteration: 715000, Loss: -157.332305908
    Iteration: 716000, Loss: -150.058197021
    Iteration: 717000, Loss: -141.942596436
    Iteration: 718000, Loss: -155.949645996
    Iteration: 719000, Loss: -150.875228882
    Iteration: 720000, Loss: -152.601974487
    Iteration: 721000, Loss: -157.36164856
    Iteration: 722000, Loss: -147.167266846
    Iteration: 723000, Loss: -153.061157227
    Iteration: 724000, Loss: -151.253753662
    Iteration: 725000, Loss: -151.803146362
    Iteration: 726000, Loss: -144.672225952
    Iteration: 727000, Loss: -146.525177002
    Iteration: 728000, Loss: -147.158920288
    Iteration: 729000, Loss: -147.671615601
    Iteration: 730000, Loss: -155.850402832
    Iteration: 731000, Loss: -147.509078979
    Iteration: 732000, Loss: -145.999450684
    Iteration: 733000, Loss: -158.074768066
    Iteration: 734000, Loss: -148.963943481
    Iteration: 735000, Loss: -151.027420044
    Iteration: 736000, Loss: -151.010665894
    Iteration: 737000, Loss: -149.212692261
    Iteration: 738000, Loss: -155.307952881
    Iteration: 739000, Loss: -161.432922363
    Iteration: 740000, Loss: -144.558334351
    Iteration: 741000, Loss: -156.098770142
    Iteration: 742000, Loss: -160.876541138
    Iteration: 743000, Loss: -145.008712769
    Iteration: 744000, Loss: -149.090835571
    Iteration: 745000, Loss: -155.904434204
    Iteration: 746000, Loss: -156.216751099
    Iteration: 747000, Loss: -143.224273682
    Iteration: 748000, Loss: -152.384841919
    Iteration: 749000, Loss: -142.012634277
    Iteration: 750000, Loss: -153.504257202
    Iteration: 751000, Loss: -148.727279663
    Iteration: 752000, Loss: -151.201904297
    Iteration: 753000, Loss: -149.510437012
    Iteration: 754000, Loss: -154.190093994
    Iteration: 755000, Loss: -149.965393066
    Iteration: 756000, Loss: -157.894714355
    Iteration: 757000, Loss: -145.10269165
    Iteration: 758000, Loss: -150.522628784
    Iteration: 759000, Loss: -150.849533081
    Iteration: 760000, Loss: -155.127532959
    Iteration: 761000, Loss: -149.612319946
    Iteration: 762000, Loss: -156.34324646
    Iteration: 763000, Loss: -152.02973938
    Iteration: 764000, Loss: -148.738098145
    Iteration: 765000, Loss: -158.131088257
    Iteration: 766000, Loss: -165.770690918
    Iteration: 767000, Loss: -152.528091431
    Iteration: 768000, Loss: -148.815933228
    Iteration: 769000, Loss: -155.426757812
    Iteration: 770000, Loss: -138.990127563
    Iteration: 771000, Loss: -159.879211426
    Iteration: 772000, Loss: -145.266693115
    Iteration: 773000, Loss: -146.721282959
    Iteration: 774000, Loss: -143.349258423
    Iteration: 775000, Loss: -143.545425415
    Iteration: 776000, Loss: -152.514877319
    Iteration: 777000, Loss: -149.467926025
    Iteration: 778000, Loss: -147.883651733
    Iteration: 779000, Loss: -156.64541626
    Iteration: 780000, Loss: -157.08215332
    Iteration: 781000, Loss: -144.882949829
    Iteration: 782000, Loss: -156.304626465
    Iteration: 783000, Loss: -165.275253296
    Iteration: 784000, Loss: -163.278793335
    Iteration: 785000, Loss: -152.299057007
    Iteration: 786000, Loss: -154.787963867
    Iteration: 787000, Loss: -153.427825928
    Iteration: 788000, Loss: -148.444961548
    Iteration: 789000, Loss: -154.632400513
    Iteration: 790000, Loss: -160.188034058
    Iteration: 791000, Loss: -149.483856201
    Iteration: 792000, Loss: -162.31137085
    Iteration: 793000, Loss: -151.075820923
    Iteration: 794000, Loss: -161.177856445
    Iteration: 795000, Loss: -147.658813477
    Iteration: 796000, Loss: -143.756713867
    Iteration: 797000, Loss: -154.508956909
    Iteration: 798000, Loss: -153.036209106
    Iteration: 799000, Loss: -148.188781738
    Iteration: 800000, Loss: -146.780670166
    Iteration: 801000, Loss: -144.407531738
    Iteration: 802000, Loss: -157.808120728
    Iteration: 803000, Loss: -154.121932983
    Iteration: 804000, Loss: -141.816833496
    Iteration: 805000, Loss: -139.589065552
    Iteration: 806000, Loss: -164.817733765
    Iteration: 807000, Loss: -153.344284058
    Iteration: 808000, Loss: -151.72694397
    Iteration: 809000, Loss: -162.801254272
    Iteration: 810000, Loss: -152.513504028
    Iteration: 811000, Loss: -142.780044556
    Iteration: 812000, Loss: -145.750717163
    Iteration: 813000, Loss: -160.924102783
    Iteration: 814000, Loss: -154.55531311
    Iteration: 815000, Loss: -149.672622681
    Iteration: 816000, Loss: -145.44833374
    Iteration: 817000, Loss: -143.839370728
    Iteration: 818000, Loss: -145.026565552
    Iteration: 819000, Loss: -164.038421631
    Iteration: 820000, Loss: -140.328018188
    Iteration: 821000, Loss: -177.224761963
    Iteration: 822000, Loss: -144.658676147
    Iteration: 823000, Loss: -162.881896973
    Iteration: 824000, Loss: -153.242263794
    Iteration: 825000, Loss: -158.743164062
    Iteration: 826000, Loss: -150.130523682
    Iteration: 827000, Loss: -145.850463867
    Iteration: 828000, Loss: -155.24281311
    Iteration: 829000, Loss: -149.869430542
    Iteration: 830000, Loss: -153.998138428
    Iteration: 831000, Loss: -165.543518066
    Iteration: 832000, Loss: -150.198852539
    Iteration: 833000, Loss: -144.305603027
    Iteration: 834000, Loss: -141.716537476
    Iteration: 835000, Loss: -147.81161499
    Iteration: 836000, Loss: -148.797134399
    Iteration: 837000, Loss: -154.868026733
    Iteration: 838000, Loss: -151.210220337
    Iteration: 839000, Loss: -142.074645996
    Iteration: 840000, Loss: -145.386566162
    Iteration: 841000, Loss: -156.000610352
    Iteration: 842000, Loss: -152.362869263
    Iteration: 843000, Loss: -154.531402588
    Iteration: 844000, Loss: -158.104644775
    Iteration: 845000, Loss: -149.148239136
    Iteration: 846000, Loss: -157.192886353
    Iteration: 847000, Loss: -156.850311279
    Iteration: 848000, Loss: -151.588150024
    Iteration: 849000, Loss: -159.527175903
    Iteration: 850000, Loss: -156.167953491
    Iteration: 851000, Loss: -177.819259644
    Iteration: 852000, Loss: -147.87020874
    Iteration: 853000, Loss: -148.176376343
    Iteration: 854000, Loss: -140.466659546
    Iteration: 855000, Loss: -150.1668396
    Iteration: 856000, Loss: -139.511444092
    Iteration: 857000, Loss: -156.17956543
    Iteration: 858000, Loss: -147.925170898
    Iteration: 859000, Loss: -151.563171387
    Iteration: 860000, Loss: -155.179336548
    Iteration: 861000, Loss: -154.869689941
    Iteration: 862000, Loss: -166.77331543
    Iteration: 863000, Loss: -150.005645752
    Iteration: 864000, Loss: -147.387512207
    Iteration: 865000, Loss: -152.540710449
    Iteration: 866000, Loss: -153.364959717
    Iteration: 867000, Loss: -147.436050415
    Iteration: 868000, Loss: -150.426712036
    Iteration: 869000, Loss: -146.082702637
    Iteration: 870000, Loss: -147.064300537
    Iteration: 871000, Loss: -155.02935791
    Iteration: 872000, Loss: -145.759811401
    Iteration: 873000, Loss: -146.695083618
    Iteration: 874000, Loss: -143.118804932
    Iteration: 875000, Loss: -158.635528564
    Iteration: 876000, Loss: -155.000793457
    Iteration: 877000, Loss: -139.07182312
    Iteration: 878000, Loss: -151.359848022
    Iteration: 879000, Loss: -147.852920532
    Iteration: 880000, Loss: -139.067153931
    Iteration: 881000, Loss: -150.835006714
    Iteration: 882000, Loss: -165.133361816
    Iteration: 883000, Loss: -145.75730896
    Iteration: 884000, Loss: -154.892349243
    Iteration: 885000, Loss: -158.299575806
    Iteration: 886000, Loss: -161.912017822
    Iteration: 887000, Loss: -167.835464478
    Iteration: 888000, Loss: -149.413024902
    Iteration: 889000, Loss: -160.982925415
    Iteration: 890000, Loss: -148.125274658
    Iteration: 891000, Loss: -162.924163818
    Iteration: 892000, Loss: -143.987442017
    Iteration: 893000, Loss: -144.408004761
    Iteration: 894000, Loss: -143.427001953
    Iteration: 895000, Loss: -192.186309814
    Iteration: 896000, Loss: -139.987136841
    Iteration: 897000, Loss: -145.731506348
    Iteration: 898000, Loss: -155.425979614
    Iteration: 899000, Loss: -143.553527832
    Iteration: 900000, Loss: -146.807754517
    Iteration: 901000, Loss: -142.612731934
    Iteration: 902000, Loss: -144.191452026
    Iteration: 903000, Loss: -154.822753906
    Iteration: 904000, Loss: -141.211776733
    Iteration: 905000, Loss: -148.055084229
    Iteration: 906000, Loss: -159.175231934
    Iteration: 907000, Loss: -151.344863892
    Iteration: 908000, Loss: -150.508377075
    Iteration: 909000, Loss: -143.713806152
    Iteration: 910000, Loss: -145.06578064
    Iteration: 911000, Loss: -144.742614746
    Iteration: 912000, Loss: -145.448684692
    Iteration: 913000, Loss: -156.951248169
    Iteration: 914000, Loss: -143.052886963
    Iteration: 915000, Loss: -144.705291748
    Iteration: 916000, Loss: -141.939193726
    Iteration: 917000, Loss: -141.88444519
    Iteration: 918000, Loss: -141.324401855
    Iteration: 919000, Loss: -140.416275024
    Iteration: 920000, Loss: -172.405609131
    Iteration: 921000, Loss: -154.595596313
    Iteration: 922000, Loss: -143.891860962
    Iteration: 923000, Loss: -149.808151245
    Iteration: 924000, Loss: -145.961654663
    Iteration: 925000, Loss: -144.932037354
    Iteration: 926000, Loss: -148.584915161
    Iteration: 927000, Loss: -150.830032349
    Iteration: 928000, Loss: -147.744567871
    Iteration: 929000, Loss: -154.855819702
    Iteration: 930000, Loss: -148.450134277
    Iteration: 931000, Loss: -142.414779663
    Iteration: 932000, Loss: -148.450271606
    Iteration: 933000, Loss: -167.594055176
    Iteration: 934000, Loss: -149.176742554
    Iteration: 935000, Loss: -150.146636963
    Iteration: 936000, Loss: -144.151794434
    Iteration: 937000, Loss: -152.319274902
    Iteration: 938000, Loss: -156.608581543
    Iteration: 939000, Loss: -142.195861816
    Iteration: 940000, Loss: -154.413467407
    Iteration: 941000, Loss: -158.648620605
    Iteration: 942000, Loss: -143.823059082
    Iteration: 943000, Loss: -148.388366699
    Iteration: 944000, Loss: -146.026031494
    Iteration: 945000, Loss: -141.033004761
    Iteration: 946000, Loss: -158.410964966
    Iteration: 947000, Loss: -150.463729858
    Iteration: 948000, Loss: -153.414077759
    Iteration: 949000, Loss: -135.458114624
    Iteration: 950000, Loss: -145.948425293
    Iteration: 951000, Loss: -148.844894409
    Iteration: 952000, Loss: -151.246520996
    Iteration: 953000, Loss: -147.288757324
    Iteration: 954000, Loss: -147.713241577
    Iteration: 955000, Loss: -153.400695801
    Iteration: 956000, Loss: -144.17918396
    Iteration: 957000, Loss: -151.809265137
    Iteration: 958000, Loss: -148.719039917
    Iteration: 959000, Loss: -151.192962646
    Iteration: 960000, Loss: -158.402481079
    Iteration: 961000, Loss: -148.626647949
    Iteration: 962000, Loss: -150.199890137
    Iteration: 963000, Loss: -146.406112671
    Iteration: 964000, Loss: -154.583709717
    Iteration: 965000, Loss: -142.019515991
    Iteration: 966000, Loss: -157.862319946
    Iteration: 967000, Loss: -152.295211792
    Iteration: 968000, Loss: -149.890090942
    Iteration: 969000, Loss: -143.260940552
    Iteration: 970000, Loss: -144.859634399
    Iteration: 971000, Loss: -143.880813599
    Iteration: 972000, Loss: -152.233825684
    Iteration: 973000, Loss: -151.933776855
    Iteration: 974000, Loss: -146.558105469
    Iteration: 975000, Loss: -169.29486084
    Iteration: 976000, Loss: -146.009338379
    Iteration: 977000, Loss: -141.553619385
    Iteration: 978000, Loss: -147.280670166
    Iteration: 979000, Loss: -152.124526978
    Iteration: 980000, Loss: -155.6431427
    Iteration: 981000, Loss: -153.308319092
    Iteration: 982000, Loss: -166.536697388
    Iteration: 983000, Loss: -145.636291504
    Iteration: 984000, Loss: -156.890899658
    Iteration: 985000, Loss: -148.798049927
    Iteration: 986000, Loss: -142.355484009
    Iteration: 987000, Loss: -144.82333374
    Iteration: 988000, Loss: -149.707046509
    Iteration: 989000, Loss: -152.624801636
    Iteration: 990000, Loss: -149.177230835
    Iteration: 991000, Loss: -146.452789307
    Iteration: 992000, Loss: -149.927047729
    Iteration: 993000, Loss: -150.112258911
    Iteration: 994000, Loss: -148.288131714
    Iteration: 995000, Loss: -145.785354614
    Iteration: 996000, Loss: -147.399139404
    Iteration: 997000, Loss: -141.588745117
    Iteration: 998000, Loss: -141.312301636
    Iteration: 999000, Loss: -150.148178101

```python
plt.figure()
plt.plot(iteration_array, variational_lower_bound_array)
plt.plot(iteration_array, KL_term_array)
plt.plot(iteration_array, log_likelihood_array)
plt.legend(['Variational Lower Bound', 'KL divergence', 'Log Likelihood'], bbox_to_anchor=(1.05, 1), loc=2)
plt.title('Loss per iteration')
```

## Results

Here, we plot the reconstructed image on test set images.

```python
import os
load_model = False
if load_model:
    saver.restore(sess, os.path.join(os.getcwd(), "Trained Bernoulli VAE"))

num_pairs = 10
image_indices = np.random.randint(0, 200, num_pairs)
for pair in range(num_pairs):
    x = np.reshape(mnist.test.images[image_indices[pair]], (1,n_pixels))
    plt.figure()
    x_image = np.reshape(x, (28,28))
    plt.subplot(121)
    plt.imshow(x_image)
    x_reconstruction = reconstruction.eval(feed_dict={X: x})
    x_reconstruction_image = (np.reshape(x_reconstruction, (28,28)))
    plt.subplot(122)
    plt.imshow(x_reconstruction_image)
```

```python

```


# 1. 2D Linear Convection

We consider the 1d linear Convection equation, under a constant velocity

$$
\partial_t u + \mathbf{a} \cdot \nabla u - \nu \nabla^2 u = 0
$$

```python
# needed imports
from numpy import zeros, ones, linspace, zeros_like
from matplotlib.pyplot import plot, contourf, show, colorbar
%matplotlib inline
```

```python
# Initial condition
import numpy as np
u0 = lambda x,y: np.exp(-(x-.3)**2/.05**2)*np.exp(-(y-.3)**2/.05**2)

ts = linspace(0., 1., 401)
x,y = np.meshgrid(ts,ts)
u = u0(x,y)
```

```python
contourf(x,y, u); colorbar() ; show()
```

### Time scheme

$$\frac{u^{n+1}-u^n}{\Delta t} + \mathbf{a} \cdot \nabla u^{n+1} - \nu \nabla^2 u_{n+1} = 0 $$

$$ \left(I + \Delta t \mathbf{a} \cdot \nabla - \nu \nabla^2 \right) u^{n+1} = u^n $$

### Weak formulation

$$
\langle v, u^{n+1} \rangle + \Delta t ~ \langle v, \mathbf{a} \cdot \nabla u^{n+1} \rangle + \nu ~\Delta t~ \langle \nabla v, \nabla u^{n+1} \rangle = \langle v, u^n \rangle
$$
if we assume $\mathbf{a} = \left( a_1, a_2 \right)^T$ is a constant, then our weak formulation writes
$$
\langle v, u^{n+1} \rangle - \Delta t ~ \langle \mathbf{a} \cdot \nabla v ,  u^{n+1} \rangle + \nu ~ \Delta t~\langle \nabla v, \nabla u^{n+1} \rangle = \langle v, u^n \rangle
$$

expending $u^n$ over the fem basis, we get the linear system

$$A U^{n+1} = M U^n$$

where

$$
M_{ij} = \langle b_i, b_j \rangle
$$

$$
A_{ij} = \langle b_i, b_j \rangle - \Delta t ~ \langle \mathbf{a} \cdot \nabla b_i, b_j \rangle + \nu ~\Delta t~ \langle \nabla b_i , \nabla b_j \rangle
$$

## Abstract Model using SymPDE

```python
from sympde.core import Constant
from sympde.expr import BilinearForm, LinearForm, integral
from sympde.topology import ScalarFunctionSpace, Square, element_of, grad, dot
from sympy import Tuple
```

```python
# ... abstract model
domain = Square()

V = ScalarFunctionSpace('V', domain)

x,y = domain.coordinates

u,v = [element_of(V, name=i) for i in ['u', 'v']]

a1 = Constant('a1')
a2 = Constant('a2')
dt = Constant('dt')
nu = Constant('nu')

a = Tuple(a1,a2)

# bilinear form
expr = v*u + dt* dot(a, grad(u))*v + nu*dt*dot(grad(u), grad(v))
a = BilinearForm((u,v), integral(domain , expr))

# bilinear form for the mass matrix
expr = u*v
m  = BilinearForm((u,v), integral(domain , expr))

# linear form for initial condition
from sympy import exp
expr = exp(-(x-.3)**2/.05**2)*exp(-(y-.3)**2/.05**2)*v
l = LinearForm(v, integral(domain, expr))
```

## Discretization using Psydac

```python
from psydac.api.discretization import discretize
```

```python
a1 = 1. ; a2 = 0. # wavespeed
nu = .03   # viscosity
T = 0.2  # T final time
dt = 0.001
niter = int(T / dt)

degree = [3,3]    # spline degree
ncells = [64,64]   # number of elements
```

```python
# Create computational domain from topological domain
domain_h = discretize(domain, ncells=ncells, comm=None)

# Discrete spaces
Vh = discretize(V, domain_h, degree=degree)

# Discretize the bilinear forms
ah = discretize(a, domain_h, [Vh, Vh])
mh = discretize(m, domain_h, [Vh, Vh])

# Discretize the linear form for the initial condition
lh = discretize(l, domain_h, Vh)
```

```python
# assemble matrices and convert them to scipy
M = mh.assemble().tosparse()
A = ah.assemble(a1=a1, a2=a2, nu=nu, dt=dt).tosparse()

# assemble the rhs and convert it to numpy array
rhs = lh.assemble().toarray()
```

```python
from scipy.sparse.linalg import cg, gmres
```

```python
# L2 projection of the initial condition
un, status = cg(M, rhs, tol=1.e-8, maxiter=5000)
```

```python
from simplines import plot_field_2d

nbasis = [W.nbasis for W in Vh.spaces]
plot_field_2d(Vh.knots, Vh.degree, un.reshape(nbasis)) ; colorbar() ; show()
```

```python
for i in range(0, niter):
    b = M.dot(un)
    un, status = gmres(A, b, tol=1.e-8, maxiter=5000)
```

```python
nbasis = [W.nbasis for W in Vh.spaces]
plot_field_2d(Vh.knots, Vh.degree, un.reshape(nbasis)) ; colorbar() ; show()
```

```python

```


```python
%matplotlib inline
import matplotlib.pyplot as p
```

```python
from sympy import *
import scipy as sc
init_printing()
```

```python
x=var('x')
```

```python
a, b, c = var("a, b, c")
```

```python
x = var('x', real=True)
```

## bio cal

```python
r_m, N, t = var("r_m N t", real=True)
```

```python
N=Function("N")
```

```python
dN_dt = Derivative(N(t), t)-r_m*N(t)
dN_dt
## cal N at any given time
```

```python
dsolve(dN_dt)
```


# Hierarchical Model for Abalone Length

Abalone were collected from various sites on the coast of California north of San Francisco. Here I'm going to develop a model to predict abalone lengths based on sites and harvest method - diving or rock-picking. I'm interested in how abalone lengths vary between sites and harvesting methods. This should be a hierarchical model as the abalone at the different sites are from the same population and should exhibit similar effects based on harvesting method. The hierarchical model will be beneficial since some of the sites are missing a harvesting method.

```python
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib.pyplot as plt

import sampyl as smp
from sampyl import np

import pandas as pd
```

Load our data here. This is just data collected in 2017.

```python
data = pd.read_csv('Clean2017length.csv')
data.head()
```

      data year
      full lengths
      group_id
      site_code
      Full_ID
      Mode

      0
      2017
      181
      1.0
      5
      2017_06_24_005_30_01_01
      R

      1
      2017
      182
      1.0
      5
      2017_06_24_005_30_01_01
      R

      2
      2017
      183
      1.0
      5
      2017_06_24_005_30_01_01
      R

      3
      2017
      191
      1.0
      5
      2017_06_24_005_30_01_01
      R

      4
      2017
      191
      1.0
      5
      2017_06_24_005_30_01_01
      R

Important columns here are:

* **full lengths:** length of abalone
* **mode:** Harvesting method, R: rock-picking, D: diving
* **site_code:** codes for 15 different sites

First some data preprocessing to get it into the correct format for our model.

```python
# Convert sites from codes into sequential integers starting at 0
unique_sites = data['site_code'].unique()
site_map = dict(zip(unique_sites, np.arange(len(unique_sites))))
data = data.assign(site=data['site_code'].map(site_map))

# Convert modes into integers as well
# Filter out 'R/D' modes, bad data collection
data = data[(data['Mode'] != 'R/D')]
mode_map = {'R':0, 'D':1}
data = data.assign(mode=data['Mode'].map(mode_map))
```

## A Hierarchical Linear Model

Here we'll define our model. We want to make a linear model for each site in the data where we predict the abalone length given the mode of catching and the site.

$$ y_s = \alpha_s + \beta_s * x_s + \epsilon $$

where $y_s$ is the predicted abalone length, $x$ denotes the mode of harvesting, $\alpha_s$ and $\beta_s$ are coefficients for each site $s$, and $\epsilon$ is the model error. We'll use this prediction for our likelihood with data $D_s$, using a normal distribution with mean $y_s$ and variance $ \epsilon^2$ :

$$  \prod_s P(D_s \mid \alpha_s, \beta_s, \epsilon) = \prod_s \mathcal{N}\left(D_s \mid y_s, \epsilon^2\right) $$

The abalone come from the same population just in different locations. We can take these similarities between sites into account by creating a hierarchical model where the coefficients are drawn from a higher-level distribution common to all sites.

$$
\begin{align}
\alpha_s & \sim \mathcal{N}\left(\mu_{\alpha}, \sigma_{\alpha}^2\right) \\
\beta_s & \sim \mathcal{N}\left(\mu_{\beta}, \sigma_{\beta}^2\right) \\
\end{align}
$$

```python
sites = data['site'].values
modes = data['mode'].values
lengths = data['full lengths'].values

# Now define the model (log-probability proportional to the posterior)
def logp(μ_α, μ_β, σ_α, σ_β, site_α, site_β, ϵ):

    model = smp.Model()

    # Population priors - normals for population means and half-Cauchy for population stds
    model.add(smp.normal(μ_α, sig=500),
              smp.normal(μ_β, sig=500),
              smp.half_cauchy(σ_α, beta=5),
              smp.half_cauchy(σ_β, beta=0.5))

    # Priors for site coefficients, sampled from population distributions
    model.add(smp.normal(site_α, mu=μ_α, sig=σ_α),
              smp.normal(site_β, mu=μ_β, sig=σ_β))

    # Prior for likelihood uncertainty
    model.add(smp.half_normal(ϵ))

    # Our estimate for abalone length, α + βx
    length_est = site_α[sites] + site_β[sites]*modes

    # Add the log-likelihood
    model.add(smp.normal(lengths, mu=length_est, sig=ϵ))

    return model()
```

```python
start = {'μ_α': 201., 'μ_β': 5., 'σ_α': 1., 'σ_β': 1.,
         'site_α': np.ones(len(site_map))*201,
         'site_β': np.zeros(len(site_map)),
         'ϵ': 1.}

# Using NUTS is slower per sample, but more likely to give good samples (and converge)
sampler = smp.NUTS(logp, start)
chain = sampler(1100, burn=100, thin=2)
```

    Progress: [##############################] 1100 of 1100 samples

There are some checks for convergence you can do, but they aren't implemented yet. Instead, we can visually inspect the chain. In general, the samples should be stable, the first half should vary around the same point as the second half.

```python
plt.plot(chain.σ_β);
```

With the posterior distribution, we can look at many different results. Here I'll make a function that plots the means and 95% credible regions (range that contains central 95% of the probability) for the coefficients $\alpha_s$ and $\beta_s$.

```python
def coeff_plot(coeff, ax=None):
    if ax is None:
        fig, ax = plt.subplots(figsize=(3,5))
    CRs = np.percentile(coeff, [2.5, 97.5], axis=0)
    means = coeff.mean(axis=0)
    ax.errorbar(means, np.arange(len(means)), xerr=np.abs(means - CRs), fmt='o')

    ax.set_yticks(np.arange(len(site_map)))
    ax.set_yticklabels(site_map.keys())
    ax.set_ylabel('Site')

    ax.grid(True, axis='x', color="#CCCCCC")
    ax.tick_params(axis='both', length=0)
    for each in ['top', 'right', 'left', 'bottom']:
        ax.spines[each].set_visible(False)

    return ax
```

Now we can look at how abalone lengths vary between sites for the rock-picking method ($\alpha_s$).

```python
ax = coeff_plot(chain.site_α)
ax.set_xlim(175, 225)
ax.set_xlabel('Abalone Length (mm)');
```

Here I'm plotting the mean and 95% credible regions (CR) of $\alpha$ for each site. This coefficient measures the average length of rock-picked abalones. We can see that the average abalone length varies quite a bit between sites. The CRs give a measure of the uncertainty in $\alpha$, wider CRs tend to result from less data at those sites.

Now, let's see how the abalone lengths vary between harvesting methods (the difference for diving is given by $\beta_s$).

```python
ax = coeff_plot(chain.site_β)
#ax.set_xticks([-5, 0, 5, 10, 15])
ax.set_xlabel('Mode effect (mm)');
```

Here I'm plotting the mean and 95% credible regions (CR) of $\beta$ for each site. This coefficient measures the difference in length of dive picked abalones compared to rock picked abalones. Most of the $\beta$ coefficients are above zero which indicates that abalones harvested via diving are larger than ones picked from the shore. For most of the sites, diving results in 5 mm longer abalone, while at site 72, the difference is around 12 mm. Again, wider CRs mean there is less data leading to greater uncertainty.

Next, I'll overlay the model on top of the data and make sure it looks right. We'll also see that some sites don't have data for both harvesting modes but our model still works because it's hierarchical. That is, we can get a posterior distribution for the coefficient from the population distribution even though the actual data is missing.

```python
def model_plot(data, chain, site, ax=None, n_samples=20):
    if ax is None:
        fig, ax = plt.subplots(figsize=(4,6))

    site = site_map[site]
    xs = np.linspace(-1, 3)

    for ii, (mode, m_data) in enumerate(data[data['site'] == site].groupby('mode')):

        a = chain.site_α[:, site]
        b = chain.site_β[:, site]

        # now sample from the posterior...
        idxs = np.random.choice(np.arange(len(a)), size=n_samples, replace=False)

        # Draw light lines sampled from the posterior
        for idx in idxs:
            ax.plot(xs, a[idx] + b[idx]*xs, color='#E74C3C', alpha=0.05)

        # Draw the line from the posterior means
        ax.plot(xs, a.mean() + b.mean()*xs, color='#E74C3C')

        # Plot actual data points with a bit of noise for visibility
        mode_label = {0: 'Rock-picking', 1: 'Diving'}
        ax.scatter(ii + np.random.randn(len(m_data))*0.04,
               m_data['full lengths'], edgecolors='none',
               alpha=0.8, marker='.', label=mode_label[mode])

    ax.set_xlim(-0.5, 1.5)
    ax.set_xticks([0, 1])
    ax.set_xticklabels('')
    ax.set_ylim(150, 250)

    ax.grid(True, axis='y', color="#CCCCCC")
    ax.tick_params(axis='both', length=0)

    for each in ['top', 'right', 'left', 'bottom']:
        ax.spines[each].set_visible(False)

    return ax
```

```python
fig, axes = plt.subplots(figsize=(10, 5), ncols=4, sharey=True)

for ax, site in zip(axes, [5, 52, 72, 162]):
    ax = model_plot(data, chain, site, ax=ax, n_samples=30)
    ax.set_title(site)

first_ax = axes[0]
first_ax.legend(framealpha=1, edgecolor='none')
first_ax.set_ylabel('Abalone length (mm)');
```

For site 5, there are few data points for the diving method so there is a lot of uncertainty in the prediction. The prediction is also pulled lower than the data by the population distribution. Similarly, for site 52 there is no diving data, but we still get a (very uncertain) prediction because it's using the population information.

Finally, we can look at the harvesting mode effect for the population. Here I'm going to print out a few statistics for $\mu_{\beta}$.

```python
fig, ax = plt.subplots()
ax.hist(chain.μ_β, bins=30);
b_mean = chain.μ_β.mean()
b_CRs = np.percentile(chain.μ_β, [2.5, 97.5])
p_gt_0 = (chain.μ_β > 0).mean()
print(
"""Mean: {:.3f}
95% CR: [{:.3f}, {:.3f}]
P(mu_b) > 0: {:.3f}
""".format(b_mean, b_CRs[0], b_CRs[1], p_gt_0))
```

We can also look at the population distribution for $\beta_s$ by sampling from a normal distribution with mean and variance sampled from $\mu_\beta$ and $\sigma_\beta$.

$$
\beta_s \sim \mathcal{N}\left(\mu_{\beta}, \sigma_{\beta}^2\right)
$$

```python
import scipy.stats as stats
```

```python
samples = stats.norm.rvs(loc=chain.μ_β, scale=chain.σ_β)
plt.hist(samples, bins=30);
plt.xlabel('Dive harvesting effect (mm)')
```

It's apparent that dive harvested abalone are roughly 5 mm longer than rock-picked abalone. Maybe this is a bias of the divers to pick larger abalone. Or, it's possible that abalone that stay in the water grow larger.


# Simplified Arm Mode

## Introduction

This notebook presents the analytical derivations of the equations of motion for
three degrees of freedom and nine muscles arm model, some of them being
bi-articular, appropriately constructed to demonstrate both kinematic and
dynamic redundancy (e.g.  $d < n < m$). The model is inspired from [1] with some
minor modifications and improvements.

## Model Constants

Abbreviations:

- DoFs: Degrees of Freedom
- EoMs: Equations of Motion
- KE: Kinetic Energy
- PE: Potential Energy
- CoM: center of mass

The following constants are used in the model:

- $m$ mass of a segment
- $I_{z_i}$ inertia around $z$-axis
- $L_i$ length of a segment
- $L_{c_i}$ length of the CoM as defined in local frame of a body
- $a_i$ muscle origin point as defined in the local frame of a body
- $b_i$ muscle insertion point as defined in the local frame of a body
- $g$ gravity
- $q_i$ are the generalized coordinates
- $u_i$ are the generalized speeds
- $\tau$ are the generalized forces

Please note that there are some differences from [1]: 1) $L_{g_i} \rightarrow
L_{c_i}$, 2) $a_i$ is always the muscle origin, 3) $b_i$ is always the muscle
insertion and 4) we don't use double indexing for the bi-articular muscles.

```python
# notebook general configuration

%load_ext autoreload
%autoreload 2

# imports and utilities
import sympy as sp
from IPython.display import display, Image
sp.interactive.printing.init_printing()

import logging
logging.basicConfig(level=logging.INFO)

# plot
%matplotlib inline
from matplotlib.pyplot import *
rcParams['figure.figsize'] = (10.0, 6.0)

# utility for displaying intermediate results
enable_display  = True
def disp(*statement):
    if (enable_display):
        display(*statement)
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

```python
# construct model
from model import ArmModel
model = ArmModel(use_gravity=1, use_coordinate_limits=0, use_viscosity=0)

disp(model.constants)
```

## Dynamics

The simplified arm model has three DoFs and nine muscles, some of them being
bi-articular. The analytical expressions of the EoMs form is given by

\begin{equation}\label{equ:eom-standard-form}
  M(q) \ddot{q} + C(q, \dot{q})\dot{q} + \tau_g(q) = \tau
\end{equation}

where $M \in \Re^{n \times n}$ represents the inertia mass matrix, $n$ the DoFs
of the model, $q, \dot{q}, \ddot{q} \in \Re^{n}$ the generalized coordinates and
their derivatives, $C \in \Re^{n \times n}$ the Coriolis and centrifugal matrix,
$\tau_g \in \Re^{n}$ the gravity contribution and $\tau$ the specified
generalized forces.

As the model is an open kinematic chain a simple procedure to derive the EoMs
can be followed. Assuming that the spatial velocity (translational, rotational)
of each body segment is given by $u_b = [v, \omega]^T \in \Re^{6 \times 1}$, the
KE of the system in body local coordinates is defined as

\begin{equation}\label{equ:spatial-ke}
  K = \frac{1}{2} \sum\limits_{i=1}^{n_b} (m_i v_i^2 + I_i \omega_i^2) =
  \frac{1}{2} \sum\limits_{i=1}^{n_b} u_i^T M_i u_i
\end{equation}

where $M_i = diag(m_i, m_i, m_i, [I_i]_{3 \times 3}) \in \Re^{6 \times 6}$
denotes the spatial inertia mass matrix, $m_i$ the mass and $I_i \in \Re^{3
\times 3}$ the inertia matrix of body $i$. The spatial quantities are related
to the generalized coordinates by the body Jacobian $u_b = J_b \dot{q}, \; J_b
\in \Re^{6 \times n}$. The total KE is coordinate invariant, thus it can be
expressed in different coordinate system

\begin{equation}\label{equ:ke-transformation}
  K = \frac{1}{2} \sum\limits_{i=1}^{n_b} q^T J_i^T M_i J_i q
\end{equation}

Following the above definition, the inertia mass matrix of the system can be
written as

\begin{equation}\label{equ:mass-matrix}
  M(q) =  \sum\limits_{i=1}^{n_b}  J_i^T M_i J_i
\end{equation}

Furthermore, the Coriolis and centrifugal forces $C(q, \dot{q}) \dot{q}$ can be
determined directly from the inertia mass matrix

\begin{equation}\label{equ:coriolis-matrix}
  C_{ij}(q, \dot{q}) = \sum\limits_{k=1}^{n} \Gamma_{ijk} \; \dot{q}_k, \; i, j
  \in [1, \dots n], \;
  \Gamma_{ijk} = \frac{1}{2} (
  \frac{\partial M_{ij}(q)}{\partial q_k} +
  \frac{\partial M_{ik}(q)}{\partial q_j} -
  \frac{\partial M_{kj}(q)}{\partial q_i})
\end{equation}

where the functions $\Gamma_{ijk}$ are called the Christoffel symbols. The
gravity contribution can be determined from the PE function

\begin{equation}\label{equ:gravity-pe}
  \begin{gathered}
    g(q) = \frac{\partial V(q)}{\partial q}, \; V(q) = \sum\limits_{i=1}^{n_b} m_i g h_i(q)
  \end{gathered}
\end{equation}

where $h_i(q)$ denotes the vertical displacement of body $i$ with respect to the
ground. In this derivation we chose to collect all forces that act on the system
in the term $f(q, \dot{q})$.

```python
# define the spatial coordinates for the CoM in terms of Lcs' and q's
disp(model.xc[1:])
# define CoM spatial velocities
disp(model.vc[1:])
#define CoM Jacobian
disp(model.Jc[1:])
```

```python
# generate the inertial mass matrix
M = model.M
for i in range(0, M.shape[0]):
    for j in range(0, M.shape[1]):
        disp('M_{' + str(i + 1) + ',' + str(j + 1) + '} = ', M[i, j])
```

```python
# total forces from Coriolis, centrafugal and gravity
f = model.f
for i in range(0, f.shape[0]):
    disp('f_' + str(i + 1) + ' = ',  f[i])
```

## Muscle Moment Arm

The muscle forces $f_m$ are transformed into joint space generalized forces
($\tau$) by the moment arm matrix ($\tau = -R^T f_m$). For a n-lateral polygon
it can be shown that the derivative of the side length with respect to the
opposite angle is the moment arm component. As a consequence, when expressing
the muscle length as a function of the generalized coordinates of the model, the
moment arm matrix is evaluated by $R = \frac{\partial l_{mt}}{\partial q}$. The
analytical expressions of the EoMs following our convention are provided below

\begin{equation}\label{equ:eom-notation}
  \begin{gathered}
    M(q) \ddot{q} + f(q, \dot{q}) = \tau \\
    \tau = -R^T(q) f_m
  \end{gathered}
\end{equation}

```python
# assert that moment arm is correctly evaluated
# model.test_muscle_geometry() # slow
# muscle length
disp('l_m = ', model.lm)
# moment arm
disp('R = ', model.R)
```

```python
# draw model
fig, ax = subplots(1, 1, figsize=(10, 10), frameon=False)
model.draw_model([60, 70, 50], True, ax, 1, False)
fig.tight_layout()
fig.savefig('results/arm_model.pdf', dpi=600, format='pdf',
            transparent=True, pad_inches=0, bbox_inches='tight')
```

[1] K. Tahara, Z. W. Luo, and S. Arimoto, “On Control Mechanism of Human-Like
Reaching Movements with Musculo-Skeletal Redundancy,” in International
Conference on Intelligent Robots and Systems, 2006, pp. 1402–1409.


# 6. Internal dynamic factor

Based on:

[1] ISO 6336-1:2006 Calculation of load capacity of spur and helical gears -- Part 1: Basic principles, introduction and general influence factors

```python
from sympy import *
from matplotlib import pyplot
from numpy import arange, array
init_printing()

def symb(x, y, z = ''):
    return symbols('{0}_{1}{2}'.format(x, y, z), type = float)
```

### 6.4.2 Method B -- Factor $K_{v-B}$
This method is not recommended if:
\begin{align}
    \frac{v z_1}{100} \sqrt{u^2/(1 + u^2)} < 3 \mathrm{m/s}
\end{align}

## 6.5 Determination using Method B
### 6.5.3 Resonance running speed:
It is given by:

```python
u, z1, cga, mRed = symbols('u z_1 c_gamma_alpha m_red')
eq6 = 30000/(pi*z1)*sqrt(cga/mRed) # n_E1
eq6
```

where $m_{red}$ is the relative mass of a gear pair, i.e. of(?) the mass per face width of each gear, referred to its base radius or to the line of action. It is equal to:

```python
eq7 = symb('m',1,'^*')*symb('m',2,'^*')/(symb('m',1,'^*') + symb('m',2,'^*'))
eq7
```

The resonance ratio is the ratio of pinion speed to resonance speed, and it given by:

```python
simplify(symb('n', 1)/eq6)
```

```python
NS = lambda x: (0.85 if (x >= 100) else 0.5 + 0.35*sqrt(x/100))
X = arange(0, 200.0)
Y = array([NS(x) for x in X])
pyplot.plot(X, Y, 'r-')
pyplot.plot(X,1.15*ones(len(X)), 'g:')
pyplot.plot(X,1.50*ones(len(X)), 'b-.')
pyplot.axis([0, 200, 0.4, 1.8])
pyplot.xlabel('specific loading, [N/mm]')
pyplot.ylabel('Resonance ratio, [-]')
```

### 6.5.8 Resonance speed for specific gear designs
#### 6.5.8.4 Planetary gears
See also 6.4.2

a) Ring gear connected to the case:

```python

```

### 6.5.9
valid for gear pais with external teeth: spur, single and double helical.

```python
dm1, db1, rho1, q1, rho2, q2, rho = symbols('d_m1 d_b1 rho_1 q_1 rho_2 q_2 rho')
eq30r = dm1**2/(1/(rho1*(1 - q1**4)) + 1/(rho2*(1 - q2**4)*u**2))
eq30l = (pi/8)*(dm1/db1)**2
eq30 = eq30l*eq30r
eq30
```

for pinions and wheels of solid construction: $q_1 = q_2 = 0$.

```python
eq30 = eq30.subs([(q1, 0), (q2,0)])
simplify(eq30)
```

Considering that the gears have the same density, leads to:

```python
eq30 = eq30.subs([(rho1, rho), (rho2, rho)])
simplify(eq30)
```

Notice that altough this expression represents the reduced mass of a gear pair, it does not has units of mass.

```python

```