```python
# Imports
import sys

from tqdm import tqdm

import torch
import numpy as np
import matplotlib.pyplot as plt

cuda = torch.cuda.is_available()
%matplotlib inline
```

# Variational Autoencoder

The variational autoencoder (VAE) was described in its current form in [Kingma 2013](https://arxiv.org/abs/1312.6114). The model consists of an encoder/inference network $q_{\phi}(z|x)$ and a decoder/generative network $p_{\theta}(x|z)$. The main idea is that it is possible to both reconstruct and generate samples from from some input distribution by learning a variational distribution over the latent variable $z$.

The VAE therefore has a bottleneck structure, where the input $x$ is encoded into a latent variable $z$. New data can then be generated by feeding a latent code into the generator network - $\widehat{x} \sim p_{\theta}(z|x)$. The diagram above shows the generative model (right) and how the latent variable $z$ is inferred from $x$ (left).

Below we will instantiate a new variational autoencoder with this bottleneck structure consisting of a 2-layer encoder network turning an input MNIST image into a latent code: $784 \to 256 \to 128 \to 32$. We also have a decoder that performs the operation in reverse: $32 \to 128 \to 256 \to 784$.

```python
from semi.models import VariationalAutoencoder
from semi.layers import GaussianSample
model = VariationalAutoencoder([784, 32, [256, 128]])
model
```

    /Users/jakobhavtorn/repos/vae/semi/models/vae.py:113: UserWarning: nn.init.xavier_normal is now deprecated in favor of nn.init.xavier_normal_.
      init.xavier_normal(m.weight.data)

    VariationalAutoencoder(
      (encoder): Encoder(
        (hidden): ModuleList(
          (0): Linear(in_features=784, out_features=256, bias=True)
          (1): Linear(in_features=256, out_features=128, bias=True)
        )
        (sample): GaussianSample(
          (mu): Linear(in_features=128, out_features=32, bias=True)
          (log_var): Linear(in_features=128, out_features=32, bias=True)
        )
      )
      (decoder): Decoder(
        (hidden): ModuleList(
          (0): Linear(in_features=32, out_features=128, bias=True)
          (1): Linear(in_features=128, out_features=256, bias=True)
        )
        (reconstruction): Linear(in_features=256, out_features=784, bias=True)
        (output_activation): Sigmoid()
      )
    )

Notice how the middle most layer consists of a `GaussianSample` layer, in which we turn the input digit into the parameters of a Normal distribution with parameters $\mu$ and $\sigma$. This allows us to use the *reparametrization trick* to sample from this distribution to introduce stochasticity into the network.

```python
from torch.autograd import Variable

gaussian = GaussianSample(10, 1)
z, mu, log_var = gaussian(Variable(torch.ones(1, 10)))

print(f"sample {float(z.data):.2f} drawn from N({float(mu.data):.2f}, {float(log_var.exp().data):.2f})")
```

    sample 2.28 drawn from N(0.68, 2.05)

## Training

How do we go about training a variational autoencoder then? We want to model the data distribution $p(x)$, from here we can introduce a variational distribution $q(z|x)$ by multiplying and dividing by this distribution. Now we employ Jensen's inequality to move the logarithm inside the integral, we can do this because $\log$ is concave and because $q(z|x)$ is a probability distribution. From here on we just rearrange and we see that a lower bound on the marginal probability of the data $p(x)$ is just an expectation over the likelihood of the data minus the KL-divergence between the variational distribution and a prior $p(z)$.

\begin{align}
\log p(x) &= \log \int p(x, z) \ dz = \log \int q(z|x) \frac{p(x, z)}{q(z|x)} \ dz\\
          &\geq \int q(z|x) \log \frac{p(x, z)}{q(z|x)} \ dz = \int q(z|x) \log p(x|z) + \log \frac{p(z)}{q(z|x)} \ dz\\
          &= \int q(z|x) \log p(x|z) \ dz + \int q(z|x) \log \frac{p(z)}{q(z|x)} \ dz\\
          &= \mathbb{E}_{q(z|x)} [\log p(x|z)] - KL(q(z|x)||p(z)) = \mathcal{L}(x)
\end{align}

To make things even more concrete, we show how we can go from this equation to an actual algorithm. Recall that the expectation is just an arithmetic mean, which can be approximated using Monte-Carlo samples. In fact for most applications we can do with just a single sample, even though this provides infinite variance.

$$\mathbb{E}_{q(z|x)} [\log p(x|z)] = \lim_{N \to \infty} \frac{1}{N} \sum_{i=1}^{N} \log p(x_i|z_i) \approx \frac{1}{M} \sum_{i=1}^{M} \log p(x_i|z_i)$$

As you can see the likelihood is just the log probability of the data given the latent variable, but the latent variable is itself derived from the data - we can just use the reconstruction error! In the MNIST case, it is most fitting to use the Bernoulli / binary cross entropy.

Finally, the second term is the Kullback-Leibler divergence. It states that whatever distribution we learn over $q(z|x)$ can never be very far from the prior $p(z)$. This is both good and bad news. Ideally we want a reconstruction that is as good as possible, i.e. only relying on the likelihood, which will only occur if $q(z|x) = p(z)$. This will never happen in practice as the q-distribution will be very complex in order to produce latent variables that result convincing samples. On the plus side, the KL-term acts as a regularizer that pulls the distribution towards the prior, which is the whole reason why we can create samples.

This term can either be computed analytically, or by sampling similarly to the way we did it in the expectation, which you can see in the printed doc string below.

```python
print(model._kld.__doc__)
```

            Computes the KL-divergence of
            some element z.

            KL(q||p) = -∫ q(z) log [ p(z) / q(z) ]
                     = -E[log p(z) - log q(z)]

            :param z: sample from q-distribuion
            :param q_param: (mu, log_var) of the q-distribution
            :param p_param: (mu, log_var) of the p-distribution
            :return: KL(q||p)

```python
from semi.data import get_mnist

_, loader_train, loader_validation = get_mnist(location="./", batch_size=64, num_workers=0)

# We use this custom BCE function until PyTorch implements reduce=False
def binary_cross_entropy(r, x):
    return -torch.sum(x * torch.log(r + 1e-8) + (1 - x) * torch.log(1 - r + 1e-8), dim=-1)

optimizer = torch.optim.Adam(model.parameters(), lr=3e-4, betas=(0.9, 0.999))
```

```python
for epoch in range(50):
    model.train()
    total_loss, m = 0, 0
    for (u, _) in tqdm(loader_train):
        u = Variable(u)

        if cuda:
            u = u.cuda(device=0)

        reconstruction = model(u)

        likelihood = -binary_cross_entropy(reconstruction, u)
        kl_divergence = model.kl_divergence

        elbo = likelihood - kl_divergence

        loss = -torch.mean(elbo)

        loss.backward()
        optimizer.step()
        optimizer.zero_grad()

        total_loss += loss.item()

        m += 1

    print(f"Epoch: {epoch}\t ELBO: {total_loss / m:.2f}\tLL: {likelihood.mean():.2f}\tKL: {kl_divergence.mean():.2f}")
```

      0%|          | 0/938 [00:00<?, ?it/s]

      0%|          | 1/938 [00:00<02:14,  6.97it/s]

      0%|          | 3/938 [00:00<01:59,  7.82it/s]

      0%|          | 4/938 [00:00<02:01,  7.67it/s]

      1%|          | 5/938 [00:00<01:57,  7.97it/s]

      1%|          | 7/938 [00:00<01:44,  8.94it/s]

      1%|          | 8/938 [00:00<01:43,  8.97it/s]

      1%|          | 10/938 [00:01<01:34,  9.78it/s]

      1%|▏         | 12/938 [00:01<01:26, 10.75it/s]

      1%|▏         | 14/938 [00:01<01:24, 10.88it/s]

      2%|▏         | 16/938 [00:01<01:24, 10.85it/s]

      2%|▏         | 18/938 [00:01<01:35,  9.61it/s]

      2%|▏         | 20/938 [00:02<01:36,  9.47it/s]

      2%|▏         | 21/938 [00:02<01:39,  9.22it/s]

      2%|▏         | 22/938 [00:02<01:44,  8.74it/s]

      2%|▏         | 23/938 [00:02<01:43,  8.84it/s]

      3%|▎         | 24/938 [00:02<01:45,  8.70it/s]

      3%|▎         | 25/938 [00:02<01:46,  8.56it/s]

      3%|▎         | 27/938 [00:02<01:37,  9.37it/s]

      3%|▎         | 28/938 [00:02<01:36,  9.41it/s]

      3%|▎         | 29/938 [00:03<01:44,  8.66it/s]

      3%|▎         | 30/938 [00:03<01:49,  8.29it/s]

      3%|▎         | 32/938 [00:03<01:41,  8.95it/s]

      4%|▎         | 33/938 [00:03<01:59,  7.59it/s]

      4%|▎         | 35/938 [00:03<01:53,  7.98it/s]

      4%|▍         | 37/938 [00:03<01:43,  8.71it/s]

      4%|▍         | 38/938 [00:04<01:49,  8.22it/s]

      4%|▍         | 39/938 [00:04<01:51,  8.04it/s]

      4%|▍         | 41/938 [00:04<01:37,  9.22it/s]

      5%|▍         | 43/938 [00:04<01:29,  9.99it/s]

      5%|▍         | 45/938 [00:04<01:30,  9.86it/s]

      5%|▌         | 47/938 [00:04<01:32,  9.64it/s]

      5%|▌         | 49/938 [00:05<01:29,  9.96it/s]

      5%|▌         | 51/938 [00:05<01:28, 10.05it/s]

      6%|▌         | 53/938 [00:05<01:26, 10.26it/s]

      6%|▌         | 55/938 [00:05<01:25, 10.31it/s]

      6%|▌         | 57/938 [00:05<01:25, 10.36it/s]
      7%|▋         | 64/938 [00:20<01:09, 12.52it/s]

      6%|▋         | 59/938 [00:06<01:38,  8.97it/s]

      6%|▋         | 60/938 [00:06<01:36,  9.08it/s]

      7%|▋         | 62/938 [00:06<01:28,  9.91it/s]

      7%|▋         | 64/938 [00:06<01:30,  9.61it/s]

      7%|▋         | 66/938 [00:06<01:29,  9.73it/s]

      7%|▋         | 67/938 [00:06<01:43,  8.43it/s]

      7%|▋         | 68/938 [00:07<01:49,  7.95it/s]

      7%|▋         | 69/938 [00:07<01:48,  7.99it/s]

      8%|▊         | 71/938 [00:07<01:38,  8.77it/s]

      8%|▊         | 72/938 [00:07<01:49,  7.92it/s]

      8%|▊         | 73/938 [00:07<01:46,  8.13it/s]

      8%|▊         | 74/938 [00:07<01:48,  8.00it/s]

      8%|▊         | 76/938 [00:08<01:40,  8.58it/s]

      8%|▊         | 77/938 [00:08<01:43,  8.31it/s]

      8%|▊         | 79/938 [00:08<01:32,  9.28it/s]

      9%|▊         | 81/938 [00:08<01:28,  9.66it/s]

      9%|▉         | 83/938 [00:08<01:25,  9.98it/s]

      9%|▉         | 85/938 [00:08<01:21, 10.50it/s]

      9%|▉         | 87/938 [00:09<01:22, 10.30it/s]

      9%|▉         | 89/938 [00:09<01:27,  9.70it/s]

     10%|▉         | 91/938 [00:09<01:26,  9.77it/s]

     10%|▉         | 92/938 [00:09<01:31,  9.23it/s]

     10%|▉         | 93/938 [00:09<01:34,  8.98it/s]

     10%|█         | 94/938 [00:09<01:41,  8.32it/s]

     10%|█         | 95/938 [00:09<01:38,  8.57it/s]

     10%|█         | 96/938 [00:10<01:38,  8.53it/s]

     10%|█         | 98/938 [00:10<01:33,  9.01it/s]

     11%|█         | 99/938 [00:10<01:31,  9.19it/s]

     11%|█         | 101/938 [00:10<01:28,  9.45it/s]

     11%|█         | 102/938 [00:10<01:37,  8.61it/s]

     11%|█         | 104/938 [00:10<01:29,  9.29it/s]

     11%|█         | 105/938 [00:10<01:28,  9.46it/s]

     11%|█▏        | 107/938 [00:11<01:31,  9.10it/s]

     12%|█▏        | 108/938 [00:11<01:31,  9.06it/s]

     12%|█▏        | 109/938 [00:11<01:42,  8.06it/s]

     12%|█▏        | 111/938 [00:11<01:43,  8.01it/s]

     12%|█▏        | 112/938 [00:11<01:44,  7.91it/s]

     12%|█▏        | 113/938 [00:12<01:41,  8.16it/s]

     12%|█▏        | 114/938 [00:12<01:43,  7.94it/s]

     12%|█▏        | 116/938 [00:12<01:31,  9.02it/s]

     13%|█▎        | 118/938 [00:12<01:25,  9.60it/s]

     13%|█▎        | 120/938 [00:12<01:25,  9.53it/s]

     13%|█▎        | 121/938 [00:12<01:32,  8.81it/s]

     13%|█▎        | 122/938 [00:12<01:30,  8.98it/s]

     13%|█▎        | 123/938 [00:13<01:31,  8.87it/s]

     13%|█▎        | 124/938 [00:13<01:30,  9.04it/s]

     13%|█▎        | 125/938 [00:13<01:27,  9.27it/s]

     14%|█▎        | 127/938 [00:13<01:20, 10.04it/s]

     14%|█▍        | 129/938 [00:13<01:23,  9.65it/s]

     14%|█▍        | 130/938 [00:13<01:24,  9.60it/s]

     14%|█▍        | 132/938 [00:13<01:23,  9.63it/s]

     14%|█▍        | 133/938 [00:14<01:27,  9.19it/s]

     14%|█▍        | 134/938 [00:14<01:31,  8.77it/s]

     14%|█▍        | 135/938 [00:14<01:49,  7.31it/s]

     14%|█▍        | 136/938 [00:14<01:47,  7.44it/s]

     15%|█▍        | 138/938 [00:14<01:38,  8.13it/s]

     15%|█▍        | 139/938 [00:14<01:36,  8.31it/s]

     15%|█▍        | 140/938 [00:14<01:33,  8.53it/s]

     15%|█▌        | 141/938 [00:15<01:32,  8.64it/s]

     15%|█▌        | 142/938 [00:15<01:39,  7.97it/s]

     15%|█▌        | 144/938 [00:15<01:28,  8.96it/s]

     16%|█▌        | 146/938 [00:15<01:20,  9.80it/s]

     16%|█▌        | 148/938 [00:15<01:25,  9.26it/s]

     16%|█▌        | 149/938 [00:15<01:48,  7.28it/s]

     16%|█▌        | 150/938 [00:16<02:00,  6.52it/s]

     16%|█▌        | 151/938 [00:16<01:54,  6.90it/s]

     16%|█▋        | 153/938 [00:16<01:38,  7.96it/s]

     16%|█▋        | 154/938 [00:16<01:34,  8.30it/s]

     17%|█▋        | 155/938 [00:16<01:35,  8.22it/s]

     17%|█▋        | 156/938 [00:16<01:33,  8.39it/s]

     17%|█▋        | 157/938 [00:16<01:30,  8.63it/s]

     17%|█▋        | 158/938 [00:17<01:38,  7.89it/s]

     17%|█▋        | 160/938 [00:17<01:28,  8.84it/s]

     17%|█▋        | 162/938 [00:17<01:20,  9.63it/s]

     17%|█▋        | 164/938 [00:17<01:15, 10.21it/s]

     18%|█▊        | 166/938 [00:17<01:13, 10.48it/s]

     18%|█▊        | 168/938 [00:17<01:22,  9.35it/s]

     18%|█▊        | 169/938 [00:18<01:27,  8.83it/s]

     18%|█▊        | 171/938 [00:18<01:20,  9.52it/s]

     18%|█▊        | 172/938 [00:18<01:27,  8.74it/s]

     19%|█▊        | 174/938 [00:18<01:29,  8.58it/s]

     19%|█▊        | 175/938 [00:18<01:38,  7.77it/s]

     19%|█▉        | 177/938 [00:18<01:27,  8.65it/s]

     19%|█▉        | 178/938 [00:19<01:37,  7.83it/s]

     19%|█▉        | 179/938 [00:19<01:34,  8.07it/s]

     19%|█▉        | 180/938 [00:19<01:32,  8.24it/s]

     19%|█▉        | 181/938 [00:19<01:29,  8.50it/s]

     19%|█▉        | 182/938 [00:19<01:26,  8.79it/s]

     20%|█▉        | 183/938 [00:19<01:26,  8.68it/s]

     20%|█▉        | 185/938 [00:19<01:23,  9.02it/s]

     20%|█▉        | 187/938 [00:20<01:18,  9.54it/s]

     20%|██        | 188/938 [00:20<01:28,  8.47it/s]

     20%|██        | 190/938 [00:20<01:20,  9.27it/s]

     20%|██        | 192/938 [00:20<01:22,  9.08it/s]

     21%|██        | 193/938 [00:20<01:30,  8.22it/s]

     21%|██        | 194/938 [00:20<01:28,  8.43it/s]

     21%|██        | 195/938 [00:21<01:26,  8.61it/s]

     21%|██        | 196/938 [00:21<01:26,  8.59it/s]

     21%|██        | 198/938 [00:21<01:18,  9.45it/s]

     21%|██▏       | 200/938 [00:21<01:15,  9.83it/s]

     22%|██▏       | 202/938 [00:21<01:13, 10.06it/s]

     22%|██▏       | 204/938 [00:21<01:13, 10.05it/s]

     22%|██▏       | 206/938 [00:22<01:13,  9.94it/s]

     22%|██▏       | 208/938 [00:22<01:12, 10.01it/s]

     22%|██▏       | 210/938 [00:22<01:11, 10.18it/s]

     23%|██▎       | 212/938 [00:22<01:11, 10.19it/s]

     23%|██▎       | 214/938 [00:22<01:09, 10.37it/s]

     23%|██▎       | 216/938 [00:23<01:11, 10.08it/s]

     23%|██▎       | 218/938 [00:23<01:14,  9.61it/s]

     23%|██▎       | 220/938 [00:23<01:07, 10.58it/s]

     24%|██▎       | 222/938 [00:23<01:08, 10.51it/s]

     24%|██▍       | 224/938 [00:23<01:17,  9.25it/s]

     24%|██▍       | 226/938 [00:24<01:14,  9.55it/s]

     24%|██▍       | 228/938 [00:24<01:07, 10.47it/s]

     25%|██▍       | 230/938 [00:24<01:04, 11.04it/s]

     25%|██▍       | 232/938 [00:24<01:04, 10.87it/s]

     25%|██▍       | 234/938 [00:24<01:13,  9.61it/s]

     25%|██▌       | 236/938 [00:25<01:11,  9.86it/s]

     25%|██▌       | 238/938 [00:25<01:10, 10.00it/s]

     26%|██▌       | 240/938 [00:25<01:04, 10.74it/s]

     26%|██▌       | 242/938 [00:25<01:03, 11.03it/s]

     26%|██▌       | 244/938 [00:25<01:05, 10.61it/s]

     26%|██▌       | 246/938 [00:25<01:06, 10.34it/s]

     26%|██▋       | 248/938 [00:26<01:08, 10.12it/s]

     27%|██▋       | 250/938 [00:26<01:03, 10.89it/s]

     27%|██▋       | 252/938 [00:26<00:59, 11.54it/s]

     27%|██▋       | 254/938 [00:26<01:01, 11.07it/s]

     27%|██▋       | 256/938 [00:26<01:01, 11.09it/s]

     28%|██▊       | 258/938 [00:27<01:17,  8.80it/s]

     28%|██▊       | 259/938 [00:27<01:23,  8.15it/s]

     28%|██▊       | 261/938 [00:27<01:19,  8.56it/s]

     28%|██▊       | 262/938 [00:27<01:18,  8.60it/s]

     28%|██▊       | 263/938 [00:27<01:16,  8.81it/s]

     28%|██▊       | 264/938 [00:27<01:17,  8.70it/s]

     28%|██▊       | 266/938 [00:28<01:11,  9.35it/s]

     28%|██▊       | 267/938 [00:28<01:19,  8.44it/s]

     29%|██▊       | 269/938 [00:28<01:11,  9.31it/s]

     29%|██▉       | 271/938 [00:28<01:07,  9.91it/s]

     29%|██▉       | 273/938 [00:28<01:08,  9.75it/s]

     29%|██▉       | 275/938 [00:28<01:09,  9.60it/s]

     29%|██▉       | 276/938 [00:29<01:09,  9.48it/s]

     30%|██▉       | 277/938 [00:29<01:11,  9.28it/s]

     30%|██▉       | 278/938 [00:29<01:13,  9.01it/s]

     30%|██▉       | 280/938 [00:29<01:06,  9.86it/s]

     30%|███       | 282/938 [00:29<01:05, 10.01it/s]

     30%|███       | 284/938 [00:29<01:06,  9.76it/s]

     30%|███       | 285/938 [00:29<01:08,  9.57it/s]

     30%|███       | 286/938 [00:30<01:11,  9.14it/s]

     31%|███       | 287/938 [00:30<01:14,  8.77it/s]

     31%|███       | 289/938 [00:30<01:05,  9.84it/s]

     31%|███       | 291/938 [00:30<01:04, 10.07it/s]

     31%|███       | 293/938 [00:30<01:04, 10.06it/s]

     31%|███▏      | 295/938 [00:30<01:07,  9.59it/s]

     32%|███▏      | 296/938 [00:31<01:09,  9.18it/s]

     32%|███▏      | 297/938 [00:31<01:20,  7.95it/s]

     32%|███▏      | 298/938 [00:31<01:17,  8.28it/s]

     32%|███▏      | 299/938 [00:31<01:19,  8.03it/s]

     32%|███▏      | 300/938 [00:31<01:18,  8.12it/s]

     32%|███▏      | 301/938 [00:31<01:17,  8.21it/s]

     32%|███▏      | 302/938 [00:31<01:21,  7.78it/s]

     32%|███▏      | 303/938 [00:32<01:19,  8.01it/s]

     32%|███▏      | 304/938 [00:32<01:17,  8.18it/s]

     33%|███▎      | 305/938 [00:32<01:18,  8.08it/s]

     33%|███▎      | 306/938 [00:32<01:16,  8.29it/s]

     33%|███▎      | 308/938 [00:32<01:13,  8.60it/s]

     33%|███▎      | 309/938 [00:32<01:17,  8.15it/s]

     33%|███▎      | 310/938 [00:32<01:13,  8.51it/s]

     33%|███▎      | 311/938 [00:32<01:12,  8.63it/s]

     33%|███▎      | 312/938 [00:33<01:11,  8.75it/s]

     33%|███▎      | 314/938 [00:33<01:09,  9.00it/s]

     34%|███▎      | 316/938 [00:33<01:03,  9.77it/s]

     34%|███▍      | 318/938 [00:33<01:02,  9.93it/s]

     34%|███▍      | 320/938 [00:33<01:00, 10.24it/s]

     34%|███▍      | 322/938 [00:33<00:59, 10.28it/s]

     35%|███▍      | 324/938 [00:34<01:02,  9.76it/s]

     35%|███▍      | 325/938 [00:34<01:09,  8.77it/s]

     35%|███▍      | 327/938 [00:34<01:05,  9.27it/s]

     35%|███▍      | 328/938 [00:34<01:07,  9.05it/s]

     35%|███▌      | 329/938 [00:34<01:14,  8.17it/s]

     35%|███▌      | 330/938 [00:34<01:16,  7.97it/s]

     35%|███▌      | 332/938 [00:35<01:12,  8.36it/s]

     36%|███▌      | 333/938 [00:35<01:09,  8.74it/s]

     36%|███▌      | 334/938 [00:35<01:29,  6.77it/s]

     36%|███▌      | 336/938 [00:35<01:19,  7.58it/s]

     36%|███▌      | 337/938 [00:35<01:32,  6.53it/s]

     36%|███▌      | 338/938 [00:35<01:24,  7.09it/s]

     36%|███▌      | 340/938 [00:36<01:16,  7.77it/s]

     36%|███▋      | 341/938 [00:36<01:12,  8.22it/s]

     36%|███▋      | 342/938 [00:36<01:12,  8.18it/s]

     37%|███▋      | 344/938 [00:36<01:09,  8.51it/s]

     37%|███▋      | 346/938 [00:36<01:04,  9.20it/s]

     37%|███▋      | 347/938 [00:36<01:07,  8.78it/s]

     37%|███▋      | 348/938 [00:37<01:06,  8.81it/s]

     37%|███▋      | 350/938 [00:37<01:02,  9.34it/s]

     37%|███▋      | 351/938 [00:37<01:12,  8.12it/s]

     38%|███▊      | 352/938 [00:37<01:09,  8.38it/s]

     38%|███▊      | 353/938 [00:37<01:16,  7.61it/s]

     38%|███▊      | 354/938 [00:37<01:23,  7.03it/s]

     38%|███▊      | 356/938 [00:38<01:22,  7.09it/s]

     38%|███▊      | 357/938 [00:38<01:15,  7.69it/s]

     38%|███▊      | 358/938 [00:38<01:22,  7.03it/s]

     38%|███▊      | 359/938 [00:38<01:21,  7.11it/s]

     38%|███▊      | 360/938 [00:38<01:14,  7.73it/s]

     38%|███▊      | 361/938 [00:38<01:20,  7.20it/s]

     39%|███▊      | 362/938 [00:38<01:17,  7.42it/s]

     39%|███▊      | 363/938 [00:39<01:26,  6.61it/s]

     39%|███▉      | 364/938 [00:39<01:18,  7.28it/s]

## Sampling from the generative model

Now that we have trained the network, we can begin to sample from it. We simply give it some random noise distributed according to the prior $p(z) = \mathcal{N}(0, I)$ and send it through the decoder. This process generates a slew of samples that look like they come from the original distribution $p(x)$.

```python
model.eval()
```

```python
# Random samples
x_mu = model.sample(Variable(torch.randn(16, 32)))

f, axarr = plt.subplots(1, 16, figsize=(18, 12))
samples = x_mu.data.view(-1, 28, 28).numpy()
for i, ax in enumerate(axarr.flat):
    ax.imshow(samples[i])
    ax.axis("off")
```

```python
# Embed two random number s and interpolate between them
num1, num2 = 1, 7

x, y = next(iter(loader_validation))
ids_num1 = np.where(y.argmax(axis=1) == num1)[0]
ids_num2 = np.where(y.argmax(axis=1) == num2)[0]
x_num1 = x[ids_num1[0]]
x_num2 = x[ids_num2[0]]

with torch.no_grad():
    z_num1, mu_num1, logvar_num1 = model.encoder(x_num1)
    z_num2, mu_num2, logvar_num2 = model.encoder(x_num2)

# Interpolate and sample
z_interpolated = np.linspace(mu_num1, mu_num2, num=16)
x_mu = model.sample(Variable(torch.Tensor(z_interpolated)))

# Plot
f, axarr = plt.subplots(1, 16, figsize=(18, 12))
samples = x_mu.data.view(-1, 28, 28).numpy()
for i, ax in enumerate(axarr.flat):
    ax.imshow(samples[i])
    ax.axis("off")
```

```python

```

## Side note: Normalizing flows

We can get a more expressive variational approximation by using flows. In the regular variational autoencoder we are limited to a diagonal Gaussian distribution as the output of our encoder, but we can do better. Introducing a full-rank covariance matrix would be an option, but we will quickly see that it becomes computationally infeasible.

What we instead can do is to use change-of-variables to calculate the probability of a random sample under some transformation. In fact, if this transformation is made in a specific way, it is simple to calculate the probability of a sample under a series of $K$ transforms.

$$\ln q_K(z_K) = \ln q_0(z_0) − \sum_{k=1}^{K} \ln \bigg | \det \frac{\partial f_k}{\partial z_{k-1}} \bigg |$$

This is the essence of [Normalizing Flows (Rezende & Mohammed, 2015)](https://arxiv.org/abs/1505.05770). This process increases the expressability of the variational distribution and will likely give a better estimate of the marginal likelihood of the data $p(x)$. However, the parameters of each flow must be learned, meaning that this process will take a long time in order to perform well.

```python
from layers import NormalizingFlows

# Add a series of 16 normalizing flows to the model
# You can do this to *all* of the models in the library.
flow = NormalizingFlows(32, n_flows=16)
model.add_flow(flow)
```

```python
for epoch in range(50):
    model.train()
    total_loss = 0
    for (u, _) in tqdm(loader_train):
        u = Variable(u)

        if cuda:
            u = u.cuda(device=0)

        reconstruction = model(u)

        likelihood = -binary_cross_entropy(reconstruction, u)
        elbo = likelihood - model.kl_divergence

        L = -torch.mean(elbo)

        L.backward()
        optimizer.step()
        optimizer.zero_grad()

        total_loss += L.data[0]

    m = len(train)

    if epoch % 10 == 0:
        print(f"Epoch: {epoch}\tL: {total_loss/m:.2f}")
```

## Additional tips

If you want to work with a problem in which the data distribution $x$ is continuous, you can do so by changing the output activation of the decoder and using a different likelihood function, as shown below.

```python
import torch.nn as nn

model.decoder.output_activation = nn.Softmax()
loss = nn.CrossEntropyLoss()
```


```python
%pylab inline
from astropy.io import fits
import numpy as np
import matplotlib.pyplot as plt
import pylab as pl
import matplotlib
import pandas as pd
from scipy import interpolate
from sympy import *
import os
import io
```

    Populating the interactive namespace from numpy and matplotlib

    /share/soft/anaconda3/lib/python3.7/site-packages/IPython/core/magics/pylab.py:160: UserWarning: pylab import has clobbered these variables: ['trace', 'gcd', 'Number', 'add', 'cbrt', 'ifft', 'interactive', 'partition', 'fft', 'plotting', 'sin', 'test', 'vectorize', 'mod', 'array', 'sinh', 'gamma', 'reshape', 'deprecated', 'binomial', 'sqrt', 'tanh', 'power', 'sinc', 'product', 'source', 'erf', 'det', 'prod', 'ones', 'diff', 'log', 'solve', 'cos', 'diag', 'trunc', 'eye', 'seterr', 'floor', 'Polygon', 'flatten', 'take', 'Line2D', 'beta', 'zeros', 'roots', 'nan', 'multinomial', 'transpose', 'cosh', 'lcm', 'tan', 'exp', 're', 'Circle', 'var', 'sign', 'poly', 'conjugate', 'plot', 'invert', 'pi']
    `%matplotlib` prevents importing * from pylab and numpy
      "\n`%matplotlib` prevents importing * from pylab and numpy"

# some basic unchanged parameters

```python
d=200.        # diameter of the telescope, in cm unit

#if(keyword_set(teld)) then d=teld

obscure=0.0  #effective central obscuration, no unit
telarea=3.14159/4.0*d*d*(1.0-obscure)  #effective area of the telescope, cm^2
darkc=0.017   #dark current, in e/s/pix

#if(keyword_set(darkcurrent)) then darkc=darkcurrent

rn=4     #read noise, in e/pix

#if(keyword_set(readnoise)) then rn=readnoise

planckh=6.626    # 10^{-27} erg*s
cc=3.0   # speed of light, 10^{17} nm/s

```

# load the filters

```python
filtersel='bessell_V.par'   #'./sdss_g0.par'
#if(keyword_set(filter)) then filtersel=filter
filterpath='../obs/filters/'
filterfile=filterpath+filtersel
print(filterfile)
# ;fluxfilter: max=1, min=0, no particular unit
i=0
with open(filterfile,'r') as fh:
    for line in fh:
        if line.startswith('#'):
            i=i+1
            continue
band=pd.read_csv(filterfile,sep='\s+',header=None,skiprows=40)
wavefilter=np.array(band[0])
fluxfilter=np.array(band[1])
wavefilter=wavefilter/10.0  # in nm
vmin=wavefilter[0]
nw=len(wavefilter)
vmax=wavefilter[nw-1]

# find the central wavelength, effective wavelength, and FWHM of the given filter

filtermid=(vmax-vmin)*0.5  #nm, central wavelength
dwave=wavefilter[1:]-wavefilter[:-1]
filtereff=np.nansum(dwave*wavefilter[1:]*fluxfilter[1:])/np.nansum(dwave*fluxfilter[1:]) #nm, effective wavelength
rmax=np.max(fluxfilter)
nnn=np.where(fluxfilter > 0.5*rmax)[0]
FWHMmin=wavefilter[nnn[0]]
FWHMmax=wavefilter[nnn[-1]]
filterwid=FWHMmax-FWHMmin  #nm, FWHM
```

    ../obs/filters/bessell_V.par

# define wavelength array,
- cover the range of 350nm to 1050nm, depend on the spectral resolution wanted.

```python
#specr0=2000  ; no unit
#if(keyword_set(specr)) then specr0=specr
sampling=2.0    #pixels per spectral resolution element ?1D or 2D/linear or area?

#if(keyword_set(specsample)) then sampling=specsample

#delta_lambda=500.0/specr0/sampling  ; in the same unit as lambda0
delta_lambda=0.1755555
#if(keyword_set(deltal)) then delta_lambda=deltal # has to be in unit of nm
narray=int((1000.0-350.0)/delta_lambda)
#figure out the wavelength array length, from 350nm to 1000nm, spacing at delta_lambda
wavearr=350.0+delta_lambda*pl.frange(narray-1)
# select out the array of V band filter
ii=np.logical_and(wavearr >= vmin, wavearr <= vmax)
wavetmp2=wavearr[ii]
x=np.interp(wavetmp2,wavefilter,fluxfilter)
integratef4=x*wavetmp2
```

    /share/soft/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:12: MatplotlibDeprecationWarning: numpy.arange
      if sys.path[0] == '':

```python
def integral(x,y):
    nn=len(x)

    dx=x[1:]-x[:-1]
    yy=0.5*(y[1:]+y[:-1])

    return np.sum(dx*yy)

```

```python
integconst=integral(wavetmp2, integratef4) # int(lambda*Rlambda*dlambda)
```

```python
integconst
```

    44654.97566743437

# some less basic parameters, may change, but not often
- qsys=0.10	; throughput of the whole system, should be a function of lambda

```python
throughput=pd.read_csv('../obs/IFU_throughput.dat',sep='\s+',header=None,skiprows=1)
lambdaq=np.array(throughput[8])
qtot=np.array(throughput[9]) #; throughput of the whole system,
#if(not keyword_set(qinput)) then qinput=1.0
qinput=1.0
qe=0.8
#;assuming the total throughput cannot reach the theory value, 0.3 is the upper limit.
qtot[qtot>=0.3]=0.3
q=qtot*qinput #*qe ;qtot of CSST already includes the CCD efficiency
fovsp=0.2  # diameter of fiber (or spaxel) in arcsec ?
#fov2=3.14159/4.0*(0.2)^2      ; fiber area in (arcsec)^2
fov2=(fovsp)**2     #*3.14159/4.0      ; fiber (or spaxel) area in (arcsec)^2
#if(keyword_set(fovp)) then fov2=(fovp)^2;*3.14159/4.0
#   for slit (point source)
#if(keyword_set(slitwidth)) then fov2=1
slitunit=0.074  # arcsec. the length of slit which conresponds to a pixel length on IFU CCD
```

# SKY
 - define V band sky brightness

```python
iskyv0=22.5  # in Johnson V mag/arcsec^2 unit
#if(keyword_set(skyv)) then iskyv0=skyv
lambdav=filtereff   #in nm

#sky brightness corresponding to this sky magnitude
iskyv0_jy=3631.0*10**(-iskyv0/2.5+3.0)  # sky flux in V in mJy/arcsec^2 unit
iskyv0_nm=iskyv0_jy*3.0/(lambdav/100.0)**2 #sky flux in V in 10^(-13)erg/s/cm^2/nm (/arcsec^2 ?)

#readin the ground sky spectrum
skybg_50=pd.read_csv('../obs/skybg_50_10.dat',sep='\s+',header=None,skiprows=14)
wavesky=np.array(skybg_50[0])
fluxsky1=np.array(skybg_50[1])
fluxsky2=fluxsky1/wavesky*1.98 #change the sky flux unit to 10^(-13)erg/s/cm^2/nm/arcsec^2

#This fluxsky is in unit of phot/s/nm/arcsec^2/m^2, to convert it to F_lambda/arcsec^2,
#need to do fluxsky(phot/s/nm/arcsec^2/m^2)*h(6.625*10^{-27}erg.s)*nu(1/s)*10{-4}(m^2/cm^2)
#=fluxsky*c(3.0*10^{17}nm/s)/lambda(nm)*6.6*10{-31} erg/s/cm^2/nm/arcsec^2
#=fluxsky/lambda*1.98*10^{-13}erg/s/cm^2/nm/arcsec^2

#find out the normalization of the sky,
#normalization=iskyv0_nm*(integrate(bandpass*lambda*dlambda)/integrate(bandpass*lambda*F_sky_lambda*dlambda))
ii=np.logical_and(wavesky >= vmin, wavesky <= vmax)
wavetmp=wavesky[ii]
fluxtmp=fluxsky1[ii]

x=np.interp(wavetmp,wavefilter,fluxfilter)
vfluxtmp=x*fluxtmp*1.98  #bandpass*lambda*F_sky_lambda(fluxsky2)=bandpass*fluxsky*1.98, x10^(-13)
skyintegrate=integral(wavetmp, vfluxtmp)
skynorm=iskyv0_nm*integconst/skyintegrate
fluxsky3=np.interp(wavearr,wavesky,fluxsky2)
fluxsky=fluxsky3*skynorm   # get the sky spectrum in wavearr grid, the unit should now be the same as fluxvega: 10^(-13) erg/s/nm/cm^2  (/arcsec^2 ?)

fluxskypp=fluxsky

print(fluxskypp)
#a second way of estimating the Sky, if know the sky electron number per pixel

'''
 if(keyword_set(skyperpixel)) then begin
    ;since the numbers are given by the main survey, our detected Sky electron will be less, so scale a rough factor of 0.9
        scaletemp=0.9
        ii=where(wavearr ge 255 and wavearr le 400, counta)
        fluxskypp[ii]=0.028/counta
        ii=where(wavearr ge 400 and wavearr le 600, countb)
        fluxskypp[ii]=0.229/countb
        ii=where(wavearr ge 600 and wavearr le 900, countc)
        fluxskypp[ii]=0.301/countc
        ii=where(wavearr ge 900, countd)
        fluxskypp[ii]=0.301/countd
        fluxskypp=fluxskypp/0.074^2*fov2*scaletemp
    end
'''
```

    [3.26635242e-05 5.87930256e-04 5.87513766e-04 ... 9.80157011e-04
     1.01348412e-03 1.06961878e-03]

    '\n if(keyword_set(skyperpixel)) then begin\n    ;since the numbers are given by the main survey, our detected Sky electron will be less, so scale a rough factor of 0.9\n        scaletemp=0.9\n        ii=where(wavearr ge 255 and wavearr le 400, counta)\n        fluxskypp[ii]=0.028/counta  \n        ii=where(wavearr ge 400 and wavearr le 600, countb)\n        fluxskypp[ii]=0.229/countb  \n        ii=where(wavearr ge 600 and wavearr le 900, countc)\n        fluxskypp[ii]=0.301/countc  \n        ii=where(wavearr ge 900, countd)\n        fluxskypp[ii]=0.301/countd \n        fluxskypp=fluxskypp/0.074^2*fov2*scaletemp\n    end \n'

# define basic target brightness, parameters constantly change

```python
itarget=22. # in Johnson V mag/arcsec^2 unit
#if(keyword_set(targetmag)) then itarget=targetmag
itarget_jy=3631.0*10**(-itarget/2.5+3.0)  # target flux in V in mJy/arcsec^2 unit
itarget_nm=itarget_jy*3.0/(lambdav/100.0)**2 #target flux in V in 10^(-13)erg/s/cm^2/nm (/arcsec^2 ?)

#readin the galaxy spectrum
'''
    ; readcol, '../obs/allgalaxy.dat', wavegal, eflux, s0f, saf, sbf, scf, /silent
    ; wavegal=wavegal*1000.0
    ; spectype0=4	;default use Sb galaxy template spectrum
    ; if(keyword_set(spectype)) then spectype0=spectype  ;unless specified
    ; if(spectype0 eq 1) then galflux1=eflux
    ; if(spectype0 eq 2) then galflux1=s0f
    ; if(spectype0 eq 3) then galflux1=saf
    ; if(spectype0 eq 4) then galflux1=sbf
    ; if(spectype0 eq 5) then galflux1=scf
'''

tplfile='../obs/SFgal_tpl/SFgal_texp_FeH0_tau5_Ew10.fits'
# if(keyword_set(galtpl)) then tplfile='../obs/SFgal_tpl/'+galtpl
sfgal=fits.open(tplfile)
wavegal=sfgal[1].data['wave']/10. #change A to nm
galflux2=sfgal[1].data['flux']
galflux1=np.interp(wavearr,wavegal,galflux2)

#;normalize the galaxy spectrum to the V band magnitude specified.
ii=np.logical_and(wavegal >= vmin, wavegal <= vmax)
wavetmp=wavegal[ii]
fluxtmp=galflux2[ii]
x=np.interp(wavetmp,wavefilter,fluxfilter)
vfluxtmp=x*wavetmp*fluxtmp #bandpass*lambda*F_gal_lambda
galintegrate=integral(wavetmp,vfluxtmp)
galnorm=itarget_nm*integconst/galintegrate
galflux=galnorm*galflux1   # the unit should now be in 10^(-13)erg/s/nm/cm^2 (/arcsec^2 ?)
```

```python
sfgal.info()
```

    Filename: ../obs/SFgal_tpl/SFgal_texp_FeH0_tau5_Ew10.fits
    No.    Name      Ver    Type      Cards   Dimensions   Format
      0  PRIMARY       1 PrimaryHDU       4   ()
      1                1 BinTableHDU     16   5138R x 4C   ['D', 'D', 'D', 'D']

```python
sfgal[1].columns
```

    ColDefs(
        name = 'wave'; format = 'D'
        name = 'flux'; format = 'D'
        name = 'stellar'; format = 'D'
        name = 'gas_emission'; format = 'D'
    )

```python
sfgal[1].header
```

    XTENSION= 'BINTABLE'           / binary table extension
    BITPIX  =                    8 / array data type
    NAXIS   =                    2 / number of array dimensions
    NAXIS1  =                   32 / length of dimension 1
    NAXIS2  =                 5138 / length of dimension 2
    PCOUNT  =                    0 / number of group parameters
    GCOUNT  =                    1 / number of groups
    TFIELDS =                    4 / number of table fields
    TTYPE1  = 'wave    '
    TFORM1  = 'D       '
    TTYPE2  = 'flux    '
    TFORM2  = 'D       '
    TTYPE3  = 'stellar '
    TFORM3  = 'D       '
    TTYPE4  = 'gas_emission'
    TFORM4  = 'D       '

# define observation information, parameters constantly change

```python
obst=300.0  # in seconds, single integration time
#if(keyword_set(obstime)) then obst=obstime
repn=1.0   # repeating time
#if(keyword_set(repeatnum)) then repn=repeatnum
npixw=3.0
#if(keyword_set(npixel_width)) then npixw=npixel_width
#   sky of slit area (slitwidth*npixw*slitlength) will go into the CCD
#if(keyword_set(slitwidth)) then fluxskypp=fluxskypp*slitwidth*npixw*slitunit

expf2=np.zeros(narray)
expfemi=np.zeros(narray)
snarray=np.zeros(narray)
mockgal=np.zeros(narray)
tmp=np.zeros(narray)
lista=np.zeros(narray*10).reshape(narray,10)
```

```python
for i in range(narray):
    lambda0=wavearr[i]
    qlambda=np.interp(lambda0,lambdaq,q)
    hv=planckh*cc/lambda0 #;10^{-10}erg
    delta_hz=cc*delta_lambda/lambda0/lambda0*sampling #;10^17 1/s

    #now that many fluxes are in 10^(-13)erg/s/nm/cm^2, to convert it to Jy, need to multiple:
    #lambda0^2/c(in nm)=lambda0^2(nm)/(3.*10^(17))*10^(-13)erg/s/Hz/cm^2
    #=lambda^2(nm)*3.33*10^(-31)erg/s/Hz/cm^2=lambda^2(nm)*3.33*10^(-8)Jy

    #find out sky value at lambda0
    #calculate n_sky/pixel
    isky=fluxsky[i]*lambda0**2*0.0333*fov2   #in uJy/spaxel unit
    iskyall=isky*telarea/1000.0   #in 10-26 erg/s/Hz /spaxel
    fsky=qlambda*iskyall*delta_hz   #10^{-9} erg/s /spaxel
    nsky=fsky/hv*10.0   #in unit of #e/s /spaxel

    '''
    if(keyword_set(skyperpixel)) then begin
        nsky=fluxskypp[i]*sampling  ; #e/s in npixw*sampling pixels
    endif
    ;print, "Sky electron counts", nsky, nsky0, fluxskypp[i]
    '''

    #calculate n_source/pixel
    isource=galflux[i]*lambda0**2*0.0333*fov2   #in uJy/spaxel unit
    isall=isource*telarea/1000.0   #in 10-26 erg/s/Hz /spaxel
    fs=qlambda*isall*delta_hz   #10^{-9} erg/s /spaxel
    ns=fs/hv*10.0   #in unit of #e/s /spaxel
    #print, "Source electron counts", ns

    darkn=(darkc*repn*obst*npixw*sampling)
    rnn2=rn**2*(repn*npixw*sampling)
    sourcenn=(ns*repn*obst)
    skynn=(nsky*repn*obst)
    tmp[i]=skynn

    #nn1=sqrt(2.0*rnn^2+2.0*darkn^2+sourcenn^2+2.0*skynn^2)
    #nn1=sqrt(rnn^2+darkn^2+sourcenn^2+skynn^2)
    nn1=np.sqrt(rnn2+darkn+skynn+sourcenn)  #total noise
    sn1=repn*ns*obst/nn1  #S/N
    snarray[i]=sn1
    #nn=sqrt(2.0*rnn^2+2.0*darkn^2+2.0*skynn^2)
    nn=np.sqrt(rnn2+darkn+skynn)  #system noise
    #print, "total noise, system noise, sn, readnoise, dark, source, sky", nn1, nn, sn1, rnn, darkn, sourcenn, skynn

    #set the detection limit
    detlimit=1.0
    #if(keyword_set(snlimit)) then detlimit=snlimit
    #N_{source}/sqrt(N_{source}+nn^2)=detlimit, ==m
    #N_{source}^2-m^2*N_{source}-m^2*nn^2=0 (solve the equation)
    #N_{source}=(m^2+sqrt(m^4+4m^2*nn^2))/2.0
    nntmp=detlimit**2+np.sqrt(detlimit**4+4.0*detlimit**2*nn**2)
    nntmp=nntmp/2.0

    #calculate detection limit in uJy and mag

    fnn=nntmp
    f1=fnn/obst/repn    #in e/s
    f2=f1*hv       #in 10^{-10} erg/s
    f3=f2/delta_lambda #in 10^{-10} erg/s/nm
    f1=f3/telarea #in 10^{-10} erg/s/nm/cm^2
    f2=f1/qlambda    #in 10^{-10} erg/s/nm/cm^2
    expf2[i]=f2/fov2*100000.   # in 10^{-15} erg/s/nm/cm^2/arcsec^2
    #expfemi[i]=expf2[i]*delta_lambda*sampling   # in 10^{-15} erg/s/cm^2/arcsec^2, the above multiplied by the spectral resolution
    #print, "detection limit is", f2,"microJy/arcsec^2"
    #print, "detection limit is", magf2, "AB mag / arcsec^2"
    mockgal[i]=galflux[i]+galflux[i]/snarray[i]*np.random.randn(1,1)[0][0]  #in 10^{-13} erg/s/nm/cm^2

    lista[i,:]=[lambda0, sn1, galflux[i], nn1,\
    np.sqrt(sourcenn), nn, np.sqrt(rnn2),np.sqrt(darkn), \
    np.sqrt(skynn), mockgal[i]]

```

```python
'''
    ;mockgal=galflux+galflux/snarray*randomn(seed,narray)
    ; plot,wavearr,galflux,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='galaxy flux'
    ; oplot,wavearr,mockgal,color=250
    ; oplot,wavearr,galflux,thick=3
    ; label=strcompress(string(obst))+'s *'+strcompress(string(repn))+' times'
    ; xyouts,800,max(galflux),label,charsize=2
    ; plot,wavearr,expf2*0.01,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='expected galaxy flux'
    ; oplot,wavearr,galflux,color=250

    ; plot,wavearr,fluxsky,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='sky flux'
    ; plot,wavearr,tmp,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='#photon'

    ; ii=where(wavearr ge vmin and wavearr le vmax)
    ; wavetmp=wavearr(ii)
    ; fluxtmp=expf2(ii)/100000.  ;10^{-10} erg/s/cm^2/arcsec^2
    ; x=interpol(fluxfilter, wavefilter, wavetmp)
    ; vfluxtmp=x*wavetmp*fluxtmp  ;bandpass*lambda*F_gal_lambda
    ; gexpintegrate=integral(wavetmp, vfluxtmp)
    ; magf2=-2.5*(alog10((gexpintegrate*(lambdav/100.0)^2)/(integconst*3631.0*3.0)))
    ; print,'magf2=',magf2
    ; plot,wavetmp,fluxtmp,xrange=[350,1000],xs=1,xtitle='lambda',ytitle='expected galaxy flux'
'''
```

    "\n    ;mockgal=galflux+galflux/snarray*randomn(seed,narray)\n    ; plot,wavearr,galflux,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='galaxy flux'\n    ; oplot,wavearr,mockgal,color=250\n    ; oplot,wavearr,galflux,thick=3\n    ; label=strcompress(string(obst))+'s *'+strcompress(string(repn))+' times'\n    ; xyouts,800,max(galflux),label,charsize=2\n    ; plot,wavearr,expf2*0.01,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='expected galaxy flux'\n    ; oplot,wavearr,galflux,color=250\n    \n    ; plot,wavearr,fluxsky,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='sky flux'\n    ; plot,wavearr,tmp,xrange=[350,1000],xs=1,xtitle='lambda (nm)',ytitle='#photon'\n    \n    \n    ; ii=where(wavearr ge vmin and wavearr le vmax)\n    ; wavetmp=wavearr(ii)\n    ; fluxtmp=expf2(ii)/100000.  ;10^{-10} erg/s/cm^2/arcsec^2\n    ; x=interpol(fluxfilter, wavefilter, wavetmp)\n    ; vfluxtmp=x*wavetmp*fluxtmp  ;bandpass*lambda*F_gal_lambda\n    ; gexpintegrate=integral(wavetmp, vfluxtmp)\n    ; magf2=-2.5*(alog10((gexpintegrate*(lambdav/100.0)^2)/(integconst*3631.0*3.0)))\n    ; print,'magf2=',magf2\n    ; plot,wavetmp,fluxtmp,xrange=[350,1000],xs=1,xtitle='lambda',ytitle='expected galaxy flux'\n"

```python
ii=np.logical_and(wavearr >= FWHMmin , wavearr <= FWHMmax)
wavetmp=wavearr[ii]
if len(snarray[ii]) % 2 ==0:
    snm=sorted(list(snarray[ii]))[int(0.5*len(snarray[ii]))]
else:
    snm=np.median(snarray[ii])    #the median SN of FWHM range to acchieve the sn limit
im=np.where(snarray[ii] == snm)[0]
print(snarra)
fact=np.reshape(expf2[ii][im]*0.01/galflux[ii][im],1)
print(fact)
fact=fact[0]
print(fact)
limitmag=-2.5*np.log10(fact)+itarget

#print,limitmag
#oplot,wavetmp, fluxtmp, color=250
snmean=np.median(snarray)
print(snmean)
```

    [37.63560326]
    37.635603262237744
    0.05607035207145872

```python
z=0.0
#if(keyword_set(redshift)) then z=redshift
waveha=656.3*(1.0+z)
ii=np.logical_and(wavearr >= (waveha-0.5) , wavearr < (waveha+0.5))  #1nm ,10A
nii=len(np.where(ii==1)[0])
ii_1=np.logical_and(wavearr >= (waveha-10) , wavearr <= (waveha-5))
ii_2=np.logical_and(wavearr <= (waveha+10) , wavearr >= (waveha+5))
icont=np.logical_or(ii_1==1,ii_2==1)

contrms=np.sqrt(np.sum(mockgal[icont]**2)/len(mockgal[icont]))
print(contrms)
h=3*contrms*np.sqrt(nii) # hight>3 con
w=1.  # width of 10A
limitemif=np.sqrt(2*np.pi)*h*w  #h=3*cont, w=10A
#print,limtmag,limitemif
```

    0.008009696733561928

```python
print('The END!')
```

    The END!

```python
print('d:', d)
print('darkc:', darkc)
print('rn:', rn)
print(filterfile)
print('sampling:', sampling)
print('delta_lambda:', delta_lambda)
print('qinput:', qinput)
print('fov2:', fov2)
print('iskyv0:', iskyv0)
print('fluxskypp:', fluxskypp)
print('itarget:', itarget)
print('tplfile:',tplfile)
print('obst:',obst)
print('repn:',repn)
print('npixw:',npixw)
print('fluxskypp:',fluxskypp)
print('z:',z)
print(integconst)
```

    d: 200.0
    darkc: 0.017
    rn: 4
    ../obs/filters/bessell_V.par
    sampling: 2.0
    delta_lambda: 0.1755555
    qinput: 1.0
    fov2: 0.04000000000000001
    iskyv0: 22.5
    fluxskypp: [3.26635242e-05 5.87930256e-04 5.87513766e-04 ... 9.80157011e-04
     1.01348412e-03 1.06961878e-03]
    itarget: 22.0
    tplfile: ../obs/SFgal_tpl/SFgal_texp_FeH0_tau5_Ew10.fits
    obst: 300.0
    repn: 1.0
    npixw: 3.0
    fluxskypp: [3.26635242e-05 5.87930256e-04 5.87513766e-04 ... 9.80157011e-04
     1.01348412e-03 1.06961878e-03]
    z: 0.0
    44654.97566743437

# write dat

```python
iddat=np.array(list(range(10)))
namedat=np.array(['lambda','S/N','tar_flux','tot_noise','sc_noise', \
       'sys_noise', 'readnoise','dark_noise', 'sky_noise', 'mockgal'])
unit=np.array(['nm', ' ','1e-13 erg/s/cm2/nm',\
       '#e','#e','#e','#e','#e','#e', '1e-13 erg/s/cm2/nm'])
```

```python
[fits.Column(name=namedat[i],array=np.array(lista[:,i]),format='k') for i in range(len(namedat))]
```

    [name = 'lambda'; format = 'k',
     name = 'S/N'; format = 'k',
     name = 'tar_flux'; format = 'k',
     name = 'tot_noise'; format = 'k',
     name = 'sc_noise'; format = 'k',
     name = 'sys_noise'; format = 'k',
     name = 'readnoise'; format = 'k',
     name = 'dark_noise'; format = 'k',
     name = 'sky_noise'; format = 'k',
     name = 'mockgal'; format = 'k']

```python
hdr=fits.Header()
for i in range(len(namedat)):
    hdr[str(i)]=unit[i]
hun1=fits.PrimaryHDU(header=hdr)

hun2=fits.BinTableHDU.from_columns([fits.Column(name=namedat[i],array=np.array(lista[:,i]),format='1E') for i in range(len(namedat))])
hdulist = fits.HDUList([hun1,hun2])
hdulist.writeto('noise.fits')
```

# read fits

```python
aaa=fits.open('noise.fits')
```

```python
aaa.info()
```

    Filename: noise.fits
    No.    Name      Ver    Type      Cards   Dimensions   Format
      0  PRIMARY       1 PrimaryHDU      14   ()
      1                1 BinTableHDU     28   3702R x 10C   [1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E]

```python
aaa[0].header
```

    SIMPLE  =                    T / conforms to FITS standard
    BITPIX  =                    8 / array data type
    NAXIS   =                    0 / number of array dimensions
    EXTEND  =                    T
    0       = 'nm      '
    1       = ''
    2       = '1e-13 erg/s/cm2/nm'
    3       = '#e      '
    4       = '#e      '
    5       = '#e      '
    6       = '#e      '
    7       = '#e      '
    8       = '#e      '
    9       = '1e-13 erg/s/cm2/nm'

```python
aaa[1].data
```

    FITS_rec([(350.     , 0.00696866, 0.00048873, 11.255384 , 0.2800625 , 11.2519   , 9.797959, 5.531727, 0.07240252, -0.02861151),
              (350.17557, 0.00651509, 0.00045201, 11.259164 , 0.27084038, 11.255906 , 9.797959, 5.531727, 0.30888805,  0.04023228),
              (350.3511 , 0.00598966, 0.000411  , 11.2589445, 0.25968695, 11.25595  , 9.797959, 5.531727, 0.31048384,  0.05475378),
              ...,
              (999.3798 , 0.0174631 , 0.00034563, 11.285209 , 0.44393095, 11.276474 , 9.797959, 5.531727, 0.7475748 ,  0.01462729),
              (999.55536, 0.01739325, 0.00034563, 11.285915 , 0.44305614, 11.277216 , 9.797959, 5.531727, 0.75868005,  0.00467519),
              (999.7309 , 0.01732252, 0.00034563, 11.287188 , 0.44217926, 11.2785225, 9.797959, 5.531727, 0.7778652 ,  0.02839942)],
             dtype=(numpy.record, [('lambda', '>f4'), ('S/N', '>f4'), ('tar_flux', '>f4'), ('tot_noise', '>f4'), ('sc_noise', '>f4'), ('sys_noise', '>f4'), ('readnoise', '>f4'), ('dark_noise', '>f4'), ('sky_noise', '>f4'), ('mockgal', '>f4')]))

```python
# if this file has existed, I need remove it
if(os.path.exists('../results/noise_'+str(filtersel)+'.fits'))==1:
    os.remove('../results/noise_'+str(filtersel)+'.fits')
```

```python
bbb=fits.open('../results/noise_bessell_V.par_SFgal_texp_FeH0_tau5_Ew10.fits.fits')
```

```python
bbb.info()
```

    Filename: ../results/noise_bessell_V.par_SFgal_texp_FeH0_tau5_Ew10.fits.fits
    No.    Name      Ver    Type      Cards   Dimensions   Format
      0  PRIMARY       1 PrimaryHDU      14   ()
      1                1 BinTableHDU     28   3702R x 10C   [1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E, 1E]

```python
bbb[0].header
```

    SIMPLE  =                    T / conforms to FITS standard
    BITPIX  =                    8 / array data type
    NAXIS   =                    0 / number of array dimensions
    EXTEND  =                    T
    0       = 'nm      '
    1       = ''
    2       = '1e-13 erg/s/cm2/nm'
    3       = '#e      '
    4       = '#e      '
    5       = '#e      '
    6       = '#e      '
    7       = '#e      '
    8       = '#e      '
    9       = '1e-13 erg/s/cm2/nm'

```python
bbb[1].columns
```

    ColDefs(
        name = 'lambda'; format = '1E'
        name = 'S/N'; format = '1E'
        name = 'tar_flux'; format = '1E'
        name = 'tot_noise'; format = '1E'
        name = 'sc_noise'; format = '1E'
        name = 'sys_noise'; format = '1E'
        name = 'readnoise'; format = '1E'
        name = 'dark_noise'; format = '1E'
        name = 'sky_noise'; format = '1E'
        name = 'mockgal'; format = '1E'
    )

```python
plt.plot(bbb[1].data['lambda'],bbb[1].data['mockgal'])
```

```python
plt.plot(wavearr,galflux)
#plt.plot(wavearr,mockgal)
```

```python

```


# Introduction

In this tutorial, we will develop a model of semantic knowledge based on the same model from the simulation-based ACT-R [tutorial](http://act-r.psy.cmu.edu/software/). As explained below, the model organizes knowledge into a semantic network in which concepts are represented by nodes and relationships are represented by connections between nodes within the network.

## Task

In the task, participants must verify whether one category is a member of another category by responding "yes" or "no". For example, a person might recieve a question such as "Is a bird a canary?" and must respond "yes" or "no" with the appropriate keys.

# Semantic Model

As illustrated in the figure below, category relationships are represented as a semantic network in which nodes correspond to concepts and directed edges correspond to membership. The model can answer questions about category membership directly through a process called direct verification, or indirectly through a process called category chaining. For example, the model can answer the question "Is a bird an animal?" through direct verification by traversing from the bird node to the animal node using a single memory retrieval. By contrast, the question "is a canary an animal?" is answered indirectly through category chaining, which involves traversing from the canary node to the bird node and then from the bird node to the animal node using successive memory retrievals.

## Declarative Memory

Declarative memory $M$ consists of 20 chunks with the slots, (i.e., domain), $Q = \{\textrm{object, attribute, value}\}$. The object slot refers to the entity being evaluated (e.g. canary); the attribute slot refers to the particular property on which the object is being evaluated (e.g. category, flies); and the value slot refers to the particular value of the attribute (e.g. bird, true). For example, chunk 20 is $c_{20}(\rm{object}) = \textrm{bird}$, $c_{20}(\textrm{attribute}) =\textrm{category}$, $c_{20}(\textrm{value}) = \textrm{animal}$, which indicates that the object bird belongs to the category animal.

### Activation

Memory activation for chunk $m$ is defined as

\begin{equation}\label{eq:semantic_activation}
a_m = \textrm{blc} + \rho_m + \epsilon_m
\end{equation}

where $\textrm{blc}$ is the base-level constant and $\rho_m$ is partial matching. We will denote the expected value as $E[a_m] = \mu_m$. The base-level constant simply scales memory activation up or down. Partial matching allows chunks to be retrieved as a function of dissimilarity to the retrieval request values, $\mathbf{r}$. For simplicity, partial matching is defined as a weighted count of mismatching slot-values:

\begin{equation}\label{eq:penalty_function}
\rho_m = -\delta  \sum_{q \in Q_r} I^c\left(r(q),  c_{m}(q)\right)
\end{equation}

where $Q_r = \{\textrm{object},\textrm{attribute}\}$ is the set of slots in the retrieval request, the mismatch penalty parameter $\delta$ controls the steepness of the dissimilarity gradient, and $I^c$ is an indicator function:

$$ I^c(x,y) =
  \begin{cases}
    1      & x \neq y\\
    0  & x = y
  \end{cases}
$$

In cases where a chunk does not have a slot corresponding to a slot in the retrieval request, we treat it as a mismatch, i.e., $I^c(x, \emptyset)=1.$  Thus, chunks that are more similar to the retrieval request are more likely to be retrieved.

## Markov Process

We will represent the Semantic model as a discrete time Markov process. In a Markov process, a system transitions from state to state, such that the next state depends only on the current state. This constraint on state transitions is known as the Markov property. As is often the case, procedural knowledge is deterministic in the model. We will omit these states, which includes stimulus encoding, to simplify the model, as they do not change the predictions.  Instead, we focus on states that are the product of stochastic declarative memory processes. Thus, the state space for the Semantic model is

$S = \{s_{\rm ir}, s_{\rm cc1},   s_{\rm cc2},  s_{\rm yes},  s_{\rm no}\}$

The states are defined as follows:
- $s_{\rm ir}$ is the initial retrieval state
- $s_{\rm cc1}$ is the first category chain retrieval state
- $s_{\rm ccs}$ is the second category chain retrieval state
- $s_{\rm yes}$ is the state in which the model responds "yes"
- $s_{\rm no}$ is the state in which model responds "no".

A convenient way to visualize a Markov process is with a directed graph in which nodes represent states and edges (e.g. arrows) that indicate possible transitions. In the graph below, the process starts at $s_{\rm ir}$ from which point it can transition to states $s_{\rm cc1}$,  $s_{\rm yes}$, or  $s_{\rm no}$. The process terminates when a response is given, Thus, $s_{\rm yes}$, and $s_{\rm no}$ are absorbing states. If the model transitions from $s_{\rm ir}$, to  $s_{\rm cc1}$, it performs second retrieval based on category chaining. From $s_{\rm cc1}$, the model can transition to $s_{\rm yes}$ or $s_{\rm no}$ where a response is made, or to $s_{\rm cc2}$ where a third memory retrieval is performed through category chainining. From $s_{\rm cc2}$, the model can only transition to $s_{\rm yes}$ or $s_{\rm no}$. In principle, there is no limit to the number of category chain states, but for the stimuli consider here, the maximum is two.

## Retrieval Probability

Although many transitions in the model are deterministic based on the assumption of well-established procedural knowledge, other transitions are stochastic due to noise in declarative memory processes. In those cases, the probability of retrieving chunk $m$ given retrieval request $\mathbf{r}$ is computed from the approximation reported in Weaver (2008):

$\Pr(\mathbf{c}_m; \mathbf{r}) = \frac{e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}},$

where $\sigma = s\sqrt{2}$ controls activation noise and $s$ is the logistic scale parameter. Typically, we are interested in the probability of transitioning into a state rather than the probability of retrieving a particular chunk. Let $R_{s_i, s_j}$ be the set of chunks that would result in a transition from a state $s_i$ to a state, $s_j$. The probability of transitioning  from $s_i$ to $s_j$ is:

$\Pr(s_i\rightarrow s_j; \mathbf{r}) = \frac{ \sum_{m|\mathbf{c}_m \in R_{s_i, s_j}}e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}}.$

## Initial State Vector

After encoding the stimulus into chunk $\mathbf{c}_{s,\textrm{imaginal}}$, the Markov process begins in state $s_{\rm ir}$ with probability 1. Formally, this is represented with the following initial state vector:

Upon entering the state $s_{\rm ir}$, the initial retrieval request $\mathbf{r} =  \{(\rm object, \mathbf{c}_{s, \textrm{imaginal}}(\rm object), (attribute, category)\}$ is issued to declarative memory.

## Transition Matrix

The probabilities of transiting from state to state is organized into a matrix. Let $\mathbf{P}$ be the transition probability matrix, defined as:

We use $p_{r,c}$ to denote the probability of transitioning from state $r$ to state $c$. Note that $s_{\rm yes}$ and $s_{\rm no}$ are absorbing states, so their self-transitions occur with probability 1 and the probabilities of transitioning to other states are zero.  After encoding the stimulus into  chunk $\mathbf{c}_{s,\rm imaginal}$, the Markov process begins in state $s_{\rm ir}$ with probability 1, where the initial retrieval request $\mathbf{r} =  \{(\rm object, c_{s,\rm imaginal}(\rm object)), (attribute, category)\}$ is issued to declarative memory. Depending on the outcome of the initial retrieval request, the model can transition to state $s_{\rm cc1}$ with probability $p_{1,2}$, state $s_{\rm yes}$ with probability $p_{1,4}$, or state $s_{\rm no}$ with probability $p_{1,5}$. The mapping between the results of the retrieval request and $s_{\rm yes}$ is given by:
\begin{equation}
R_{{\rm ir},{\rm yes}} = \{\mathbf{c}_m \in M: \forall {q\in Q}, p_{j}(q) = c_m(q) \},
\label{eq:map_yes}
\end{equation}
where $\mathbf{p}_j = \{(\rm object, c_{s,\rm imaginal}(\rm object)),  (\rm attribute, category),(value, c_{s,\rm imaginal}(category))\}$. $R_{{\rm ir},{\rm yes}}$ contains a single chunk because $\mathbf{p}_j$ requires an exact match. The mapping between the result of the retrieval request and a category chain state is given by:
\begin{equation}
R_{{\rm ir},{\rm cci}} = \{\mathbf{c}_m  \in M: \forall {q\in Q}, p_k(q) = c_m(q) \},
\label{eq:map_cc}
\end{equation}
where $\mathbf{p}_k = \{(\rm object, c_{s,\rm imaginal}(\rm object)), (\rm attribute, category), (\rm value, \neg c_{s,\rm imaginal}(\rm category)\}$ denotes the conditions for production rule $k$.  The mapping between the result of the retrieval request and $s_{\rm no}$ is given by:
\begin{equation}
R_{{\rm ir},{\rm no}} = \{\mathbf{c}_m  \in M: \exists {q \in Q_{j}} \textrm{ s.t. } p_j(q) \neq c_m(q)\} \cup \mathbf{c}_{m^\prime},
\label{eq:map_no}
\end{equation}
where $Q_{j} = \{\rm object, attribute\}$. If the model transitions into absorbing states $s_{\rm yes}$ or $s_{\rm no}$, a response is emitted and the model terminates. However, if the model transitions to state $s_{\rm cc1}$, a new retrieval request $\mathbf{r}_{\rm cc1}$ is formed by assigning the value of the value slot of retrieved chunk $\mathbf{c}_{r1} \in R_{{\rm ir}, {\rm cc1}}$, which we denote $c_{r1}({\rm value})$, to the value of the object slot in the new retrieval request: $\mathbf{r}_{\rm cc1} = \{\rm (object, c_{r1}(value)), (attribute, category)\}$.
 In addition, $\mathbf{c}_{s, \rm imaginal}$ is modified after each category chain $i \in \{1,2\}$ as follows: $c_{s,\rm imaginal}(\rm object) = c_{ri}(value)$.

 For some stimuli considered here it is possible to transition to $s_{\rm cc2}$ where a second category chain is performed.  The mapping between the results of retrieval request $\mathbf{r}_{cc1}$ and state $s_{\rm cc2}$ is $R_{\rm cc1,cc2}$. In state $s_{\rm cc2}$, a new retrieval retrieval request $\mathbf{r}_{cc2} = \{\rm (object, c_{r2}), (attribute,category) \}$ is formed from retrieved chunk $c_{r2} \in R_{\rm cc1,cc2}$.

### A worked example

As a concrete example, consider the question for canary-animal in which the model arrives at the correct answer "yes" through the process of category chaining. The model encodes the stimulus into chunk $\mathbf{c}_{s,\rm imaginal} = \{\rm (object, canary), (category, animal)\}$ before issuing retrieval request $\mathbf{r} = \{(\textrm{object},\textrm{canary}), (\textrm{attribute},\textrm{category})\}$. Let $\mathbf{p}_{j} = \{(\textrm{object},\textrm{canary}), (\textrm{attribute},\textrm{category}), (\textrm{value},  \textrm{animal})\}$ be the conditions for the "yes" production rule, and $\mathbf{p}_{k} = \{(\textrm{object},\textrm{canary}), (\textrm{attribute},\textrm{category}), (\textrm{value}, \lnot \textrm{animal})\}$ be the conditions for the category chaining production rule.  Using the mappings defined above, the probability of transitioning from the initial retrieval state, $s_{\rm ir},$ to the first category chain retrieval state, $s_{\rm cc1},$ is given by,
$p_{1,2} = \frac{\sum_{m|\mathbf{c}_m  \in R_{\rm ir, cc1}} e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}}.$

The probability of transitioning from the initial retrieval state, $s_{\rm ir}$, to a no response, $s_{\rm no},$ is given by,

$p_{1,5} = \frac{\sum_{m|\mathbf{c}_m  \in R_{\rm ir,no}} e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}}.$

The model cannot transition from $s_{\rm ir}$ to $s_{\rm yes}$ to verify that a canary is an animal upon the first retrieval because $R_{\rm ir,yes} = \emptyset$ at this point in the process. Thus, $p_{1,4} = 0$, which implies $p_{1,5} = 1 - p_{1,2}$.

Let us assume that the model transitions from $s_{\rm ir}$ to  $s_{\rm cc1}$ by retrieving $\mathbf{c}_{r1} = \{(\rm object, canary),(\rm attribute, category), (\rm value, bird)\}$.  A new retrieval request $\mathbf{r}_{\rm cc1} = \{(\rm object, bird),(\rm attribute, category) \}$ is used to perform a retrieval for category chaining and the chunk in the imaginal buffer becomes  $\mathbf{c}_{s,\rm imaginal} = \{\rm (object, bird), (category, animal)\}$. From $s_{\rm cc1}$, the model can transition to $s_{\rm cc2}$ with probability $p_{2,3}$, to $s_{\rm yes}$ with probability $p_{2,4}$, or to $s_{\rm no}$ with probability $p_{2,5}$ depending on the result of the retrieval request $\mathbf{r}_{cc1}$. The probability of $p_{2,3} = 0$ because $R_{\rm cc1,cc2} = \emptyset$, i.e., there are no chunks in memory for which a bird is not an animal. The probability of transitioning to $s_{\rm yes}$ is,

$p_{2,4} = \frac{\sum_{m|\mathbf{c}_m \in R_{\rm cc1,yes}} e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}},$

which is obtained by retrieving $\mathbf{c}_{r2} = \{(\rm object, bird), (\rm attribute, category), (\rm value, animal)\}$. The complementary probability of transitioning to $s_{\rm no}$ is obtained by:

$p_{2,5} = \frac{\sum_{m|\mathbf{c}_m \in R_{\rm cc1, no}} e^{\mu_m/\sigma}}{\sum_{k|\mathbf{c}_k \in M} e^{\mu_k/\sigma} + e^{\mu_{m^\prime}/\sigma}} = 1-p_{2,4}$

#### Generating the Transition Matrix

In the code block below, we will generate the transition matrix for the question "Is a canary an animal?". Before doing so, we will first load the required dependencies.

```julia
cd(@__DIR__)
using Pkg
Pkg.activate("../../")
using StatsPlots, ACTRModels, Distributions, Turing, NamedArrays
include("Semantic_Model.jl")
Random.seed!(354301);
```

      Activating environment at `~/.julia/dev/ACTRFundamentalTools/Project.toml`

In the following example, the transition matrix is generated with the function `transition_matrix`. We begin by defining the `stimulus` and the model parameters in `parm`. Next we create an ACT-R model object and pass it to to `transition_matrix`. Lastly, we add labels to the transition matrix with `NamedArray`.

```julia
blc = 1.0
parms = (noise = true, τ = 0.0, s = 0.2, mmp = true, δ = 1.0)
stimulus = (object = :canary, category = :animal, ans = :yes)
# populate declarative memory
chunks = populate_memory()
# create declarative memory object
memory = Declarative(;memory=chunks)
# create act-r object
actr = ACTR(;declarative=memory, parms..., blc)
# generate transition matrix
tmat = transition_matrix(actr, stimulus, blc)
# states: initial retrieval, category chain 1, category chain 2, respond yes and respond no
states = ["Sir", "Scc1", "Scc2", "Syes", "Sno"]
# name the rows and columns
NamedArray(tmat, (states, states), ("t","t+1")) |> x->round.(x, digits=2)
```

    5×5 Named Matrix{Float64}
    t ╲ t+1 │  Sir  Scc1  Scc2  Syes   Sno
    ────────┼─────────────────────────────
    Sir     │  0.0   0.8   0.0   0.0   0.2
    Scc1    │  0.0   0.0   0.0   0.8   0.2
    Scc2    │  0.0   0.0   1.0   0.0   0.0
    Syes    │  0.0   0.0   0.0   1.0   0.0
    Sno     │  0.0   0.0   0.0   0.0   1.0

#### Response Probabilities

Based on the transition matrix above, there is one path to a "yes" response and two paths to a "no" response. The probability of responding yes is

$\rm Pr(\rm Response = yes) = p_{1,2}\times p_{2,4} = .80 \times .80 = .64$

The probability of responding "no" is

$\rm Pr(\rm Response = no) = \underbrace{p_{1,5}}_{\textrm{ no on initial retrieval}} + \underbrace{p_{1,2}\times p_{2,5}}_{\textrm{no on category chain}} = .20 + .80 \times .20 = .36$

As expected, $\rm Pr(\rm Response = no) = 1 - \rm Pr(\rm Response = yes)$.

# Generate Data

The data generation process uses four primary functions, shown below. Each function is annotated.  The function `simulate` is the top-level function that initializes the data generation process. It accepts the following inputs

- parms: a `NamedTuple` of fixed parameters
- stimulus: a `NamedTuple` of stimulus slot-value pairs
- n_reps: the number of trials or repetitions for a given stimulus
- blc: keyword argument for the base-level constant parameter.

`simulate` performs two primary functions. First, it initializes the ACT-R model with the desired parameters. Second, it iterates through simulated trials with the same stimulus and counts the number of correct responses `k`. `simulate` returns a `NamedTuple` containing the stimulus, the number of simulated trials, and the number of correct responses.

Inside the for loop of `simulate`, the function `simulate_trial` generates data for a single simulated trial. `simulate_trial` requires two inputs:

- `actr`: an ACT-R model object
- `stimulus`: a stimulus containing values for object and category

`simulate_trial` performs chained retrievals until a yes or no state is researched in the markov process. On each iteration of the while loop the model performs the following actions:

1. retrieve a chunk
2. evaluate whether to respond yes, respond no, or to chain the category in a subsequent memory retrieval

If the a retrieval failure occurs, `k` remains zero indicating a no response, `retrieving` is set to false and the simulation terminates for the present trial. If  the function `direct_verify` returns true, the model responds "yes". `direct_verify` corresponds to production rule $p_j$ described above. In this case, `k` = 1 and `retrieving` is set to false to terminate the simulation. If `chain_category` returns true, the model chains the category and attempts an additional memory retrieval by seting the object slot of the probe to the value of the object slot in the retrieved chunk. The remaining condition is that a mismatching chunk was retrieved, in which case `retrieving` is set to false, and `k` remains zero, indicating a "no" response.

```julia
function simulate(parms, stimulus, n_reps; blc)
    # populate declarative memory
    chunks = populate_memory()
    # generate declarative memory object
    memory = Declarative(;memory=chunks)
    # generate ACTR object
    actr = ACTR(;declarative=memory, parms..., blc)
    # the number of correct responses
    k = 0
    # count the number of correct answers, k
    for rep in 1:n_reps
        k += simulate_trial(actr, stimulus)
    end
    # return data that constains stimulus information, number of trials,
    # and correct answers
    return (stimulus..., N = n_reps, k = k)
end

function simulate_trial(actr, stimulus)
    retrieving = true
    # create memory probe or retrieval request
    probe = stimulus
    chunks = actr.declarative.memory
    # k = 1 if answer is "yes", 0 otherwise
    k = 0
    while retrieving
        # generate retrieval probabilities
        p,_ = retrieval_probs(actr; object=probe.object, attribute=:category)
        # sample a chunk index proportional to retrieval probabilities
        idx = sample(1:length(p), weights(p))
        # Last element corresponds to a retrieval failure
        # stop retrieval processes
        if idx == length(p)
            retrieving = false
        # retrieved chunk matches retrieval request, stop retrieving
        # and set k = 1 for "yes" response
        elseif direct_verify(chunks[idx], probe)
            retrieving = false
            k += 1
        # perform another retrieval with category chaining
        # modify the retrieval request based on the retrieved chunk
        elseif chain_category(chunks[idx], probe)
            probe = delete(probe, :object)
            probe = (object = chunks[idx].slots.value, probe...)
        # no chunks match, stop retrieving and respond "no" with k = 0
        else
            retrieving = false
        end
    end
    return k
end

"""
Answer yes via direct verification if retrieved chunk matches
probe on the object slot, the attribute slot equals category and the
value slot matches the value of the probe's category slot
"""
function direct_verify(chunk, probe)
    return match(chunk, object=probe.object,
        value=probe.category, attribute=:category)
end

"""
Chain category if retrieved chunk matches
probe on the object slot, the attribute slot equals category and the
value slot does not match the value of the probe's category slot
"""
function chain_category(chunk, probe)
    return match(chunk, ==, !=, ==, object=probe.object,
        value=probe.category, attribute=:category)
end
```

    chain_category

We will use the questions "Is a canary a bird" and "Is a canary an animal" as the stimuli for the simulation. In the code below, 10 trials are generated for each question. The data are formated as an array of `NamedTuples` where

- object: is the object slot

- category: the category slot

- ans: the correct answer

- N: the number of trials

- k: the number of trials in which yes is given as a response

```julia
# true blc value
blc = 1.0
# fixed parameters
parms = (noise = true, τ = 0.0, s = .2, mmp = true, δ = 1.0)
stimuli = get_stimuli()
n_reps = 10
data = map(x -> simulate(parms, x, n_reps; blc), stimuli)
```

    2-element Vector{NamedTuple{(:object, :category, :ans, :N, :k), Tuple{Symbol, Symbol, Symbol, Int64, Int64}}}:
     (object = :canary, category = :bird, ans = :yes, N = 10, k = 8)
     (object = :canary, category = :animal, ans = :yes, N = 10, k = 7)

## Define Likelihood Function

The probability of responding "Yes" is obtained from the stationary distribution vector, $\boldsymbol{\pi} \in \mathbb{R}^{1XN}$. The stationary distribution vector describes the limiting behavior of the Markov process, such that $\boldsymbol{\pi} = \lim_{n \to \infty}  \mathbf{s} \mathbf{P}^n$ and $\boldsymbol{\pi} = \boldsymbol{\pi}{P}$. The probability of responding "Yes" is given by $\theta = \pi_4$. The likelihood of responding "Yes" on $k$ trials out of a total of $n_t$ trials is given by the binomial likelihood function:

$\mathcal{L}(\theta ; k, n_t) = {n_t \choose k} \theta^k (1-\theta)^{n_t-k} $

The annotated code below impliments the likelihood function defined mathematically above. The function `computeLL` is the top-level function that computes the log likelihood across all data. The function `initial_state` generates the initial state vector. `transition_matrix` generates the transition matrix and the function `probability_yes` computes the log likelihood of $k$ "yes" responses out of $N$ trials using the binomial log PDF function. The most complex part of the code is the function `transition_matrix`. At a high level, the code matches the formal discription above, which involves the following steps

1. compute a vector p that represents the retrieval probability of each chunk
2. define the mapping between states by dividing chunk indices into sets that correspond to direct verification (yes response), category chain, and mismatching chunks (no response)
3. request chunk that matches category chain production rule
4. modify probe/retrieval request for the next category chain
5. continue steps 1-4 while there is a chunk that matches the category chain production rule

```julia
using Parameters, StatsBase, NamedTupleTools

import Distributions: logpdf, loglikelihood

struct Semantic{T1,T2} <: ContinuousUnivariateDistribution
    blc::T1
    parms::T2
end

loglikelihood(d::Semantic, data::Array{<:NamedTuple,1}) = logpdf(d, data)

Semantic(;blc, parms) = Semantic(blc, parms)

function initial_state(blc)
    s0 = zeros(typeof(blc), 5)
    s0[1] = 1
    return s0
end

function computeLL(parms, data; blc)
    act = zero(typeof(blc))
    # populate declarative memory
    chunks = populate_memory(act)
    # create declarative memory object
    memory = Declarative(;memory=chunks)
    # create act-r object
    actr = ACTR(;declarative=memory, parms..., blc)
    # create initial state vector
    s0 = initial_state(blc)
    LL = 0.0
    for d in data
        # create transition matrix
        tmat = transition_matrix(actr, d, blc)
        # compute probability of "yes"
        LL += probability_yes(tmat, s0, d)
    end
    return LL
end

function probability_yes(tmat, s0, d)
    z = s0' * tmat^3; θ = z[4]
    # sometimes θ is nan because of exponentiation of activation
    return isnan(θ) ? (return -Inf) : logpdf(Binomial(d.N, θ), d.k)
end

"""
populatates transition matrix consisting of 5 states:
* `s1`: initial retrieval
* `s2 `: chain category 1
* `s3`: chain category 2
* `s4`: respond yes
* `s5`: respond no
"""
function transition_matrix(actr, stim, blc)
    chunks = actr.declarative.memory
    Nc = length(chunks) + 1
    probe::typeof(stim) = stim
    probe = stim
    N = 5
    # populate transition matrix
    tmat = zeros(typeof(blc), N, N)
    # compute retrieval probabilities, p
    p,_ = retrieval_probs(actr; object=get_object(probe), attribute=:category)
    # find indices of chunks associated with direct verification, category chaining and mismatching conditions
    direct_indices = find_indices(actr, object=get_object(probe), value=get_category(probe))
    chain_indices = find_indices(actr, ==, !=, ==, object=get_object(probe), value=get_category(probe), attribute=:category)
    mismatch_indices = setdiff(1:Nc, direct_indices, chain_indices)
    # use indices to compute probability of category chain, direct verification (yes), and mismatch (no)
    tmat[1,2] = sum(p[chain_indices])
    tmat[1,4] = sum(p[direct_indices])
    tmat[1,5] = sum(p[mismatch_indices])
    # attempt to extract chunk associated with category chaining
    chain_chunk = get_chunks(actr,==,!=,==, object = get_object(probe),
    value = get_category(probe), attribute=:category)
    cnt = 1
    # continue the process above as long as category chaining can be performed.
    while !isempty(chain_chunk)
        cnt += 1
        probe = (object = get_chunk_value(chain_chunk[1]), delete(probe, :object)...)
        p,_ = retrieval_probs(actr; object=get_object(probe), attribute=:category)
        direct_indices = find_indices(actr, object=get_object(probe), value=get_category(probe))
        chain_indices = find_indices(actr, ==, !=, ==, object=get_object(probe), value=get_category(probe), attribute=:category)
        mismatch_indices = setdiff(1:Nc, direct_indices, chain_indices)
        tmat[cnt,2] = sum(p[chain_indices])
        tmat[cnt,4] = sum(p[direct_indices])
        tmat[cnt,5] = sum(p[mismatch_indices])
        chain_chunk = get_chunks(actr,==,!=,==, object = get_object(probe),
        value = get_category(probe), attribute=:category)
    end
    # set self-transitions to 1 if row i sums to 0.0
    map(i -> sum(tmat[i,:]) == 0.0 ? (tmat[i,i] = 1.0) : nothing, 1:size(tmat, 2))
    return tmat
end
```

    transition_matrix

## Define Model

The prior distributions and model is summarized as follows:

\begin{align}
\rm blc \sim Normal(1,1)
\end{align}

\begin{align}
\theta = \mathbf{\pi_4}
\end{align}

\begin{align}
k \sim \rm Binomial(\theta, N)
\end{align}

where $ \mathbf{\pi}$ is the stationary distribution and index 4 corresponds to state $s_{\rm yes}$. In computer code, the model is specified as follows:

```julia
@model model(data, parms) = begin
    blc ~ Normal(1.0, 1.0)
    data ~ Semantic(blc, parms)
end
```

    model (generic function with 1 method)

## Estimate Parameters

Now that the priors, likelihood and Turing model have been specified, we can now estimate the parameters. In the following code, we will run four MCMC chains with the NUTS sample for 2,000 iterations and omit the first 1,000 warmup samples.

```julia
# Settings of the NUTS sampler.
n_samples = 1500
n_adapt = 1500
specs = NUTS(n_adapt, 0.65)
n_chains = 4
chain = sample(model(data, parms), specs, MCMCThreads(), n_samples, n_chains, progress=true)
```

    ┌ Info: Found initial step size
    │   ϵ = 0.41250000000000003
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/y0DW3/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.4
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/y0DW3/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.4
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/y0DW3/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.4
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/y0DW3/src/inference/hmc.jl:188
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, true, false, true)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, true, false, true)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/yd6UP/src/hamiltonian.jl:47

    Chains MCMC chain (1500×13×4 Array{Float64, 3}):

    Iterations        = 1501:1:3000
    Number of chains  = 4
    Samples per chain = 1500
    Wall duration     = 13.72 seconds
    Compute duration  = 52.29 seconds
    parameters        = blc
    internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, max_hamiltonian_energy_error, tree_depth, numerical_error, step_size, nom_step_size

    Summary Statistics
      parameters      mean       std   naive_se      mcse         ess      rhat    ⋯
          Symbol   Float64   Float64    Float64   Float64     Float64   Float64    ⋯

             blc    1.5328    0.7085     0.0091    0.0204   1166.2415    1.0023    ⋯
                                                                    1 column omitted

    Quantiles
      parameters      2.5%     25.0%     50.0%     75.0%     97.5%
          Symbol   Float64   Float64   Float64   Float64   Float64

             blc    0.4439    0.9933    1.4321    1.9695    3.1418

## Results

In the summary output above, rhat is very close to 1, indicating convergence of the MCMC chains.The trace plot below also shows good mixing: the trace of each chain fluctuates randomly, and the traces are plotted on top of each other.

The autocorrelation plot in the second pannel shows low autocorrelation, indicating efficient sampling of the posterior distribution. In the third panel, the density plot shows that the posterior distribution of blc encompasses the data generating value of blc = 1 and is highly skewed.

```julia
pyplot()
ch = group(chain, :blc)
font_size = 12
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcτ = plot(p1, p2, p3, layout=(3,1), size=(600,600))
```

### Posterior Predictive Distribution

The plots below show the posterior predictive distributions for percent correct for the question "is a canary a bird", which requires no category chains and the question "is a canary an animal?", which requires 1 category chain. Comparing the distributions, we see that, on average, category chaining reduces accuracy. The reduction in accuracy can be attributed to the increase in opportunities to incorrectly respond "no": one opportunity following the initial retrieval and a second opportunity following the category chain retrieval.

```julia
font_size = 12
hit_rates(s) = posterior_predictive(x -> hit_rate(parms, s, n_reps; x...), chain, 1000)
preds = map(s -> hit_rates(s), stimuli)
predictive_plot = histogram(preds, xlabel="% Correct" ,ylabel="Probability", xaxis=font(font_size), yaxis=font(font_size),
    grid=false, color=:grey, leg=false, titlefont=font(font_size), xlims=(0,1.1),
    layout=(2,1), ylims=(0,0.4), normalize=:probability, size=(600,600), title=["Is a canary a bird?" "Is a canary an animal?"])
```

# References

Weaver, R. (2008). Parameters, predictions, and evidence in computational modeling: A statistical view informed by ACT–R. Cognitive Science, 32(8), 1349-1375.


```python
import numpy as np
import sympy as sym
import numba
import pydae.build as db
```

```python

```

## Formulation

Backward solution:

$$
\begin{split}
\mathbf {\dot x}  &  =  \mathbf {f (x,y^{ini},u^{ini}) } \\
\mathbf 0 & =  \mathbf {g (x,y^{ini},u^{ini}) }
\end{split}
$$

Foreward solution:

$$
\begin{split}
\mathbf {\dot x}  &  =  \mathbf {f (x,y^{run},u^{run}) } \\
\mathbf 0 & =  \mathbf {g (x,y^{run},u^{run}) }
\end{split}
$$

### Auxiliar  equations

\begin{eqnarray}
v_d &=& v_1\sin(\delta - \theta_1) \\
v_q &=& v_1\cos(\delta - \theta_1)
\end{eqnarray}

\begin{eqnarray}
p_e = i_d \left(v_d + R_a i_d\right) + i_q \left(v_q + R_a i_q\right)
\end{eqnarray}

### Differential  equations

$$
\begin{eqnarray}
 f_1 = \dot \delta &=&  \Omega_b \left( \omega -1 \right) \\
 f_2 = \dot \omega &=&  \frac{1}{2H} \left( p_m - p_e - D   \left( \omega - 1 \right) \right)  \\
 f_3 = \dot e_q &=& \frac{1}{T'_{d0}} \left( -e'_q - \left(X_d - X'_d \right) i_d + v_f^\star \right) \\
 f_4 = \dot e_d &=&  \frac{1}{T'_{q0}} \left( -e'_d - \left(X_q - X'_q \right) i_q \right)
\end{eqnarray}
$$

### Algebraic equations

$$
\begin{eqnarray}
g_1 &=& v_q + R_a i_q + X_d' i_d - e_q \\
g_2 &=& v_d + R_a i_d - X_q' i_q - e_d \\
g_3 &=& p_t - \left(v_1 v_0 \sin \left(\theta_1 - \theta_0 \right) \right)/X_l \\
g_4 &=& q_t + \left(v_1 v_0 \cos \left(\theta_1 - \theta_0 \right) \right)/X_l - v_1^2/X_l \\
g_5 &=& i_d v_d + i_q v_q - p_t \\
g_6 &=& i_d v_q - i_q v_d - q_t
\end{eqnarray}
$$

The dynamic states are:

$$
\mathbf{f} =
\left[
\begin{array}{c}
f_1\\
f_2\\
f_3\\
f_4
\end{array}
\right]
\;\;\;\;\;\;
\mathbf{g} =
\left[
\begin{array}{c}
g_1\\
g_2\\
g_3\\
g_4\\
g_5\\
g_6
\end{array}
\right]
\;\;\;\;\;\;
$$

$$
\mathbf x = \left[
\begin{array}{c}
\delta \\
\omega \\
 e_q'\\
 e_d'
\end{array} \right]
\;\;\;\;
\mathbf {y^{ini}} = \left[
\begin{array}{c}
i_d\\
i_q\\
v_1\\
\theta_1\\
p_m\\
v_f
\end{array} \right]
\;\;\;\;
\mathbf {y^{run}} = \left[
\begin{array}{c}
i_d\\
i_q\\
v_1\\
\theta_1\\
p_t\\
q_t
\end{array} \right]
\;\;\;\;
\mathbf {u^{ini}} = \left[
\begin{array}{c}
 p_t\\
 q_t
\end{array} \right]
\;\;\;\;
\mathbf {u^{run}} = \left[
\begin{array}{c}
p_m \\
v_f
\end{array} \right]
$$

### Outputs

$$
\mathbf{h} =
\left[
\begin{array}{c}
p_e \\
p_m
\end{array}
\right]
\;\;\;\;\;\;
\mathbf{z} =
\left[
\begin{array}{c}
p_e\\
p_m
\end{array}
\right]
$$

## System definition

```python
params_dict = {'X_d':1.81,'X1d':0.3,'T1d0':8.0,
               'X_q':1.76,'X1q':0.65,'T1q0':1.0,
               'R_a':0.003,'X_line': 0.05,
               'H':3.5,'D':1.0,
               'Omega_b':2*np.pi*50,'omega_s':1.0,
               'v_0':1.0,'theta_0':0.0, 'B_shunt':0.0}

u_ini_dict = {'p_t':0.8, 'q_t':0.2}  # for the initialization problem
u_run_dict = {'p_m':0.8,'v_f':1.0}  # for the running problem (here initialization and running problem are the same)

x_list = ['delta','omega','e1q','e1d']    # [inductor current, PI integrator]
y_ini_list = ['i_d','i_q','v_1','theta_1','p_m','v_f'] # for the initialization problem
y_run_list = ['i_d','i_q','v_1','theta_1','p_t','q_t'] # for the running problem (here initialization and running problem are the same)

sys_vars = {'params_dict':params_dict,
            'u_list':u_run_dict,
            'x_list':x_list,
            'y_list':y_run_list}

exec(db.sym_gen_str())  # exec to generate the required symbolic varables and constants
```

```python
# Auxiliar equations
v_d = v_1*sin(delta - theta_1)
v_q = v_1*cos(delta - theta_1)

p_e = i_d*(v_d + R_a*i_d) + i_q*(v_q + R_a*i_q)

# Differential equations
ddelta = Omega_b*(omega - omega_s)
domega = 1/(2*H)*(p_m - p_e - D*(omega - omega_s))
de1q = 1/T1d0*(-e1q - (X_d - X1d)*i_d + v_f)
de1d = 1/T1q0*(-e1d + (X_q - X1q)*i_q)

# Algebraic equations
g_1 = v_q + R_a*i_q + X1d*i_d - e1q
g_2 = v_d + R_a*i_d - X1q*i_q - e1d
g_3 = p_t - (v_1*v_0*sin(theta_1 - theta_0))/X_line
g_4 = q_t + (v_1*v_0*cos(theta_1 - theta_0))/X_line - v_1**2/X_line - v_1**2*B_shunt
g_5 = i_d*v_d + i_q*v_q - p_t
g_6 = i_d*v_q - i_q*v_d - q_t

# Outputs
h_1 = p_m
h_2 = p_e

# System dictionary
sys = {'name':'smib_milano_ex8p1_4ord',
       'params_dict':params,
       'f_list':[ddelta,domega,de1q,de1d],
       'g_list':[g_1,g_2,g_3,g_4,g_5,g_6],
       'x_list':x_list,
       'y_ini_list':y_ini_list,
       'y_run_list':y_run_list,
       'u_run_dict':u_run_dict,
       'u_ini_dict':u_ini_dict,
       'h_dict':{'p_m':h_1,'p_e':h_2}}

sys = db.system(sys)
db.sys2num(sys)      # building system module

```

```python

```