## pomegranate: fast and flexible probabilistic modelling

Author: Jacob Schreiber

Contact: <jmschreiber91@gmail.com>

```python
%pylab inline
import time
import pandas
import random
import seaborn; seaborn.set_style('whitegrid')
import itertools

from pomegranate import *

random.seed(0)
numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,pomegranate
```

    Populating the interactive namespace from numpy and matplotlib
    Sun Sep 30 2018

    numpy 1.14.2
    scipy 1.0.0
    pomegranate 0.10.0

    compiler   : GCC 7.2.0
    system     : Linux
    release    : 4.15.0-34-generic
    machine    : x86_64
    processor  : x86_64
    CPU cores  : 4
    interpreter: 64bit

pomegranate is a probabilistic modeling library for Python that aims to be easy to use, flexible, and fast. It is easy to use because it maintains a consistent and minimal API that mirrors the scikit-learn API when possible. It is flexible because it allows complicated probabilistic models to be built from simpler ones by easily stacking components on top of each other. It is fast due to its Cython backend which allows for fast numerics and multi-threading without cluttering the interface. pomegranate was developed at the University of Washington with the generous support from the following organizations:

### The API

One of the core tenant of pomegranate is that everything is fundamentally a probability distribution and should be treated the same way. Most people are familiar with simple probability distributions such as the normal or the uniform distribution. However, a mixture of many simple distributions is still a probability distribution because the density under it still adds up to 1. A hidden Markov model is still a probability distribution over sequences because the probability of all potential sequences must add up to 1. A Bayesian network is literally a probability distribution that has been factorized along a graphical structure, with a discrete Bayesian network factorizing a very large joint probability table and a linear Gaussian Bayesian network factorizing a multivariate Gaussian distribution that has many 0's in the inverse covariance matrix.

A benefit of considering all methods as probability distributions is that it simplifies and unifies the API across all of the models. The common API is the following methods. Let's specify some distribution and take a look.

```python
model = NormalDistribution(5, 1)
```

***model.probability(X)*** and ***model.log_probability(X)***

These methods return the probability of a single sample or a vector of probabilities if provided a vector of samples. This is equal to P(D|M) where D is the data and M is the model.

```python
print(model.probability([4., 6., 7.]))
print(model.log_probability([4., 6., 7.]))
```

    [0.24197072 0.24197072 0.05399097]
    [-1.41893853 -1.41893853 -2.91893853]

***model.sample(n=1)***

This method will return a random sample from the probability distribution or a vector of random samples if n is set to be greater than 1.

```python
model.sample(n=10)
```

    array([6.76405235, 5.40015721, 5.97873798, 7.2408932 , 6.86755799,
           4.02272212, 5.95008842, 4.84864279, 4.89678115, 5.4105985 ])

***model.fit(X, weights=None, inertia=0.0, other arguments)***

This method will fit the distribution to the data. For simple distributions and Bayesian networks this corresponds to weighted maximum likelihood estimates. For other compositional distributions such as hidden Markov models this corresponds to an iterative expectation-maximization allgorithm. Optionally, weights can be provided for each of the samples. Different models will have different arguments that can be specified as well.

```python
X = numpy.random.normal(7, 2, size=(100,))

model.fit(X)
model
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            7.036955079959913,
            2.0668514243367655
        ],
        "name" :"NormalDistribution"
    }

***model.summary(X, weights=None)***

This method implements the first part of the fitting process, which is summarizing a batch of data down to its sufficient statistics and storing them. These sufficient statistics are additive and will be updated for each successive batch that's seen. The sufficient statistics can be reset either through a call to `from_summaries` or a call to `clear_summaries`.

```python
X = numpy.random.normal(8, 1.5, size=(100,))

model.summarize(X)
model
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            7.036955079959913,
            2.0668514243367655
        ],
        "name" :"NormalDistribution"
    }

***model.from_summaries(inertia=0.0)***

This method implements the second part of the fitting process, which is using the stored sufficient statistics in order to update the model parameters.

```python
model.from_summaries()
model
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            8.068146018374335,
            1.4490322240381275
        ],
        "name" :"NormalDistribution"
    }

***model.clear_summaries()***

This method resets the sufficient statistics stored to the model.

```python
X = numpy.random.normal(3, 0.2, size=(100,))

model.summarize(X)
model.clear_summaries()
model.from_summaries()
model
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            8.068146018374335,
            1.4490322240381275
        ],
        "name" :"NormalDistribution"
    }

***Model.from_samples(distributions, n_components, X, weights=None)*** or ***Model.from_samples(X, weights=None)***

This class method will initialize and then fit the parameters of a model to some data. This differs from the `fit` function in that the `fit` function will update the parameters of a pre-defined model whereas the `from_samples` method will initialize a model to data and return the best parameters given the data.

```python
X = numpy.random.normal(6, 1, size=(250, 1))

model = NormalDistribution.from_samples(X)
model
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            5.878631757664192,
            1.0120288481111088
        ],
        "name" :"NormalDistribution"
    }

```python
model = GeneralMixtureModel.from_samples(NormalDistribution, 3, X)
model
```

    {
        "weights" : [
            0.3148521575093671,
            0.42381846635660686,
            0.261329376134026
        ],
        "distributions" : [
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    5.901815709887363,
                    0.47865628749453537
                ],
                "name" : "NormalDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    5.175959799664007,
                    0.8774489898709738
                ],
                "name" : "NormalDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    6.9902780582522634,
                    0.597808757836137
                ],
                "name" : "NormalDistribution"
            }
        ],
        "class" : "GeneralMixtureModel"
    }

***model.to_json(separators=(',', ' :'), indent=4)***

This method returns the JSON serialization of the distribution as a string.

```python
print(model.to_json())
```

    {
        "weights" : [
            0.3148521575093671,
            0.42381846635660686,
            0.261329376134026
        ],
        "distributions" : [
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    5.901815709887363,
                    0.47865628749453537
                ],
                "name" : "NormalDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    5.175959799664007,
                    0.8774489898709738
                ],
                "name" : "NormalDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    6.9902780582522634,
                    0.597808757836137
                ],
                "name" : "NormalDistribution"
            }
        ],
        "class" : "GeneralMixtureModel"
    }

***Model.from_json(s)***

This class method returns the deserialization of the JSON string back to the model object. s can either be the JSON string or a filename ending in `.json` to read from.

```python
model = NormalDistribution(5, 2)

model2 = Distribution.from_json(model.to_json())
model2
```

    {
        "frozen" :false,
        "class" :"Distribution",
        "parameters" :[
            5.0,
            2.0
        ],
        "name" :"NormalDistribution"
    }

Compositional models, i.e., those that are not simple distributions, have three additional methods whose named are inspired by scikit-learn. These methods relate to the posterior probabilities P(M|D) of each of the components of the model given some data.

```python
d1 = ExponentialDistribution(5.0)
d2 = ExponentialDistribution(0.3)

model = GeneralMixtureModel([d1, d2])
model
```

    {
        "weights" : [
            0.5,
            0.5
        ],
        "distributions" : [
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    5.0
                ],
                "name" : "ExponentialDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    0.3
                ],
                "name" : "ExponentialDistribution"
            }
        ],
        "class" : "GeneralMixtureModel"
    }

***model.predict(X)***

This method returns the most likely component for each sample. In the case of a mixture model it returns the component that is most likely, in the case of a hidden Markov model it returns the most likely component for each observation in the sequence.

```python
X = numpy.random.exponential(3, size=(10,1))

model.predict(X)
```

    array([1, 0, 0, 0, 1, 0, 0, 1, 0, 1], dtype=int32)

***model.predict_proba(X)***

This method returns the probability of each component for each sample. It is similar to the `predict` method except that it returns the probabilities instead of simply the most likely component.

```python
model.predict_proba(X)
```

    array([[0.00046883, 0.99953117],
           [0.74132615, 0.25867385],
           [0.58846532, 0.41153468],
           [0.89449455, 0.10550545],
           [0.        , 1.        ],
           [0.94041259, 0.05958741],
           [0.50991095, 0.49008905],
           [0.00002574, 0.99997426],
           [0.83434281, 0.16565719],
           [0.        , 1.        ]])

***model.predict_log_proba(X)***

Like predict_proba except that it returns the log probabilities instead of the probabilities.

```python
model.predict_log_proba(X)
```

    array([[ -7.66527111,  -0.00046894],
           [ -0.29931461,  -1.35218726],
           [ -0.53023729,  -0.88786198],
           [ -0.11149647,  -2.24899266],
           [-46.97709403,   0.        ],
           [ -0.06143657,  -2.820311  ],
           [ -0.67351918,  -0.71316817],
           [-10.56732239,  -0.00002574],
           [ -0.18111092,  -1.79783472],
           [-27.30692393,  -0.        ]])

### Flexibility

#### Modeling different features as different distributions

A second benefit of treating all models as probability distributions is that it greatly increases the flexibility that pomegranate provides. When people build naive Bayes classifiers, they typically will use a Gaussian distribution. However, there's no reason that one has to use a Gaussian distribution, you can drop in any type of distribution that you'd like. For example:

```python
X = numpy.random.normal(5, 1, size=(100, 2))
X[50:] += 1

y = numpy.zeros(100)
y[50:] = 1

model1 = NaiveBayes.from_samples(NormalDistribution, X, y)
model2 = NaiveBayes.from_samples(LogNormalDistribution, X, y)
```

It is easy to drop in whatever probability distribution you'd like because it should be easy. Mathematically, the naive Bayes model relies on Bayes' rule, which says:

\begin{equation}
P(M|D) = \frac{P(D|M)P(M)}{P(D)}
\end{align}

Because the "naive" part of a "naive" Bayes model means that the model treats each feature independently, we can rewrite the $P(D|M)$ aspect as the product of these probabilities over all $d$ features:

\begin{equation}
P(M|D) = \frac{P(M) \prod\limits_{i=1}^{d} P(D_{i}|M)}{P(D)}
\end{equation}

Now, because each feature is independent, they can be modeled by different probability distributions.

```python
mu = numpy.random.normal(7, 2, size=1000)
std = numpy.random.lognormal(-0.8, 0.8, size=1000)
dur = numpy.random.exponential(50, size=1000)

data = numpy.concatenate([numpy.random.normal(mu_, std_, int(t)) for mu_, std_, t in zip(mu, std, dur)])

plt.figure(figsize=(14, 4))
plt.title("Randomly Generated Signal", fontsize=16)
plt.plot(data)
plt.xlabel("Time", fontsize=14)
plt.ylabel("Signal", fontsize=14)
plt.xlim(0, 3000)
plt.show()
```

If someone was trying to model this signal, they could potentially try to segment it and then extract properties of those segments, such as the mean, the variance, and the duration. We can plot what thos would look like:

```python
plt.figure(figsize=(12, 3))
plt.subplot(131)
plt.title("mu", fontsize=14)
plt.hist(mu, bins=numpy.arange(0, 15))

plt.subplot(132)
plt.title("sigma", fontsize=14)
plt.hist(std, bins=numpy.arange(0.00, 1.75, 0.05))

plt.subplot(133)
plt.title("Duration", fontsize=14)
plt.hist(dur, bins=numpy.arange(0, 150, 10))
plt.show()
```

We can see that the mean of the segments does have a Gaussian distribution, but that neither the standard deviation or the duration do. It would be suboptimal to assume all features were Gaussian, as a simple Gaussian naive Bayes model would. pomegranate allows you to define a different distribution for each feature, like the following:

```python
X1 = numpy.array([numpy.random.normal(7, 2, size=400),
                  numpy.random.lognormal(-0.8, 0.8, size=400),
                  numpy.random.exponential(50, size=400)]).T

X2 = numpy.array([numpy.random.normal(8, 2, size=600),
                  numpy.random.lognormal(-1.2, 0.6, size=600),
                  numpy.random.exponential(100, size=600)]).T

X = numpy.concatenate([X1, X2])
y = numpy.zeros(1000)
y[400:] = 1

NaiveBayes.from_samples([NormalDistribution, LogNormalDistribution, ExponentialDistribution], X, y)
```

    {
        "models" : [
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    [
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                6.905835442537852,
                                2.09604452629164
                            ],
                            "name" : "NormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                -0.880499005246798,
                                0.7642112111076668
                            ],
                            "name" : "LogNormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                0.01979944881580867
                            ],
                            "name" : "ExponentialDistribution"
                        }
                    ],
                    [
                        1.0,
                        1.0,
                        1.0
                    ]
                ],
                "name" : "IndependentComponentsDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    [
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                8.03534029261696,
                                2.063613757125447
                            ],
                            "name" : "NormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                -1.2016720901139963,
                                0.5891919855702662
                            ],
                            "name" : "LogNormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                0.010167268183697146
                            ],
                            "name" : "ExponentialDistribution"
                        }
                    ],
                    [
                        1.0,
                        1.0,
                        1.0
                    ]
                ],
                "name" : "IndependentComponentsDistribution"
            }
        ],
        "weights" : [
            0.4,
            0.6
        ],
        "class" : "NaiveBayes"
    }

Modeling each feature independently is allowed for naive Bayes, mixtures, and hidden Markov models. This is very useful when trying to capture different dynamics in different features.

#### Stacking models on top of each other

Next, another feature that emerges when you treat all models as probability distributions is that they can be easily stacked within each other. For example, if a mixture is just a probability distribution, then naturally you should be able to make a mixture Bayes classifier by dropping a `GeneralMixtureModel` into a `BayesClassifier` just as easily as a Gaussian Bayes classifier by dropping a `MultivariateGaussianDistribution` into the `BayesClassifier`.

```python
X = numpy.concatenate([numpy.random.normal((5, 1), 1, size=(200, 2)),
                       numpy.random.normal((6, 4), 1, size=(200, 2)),
                       numpy.random.normal((3, 5), 1, size=(350, 2)),
                       numpy.random.normal((7, 6), 1, size=(250, 2))])

y = numpy.zeros(1000)
y[400:] = 1

model = BayesClassifier.from_samples(MultivariateGaussianDistribution, X, y)
print model.log_probability(X).sum()

d1 = GeneralMixtureModel.from_samples(MultivariateGaussianDistribution, 2, X[y == 0])
d2 = GeneralMixtureModel.from_samples(MultivariateGaussianDistribution, 2, X[y == 1])
model2 = BayesClassifier([d1, d2], [0.4, 0.6])
print model2.log_probability(X).sum()
```

    -4012.4121095570604
    -3904.7230177949655

In this case we build data that intentionally has 4 clusters, of which each class is composed of two of the clusters. A simple normal distribution is unlikely to capture this well, but a mixture model is built specifically to model distributions that are composed of multiple parts. We can see that the correspond log probability is much higher when we incorporate the mixture model in. There is an implementation detail that doesn't allow you yet to pass in a compositional model into the `from_samples` method quite yet, but in this case it is fairly simple to break up the data such that we build the model ourselves.

### Speed

pomegranate uses a Cython backend for the computationally intensive aspects of the calculation, both dramatically speeding up calculations and allowing for multithreading to be utilized.

#### numpy

pomegranate used to be consistently faster than numpy at fitting probability distributions to data, when compared to performing the same operations in numpy. However, due to consistent improvements in numpy, it is now a more complicated picture. For example, let's look at fitting a normal distribution to 1,000 samples.

```python
X = numpy.random.normal(4, 1, size=1000)

%timeit -n 1 -r 1 numpy.mean(X), numpy.std(X)
%timeit -n 1 -r 1 NormalDistribution.from_samples(X)
```

    1 loop, best of 1: 137 µs per loop
    1 loop, best of 1: 152 µs per loop

pomegranate appears to be around the same time, but the total amount of time is negligble one way or another. If we increase the size of the data we're using to 10 million samples:

```python
X = numpy.random.normal(4, 1, size=10000000)

%timeit numpy.mean(X), numpy.std(X)
%timeit NormalDistribution.from_samples(X)
```

    10 loops, best of 3: 65.8 ms per loop
    10 loops, best of 3: 109 ms per loop

It looks like pomegranate is now slower than numpy, but still roughly comparable.

Let's now look at fitting a multivariate Gaussian distribution. The calculation of a covariance matrix requires a dot product, which numpy accelerates using BLAS. pomegranate uses a Cython wrapper of BLAS and so can take advantage of those speed improvements. Let's look at fitting to one million data points with 3 dimensions.

```python
X = numpy.random.normal(4, 1, size=(1000000, 3))

%timeit numpy.mean(X, axis=0), numpy.cov(X, rowvar=False, ddof=0)
%timeit MultivariateGaussianDistribution.from_samples(X)
```

    10 loops, best of 3: 42.6 ms per loop
    10 loops, best of 3: 41.1 ms per loop

It looks like pomegranate is around the same speed here. However, if we fit to a 1000 dimensional sample rather than a 3 dimensional one:

```python
X = numpy.random.normal(4, 1, size=(100000, 1000))

%timeit numpy.mean(X, axis=0), numpy.cov(X, rowvar=False, ddof=0)
%timeit MultivariateGaussianDistribution.from_samples(X)
```

    1 loop, best of 3: 5.28 s per loop
    1 loop, best of 3: 10.9 s per loop

Now it looks like pomegranate is slower than numpy.

The main take-away when compared to numpy is that pomegranate and numpy appear to perform similarly. As the samples become larger numpy becomes faster than pomegranate, but they're both within an order of magnitude of each other one way or another. If you're trying to do basic operations on a large amount of data, you may want to stick with using numpy.

#### scipy

scipy can be used to calculate probabilities of samples given a distribution. This corresponds to the `probability` and `log_probability` functions in pomegranate. Let's see how long it takes to calculate these probabilities in the two packages given a normal distrubution.

```python
from scipy.stats import norm

d = NormalDistribution(0, 1)
x = numpy.random.normal(0, 1, size=(10000000,))

%timeit norm.logpdf(x, 0, 1)
%timeit NormalDistribution(0, 1).log_probability(x)

print "\nlogp difference: {}".format((norm.logpdf(x, 0, 1) - NormalDistribution(0, 1).log_probability(x)).sum())
```

    1 loop, best of 3: 705 ms per loop
    10 loops, best of 3: 84.8 ms per loop

    logp difference: -3.99015110908e-06

Looks like it can be significantly faster. Let's also look at a large multivariate normal distribution with 2500 dimensions.

```python
from scipy.stats import multivariate_normal

dim = 2500
n = 1000

mu = numpy.random.normal(6, 1, size=dim)
cov = numpy.eye(dim)

X = numpy.random.normal(8, 1, size=(n, dim))

d = MultivariateGaussianDistribution(mu, cov)

%timeit multivariate_normal.logpdf(X, mu, cov)
%timeit MultivariateGaussianDistribution(mu, cov).log_probability(X)
%timeit d.log_probability(X)

print "\nlogp difference: {}".format((multivariate_normal.logpdf(X, mu, cov) - d.log_probability(X)).sum())
```

    1 loop, best of 3: 4.57 s per loop
    1 loop, best of 3: 2.37 s per loop
    1 loop, best of 3: 557 ms per loop

    logp difference: 8.18359694676e-07

One of the reasons which pomegranate can be so fast at calculating log probabilities is that it is able to cache parts of the logpdf equation so that it doesn't need to do all of the calculations each time. For example, let's look at the Normal distribution pdf equation:

\begin{equation}
P(X|\mu, \sigma) = \frac{1}{\sqrt{2\pi}\sigma} exp \left( -\frac{(x - \mu)^{2}}{2\sigma^{2}} \right) \\
\end{equation}

We can take the log of this to simplify it.

\begin{equation}
logP(X|\mu, \sigma) = -\log \left(\sqrt{2\pi}\sigma \right) - \frac{(x-\mu)^{2}}{2\sigma^{2}}
\end{equation}

pomegranate speeds up this calculation by caching $-\log(\sqrt{2\pi}\sigma)$ and $2\sigma^{2}$ when the object is created. This means that the equation is simplified to the following:

\begin{equation}
logP(X|\mu, \sigma) = \alpha - \frac{(x - \mu)^{2}}{\beta}
\end{equation}

We don't need to calculate any logs or exponentials here, just a difference, a multiplication, a division, and a subtraction.

#### scikit-learn

scikit-learn and pomegranate overlap when it comes to naive Bayes classifiers and mixture models. In pomegranate, both of these model types can be used with any distribution, allowing mixtures of exponentials or log-normals to be made just as easily as mixtures of Gaussians, whereas in scikit-learn only Gaussians or multinomials are allowed. Let's compare speed of the overlap.

```python
from sklearn.mixture import GaussianMixture

X = numpy.random.normal(8, 1, size=(10000, 100))

%timeit model1 = GaussianMixture(5, max_iter=10).fit(X)
%timeit model2 = GeneralMixtureModel.from_samples(MultivariateGaussianDistribution, 5, X, max_iterations=10)
```

    /home/jmschr/anaconda2/lib/python2.7/site-packages/sklearn/mixture/base.py:268: ConvergenceWarning: Initialization 1 did not converge. Try different init parameters, or increase max_iter, tol or check for degenerate data.
      % (init + 1), ConvergenceWarning)

    1 loop, best of 3: 1.8 s per loop
    1 loop, best of 3: 1.92 s per loop

It looks like the two are approximately the same speed on a single thread. Let's look at naive Bayes now.

```python
from sklearn.naive_bayes import GaussianNB

X = numpy.random.normal(8, 1, size=(100000, 500))
X[:50000] += 1

y = numpy.zeros(100000)
y[:50000] = 1

%timeit GaussianNB().fit(X, y)
%timeit NaiveBayes.from_samples(NormalDistribution, X, y)
```

    1 loop, best of 3: 908 ms per loop
    1 loop, best of 3: 1.45 s per loop

It looks like scikit-learn is a bit faster when it comes to training a Gaussian naive Bayes model.

Much like with the comparisons to numpy, if scikit-learn has something easily implemented, it's likely better to try to use their implementation. The goal of pomegranate is not to step on the feet of these other packages, but rather to provide similarly-fast implementations that extend to other probability distributions. For example, you can train a log normal naive Bayes in the following manner:

```python
model = NaiveBayes.from_samples(LogNormalDistribution, X, y)
```

No such functionality yet exists in scikit-learn to train naive Bayes models for arbitrary distributions.


```
!pip install tensorflow-quantum
```

```
#imports
import os

import numpy as np
np.random.seed(11)
import pandas as pd
import os
from glob import glob
import seaborn as sns
from PIL import Image
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split, KFold, cross_val_score, GridSearchCV
from sklearn.metrics import accuracy_score
import itertools

from tqdm import tqdm
from sklearn.preprocessing import OneHotEncoder

import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np
import seaborn as sns
import collections

import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

```
from google.colab import drive
drive.mount('/content/drive')
```

```
folder_benign_train = '/content/drive/My Drive/binary-data/train/benign'
folder_malignant_train = '/content/drive/My Drive/binary-data/train/malignant'

folder_benign_test = '/content/drive/My Drive/binary-data/test/benign'
folder_malignant_test = '/content/drive/My Drive/binary-data/test/malignant'

read = lambda imname: np.asarray(Image.open(imname).convert("RGB"))

# Load in training pictures
ims_benign = [read(os.path.join(folder_benign_train, filename)) for filename in tqdm(os.listdir(folder_benign_train))]
X_benign = np.array(ims_benign, dtype='uint8')
ims_malignant = [read(os.path.join(folder_malignant_train, filename)) for filename in tqdm(os.listdir(folder_malignant_train))]
X_malignant = np.array(ims_malignant, dtype='uint8')

# Load in testing pictures
ims_benign = [read(os.path.join(folder_benign_test, filename)) for filename in tqdm(os.listdir(folder_benign_test))]
X_benign_test = np.array(ims_benign, dtype='uint8')
ims_malignant = [read(os.path.join(folder_malignant_test, filename)) for filename in tqdm(os.listdir(folder_malignant_test))]
X_malignant_test = np.array(ims_malignant, dtype='uint8')

```

```
#create labels
y_benign = np.zeros(X_benign.shape[0])
y_malignant = np.ones(X_malignant.shape[0])

y_benign_test = np.zeros(X_benign_test.shape[0])
y_malignant_test = np.ones(X_malignant_test.shape[0])

#merge data
X_train = np.concatenate((X_benign, X_malignant), axis = 0) #training data
y_train = np.concatenate((y_benign, y_malignant), axis = 0) #training labels

X_test = np.concatenate((X_benign_test, X_malignant_test), axis = 0) #test data
y_test = np.concatenate((y_benign_test, y_malignant_test), axis = 0) #test labels
```

```
#shuffle data
s = np.arange(X_train.shape[0])
np.random.shuffle(s)
X_train = X_train[s]
y_train = y_train[s]

s = np.arange(X_test.shape[0])
np.random.shuffle(s)
X_test = X_test[s]
y_test = y_test[s]
```

```
# Display first 15 images of moles, and how they are classified
w=40
h=30
fig=plt.figure(figsize=(12, 8))
columns = 5
rows = 3
```

```
y_train = y_train.reshape(-1,1)
y_test = y_test.reshape(-1,1)

enc = OneHotEncoder(handle_unknown='ignore')
enc.fit(y_train)

y_train = enc.transform(y_train).toarray()
y_test = enc.transform(y_test).toarray()

```

```
# With data augmentation to prevent overfitting
X_train = X_train/255.
X_test = X_test/255.

```

```
import cv2

#grayscaled & resized data
gray_X_train = []
gray_X_test = []

def rgb2gray(rgb):
    return np.dot(rgb[...,:3], [0.2989, 0.5870, 0.1140])

for image in X_train:
  img = rgb2gray(image)
  gray_X_train.append(cv2.resize(img, (4,4)))

for image in X_test:
  img = rgb2gray(image)
  gray_X_test.append(cv2.resize(img, (4,4)))
```

```
def convert_to_circuit(image):
    """Encode truncated classical image into quantum datapoint."""
    values = np.ndarray.flatten(image)
    qubits = cirq.GridQubit.rect(4, 4)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        if value:
            circuit.append(cirq.X(qubits[i]))
    return circuit

```

```
x_train_circ = [convert_to_circuit(x) for x in gray_X_train]
x_test_circ = [convert_to_circuit(x) for x in gray_X_test]
```

```
x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)
x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)
```

```
#QNN Implementation: classification is based on the expectation of the readout qubit --> use 2 qubit gates with the readout qubit always acted on

class CircuitLayerBuilder():
    def __init__(self, data_qubits, readout):
        self.data_qubits = data_qubits
        self.readout = readout

    def add_layer(self, circuit, gate, prefix):
        for i, qubit in enumerate(self.data_qubits):
            symbol = sympy.Symbol(prefix + '-' + str(i))
            circuit.append(gate(qubit, self.readout)**symbol)

#2 layered model that matches the data-circuit size
#preparation and readout operations

def create_quantum_model():
    data_qubits = cirq.GridQubit.rect(4, 4)  # a 4x4 grid.
    readout = cirq.GridQubit(-1, -1)         # a single qubit at [-1,-1]
    circuit = cirq.Circuit()

    # Prepare the readout qubit.
    circuit.append(cirq.X(readout))
    circuit.append(cirq.H(readout))

    builder = CircuitLayerBuilder(
        data_qubits = data_qubits,
        readout=readout)

    # Then add layers (experiment by adding more).
    builder.add_layer(circuit, cirq.XX, "xx1")
    builder.add_layer(circuit, cirq.ZZ, "zz1")

    # Finally, prepare the readout qubit.
    circuit.append(cirq.H(readout))

    return circuit, cirq.Z(readout)

model_circuit, model_readout = create_quantum_model()
```

```
# Build the Keras model.
model = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])

y_train_hinge = 2.0*y_train-1.0
y_test_hinge = 2.0*y_test-1.0

#hinge_accuracy = handles [-1, 1] as y_true labels argument
def hinge_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true) > 0.0
    y_pred = tf.squeeze(y_pred) > 0.0
    result = tf.cast(y_true == y_pred, tf.float32)

    return tf.reduce_mean(result)

model.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])

print(model.summary())
```

```
EPOCHS = 3
BATCH_SIZE = 8

NUM_EXAMPLES = len(x_train_tfcirc)

x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]
y_train_hinge_sub = y_train_hinge[:NUM_EXAMPLES]

qnn_history = model.fit(
      x_train_tfcirc_sub, y_train_hinge_sub,
      batch_size=8,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc, y_test_hinge))

qnn_results = model.evaluate(x_test_tfcirc, y_test)
```

```
model.save_weights("model.h5")
```

```
!pip install coremltools

output_labels = ['0', '1']
ios_model = coremltools.converters.keras.convert('model.h5', input_names=['image'], output_names=['output'],
                                                   class_labels=output_labels, image_input_names='image')

ios_model.save('your_model_name.mlmodel')
```


# Introduction

In this tutorial, we will develop a model of paired associates learning as described in the standard ACT-R tutorial. This model differs from other models in the tutorial series in that the trials are not independent as a result of learning and forgetting.

## Task

Over the course of eight blocks,  subjects learn paired associates consisting of a word paired with a unique number. In each block, the same 20 paired associates are presented in a randomized order. On each trial, the word component of a randomly selected paired associate is presented for five seconds. Subjects are instructed to press the number key associated with the word. In contrast to the task described in the standard ACT-R tutorial, subjects are asked to press a designated key in the event of a retrieval failure as opposed to omitting a response. This modification to the task can improve the estimation of the retrieval threshold. After the five second deadline, the corresponding number is presented as feedback for a duration of five seconds.

# Paired Associates Model

On each trial, the model visually encodes the word associate and creates a chunk in the imaginal buffer representing the paired associate for the current trial. After visual encoding, the word is added to the word slot of the chunk in the imaginal buffer and the model attempts to retrieve a chunk with the word associate in the word slot. If the chunk is retrieved, the model responds with the corresponding number located in the number slot of the retrieved chunk. The model responds with a a designated key in the case of a retrieval failure. After the five second deadline, the number associate is presented as feedback and the model visually encodes and adds the number to the number slot of the chunk in the imaginal buffer. The chunk is then cleared from the imaginal buffer, at which time the number of uses for that chunk is incremented by one. In some small number of cases, the response is truncated because the deadline is exceeded. In such cases, the model omits a response and proceeds to process the feedback.

## Declarative memory

Let $M$ denote the set chunks in declarative memory, where  $n_{\rm items}$ represents the number of associative pairs used in the experiment. $M$ is empty at the beginning of the experiment. New chunks are added to $M$ as they are encountered. Each chunk consists of the following set of slots: $Q= \rm \{word,number\}$.

### Encoding

On each trial, a word $v$ is presented on a screen and the model encodes the stimulus into a chunk that is stored in the imaginal buffer. Formally, we define the encoded chunk as:

\begin{align}
\mathbf{c}_{s,\rm \textrm{imaginal}} = \{ (\rm word, v)\}
\end{align}

The process of encoding requires .05 seconds for conflict resolution, .085 seconds to shift attention, and .200 to create a new chunk in the imaginal buffer. The time to create a new chunk in the imaginal buffer will be ignored for simplicity because it happens conccurently with a memory retrieval, which is typically slower. Thus, the .200 seconds tends not to influence the processing time dynamics of the model.

### Retrieval Request

The retrieval request is defined as:

\begin{align}
\mathbf{r} = \{(\textrm{word},c_{s,\rm imaginal}(\rm word)\}
\end{align}

where $c_{s,\rm imaginal}(\rm word)$ is the word value of the chunk in the imaginal buffer. The slot set for the retrieval request is defined as $Q_r = \rm \{word\}$

### Response Mapping

We define resp $\in \{0,1,2,\cdots, 9, \emptyset, \textrm{uk} \}$ as the response. As before, uk indicates a retrieval failure. $\emptyset$ denotes a case in which a response failed to meet the five second deadline. The mapping between the retrieval outcome and the observed response is:
\begin{align*}
c_{\rm resp} &= \mathbf{c}\in M : c(\rm number) = \textrm{resp}  &\textrm{ for resp }\ne \textrm{uk} \textrm{ and rt } < 5\\
c_{\rm uk} &= \mathbf{c}_{m^\prime}  & \textrm{ rt } < 5.
\end{align*}
where $|R_{\rm resp}| = 1$ because the response mapping is one-to-one. If a chunk is retrieved, it is cleared from the buffer as soon as the response production rule is executed and $N_m$ is incremented by 1. After encoding the feedback, the imaginal buffer is cleared and $N_m$ is incremented by 1 and the  timestamp $t_{m}$ is updated with the current simulated time.

## Activation

Activation for chunk $m$ is defined as:

\begin{align}
a_m = \textrm{blc} + \epsilon_m
\end{align}

where bll is base level learning, and $\epsilon_m \sim \rm normal(0, \sigma)$ is normally distributed noise.
Base level learning captures the dynamics of learning and forgetting. As a chunk is used more, activation increases. By contrast, as the time elapsed between uses increases, activation decays. We use the hybrid approximation to base-level learning because it reduces computational complexity with little sacrifice in terms of (Petrov, 2006, Fisher et al., 2018). The hybrid approximation tracks the exact time stamps of the last $k$ uses of each chunk. Previous analyses of the hybrid approximation has demonstrated that setting $k=1$ produces a highly accurate approximation (Petrov, 2006, Fisher et al., 2018). Without loss of generalizability, we present a simplified version of the hybrid approximation in which $k=1$:
\begin{equation}
\textrm{bll}_m \approx  \ln\left(t_{m}^{-d} + \frac{(N_m-1)(L_m^{1-d} - t_{m}^{1-d}) }{(1-d)(L_m - t_{m})}\right)
\end{equation}
where $d \in [0,1]$ is the decay parameter, $L_m$ is the time since the creation of chunk $m$, $N_m$ is the number of uses of chunk $m$, and $t_{m}$ is the time since the last use of chunk $m$.

## Conflict resolution and perceptual-motor time

We will fix $t_{\rm er}$ to the following value based on the default ACT-R parameters

\begin{align}
 t_\textrm{er} = \rm (cr + e) + cr + (cr + me) = .535
\end{align}

where cr is the default conflict resolution time of .05 seconds, $e$ is the default encoding time of .085 seconds and me is the approximate motor execution time of .300 seconds.

## Likelihood Function

The likelihood function consists of two components. The following component is used when a response is given:
\begin{eqnarray}\label{eq:LNR_paired}
\mathcal{L}(\Theta; \textrm{resp},\textrm{rt})
= f(\textrm{rt}; -\mu_{\rm resp} + \upsilon, \sigma)\left[1-G(\textrm{rt}; -\mu_{m'} + \upsilon, \sigma)\right]\\
\mathcal{L}(\Theta; \textrm{uk}, \textrm{rt})
= f(\textrm{rt}; -\mu_{m'} + \upsilon, \sigma)\left[1-G(\textrm{rt}; -\mu_{\rm resp} + \upsilon, \sigma)\right],
\end{eqnarray}
where $\Theta$ represents the set of model parameters. The other component applies when a response is not given.  Formally, the probability of failing to respond within the deadline is defined as:
\begin{equation}\label{eq:truncate}
\textrm{Pr}(\textrm{resp}=\emptyset) = \left[1-G(5 \mid -\mu_{\rm resp} + \upsilon,\sigma)\right]\left[1-G(5 \mid -\mu_{\rm uk} + \upsilon,\sigma)\right]
\end{equation}

Next, we will generate some simulated data to use for parameter estimation. First, we will load the required packages and functions.

```julia
# change directory of this file
cd(@__DIR__)
# pakcage manager
using Pkg
# activate this environment of this project
Pkg.activate("../../")
# load dependencies
using Turing, DataFrames, StatsPlots, Revise, ACTRModels
# load stimuli
include("Stimuli.jl")
# load model functions
include("Paired_Model.jl")
# set seed for RNG
Random.seed!(589)
```

      Activating environment at `~/.julia/dev/ACTRFundamentalTools/Project.toml`

    MersenneTwister(589)

# Generate Data

The data generation process is comprised of the following three functions:

- `simulate`: top-level function that runs the simulation
- `simulate_block`: iterates through a block of trials
- `simulate_trial`: simulates the model for a single trial

The function `simulate` simulates the experiment for a specified number of blocks and trials within blocks. `simulate` accepts the following arguments:

- `all_stimuli`: all 20 word-number pairs
- `fixed_parms`: a `NamedTuple` of fixed parameters in the model
- `n_blocks`: the number of blocks in the experiment
- `n_trials`: the number of trials in each block, which is also the number of unique paired associates
- `deadline`: the deadline for responding
- `isi`: length of inter-stimulus interval during which the number associate is presented as feedback
- `d`: keyword argument for decay parameter

`simulate` performs the following steps. First, it selects a subset of paired associates based on `n_trials` with a max of 20. Next, the model is initialized with empty declarative memory and the parameters specified in `fixed_parms` and `d`. Finally, it iterates through each block of trials, calling `simulate_block` each time. Inside the block loop, the current time is updated with `set_time!` and simulated data is added to the array called `data`.

The function `simulate_block` iterates through a block of trials and requires the following inputs:

- `actr`: ACT-R model object
- `stimuli`: a randomized set of paired associates, one for each trial
- `cur_time`: the current time in the simulation
- `block`: current block number
- `deadline`: the deadline for responding
- `isi`: length of inter-stimulus interval during which the number associate is presented as feedback

In `simulate_block`, the array `data` stores the simulated data for the current block of trials. The for loop iterators through the paired associates for the current block of trials and advanced the time by 10 seconds after each loop because a new trial always begins after 10 seconds.

Finally, `simulate_trial` executes the simulation for a single trial. `simulate_trial` requires the following inputs:

- `actr`: ACT-R model object
- `stimulus`: a paired associate
- `block`: current block number
- `cur_time`: the current time in the simulation
- `deadline`: the deadline for responding

`simulate_trial` performs the following basic functions: execute a retreival attempt, handle the response and data, and process the feedback following the 5 second deadline. Throughout the process, time is incremented with the function `add_time!` The function `retrieve` submits a retrieval request for a chunk containing a slot for the presented word.

The following block of if statements handles the data recording and updating of the chunks. If the rt is greater than the deadline, the response is classified as truncated. Next, if a chunk is retrieved, the number of uses is updated and the response is classified as retrieved. Finally, if a retrieval failure occurs, the response is classified as failure.

In the last part of `simulate_trial`, the model encodes the number associate and `cur_time` is incremented accordingly. Finally, the number of uses is incremented for the correct chunk.

All three functions are annotated below:

```julia
function simulate(all_stimuli, fixed_parms, n_blocks, n_trials, deadline=5.0, isi=5.0; d)
    # sample unique stimului for the simulation
    stimuli = sample_stimuli(all_stimuli, n_trials)
    # initialize declarative memory
    chunks = [Chunk(word=:_,number=1)] |> empty!
    memory = Declarative(;memory=chunks)
    # create act-r object
    actr = ACTR(;declarative=memory, fixed_parms..., d)
    # preallocate data
    data = Array{NamedTuple,1}()
    # iterate through all blocks
    for block in 1:n_blocks
        # randomize stimulus order
        shuffle!(stimuli)
        temp = simulate_block(actr, stimuli, block, deadline, isi)
        push!(data, temp...)
    end
    return vcat(data...)
end

function simulate_block(actr, stimuli, block, deadline=5.0, isi=5.0)
    # initialize data for the current block
    data = Array{NamedTuple,1}()
    start_time = get_time(actr)
    for stimulus in stimuli
        temp = simulate_trial(actr, stimulus, block, deadline)
        push!(data, temp)
        # 5 second response deadline + additional 5 seconds
        start_time += (deadline + isi)
        set_time!(actr, start_time)
    end
    return data
end

function simulate_trial(actr, stimulus, block, deadline=5.0)
    N=0;L=0.0;recent=Float64[];retrieved=:failed;time_created=0.0
    trial_start = get_time(actr)
    # encode and conflict resolution time
    e_time = 0.085 + 0.05 + 0.05
    add_time!(actr, e_time)
    # retrieve paired associate based on word
    chunk = retrieve(actr; word=stimulus.word)
    # retrieval time
    r_time = compute_RT(actr, chunk)
    # add encoding time and retrieval time to reaction time
    rt = e_time + r_time + .05 + 0.300
    # interrupt retrieval if exceeds deadline
    if rt > deadline
        # set to truncated value
        rt = deadline
        # advance current time to deadline
        set_time!(actr, trial_start + deadline)
        retrieved = :truncated
        # retrieval info if no chunk is retrieved
        chunk = get_chunks(actr; stimulus...)
        if !isempty(chunk)
            # get chunk state at time of retrieval
            @unpack N,L,recent,time_created = chunk[1]
            recent = copy(recent)
        end
    elseif !isempty(chunk)
        # add retreival time to current time
        add_time!(actr, r_time)
        # next conflict resolution
        add_time!(actr, .05)
        retrieved = :retrieved
        # get chunk state at time of retrieval
        @unpack N,L,recent,time_created = chunk[1]
        recent = copy(recent)
        # add chunk if new, or update N and time stamps
        add_chunk!(actr; stimulus...)
    else
        # add retreival failure time to current time
        add_time!(actr, r_time)
        # next conflict resolution
        add_time!(actr, .05)
        # retrieval info if no chunk is retrieved
        chunk = get_chunks(actr; stimulus...)
        if !isempty(chunk)
            # get chunk state at time of retrieval
            @unpack N,L,recent,time_created = chunk[1]
            recent = copy(recent)
        end
    end
    # feedback is always presented after the deadline
    set_time!(actr, trial_start + deadline)
    # encode feedback
    add_time!(actr, 0.05 + 0.05 + 0.085)
    # add chunk: create new chunk if does not exist, otherwise increment N and add time stamp
    add_chunk!(actr; stimulus...)
    data = (stimulus=stimulus,N=N,L=L,block=block,time_created=time_created,
        recent=recent,rt=rt,retrieved=retrieved)
    return data
end
```

    simulate_trial (generic function with 2 methods)

It is possible to reverse engineer the memory state prior to each response based on timestamps and assumptions of the model. To eliminate this post-processing step, we will record the memory state in the data generation simulation. `simulate` outputs a `NamedTuple` with all the information needed to reconstruct base-level learning.

Its worth noting that we changed two parameter values that were used in the standad ACT-R tutorial. First, we changed $\tau$ from -2.0 to -2.5. Second, we changed s from .5 to .2. The reason we changed these parameters is that the the lognormal race model behaves differently compared to standard ACT-R, even though there are many similarities. First, adding noise to the retrieval threshold increases the number of retrieval failures, thus requring the value to be adjusted downward. However, decreasing the retrieval threshold too much would lead to extremely long retrieval failures. As a result, we decreased s from .5 to .2 to further adjust the learning trajectory.

The code block below will show the data for one simulated trial:

```julia
# true decay parameter
d = 0.5
# number of block repetitions
n_blocks = 8
# number of trials within a block
n_trials = 20
# fixed parameters
fixed_parms = (
    τ = -2.5,        # retrieval threshold
    noise = true,    # noise "on"
    bll = true,      # base-level learning "on"
    s = 0.2,         # scale parameter for activation noise
    lf = 0.4,        # latency factor
    ter = 0.535      # encoding, motor and conflict resolution time
)
# generate data
data = simulate(all_stimuli, fixed_parms, n_blocks, n_trials; d)
data[1]
```

    (stimulus = (word = :sock, number = 4), N = 0, L = 0.0, block = 1, time_created = 0.0, recent = Float64[], rt = 5.0, retrieved = :truncated)

The keys in each `NamedTuple` are:

- `stimulus`: the word-number pair
- `N`: the number of uses of the chunk
- `L`: the lifetime of the chunk defined as the time since the chunk was created
- `block`: block number
- `time_created`: the time at which the chunk was created
- `recent`: a vector that trackets the most recent use (in this case a single element becase k=1)
- `rt`: the reaction time
- `retrieved`: indicates whether a chunk was retrieved, a retrieval failure occured, or the response was truncated

## Define Likelihood Function

The code below shows the log likelihood function for the paired model. The function `computeLL` accepts the following arguments:

- data: a `NamedTuple` of trial-level response information
- fixed_parms: a `NamedTuple` of fixed parameters
- $d$: the decay parameter

`computeLL` begins by creating an ACT-R model object and extracting the required parameters. After initializing the log likelihood `LL` at zero, the log likelihood is computed for each trial. There are three cases corresponding to the three retrieval outcomes: a successful retrieval, a retrieval failure, and a truncated retrieval. In each case, the retrieval state is created with the function `modify!`. Once the retrieval state is recreated, the activation of the chunk can be computed accurate for the parameters provided. Next, a Lognormal Race distribution object, `LNR` is created using the chunk activation value and retrieval threshold. With these quantities computed, the log likelihood for the current trial can be computed and added to `LL`. Each of the three cases include a special case for the first block in declarative memory does not yet contain a chunk for each paired associate. In such cases, the Lognormal Race contains only the retrieval threshold and thus reduces to a shifted Lognormal.

```julia
function computeLL(data, fixed_parms; d)
    chunk = Chunk(act=zero(d))
    # initialize declarative memory
    memory = Declarative(;memory=[chunk])
    # create ACTR object with declarative memory and parameters
    actr = ACTR(;declarative=memory, fixed_parms..., d)
    # extract parameters
    @unpack s,lf,τ,ter=actr.parms
    # do not add noise activation values
    actr.parms.noise = false
    σ = s * pi / sqrt(3)
    # initialize log likelihood
    LL = 0.0
    for k in data
        cur_time = k.L + k.time_created
        if k.retrieved == :retrieved
            # reproduce the memory state right before retrieval
            modify!(chunk; N=k.N, recent=k.recent, time_created=k.time_created)
            # compute activation
            compute_activation!(actr, chunk, cur_time)
            # adjust activation values for latency factor
            v = [chunk.act,τ] .- log(lf)
            # log normal race distribution object
            dist = LNR(-v, σ, ter)
            # compute log likelihood of retrieving chunk at time k.rt
            LL += logpdf(dist, 1, k.rt)
        elseif k.retrieved == :truncated
            # special case in which chunk has not been created yet
            if k.N == 0
                v = τ .- log(lf)
                # compute log likelihood of failing to respond within deadline
                LL += logccdf(LogNormal(-v, σ), 5)
            else
                # reproduce the memory state right before retrieval
                modify!(chunk; N=k.N, recent=k.recent, time_created=k.time_created)
                # compute activation
                compute_activation!(actr, chunk, cur_time)
                # adjust activation values for latency factor
                v = [chunk.act,τ] .- log(lf)
                # compute log likelihood of failing to respond within deadline
                LL += logccdf(LogNormal(-v[1], σ), 5) + logccdf(LogNormal(-v[2], σ), 5)
            end
        else
            # retrieval failure for novel pair
            if k.N == 0
                dist = LNR(-[τ-log(lf)], σ, ter)
                LL += logpdf(dist, 1, k.rt)
            else
                # reproduce the memory state immediately prior to retrieval
                modify!(chunk; N=k.N, recent=k.recent, time_created=k.time_created)
                # compute activation
                compute_activation!(actr, chunk, cur_time)
                # mean activation values
                v = [chunk.act,τ] .- log(lf)
                # log normal race distribution object
                dist = LNR(-v, σ, ter)
                # compute log likelihood of failing to retrieve chunk at time k.rt
                LL += logpdf(dist, 2, k.rt)
            end
        end
    end
    return LL
end
```

    computeLL (generic function with 1 method)

## Define Model

The model and prior distributions are summarized as follows:

\begin{align}
d \sim \rm Beta(5,5)
\end{align}

\begin{align}
y_{i} \sim \rm f(\textrm{resp}_i,\textrm{rt}_i \mid \Theta)
\end{align}

In computer code, the model is specified as follows:

```julia
@model model(data, parms) = begin
    d ~ Beta(5, 5)
    data ~ Paired(d, parms)
end
```

    model (generic function with 1 method)

## Estimate Parameters

Now that the priors, likelihood and Turing model have been specified, we can now estimate the parameters. In the following code, we will run four MCMC chains with the NUTS sample for 2,000 iterations and omit the first 1,000 warmup samples.

```julia
# number of samples
n_samples = 1000
delta = 0.85
# number of adaption samples
n_adapt = 1000
# number of independent chains
n_chains = 4
# sampler object
specs = NUTS(n_adapt, delta)
# Start sampling.
chain = sample(model(data, fixed_parms), specs, MCMCThreads(), n_samples, n_chains, progress=true)
```

    ┌ Info: Found initial step size
    │   ϵ = 0.05
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/Ysmbf/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.05
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/Ysmbf/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.025
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/Ysmbf/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.05
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/Ysmbf/src/inference/hmc.jl:188
    ┌ Warning: The current proposal will be rejected due to numerical error(s).
    │   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
    └ @ AdvancedHMC /home/dfish/.julia/packages/AdvancedHMC/bv9VV/src/hamiltonian.jl:47

    Chains MCMC chain (1000×13×4 Array{Float64, 3}):

    Iterations        = 1001:1:2000
    Number of chains  = 4
    Samples per chain = 1000
    Wall duration     = 9.31 seconds
    Compute duration  = 36.55 seconds
    parameters        = d
    internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, max_hamiltonian_energy_error, tree_depth, numerical_error, step_size, nom_step_size

    Summary Statistics
      parameters      mean       std   naive_se      mcse         ess      rhat    ⋯
          Symbol   Float64   Float64    Float64   Float64     Float64   Float64    ⋯

               d    0.4988    0.0055     0.0001    0.0001   1584.5963    0.9997    ⋯
                                                                    1 column omitted

    Quantiles
      parameters      2.5%     25.0%     50.0%     75.0%     97.5%
          Symbol   Float64   Float64   Float64   Float64   Float64

               d    0.4881    0.4950    0.4988    0.5027    0.5094

## Results

In the summary output above, `rhat` for the decay parameter is very close to 1, suggesting that the chains converged. In the trace plot below (top panel), each chain (denoted by a different color) is superimosed and fluxuates randomly. This pattern provides visual confirmation of convergence. The autocorrelation plot in the second panel shows a fast drop in autocorrelation, indicating efficient sampling. In the third panel, the density plot shows that the posterior distribution of $d$ is centered near the true value of .5. Taken together, these results suggest the we are able to estimate the decay parameter properly.

```julia
pyplot()
ch = group(chain, :d)
font_size = 12
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcτ = plot(p1, p2, p3, layout=(3,1), size=(600,600))
```

### Posterior Predictive Distribution

The plot below shows mean reaction time across blocks for 100 posterior samples. Each red line is the mean retrieval failure time across the 8 blocks for a given posterior sample. Similarly, each black line is the mean reaction time for correct responses across blocks for a given posterior sample. As expected, both retrieval failures and successful retrievals become faster across blocks. Retrieval failures are slower on average than successful retrievals. In addition, it is clear that retrieval failures become less frequent across blocks.

```julia
preds = posterior_predictive(x -> show_learning(all_stimuli, fixed_parms; x...), chain, 100)
preds = vcat(preds..., source=:rep)
groups = groupby(preds, [:rep,:retrieved,:block])
mean_rts = combine(groups, :rt=>mean=>:mean_rt)
sort!(mean_rts, [:rep,:block,:retrieved])
fmean_rts = filter(x->x.retrieved != :truncated, mean_rts)
p4 = @df fmean_rts plot(:block, :mean_rt, group=(:rep,:retrieved), xlabel="Block", ylabel="Mean RT", xaxis=font(12),
    yaxis=font(12), color=[:darkred :black], grid=false, size=(600,300), titlefont=font(12), linewidth=.5,
    labeltitle="Retrieved", legendtitlefontsize=8, ylims=(0,5.1), leg=false, alpha=.5)
```

In the plot below, mean accuracy across posterior samples increases over the course of eight blocks. Initially, accuracy is zero because the associative pairs are novel. You may have noticed that these predictions differ somewhat from the predictions in the standard ACT-R tutorial. As noted above, the reason for this difference is mostly due to the addition of noise to the retrieval threshold. Not only does this change the retrieval dynamics, but the mean time for retrieval failures increases with increasing values of noise. In addition, using normally distributed noise rather than logistically distributed noise for activation will cause the predictions to differ by some small amount because the logistic distribution has a heavier tail than the normal distribution.

```julia
groups = groupby(preds, :block)
accuracy = combine(groups, :retrieved=>(x->mean(x .== :retrieved))=>:accuracy)
sort!(accuracy)
p5 = @df accuracy plot(:block, :accuracy, xlabel="Block", ylabel="Mean Accuracy", xaxis=font(12),
    yaxis=font(12), color=:black, grid=false, size=(600,300), titlefont=font(12), linewidth=2,
    leg=false, legendtitlefontsize=8)
```

# References

Fisher, C. R., Houpt, J., & Gunzelmann, G. (2018). A Comparison of Approximations for Base-Level Activation in ACT-R. Computational Brain & Behavior, 1(3-4), 228-236.

Petrov, A. A. (2006). Computationally efficient approximation of the base-level learning equation in ACT-R. In Proceedings of the seventh international conference on cognitive modeling (pp. 391-392). Edizioni Goliardiche: Trieste, ITA.