```python
# Alpha_RNNs-Bitcoin
# Author: Matthew Dixon
# Version: 1.1 (27.2.2020)
# License: MIT
# Email: matthew.dixon@iit.edu
# Notes: tested on Mac OS X running Python 3.6.9 with the following packages:
# tensorflow=2.0.0, keras=2.3.1, scikit-learn=0.22.1, numpy=1.18.1, matplotlib=3.1.3, pandas=1.0.3, statsmodels=0.10.1
# Citation: Please cite the following reference if this notebook is used for research purposes:
# Bilokon P., Dixon M.F. and Halperin I., Machine Learning in Finance: From Theory to Practice, Springer Graduate Textbook Series, 2020.
```

## An Introduction to Prediction with RNNs

### Overview
- This notebook provides an example of how Keras can be used to train and test TensorFlow RNNs for time series prediction. The example dataset is for predicting from noisy, non-stationary data.
- Statistical methods used for autoregressive models shall be used to identify the sequence length needed in the RNN and to diagnose the model error.
- Plain RNNs are not suited to non-stationary time series modeling. We can use a GRU or LSTM to model non-stationary data, since these models exhibit dynamic auto-correlation structure.
- Unlike classical time series methods, e.g. ARIMA, there are no parametric assumptions on the distribution of the errors, and non-linear relationships between response and predictors can be captured.
- The data is snapshots of the USD value of Coinbase every minute over 2018.

#### Statistician's note
- We choose to build a model which provides strong predictive power, at the expense of reduced explanatory power.
- Our choice to use a recurrent neural network is predicated on each observation in the time series being dependent on previous observations. The ordering of the observations therefore matters and $X$ is not iid.
- Once the input data is appropriately scaled, model building starts with 'feature selection' - identifying the relevant features to include in the model.

- In this notebook, we assume that we've already identifed the relevant set of features (i.e. there is only one time series provided).
- Our primary concern is assessing the extent to which the model is over-fitting, by comparing the in- and out-of-sample MSEs.

#### Implementation notes
- It is important to ensure that `shuffle=False` in the fit function, otherwise the ordering of sequences is not preserved. This is especially important for methods which have memory beyond the current sequence (i.e. all methods except RNNs).
- Time series cross-validation must be used for hyper-parameter tuning because the ordering of the data matters. In particular, the model must never use training data more recent than the forecasting observation date.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
import tensorflow as tf
from datetime import timedelta

from sklearn.metrics import mean_squared_error
from sklearn.model_selection import KFold, TimeSeriesSplit, GridSearchCV

import keras.initializers
from keras.layers import Dense, Layer, LSTM, GRU, SimpleRNN, RNN
from keras.models import Sequential
from keras.models import load_model
from keras.regularizers import l1, l2
from keras.callbacks import EarlyStopping
from keras.wrappers.scikit_learn import KerasRegressor

# note that the directory containing these two .py's must be in the path variable:
from alphaRNN import AlphaRNN
from alphatRNN import AlphatRNN
```

```python
%matplotlib inline
```

### Example Data
- The example dataset $X$ is a chronologically ordered time series. The ordering of the observations matters and each observation is not assumed to be independent (as with cross-sectional classification data).

- Each observation in $X$ has one variable (a.k.a. univariate time series).

Loading the data into a Pandas Dataframe, then viewing the first ten observations and the distribution of the labels:

```python
df = pd.read_csv('../data/coinbase.csv', index_col=1)

df.index = pd.to_datetime(df.index, infer_datetime_format=True)
```

The dataset contains missing values; in order to prevent this causing errors, we replace these with adjacent values from the time series.

```python
nof_missing_values = sum(np.isnan(df['USD']))

print(nof_missing_values, 'observations are missing.')
print('This is {:.3f}% of the total.'.format(nof_missing_values*100/len(df)))
```

    1662 observations are missing.
    This is 0.368% of the total.

```python
df = df.fillna(method="backfill")

nof_missing_values = sum(np.isnan(df['USD']))

print('Now', nof_missing_values, 'observations are missing.')
```

    Now 0 observations are missing.

# RNN Regression
We consider a univariate prediction problem where the time series is given by 'USD' in the data frame, and for each input sequence we predict the value 4 time-steps into the future.

```python
use_features = ['USD'] # continuous input
target = ['USD'] # continuous output
n_steps_ahead = 4 # forecasting horizon
```

### Stationarity
It is essential to determine whether the time series is "stationary". Informally, stationarity is when the auto-covariance is independent of time. Failure to establish stationarity will almost certainly lead to misinterpretation of model identification and diagnostic tests. Moreover, stationarity is decisive in characterizing the prediction problem and whether to use a more advanced architecture. In particular, we can expect a plain RNN to perform poorly if the data is non-stationary as the RNN exhibits fixed auto-covariance.

We perform an Augmented Dickey-Fuller test to establish stationarity under the assumption that the time series has a constant bias but does not exhibit a time trend. In other words, we assume that the time series is already de-trended.

If the stationarity test fails, even after first de-trending the time series, then one potential recourse is to simply take differences of time series and predict $\Delta y_t$.

The null hypothesis of the Augmented Dickey-Fuller is that there is a unit root, with the alternative that there is no unit root. If the p-value is above $(1-\alpha)$, then we cannot reject that there is a unit root. Note that a subset of the time series is used to reduce the memory requirements of the test. We use the first 200,000 samples to test for stationarity. While the test statistic is sensitive to the data size, the ADF test is always accepted at the 99\% level.

```python
sample = df['USD'][:200000]
adf, p, usedlag, nobs, cvs, aic = sm.tsa.stattools.adfuller(sample)
```

```python
adf_results_string = 'ADF: {}\np-value: {},\nN: {}, \ncritical values: {}'
print(adf_results_string.format(adf, p, nobs, cvs))
```

    ADF: -1.628446514215208
    p-value: 0.46830397402696317,
    N: 19953,
    critical values: {'1%': -3.4306777773505996, '5%': -2.86168486605018, '10%': -2.566847108243414}

Here we accept the null as the p-value is larger than 0.01, thus we can not reject the test at the 99% confidence level. This suggests that the time series is **non-stationary**.

#### Autoregressive Model Identification: The partial auto-correlation
It is important to determine the number of lags, the sequence length, required in the RNN by statistical analysis. A brute-force approach will in general be too time-consuming.

A partial auto-correlation at lag $h\geq 2$ is a conditional auto-correlation between a variable, $X_t$, and its $h^{th}$ lag, $X_{t-h}$ under the assumption that we control for the values of the intermediate lags, $X_{t-1},\dots, X_{t-h+1}$:

$$\begin{align}\tau_h&:=\tau(X_t, X_{t-h}; X_{t-1},\dots, X_{t-h+1})\\
&:=\frac{\gamma(X_t, X_{t-h}; X_{t-1},\dots, X_{t-h+1})}{\sqrt{\gamma(X_t |X_{t-1},\dots, X_{t-h+1})\gamma(X_{t-h} |X_{t-1},\dots, X_{t-h+1}))}},
\end{align}$$
where $\gamma_h:=\gamma(X_tX_{t-h})$ is the lag-$h$ autocovariance. The partial autocorrelation function $\tau_h:\mathbb{N} \rightarrow [-1,1]$ is a map $h:\mapsto \tau_h$.

The estimated partial auto-correlation function (PACF) can be used to identify the order of an autoregressive time series model. Values of $|\tau_h|$ greater or equal to $\frac{\Phi^{-1}(\alpha)}{\sqrt{T}}$, where $T$ is the number of observations and $\Phi(z)$ is the standard normal CDF, are significant lag $h$ partial autocorelations at the $\alpha$ confidence level.

We use the stattools package to estimate the PACF. The `nlags` parameter is the maximum number of lags used for PACF estimation.

```python
pacf = sm.tsa.stattools.pacf(df[use_features], nlags=30)
```

Since $\Phi^{-1}(0.99) \simeq 2.58$:

```python
T = len(df[use_features])

sig_test = lambda tau_h: np.abs(tau_h) > 2.58/np.sqrt(T)
```

We find the first lag which isn't significant at the 99% level and automatically determine the number of lags needed in our autoregressive model as one below this value.

```python
for i in range(len(pacf)):
    if sig_test(pacf[i]) == False:
        n_steps = i - 1
        print('n_steps set to', n_steps)
        break
```

    n_steps set to 4

This may lead to a high order model, with more lags than strictly necessary. We could view this value, informally, as an upper bound on the number of lags needed. We can also simply identify the order of the model based on the plot of the PACF. In this case, a minimum of 4 lags appears satisfactory, although more may be needed. Unlike autoregressive models, the advantage of using fewer parameters is purely computational as adding more lags does not increase the number of parameters, only the size of the tensorial representation of the sequence data in TensorFlow.

```python
plt.plot(pacf, label='pacf')
plt.plot([2.58/np.sqrt(T)]*30, label='99% confidence interval (upper)')
plt.plot([-2.58/np.sqrt(T)]*30, label='99% confidence interval (lower)')
plt.xlabel('number of lags')
plt.legend();
```

### Splitting the time series into training and testing sets
Split the training and test set by using the first 80% of the time series and the remaining 20% for the test set. Note that the test set must be in the future of the training set to avoid look-ahead bias. Also, random sampling of the data can not be used as this would eliminate the auto-correlation structure.

```python
train_weight = 0.8
split = int(len(df) * train_weight)

df_train = df[use_features].iloc[:split]
df_test = df[use_features].iloc[split:]
```

### Scaling
Standardization of the data is important to avoid potential scaling difficulties in the fitting of the model. When there is more than one feature (covariate), scaling avoids one feature dominating over another due to disparate scales.

To avoid introducing a look-ahead bias into the prediction, we must re-scale the training data without knowledge of the test set. Hence, we will simply standardize the training set using the mean and standard deviation of the training set and not the whole time series. Additionally, to avoid introducing a systematic bias into test set, we use the identical normalization for the test set - the mean and standard deviation of the training set are used to normalize the test set.

```python
# note that for a multivariate time series, you would need to scale
# each variable by its own mean and standard deviation in the training set
mu = np.float(df_train.mean())
sigma = np.float(df_train.std())

stdize_input = lambda x: (x - mu) / sigma

df_train = df_train.apply(stdize_input)
df_test = df_test.apply(stdize_input)
```

### Data formatting for RNNs

TensorFlow uses tensors to represent data. To perform sequence learning, the time series of variables must be transformed to a series of over-lapping sub-sequences.

For example, consider a univariate time series of increasing integers.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Setting the sequence length to 10, for example, we move the window forward by one observation at a time and construct new sequences:

1 2 3 4 5 6 7 8 9 10

2 3 4 5 6 7 8 9 10 11

3 4 5 6 7 8 9 10 11 12

4 5 6 7 8 9 10 11 12 13

5 6 7 8 9 10 11 12 13 14

6 7 8 9 10 11 12 13 14 15

Let's define the following function for reshaping the data into one-step ahead times series prediction format.

```python
def get_lagged_features(df, n_steps, n_steps_ahead):
    """
    df: pandas DataFrame of time series to be lagged
    n_steps: number of lags, i.e. sequence length
    n_steps_ahead: forecasting horizon
    """
    lag_list = []

    for lag in range(n_steps + n_steps_ahead - 1, n_steps_ahead - 1, -1):
        lag_list.append(df.shift(lag))
    lag_array = np.dstack([i[n_steps+n_steps_ahead-1:] for i in lag_list])
    # We swap the last two dimensions so each slice along the first dimension
    # is the same shape as the corresponding segment of the input time series
    lag_array = np.swapaxes(lag_array, 1, -1)
    return lag_array
```

We shall first transform the training input and output data.

```python
x_train = get_lagged_features(df_train, n_steps, n_steps_ahead)
y_train =  df_train.values[n_steps + n_steps_ahead - 1:]
y_train_timestamps = df_train.index[n_steps + n_steps_ahead - 1:]

x_test = get_lagged_features(df_test, n_steps, n_steps_ahead)
y_test =  df_test.values[n_steps + n_steps_ahead - 1:]
y_test_timestamps = df_test.index[n_steps + n_steps_ahead - 1:]
```

Print the shapes of each tensor. The first digit is the number of observations. For feature arrays, the second digit is the sequence length (i.e. the number of lags in the model) and the final digit is the dimension of each element in the sequence or output vector respectively. In this case, the example performs univariate time series analysis and so the dimension of the input and output is 1.

```python
print([tensor.shape for tensor in (x_train, y_train, x_test, y_test)])
```

    [(361337, 4, 1), (361337, 1), (90330, 4, 1), (90330, 1)]

### Model Specification

```python
def AlphatRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(AlphatRNN(n_units, activation='tanh', recurrent_activation='sigmoid', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def AlphaRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(AlphaRNN(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def SimpleRNN_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(SimpleRNN(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True, stateful=False))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def GRU_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(GRU(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model

def LSTM_(n_units = 10, l1_reg=0, seed=0):
  model = Sequential()
  model.add(LSTM(n_units, activation='tanh', kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), recurrent_initializer=keras.initializers.orthogonal(seed), kernel_regularizer=l1(l1_reg), input_shape=(x_train.shape[1], x_train.shape[-1]), unroll=True))
  model.add(Dense(1, kernel_initializer=keras.initializers.glorot_uniform(seed), bias_initializer=keras.initializers.glorot_uniform(seed), kernel_regularizer=l1(l1_reg)))
  model.compile(loss='mean_squared_error', optimizer='adam')
  return model
```

```python
max_epochs = 1000
batch_size = 1000
```

Use a batch size of 1000 as the dataset is reasonably large and the training time would be too long otherwise. 20 epochs have been used here, but a better approach would be to use a stopping criteria through a call back.

```python
es = EarlyStopping(monitor='loss', mode='min', verbose=1, patience=100, min_delta=1e-7, restore_best_weights=True)
```

```python
params = {
    'rnn': {
        'model': None, 'function': SimpleRNN_, 'l1_reg': 0.0, 'H': 20,
        'color': 'blue', 'label':'RNN'},
    'alpharnn': {
        'model': None, 'function': AlphaRNN_, 'l1_reg': 0.0, 'H': 10,
        'color': 'green', 'label': '$\\alpha$-RNN' },
    'alphatrnn': {
        'model': None, 'function':AlphatRNN_, 'l1_reg': 0.0, 'H': 5,
        'color': 'cyan', 'label': '$\\alpha_t$-RNN'},
    'gru': {
        'model': None, 'function':GRU_,'l1_reg': 0.0, 'H': 10,
        'color': 'orange', 'label': 'GRU'},
    'lstm': {
        'model': None, 'function': LSTM_,'l1_reg': 0.0, 'H': 10,
        'color':'red', 'label': 'LSTM'}
}
```

# Optionally, load pre-trained models
Training the models takes several hours. To save time, you may load the already fitted models instead:

```python
do_training = False # Set to True if you wish to train your own models
```

```python
if do_training is False:
    custom_objects = {'AlphaRNN': AlphaRNN, 'AlphatRNN': AlphatRNN}
    for key in params.keys():
        params[key]['model'] = load_model('trained-RNNs/RNNs-Bitcoin-' + key + '.hdf5', custom_objects=custom_objects)
```

# Cross-validation

The cell below performs a grid search to optimise the `n_units` and `l1_reg` for each of the models using the training data.

The results are cross-validated to avoid over-fitting. Scikit-Learn's `TimeSeriesSplit` function is used to partition the data into 5 pairs of training and testing sets, where the test data is always ahead of the training data in time.

```python
cross_val = False # WARNING: Changing this to True will take many hours to run

if do_training and cross_val:
    n_units = [5, 10, 20]
    l1_reg = [0, 0.001, 0.01, 0.1]

    # A dictionary containing a list of values to be iterated through
    # for each parameter of the model included in the search
    param_grid = {'n_units': n_units, 'l1_reg': l1_reg}

    # In the kth split, TimeSeriesSplit returns first k folds
    # as training set and the (k+1)th fold as test set.
    tscv = TimeSeriesSplit(n_splits = 5)

    # A grid search is performed for each of the models, and the parameter set which
    # performs best over all the cross-validation splits is saved in the `params` dictionary
    for key in params.keys():
        print('Performing cross-validation. Model:', key)
        model = KerasRegressor(build_fn=params[key]['function'], epochs=max_epochs,
                               batch_size=batch_size, verbose=2)
        grid = GridSearchCV(estimator=model, param_grid=param_grid,
                            cv=tscv, n_jobs=1, verbose=2)
        grid_result = grid.fit(x_train, y_train, callbacks=[es])
        print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))

        means = grid_result.cv_results_['mean_test_score']
        stds = grid_result.cv_results_['std_test_score']
        params_ = grid_result.cv_results_['params']
        for mean, stdev, param_ in zip(means, stds, params_):
            print("%f (%f) with %r" % (mean, stdev, param_))

        params[key]['H'] = grid_result.best_params_['n_units']
        params[key]['l1_reg']= grid_result.best_params_['l1_reg']
```

# Train cross-validated model

If the grid search was performed, the parameters `n_units` and `l1_reg` with the best average performance across the cross-validation splits are used to train the models on the entire training set. If not, the values from the initialisation of `params` above are used.

```python
if do_training is True:
    for key in params.keys():
        tf.random.set_seed(0)
        print('Training', key, 'model')
        model = params[key]['function'](params[key]['H'], params[key]['l1_reg'])
        model.fit(x_train, y_train, epochs=max_epochs,
                  batch_size=batch_size, callbacks=[es], shuffle=False)
        params[key]['model'] = model
```

Optionally save the fitted models:

```python
for key in params.keys():
    params[key]['model'].save('RNNs-Bitcoin-SAVED-' + key + '.hdf5', overwrite=True)  # creates a HDF5 file
```

### Print out the value of $\alpha \in [0,1]$ for the alpha-RNN model:

```python
def sigmoid(x):
    return (1 / (1 + np.exp(-x)))
```

```python
model = params['alpharnn']['model']
names = [weight.name for layer in model.layers for weight in layer.weights]
weights = model.get_weights()

for name, weight in zip(names, weights):
    if 'alpha:0' in name:
        print("alpha = " + str(sigmoid(weight)))
```

    alpha = [0.24113195]

### Prediction

We shall now apply the fitted RNN models to the training set and the test set, separately. We can then informally assess the extent of over-fitting.

```python
for key in params.keys():
    model = params[key]['model']
    model.summary()

    params[key]['pred_train'] = model.predict(x_train, verbose=1)
    params[key]['MSE_train'] = mean_squared_error(y_train, params[key]['pred_train'])

    params[key]['pred_test'] = model.predict(x_test, verbose=1)
    params[key]['MSE_test'] = mean_squared_error(y_test, params[key]['pred_test'])
```

    Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    simple_rnn_1 (SimpleRNN)     (None, 20)                440
    _________________________________________________________________
    dense_1 (Dense)              (None, 1)                 21
    =================================================================
    Total params: 461
    Trainable params: 461
    Non-trainable params: 0
    _________________________________________________________________
    361337/361337 [==============================] - 15s 40us/step
    90330/90330 [==============================] - 4s 39us/step
    Model: "sequential_2"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    alpha_rnn_1 (AlphaRNN)       (None, 10)                121
    _________________________________________________________________
    dense_2 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 132
    Trainable params: 132
    Non-trainable params: 0
    _________________________________________________________________
    361337/361337 [==============================] - 16s 44us/step
    90330/90330 [==============================] - 4s 43us/step
    Model: "sequential_3"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    alphat_rnn_1 (AlphatRNN)     (None, 5)                 80
    _________________________________________________________________
    dense_3 (Dense)              (None, 1)                 6
    =================================================================
    Total params: 86
    Trainable params: 86
    Non-trainable params: 0
    _________________________________________________________________
    361337/361337 [==============================] - 18s 51us/step
    90330/90330 [==============================] - 4s 49us/step
    Model: "sequential_4"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    gru_1 (GRU)                  (None, 10)                360
    _________________________________________________________________
    dense_4 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 371
    Trainable params: 371
    Non-trainable params: 0
    _________________________________________________________________
    361337/361337 [==============================] - 24s 67us/step
    90330/90330 [==============================] - 6s 63us/step
    Model: "sequential_5"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    lstm_1 (LSTM)                (None, 10)                480
    _________________________________________________________________
    dense_5 (Dense)              (None, 1)                 11
    =================================================================
    Total params: 491
    Trainable params: 491
    Non-trainable params: 0
    _________________________________________________________________
    361337/361337 [==============================] - 24s 67us/step
    90330/90330 [==============================] - 6s 66us/step

### Model Performance

Finally assess the performance of the model in and out-of-sample using the MSE. We expect the mean error to be larger on the test set than on the training set.

```python
print('training set:', len(y_train))
print('testing set:', len(y_test))

# Upper limits for `l` & `u` in the cells below:
```

    training set: 361337
    testing set: 90330

```python
print(params.keys())

# Set `compare` in the cells below to a list
# containing any subset of these:
```

    dict_keys(['rnn', 'alpharnn', 'alphatrnn', 'gru', 'lstm'])

```python
max_pts = 10**4
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling ratio for under `max_pts`
                                        # per series.  Set `None` to disable.

fig = plt.figure(figsize=(15,8))
x_vals = y_train_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds]
    label = params[key]['label'] + ' (train MSE: %.2e)' % params[key]['MSE_train']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.plot(x_vals, y_train[l:u:ds], c="black", label="Observed", lw=1)
start, end = x_vals.min(), x_vals.max()
xticks =  [start.date() + timedelta(days=(1+i)) for i in range(1 + (end - start).days)]
xticks = xticks[::max(1, len(xticks)//30)]
for t in xticks: plt.axvline(x=t, c='gray', linewidth=0.5, zorder=0)
plt.xticks(xticks, rotation=70)
plt.xlim(start, end)
plt.ylabel('$\hat{Y}$', rotation=0, fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Outputs (Training)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling ratio for under `max_pts`
                                        # per series.  Set `None` to disable.
fig = plt.figure(figsize=(15,8))
x_vals = y_test_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds]
    label = params[key]['label'] + ' (test MSE: %.2e)' % params[key]['MSE_test']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
plt.plot(x_vals, y_test[l:u:ds], c="black", label="Observed", lw=1)
start, end = x_vals.min(), x_vals.max()
xticks =  [start.date() + timedelta(days=(1+i)) for i in range(1 + (end - start).days)]
xticks = xticks[::max(1, len(xticks)//30)]
for t in xticks: plt.axvline(x=t, c='gray', linewidth=0.5, zorder=0)
plt.xticks(xticks, rotation=70)
plt.xlim(start, end)
plt.ylabel('$\hat{Y}$', rotation=0, fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Outputs (Testing)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # e.g. (None, 100000) lower and upper indices of range to plot
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling ratio for under `max_pts`
                                        # per series.  Set `None` to disable.
fig = plt.figure(figsize=(15,8))
x_vals = y_test_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_test'][l:u:ds] - y_test[l:u:ds]
    label = params[key]['label'] + ' (test MSE: %.2e)' % params[key]['MSE_test']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
start, end = x_vals.min(), x_vals.max()
xticks =  [start.date() + timedelta(days=(1+i)) for i in range(1 + (end - start).days)]
xticks = xticks[::max(1, len(xticks)//30)]
plt.axhline(0, linewidth=0.8)
for t in xticks: plt.axvline(x=t, c='gray', linewidth=0.5, zorder=0)
plt.xticks(xticks, rotation=80)
plt.xlim(start, end)
plt.ylabel('$\hat{Y}-Y$', fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Error (Testing)', fontsize=16);
```

```python
compare = params.keys() # e.g. ['rnn', 'alpharnn'] or ['lstm']
l, u = (None, None) # lower and upper indices of range to plot - e.g. (None, 10000)
ds = max(1, len(y_train[l:u])//max_pts) # Downsampling ratio for under `max_pts`
                                        # per series.  Set `None` to disable.
fig = plt.figure(figsize=(15,8))
x_vals = y_train_timestamps[l:u:ds]
for key in compare:
    y_vals = params[key]['pred_train'][l:u:ds] - y_train[l:u:ds]
    label = params[key]['label'] + ' (train MSE: %.2e)' % params[key]['MSE_train']
    plt.plot(x_vals, y_vals, c=params[key]['color'], label=label, lw=1)
start, end = x_vals.min(), x_vals.max()
xticks =  [start.date() + timedelta(days=(1+i)) for i in range(1 + (end - start).days)]
xticks = xticks[::max(1, len(xticks)//30)]
plt.axhline(0, linewidth=0.8)
for t in xticks: plt.axvline(x=t, c='gray', linewidth=0.5, zorder=0)
plt.xticks(xticks, rotation=80)
plt.xlim(start, end)
plt.ylabel('$\hat{Y}-Y$', fontsize=14)
plt.legend(loc="best", fontsize=12)
plt.title('Observed vs Model Error (Training)', fontsize=16);
```

### Model Diagnostics

A fitted time series model must be examined for underfitting with a white noise test. We analyze the model residuals (i.e. the error $u_t$) to determine whether it is white noise or whether it is auto-correlated. The latter case provides statistical evidence that more lags are needed in the RNN. Box and Pierce propose the Portmanteau statistic:

$$Q^*(m)=T\sum_{l=1}^m\hat{\tau}_l^2,$$
as a test statistic for the null hypothesis $H_0:\tau_1=\dots=\tau_m=0$ against the alternative hypothesis $H_a:\tau_i\neq 0$ for some $i\in\{1,\dots,m\}$, where $T$ is the number of observations, $\hat{\tau}_i$ are the sample autocorrelations of the residual, and $m$ is the maximum lag used in the test. There are several heuristics in the statistics literature to determine the maximum lag such as the Schwert statistic.

The Box-Pierce statistic follows an asymptotically chi-squared distribution with $m$ degrees of freedom.

The Ljung-Box test statistic increases the power of the test in finite samples:
$$Q(m)=T(T+2)\sum_{l=1}^m\frac{\hat{\tau}_l^2}{T-l}$$
This statistic also follows an asymptotically chi-squared distribution with $m$ degrees of freedom. The decision rule is to reject $H_0$ if $Q(m)>\chi_{\alpha}^2$ where $\chi_{\alpha}^2$ denotes the $100(1-\alpha)^{th}$ percentile of a chi-squared distribution with m degrees of freedom and is the significance level for rejecting $H_0$.

The test can be time consuming and we select a subset of the residuals. Here we simply set the maximum lag to 20. In the results below, we find that the p-values are all smaller than 0.01, indicating that we can reject the null at the 99% confidence level for any lag. This is strong evidence that the model is under-fitting and more lags are needed in our model. Unlike an auto-regressive model, increasing the number of lags in the RNN does not increase the number of weights. Thus there is no danger of over-fitting by increasing the lag, although there will be an increase in the training time.

```python
# number of samples to use for computing test statistic
n = 100000
```

```python
params.keys()
```

    dict_keys(['rnn', 'alpharnn', 'alphatrnn', 'gru', 'lstm'])

```python
key = 'alpharnn'
predicted = params[key]['pred_test']
residual = y_test[-n:] - predicted[-n:]
```

```python
lb, p = sm.stats.diagnostic.acorr_ljungbox(residual, lags=20, boxpierce=False)
```

The Box-Ljung test statistics are

```python
lb
```

    array([61562.07460753, 90466.80953156, 98349.37299933, 98710.33220986,
           98710.73332643, 98756.13543138, 98854.37711435, 98948.93155159,
           98986.97252624, 98987.50348873, 99007.36449068, 99071.19145694,
           99135.55844865, 99194.2390851 , 99238.35964502, 99290.24934538,
           99360.44600305, 99421.8768069 , 99463.51398864, 99467.77658386])

The p-values are

```python
p
```

    array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
           0., 0., 0.])


```python
import numpy as np
import warnings
warnings.filterwarnings("ignore")
```

```python
l = .8
M = 1.
m =.1
g = 9.8
dT = 0.02
```

## Pure Python

```python
import math

def calc_rhe_pure_python(q):
    return [q[2],
            q[3],
             (g*m*math.sin(2*q[1])/2 + l*m*q[3]**2*math.sin(q[1]))/(M + m*math.sin(q[1])**2),
            -(g*(M + m)*math.sin(q[1]) + (l*m*q[3]**2*math.sin(q[1]))*math.cos(q[1]))/(l*(M + m*math.sin(q[1])**2))]

def RK4_pure_python(q, dt):
    k1 = calc_rhe_pure_python(q)
    k2 = calc_rhe_pure_python([e+d*dt/2 for e, d in zip(q, k1)])
    k3 = calc_rhe_pure_python([e+d*dt/2 for e, d in zip(q, k2)])
    k4 = calc_rhe_pure_python([e+d*dt/2 for e, d in zip(q, k3)])
    return [x+dt*(k1x/6+k2x/3+k3x/3+k4x/6) for x, k1x, k2x, k3x, k4x in zip(q, k1, k2, k3, k4)]
```

```python
x0 = [0., -math.pi+0.1, 0., 0.]

print("1. Pure Python\n\t:", end='')
%timeit x1 = RK4_pure_python(x0, dT)
```

    1. Pure Python
    	:12.5 µs ± 283 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

## Numpy

```python
def calc_rhe_numpy(q):
    return np.array([q[2],
            q[3],
             (g*m*np.sin(2*q[1])/2 + l*m*q[3]**2*np.sin(q[1]))/(M + m*np.sin(q[1])**2),
            -(g*(M + m)*np.sin(q[1]) + (l*m*q[3]**2*np.sin(q[1]))*np.cos(q[1]))/(l*(M + m*np.sin(q[1])**2))])

def RK4_numpy(q, dt):
    k1 = calc_rhe_numpy(q)
    k2 = calc_rhe_numpy(q+k1*dt/2)
    k3 = calc_rhe_numpy(q+k2*dt/2)
    k4 = calc_rhe_numpy(q+k3*dt/2)
    return q+dt*(k1/6+k2/3+k3/3+k4/6)
```

```python
x0 = np.array([0., -math.pi+0.1, 0., 0.])

print("2. Numpy\n\t:", end='')
%timeit x1 = RK4_numpy(x0, dT)
```

    2. Numpy
    	:66.2 µs ± 1.61 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

## Numba - Pure python

```python
from numba import jit

@jit(nopython=True)
def calc_rhe_pure_python_numba(q):
    return [q[2],
            q[3],
             (g*m*math.sin(2*q[1])/2 + l*m*q[3]**2*math.sin(q[1]))/(M + m*math.sin(q[1])**2),
            -(g*(M + m)*math.sin(q[1]) + (l*m*q[3]**2*math.sin(q[1]))*math.cos(q[1]))/(l*(M + m*math.sin(q[1])**2))]

@jit(nopython=True)
def RK4_pure_python_numba(q, dt):
    k1 = calc_rhe_pure_python_numba(q)
    k2 = calc_rhe_pure_python_numba([e+d*dt/2 for e, d in zip(q, k1)])
    k3 = calc_rhe_pure_python_numba([e+d*dt/2 for e, d in zip(q, k2)])
    k4 = calc_rhe_pure_python_numba([e+d*dt/2 for e, d in zip(q, k3)])
    return [x+dt*(k1x/6+k2x/3+k3x/3+k4x/6) for x, k1x, k2x, k3x, k4x in zip(q, k1, k2, k3, k4)]
```

```python
x0 = [0., -math.pi+0.1, 0., 0.]

# initial call for JIT compiler
x1 = RK4_pure_python_numba(x0, dT)

print("3. Numba - Pure Python\n\t:", end='')
%timeit x1 = RK4_pure_python_numba(x0, dT)
```

    3. Numba - Pure Python
    	:8.82 µs ± 72.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

## Numba - Numpy

```python
from numba import jit

@jit(nopython=True)
def calc_rhe_numpy_numba(q):
    return np.array([q[2],
            q[3],
             (g*m*np.sin(2*q[1])/2 + l*m*q[3]**2*np.sin(q[1]))/(M + m*np.sin(q[1])**2),
            -(g*(M + m)*np.sin(q[1]) + (l*m*q[3]**2*np.sin(q[1]))*np.cos(q[1]))/(l*(M + m*np.sin(q[1])**2))])

@jit(nopython=True)
def RK4_numpy_numba(q, dt):
    k1 = calc_rhe_numpy_numba(q)
    k2 = calc_rhe_numpy_numba(q+k1*dt/2)
    k3 = calc_rhe_numpy_numba(q+k2*dt/2)
    k4 = calc_rhe_numpy_numba(q+k3*dt/2)
    return q+dt*(k1/6+k2/3+k3/3+k4/6)
```

```python
x0 = np.array([0., -math.pi+0.1, 0., 0.])

# initial call for JIT compiler
x1 = RK4_numpy_numba(x0, dT)

print("4. Numba Numpy\n\t:", end='')
%timeit x1 = RK4_numpy_numba(x0, dT)
```

    4. Numba Numpy
    	:1.46 µs ± 22.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

## Cython

```python
%load_ext Cython
```

```cython
%%cython
import numpy as np
cimport numpy as np
cimport cython

ctypedef double DTYPE_t

cdef double l = .8
cdef double M = 1.
cdef double m =.1
cdef double g = 9.8
cdef double dT = 0.02

@cython.boundscheck(False)
@cython.wraparound(False)
cdef calc_rhe_numpy_cython(np.ndarray[DTYPE_t, ndim=1] q):
    return np.array([q[2],
            q[3],
             (g*m*np.sin(2*q[1])/2 + l*m*q[3]**2*np.sin(q[1]))/(M + m*np.sin(q[1])**2),
            -(g*(M + m)*np.sin(q[1]) + (l*m*q[3]**2*np.sin(q[1]))*np.cos(q[1]))/(l*(M + m*np.sin(q[1])**2))])

@cython.boundscheck(False)
@cython.wraparound(False)
def RK4_numpy_cython(np.ndarray[DTYPE_t, ndim=1] q, double dt):
    cdef np.ndarray[DTYPE_t, ndim=1] k1
    cdef np.ndarray[DTYPE_t, ndim=1] k2
    cdef np.ndarray[DTYPE_t, ndim=1] k3
    cdef np.ndarray[DTYPE_t, ndim=1] k4

    k1 = calc_rhe_numpy_cython(q)
    k2 = calc_rhe_numpy_cython(q+k1*dt/2)
    k3 = calc_rhe_numpy_cython(q+k2*dt/2)
    k4 = calc_rhe_numpy_cython(q+k3*dt/2)
    return q+dt*(k1/6+k2/3+k3/3+k4/6)
```

```python
x0 = np.array([0., -math.pi+0.1, 0., 0.])

print("6. Cython - Numpy\n\t:", end='')
%timeit x1 = RK4_numpy_cython(x0, dT)
```

    6. Cython - Numpy
    	:52 µs ± 693 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

## Sympy

```python
 import sympy as sy
 def gen_rhe_sympy():
        q  = sy.symbols('q:{0}'.format(4))
        qd = q[2:4]

        I = sy.Matrix([[1, 0, 0, 0],
                      [0, 1, 0, 0],
                      [0, 0, M + m, l*m*sy.cos(q[1])],
                      [0, 0, l*m*sy.cos(q[1]), l**2*m]])
        f = sy.Matrix([
                       qd[0],
                       qd[1],
                       l*m*sy.sin(q[1])*qd[1]**2,
                      -g*l*m*sy.sin(q[1])])
        return sy.lambdify([q], sy.simplify(I.inv()*f))

calc_rhe_sympy = gen_rhe_sympy()

def RK4_sympy(q, dt):
    k1 = calc_rhe_sympy(q)
    k2 = calc_rhe_sympy(q+k1*dt/2)
    k3 = calc_rhe_sympy(q+k2*dt/2)
    k4 = calc_rhe_sympy(q+k3*dt/2)
    return q+dt*(k1/6+k2/3+k3/3+k4/6)
```

```python
x0 = np.array([0., -math.pi+0.1, 0., 0.])

print("7. Sympy - Numpy\n\t:", end='')
%timeit x1 = RK4_sympy(x0, dT)
```

    7. Sympy - Numpy
    	:97.3 µs ± 1.29 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

## Eigen - Swig
```
python setup.py build_ext --inplace
```

```python
from swig_eigen.swig_eigen_mod import RK4_eigen
```

```python
x0 = np.array([0., -math.pi+0.1, 0., 0.])

print("8. Swig -Eigen\n\t:", end='')
%timeit x1 = RK4_eigen(x0, dT)
```

    8. Swig -Eigen
    	:1.89 µs ± 15.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)


# Taylor Problem 16.14 version A (Update 23-April-2021)

We'll plot at various times a wave $u(x,t)$ that is defined by its initial shape at $t=0$ from $x=0$ to $x=L$, using a Fourier sine series to write the result at a general time t:

$\begin{align}
  u(x,t) = \sum_{n=1}^{\infty} B_n \sin(k_n x)\cos(\omega_n t)
  \;,
\end{align}$

with $k_n = n\pi/L$ and $\omega_n = k_n c$, where $c$ is the wave speed.   Here the coefficients are given by

$\begin{align}
  B_n = \frac{2}{L}\int_0^L u(x,0) \sin\frac{n\pi x}{L} \, dx
  \;.
\end{align}$

* Created 28-Mar-2019. Last revised 30-Mar-2019 by Dick Furnstahl (furnstahl.1@osu.edu).
* This version sums only over odd n, which are called $m = 2n + 1$.

```python
%matplotlib inline
```

```python
import numpy as np
import matplotlib.pyplot as plt

from matplotlib import animation, rc
from IPython.display import HTML
```

```python
print("Hello")
```

    Hello

First define functions for the $t=0$ wave function form (here a triangle) and for the subsequent shape at any time $t$ based on the wave speed `c_wave`.

```python
def B_coeff(m):
    """Fourier coefficient for the n = 2*m + 1 term in the expansion.
    """
    temp = 2.*m + 1
    return (-1)**m * 8. / (temp * np.pi)**2
```

```python
def k(m, L):
    """Wave number for n = 2*m + 1.
    """
    return (2.*m + 1.) * np.pi / L
```

```python
def u_triangular(x_pts, t, m_max=20, c_wave=1., L=1.):
    """Returns the wave from the sum of Fourier components.
    """
    y_pts = np.zeros(len(x_pts))
    # define y_pts as the same size as x_pts
    #14.12
    #     h = 1.05 * 10**-34
#     m = 1.675 * 10**-27
#     E = 1.6022 *10**-12
#     coeff = (h * h) / (2.0 * m * E)

    #coeff* (0.25 + 0.75 * np.cos(x_pts) + 0.75 * np.cos(x_pts) + 2.25 * np.square(np.cos(x_pts)))
    # 14.15
    # g = 0.5
#     g2 = g * g
#     R = 2.0
#     R2 = R * R
#     b = 1.0
#     b2 = b * b
#     factor = np.sqrt((R2-b2) * (R2- g2*b2))/(2* (np.sqrt(R2- g2*b2) - g * np.sqrt(R2-b2) ))
#     ((b/np.sin(x_pts)) * abs(factor))/R2

#14.23

    g = 1.0
    E = 1.0
    y_pts += (g/E) * (np.square(np.pi) * (np.pi - x_pts))/(np.square(x_pts) * np.square(2* np.pi - x_pts) * np.sin(x_pts))
    return y_pts
```

First look at the initial ($t=0$) wave form.

```python
L = 1.
m_max = 20
c_wave = 1
omega_1 = np.pi * c_wave / L
tau = 2.*np.pi / omega_1

# Set up the array of x points (whatever looks good)
x_min = 0.1
x_max = np.pi-0.1
delta_x = 0.1
x_pts = np.arange(x_min, x_max, delta_x)

# Make a figure showing the initial wave.
t_now = 0.

fig = plt.figure(figsize=(6,4), num='Standing wave')
ax = fig.add_subplot(1,1,1)
ax.set_xlim(x_min-0.1, x_max + 0.1)
gap = 0.1
ax.set_xlabel(r'$Theta$')
ax.set_ylabel(r'$Cross Section$')
ax.set_title(rf'$Cross Section$')

line, = ax.semilogy(x_pts,
                u_triangular(x_pts, t_now, m_max, c_wave, L),
                color='blue', lw=2)

fig.tight_layout()

```

Next make some plots at an array of time points.

```python
t_array = tau * np.arange(0., 1.125, .125)

fig_array = plt.figure(figsize=(12,12), num='Standing wave')

for i, t_now in enumerate(t_array):
    ax_array = fig_array.add_subplot(3, 3, i+1)
    ax_array.set_xlim(x_min, x_max)
    gap = 0.1
    ax_array.set_ylim(-1. - gap, 1. + gap)
    ax_array.set_xlabel(r'$x$')
    ax_array.set_ylabel(r'$u(x, t)$')
    ax_array.set_title(rf'$t = {t_now/tau:.3f}\tau$')

    ax_array.plot(x_pts,
                  u_triangular(x_pts, t_now, m_max, c_wave, L),
                  color='blue', lw=2)

fig_array.tight_layout()
fig_array.savefig('Taylor_Problem_16p14.png',
                   bbox_inches='tight')
```

Now it is time to animate!

```python
# Set up the t mesh for the animation.  The maximum value of t shown in
#  the movie will be t_min + delta_t * frame_number
t_min = 0.   # You can make this negative to see what happens before t=0!
t_max = 2.*tau
delta_t = t_max / 100.
t_pts = np.arange(t_min, t_max + delta_t, delta_t)
```

We use the cell "magic" `%%capture` to keep the figure from being shown here.  If we didn't the animated version below would be blank.

```python
%%capture

fig_anim = plt.figure(figsize=(6,3), num='Triangular wave')
ax_anim = fig_anim.add_subplot(1,1,1)
ax_anim.set_xlim(x_min, x_max)
gap = 0.1
ax_anim.set_ylim(-1. - gap, 1. + gap)

# By assigning the first return from plot to line_anim, we can later change
#  the values in the line.
line_anim, = ax_anim.plot(x_pts,
                          u_triangular(x_pts, t_min, m_max, c_wave, L),
                          color='blue', lw=2)

fig_anim.tight_layout()
```

```python
def animate_wave(i):
    """This is the function called by FuncAnimation to create each frame,
        numbered by i.  So each i corresponds to a point in the t_pts
        array, with index i.
    """
    t = t_pts[i]
    y_pts = u_triangular(x_pts, t, m_max, c_wave, L)

    line_anim.set_data(x_pts, y_pts)  # overwrite line_anim with new points
    return (line_anim,)   # this is needed for blit=True to work
```

```python
frame_interval = 80.  # time between frames
frame_number = 101    # number of frames to include (index of t_pts)
anim = animation.FuncAnimation(fig_anim,
                               animate_wave,
                               init_func=None,
                               frames=frame_number,
                               interval=frame_interval,
                               blit=True,
                               repeat=False)

```

```python
HTML(anim.to_jshtml())  # animate using javascript

```

```python

```


# Introduction

The goal of this tutorial is to develop a memory based model of addition that predicts choices and reaction times. The model illustrates the use of the Lognormal Race model to describe memory retrieval in ACT-R and the use of marginalization to characterize the many-to-one mapping between retrieved chunks and responses.

# Siegler Addition Task

The Siegler addition task is based on a study of childrens' arithmatic ability conducted by Siegler (1984). In the task, children are asked to sum two verbally presented numbers (e.g. 2 + 2), and provide the solution verbally. If the participant does not know the solution, he or she responds "I don't know".  Each block consists of the following problems:

- 1 + 1
- 1 + 2
- 1 + 3
- 2 + 2
- 2 + 3
- 3 + 3

The block is repeated five times in the present simulation.

# Siegler Addition Model

## Overview

On each trial, the Siegler addition model proceeds through a deterministic chain of production rules that build the problem representation in the imaginal buffer, retrieves the answer, and responds. For each number, the model fires a production rule that listens to the number, retrieves the number from declarative memory, and adds the number to a problem representation chunk in the imaginal buffer. The resulting chunk is used as a retrieval request in the next production rule to obtain the answer from declarative memory. When the next production rule fires, it "harvests" the answer, which entails assigning the answer to a sum slot in the problem representation chunk. Finally, the model fires a production rule to vocalize the answer and merge the problem representation into declarative memory.

## Declarative memory

Declarative memory $M$ is populated with 35 chunks representing addition facts. As an example, consider the chunk $m$:

\begin{align}
\mathbf{c}_m = \{\rm (addend1,2),(addend2,2), (sum,4)\}
\end{align}

Each chunk contains the following three slots: $Q = \{\textrm{addend1},\textrm{addend2}, \textrm{sum}\}$. The full set of 35 addition facts is generated by permuting integers ranging from 0 to 5 for addend1 and addend2, such that the sum is less 10. Formally, this set is defined as:

\begin{align}
M = \{ \forall_{i,j} \rm \{(addend1, j), (addend2, i), (sum, i + j)\} : i,j \in \{0,1,\dots 5\}, i + j < 10  \}
\end{align}

## Retrieval Request

The details of the problem are encoded through the aural module where they are transferred and stored in a chunk $\mathbf{c}_{s,\textrm{imaginal}}$ located in the imaginal buffer. A retrieval request on trial $i$ is formed from the slots in $\mathbf{c}_{s,\textrm{imaginal}}$, which is defined as

\begin{align}
\mathbf{r}_i = \{(\rm addend1, c_{s,\textrm{imaginal}}(addend1), (\rm addend2, c_{s,\textrm{imaginal}}(addend2) \}
\end{align}

where $Q_r = \{\rm addend1, addend2\}$ is the set of slots for $\mathbf{r}_i$.

## Activation

Activation for chunk $\mathbf{c}_m$ is defined as:

\begin{align}
a_m = \textrm{blc}_m + \rho_m + \epsilon_m
\end{align}

where $\textrm{blc}_m$ represents degree of prior practice, $\rho_m$ is partial matching activation and $\epsilon \sim \rm Normal(0,\sigma)$. Note that $\sigma = \frac{s \pi}{\sqrt{3}}$ and $E[a_m] = \mu_m$. Unlike previous models, blc is indexed by $m$ to reflect the fact that its value depends on the chunk. The rationale for allowing blc to vary as function of $m$ is due to the fact that children have more practice with addition problems in which the sum is less than five, and thus, perform better with these problems. Based on this fact, we can define blc with the following piecewise equation:

\begin{align}\label{eq:penalty_activation_siegler}
\textrm{blc}_m  = \begin{cases}
    .65 & \text{if } c_m(\rm sum) < 5 \\
    0 & \text{otherwise} \\
  \end{cases}\
\end{align}

The model uses the following graded mismatch penalty function

\begin{align}
\rho_m  = \textrm{sim} \times \delta \sum_{q \in Q_r}| c_m(q) - r(q) |
\end{align}

where sim = .1 is a similarity scalar. The penalty function applies greater penalty when the slot values differ by a larger degree.

## Response Mapping

Due to the partial matching mechanism, a given response could be associated with one of several possible chunks. For example, the response $y = 4$ could have resulted from retrieving any chunk that represents the following addition facts:
- $2+2$
- $1+3$
- $3+1$
- $4+0$
- $0+4$

Let $\textrm{resp} \in \{0,1,..9,\textrm{uk}\}$ represent the response, with uk indicating a retrieval failure. A retrieval failure indicates that a participant does not know the answer (hence uk for unknown) and this response is mapped to a single key, such as spacebar. Due to the partial matching mechanism, a given response could be associated with one of several possible chunks. In order to account for this in the likelihood function, we define the set $R_{\rm resp}$, which consists of all possible chunks associated with resp. Formally, this is given by:
\begin{align*}
R_{\rm resp} &= \{\mathbf{c}_m \in M : c_m(\rm sum) = \textrm{resp} \} &\textrm{ for resp }\ne \textrm{uk}\\
R_{\rm resp} &= \{\mathbf{c}_{m^\prime}\}  &\textrm{ for resp }=\textrm{uk}.
\end{align*}

## Conflict resolution and perceptual-motor time

We will fix $t_{\rm er}$ to 2.05 based on the default ACT-R parameters.

## Likelihood Function

Let $\mathbf{Y} = \{\mathbf{y}_1, \mathbf{y}_2, \dots, \mathbf{y}_{N}\}$ be joint choice-response time pairs, such that $\mathbf{y}_i = \{\rm resp_i,rt_i\}$ contains the response and reaction time for trial $i$, respectively.

\begin{equation}\label{eq:genericlikelihood}
\mathcal{L}_{\rm LNR}(\Theta; \mathbf{c}_m, \textrm{rt})
=
g(\textrm{rt}-\textrm{t}_{\textrm{er}}|-\mu_m + \upsilon,\sigma)
\prod_{\{z|\mathbf{c}_z \in M \cup \mathbf{c}_{m^\prime} \} \setminus \mathbf{c}_m}\left[1-G(\textrm{rt}-\textrm{t}_{\textrm{er}}|-\mu_z + \upsilon,\sigma)\right],
\end{equation}
where $\mu_m$ is the mean activation of chunk $m$, $\sigma$ is the standard deviation in log-space, $\upsilon = \ln(\textrm{lf})$, $g$ is the Lognormal PDF, $G$ is the Lognormal CDF, $t_{\textrm{er}}$ is an offset that captures residual processing time for perceptual encoding, conflict resolution, and motor response, and $\Theta = \{\mu_1,\dots \mu_n, \sigma, \upsilon, t_{\textrm{er}}\}$ is the set of parameters for the LNR model. Note that the product set represents all chunks that failed to be retrieved---which is all chunks excluding chunk $m$, plus the retrieval failure option, denoted as $\mathbf{c}_{m^\prime}$. Note that $\upsilon$ drops out because lf $=1$ for this model.

The likelihood function for a single trial of the Siegler model is given as:

\begin{equation}\label{eq:LNR}
\mathcal{L}(\Theta; \textrm{resp},\textrm{rt})
= \sum_{\mathbf{c}_m \in R_{\textrm{resp}}} f_{\rm LNR}(\mathbf{c}_m,\textrm{rt};\Theta),
\end{equation}

Across all trials, the likelihood function is defined as:

\begin{align}
\mathcal{L}(\Theta; \mathbf{Y}) = \Pi_{i=1}^{n} \mathcal{L}(\Theta; \textrm{resp}_i,\textrm{rt}_i)
\end{align}

```julia
# set the working directory to the directory in which this file is contained
cd(@__DIR__)
# load the package manager
using Pkg
# activate the project environment
Pkg.activate("../../")
# load the required packages
using StatsPlots, Revise, ACTRModels, Distributions, Turing, DataFrames
# load all model functions
include("Siegler_Model.jl")
# seed random number generator
Random.seed!(82210)
```

      Activating environment at `~/.julia/dev/FundamentalToolsACTR/Project.toml`
    ┌ Info: Precompiling StatsPlots [f3b207a7-027a-5e70-b257-86293d7955fd]
    └ @ Base loading.jl:1342
    ┌ Info: Precompiling Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0]
    └ @ Base loading.jl:1342

    MersenneTwister(82210)

# Generate Data

In the code block below,  we will define a function to generate simulated data from the model. The `simulate` function accepts the following arguments:

- stimuli: an array of `NamedTuples` containing the stimuli
- parms: a `NamedTuple` of fixed parameters
- args...: a variable list of keyword arguments for parameters.

In the annotated `simulate` function below, the model objects are initialized and data are generated for each trial in the for loop. On each iteration of the for loop, a chunk is retrieved based on retrieval request `s`. If a retrieval failure occurs, resp = -100. Otherwise, the retrieved answer is added to the data. The output of `simulate` is a vector of `NamedTuples` in which each `NamedTuple` contains the first number of the problem, the second number of the problem, the response, and the reaction time.

```julia
function simulate(stimuli, parms; args...)
    # populate chunks
    chunks = populate_memory()
    # set the base level constant based on sum slot
    set_baselevels!(chunks)
    # create a declarative memory object
    memory = Declarative(;memory=chunks)
    # create an ACTR model object
    actr = ACTR(;declarative=memory, parms..., args...)
    shuffle!(stimuli)
    N = length(stimuli)
    data = Array{NamedTuple,1}(undef, N)
    # loop through trials
    for (i,s) in enumerate(stimuli)
        # retrieve a chunk based on retrieval request s
        chunk = retrieve(actr; s...)
        # compute reaction time
        rt = compute_RT(actr, chunk) + parms.ter
        if isempty(chunk)
            # retrieval failure
            data[i] = (s...,resp = -100,rt = rt)
        else
            # chunk retrieved.
            data[i] = (s...,resp = chunk[1].slots.sum,rt = rt)
        end
    end
    return data
end
```

    simulate (generic function with 1 method)

Now that our `simulate` function has been defined, we can now generate some data. The code block below generates 5 blocks  trials and stores the simulated data in an array where each `NamedTuple` contains stimulus and response information. `num1` corresponds to the first addend, `num2` corresponds to the second addend, `resp` corresponds to the response, and `rt` corresponds to the reaction time.

```julia
# mismatch penalty parameter
δ = 16.0
# retrieval threshold parameter
τ = -0.45
# logistic scalar for activation noise
s = 0.5
# fixed parameters
parms = (mmp = true,noise = true,mmp_fun = sim_fun,ter = 2.05)
# number of blocks
n_blocks = 5
# stimuli for simulation
stimuli = [(num1 = 1,num2 = 1), (num1 = 1,num2 = 2), (num1 = 1,num2 = 3), (num1 = 2,num2 = 2),
    (num1 = 2,num2 = 3), (num1 = 3,num2 = 3)]
# generate data for each block
temp = map(x -> simulate(stimuli, parms; δ, τ, s), 1:n_blocks)
# flatten data
data = vcat(vcat(temp...)...)
data[1]
```

    (num1 = 1, num2 = 3, resp = 4, rt = 2.8563793860458158)

## Define Likelihood Function

In the code below, we define a custom distribution object for the Siegler model with the following fields:

- $\delta$: mismatch penalty parameter
- $\tau$: retrieval threshold parameter
- s: activation noise parameter
- parms: a `NamedTuple` of fixed parameters

The function `computeLL` operates in a similar manner to the function `simulate`, which generates data from the model. The function `computeLL` begins by initializing the declarative memory and ACT-R objects. The for loop iterates through the data and breaks down the likelihood in two cases:

1. If a retrieval failure occurs, as indexed by -100, the log likelihood is computed using a simple Lognormal race model.
2. However, all other responses are a mixture of Lognormal races across the different chunks in $R_{\textrm{resp}}$. This requires computing the log likelihood of each mixture component and finding the marginal log likelihood with the function `logsumexp`, which provides a numerically stable way to sum logarithms.

```julia
struct Siegler{T1,T2,T3,T4} <: ContinuousUnivariateDistribution
    δ::T1
    τ::T2
    s::T3
    parms::T4
end

Siegler(;δ, τ, s, parms) = Siegler(δ, τ, s, parms)

loglikelihood(d::Siegler, data::Array{<:NamedTuple,1}) = logpdf(d, data)

function logpdf(d::Siegler, data::Array{<:NamedTuple,1})
    LL = computeLL(d.parms, data; δ=d.δ, τ=d.τ, s=d.s)
    return LL
end

function computeLL(parms, data; δ, τ, s)
    type = typeof(δ)
    # populate chunks
    chunks = populate_memory(zero(δ))
    # set base level constant based on sum slot
    set_baselevels!(chunks)
    # create declarative memory object
    memory = Declarative(;memory=chunks)
    # create ACTR model object
    actr = ACTR(;declarative=memory, parms..., δ, τ, s)
    # remove random values from activation
    actr.parms.noise = false
    N = length(chunks) + 1
    @unpack s,ter,τ = actr.parms
    LL = 0.0; idx = 0; μ = Array{type,1}(undef, N)
    σ = s * pi / sqrt(3)
    ϕ = ter
    for (num1,num2,resp,rt) in data
        # compute activation
        compute_activation!(actr; num1, num2)
        # extract mean activation values
        map!(x -> x.act, μ, chunks)
        # last mean activation is for retrieval failure
        μ[end] = τ
        # log normal race distribution
        dist = LNR(;μ=-μ, σ, ϕ)
        # no retrieval error
        if resp != -100
            # get all chunk indices such that sum = response
            indices = find_indices(actr; sum=resp)
            log_probs = zeros(type, length(indices))
            # loop over each component of mixture and compute log likelihood
            for (c,idx) in enumerate(indices)
                log_probs[c] = logpdf(dist, idx, rt)
            end
            # compute marginal likelihood
            LL += logsumexp(log_probs)
        else
            # retrieval failure
            LL += logpdf(dist, N, rt)
        end
    end
    return LL
end
```

    computeLL (generic function with 1 method)

## Define Model
The following summaries the prior distributions and the likelihood.

\begin{align}
\delta \sim \textrm{Normal}(16,8)
\end{align}

\begin{align}
\tau \sim \textrm{Normal}(-.45,1)
\end{align}

\begin{align}
 s \sim \textrm{TNormal}(.5,.5)_{0}^{\infty}
\end{align}

\begin{align}
\mathbf{y}_i \sim \textrm{LNR}(\Theta)
\end{align}

In the code block below, the model is specified for Turing.

```julia
@model model(data, parms) = begin
    δ ~ Normal(16, 8)
    τ ~ Normal(-0.45, 1)
    s ~ truncated(Normal(0.5, 0.5), 0.0, Inf)
    data ~ Siegler(δ, τ, s, parms)
end
```

    model (generic function with 1 method)

## Estimate Parameters

Now that the priors, likelihood and Turing model have been specified, we can now estimate the parameters. In the following code, we will run four MCMC chains with the NUTS sample for 2,000 iterations and omit the first 1,000 warmup samples.

```julia
# Settings of the NUTS sampler.
n_samples = 1000
delta = 0.85
n_adapt = 1000
n_chains = 4
specs = NUTS(n_adapt, delta)
# Start sampling.
chain = sample(model(data, parms), specs, MCMCThreads(), n_samples, n_chains, progress=true)
```

    ┌ Info: Found initial step size
    │   ϵ = 0.05
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/bzDgQ/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.05
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/bzDgQ/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.046875
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/bzDgQ/src/inference/hmc.jl:188
    ┌ Info: Found initial step size
    │   ϵ = 0.0125
    └ @ Turing.Inference /home/dfish/.julia/packages/Turing/bzDgQ/src/inference/hmc.jl:188

    Chains MCMC chain (1000×15×4 Array{Float64, 3}):

    Iterations        = 1001:1:2000
    Number of chains  = 4
    Samples per chain = 1000
    Wall duration     = 23.02 seconds
    Compute duration  = 89.92 seconds
    parameters        = s, τ, δ
    internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, max_hamiltonian_energy_error, tree_depth, numerical_error, step_size, nom_step_size

    Summary Statistics
      parameters      mean       std   naive_se      mcse         ess      rhat    ⋯
          Symbol   Float64   Float64    Float64   Float64     Float64   Float64    ⋯

               δ   17.5353    2.6226     0.0415    0.0640   1704.9773    1.0015    ⋯
               τ   -0.4918    0.2858     0.0045    0.0059   2318.0600    1.0002    ⋯
               s    0.5486    0.0710     0.0011    0.0016   1621.0790    1.0007    ⋯
                                                                    1 column omitted

    Quantiles
      parameters      2.5%     25.0%     50.0%     75.0%     97.5%
          Symbol   Float64   Float64   Float64   Float64   Float64

               δ   12.9376   15.6747   17.3363   19.1714   23.2999
               τ   -1.0917   -0.6733   -0.4777   -0.2853    0.0118
               s    0.4295    0.4978    0.5432    0.5912    0.7073

## Results

Overall, the quality of the MCMC chains is good. In the summary printout above, $\hat{r} \approx 1$ for each parameter, indicating good mixing. The first panel for each parameter does not show any sign of anomolous behavior, such as non-overlapping values or the same value across many iterations. The second panel for each parameter shows that autocorrelation is low, indicating efficient sampling. The density plots for each parameter show thatthe posterior distribution is centered near the data generating of $\delta = 16$, $\tau = -45$, and $s = .5$.

```julia
pyplot()
font_size = 12
ch = group(chain, :δ)
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcδ = plot(p1, p2, p3, layout=(3,1), size=(800,600))
```

```julia
ch = group(chain, :τ)
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcτ = plot(p1, p2, p3, layout=(3,1), size=(800,600))
```

```julia
ch = group(chain, :s)
p1 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:traceplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p2 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:autocorplot),
  grid=false, size=(250,100), titlefont=font(font_size))
p3 = plot(ch, xaxis=font(font_size), yaxis=font(font_size), seriestype=(:mixeddensity),
  grid=false, size=(250,100), titlefont=font(font_size))
pcs = plot(p1, p2, p3, layout=(3,1), size=(800,600))
```

## Correlation of posterior distribution

Inspecting correlations in the posterior distribution can be helpful for identifying tradeoffs between parameters or non-identifiability. Moderate correlations indicate tradeoffs between parameters whereas correlations close to 1 or -1 indicate problems with identifiability. The correlation matrix below indicates a moderate tradeoff between parameters $\delta$ and $s$.

```julia
cor(chain)
```

    Correlation
      parameters         δ         τ         s
          Symbol   Float64   Float64   Float64

               δ    1.0000   -0.2986    0.6899
               τ   -0.2986    1.0000   -0.4182
               s    0.6899   -0.4182    1.0000

### Posterior Predictive Distribution

#### Reaction Times

The code block below generates the posterior distribution of  mean reaction time for each of the six problems. One notable pattern is that the mean reaction time for the correct answer tends to be faster than the mean reaction time for incorrect answers.

```julia
preds = posterior_predictive(x -> simulate(stimuli, parms; x...), chain, 1000)
preds = vcat(vcat(preds...)...)
df = DataFrame(preds)
sort!(stimuli)
p4 = rt_histogram(df, stimuli, size=(800,400))
```

#### Responses
The plot below shows the response probabilities for each of the six addition problems. As expected, response probabilities are highest for correct answers, and decrease as a function of distance from the correct answer.

```julia
p5 = response_histogram(df, stimuli; size=(800,400))
```

```julia

```


# Decoupling!

```python
%matplotlib inline

from __future__ import division

import numpy as np
import matplotlib.pyplot as pl
import matplotlib as mpl

#----- MATPLOTLIB paramaters ---------
mpl.rcParams.update({'font.size': 18,'font.family':'serif'})

mpl.rcParams['xtick.major.size'] = 7
mpl.rcParams['xtick.major.width'] = 1
mpl.rcParams['xtick.minor.size'] = 3
mpl.rcParams['xtick.minor.width'] = 1
mpl.rcParams['ytick.major.size'] = 7
mpl.rcParams['ytick.major.width'] = 1
mpl.rcParams['ytick.minor.size'] = 3
mpl.rcParams['ytick.minor.width'] = 1
#--------------------------------------

from scipy.interpolate import interp1d
from scipy.integrate import quad, odeint, solve_ivp, ode
import sympy as sp

x1 = sp.symbols('x1')
```

**Set up the equations of motion**

```python
def h_fun(s):
    return sp.sqrt(s**-3 + s**-4)

deriv = sp.diff(h_fun(x1), x1)

def hprime_fun(s):
    return float(deriv.subs(x1, s))

def jacob(y,s):
    chi = y[0]
    chiprime = y[1]
    h = 1.0*h_fun(s)
    hp = 1.0*hprime_fun(s)
    return [[0,1],[(s*hp + h)/(s**2*h) - 2*np.sign(chi)*chi**-3*(L*(s*h)**2)**-1, -s*(s*hp + h)/(s**2*h) ]]

def dyds(y,s):
    chi = y[0]
    chiprime = y[1]

    h = 1.0*h_fun(s)
    hp = 1.0*hprime_fun(s)

    #term2 = 0.0
    #if (chi/L > 1e-5):
    #    term2 = (L*(s*h)**2*(chi**2))**-1

    eqn = -((s*chiprime - chi)*(s*hp + h)/(s**2*h) + np.sign(chi)*(L*(s*h*chi)**2)**-1)#
    return [chiprime, eqn]
```

```python

```

**Solving the equations of motions**

```python
L = 0.001
s0 = 1e-4*L

#Initial conditions
chi_init = [s0, 1.0]

s_list = np.logspace(np.log10(s0), np.log10(0.75*L), 100)
ys,output = odeint(dyds, chi_init, s_list, Dfun=jacob, full_output=True)

#print ys
```

```python

```

```python
pl.figure()

pl.plot(s_list/L, ys[:,0]/L)
pl.xlabel(r"$s/\lambda$")
pl.ylabel(r"$\chi/\lambda$")
pl.axvline(1.0/3.0, linestyle='--', color='k')
pl.title(r"$\lambda = " + str(L)+"$")
pl.show()

pl.figure()

pl.plot(s_list/L, ys[:,1]/L)
pl.xlabel(r"$s/\lambda$")
pl.ylabel(r"$\chi'/\lambda$")
pl.title(r"$\lambda = " + str(L)+"$")
pl.show()

```

```python
chi_interp = interp1d(s_list, ys[:,0])

y_list = np.linspace(1e-3*L, 0.5*L)

integ1 = lambda y: y**-4*h_fun(y)**-1*(chi_interp(y))**2
integ2 = lambda y: (1+y**(3.0/2.0))**2*y**-4*h_fun(y)**-1*(chi_interp(y))**2

pl.figure()
pl.plot(y_list, np.vectorize(integ1)(y_list), label='Without halo')
pl.plot(y_list, np.vectorize(integ2)(y_list), label='With halo')
pl.legend()
pl.show()

print(quad(integ1, s0, 0.5*L)[0]/L)
print(quad(integ2, s0, 0.5*L)[0]/L)
```

**Solving it a different way...**

```python
r = ode(dyds, None).set_integrator('dopri5', safety=0.5, beta=0.1)
r.set_initial_value(chi_init, s0)
dt = 1e-2*L
t1 = 2.0*L

r_list = [s0,]
t_list = [s0,]
while r.successful() and r.t < t1:
    res = r.integrate(r.t+dt)
    t_list = np.append(t_list, r.t+dt)
    print(r.y)
    r_list = np.append(r_list,r.y[0])
    #print(r.t+dt, r.integrate(r.t+dt))

```

```python
pl.figure()
pl.plot(t_list,r_list)
pl.ylim(-1, 1)
pl.show()
```

```python

```