# Information-theoretic analysis of EEG microstates in Python

Our software is provided as a single Python source code file (eeg_microstates.py) that contains all functions necessary to:
* import EEG data in the '.edf' file format
* pre-process (band-pass filter) the EEG data
* compute microstate maps by the modified K-means algorithm [1]
* compute microstate time series by competitively back-fitting the microstate maps into the EEG data
* perform information-theoretical calculations we recently introduced [2]

[1] Pascual-Marqui RD, Michel CM, Lehmann D. Segmentation of brain electrical activity into microstates: model estimation and validation. IEEE Trans Biomed Eng 42 (1995) 658–665.

[2] von Wegner F, Tagliazucchi E, Laufs H. Information-theoretical analysis of resting state EEG microstate sequences - non-Markovianity, non-stationarity and periodicities. NeuroImage 158 (2017) 99–111.

[3] von Wegner F, Laufs H. Information-theoretical analysis of EEG microstate sequences in Python. Front Neuroinformatics (2018) doi: 10.3389/fninf.2018.00030.

[4] von Wegner F, Knaut P, Laufs H. EEG microstate sequences from different clustering algorithms are information-theoretically invariant. Front Comp Neurosci (2018) doi: 10.3389/fncom.2018.00070.

### Github link of this project

The current version of the software can be found on the [GitHub project page](https://github.com/Frederic-vW/eeg_microstates/).

### Current literature about EEG microstates and information theory

To find out about current publications and directions in microstate research, in particular related to information-theoretical approaches, we provide an active link to the PubMed page:
https://www.ncbi.nlm.nih.gov/pubmed/?term=eeg+microstates+information+theory

## Background

Electroencephalography (EEG) records the brain's electrical potential at discrete spatial sampling sites, defined by the locations of the electrodes on the skull. We also call the potential distribution at a given time a topography, or a map. The idea behind the microstate algorithm is to represent the EEG time series by a small number of maps that yield an optimum representation of the full data set. As in other areas of data compression, optimality can be defined in different ways. For EEG microstates, the criterion of choice is that of maximum explained variance. To this end, a (large) set of EEG samples is fed into a clustering algorithm whose output is a reduced set of representative EEG topographies, also called termed microstates, or microstate maps.
The EEG time series can then be represented by a sequence of microstate labels, where each time point of the original EEG data is mapped to the microstate with the maximum spatial correlation to the actual EEG topography. Polarity is usually ignored, so the squared correlation coefficient between each candidate microstate map and the EEG topography is used as a similarity measure.
With a set of representative microstates at hand, the multi-channel EEG data set can be compressed into a time series of symbols, i.e. microstate labels. Most frequently, four microstates are used to compress the EEG data set, and the microstates are labelled A, B, C, D. The EEG data set is therefore transformed into a microstate sequence that could read like ...AAABBDCDABBDAB...
Due to the special, non-metric nature of the new state space {A, B, C, D}, many common time-series methods such as the autocorrelation function cannot be applied to the microstate time series. We therefore test several entropy-based measures based on Shannon's information theory to quantify these time series.

The details of the procedure are given in the following sections and the whole procedure is illustrated in the figure below.
Step 1: The EEG data set is submitted to the modified K-means algorithm to obtain the microstate maps A-D. The figure below shows a section of resting state EEG for a selection of the 30 channels recorded. At each time point, the spatial standard deviation of the EEG topography is also called global field power (GFP, blue). Topographies at local maxima of the GFP time series (MAX, red dots) define the input to the modified K-means clustering algorithm.
Step 2: The microstate time series is computed by competitive back-fitting of the microstate maps into the EEG time series.
Step 3: Several information-theoretical tests are run on the microstate sequence and finally, the time-lagged mutual information function and a first-order Markov confidence interval are computed and displayed.

## Import package

In order to work with the package, the following import statement can be used in Python standalone scripts, e.g. for batch processing, or in any Python-based EEG software to add extra functionality:

```python
from eeg_microstates3 import *
```

## (1) EEG data input

The package contains a basic [EDF](http://www.edfplus.info/specs/edf.html) file format reader. EEG data is returned as a NumPy array with time running along the rows (first index) and channels across the columns (2nd index), i.e. the array shape is (n_samples, n_channels). If our code is interfaced with other Python programs, EEG data must be provided in that shape in order to be processed correctly by our package functions.
The return variables of the function read_edf are (1) a list of channel names, (2) the sampling rate in Hz, and (3) the EEG data in the mentioned format.
To run this tutorial, you must have the test.edf EEG file in your current folder. The file is part of the GitHub repository. The record contains 192 sec. from an eyes-closed resting state experiment of a healthy male subject recorded with a 30 channel EEG cap using a standard 10-10 electrode configuration. The sampling rate is 250 Hz and the data is band-pass filtered to the 1-40 Hz range.

```python
filename = "test.edf"
chs, fs, data_raw = read_edf(filename)
print("File: {}".format(filename))
print("Sampling rate: {:.2f} Hz".format(fs))
print("Data shape: {:d} samples x {:d} channels".format(data_raw.shape[0], data_raw.shape[1]))
print("Channels:")
for ch in chs:
    print(f"{ch.decode():s}", end=', ')
print("")
```

    File: test.edf
    Sampling rate: 250.00 Hz
    Data shape: 48000 samples x 30 channels
    Channels:
    Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T7, T8, P7, P8, Fz, Cz, Pz, AFz, AF3, AF4, FC3, FC4, FT9, FT10, TP9, TP10, CP5, CP6,

## (2) Pre-processing

EEG data is usually band-pass filtered before microstate clustering. The package contains a zero-phase 6-th order Butterworth band-pass filter as provided by SciPy. The input is (1) the data to be filtered as a NumPy array of shape (n_samples, n_channels), (2) a tuple defining the desired frequency band in Hz, and (3) the EEG sampling rate in Hz.

```python
data = bp_filter(data_raw, f_lo=1, f_hi=35, fs=fs)
print(data.shape)
```

    (48000, 30)

## (3) Visualize data (PCA-1)

A basic data visualization tool is provided to get an idea of the temporal structure of the EEG data set. We use principle component analysis as provided in the scikit-learn package and plot the time course of the first principal component to represent the dominant component of the data set. Using the test.edf file, we find amplitude modulated oscillations, mainly in the alpha frequency band (8-12), as expected in a wakeful resting condition.

```python
pca = PCA(copy=True, n_components=1, whiten=False)
pca1 = pca.fit_transform(data)[:,0]
plot_data(pca1, fs)
```

Alternatively, only a segment of data can be visualized to better recognize alpha oscillations.

```python
t0, t1 = 1000, 3000
plot_data(pca1[t0:t1], fs)
```

## (4) Clustering of the microstate maps

We are now ready to apply the microstate clustering algorithm to the filtered data. We implemented five different clustering methods (AAHC, modified k-means, k-medoids, PCA, Fast-ICA), as detailed in [4].
Here, we will run the modified K-means algorithm as given by Pascual-Marqui [1], implemented as the function 'kmeans' in the eeg_microstates.py file [3]. The function variables should be self-explaining and are further documented in the source file following the PEP 8 styleguide.
As the initial microstate ordering returned by the stochastic K-means algorithm is random, we provide a tool to re-label the microstates interactively. The standard ordering follows the geometry of the maps. Map A has a diagonal direction connecting the upper (frontal) left and lower (occipital) right corners, map B is diagonal in the opposite direction, map C has a horizontal orientation and map D is often circular. Sometimes, slightly different maps are generated. We mostly observed the occurrence of a map D with a vertical axis instead of a circular pattern. In this case, you can either accept the results and proceed, or re-cluster the data set as the results of the K-means algorithm differ between runs.
Modified K-means clustering and all information-theoretical computations can be run for any number of microstates. The desired number of microstates is given by the user as the argument 'n_maps' of the 'kmeans' function. If n_maps is not provided, the default value n_maps=4 is used.

```python
mode = ["aahc", "kmeans", "kmedoids", "pca", "ica"][1]
print(f"Clustering algorithm: {mode:s}")
```

    Clustering algorithm: kmeans

```python
n_maps = 4
locs = []
maps, x, gfp_peaks, gev = clustering(data, fs, chs, locs, mode, n_maps, interpol=False, doplot=True)
```

You can also inspect the source code of the function:

```python
import inspect
#c=inspect.getsourcelines(kmeans) # activate by uncomment
#for l in c[0]: print(l)
```

When the algorithm terminates, the global explained variance (GEV) of the computed microstate time series is shown. For the test file, the GEV should give approximately 0.73. In other words, the microstate sequence explains ca. 73% of the electrical potential's spatial variance over time.

The first return variable of the kmeans function is an array containing the microstate maps row-wise. The second return variable is the microstate sequence corresponding to the EEG data that was used as input. The sequence was generated as part of the kmeans optimizations by back-fitting into the EEG time series, and the sequence corresponding to the best K-means run is returned.
On this sequence all subsequent tests will be run. The sequence contains the microstate labels as numbers, $\{A, B, C, D\} \rightarrow \{0, 1, 2, 3\}$, in order to accelerate information-theoretical computations. Using numbers, the microstate label can directly be used as an index in arrays and matrices. If you want to test microstate sequences generated by other another software, the sequence must be converted to numerical values $0, ..., n_{maps}-1$ before using the functions of this package.

## (5) Basic microstate statistics

We will now look at some basic statistical properties of the microstate sequence returned by the kmeans function. The empirical symbol distribution $\hat{p}$ quantifies the relative frequency of each microstate map. It is equivalent to the so-called 'ratio of time covered' (RTT) used in some EEG microstate publications. The transition matrix $\hat{T}$ contains the transition probabilities between all pairs of microstate labels. To get a feeling of the temporal dynamics of the EEG's spatial variance, we also compute the number of GFP peaks per second. The value is roughly 2 x the dominant EEG frequency band. In our case, the dominant frequency is a 10 Hz alpha rhythm and we observe approx. 21.5 GFP peaks per second.

```python
p_hat = p_empirical(x, n_maps)
T_hat = T_empirical(x, n_maps)
print("\nEmpirical symbol distribution (RTT):\n")
for i in range(n_maps):
    print(f"p_{i:d} = {p_hat[i]:.3f}")
print("\nEmpirical transition matrix:\n")
print_matrix(T_hat)
pps = len(gfp_peaks) / (len(x)/fs)  # peaks per second
print(f"\nGFP peaks per sec.: {pps:.2f}")
print("\nGlobal explained variance (GEV) per map:")
for i, g in enumerate(gev): print(f"GEV(ms-{i:d}) = {gev[i]:.2f}")
print(f"\ntotal GEV: {gev.sum():.3f}")
```

    Empirical symbol distribution (RTT):

    p_0 = 0.237
    p_1 = 0.282
    p_2 = 0.219
    p_3 = 0.261

    Empirical transition matrix:

    |0.7770.0800.0880.055|
    |0.0630.8030.0580.076|
    |0.0600.0810.7610.098|
    |0.0850.0720.0570.786|

    GFP peaks per sec.: 21.51

    Global explained variance (GEV) per map:
    GEV(ms-0) = 0.25
    GEV(ms-1) = 0.17
    GEV(ms-2) = 0.10
    GEV(ms-3) = 0.16

    total GEV: 0.686

## (6) Shannon entropy and entropy rate of the microstate sequence

The information-theoretical characterization of the microstate time series starts with the calculation of the empirical Shannon entropy $\hat{H}$ of the sequence:

$$
\begin{align}
\hat{H} & = -\sum_i \hat{p}(i) \log \hat{p}(i)
\end{align}
$$

where the index $i$ runs over the microstate labels. The base of the logarithm is not important. We use the natural logarithm to the base $e$ (Euler's number) and therefore measure information in 'nats', in analogy to bits when using $\log_2$.

```python
h_hat = H_1(x, n_maps)
h_max = max_entropy(n_maps)
print(f"\nEmpirical entropy H = {h_hat:.2f} (max. entropy: {h_max:.2f})")
```

    Empirical entropy H = 1.38 (max. entropy: 1.39)

The maximum entropy possible for any sequence composed of four symbols is $\log(4) \approx 1.39$, and the minimum entropy is zero. We see that the test sequence almost achieves the maximum entropy. As entropy can be interpreted as the amount of surprise or randomness, this result suggests a lack of predictable temporal features in the microstate sequence. However, we will show that the sequence does contain temporal structure, mainly in the form of periodicities in the range of the main EEG frequency bands.

Next, the entropy rate of the sequence is estimated from a set of joint Shannon entropies $H_k=H(\mathbf{X}_{n}^{(k)})$ for different history lengths $k$. The estimated entropy rate is the slope of the linear fit $k$ vs. $H_k$. The intercept is the so-called excess entropy, which is not further investigated here.

```python
h_rate, _ = excess_entropy_rate(x, n_maps, kmax=8, doplot=True)
h_mc = mc_entropy_rate(p_hat, T_hat)
print(f"\nEmpirical entropy rate h = {h_rate:.2f}")
print(f"Theoretical MC entropy rate h = {h_mc:.2f}")
```

## (7) Markov tests

We will first test the Markov properties of order 0, 1 and 2 for the EEG test file. For the details of the testing procedure, see the attached manuscript and the code documentation in the source file.

```python
alpha = 0.01
p_M0 = testMarkov0(x, n_maps, alpha)
p_M1 = testMarkov1(x, n_maps, alpha)
p_M2 = testMarkov2(x, n_maps, alpha)
p_vals_geo = geoTest(x, n_maps, 1000./fs, alpha)
```

    \ZERO-ORDER MARKOVIANITY:
    p: 0.00e+00 | t: 59954.303 | df: 9.0

    FIRST-ORDER MARKOVIANITY:
    p: 3.83e-123 | t: 695.807 | df: 36.0

    SECOND-ORDER MARKOVIANITY:
    p: 2.38e-81 | t: 742.497 | df: 144.0

    GEOMETRIC DISTRIBUTION of lifetimes:

    Testing the distribution of symbol # 0
    p: 1.21e-16 | t: 136.814 | df: 27.0
    G-test (log-likelihood) p: 2.31e-06, g: 74.737, df: 27.0
    G-test (Pearson Chi2) p: 2.38e-05, g: 67.687, df: 27.0

    Testing the distribution of symbol # 1
    p: 1.39e-65 | t: 390.859 | df: 28.0
    G-test (log-likelihood) p: 8.90e-31, g: 215.175, df: 28.0
    G-test (Pearson Chi2) p: 8.43e-27, g: 194.229, df: 28.0

    Testing the distribution of symbol # 2
    p: 2.91e-13 | t: 123.242 | df: 30.0
    G-test (log-likelihood) p: 5.48e-05, g: 69.609, df: 30.0
    G-test (Pearson Chi2) p: 5.12e-04, g: 62.079, df: 30.0

    Testing the distribution of symbol # 3
    p: 2.39e-23 | t: 180.145 | df: 30.0
    G-test (log-likelihood) p: 5.98e-09, g: 96.762, df: 30.0
    G-test (Pearson Chi2) p: 6.35e-08, g: 90.097, df: 30.0

The statistical Markovianity test show that our test sequence does not have the Markov property of the tested orders. This means that there are memory effects in the series which we will further explore.

## (8) Stationarity test

Next, we will test if the transition matrix $\hat{T}$ is time-stationary or if the matrix elements change significantly over time. You can provide a specific window length L for testing or use the default value of 5000 samples.

```python
try:
    L = int(input("Enter block size: "))
except:
    L = 5000
p3 = conditionalHomogeneityTest(x, n_maps, L, alpha)
```

    Enter block size: 2500

    CONDITIONAL HOMOGENEITY (three-way table):
    Split data in r = 19 blocks of length 2500.
    p: 2.10e-12 | t: 392.765 | df: 216.0

We find that the given transition matrix is not time-stationary.

## (9) Symmetry test

We will now test if the transition matrix is symmetric. In other words, we test if the transitions $A \rightarrow B$ and $B \rightarrow A$ occur with the same probability or not.

```python
p4 = symmetryTest(x, n_maps, alpha)
```

    SYMMETRY:
    p: 2.71e-55 | t: 269.547 | df: 6.0

The transition matrix of the test microstate sequence is not symmetric.

## (10) Markov surrogate example

To test our statistical procedures and to construct surrogate data sets from which confidence intervals for further information-theoretical quantities can be computed, we will synthesize first-order Markov chains with symbol distribution and transition matrix identical to the experimental EEG data set. This is possible because a first-order Markov chain is completely defined by an initial symbol distribution and a transition matrix for first-order (single time step) transitions.

```python
x_mc = surrogate_mc(p_hat, T_hat, n_maps, len(x))
p_surr = p_empirical(x_mc, n_maps)
T_surr = T_empirical(x_mc, n_maps)
print("\nSurrogate symbol distribution:\n")
for i in range(n_maps):
    print(f"p_{i:d} = {p_surr[i]:.3f}")
print( "\nSurrogate transition matrix:\n" )
print_matrix(T_surr)
```

    Surrogate symbol distribution:

    p_0 = 0.228
    p_1 = 0.283
    p_2 = 0.220
    p_3 = 0.268

    Surrogate transition matrix:

    |0.7690.0830.0920.057|
    |0.0640.8030.0590.075|
    |0.0570.0790.7650.100|
    |0.0830.0730.0530.791|

We observe that the symbol distribution and the transition matrix are equivalent, within some stochastic margin, to the experimental data. A quantitative analysis of Markov surrogate errors is contained in [2].

## (11) Markov tests for surrogate Markov data

It will now be shown that the Markov surrogates fulfill the expected properties, or vice versa, that our Markovianity and stationarity tests work well with our surrogate data. This means that we can analyze experimental data series to find memory effects beyond a first-order Markov model.

```python
p_M0_mc = testMarkov0(x_mc, n_maps, alpha)
p_M1_mc = testMarkov1(x_mc, n_maps, alpha)
p_M2_mc = testMarkov2(x_mc, n_maps, alpha)
p_vals_geo_mc = geoTest(x_mc, n_maps, 1000./fs, alpha)
```

    \ZERO-ORDER MARKOVIANITY:
    p: 0.00e+00 | t: 59982.540 | df: 9.0

    FIRST-ORDER MARKOVIANITY:
    p: 8.83e-01 | t: 26.268 | df: 36.0

    SECOND-ORDER MARKOVIANITY:
    p: 3.53e-01 | t: 149.818 | df: 144.0

    GEOMETRIC DISTRIBUTION of lifetimes:

    Testing the distribution of symbol # 0
    p: 4.78e-01 | t: 28.756 | df: 29.0
    G-test (log-likelihood) p: 9.91e-01, g: 14.005, df: 29.0
    G-test (Pearson Chi2) p: 9.95e-01, g: 13.251, df: 29.0

    Testing the distribution of symbol # 1
    p: 1.61e-02 | t: 50.167 | df: 31.0
    G-test (log-likelihood) p: 8.49e-01, g: 23.011, df: 31.0
    G-test (Pearson Chi2) p: 8.78e-01, g: 22.153, df: 31.0

    Testing the distribution of symbol # 2
    p: 7.03e-02 | t: 48.015 | df: 35.0
    G-test (log-likelihood) p: 9.80e-01, g: 20.073, df: 35.0
    G-test (Pearson Chi2) p: 9.94e-01, g: 17.497, df: 35.0

    Testing the distribution of symbol # 3
    p: 8.09e-01 | t: 26.701 | df: 34.0
    G-test (log-likelihood) p: 1.00e+00, g: 13.038, df: 34.0
    G-test (Pearson Chi2) p: 1.00e+00, g: 12.138, df: 34.0

The Markov surrogate clearly fulfills the expected theoretical properties. The zero-order Markov hypothesis is rejected while the first-order and second-order properties are accepted. The theoretically equivalent, but numerically different test for geometric lifetime distributions asserts the geometric lifetime distribution of all microstate maps.

## (12) Stationarity test for surrogate Markov data

We expect the Markov surrogate to be time-stationary as the same transition matrix has been used to generate the entire sequence. The test results show that indeed, the Markov surrogate is correctly classified as time-stationary. This means that we can identify non-stationarity in experimental data sets statistically.

```python
try:
    L = int(input("Enter block size: "))
except:
    L = 5000
p3_mc = conditionalHomogeneityTest(x_mc, n_maps, L, alpha)
```

    Enter block size: 3000

    CONDITIONAL HOMOGENEITY (three-way table):
    Split data in r = 16 blocks of length 3000.
    p: 8.58e-01 | t: 159.814 | df: 180.0

## (13) Symmetry test for surrogate Markov data

As the experimental EEG transtion matrix is used to generate the Markov surrogate, the surrogate's transition matrix is also asymmetric.

```python
p4_mc = symmetryTest(x_mc, n_maps, alpha)
```

    SYMMETRY:
    p: 1.03e-63 | t: 308.866 | df: 6.0

## (14) Time-lagged mutual information: empirical vs. Markov

Mutual information for time lag k, $I(k)$, is defined as the difference between the entropy (uncertainty) of the microstate sequence at time point $t+k$, and the conditional entropy at time point $t+k$ given the state at time $t$. Mutual information thus quantifies the uncertainty about the state at $t+k$ that is not represented by the previous time point $t$.

$$
\begin{align}
I(k) & = H(X_{t+k}) - H(X_{t+k} \mid X_k)
\end{align}
$$

As an analogy to the autocorrelation function (ACF) for metric time series, we will call this information-theoretic quantity the autoinformation function (AIF). Whilte the ACF uses linear correlation as a dependency measure for metric time series, the AIF uses mutual information to quantify dependencies in symbolic sequences.
We can now compute the time-lagged mutual information function of the microstate sequence and compare it to a set of Markov surrogates, from which a confidence interval under the first-order Markovian null hypothesis is constructed. We here use 10 Markov surrogates and compute the two-sided 99% percentile.

```python
l_max = 100
n_mc = 10
aif = mutinf(x, n_maps, l_max)
print(f"\nComputing n = {n_mc:d} Markov surrogates...\n")
aif_array = mutinf_CI(p_hat, T_hat, len(x), alpha, n_mc, l_max)
pct = np.percentile(aif_array,[100.*alpha/2.,100.*(1-alpha/2.)],axis=0)
print("\nMarkov surrogates and AIF confidence interval computed")
```

    mutual information lag: 100

    Computing n = {n_mc:d} Markov surrogates...

    surrogate MC # 1/10
    mutual information lag: 100

    surrogate MC # 2/10
    mutual information lag: 100

    surrogate MC # 3/10
    mutual information lag: 100

    surrogate MC # 4/10
    mutual information lag: 100

    surrogate MC # 5/10
    mutual information lag: 100

    surrogate MC # 6/10
    mutual information lag: 100

    surrogate MC # 7/10
    mutual information lag: 100

    surrogate MC # 8/10
    mutual information lag: 100

    surrogate MC # 9/10
    mutual information lag: 100

    surrogate MC # 10/10
    mutual information lag: 100

```python
plt.ioff()
fig = plt.figure(1, figsize=(20,5))
fig.patch.set_facecolor('white')
t = np.arange(l_max)*1000./fs
plt.semilogy(t, aif, '-sk', linewidth=3, label='AIF (EEG)')
plt.semilogy(t, pct[0,:], '-k', linewidth=1)
plt.semilogy(t, pct[1,:], '-k', linewidth=1)
plt.fill_between(t, pct[0,:], pct[1,:], facecolor='gray', alpha=0.5, label='AIF (Markov)')
plt.xlabel("time lag [ms]", fontsize=20, fontweight="bold")
plt.ylabel("AIF [nats]", fontsize=20, fontweight="bold")
plt.legend(fontsize=22)
ax = plt.gca()
ax.tick_params(axis='both', labelsize=18)
plt.tight_layout()
plt.show()
```

The figure generated by the code above shows the autoinformation function (AIF) along with Markovian confidence intervals. The AIF summarizes memory effects in the test microstate sequence as identified by mutual information. It is observed that the AIF of experimental data (black line and squares) shows clear periodicities and comparison with the Markovian confidence interval (gray-shaded area) shows that these peaks are statistically significant.
We can thus show that EEG microstates recur in a regular way. In other words, the microstate sequences passes through the graph of state transitions in a circular way. The transitions do not do so in a completely deterministic way, as any state transition is possible at any time, but we show that there are preferred closed paths along the state transition graph.

## (15) Summary and reporting results

The results can be summarized for scientific reporting as follows. Please note that *fixed numerical values are shown here*, i.e. the results of a specifc run of the algorithm. As clustering results vary slightly between runs, the exact numerical values may differ from the results obtained above.

1. We analyzed 192 seconds of resting state EEG sampled at 250 Hz. The 1-35 Hz band-pass filtered data set showed  21.51 GFP peaks per second and was clustered into 4 microstates, labelled A-D.

2. Static properties:
    * The microstate distribution was $p(A)=0.222$, $p(B)=0.240$, $p(C)=0.264$, and $p(D)=0.275$.
    * The four microstates explained 69% of the total data variance (GEV).
    * In information-theoretical terms, the four microstates produced a Shannon entropy of $H=1.38$ nats, where the upper bound for a 4-state process is $H_{max}=1.39$ nats. The empirical entropy rate was $h=0.73$ nats, and the expected value for the analogous first-order Markov process is $h=0.76$ nats.

3. Dynamic properties:
    * The transition probabilities between all pairs of microstates are summarized in the transition matrix T:
    \begin{equation*}
    T =  \begin{vmatrix}
    0.771 & 0.094 & 0.058 & 0.076 \\
	0.063 & 0.769 & 0.112 & 0.056 \\
	0.063 & 0.055 & 0.786 & 0.096 \\
	0.070 & 0.072 & 0.061 & 0.797 \\
    \end{vmatrix}
    \end{equation*}
    * The transition matrix T proved to be asymmetric ($p \lt 10^{-80}$).

4. Information-theoretical properties. Using a significance level of $\alpha = 0.01$, we found that:
    * The microstate sequence does not fulfill the Markov property of order 0, 1, or 2. We conclude that the microstate sequence has extended memory effects.
    * During the EEG recording, the transition matrix of the microstate sequence is not constant (it is non-stationary), when tested for a partition into 9 blocks and 5000 samples per block ($p=3.37 \times 10^{-6}$). This result should be taken as a precaution when using methods that require time stationarity (e.g. long-range memory interpretations of Hurst exponents).
    * Non-Markovianity and non-stationarity are at least partially explained by a periodic recurrence of the microstates, as shown by the time-lagged mutual information function (or autoinformation function, AIF). For time lags up to 400 msec., the EEG's AIF lies above the (first-order) Markov confidence interval and shows distinct peaks at multiples of approx. 50 msec (half the period of the underlying alpha rhythm).

### Code acceleration

If you ran the code above on your computer, you will have noticed that the AIF calculation determines the computation time of the procedure. The code can be accelerated using Cython, Numba or an external Julia function, for instance. To maximize portability and minimize the number of operating system dependencies, we did not include these accelerations in the code. However, the mentioned methods are implemented with little effort. Accelerated code can also be obtained from the author.
The eeg_microstates.py script can be run from the console using the -mc parameter that defines the number of Markov surrogates. The number of surrogates will define the width and smoothness of the confidence interval. Empirically, we suggest a minimum number of 50-100 surrogates.

## Conclusions

If you have arrived here, you should have completed the tutorial successfully. You should be familiar with the functionality provided by this software and have a good impression how to process larger data sets to obtain group statistics.


Given a string s, find the longest palindromic substring in s. You may assume that the maximum length of s is 1000.

Example 1:
```
Input: "babad"
Output: "bab"
```
Note: "aba" is also a valid answer.
Example 2:
```
Input: "cbbd"
Output: "bb"
```

### Solution 1: Standard method, expanding from the center

```python
# 回纹序列
# 解法一：比较好的办法：从中间向两边扩展，找到并返回最长的字符串

class Solution(object):
    def longestPalindrome(self, s):
        """
        :type s: str
        :rtype: str
        """
        ret = p_str = ""
        for i in range(len(s)):
            p_str = self.getPalindrome(s, i, i)
            if len(p_str) >= len(ret):
                ret = p_str
            p_str = self.getPalindrome(s, i, i+1)
            if len(p_str) >= len(ret):
                ret = p_str
        return ret

    def getPalindrome(self, s, l, r):
        """
        :type s: str
        :type l: int
        :type r: int
        :rtype: str
        """
        while(l >= 0 and r < len(s) and s[l] == s[r]):
            l = l - 1;
            r = r + 1;
        return s[l+1:r] # 这里的+1 非常关键，由于在while循环中已经减1，此处需要加1
```

```python
if __name__ =="__main__":
    t_str = "babad"
    print(Solution().longestPalindrome(t_str))
```

    aba

### Approach 2: Dynamic Programming
To improve over the brute force solution, we first observe how we can avoid unnecessary re-computation while validating palindromes. Consider the case "ababa". If we already knew that "bab" is a palindrome, it is obvious that "ababa" must be a palindrome since the two left and right end letters are the same.

We define P(i,j)P(i,j) as following:
\begin{equation}
P(𝑖,𝑗) =
    \begin{cases}
      true & \text{if the substring 𝑆𝑖…𝑆𝑗 is a palindrome}\\
      false & \text{otherwise}
    \end{cases}
\end{equation}

Therefore,
$P(i, j) = P(i+1, j-1) \text{ and } S_i == S_j$

The base cases are:
$P(i, i) = true$,
$P(i, i+1) = ( S_i == S_{i+1} )$

This yields a straight forward DP solution, which we first initialize the one and two letters palindromes, and work our way up finding all three letters palindromes, and so on...

Complexity Analysis:
- Time complexity : $O(n^2)$. This gives us a runtime complexity of $O(n^2)$.
- Space complexity : $O(n^2)$. It uses $O(n^2)$ space to store the table.

```python
# 回纹序列
# 解法二：动态规划方法 Dynamic Programming Method
import numpy as np

class Solution(object):
    def longestPalindrome(self, s):
        n = len(s)
        P = np.zeros((n, n), dtype=bool)
        max_len = 1
        start = 0

        for i in range(n):
            P[i,i] = True
            for j in range(i):
                if s[j] == s[i] and j == i-1: # babaad, i=5, j=4
                    P[j,i] = True
                if s[j] == s[i] and P[j+1,i-1]: # babaad, i=4,j=2
                    P[j,i] = True

                if P[j, i] and (i-j+1) >= max_len:
                    # if currently the longest palindrome, store the length and start
                    max_len = i-j+1
                    start = j

        return s[start:start + max_len]
```

```python
if __name__ =="__main__":
    t_str = "babad"
    print(Solution().longestPalindrome(t_str))
```

    aba

```python

```


```python
from scipy.special import psi, polygamma, gammaln
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
```

3-1. Latent Dirichlet Allocation

1  Data: Reuters News Titles

2  Model: Basic LDA

2.1  Variational EM

2.1.1  E-step

2.1.2  M-step

2.1.3  Variational lower bound

2.1.4  Training

3  Model: Smoothed LDA

3.1  Collapsed Gibbs Sampling

3.1.1  Run Gibbs sampler

3.1.2  Recover $\beta$ and $\theta$ from the sample

3.2  Variational EM

3.2.1  E-step

3.2.2  M-step

## Data: Reuters News Titles

Reuters is a multi-class, multi-label dataset.

* 90 classes
* 10788 documents
    * 7769 training documents
    * 3019 testing documents

```python
from nltk.corpus import reuters
from nltk.corpus import stopwords
```

* train-test split
: The data is already splitted. Just sort it out.

```python
stops = stopwords.words("english")
stops += [
    "a", "about", "above", "across", "after", "afterwards", "again", "against",
    "all", "almost", "alone", "along", "already", "also", "although", "always",
    "am", "among", "amongst", "amoungst", "amount", "an", "and", "another",
    "any", "anyhow", "anyone", "anything", "anyway", "anywhere", "are",
    "around", "as", "at", "back", "be", "became", "because", "become",
    "becomes", "becoming", "been", "before", "beforehand", "behind", "being",
    "below", "beside", "besides", "between", "beyond", "bill", "both",
    "bottom", "but", "by", "call", "can", "cannot", "cant", "co", "con",
    "could", "couldnt", "cry", "de", "describe", "detail", "do", "done",
    "down", "due", "during", "each", "eg", "eight", "either", "eleven", "else",
    "elsewhere", "empty", "enough", "etc", "even", "ever", "every", "everyone",
    "everything", "everywhere", "except", "few", "fifteen", "fifty", "fill",
    "find", "fire", "first", "five", "for", "former", "formerly", "forty",
    "found", "four", "from", "front", "full", "further", "get", "give", "go",
    "had", "has", "hasnt", "have", "he", "hence", "her", "here", "hereafter",
    "hereby", "herein", "hereupon", "hers", "herself", "him", "himself", "his",
    "how", "however", "hundred", "i", "ie", "if", "in", "inc", "indeed",
    "interest", "into", "is", "it", "its", "itself", "keep", "last", "latter",
    "latterly", "least", "less", "ltd", "made", "many", "may", "me",
    "meanwhile", "might", "mill", "mine", "more", "moreover", "most", "mostly",
    "move", "much", "must", "my", "myself", "name", "namely", "neither",
    "never", "nevertheless", "next", "nine", "no", "nobody", "none", "noone",
    "nor", "not", "nothing", "now", "nowhere", "of", "off", "often", "on",
    "once", "one", "only", "onto", "or", "other", "others", "otherwise", "our",
    "ours", "ourselves", "out", "over", "own", "part", "per", "perhaps",
    "please", "put", "rather", "re", "same", "see", "seem", "seemed",
    "seeming", "seems", "serious", "several", "she", "should", "show", "side",
    "since", "sincere", "six", "sixty", "so", "some", "somehow", "someone",
    "something", "sometime", "sometimes", "somewhere", "still", "such",
    "system", "take", "ten", "than", "that", "the", "their", "them",
    "themselves", "then", "thence", "there", "thereafter", "thereby",
    "therefore", "therein", "thereupon", "these", "they", "thick", "thin",
    "third", "this", "those", "though", "three", "through", "throughout",
    "thru", "thus", "to", "together", "too", "top", "toward", "towards",
    "twelve", "twenty", "two", "un", "under", "until", "up", "upon", "us",
    "very", "via", "was", "we", "well", "were", "what", "whatever", "when",
    "whence", "whenever", "where", "whereafter", "whereas", "whereby",
    "wherein", "whereupon", "wherever", "whether", "which", "while", "whither",
    "who", "whoever", "whole", "whom", "whose", "why", "will", "with",
    "within", "without", "would", "yet", "you", "your", "yours", "yourself",
    "yourselves", ".", "!", "?", ",", ";", ":", "[", "]", "{", "}", "-", "+",
    "_", "/", "@", "#", "$", "%", "^", "&", "*", "(", ")", "<", ">", "|", "=",
    ".-", ".,", "'", '"', ',"'
]
```

```python
reuters.words()[:10]
```

    ['ASIAN', 'EXPORTERS', 'FEAR', 'DAMAGE', 'FROM', 'U', '.', 'S', '.-', 'JAPAN']

```python
# fetch titles only
# fetch 2000 docs only
trainset, testset = [], []
vocab = []

i = 0
for file_id in reuters.fileids():
    if file_id.startswith("train"):
        doc = [w.lower() for w in reuters.words(file_id) \
                 if (w.isupper()) \
                 if (w.lower() not in stops) \
                 and (not w.isnumeric())]
        if doc:
            trainset.append(doc)
            vocab += doc
            i += 1
    else:
        testset.append([w.lower() for w in reuters.words(file_id) \
                         if (w.isupper()) \
                         and (w.lower() not in stops) \
                         and (not w.isnumeric())])
    if i >= 2000:
        break
```

```python
vocab = list(set(vocab))
word_to_ix = {w: i for i, w in enumerate(vocab)}
```

```python
def seq_to_ix(seq, vocab=vocab):
    # len(vocab), which is the last index, is for the <unk> (unknown) token
    unk_idx = len(vocab)
    return np.array(list(map(lambda w: word_to_ix.get(w, unk_idx), seq)))

data = {
    "train": list(map(seq_to_ix, trainset)),
    "test": list(map(seq_to_ix, testset))
}
```

```python
data["train"][0][:5]  # word indices
```

    array([3730, 1498, 3273, 1292,  695])

## Model: Basic LDA

### Variational EM

For each document $\mathbf{w}$ in a corpus $D$, generate

$$
N \sim \mathcal{P}(\xi) \\
\theta \sim \text{Dir}(\alpha)
$$

and for $n = 1, \cdots, N$, generate

$$
z_n \sim \text{Multi}(\theta) \\
w_n \sim P(w_n | z_n, \beta)
$$

where $\beta \in \mathbb{R}^{k \times V}$, $\beta_{ij} = P(w^j = 1| z^i = 1)$.

* $\alpha, \beta$: hyperparameters (Dirichlet, Multinomial).
* $N$: The number of words in the document. (ancillary variable)
* $\theta$: A topic mixture.
* (For $i$ in $1\cdots N$)
  * $z_n$: A topic variable.
  * $w_n$: A generated word.

```python
def init_lda(docs, vocab, n_topic, gibbs=False, random_state=0):
    if gibbs:
        global V, k, N, M, alpha, eta, n_iw, n_di
    else:
        global V, k, N, M, alpha, beta, gamma, phi

    np.random.seed(random_state)

    V = len(vocab)
    k = n_topic  # number of topics
    N = np.array([doc.shape[0] for doc in docs])
    M = len(docs)

    print(f"V: {V}\nk: {k}\nN: {N[:10]}...\nM: {M}")

    # initialize α, β

    if gibbs:
        alpha = np.random.gamma(shape=100, scale=0.01, size=1)  # one for all k
        eta = np.random.gamma(shape=100, scale=0.01, size=1)  # one for all V
        print(f"α: {alpha}\nη: {eta}")

        n_iw = np.zeros((k, V), dtype=int)
        n_di = np.zeros((M, k), dtype=int)
        print(f"n_iw: dim {n_iw.shape}\nn_di: dim {n_di.shape}")
    else:
        alpha = np.random.gamma(shape=100, scale=0.01, size=k) #np.random.rand(k)
        beta = np.random.dirichlet(np.ones(V), k)
        print(f"α: dim {alpha.shape}\nβ: dim {beta.shape}")

        # initialize ϕ, γ
        ## ϕ: (M x max(N) x k) arrays with zero paddings on the right
        gamma = alpha + np.ones((M, k)) * N.reshape(-1, 1) / k

        phi = np.ones((M, max(N), k)) / k
        for m, N_d in enumerate(N):
            phi[m, N_d:, :] = 0  # zero padding for vectorized operations

        print(f"γ: dim {gamma.shape}\nϕ: dim ({len(phi)}, N_d, {phi[0].shape[1]})")
```

#### E-step

Let $\phi_d \in \mathbb{R}^{N \times k}, \gamma_d \in \mathbb{R}^k$ be variational parameters for $\alpha, \beta$.

For a document $\mathbf{w}_d$, $d = 1,\cdots,M$,

1. initialize $\phi_{dni}^0 := 1/k$ for all $i,n$.
2. initialize $\gamma_{di}^0 := \alpha_i + N/k$ for all $i$.
3. **repeat until** convergence
    1. for $n=1$ to $N$
        1. for $i=1$ to $k$
            1. $\phi_{dni}^{t+1} := \beta_{i\mathbf{w}_{dn}}\exp\left(\Psi(\gamma_{di}^t) - \Psi(\sum_{j=1}^k \gamma_{dj}^t)\right)$
        2. normalize $\phi_{dn}^{t+1}$ to sum to 1
    2. $\gamma_d^{t+1} := \alpha + \sum_{n=1}^N \phi_{dn}^{t+1}$

where $\Psi$ is the first derivative of the $\log\Gamma$ function.

```python
def E_step(docs, phi, gamma, alpha, beta):
    """
    Minorize the joint likelihood function via variational inference.
    This is the E-step of variational EM algorithm for LDA.
    """
    # optimize phi
    for m in range(M):
        phi[m, :N[m], :] = (beta[:, docs[m]] * np.exp(psi(gamma[m, :]) - psi(gamma[m, :].sum())).reshape(-1, 1)).T

        # Normalize phi
        phi[m, :N[m]] /= phi[m, :N[m]].sum(axis=1).reshape(-1, 1)
        if np.any(np.isnan(phi)):
            raise ValueError("phi nan")

    # optimize gamma
    gamma = alpha + phi.sum(axis=1)

    return phi, gamma
```

#### M-step

$$
\beta_{ij} \propto \sum_{d=1}^M \sum_{n=1}^N \phi_{dni} \mathbf{w}_{dn}^j
$$

$\alpha$ is updated via Newton-Raphson method:

$$
\frac{\partial L}{\partial \alpha_i}
  = M\left( \Psi\left(\sum_{j=1}^k \alpha_j\right) - \Psi(\alpha_i) \right)
    - \sum_{d=1}^M \left( \Psi(\gamma_{di}) - \Psi\left(\sum_{j=1}^k \gamma_{dj}\right) \right) \\
\frac{\partial^2 L}{\partial \alpha_i \alpha_j} = M \left( \Psi'\left(\sum_{j=1}^k \alpha_j\right) - \delta(i,j) \Psi'(\alpha_i) \right)
$$

where $\delta(i,j) = 1$ if $i=j$, $0$ otherwise.

```python
def M_step(docs, phi, gamma, alpha, beta, M):
    """
    maximize the lower bound of the likelihood.
    This is the M-step of variational EM algorithm for (smoothed) LDA.

    update of alpha follows from appendix A.2 of Blei et al., 2003.
    """
    # update alpha
    alpha = _update(alpha, gamma, M)

    # update beta
    for j in range(V):
        beta[:, j] = np.array([_phi_dot_w(docs, phi, m, j) for m in range(M)]).sum(axis=0)
    beta /= beta.sum(axis=1).reshape(-1, 1)

    return alpha, beta
```

```python
import warnings

def _update(var, vi_var, const, max_iter=10000, tol=1e-6):
    """
    From appendix A.2 of Blei et al., 2003.
    For hessian with shape `H = diag(h) + 1z1'`

    To update alpha, input var=alpha and vi_var=gamma, const=M.
    To update eta, input var=eta and vi_var=lambda, const=k.
    """
    for _ in range(max_iter):
        # store old value
        var0 = var.copy()

        # g: gradient
        psi_sum = psi(vi_var.sum(axis=1)).reshape(-1, 1)
        g = const * (psi(var.sum()) - psi(var)) \
            + (psi(vi_var) - psi_sum).sum(axis=0)

        # H = diag(h) + 1z1'
        z = const * polygamma(1, var.sum())  # z: Hessian constant component
        h = -const * polygamma(1, var)       # h: Hessian diagonal component
        c = (g / h).sum() / (1./z + (1./h).sum())

        # update var
        var -= (g - c) / h

        # check convergence
        err = np.sqrt(np.mean((var - var0) ** 2))
        crit = err < tol
        if crit:
            break
    else:
        warnings.warn(f"max_iter={max_iter} reached: values might not be optimal.")

    #print(err)
    return var

def _phi_dot_w(docs, phi, d, j):
    """
    \sum_{n=1}^{N_d} ϕ_{dni} w_{dn}^j
    """
    # doc = np.zeros(docs[m].shape[0] * V, dtype=int)
    # doc[np.arange(0, docs[m].shape[0] * V, V) + docs[m]] = 1
    # doc = doc.reshape(-1, V)
    # lam += phi[m, :N[m], :].T @ doc
    return (docs[d] == j) @ phi[m, :N[d], :]
```

#### Variational lower bound

$$\begin{align}
L(\gamma, \phi; \alpha, \beta)
  &= \log\Gamma(\sum_{j=1}^k \alpha_j) - \sum_{i=1}^k \log\Gamma(\alpha_i) + \sum_{i=1}^k (\alpha_i - 1) \left(\Psi(\gamma_i) - \Psi(\sum_{i=1}^k \gamma_i)\right) \\
  &+ \sum_{n=1}^N \sum_{i=1}^k \phi_{ni} \left(\Psi(\gamma_i) - \Psi(\sum_{i=1}^k \gamma_i)\right) \\
  &+ \sum_{n=1}^N \sum_{i=1}^k \sum_{j=1}^V \phi_{ni} \mathbf{w}_{n}^j \log\beta_{ij} \\
  &- \log\Gamma(\sum_{i=1}^k \gamma_i) + \sum_{i=1}^k \log\Gamma(\gamma_i) - \sum_{i=1}^k (\gamma_i - 1) \left(\Psi(\gamma_i) - \Psi(\sum_{i=1}^k \gamma_i)\right) \\
  &- \sum_{n=1}^N \sum_{i=1}^k \phi_{ni} \log\phi_{ni}
\end{align}$$

```python
def dg(gamma, d, i):
    """
    E[log θ_t] where θ_t ~ Dir(gamma)
    """
    return psi(gamma[d, i]) - psi(np.sum(gamma[d, :]))

def dl(lam, i, w_n):
    """
    E[log β_t] where β_t ~ Dir(lam)
    """
    return psi(lam[i, w_n]) - psi(np.sum(lam[i, :]))

def vlb(docs, phi, gamma, alpha, beta, M, N, k):
    """
    Average variational lower bound for joint log likelihood.
    """
    lb = 0
    for d in range(M):
        lb += (
            gammaln(np.sum(alpha))
            - np.sum(gammaln(alpha))
            + np.sum([(alpha[i] - 1) * dg(gamma, d, i) for i in range(k)])
        )

        lb -= (
            gammaln(np.sum(gamma[d, :]))
            - np.sum(gammaln(gamma[d, :]))
            + np.sum([(gamma[d, i] - 1) * dg(gamma, d, i) for i in range(k)])
        )

        for n in range(N[d]):
            w_n = int(docs[d][n])

            lb += np.sum([phi[d][n, i] * dg(gamma, d, i) for i in range(k)])
            lb += np.sum([phi[d][n, i] * np.log(beta[i, w_n]) for i in range(k)])
            lb -= np.sum([phi[d][n, i] * np.log(phi[d][n, i]) for i in range(k)])

    return lb / M
```

#### Training

* Only on 2,000 documents

```python
import os
# for beutiful plot later, reorder training set
# (don't need to do this)

if "lda_trainset.idx" in os.listdir():
    with open("lda_trainset.idx") as r:
        idx = eval(r.read())
    docs = np.array(data["train"])[idx].tolist()
else:
    docs = data["train"]
```

    <ipython-input-29-53338b2e59c4>:8: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
      docs = np.array(data["train"])[idx].tolist()

```python
init_lda(docs, vocab, n_topic=10)
```

    V: 4048
    k: 10
    N: [16  5  6  7  9  7  8  8  6  5]...
    M: 2000
    α: dim (10,)
    β: dim (10, 4048)
    γ: dim (2000, 10)
    ϕ: dim (2000, N_d, 10)

```python
%%time

N_EPOCH = 1000
TOL = 0.1

verbose = True
lb = -np.inf

for epoch in range(N_EPOCH):
    # store old value
    lb_old = lb

    # Variational EM
    phi, gamma = E_step(docs, phi, gamma, alpha, beta)
    alpha, beta = M_step(docs, phi, gamma, alpha, beta, M)

    # check anomaly
    if np.any(np.isnan(alpha)):
        print("NaN detected: alpha")
        break

    # check convergence
    lb = vlb(docs, phi, gamma, alpha, beta, M, N, k)
    err = abs(lb - lb_old)

    # check anomaly
    if np.isnan(lb):
        print("NaN detected: lb")
        break

    if verbose:
        print(f"{epoch: 04}:  variational_lb: {lb: .3f},  error: {err: .3f}")

    if err < TOL:
        break
else:
    warnings.warn(f"max_iter reached: values might not be optimal.")

print(" ========== TRAINING FINISHED ==========")
```

     000:  variational_lb: -63.016,  error:  inf
     001:  variational_lb: -62.239,  error:  0.777
     002:  variational_lb: -61.789,  error:  0.450
     003:  variational_lb: -61.427,  error:  0.362
     004:  variational_lb: -61.108,  error:  0.319
     005:  variational_lb: -60.816,  error:  0.292
     006:  variational_lb: -60.545,  error:  0.270
     007:  variational_lb: -60.285,  error:  0.261
     008:  variational_lb: -60.018,  error:  0.267
     009:  variational_lb: -59.734,  error:  0.284
     010:  variational_lb: -59.426,  error:  0.308
     011:  variational_lb: -59.092,  error:  0.334
     012:  variational_lb: -58.737,  error:  0.355
     013:  variational_lb: -58.366,  error:  0.371
     014:  variational_lb: -57.988,  error:  0.379
     015:  variational_lb: -57.614,  error:  0.374
     016:  variational_lb: -57.259,  error:  0.355
     017:  variational_lb: -56.931,  error:  0.328
     018:  variational_lb: -56.632,  error:  0.299
     019:  variational_lb: -56.362,  error:  0.270
     020:  variational_lb: -56.120,  error:  0.242
     021:  variational_lb: -55.899,  error:  0.221
     022:  variational_lb: -55.693,  error:  0.206
     023:  variational_lb: -55.500,  error:  0.193
     024:  variational_lb: -55.320,  error:  0.180
     025:  variational_lb: -55.150,  error:  0.170
     026:  variational_lb: -54.990,  error:  0.160
     027:  variational_lb: -54.845,  error:  0.145
     028:  variational_lb: -54.718,  error:  0.128
     029:  variational_lb: -54.604,  error:  0.113
     030:  variational_lb: -54.501,  error:  0.103
     031:  variational_lb: -54.409,  error:  0.092
     ========== TRAINING FINISHED ==========
    CPU times: user 7min 49s, sys: 6.84 s, total: 7min 56s
    Wall time: 7min 56s

* Training result

1. Topic extraction

```python
def n_most_important(beta_i, n=30):
    """
    find the index of the largest `n` values in a list
    """

    max_values = beta_i.argsort()[-n:][::-1]
    return np.array(vocab)[max_values]
```

```python
for i in range(k):
    print(f"TOPIC {i:02}: {n_most_important(beta[i], 9)}")
```

    TOPIC 00: ['ec' 'csr' 'loss' 'bank' 'icco' 'unit' 'cocoa' 'fob' 'petroleum']
    TOPIC 01: ['raises' 'acquisition' 'prices' 'prime' 'rate' 'w' 'completes' 'mar'
     'imports']
    TOPIC 02: ['year' 'sets' 'sees' 'net' 'stock' 'dividend' 'l' 'industries' 'corp']
    TOPIC 03: ['pct' 'cts' 'gdp' 'opec' 'shr' 'february' 'plc' 'ups' 'rose']
    TOPIC 04: ['u' 'japan' 'fed' 'trade' 'gaf' 'ems' 'says' 'dlr' 'gnp']
    TOPIC 05: ['usda' 'f' 'ag' 'international' 'sell' 'report' 'ge' 'corn' 'wheat']
    TOPIC 06: ['k' 'market' 'money' 'rate' 'eep' 'treasury' 'prime' 'says' 'mln']
    TOPIC 07: ['qtr' 'note' '4th' 'net' 'loss' 'ico' 'corp' '1st' 'group']
    TOPIC 08: ['dlrs' 'mln' 'bp' 'march' 'corp' 'week' 'canada' 'e' 'bank']
    TOPIC 09: ['unit' 'buy' 'says' 'sale' 'c' 'sells' 'completes' 'american' 'grain']

2. Topic-word & document-document distribution

```python
n_sample = 10000
theta_hat = np.array([np.random.dirichlet(gamma[d], n_sample).mean(0) for d in range(M)])
theta_hat /= theta_hat.sum(1).reshape(-1, 1)
```

```python
plt.figure(figsize=(8,8))
plt.subplot(121)
n_plot_words = 150
sns.heatmap(beta.T[:n_plot_words], xticklabels=[], yticklabels=[])
plt.xlabel("Topics", fontsize=14)
plt.ylabel(f"Words[:{n_plot_words}]", fontsize=14)
plt.title("topic-word distribution", fontsize=16)

plt.subplot(122)
sns.heatmap(theta_hat, xticklabels=[], yticklabels=[])
plt.xlabel("Topics", fontsize=14)
plt.ylabel("Documents", fontsize=14)
plt.title("document-topic distribution", fontsize=16)

plt.tight_layout();
```

---

## Model: Smoothed LDA

For each document $\mathbf{w}$ in a corpus $D$, generate

$$
N \sim \mathcal{P}(\xi) \\
\beta \sim \text{Dir}(\lambda) \\
\theta \sim \text{Dir}(\alpha)
$$

and for $n = 1, \cdots, N$, generate

$$
z_n \sim \text{Multi}(\theta) \\
w_n \sim P(w_n | z_n, \beta)
$$

where $\beta \in \mathbb{R}^{k \times V}$, $\beta_{ij} = P(w^j = 1| z^i = 1)$.

* $\alpha, \eta$: Dirichlet hyperparameters.
* $\beta$: Unsmoothed multinomial hyperparameter.
* $N$: The number of words in the document. (ancillary variable)
* $\theta$: A topic mixture.
* (For $i$ in $1\cdots N$)
  * $z_n$: A topic variable.
  * $w_n$: A generated word.

### Collapsed Gibbs Sampling

$$
P(z_{dn}^i=1 | \mathbf{z}_{(-dn)},\mathbf{w}) \propto \frac{n_{(-dn),iw_{dn}}+\eta}{n_{(-dn),i\cdot}+V\eta} \frac{n_{(-dn),dj}+\alpha}{n_{(-dn),d\cdot}+k\alpha}.
$$

#### Run Gibbs sampler

```python
def _init_gibbs(docs, vocab, n_topic, n_gibbs=2000):
    """
    Initialize t=0 state for Gibbs sampling.
    Replace initial word-topic assignment ndarray (M, N, N_GIBBS) in-place.
    """
    # initialize variables
    init_lda(docs, vocab, n_topic=n_topic, gibbs=True)

    # word-topic assignment
    global assign
    N_max = max(N)
    assign = np.zeros((M, N_max, n_gibbs+1), dtype=int)
    print(f"assign: dim {assign.shape}")

    # initial assignment
    for d in range(M):
        for n in range(N[d]):
            # randomly assign topic to word w_{dn}
            w_dn = docs[d][n]
            assign[d, n, 0] = np.random.randint(k)

            # increment counters
            i = assign[d, n, 0]
            n_iw[i, w_dn] += 1
            n_di[d, i] += 1
```

```python
def _conditional_prob(w_dn, d):
    """
    P(z_{dn}^i=1 | z_{(-dn)}, w)
    """
    prob = np.empty(k)

    for i in range(k):
        # P(w_dn | z_i)
        _1 = (n_iw[i, w_dn] + eta) / (n_iw[i, :].sum() + V*eta)
        # P(z_i | d)
        _2 = (n_di[d, i] + alpha) / (n_di[d, :].sum() + k*alpha)

        prob[i] = _1 * _2

    return prob / prob.sum()
```

```python
def run_gibbs(docs, vocab, n_topic, n_gibbs=2000, verbose=True):
    """
    Run collapsed Gibbs sampling
    """
    # initialize required variables
    _init_gibbs(docs, vocab, n_topic, n_gibbs)

    if verbose:
        print("\n", "="*10, "START SAMPLER", "="*10)

    # run the sampler
    for t in range(n_gibbs):
        for d in range(M):
            for n in range(N[d]):
                w_dn = docs[d][n]

                # decrement counters
                i_t = assign[d, n, t]  # previous assignment
                n_iw[i_t, w_dn] -= 1
                n_di[d, i_t] -= 1

                # assign new topics
                prob = _conditional_prob(w_dn, d)
                i_tp1 = np.argmax(np.random.multinomial(1, prob))

                # increment counter according to new assignment
                n_iw[i_tp1, w_dn] += 1
                n_di[d, i_tp1] += 1
                assign[d, n, t+1] = i_tp1

        # print out status
        if verbose & ((t+1) % 50 == 0):
            print(f"Sampled {t+1}/{n_gibbs}")
```

```python
run_gibbs(docs, vocab, n_topic=10, n_gibbs=2000)
```

    V: 4048
    k: 10
    N: [16  5  6  7  9  7  8  8  6  5]...
    M: 2000
    α: [1.18335424]
    η: [1.03715177]
    n_iw: dim (10, 4048)
    n_di: dim (2000, 10)
    assign: dim (2000, 88, 2001)

     ========== START SAMPLER ==========
    Sampled 50/2000
    Sampled 100/2000
    Sampled 150/2000
    Sampled 200/2000
    Sampled 250/2000
    Sampled 300/2000
    Sampled 350/2000
    Sampled 400/2000
    Sampled 450/2000
    Sampled 500/2000
    Sampled 550/2000
    Sampled 600/2000
    Sampled 650/2000
    Sampled 700/2000
    Sampled 750/2000
    Sampled 800/2000
    Sampled 850/2000
    Sampled 900/2000
    Sampled 950/2000
    Sampled 1000/2000
    Sampled 1050/2000
    Sampled 1100/2000
    Sampled 1150/2000
    Sampled 1200/2000
    Sampled 1250/2000
    Sampled 1300/2000
    Sampled 1350/2000
    Sampled 1400/2000
    Sampled 1450/2000
    Sampled 1500/2000
    Sampled 1550/2000
    Sampled 1600/2000
    Sampled 1650/2000
    Sampled 1700/2000
    Sampled 1750/2000
    Sampled 1800/2000
    Sampled 1850/2000
    Sampled 1900/2000
    Sampled 1950/2000
    Sampled 2000/2000

#### Recover $\beta$ and $\theta$ from the sample

```python
β̂ = np.empty((k, V))
θ̂ = np.empty((M, k))

for j in range(V):
    for i in range(k):
        β̂[i, j] = (n_iw[i, j] + eta) / (n_iw[i, :].sum() + V*eta)

for d in range(M):
    for i in range(k):
        θ̂[d, i] = (n_di[d, i] + alpha) / (n_di[d, :].sum() + k*alpha)
```

```python
plt.figure(figsize=(8,8))
plt.subplot(121)
n_plot_words = 150
sns.heatmap(β̂.T[:n_plot_words], xticklabels=[], yticklabels=[])
plt.xlabel("Topics", fontsize=14)
plt.ylabel(f"Words[:{n_plot_words}]", fontsize=14)
plt.title("topic-word distribution", fontsize=16)

plt.subplot(122)
sns.heatmap(θ̂, xticklabels=[], yticklabels=[])
plt.xlabel("Topics", fontsize=14)
plt.ylabel("Documents", fontsize=14)
plt.title("document-topic distribution", fontsize=16)

plt.tight_layout();
```

### Variational EM

#### E-step

Let $\phi_d \in \mathbb{R}^{N \times k}, \gamma_d \in \mathbb{R}^k, \lambda \in \mathbb{R}^{k \times V}$ be variational parameters for $\alpha, \beta, \eta$.
Suppose further that for $\beta \in \mathbb{R}^{k \times V}$, $\beta_i^0 \sim \text{Dir}(\lambda^0)$ where $\lambda_i^0 = \eta$ for all $i$.

For a document $\mathbf{w}_d$, $d = 1,\cdots,M$,

1. initialize $\phi_{dni}^0 := 1/k$ for all $i,n$.
2. initialize $\gamma_{di} := \alpha_i + N/k$ for all $i$.
3. **repeat until** convergence
    1. for $n=1$ to $N$
        1. for $i=1$ to $k$
            1. $\phi_{dni}^{t+1} := \exp\left(\Psi(\lambda_{iw_{dn}}^t) - \Psi(\sum_{j=1}^V \lambda_{ij}^t) + \Psi(\gamma_{di}^t) - \Psi(\sum_{j=1}^k \gamma_{dj}^t)\right)$
            1. for $j=1$ to $V$
                1. $\lambda_{ij} = \eta + \sum_{d=1}^M \sum_{n=1}^{N_d} \phi_{dni} w_{dn}^j$
        2. normalize $\phi_{dn}^{t+1}$ to sum to 1
    2. $\gamma_d^{t+1} := \alpha + \sum_{n=1}^N \phi_{dn}^{t+1}$

where $\Psi$ is the first derivative of the $\log\Gamma$ function.

#### M-step

$$
\beta_{ij} \propto \sum_{d=1}^M \sum_{n=1}^N \phi_{dni} \mathbf{w}_{dn}^j
$$

$\alpha$ is updated via Newton-Raphson method:

$$
\frac{\partial L}{\partial \alpha_i}
  = M\left( \Psi\left(\sum_{j=1}^k \alpha_j\right) - \Psi(\alpha_i) \right)
    - \sum_{d=1}^M \left( \Psi(\gamma_{di}) - \Psi\left(\sum_{j=1}^k \gamma_{dj}\right) \right) \\
\frac{\partial^2 L}{\partial \alpha_i \alpha_j} = M \left( \Psi'\left(\sum_{j=1}^k \alpha_j\right) - \delta(i,j) \Psi'(\alpha_i) \right)
$$

where $\delta(i,j) = 1$ if $i=j$, $0$ otherwise.

```python
class SmoothedLDA:

    def __init__(self, docs, vocab, k):
        self.docs = docs

        self.V = len(vocab)
        self.k = k  # number of topics
        self.N = np.array([doc.shape[0] for doc in docs])
        self.M = len(docs)

        V = self.V
        N = self.N
        M = self.M

        # initialize model parameters
        ##self.beta = np.ones((k, V)) / V
        self.alpha = np.random.gamma(100, 0.01, k)
        self.eta = np.ones(V)

        # initialize variational parameters
        # ϕ: (M x max(N) x k) arrays with zero paddings on the right
        self.phi = [np.ones((N[d], k)) / k for d in range(M)]
        self.gamma = alpha + (N / k).reshape(-1, 1)
        self.lam = np.random.gamma(shape=100, scale=0.01, size=(k, V))

    def _update_phi(self):
        """
        Update variational parameter phi
        ϕ_{n, j} ∝ e^[ (Ψ(λ_j) - Ψ(Σλ_j)) + ( Ψ(γ_j) - Ψ(Σγ_j) ) ]
        """
        M = self.M
        N = self.N
        k = self.k

        phi = self.phi
        gamma = self.gamma
        docs = self.docs

        for d in range(M):
            for n in range(N[d]):
                for i in range(k):
                    w_n = int(docs[d][n])
                    phi[d][n, i] = np.exp(dl(lam, i, w_n) + dg(gamma, d, i))

                # Normalize over topics
                phi[d][n, :] = phi[d][n, :] / np.sum(phi[d][n, :])

        return phi

    def _update_gamma(self):
        """
        Update variational parameter gamma
        γ_t = α_t + Σ_{n=1}^{N_d} ϕ_{t, n}
        """
        M = self.M
        phi = self.phi
        alpha = self.alpha

        gamma = alpha + np.array(
            list(map(lambda x: x.sum(axis=0), phi))
        )

        return gamma

    def _update_lam(self):
        V = self.V
        N = self.N
        M = self.M
        phi = self.phi
        lam = self.lam
        eta = self.eta
        docs = self.docs

        lam[:] = eta
        for d in range(M):  #, desc="MINORIZE lam"):
            doc = np.zeros(N[d] * V, dtype=int)
            doc[np.arange(0, N[d] * V, V) + docs[d]] = 1
            doc = doc.reshape(-1, V)

            lam += phi[d].T @ doc

        return lam

    def _update_alpha(self, max_iter=1000, tol=0.1):
        """
        Update alpha with linear time Newton-Raphson.
        """
        M = self.M
        k = self.k

        alpha = self.alpha
        gamma = self.gamma

        for _ in range(max_iter):
            alpha_old = alpha

            #  Calculate gradient
            g = M * (psi(np.sum(alpha)) - psi(alpha)) +\
                (psi(gamma) - psi(np.sum(gamma, axis=1)).reshape(-1, 1)).sum(axis=0)

            #  Calculate Hessian diagonal component
            h = -M * polygamma(1, alpha)

            #  Calculate Hessian constant component
            z = M * polygamma(1, np.sum(alpha))

            #  Calculate constant
            c = np.sum(g / h) / (z ** (-1.0) + np.sum(h ** (-1.0)))

            #  Update alpha
            alpha = alpha - (g - c) / h

            #  Check convergence
            if np.sqrt(np.mean(np.square(alpha - alpha_old))) < tol:
                break
        else:
            warnings.warn("_update_alpha(): max_iter reached.")

        return alpha

    def _update_eta(self, max_iter=1000, tol=0.1):
        """
        Update eta with linear time Newton-Raphson.
        """
        M = self.M
        k = self.k

        eta = self.eta
        lam = self.lam

        for _ in range(max_iter):
            eta_old = eta

            #  Calculate gradient
            g = k * (psi(np.sum(eta)) - psi(eta)) +\
                (psi(lam) - psi(np.sum(lam, axis=1)).reshape(-1, 1)).sum(axis=0)

            #  Calculate Hessian diagonal component
            h = -k * polygamma(1, eta)

            #  Calculate Hessian constant component
            z = k * polygamma(1, np.sum(eta))

            #  Calculate constant
            c = np.sum(g / h) / (z ** (-1.0) + np.sum(h ** (-1.0)))

            #  Update alpha
            eta = eta - (g - c) / h

            #  Check convergence
            if np.sqrt(np.mean(np.square(eta - eta_old))) < tol:
                break
        else:
            warnings.warn("_update_eta(): max_iter reached.")

        return eta

    def _E_step(self):
        """
        E-step of the variational EM algorithm.
        Update ϕ, γ, λ.
        """
        self.phi = self._update_phi()
        self.gamma = self._update_gamma()
        self.lam = self._update_lam()

    def _M_step(self):
        """
        M-step of the variational EM algorithm.
        Update α, η.
        """
        self.alpha = self._update_alpha()
        self.eta = self._update_eta()

    def vlb(self):
        """
        lower bound from variational inference
        """
        phi = self.phi
        gamma = self.gamma
        lam = self.lam
        alpha = self.alpha
        eta = self.eta
        docs = self.docs

        M = self.M
        k = self.k
        N = self.N

        a0, a1, a2, a3_1, a3_2, a4, a5 = 0., 0., 0., 0., 0., 0., 0.
        for d in range(M):
            a0 += (
                k * (
                    gammaln(np.sum(eta))
                    - np.sum(gammaln(eta))
                )
                + np.sum([(eta[j] - 1) * dl(lam, i, j) for j in range(V) for i in range(k)])
            )
            a1 += (
                gammaln(np.sum(alpha))
                - np.sum(gammaln(alpha))
                + np.sum([(alpha[i] - 1) * dg(gamma, d, i) for i in range(k)])
            )

            a4 += (
                gammaln(np.sum(gamma[d, :]))
                - np.sum(gammaln(gamma[d, :]))
                + np.sum([(gamma[d, i] - 1) * dg(gamma, d, i) for i in range(k)])
            )

            for i in range(k):
                for j in range(V):
                    a3_2 += (
                        gammaln(np.sum(lam[i, j]))
                        - np.sum(gammaln(lam[i, :]))
                        + np.sum((lam[i, j] - 1) * dl(lam, i, j))
                    )

            for n in range(N[d]):
                w_n = int(docs[d][n])
                a2 += np.sum([phi[d][n, i] * dg(gamma, d, i) for i in range(k)])
                a3_1 += np.sum([phi[d][n, i] * dl(lam, i, w_n) for i in range(k)])
                a5 += np.sum([phi[d][n, i] * np.log(phi[d][n, i]) for i in range(k)])

        return a0 + a1 + a2 + a3_1 - a3_2 - a4 - a5

    def train(self, max_iter=1000, tol=5, verbose=True):
        vlb = -np.inf

        for it in range(max_iter):
            old_vlb = vlb
            self._E_step()
            self._M_step()

            vlb = self.vlb()
            err = vlb - old_vlb

            if verbose:
                print(f"Iteration {it+1}: {vlb: .3f} (delta: {err: .2f})")

            if err < tol:
                break
        else:
            warnings.warn("max_iter reached.")
```

```python
lda = SmoothedLDA(docs, vocab, k)
lda.train()
```

```python

```

NLP_with_PyTorch/3_document-embedding/3-1. latent dirichlet allocation.ipynb


# Modelo de Lack

El modelo del proceso de extracción supercritica parte de las ecuaciones de tranferencia de masa para describir la difusión del componente de interes dentro de una matriz vegetal como se muestra en la ecuación 9

Ecuación de tranferencia de masa

\begin{equation}
p_{f} E \frac{\partial y}{\partial t}+ p_{f} U\frac{\partial y}{\partial h} = J(x,y)
\end{equation}

en los trabajos de Sovova [], se desarrolló el tema de la manipulación de la ecuación de transferencia de masa para obtener una solución análitica de este modelo, dando lugar a una función definida por partes para describir la cinética del rendidmiento de extracción, tal como se muestra en las ecuaciones (2)- (6)

\begin{equation}
\tau_{1} = \frac{x_o-x_k}{yAy_r}
\end{equation}

\begin{equation}
\tau_{2} = \tau_{1} + \frac{xk}{\gamma A y_r}ln \left( \frac{x_k}{x_o} + \left( 1- \frac{x_k}{x_o}\right) e^\frac{x_oA}{x_k}   \right)
\end{equation}

\begin{equation}
z_{k} = \frac{x_k}{Ax_o} ln \frac{(x_o e^\frac{y A y_r}{x_k} (\tau-\tau_1) - x_k)}{x_o-x_k}
\end{equation}

\begin{equation}
s_1 = \gamma A y_r \tau e^{(-Az)}
\end{equation}

\begin{equation}
s_2 = (x_o - x_k) e^{(-A(z - z_k))}
\end{equation}

\begin{equation}
s_3 = 1 + \left( \frac{x_o}{x_k} e^{\left( \frac{\gamma Ay_r(\tau-\tau_1)} {x_k} \right)}-1 \right) e^{\left( \frac{-x_oAz} {x_k} \right)}
\end{equation}

\begin{equation}
x(\tau,z) =
\begin{cases}
x_o - s_1 & \tau \leq \tau_1\\
x_o - s_2 & \tau > \tau_1, z \geq z_k \\
\frac{x_o}{s_3} & \tau > \tau_1, z<z_k
\end{cases}
\end{equation}

\begin{equation}
u = 1 + \frac{x_k}{x_o} \left( e^\frac{x_oA}{x_k} - 1 \right)
\end{equation}

\begin{equation}
e =
\begin{cases}
\gamma y_r \tau (1- e^{(-A)} & \tau \leq \tau_1\\
\gamma y_r  (\tau - \tau_1 e^{(-A(1 - z_k))} & \tau_1 < \tau \leq \tau_2\\
x_o - \frac{x_k}{A}ln(u) e^\frac{yAy_r}{x_k}(\tau_1 - \tau) & \tau \geq \tau_2
\end{cases}
\end{equation}

```python

```

## Nomenclatura

- $a_0$ Surface, $\frac{m^2}{m^3}$
- A : (=tr /tf ), inverse de l'effet de transfert de masse de la phase fluide sur le taux d'extraction
- e : rendement de l'extraction, kg kgfeed−1
- h : coordonnée axiale le long du lit d'extraction, m
- H : hauteur du lit d'extraction, m
- J : taux du transfère de matière, kg m−3 s−1
- $k_{f}$: coefficient de transfère de matière phase liquide , m s−1
- $k_{s}$: coefficient de transfère de matière phase solide, m s−1
- K : coefficient de partition, kgfeed kgCO2−1
- n : nombre de mélangeur
- q : (=q’t), rapport solvant-alimentation, kgCO2 kgfeed−1
- q’: débit spécifique, kgCO2 s−1 kgfeed−1
- t : durée de l'extraction, s
- $t_{f}$: (=E/kf a0), temps caractéristique de transfert de matière dans le fluide, s
- $t_{r}$: temps de résidence, s
- $t_{s}$: (= (1-E)/kf a0), temps caractéristique de diffusion interne,s
- U : vélocité superficielle, m s−1
- v : facteur de taux de transfert de matière dans le modèle de Lack
- x : contenu de l'extrait dans la phase solide, kg kgsolid−1
- $x_{d}$: teneur critique de l'extrait dans la phase solide
- $x_{B}$: contenu de l'extrait au sein des cellules brisées,
- $k_{g}$: kgsolid−1
- $x_{I}$: contenu de l'extrait au sein des cellules intactes,
- $k_{g}$ : kgsolid−1
- $x_{k}$ : contenu minimal de l'extrait dans la phase solide pour lequel y+ = yr , kg kgsolid−1
- $x_{u}$: contenu de l'extrait dans les particules placées dans le réacteur , kg kgsolid−1
- $x_{uI}$ : contenu de l'extrait dans les cellules intactes,kg kgsolid−1
- y :contenu de l'extrait dans la phase liquid, kg kgCO2−1
- $y_{eq}$: concentration de la phase liquide à l'équilibre, kg kgCO2−1
- $y_{r}$: solubilité de l'extrait dans le CO2, kg kgCO2−1
- z :(h/H), coordonnée axiale sans dimension
- $z_{k}$:coordonnée axiale sans dimension où x = xk
- ε : porosité du lit (degré de vide)
- gamma : Rapport massique du CO2 dans le volume de vide du lit d'extraction, kgCO2 kgfeed−1
- $p_{f}$: densité du CO2, kgCO2 m−3
- $p_{s}$: densité des particules, kg m−3
- tao: (=t/tr ),temps (sans unité)

```python

```

```python

```

```python

```


# Autoregressive processes

- Once we learn the general equation and properties of the line / hyperplane we can turn around and perform various learning tasks - like regression and classification - using it as a model.

- By the same token now that we have a basic understanding of how to model general ordered data, we can use those models to perform various learning tasks.

- In this Section we use the generic fixed order dynamic system introduced previously to perform simple learning tasks on time series.  This is often referred to as *autoregressive modeling or prediction*.

Activate next cell to toggle code on and off

```python
from IPython.display import display
from IPython.display import HTML
import IPython.core.display as di # Example: di.display_html('<h3>%s:</h3>' % str, raw=True)

# This line will hide code by default when the notebook is eåxported as HTML
di.display_html('', raw=True)

# This line will add a button to toggle visibility of code blocks, for use with the HTML export version
di.display_html('''<button onclick="jQuery('.input_area').toggle(); jQuery('.prompt').toggle();">Toggle code</button>''', raw=True)
```

<button onclick="jQuery('.input_area').toggle(); jQuery('.prompt').toggle();">Toggle code</button>

```python
## This code cell will not be shown in the HTML version of this notebook
# imports from custom library
import sys
sys.path.append('../../')

# import custom lib
from mlrefined_libraries import recurrent_library as reclib

# import dataset path
datapath = '../../mlrefined_datasets/recurrent_datasets/'

# import autograd functionality to bulid function's properly for optimizers
import autograd.numpy as np

# this is needed to compensate for %matplotlib notebook's tendancy to blow up images when plotted inline
%matplotlib notebook
from matplotlib import rcParams
rcParams['figure.autolayout'] = True

%load_ext autoreload
%autoreload 2
```

# Using limited memory dynamic systems for time series prediction

- A very natural way of dealing with time series prediction problems:

"window" the data producing a set of input/output data

 train favorite standard ML scheme on input/output pairs

 to predict next future value feed in most recent window of series into trained model

- Making a training set of input/output data for a time series by "windowing" with a window size of $D = 3$ looks like this

- For example, using a linear `model` we press $x_{p}$ against a linear combination of its $D$ recent predecessors

- This takes the *exact* form of a *linear regression*, since we want to tune $w_0$ through $w_{D}$ so that the following holds

\begin{equation}
 w_0 + x_{p-1}w_1 + x_{p-2}w_2 + \cdots + x_{p-D}w_{D} \approx x_{p}
\end{equation}

- More compactly we can write this as

\begin{equation}
\mathring{\mathbf{x}}_{p-1}^T\mathbf{w} \approx x_{p}
\end{equation}

where $
\mathring{\mathbf{x}}_{p-1}=\begin{bmatrix}
1 \\
x_{p-1}\\
\vdots\\
x_{p-D}
\end{bmatrix}
\,\,\,\,
\mathbf{w}=\begin{bmatrix}
w_{0}\\
w_{1}\\
\vdots\\
w_{D}
\end{bmatrix}
$

- Hence it makes sense to tune these weights by minimizing e.g., a Least Squares cost for linear regression

\begin{equation}
\frac{1}{P-D}\sum_{p=D}^{P}\left(\mathring{\mathbf{x}}_{p-1}^T\mathbf{w} - x_{p}\right)^2
\end{equation}

- This is standard *supervised learning*, denote by

$$
\,\,\,\,\, \,\,\, \hat{y}_p:=\mathring{\mathbf{x}}_{p-1}^T\mathbf{w} \\
y_p:= x_p \\
\,\,\,\,\,  \,\,\,\,\,  \,\,\,\,\,  \,\,\,\,\,  \,\,  \ell\left(a,b\right) := \left(a-b \right)^2
$$

- We have a standard Least Squares regression

\begin{equation}
\frac{1}{P-D}\sum_{p=D}^{P}\ell\left(\hat{y}_p,y_p\right)
\end{equation}

- And we can extend this via nonlinear modeling precisely as we do with non time series data e.g., with a multilayer perceptron

- We simply replace the linear model $\text{model}\left(\mathbf{x},\mathbf{w}\right) = \mathring{\mathbf{x}}_{\,}^T\mathbf{w}$ with a nonlinear one $\text{model}\left(\mathbf{x},\Theta\right)$ built from units of a universal approximator (see [Chapters 10 - 13](https://github.com/jermwatt/machine_learning_refined))

- Another perspective: this approach turns the general framework of a [recurrence relation](https://jermwatt.github.io/control-notes/posts/recurrence_relations/Recurrence_relations.html) on its head, where proper parameters are *tuned* to data (instead of being set "by hand")

#### Example 1.   Time series prediction using an autoregressive model

- Below we load in a time series dataset we will perform prediction on

- Note: we will first ["contrast normalize"](https://jermwatt.github.io/control-notes/posts/zca_sphereing/ZCA_Sphereing.html) this series - that is we mean center and re-scale by its standard deviation

```python
## This code cell will not be shown in the HTML version of this notebook
# load in driver
csvname = datapath + 'normalized_apple_prices.csv'
x_series = np.loadtxt(csvname,delimiter = ',')[np.newaxis,:]

# standard normalize input series
normalizer,inverse_normalizer = reclib.time_series_lib.normalizers.standard(x_series)
x_series = normalizer(x_series)

# Plot the standard normalized series
reclib.time_series_lib.fixed_order_plotters.plot_sequences(x_series)
```

    <IPython.core.display.Javascript object>

- Next we choose an order for modeling, here we will use $D = 5$, and we produce a set of input/output data out of our time series as detailed above

- We can then use a [standard machine learning techniques](https://github.com/jermwatt/machine_learning_refined) - including [standard normalization](https://jermwatt.github.io/machine_learning_refined/notes/9_Feature_engineer_select/9_3_Scaling.html) the feature transforms like [multilayer perceptrons](https://jermwatt.github.io/machine_learning_refined/notes/13_Multilayer_perceptrons/13_2_Multi_layer_perceptrons.html), etc., - to learn an autoregressive model to this data to perfrom time series prediction

- Here we use the entire series as training, and tune a linear model via gradient descent

```python
# This code cell will not be shown in the HTML version of this notebook
# create windowed series
order = 5
x,y = reclib.time_series_lib.utils.window_series(x_series,order = order)

# initialize with input/output data
mylib1 = reclib.time_series_lib.basic_lib.super_setup.Setup(x,y)

# perform preprocessing step(s) - especially input normalization
mylib1.preprocessing_steps(normalizer = 'standard')

# split into training and validation sets
mylib1.make_train_val_split(train_portion = 1)

# choose cost
mylib1.choose_cost(name = 'least_squares')

# choose dimensions of fully connected multilayer perceptron layers
layer_sizes = [1]
mylib1.choose_features(feature_name = 'multilayer_perceptron',layer_sizes = layer_sizes,activation = 'linear',scale = 0.1)

# fit an optimization
mylib1.fit(max_its = 100,alpha_choice = 10**(-1),verbose = False)

# show cost function history
mylib1.show_histories()
```

    <IPython.core.display.Javascript object>

- Lets take a look - the series is plotted below along with the training fit in blue

```python
# Plot the standard normalized series and its training fit
reclib.time_series_lib.fixed_order_plotters.plot_train_val_sequences(mylib1,x_series)
```

    <IPython.core.display.Javascript object>

# Cross-validation for time series

- With an (assumed) i.i.d. dataset we split our data into training and validation sets [*randomly*](https://jermwatt.github.io/machine_learning_refined/notes/11_Feature_learning/11_3_Real_approximation.html)

- But this doesn't make sense with time series data.  Why?  Because validation data is supposed to model data, generated by the same process, that we will receive in the future

- Time series data doesn't arrive in this fashion - it arrives in contiguous chunks

- So cutting out validation data at random we take out a *contiguous portion*

- For example, we can take the first large contiguous chunk of the series as training, and validate on a final contiguous portion

- In terms of what portion of the data to assign to training / validation, the same rules of thumb still hold from ['standard' machine learning](https://jermwatt.github.io/machine_learning_refined/notes/11_Feature_learning/11_3_Real_approximation.html) (e.g., 60% - 90% commonly used to train, the remainder for validation)

- In terms of efficient cross-validation schemes - typically regularization-based methods like early-stopping are used (to decide on the proper model parameter settings based on minimum validation error)

#### Example 2.   Early stopping based cross-validation

- Below is an example where we lop off the last third of the previously shown time series example for validation

- We use a linear model, tune its parameters via gradient descent, and use early stopping to make the final parameter choice

- The training fit is shown in blue, and validation in yellow

```python
# This code cell will not be shown in the HTML version of this notebook
# initialize with input/output data
mylib2 = reclib.time_series_lib.basic_lib.super_setup.Setup(x,y)

# perform preprocessing step(s) - especially input normalization
mylib2.preprocessing_steps(normalizer = 'standard')

# split into training and validation sets
mylib2.make_train_val_split(train_portion = 0.66)

# choose cost
mylib2.choose_cost(name = 'least_squares')

# choose dimensions of fully connected multilayer perceptron layers
layer_sizes = [1]
mylib2.choose_features(feature_name = 'multilayer_perceptron',layer_sizes = layer_sizes,activation = 'linear',scale = 0.1)

# fit an optimization
mylib2.fit(max_its = 500,alpha_choice = 10**(-1),verbose = False)

# show cost function history
mylib2.show_histories()
```

    <IPython.core.display.Javascript object>

- We can then choose the best model and weights based on lowest validation error.

- Lets take a look - the series is plotted by the next cell.

- The training fit is shown in blue, validation shown in yellow.

```python
# Plot the standard normalized series
reclib.time_series_lib.fixed_order_plotters.plot_train_val_sequences(mylib2,x_series)
```

    <IPython.core.display.Javascript object>

# Using a trained model as a recurrence relation or "generative model"

- With a trained model in hand, we can then use it as a [recurrence relation](https://jermwatt.github.io/control-notes/posts/recurrence_relations/Recurrence_relations.html) or "generative model" to produce predictions as far forward in the future as we like

- How do we do this if we - in order to make a single prediction - we need a contiguous window of the actual series?

- By treating the previously *predicted values* as real elements of the series, as shown below for a prototypical series and window length $D = 3$.

- This provides *very rough* prediction accuracy, since using the model as a generator means eventually it takes in all of its own predictions as input (which eventually degrades its predictive power)

```python
# Plot the standard normalized series
reclib.time_series_lib.fixed_order_plotters.plot_train_gen_sequences(mylib2,x_series)
```

    <IPython.core.display.Javascript object>


## Helmholtz energy conversion of cubics

As in Michelsen's book, PR and SRK can be given the common form:
$$ p = \frac{RT}{v-b}-\frac{a(T)}{(v+\Delta_1b)(v+\Delta_2b)} $$
which can be ultimately converted to
$$ \alpha^r = - \log{\left (- b \delta \rho_{r} + 1 \right )} - \frac{\tau a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )} $$

```python
from __future__ import division
from sympy import *
from IPython.display import display, Math, Latex
from IPython.core.display import display_html
init_session(quiet=True, use_latex='mathjax')
init_printing()
```

    IPython console for SymPy 0.7.6 (Python 2.7.10-64-bit) (ground types: python)

```python
b,rho_r,tau,delta,T_c,R,Delta_1,Delta_2 = symbols('b,rho_r,tau,delta,T_c,R,Delta_1,Delta_2')
a = symbols('a', cls=Function)(tau)
```

```python
alphar = -log(1-b*rho_r*delta)-a*tau/(R*b*T_c*(Delta_1-Delta_2))*log((1+Delta_1*b*rho_r*delta)/(1+Delta_2*b*rho_r*delta))
display(alphar)
```

```python
def format_deriv(arg, itau, idel, RHS):
    """
    A function for giving a nice latex representation of
    the partial derivative in question
    """
    if itau+idel == 1:
        numexp = ''
    else:
        numexp = '^{{{s:d}}}'.format(s=itau+idel)

    if itau == 0:
        tau = ''
    elif itau == 1:
        tau = '\\partial \\tau'
    else:
        tau = '\\partial \\tau^{{{s:d}}}'.format(s=itau)

    if idel == 0:
        delta = ''
    elif idel == 1:
        delta = '\\partial \\delta'
    else:
        delta = '\\partial \\delta^{{{s:d}}}'.format(s=idel)

    temp = '\\frac{{\\partial{{{numexp:s}}} {arg:s}}}{{{{{tau:s}}}{{{delta:s}}}}} = '
    return Math(temp.format(**locals()) + latex(RHS))
```

```python
for deriv_count in range(1,5):
    for dtau in range(deriv_count+1):
        ddelta = deriv_count-dtau
        #print dtau, ddelta
        display(format_deriv('\\alpha^r', dtau, ddelta, diff(diff(alphar,tau,dtau),delta,ddelta)))
```

$$\frac{\partial{} \alpha^r}{{}{\partial \delta}} = \frac{b \rho_{r}}{- b \delta \rho_{r} + 1} - \frac{\tau a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\frac{\Delta_{1} b \rho_{r}}{\Delta_{2} b \delta \rho_{r} + 1} - \frac{\Delta_{2} b \rho_{r} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)$$

$$\frac{\partial{} \alpha^r}{{\partial \tau}{}} = - \frac{\tau \frac{d}{d \tau} a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )} - \frac{a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )}$$

$$\frac{\partial{^{2}} \alpha^r}{{}{\partial \delta^{2}}} = b \rho_{r}^{2} \left(\frac{\Delta_{1} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{2}} + \frac{\Delta_{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)} + \frac{b}{\left(b \delta \rho_{r} - 1\right)^{2}}\right)$$

$$\frac{\partial{^{2}} \alpha^r}{{\partial \tau}{\partial \delta}} = - \frac{\tau \frac{d}{d \tau} a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\frac{\Delta_{1} b \rho_{r}}{\Delta_{2} b \delta \rho_{r} + 1} - \frac{\Delta_{2} b \rho_{r} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right) - \frac{a{\left (\tau \right )}}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\frac{\Delta_{1} b \rho_{r}}{\Delta_{2} b \delta \rho_{r} + 1} - \frac{\Delta_{2} b \rho_{r} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)$$

$$\frac{\partial{^{2}} \alpha^r}{{\partial \tau^{2}}{}} = - \frac{1}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \left(\tau \frac{d^{2}}{d \tau^{2}}  a{\left (\tau \right )} + 2 \frac{d}{d \tau} a{\left (\tau \right )}\right) \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )}$$

$$\frac{\partial{^{3}} \alpha^r}{{}{\partial \delta^{3}}} = 2 b^{2} \rho_{r}^{3} \left(- \frac{\Delta_{1}^{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{3}} - \frac{\Delta_{1} \Delta_{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{2} \left(\Delta_{2} b \delta \rho_{r} + 1\right)} - \frac{\Delta_{2}^{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}} - \frac{b}{\left(b \delta \rho_{r} - 1\right)^{3}}\right)$$

$$\frac{\partial{^{3}} \alpha^r}{{\partial \tau}{\partial \delta^{2}}} = \frac{b \rho_{r}^{2}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) \left(\frac{\Delta_{1} \tau \frac{d}{d \tau} a{\left (\tau \right )}}{\Delta_{1} b \delta \rho_{r} + 1} + \frac{\Delta_{1} a{\left (\tau \right )}}{\Delta_{1} b \delta \rho_{r} + 1} + \frac{\Delta_{2} \tau \frac{d}{d \tau} a{\left (\tau \right )}}{\Delta_{2} b \delta \rho_{r} + 1} + \frac{\Delta_{2} a{\left (\tau \right )}}{\Delta_{2} b \delta \rho_{r} + 1}\right)$$

$$\frac{\partial{^{3}} \alpha^r}{{\partial \tau^{2}}{\partial \delta}} = - \frac{1}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\tau \frac{d^{2}}{d \tau^{2}}  a{\left (\tau \right )} + 2 \frac{d}{d \tau} a{\left (\tau \right )}\right) \left(\frac{\Delta_{1} b \rho_{r}}{\Delta_{2} b \delta \rho_{r} + 1} - \frac{\Delta_{2} b \rho_{r} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)$$

$$\frac{\partial{^{3}} \alpha^r}{{\partial \tau^{3}}{}} = - \frac{1}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \left(\tau \frac{d^{3}}{d \tau^{3}}  a{\left (\tau \right )} + 3 \frac{d^{2}}{d \tau^{2}}  a{\left (\tau \right )}\right) \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )}$$

$$\frac{\partial{^{4}} \alpha^r}{{}{\partial \delta^{4}}} = 6 b^{3} \rho_{r}^{4} \left(\frac{\Delta_{1}^{3} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{4}} + \frac{\Delta_{1}^{2} \Delta_{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{3} \left(\Delta_{2} b \delta \rho_{r} + 1\right)} + \frac{\Delta_{1} \Delta_{2}^{2} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)^{2} \left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}} + \frac{\Delta_{2}^{3} \tau \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) a{\left (\tau \right )}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)^{3}} + \frac{b}{\left(b \delta \rho_{r} - 1\right)^{4}}\right)$$

$$\frac{\partial{^{4}} \alpha^r}{{\partial \tau}{\partial \delta^{3}}} = \frac{2 b^{2} \rho_{r}^{3}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) \left(- \frac{\Delta_{1}^{2} \tau \frac{d}{d \tau} a{\left (\tau \right )}}{\left(\Delta_{1} b \delta \rho_{r} + 1\right)^{2}} - \frac{\Delta_{1}^{2} a{\left (\tau \right )}}{\left(\Delta_{1} b \delta \rho_{r} + 1\right)^{2}} - \frac{\Delta_{1} \Delta_{2} \tau \frac{d}{d \tau} a{\left (\tau \right )}}{\left(\Delta_{1} b \delta \rho_{r} + 1\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)} - \frac{\Delta_{1} \Delta_{2} a{\left (\tau \right )}}{\left(\Delta_{1} b \delta \rho_{r} + 1\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)} - \frac{\Delta_{2}^{2} \tau \frac{d}{d \tau} a{\left (\tau \right )}}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}} - \frac{\Delta_{2}^{2} a{\left (\tau \right )}}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right)$$

$$\frac{\partial{^{4}} \alpha^r}{{\partial \tau^{2}}{\partial \delta^{2}}} = \frac{b \rho_{r}^{2}}{R T_{c} \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\Delta_{1} - \frac{\Delta_{2} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\Delta_{2} b \delta \rho_{r} + 1}\right) \left(\frac{\Delta_{1}}{\Delta_{1} b \delta \rho_{r} + 1} + \frac{\Delta_{2}}{\Delta_{2} b \delta \rho_{r} + 1}\right) \left(\tau \frac{d^{2}}{d \tau^{2}}  a{\left (\tau \right )} + 2 \frac{d}{d \tau} a{\left (\tau \right )}\right)$$

$$\frac{\partial{^{4}} \alpha^r}{{\partial \tau^{3}}{\partial \delta}} = - \frac{1}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right) \left(\Delta_{1} b \delta \rho_{r} + 1\right)} \left(\tau \frac{d^{3}}{d \tau^{3}}  a{\left (\tau \right )} + 3 \frac{d^{2}}{d \tau^{2}}  a{\left (\tau \right )}\right) \left(\frac{\Delta_{1} b \rho_{r}}{\Delta_{2} b \delta \rho_{r} + 1} - \frac{\Delta_{2} b \rho_{r} \left(\Delta_{1} b \delta \rho_{r} + 1\right)}{\left(\Delta_{2} b \delta \rho_{r} + 1\right)^{2}}\right) \left(\Delta_{2} b \delta \rho_{r} + 1\right)$$

$$\frac{\partial{^{4}} \alpha^r}{{\partial \tau^{4}}{}} = - \frac{1}{R T_{c} b \left(\Delta_{1} - \Delta_{2}\right)} \left(\tau \frac{d^{4}}{d \tau^{4}}  a{\left (\tau \right )} + 4 \frac{d^{3}}{d \tau^{3}}  a{\left (\tau \right )}\right) \log{\left (\frac{\Delta_{1} b \delta \rho_{r} + 1}{\Delta_{2} b \delta \rho_{r} + 1} \right )}$$

```python

```