# Information-theoretic analysis of EEG microstates in Python

The following CSS code will render the document style for better visualization (https://stackoverflow.com/questions/34303422).

```python
from IPython.core.display import HTML
HTML("""
<style>

div.cell { /* Tunes the space between cells */
margin-top:1em;
margin-bottom:1em;
}

div.text_cell_render h1 { /* Main titles bigger, centered */
font-size: 2.2em;
line-height:1.4em;
text-align:center;
}

div.text_cell_render h2 { /*  Parts names nearer from text -0.4em*/
margin-bottom: 0.0em;
}

div.text_cell_render { /* Customize text cells */
font-family: 'Times New Roman';
font-size:1.5em;
line-height:1.4em;
padding-left:3em;
padding-right:3em;
}
</style>
""")
```

<style>

div.cell { /* Tunes the space between cells */
margin-top:1em;
margin-bottom:1em;
}

div.text_cell_render h1 { /* Main titles bigger, centered */
font-size: 2.2em;
line-height:1.4em;
text-align:center;
}

div.text_cell_render h2 { /*  Parts names nearer from text -0.4em*/
margin-bottom: 0.0em;
}

div.text_cell_render { /* Customize text cells */
font-family: 'Times New Roman';
font-size:1.5em;
line-height:1.4em;
padding-left:3em;
padding-right:3em;
}
</style>

Our software is provided as a single Python source code file (eeg_microstates.py) that contains all functions necessary to:
* import EEG data in the '.edf' file format
* pre-process (band-pass filter) the EEG data
* compute microstate maps by the modified K-means algorithm [1]
* compute microstate time series by competitively back-fitting the microstate maps into the EEG data
* perform information-theoretical calculations we recently introduced [2]

[1] Pascual-Marqui RD, Michel CM, Lehmann D. Segmentation of brain electrical activity into microstates: model estimation and validation. IEEE Trans Biomed Eng 42 (1995) 658–665.

[2] von Wegner F, Tagliazucchi E, Laufs H. Information-theoretical analysis of resting state EEG microstate sequences - non-Markovianity, non-stationarity and periodicities. NeuroImage 158 (2017) 99–111.

[3] von Wegner F, Laufs H. Information-theoretical analysis of EEG microstate sequences in Python. Front Neuroinformatics (2018) doi: 10.3389/fninf.2018.00030.

[4] von Wegner F, Knaut P, Laufs H. EEG microstate sequences from different clustering algorithms are information-theoretically invariant. Front Comp Neurosci (2018) doi: 10.3389/fncom.2018.00070.

In order to display images, we import the following notebook utility.

```python
from IPython.core.display import Image
```

In order to work with the package, the following import statement can be used in Python standalone scripts, e.g. for batch processing, or in any Python-based EEG software to add extra functionality:

```python
from eeg_microstates import *
```

### Github link of this project

The current version of the software can be found on the [GitHub project page](https://github.com/Frederic-vW/eeg_microstates/tree/master/).

### Current literature about EEG microstates and information theory

To find out about current publications and directions in microstate research, in particular related to information-theoretical approaches, we provide an active link to the PubMed page. If using the interactive IPython notebook, you can use the PubMed search bar to adapt the search term.

```python
url = 'https://www.ncbi.nlm.nih.gov/pubmed/?term=eeg+microstates+information+theory'
iframe = ''
HTML(iframe)
```

## Background

Electroencephalography (EEG) records the brain's electrical potential at discrete spatial sampling sites, defined by the locations of the electrodes on the skull. We also call the potential distribution at a given time a topography, or a map. The idea behind the microstate algorithm is to represent the EEG time series by a small number of maps that yield an optimum representation of the full data set. As in other areas of data compression, optimality can be defined in different ways. For EEG microstates, the criterion of choice is that of maximum explained variance. To this end, a (large) set of EEG samples is fed into a clustering algorithm whose output is a reduced set of representative EEG topographies, also called termed microstates, or microstate maps.
The EEG time series can then be represented by a sequence of microstate labels, where each time point of the original EEG data is mapped to the microstate with the maximum spatial correlation to the actual EEG topography. Polarity is usually ignored, so the squared correlation coefficient between each candidate microstate map and the EEG topography is used as a similarity measure.
With a set of representative microstates at hand, the multi-channel EEG data set can be compressed into a time series of symbols, i.e. microstate labels. Most frequently, four microstates are used to compress the EEG data set, and the microstates are labelled A, B, C, D. The EEG data set is therefore transformed into a microstate sequence that could read like ...AAABBDCDABBDAB...
Due to the special, non-metric nature of the new state space {A, B, C, D}, many common time-series methods such as the autocorrelation function cannot be applied to the microstate time series. We therefore test several entropy-based measures based on Shannon's information theory to quantify these time series.

The details of the procedure are given in the following sections and the whole procedure is illustrated in the figure below.
Step 1: The EEG data set is submitted to the modified K-means algorithm to obtain the microstate maps A-D. The figure below shows a section of resting state EEG for a selection of the 30 channels recorded. At each time point, the spatial standard deviation of the EEG topography is also called global field power (GFP, blue). Topographies at local maxima of the GFP time series (MAX, red dots) define the input to the modified K-means clustering algorithm.
Step 2: The microstate time series is computed by competitive back-fitting of the microstate maps into the EEG time series.
Step 3: Several information-theoretical tests are run on the microstate sequence and finally, the time-lagged mutual information function and a first-order Markov confidence interval are computed and displayed.

```python
Image(filename='Figure_1.jpg')
```

## (1) EEG data input

The package contains a basic [EDF](http://www.edfplus.info/specs/edf.html) file format reader. EEG data is returned as a NumPy array with time running along the rows (first index) and channels across the columns (2nd index), i.e. the array shape is (n_samples, n_channels). If our code is interfaced with other Python programs, EEG data must be provided in that shape in order to be processed correctly by our package functions.
The return variables of the function read_edf are (1) a list of channel names, (2) the sampling rate in Hz, and (3) the EEG data in the mentioned format.
To run this tutorial, you must have the test.edf EEG file in your current folder. The file is part of the GitHub repository. The record contains 192 sec. from an eyes-closed resting state experiment of a healthy male subject recorded with a 30 channel EEG cap using a standard 10-10 electrode configuration. The sampling rate is 250 Hz and the data is band-pass filtered to the 1-40 Hz range.

```python
filename = "test.edf"
chs, fs, data_raw = read_edf(filename)
print("\n\tFile: {}".format(filename))
print("\tSampling rate: {:.2f} Hz".format(fs))
print("\tData shape: {:d} samples x {:d} channels".format(data_raw.shape[0], data_raw.shape[1]))
print("\tChannels:")
for ch in chs: print "\t{:s}".format(ch),
```

    {'startdate': '14.10.08', 'digimin': [-30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001, -30001], 'physmin': [-28.195, -31.263, -31.614, -25.298, -27.77, -25.518, -40.338, -39.106, -57.314, -52.697, -29.888, -32.122, -27.329, -25.885, -39.29, -56.341, -30.092, -30.501, -37.138, -28.687, -30.758, -25.496, -30.673, -19.727, -26.854, -31.31, -48.835, -44.015, -32.758, -36.985], 'reserved': '                                            ', 'samples_per_record': [250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0, 250.0], 'recordingID': 'Startdate 14-OCT-2008 X X X                                                     ', 'physdime': ['uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV', 'uV'], 'patientID': 'X X 01-JAN-0000 X                                                               ', 'digimax': [30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000, 30000], 'physmax': [28.195, 31.263, 31.614, 25.298, 27.77, 25.518, 40.338, 39.106, 57.314, 52.697, 29.888, 32.122, 27.329, 25.885, 39.29, 56.341, 30.092, 30.501, 37.138, 28.687, 30.758, 25.496, 30.673, 19.727, 26.854, 31.31, 48.835, 44.015, 32.758, 36.985], 'channels': 30, 'records': 192, 'length': 7936, 'version': '0       ', 'transducer': ['AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode', 'AgAgCl', 'electrode'], 'starttime': '20.48.12', 'duration': 1.0, 'channelname': 'Fp1             Fp2             F3              F4              C3              C4              P3              P4              O1              O2              F7              F8              T7              T8              P7              P8              Fz              Cz              Pz              AFz             AF3             AF4             FC3             FC4             FT9             FT10            TP9             TP10            CP5             CP6             ', 'prefilt': []}

    	File: test.edf
    	Sampling rate: 250.00 Hz
    	Data shape: 48000 samples x 30 channels
    	Channels:
    	Fp1 	Fp2 	F3 	F4 	C3 	C4 	P3 	P4 	O1 	O2 	F7 	F8 	T7 	T8 	P7 	P8 	Fz 	Cz 	Pz 	AFz 	AF3 	AF4 	FC3 	FC4 	FT9 	FT10 	TP9 	TP10 	CP5 	CP6

## (2) Pre-processing

EEG data is usually band-pass filtered before microstate clustering. The package contains a zero-phase 6-th order Butterworth band-pass filter as provided by SciPy. The input is (1) the data to be filtered as a NumPy array of shape (n_samples, n_channels), (2) a tuple defining the desired frequency band in Hz, and (3) the EEG sampling rate in Hz.

```python
data = bp_filter(data_raw, (1, 35), fs)
```

    /usr/local/lib/python2.7/dist-packages/scipy/signal/_arraytools.py:45: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
      b = a[a_slice]

## (3) Visualize data (PCA-1)

A basic data visualization tool is provided to get an idea of the temporal structure of the EEG data set. We use principle component analysis as provided in the scikit-learn package and plot the time course of the first principal component to represent the dominant component of the data set. Using the test.edf file, we find amplitude modulated oscillations, mainly in the alpha frequency band (8-12), as expected in a wakeful resting condition.

```python
pca = PCA(copy=True, n_components=1, whiten=False)
pca1 = pca.fit_transform(data)[:,0]
plot_data(pca1, fs)
```

Alternatively, only a segment of data can be visualized to better recognize alpha oscillations.

```python
t0, t1 = 1000, 3000
plot_data(pca1[t0:t1], fs)
```

## (4) Clustering of the microstate maps

We are now ready to apply the microstate clustering algorithm to the filtered data. We implemented five different clustering methods (AAHC, modified k-means, k-medoids, PCA, Fast-ICA), as detailed in [4].
Here, we will run the modified K-means algorithm as given by Pascual-Marqui [1], implemented as the function 'kmeans' in the eeg_microstates.py file [3]. The function variables should be self-explaining and are further documented in the source file following the PEP 8 styleguide.
As the initial microstate ordering returned by the stochastic K-means algorithm is random, we provide a tool to re-label the microstates interactively. The standard ordering follows the geometry of the maps. Map A has a diagonal direction connecting the upper (frontal) left and lower (occipital) right corners, map B is diagonal in the opposite direction, map C has a horizontal orientation and map D is often circular. Sometimes, slightly different maps are generated. We mostly observed the occurrence of a map D with a vertical axis instead of a circular pattern. In this case, you can either accept the results and proceed, or re-cluster the data set as the results of the K-means algorithm differ between runs.
Modified K-means clustering and all information-theoretical computations can be run for any number of microstates. The desired number of microstates is given by the user as the argument 'n_maps' of the 'kmeans' function. If n_maps is not provided, the default value n_maps=4 is used.

```python
mode = ["aahc", "kmeans", "kmedoids", "pca", "ica"][1]
print("Clustering algorithm: {:s}".format(mode))
```

    Clustering algorithm: kmeans

```python
n_maps = 4
locs = []
maps, x, gfp_peaks, gev = clustering(data, fs, chs, locs, mode, n_maps, doplot=True)
```

You can also inspect the source code of the function:

```python
import inspect
# c=inspect.getsourcelines(kmeans) # activate by uncomment
# for l in c[0]: print l
```

When the algorithm terminates, the global explained variance (GEV) of the computed microstate time series is shown. For the test file, the GEV should give approximately 0.73. In other words, the microstate sequence explains ca. 73% of the electrical potential's spatial variance over time.

The first return variable of the kmeans function is an array containing the microstate maps row-wise. The second return variable is the microstate sequence corresponding to the EEG data that was used as input. The sequence was generated as part of the kmeans optimizations by back-fitting into the EEG time series, and the sequence corresponding to the best K-means run is returned.
On this sequence all subsequent tests will be run. The sequence contains the microstate labels as numbers, $\{A, B, C, D\} \rightarrow \{0, 1, 2, 3\}$, in order to accelerate information-theoretical computations. Using numbers, the microstate label can directly be used as an index in arrays and matrices. If you want to test microstate sequences generated by other another software, the sequence must be converted to numerical values $0, ..., n_{maps}-1$ before using the functions of this package.

## (5) Basic microstate statistics

We will now look at some basic statistical properties of the microstate sequence returned by the kmeans function. The empirical symbol distribution $\hat{p}$ quantifies the relative frequency of each microstate map. It is equivalent to the so-called 'ratio of time covered' (RTT) used in some EEG microstate publications. The transition matrix $\hat{T}$ contains the transition probabilities between all pairs of microstate labels. To get a feeling of the temporal dynamics of the EEG's spatial variance, we also compute the number of GFP peaks per second. The value is roughly 2 x the dominant EEG frequency band. In our case, the dominant frequency is a 10 Hz alpha rhythm and we observe approx. 21.5 GFP peaks per second.

```python
p_hat = p_empirical(x, n_maps)
T_hat = T_empirical(x, n_maps)
print("\n\t\tEmpirical symbol distribution (RTT):\n")
for i in range(n_maps): print("\t\tp_{:d} = {:.3f}".format(i, p_hat[i]))
print("\n\t\tEmpirical transition matrix:\n")
print_matrix(T_hat)
pps = len(gfp_peaks) / (len(x)/fs)  # peaks per second
print("\n\t\tGFP peaks per sec.: {:.2f}".format(pps))
print("\n\t\tGlobal explained variance (GEV) per map:")
print "\t\t" + str(gev)
print("\n\t\ttotal GEV: {:.2f}".format(gev.sum()))
```

    		Empirical symbol distribution (RTT):

    		p_0 = 0.222
    		p_1 = 0.240
    		p_2 = 0.264
    		p_3 = 0.275

    		Empirical transition matrix:

    		|0.771 0.094 0.058 0.076|
    		|0.063 0.769 0.112 0.056|
    		|0.063 0.055 0.786 0.096|
    		|0.070 0.072 0.061 0.797|

    		GFP peaks per sec.: 21.51

    		Global explained variance (GEV) per map:
    		[0.10284771 0.13338401 0.27586287 0.17435817]

    		total GEV: 0.69

## (6) Shannon entropy and entropy rate of the microstate sequence

The information-theoretical characterization of the microstate time series starts with the calculation of the empirical Shannon entropy $\hat{H}$ of the sequence:

$$
\begin{align}
\hat{H} & = -\sum_i \hat{p}(i) \log \hat{p}(i)
\end{align}
$$

where the index $i$ runs over the microstate labels. The base of the logarithm is not important. We use the natural logarithm to the base $e$ (Euler's number) and therefore measure information in 'nats', in analogy to bits when using $\log_2$.

```python
h_hat = H_1(x, n_maps)
h_max = max_entropy(n_maps)
print("\n\t\tEmpirical entropy H = {:.2f} (max. entropy: {:.2f})\n".format(h_hat, h_max) )
```

    		Empirical entropy H = 1.38 (max. entropy: 1.39)

The maximum entropy possible for any sequence composed of four symbols is $\log(4) \approx 1.39$, and the minimum entropy is zero. We see that the test sequence almost achieves the maximum entropy. As entropy can be interpreted as the amount of surprise or randomness, this result suggests a lack of predictable temporal features in the microstate sequence. However, we will show that the sequence does contain temporal structure, mainly in the form of periodicities in the range of the main EEG frequency bands.

Next, the entropy rate of the sequence is estimated from a set of joint Shannon entropies $H_k=H(\mathbf{X}_{n}^{(k)})$ for different history lengths $k$. The estimated entropy rate is the slope of the linear fit $k$ vs. $H_k$. The intercept is the so-called excess entropy, which is not further investigated here.

```python
h_rate, _ = excess_entropy_rate(x, n_maps, kmax=8, doplot=True)
h_mc = mc_entropy_rate(p_hat, T_hat)
print("\n\t\tEmpirical entropy rate h = {:.2f}".format(h_rate))
print("\t\tTheoretical MC entropy rate h = {:.2f}".format(h_mc))
```

## (7) Markov tests

We will first test the Markov properties of order 0, 1 and 2 for the EEG test file. For the details of the testing procedure, see the attached manuscript and the code documentation in the source file.

```python
alpha = 0.01
p_M0 = testMarkov0(x, n_maps, alpha)
p_M1 = testMarkov1(x, n_maps, alpha)
p_M2 = testMarkov2(x, n_maps, alpha)
p_vals_geo = geoTest(x, n_maps, 1000./fs, alpha)
```

    		ZERO-ORDER MARKOVIANITY:
    		p: 0.00e+00 | t: 59858.388 | df: 9.0

    		FIRST-ORDER MARKOVIANITY:
    		p: 1.64e-131 | t: 736.258 | df: 36.0

    		SECOND-ORDER MARKOVIANITY:
    		p: 1.33e-84 | t: 760.946 | df: 144.0

    		GEOMETRIC DISTRIBUTION of lifetimes:

    		Testing the distribution of symbol # 0
    		p: 8.50e-21 | t: 164.047 | df: 29.0
    		G-test (log-likelihood) p: 2.40e-08, g: 91.168, df: 29.0
    		G-test (Pearson Chi2) p: 3.83e-07, g: 83.258, df: 29.0

    		Testing the distribution of symbol # 1
    		p: 5.84e-14 | t: 123.501 | df: 28.0
    		G-test (log-likelihood) p: 9.17e-05, g: 64.943, df: 28.0
    		G-test (Pearson Chi2) p: 2.12e-04, g: 62.187, df: 28.0

    		Testing the distribution of symbol # 2
    		p: 1.15e-23 | t: 173.008 | df: 26.0
    		G-test (log-likelihood) p: 1.01e-09, g: 94.568, df: 26.0
    		G-test (Pearson Chi2) p: 6.40e-08, g: 83.316, df: 26.0

    		Testing the distribution of symbol # 3
    		p: 2.13e-49 | t: 304.976 | df: 26.0
    		G-test (log-likelihood) p: 6.41e-22, g: 163.645, df: 26.0
    		G-test (Pearson Chi2) p: 1.99e-19, g: 150.128, df: 26.0

The statistical Markovianity test show that our test sequence does not have the Markov property of the tested orders. This means that there are memory effects in the series which we will further explore.

## (8) Stationarity test

Next, we will test if the transition matrix $\hat{T}$ is time-stationary or if the matrix elements change significantly over time. You can provide a specific window length L for testing or use the default value of 5000 samples.

```python
try:
    L = int(raw_input("\t\tEnter block size: "))
except:
    L = 5000
p3 = conditionalHomogeneityTest(x, n_maps, L, alpha)
```

    		Enter block size:

    		CONDITIONAL HOMOGENEITY (three-way table):
    		Split data in r = 9 blocks of length 5000.
    		p: 3.37e-06 | t: 171.631 | df: 96.0

We find that the given transition matrix is not time-stationary.

## (9) Symmetry test

We will now test if the transition matrix is symmetric. In other words, we test if the transitions $A \rightarrow B$ and $B \rightarrow A$ occur with the same probability or not.

```python
p4 = symmetryTest(x, n_maps, alpha)
```

    		SYMMETRY:
    		p: 1.61e-82 | t: 396.469 | df: 6.0

The transition matrix of the test microstate sequence is not symmetric.

## (10) Markov surrogate example

To test our statistical procedures and to construct surrogate data sets from which confidence intervals for further information-theoretical quantities can be computed, we will synthesize first-order Markov chains with symbol distribution and transition matrix identical to the experimental EEG data set. This is possible because a first-order Markov chain is completely defined by an initial symbol distribution and a transition matrix for first-order (single time step) transitions.

```python
x_mc = surrogate_mc(p_hat, T_hat, n_maps, len(x))
p_surr = p_empirical(x_mc, n_maps)
T_surr = T_empirical(x_mc, n_maps)
print("\n\t\tSurrogate symbol distribution:\n")
for i in range(n_maps): print("\t\tp_{:d} = {:.3f}".format(i, p_surr[i]))
print( "\n\t\tSurrogate transition matrix:\n" )
print_matrix(T_surr)
```

    		Surrogate symbol distribution:

    		p_0 = 0.217
    		p_1 = 0.251
    		p_2 = 0.259
    		p_3 = 0.273

    		Surrogate transition matrix:

    		|0.767 0.098 0.056 0.079|
    		|0.064 0.773 0.104 0.059|
    		|0.060 0.060 0.787 0.093|
    		|0.069 0.074 0.062 0.795|

We observe that the symbol distribution and the transition matrix are equivalent, within some stochastic margin, to the experimental data. A quantitative analysis of Markov surrogate errors is contained in [2].

## (11) Markov tests for surrogate Markov data

It will now be shown that the Markov surrogates fulfill the expected properties, or vice versa, that our Markovianity and stationarity tests work well with our surrogate data. This means that we can analyze experimental data series to find memory effects beyond a first-order Markov model.

```python
p_M0_mc = testMarkov0(x_mc, n_maps, alpha)
p_M1_mc = testMarkov1(x_mc, n_maps, alpha)
p_M2_mc = testMarkov2(x_mc, n_maps, alpha)
p_vals_geo_mc = geoTest(x_mc, n_maps, 1000./fs, alpha)
```

    		ZERO-ORDER MARKOVIANITY:
    		p: 0.00e+00 | t: 59734.431 | df: 9.0

    		FIRST-ORDER MARKOVIANITY:
    		p: 2.01e-01 | t: 42.847 | df: 36.0

    		SECOND-ORDER MARKOVIANITY:
    		p: 2.64e-01 | t: 154.297 | df: 144.0

    		GEOMETRIC DISTRIBUTION of lifetimes:

    		Testing the distribution of symbol # 0
    		p: 8.01e-01 | t: 20.680 | df: 27.0
    		G-test (log-likelihood) p: 9.99e-01, g: 9.997, df: 27.0
    		G-test (Pearson Chi2) p: 1.00e+00, g: 8.853, df: 27.0

    		Testing the distribution of symbol # 1
    		p: 3.23e-01 | t: 40.379 | df: 37.0
    		G-test (log-likelihood) p: 9.86e-01, g: 20.632, df: 37.0
    		G-test (Pearson Chi2) p: 9.93e-01, g: 19.137, df: 37.0

    		Testing the distribution of symbol # 2
    		p: 2.51e-01 | t: 42.365 | df: 37.0
    		G-test (log-likelihood) p: 9.89e-01, g: 20.253, df: 37.0
    		G-test (Pearson Chi2) p: 9.94e-01, g: 18.783, df: 37.0

    		Testing the distribution of symbol # 3
    		p: 4.84e-04 | t: 59.410 | df: 28.0
    		G-test (log-likelihood) p: 4.57e-01, g: 28.132, df: 28.0
    		G-test (Pearson Chi2) p: 6.01e-01, g: 25.482, df: 28.0

The Markov surrogate clearly fulfills the expected theoretical properties. The zero-order Markov hypothesis is rejected while the first-order and second-order properties are accepted. The theoretically equivalent, but numerically different test for geometric lifetime distributions asserts the geometric lifetime distribution of all microstate maps.

## (12) Stationarity test for surrogate Markov data

We expect the Markov surrogate to be time-stationary as the same transition matrix has been used to generate the entire sequence. The test results show that indeed, the Markov surrogate is correctly classified as time-stationary. This means that we can identify non-stationarity in experimental data sets statistically.

```python
try:
    L = int(raw_input("\t\tEnter block size: "))
except:
    L = 5000
p3_mc = conditionalHomogeneityTest(x_mc, n_maps, L, alpha)
```

    		Enter block size:

    		CONDITIONAL HOMOGENEITY (three-way table):
    		Split data in r = 9 blocks of length 5000.
    		p: 3.42e-01 | t: 101.058 | df: 96.0

## (13) Symmetry test for surrogate Markov data

As the experimental EEG transtion matrix is used to generate the Markov surrogate, the surrogate's transition matrix is also asymmetric.

```python
p4_mc = symmetryTest(x_mc, n_maps, alpha)
```

    		SYMMETRY:
    		p: 8.39e-60 | t: 290.613 | df: 6.0

## (14) Time-lagged mutual information: empirical vs. Markov

Mutual information for time lag k, $I(k)$, is defined as the difference between the entropy (uncertainty) of the microstate sequence at time point $t+k$, and the conditional entropy at time point $t+k$ given the state at time $t$. Mutual information thus quantifies the uncertainty about the state at $t+k$ that is not represented by the previous time point $t$.

$$
\begin{align}
I(k) & = H(X_{t+k}) - H(X_{t+k} \mid X_k)
\end{align}
$$

As an analogy to the autocorrelation function (ACF) for metric time series, we will call this information-theoretic quantity the autoinformation function (AIF). Whilte the ACF uses linear correlation as a dependency measure for metric time series, the AIF uses mutual information to quantify dependencies in symbolic sequences.
We can now compute the time-lagged mutual information function of the microstate sequence and compare it to a set of Markov surrogates, from which a confidence interval under the first-order Markovian null hypothesis is constructed. We here use 10 Markov surrogates and compute the two-sided 99% percentile.

```python
l_max = 100
n_mc = 10
aif = mutinf(x, n_maps, l_max)
print("\n\t\tComputing n = {:d} Markov surrogates...\n".format(n_mc))
aif_array = mutinf_CI(p_hat, T_hat, len(x), alpha, n_mc, l_max)
pct = np.percentile(aif_array,[100.*alpha/2.,100.*(1-alpha/2.)],axis=0)
plt.ioff()
fig = plt.figure(1, figsize=(20,5))
fig.patch.set_facecolor('white')
t = np.arange(l_max)*1000./fs
plt.semilogy(t, aif, '-sk', linewidth=3, label='AIF (EEG)')
plt.semilogy(t, pct[0,:], '-k', linewidth=1)
plt.semilogy(t, pct[1,:], '-k', linewidth=1)
plt.fill_between(t, pct[0,:], pct[1,:], facecolor='gray', alpha=0.5, label='AIF (Markov)')
plt.xlabel("time lag [ms]", fontsize=20, fontweight="bold")
plt.ylabel("AIF [nats]", fontsize=20, fontweight="bold")
plt.legend(fontsize=22)
ax = plt.gca()
ax.set_xticklabels(ax.get_xticks(), fontsize=18, fontweight='normal')
ax.set_yticklabels(ax.get_yticks(), fontsize=18, fontweight='normal')
plt.tight_layout()
plt.show()
```

The figure generated by the code above shows the autoinformation function (AIF) along with Markovian confidence intervals. The AIF summarizes memory effects in the test microstate sequence as identified by mutual information. It is observed that the AIF of experimental data (black line and squares) shows clear periodicities and comparison with the Markovian confidence interval (gray-shaded area) shows that these peaks are statistically significant.
We can thus show that EEG microstates recur in a regular way. In other words, the microstate sequences passes through the graph of state transitions in a circular way. The transitions do not do so in a completely deterministic way, as any state transition is possible at any time, but we show that there are preferred closed paths along the state transition graph.

## (15) Summary and reporting results

The results can be summarized for scientific reporting as follows. Please note that *fixed numerical values are shown here*, i.e. the results of a specifc run of the algorithm. As clustering results vary slightly between runs, the exact numerical values may differ from the results obtained above.

1. We analyzed 192 seconds of resting state EEG sampled at 250 Hz. The 1-35 Hz band-pass filtered data set showed  21.51 GFP peaks per second and was clustered into 4 microstates, labelled A-D.

2. Static properties:
    * The microstate distribution was $p(A)=0.222$, $p(B)=0.240$, $p(C)=0.264$, and $p(D)=0.275$.
    * The four microstates explained 69% of the total data variance (GEV).
    * In information-theoretical terms, the four microstates produced a Shannon entropy of $H=1.38$ nats, where the upper bound for a 4-state process is $H_{max}=1.39$ nats. The empirical entropy rate was $h=0.73$ nats, and the expected value for the analogous first-order Markov process is $h=0.76$ nats.

3. Dynamic properties:
    * The transition probabilities between all pairs of microstates are summarized in the transition matrix T:
    \begin{equation*}
    T =  \begin{vmatrix}
    0.771 & 0.094 & 0.058 & 0.076 \\
	0.063 & 0.769 & 0.112 & 0.056 \\
	0.063 & 0.055 & 0.786 & 0.096 \\
	0.070 & 0.072 & 0.061 & 0.797 \\
    \end{vmatrix}
    \end{equation*}
    * The transition matrix T proved to be asymmetric ($p \lt 10^{-80}$).

4. Information-theoretical properties. Using a significance level of $\alpha = 0.01$, we found that:
    * The microstate sequence does not fulfill the Markov property of order 0, 1, or 2. We conclude that the microstate sequence has extended memory effects.
    * During the EEG recording, the transition matrix of the microstate sequence is not constant (it is non-stationary), when tested for a partition into 9 blocks and 5000 samples per block ($p=3.37 \times 10^{-6}$). This result should be taken as a precaution when using methods that require time stationarity (e.g. long-range memory interpretations of Hurst exponents).
    * Non-Markovianity and non-stationarity are at least partially explained by a periodic recurrence of the microstates, as shown by the time-lagged mutual information function (or autoinformation function, AIF). For time lags up to 400 msec., the EEG's AIF lies above the (first-order) Markov confidence interval and shows distinct peaks at multiples of approx. 50 msec (half the period of the underlying alpha rhythm).

### Code acceleration

If you ran the code above on your computer, you will have noticed that the AIF calculation determines the computation time of the procedure. The code can be accelerated using Cython, Numba or an external Julia function, for instance. To maximize portability and minimize the number of operating system dependencies, we did not include these accelerations in the code. However, the mentioned methods are implemented with little effort. Accelerated code can also be obtained from the author.
The eeg_microstates.py script can be run from the console using the -mc parameter that defines the number of Markov surrogates. The number of surrogates will define the width and smoothness of the confidence interval. Empirically, we suggest a minimum number of 50-100 surrogates.

## Conclusions

If you have arrived here, you should have completed the tutorial successfully. You should be familiar with the functionality provided by this software and have a good impression how to process larger data sets to obtain group statistics.


```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

```python
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

```python
from sympy import init_printing, Matrix, symbols, eye
from warnings import filterwarnings
```

```python
init_printing(use_latex = 'mathjax')
filterwarnings('ignore')
```

# Markov matrices and steady state
# Projections and Fourier series

## Markov matrix

* Consider the follow Markov matrix (transition matrix)

```python
A = Matrix([[0.1, 0.01, 0.3], [0.2, 0.99, 0.3], [0.7, 0, 0.4]])
A
```

$$\left[\begin{matrix}0.1 & 0.01 & 0.3\\0.2 & 0.99 & 0.3\\0.7 & 0 & 0.4\end{matrix}\right]$$

* All the column entries add to 1 (also true for powers of the matrix)
* All entries ≥ 0 (also true for powers of the matrix)
* There will be an eigenvalue of 1
* All other eigenvalues will have an absolute value of at most 1
* For difference equations we will have the following
$$ {u}_{k}={A}^{k}{u}_{0}={c}_{1}{\lambda}_{1}^{k}{x}_{1}+{c}_{1}{\lambda}_{2}^{k}{x}_{2}+{\dots} $$
* Thus for the eigenvalues other than 1, successive powers will lead to those terms approaching zero and a steady state being reached by the term with the eigenvalue of 1

```python
A.eigenvals()
```

$$\begin{Bmatrix}1 : 1, & \frac{49}{200} + \frac{3 \sqrt{1009}}{200} : 1, & - \frac{3 \sqrt{1009}}{200} + \frac{49}{200} : 1\end{Bmatrix}$$

* All the components of the eigenvector of the eigenvalue 1 is positive

```python
A.eigenvects()
```

$$\begin{bmatrix}\begin{pmatrix}1.0, & 1, & \begin{bmatrix}\left[\begin{matrix}0.857142857142857\\47.1428571428571\\1.0\end{matrix}\right]\end{bmatrix}\end{pmatrix}, & \begin{pmatrix}0.721471405228058, & 1, & \begin{bmatrix}\left[\begin{matrix}0.459244864611511\\-1.45924486461151\\1.0\end{matrix}\right]\end{bmatrix}\end{pmatrix}, & \begin{pmatrix}-0.231471405228058, & 1, & \begin{bmatrix}\left[\begin{matrix}-0.902102007468654\\-0.0978979925313461\\1.0\end{matrix}\right]\end{bmatrix}\end{pmatrix}\end{bmatrix}$$

* If the matrix A - 1λ is singular, then the eigenvalue is 1

```python
A - eye(3) # A minus the identity matrix
```

$$\left[\begin{matrix}-0.9 & 0.01 & 0.3\\0.2 & -0.01 & 0.3\\0.7 & 0 & -0.6\end{matrix}\right]$$

```python
(A - eye(3)).det() # A computer peculiarity to inidcate 0
```

$$-3.03576608295941 \cdot 10^{-18}$$

* The sum of the entries in each column is now zero
* We would like a proof involving this (sum of column entries equal zero) as an assumption to give a singular matrix, without having to calculate the determinant
* It is easy to see that the rows are linearly dependent and that would give proof of a singular matrix
* Look also at the nullspace of AT

```python
A_1t = A - eye(3)
(A_1t.transpose()).nullspace()
```

$$\begin{bmatrix}\end{bmatrix}$$

```python
(A.transpose()).nullspace()
```

$$\begin{bmatrix}\end{bmatrix}$$

```python
A.nullspace()
```

$$\begin{bmatrix}\end{bmatrix}$$

```python
(A - eye(3)).rref()
```

$$\begin{pmatrix}\left[\begin{matrix}1.0 & 0 & 0\\0 & 1.0 & 0\\0 & 0 & 1.0\end{matrix}\right], & \begin{bmatrix}0, & 1, & 2\end{bmatrix}\end{pmatrix}$$

* Note how the eigenvalues of A and AT are the same

```python
A.eigenvals() == A.transpose().eigenvals()
```

    True

* The proof of this lies in the fact that we calculate the eigenvalue(s) by the following equation
$$ \left|{A}-{\lambda}{I}\right|={0} $$
* The eigenvalue(s) must also solve this equation
$$ \left|{A}^{T}-{\lambda}{I}\right|={0} $$
* Since λI=λIT and both of these equations eqal the same value (0), the eigenvalue(s) must be equal

### Example of a Markov matrix

* Consider the population of two (isolated) states (*c* and *m*)
* Over time there is movement between the two (no loss or entry from outside the system)
* We consider the following system fo movement (difference equation)
$$ {u}_{k+1}={A}{u}_{k} $$
* We can create the following matrix equation
$$ \begin{bmatrix} { u }_{ c } \\ { u }_{ m } \end{bmatrix}_{k+1}=\begin{bmatrix} 0.9 & 0.2 \\ 0.1 & 0.8 \end{bmatrix}\begin{bmatrix} { u }_{ c } \\ { u }_{ m } \end{bmatrix}_{k} $$

```python
A = Matrix([[0.9, 0.2], [0.1, 0.8]])
A
```

$$\left[\begin{matrix}0.9 & 0.2\\0.1 & 0.8\end{matrix}\right]$$

* Let's start at time, *t*=0 (or in our case *k*=0), with the whole population in state *m*
* Because the system is closed, the total population stays at 1000

```python
u0 = Matrix([0, 1000])
```

* After 1 time period *k*+1 will be given by the following

```python
u1 = A * u0
u1
```

$$\left[\begin{matrix}200.0\\800.0\end{matrix}\right]$$

```python
u2 = A * u1
u2
```

$$\left[\begin{matrix}340.0\\660.0\end{matrix}\right]$$

* Where will we end up?
* Let's look at the eigenvalues
* One will be one and the other must be the trace minus 1 = 0.7

```python
A.eigenvals()
```

$$\begin{Bmatrix}\frac{7}{10} : 1, & 1 : 1\end{Bmatrix}$$

```python
A.eigenvects()
```

$$\begin{bmatrix}\begin{pmatrix}0.7, & 1, & \begin{bmatrix}\left[\begin{matrix}-1.0\\1.0\end{matrix}\right]\end{bmatrix}\end{pmatrix}, & \begin{pmatrix}1.0, & 1, & \begin{bmatrix}\left[\begin{matrix}2.0\\1.0\end{matrix}\right]\end{bmatrix}\end{pmatrix}\end{bmatrix}$$

* When can look at what happens after a hundred steps using a small *while* loop

```python
u = Matrix([0, 1000])
i = 0

while i < 101:
    u = A * u
    i += 1
u
```

$$\left[\begin{matrix}666.666666666668\\333.333333333334\end{matrix}\right]$$

* It seems that we are at about ⅔ vs ⅓
* It looks suspiciously like the eigenvector of the eigenvalue 1

* Remember the equation for the difference equation we used above?
$$ {u}_{k}={A}^{k}{u}_{0}={c}_{1}{\lambda}_{1}^{k}{x}_{1}+{c}_{1}{\lambda}_{2}^{k}{x}_{2}+{\dots} $$
* We only have two terms, so this will become somewhat shortened

$$ {u}_{k}={A}^{k}{u}_{0}={c}_{1}{\lambda}_{1}^{k}{x}_{1}+{c}_{1}{\lambda}_{2}^{k}{x}_{2} \\ {u}_{k}={c}_{1}{1}^{k}\begin{bmatrix}2\\1\end{bmatrix}+{c}_{1}{\frac{7}{10}}^{k}{\begin{bmatrix}-1\\1\end{bmatrix}} \\ {u}_{0}={c}_{1}{1}^{0}\begin{bmatrix}2\\1\end{bmatrix}+{c}_{1}{\left({\frac{7}{10}}\right)}^{0}{\begin{bmatrix}-1\\1\end{bmatrix}} \\ {u}_{0}={c}_{1}\begin{bmatrix}2\\1\end{bmatrix}+{c}_{1}{\begin{bmatrix}-1\\1\end{bmatrix}}=\begin{bmatrix}0\\1000\end{bmatrix} \\  $$

```python
result = Matrix([[2, -1, 0], [1, 1, 1000]])
result.rref()
```

$$\begin{pmatrix}\left[\begin{matrix}1 & 0 & \frac{1000}{3}\\0 & 1 & \frac{2000}{3}\end{matrix}\right], & \begin{bmatrix}0, & 1\end{bmatrix}\end{pmatrix}$$

* So we have *c*1=1000÷3 and *c*2=2000÷3

* This results in the final solution
$$ {u}_{k}=\frac{1000}{3}{1}^{k}\begin{bmatrix}2\\1\end{bmatrix}+\frac{2000}{3}{\left({\frac{7}{10}}\right)}^{k}{\begin{bmatrix}-1\\1\end{bmatrix}} $$
* Now we can work out the solution for any time step *k* and even see what happens at time approaches infinity
* We note that the second expression disappears as *k* approaches infinity, which represents the steady state

## Projections and Fourier series

* Consider projections with **orthonormal** basis *q*1, *q*2, ..., *q*n
* Any vector **v** can be expressed (expanded) as an combination of this basis
$$ {v}={x}_{1}{q}_{1}+{x}_{2}{q}_{2}+\dots+{x}_{n}{q}_{n} $$

* Because the basis *q*i were chosen to be orthogonal taking the dot product of *q*1 on both sides will cancel out all *q*≠1 factors
$$ {q}_{1}^{T}{v}={x}_{1}{q}_{1}^{T}{q}_{1}+{0}+{0}+\dots \\ {q}_{1}^{T}{v}={x}_{1} $$

* We can see this clearly in matrix form below
$$ \begin{bmatrix} \vdots  & \vdots  & \vdots  \\ { q }_{ 1 } & \cdots  & { q }_{ n } \\ \vdots  & \vdots  & \vdots  \end{bmatrix}\begin{bmatrix} { x }_{ 1 } \\ \vdots  \\ { x }_{ n } \end{bmatrix}=v\\ Qx=v\\ x={ Q }^{ -1 }v\\ \because \quad { Q }^{ -1 }={ Q }^{ T }\\ x={ Q }^{ T }v $$
* The first component of **x**, *x*1 is the the first row of QT times **v**
* The first row of QT is just q1, with a bunch of zeros and just ends as we had above
$$ {q}_{1}^{T}{v}={x}_{1} $$

* The key here was to choose orthonormal basis vectors
* Now for how this relates to Fourier series
* We might want to write a function as an **infinite** series of expressions
$$ f\left( x \right)={ a }_{ 0 }+{ a }_{ 1 }\cos { x } +{ b }_{ 1 }\sin { x } +{ a }_{ 2 }\cos { 2x } +{ b }_{ 2 }\sin { 2x } +\dots  $$
* The idea here is that there is still something orthogonal in each of these expressions (sine and cosine)
* Joseph Fourier realized that he could work in function space
* The vectors are now functions in an infinitely large space
* The basis vectors are *cos*(*x*), *sin*(*x*), *cos*(2*x*)...

* What is the inner (dot) product of functions that make them orthogonal?
* For vectors is was the following
$$ {v}^{T}{w}={v}_{1}{w}_{1}+{v}_{2}{w}_{2}+\dots+{v}_{n}{w}_{n} $$
* For functions *f* and *g* the analogue is to multiply them and the analogue of all the addition is integration
$$ { f }^{ T }g=\int { f\left( x \right)g\left( x \right) } dx $$
* Now we need to know the lower and upper limit of integration
* We note that the sine and cosine functions are periodic and repeat every 2π (these are periodic functions)
$$ \because \quad f\left( x \right)=f\left( x+2\pi  \right)\\ { f }^{ T }g=\int _{ 0 }^{ 2\pi  }{ f\left( x \right) g\left( x \right)  } dx $$
* Just like the inner product of *pairs* gave zero because of orthogonality we have the same here
$$ \int _{ 0 }^{ 2\pi  }{ \sin { x } \cos { x }  } dx\\ u\left( x \right) =\sin { x } \\ \frac { du }{ dx } =\cos { x } \\ \cos { x } dx=du\\ { u }_{ 2 }\left( 2\pi  \right) =\sin { 2\pi  } =0\\ { u }_{ 1 }\left( 0 \right) =\sin { 0 } =0\\ \int _{ 0 }^{ 0 }{ u } du=0 $$
* With some trigonometric identities we can show the same for all the other *pairs*

* A Fourier series is then an expression of a function expanded on this orthonormal basis pairs

* How do we get the coefficients then?
* Same as above, i.e. taking the inner product of both sides with *cos*(*x*)
$$ \int _{ 0 }^{ 2\pi  }{ f\left( x \right) \cos { x }  } dx={ a }_{ 1 }\int _{ 0 }^{ 2\pi  }{ { \left( \cos { x }  \right)  }^{ 2 } } dx+0+0+\dots =\pi \\ { a }_{ 1 }=\frac { 1 }{ \pi  } \int _{ 0 }^{ 2\pi  }{ f\left( x \right) \cos { x }  } dx $$

## Example problem (Markov matrixes)

### Example problem 1

* A particle jumps between positions A and B with the following probabilities
    * A to A (stays in A) probability is 0.6
    * A to B probability is 0.4 (so all states from A add to 1.0)
    * B to B probability is 0.8
    * B to A probability is 0.2
* If the particle starts in position A, what is the probability that it will be at A and B after the first step, *n*-steps, and ∞-steps

#### Solution

```python
x_vect = Matrix([1, 0])
A = Matrix([[0.6, 0.2], [0.4, 0.8]]) # Look carefully at what goes where
A, x_vect # Displaying the two matrices
```

$$\begin{pmatrix}\left[\begin{matrix}0.6 & 0.2\\0.4 & 0.8\end{matrix}\right], & \left[\begin{matrix}1\\0\end{matrix}\right]\end{pmatrix}$$

* After a single step we will have the following

```python
A * x_vect
```

$$\left[\begin{matrix}0.6\\0.4\end{matrix}\right]$$

* A probability of 0.6 of being in position A and a probability of 0.4 of being in position B

* Now look at the following trend
$$ { p }_{ 1 }=A{ p }_{ 0 }\\ { p }_{ 2 }=A{ p }_{ 1 }=A\left( { Ap }_{ 0 } \right) ={ A }^{ 2 }{ p }_{ 0 }\\ { p }_{ 3 }={ A }^{ 3 }{ p }_{ 0 }\\ \vdots \\ { p }_{ n }={ A }^{ n }{ p }_{ 0 } $$

* To take the power of a matrix in python is easy, but let's remind ourselves that we need to wotk with eignevalues and eigenvectors
* Because all the column entries add to 1, we are dealing with a Markov matrix
* One of the eigenvalues will be 1, the trace is 1.4, therefor the other eigenvalue is 0.4
* Remember diagonalization?

```python
S, D = A.diagonalize()
S, D
```

$$\begin{pmatrix}\left[\begin{matrix}-1.0 & 1.0\\1.0 & 2.0\end{matrix}\right], & \left[\begin{matrix}0.4 & 0\\0 & 1.0\end{matrix}\right]\end{pmatrix}$$

* Note the matrix of eigenvalues (diagonal) *D*
* Note the matrix of eigenvectors
* So for the eigenvalue of 1 we have the following eigenvector (steady state)
$$ \begin{bmatrix}1\\2\end{bmatrix} $$
* For the other eigenvalue we have the following eigenvector (decay)
$$ \begin{bmatrix}-1\\0\end{bmatrix} $$

$$ {u}_{k}={A}^{k}{u}_{0}={c}_{1}{\lambda}_{1}^{k}{x}_{1}+{c}_{1}{\lambda}_{2}^{k}{x}_{2} \\ {u}_{k}={c}_{1}{1}^{k}\begin{bmatrix}1\\2\end{bmatrix}+{c}_{1}{\frac{4}{10}}^{k}{\begin{bmatrix}-1\\1\end{bmatrix}} \\ {u}_{0}={c}_{1}{1}^{0}\begin{bmatrix}1\\2\end{bmatrix}+{c}_{1}{\left({\frac{4}{10}}\right)}^{0}{\begin{bmatrix}-1\\1\end{bmatrix}} \\ {u}_{0}={c}_{1}\begin{bmatrix}1\\2\end{bmatrix}+{c}_{1}{\begin{bmatrix}-1\\1\end{bmatrix}}=\begin{bmatrix}1\\0\end{bmatrix} \\  $$

* Solving for the constants we can create an augmented matrix

```python
u = Matrix([[1, -1, 1], [2, 1, 0]])
u.rref()
```

$$\begin{pmatrix}\left[\begin{matrix}1 & 0 & \frac{1}{3}\\0 & 1 & - \frac{2}{3}\end{matrix}\right], & \begin{bmatrix}0, & 1\end{bmatrix}\end{pmatrix}$$

* This gives us the following solution
$$ {u}_{n}=\frac{1}{3}\left({1}^{n}\right)\begin{bmatrix}1\\2\end{bmatrix}+\frac{-2}{3}{\left({\frac{4}{10}}\right)}^{n}\begin{bmatrix}-1\\1\end{bmatrix} \\  { p\left( A \right)  }_{ n }=\frac { 1 }{ 3 } +\frac { -2 }{ 3 } { \left( \frac { 4 }{ 10 }  \right)  }^{ n }\left( -1 \right) \\ { p\left( B \right)  }_{ n }=\frac { 1 }{ 3 } \left( 2 \right) +\frac { -2 }{ 3 } { \left( \frac { 4 }{ 10 }  \right)  }^{ n }\left( 1 \right) $$

* So at infinity it will be at position A with a probability of ⅓ and at position B with a probability of ⅔

```python
(A ** 1000000) * x_vect # Just to show how easy it is just to take the power of A in python
```

$$\left[\begin{matrix}0.333333333362792\\0.666666666725585\end{matrix}\right]$$

```python

```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Lecture_24_Markov_matrices_Projections_and_Fourier_series.ipynb