# Patch Discord
## The Aleatoric MIDI Router

### Introduction

The aletoric MIDI router is a device that serves the purpose of erasing the labels from the control knobs of a synthesizer.

In a system comprising a controller and a synthesizer, the router can be placed in the middle, receiving a number of control signals and outputting altered control signals. As can be guessed from the name, the router randomly permutes signal destinations. However, it does not simply route its inputs to different channels. This would have resulted in simply renaming the knobs, and not yet erasing their labels. Instead, the router performs a permutation such that *all* the input channels affect *all* the output channels. This way, a musician using a controller cannot know what the result of a turn of a knob would be, and thus cannot apply his or her knowledge and experience, but has to completely rely on listening instead. Furthermore, to exclude the possibility of the musician getting accustomed to the routing with time and mentally mapping the controls to their new destinations, the internal state of the router gradually changes, so that the routing is never static. The output changes even when the musician does not provide any input. This side effect serves as an invitation for the musician to actively work even when just maintaining a certain timbre.

The aleatoric router is clearly not a utility device, but rather a catalyst of a creative practice.

### Notes on implementation

While the general description above does not imply a single implementation, this document develops one possible implementation of the router.

The router can be realised as a MIDI effect plugin for DAWs, as well as a standalone hardware device. Moreover, an analog variant is also concievable, replacing MIDI with CV signals. However, the current implementation focuses on MIDI.

### High-level structure

In a setup with a separate controller and a sound engine, the router fits in the middle:

```
+---+          +---+           +---+
| C | == M ==> | R |  == M ==> | S |
+---+          +---+           +---+

C - controller
M - MIDI bus
R - router
S - synth
```

---

The internal structure of the router is demonstated by the figure below:

```
     ⎡ ⎤    ⎡   ⎤    ⎡ ⎤
M ==>⎢I⎢ -> ⎢ P ⎢ -> ⎢O⎢ ==> M
     ⎣ ⎦    ⎣   ⎦    ⎣ ⎦
                ^
                ^
                G

M - MIDI bus
I - input accumulator
P - permutation
G - internal state generator
O - output accumulator
```

---

The principal operation performed by the router is a permutation of an input vector (permutation here is not to be understood in its combinatorial meaning). To adapt to the event-driven model of the MIDI protocol, input and output accumulators are employed. The input accumulator receives MIDI CC events and outputs a vector of the latest values on each of the preconfigured channels. The output accumulator perfoms the reverse operation: it receives the result of the permutation and emits MIDI CC events on channels on which the value has changed.

One cycle of the router's operation consists of:
1. freezing the input vector
2. performing a permutation
3. sending to the bus the difference between the permutation result and the previous cycle's output

As has been already mentioned, the permutation is not static. The purpose of the internal state generator is to periodically generate the permutation parameters. The state generator is not clocked at the same rate as the main cycle.

The clock rate of the main cycle should be approximately the rate at which the knobs are scanned by the controller, so that CC events are not missed, otherwise the user will experience unresponsiveness.

The clock rate of the state generator should be substantially slower, otherwise something akin to a frantic LFO on all the configured channels will be heard. In general, the state change rate should be a user-controllable parameter.

### Permutation

The main operation of the router is the permutation of an input vector. The term permutation here is not used in its combinatorial meaning, but is defined below instead.

First of all, it should be pointed out that all the values on the MIDI bus are taken to be integers in the range $[0; 127]$. The input and the output of the permutation are vectors of values in this domain.

The term channel is used below to signify a MIDI CC number. In more conventional MIDI terminology, the router operates on a single MIDI channel, but on multiple controllers (aka CC).

Before the permutation is derived, its desired characteristics should be clarified.
1. Number of input and output channels need not necessarily be equal.
2. All the input channels affect all the output channels to a certain degree.
3. The mapping should be continuous (in the discreet approximation, as we are operating in integers), so that close input vectors map to close output vectors.
4. The mapping should **not** be intuitively understandable by the musician. This implies that a permutation that maps all-zero input to all-zero output and/or all-maximum input to all-maximum output is unsatisfactory.
5. The output vector should not overflow.
6. The full range of each output channel should be covered by the input space. In other words, for each output channel for each possible value there exists at least one input vector that maps to an output vector that contains this value.
7. The state of the permutation should be gradually variable. This means that the permutation is parameterised, and changing the permutation parameter results in smooth changes in the output given the same input.

It should also be noted that the following properties are not specifically aimed for:
- Covering the entire output space is not required. Any specific output vector it is not guaranteed to be reachable.
- Unique mapping is not required. For any reachable output vector there might exist more than one input vector that maps to it.

All these requirements do not describe a single possible permutation. In the context of the musical device, a precise mathematical destination is probably not relevant. One permutation that fulfills the requirements is derived below, using some basic linear algebra and some intuitions.

To satisfy the first requirement, let's immediately consider the general case of $n$ input channels and $m$ output channels. The task is now to find a mapping from $n$ dimensional vector space to $m$ dimensional vector space.

\begin{equation}
\begin{bmatrix}i_{1} \\ i_{2} \\ ... \\ i_{n}\end{bmatrix}
\mapsto
\begin{bmatrix}o_{1} \\ o_{2} \\ ... \\ o_{m}\end{bmatrix}
\end{equation}

We begin with a linear mapping and introduce a matrix $P$ with $m$ rows and $n$ columns.  This satisfies the second requirement, as the element $p_{mn}$ of the matrix is a weigth with which $n$-th input contributes to $m$-th output.

\begin{equation}
\begin{bmatrix}p_{11} & p_{12} & ... & p_{1n} \\ p_{21} & ... \\ ... & & ... \\ p_{m1} & & & p_{mn}\end{bmatrix}
\times
\begin{bmatrix}i_{1} \\ i_{2} \\ ... \\ i_{n}\end{bmatrix}=
\begin{bmatrix}o_{1} \\ o_{2} \\ ... \\ o_{m}\end{bmatrix}
\end{equation}

Linear mapping also satisfies the third requirement: there are no abrupt jumps in output.

However, the fourth requirement, namely that the all-zero input should not map to all-zero output renders a linear mapping unsatisfactory. The fourth requirement calls for a translation. Let's introduce a translation vector that is added to the input vector before the multiplication by the matrix.

\begin{equation}
\begin{bmatrix}p_{11} & p_{12} & ... & p_{1n} \\ p_{21} & ... \\ ... & & ... \\ p_{m1} & & & p_{mn}\end{bmatrix}
\times
\left(
\begin{bmatrix}i_{1} \\ i_{2} \\ ... \\ i_{n}\end{bmatrix}
+
\begin{bmatrix}t_{1} \\ t_{2} \\ ... \\ t_{n}\end{bmatrix}
\right)=
\begin{bmatrix}o_{1} \\ o_{2} \\ ... \\ o_{m}\end{bmatrix}
\end{equation}

The fifth requirement is that the output doesn't overflow. Let's consider each output channel separately. The value of the $k$-th output channel is defined by the input vector and the $k$-th row of the matrix.

\begin{equation}
o_{k} = p_{k1} (i_{1} + t_{k}) + p_{k2} (i_{2} + t_{k}) + ... + p_{kn} (i_{n} + t_{k})
\end{equation}

By carefully picking the $p$ coeffiecients, it can be ensured that the output doesn't overflow. For example, if we ignore the translation, we can achieve that by choosing coefficients to be in the range of $[0; 1]$ in such a way that the sum of the row is $1$. Then each coefficient is the percentage of contribution of the corresponding input.

This is a viable solution. However, in this case  every input channel affects any output channel only to a limited degree. No input channel can reach the full range of an output channel.

It appears that in order to be more musically interesting, the control should be more redundant. Some input channels should be able to reach the full range of output, and some other input channels should be able to reverse the effect of the former channels.

So the internal calculations are performed in a broader domain. The translation and multiplication results are allowed to overflow the MIDI range. The multiplication product is mapped back to the domain afterwards.

\begin{equation}
wrap
\left(
\begin{bmatrix}p_{11} & p_{12} & ... & p_{1n} \\ p_{21} & ... \\ ... & & ... \\ p_{m1} & & & p_{mn}\end{bmatrix}
\times
\left(
\begin{bmatrix}i_{1} \\ i_{2} \\ ... \\ i_{n}\end{bmatrix}
+
\begin{bmatrix}t_{1} \\ t_{2} \\ ... \\ t_{n}\end{bmatrix}
\right)
\right)=
\begin{bmatrix}o_{1} \\ o_{2} \\ ... \\ o_{m}\end{bmatrix}
\end{equation}

We need to employ such a way of wrapping that respects the continuity requirement. Wrapping using a modulo operation does not fulfill this requirment, as it involves a jump from one boundary to the other:

```python
import matplotlib.pyplot as plt
import numpy as np

x = np.arange(-300, 300, 5)
y = x % 127

fig, ax = plt.subplots(figsize=(15, 2))
ax.plot(x, y, '.')
plt.show()
```

Instead of making a jump, the value could bounce from the boundary and start moving in the opposite direction:

```python
def wrap(x, limit):
    '''The lower limit is assumed to be 0.'''
    x %= 2 * limit
    if x > limit:
        x = 2 * limit - x
    return x

y = np.array([wrap(xi, 127) for xi in x])

fig, ax = plt.subplots(figsize=(15, 2))
ax.plot(x, y, '.')
plt.show()
```

Wrapping is applied to each element of the output vector.

With wrapping, the elements of the matrix can be picked almost arbitrarily.

However, it is desirable that input covers the output range more than once. The value of a coefficient can be understood as the number of times an input channel covers an output channel (ignoring translation). Large values of coefficients decrease the resolution of the input controls, which is to be avoided. Intuitively, the values of the coefficients should remain approximately  in the range $[-2, 2]$, where negative coefficients mean inversion of the input channel.

Also by intuition, the elements of the translation vector are in the MIDI domain.

The remaining requirement is that the permutation should be parameterized. This is detailed in the next section.

### Internal State

The internal state of the router comprises the translation vector and the matrix. The state should be gradually variable, so that varying the state does not result in jumps in the output.

The translation vector can be randomly chosen at the inititialization time and fixed from then on.

The matrix should be periodically regenerated, and the next matrix should be closely related to the previous matrix. The elements of the matrix should be in the range $[-2; 2]$. A parameterized function that always yields values within a certain range is necessary.

A simple example of such a function is the sine function:
\begin{equation}
x = a \cdot sin{(f t + \phi)} + c
\end{equation}

Since the range is symmetrical with respect to $0$, the constant $c$ is $0$.

$a$ is the scaling factor controlling the output range. It should be set to $2$.

Each row of coefficients can be picked as samples from a section of a sine wave one period long.
The equation for the elements of a row is:

\begin{equation}
p_{i} = 2 \cdot sin{((i - 1) \frac{2 \pi}{n} + \phi)}
\end{equation}

$n$ is the number of coefficients, which is the number of inputs. $i$ is the index of the element in the row, in the range $[1; n]$.

This way there is one peak and one valley per row, that is per output channel. This translates to one input channel being the dominant control, and another input channel being its opposite.

The phase $\phi$ is the variable parameter of the permutation. As it changes, the row of coefficients slightly shifts along the sine wave, but never changes abruptly. Initial phases per row can be randomly chosen at initialization time, just like the translation vector.

$\phi$ is gradually incremented. The rate of change of $\phi$ is the rate of the state generator mentioned in the section on high-level structure. The precise value can be user-controlled, but the full cycle (change of $\phi$ by $2\pi$ radians) should take several minutes, so that the fact that the state is periodic is not obvious to the user.

Here is a demonstration of sampling of the coefficients:

```python
import matplotlib.pyplot as plt
import numpy as np

PHI = .333
N = 8 # number of coefficients

def coefficient(index):
    return 2 * np.sin((index * 2 * np.pi) / (N - 1) + PHI)

indices = [i for i in range(N)]
samples = np.array([coefficient(i) for i in indices])

# detailed function for demonstration
ref_x = np.arange(0, N - 0.9, 0.1)
ref_y = np.array([coefficient(i) for i in ref_x])

fig, ax = plt.subplots(figsize=(15, 2))
ax.plot(indices, samples, 'o', ref_x, ref_y, '-')
plt.show()
```

There is one more consideration. With the coefficients sampled from a sine wave, the user might notice a spatial pattern of the dominant control and the opposite control being $n/2$ knobs apart if there are $n$ inputs. In order to conceal this spatial pattern, one more step is added. After the coefficients are sampled, they are shuffled, so that they appear in the matrix in a different order.

The shuffling can be represented with a shuffling matrix $S$ that has the same dimensions as the permutation matrix. The element of each row of the shuffling matrix $s_{i}$ represents the column in the permutation matrix to which the $i$-th sample from the sine wave will go. So the rows of the permutation matrix are shufflings of numbers from 1 to $n$, where $n$ is the number of coefficients.

```python
import random

shuffling = random.sample(indices, len(indices))

coefficients = [0 for i in range(N)]
for i in indices:
    coefficients[i] = samples[shuffling[i]]

print("shuffling:")
print("\n".join(["from {0} to {1}".format(i, v) for i, v in enumerate(shuffling)]))
print("\nsamples:")
print(" | ".join(["{:.3f}".format(i) for i in samples]))
print("\ncoefficients:")
print(" | ".join(["{:.3f}".format(i) for i in coefficients]))
```

    shuffling:
    from 0 to 7
    from 1 to 0
    from 2 to 1
    from 3 to 5
    from 4 to 4
    from 5 to 3
    from 6 to 2
    from 7 to 6

    samples:
    0.654 | 1.885 | 1.697 | 0.231 | -1.409 | -1.988 | -1.070 | 0.654

    coefficients:
    0.654 | 0.654 | 1.885 | -1.988 | -1.409 | 0.231 | 1.697 | -1.070

Finally, the full state of the router is represented by:
- translation vector
- initial phases vector
- shuffling matrix
- current phase shift

All of these except for the phase shift are fixed at the initialization stage.
The phase shift is incremented at state change rate. The permutation matrix is then regenerated with the new phase shift.


# 3. Imagined movement

In this tutorial we will look at imagined movement. Our movement is controlled in the motor cortex where there is an increased level of mu activity (8–12 Hz) when we perform movements. This is accompanied by a reduction of this mu activity in specific regions that deal with the limb that is currently moving. This decrease is called Event Related Desynchronization (ERD). By measuring the amount of mu activity at different locations on the motor cortex, we can determine which limb the subject is moving. Through mirror neurons, this effect also occurs when the subject is not actually moving his limbs, but merely imagining it.

## Credits
The CSP code was originally written by Boris Reuderink of the Donders
Institute for Brain, Cognition and Behavior. It is part of his Python EEG
toolbox: https://github.com/breuderink/eegtools

Inspiration for this tutorial also came from the excellent code example
given in the book chapter:

Arnaud Delorme, Christian Kothe, Andrey Vankov, Nima Bigdely-Shamlo,
Robert Oostenveld, Thorsten Zander, and Scott Makeig. MATLAB-Based Tools
for BCI Research, In _(B+H)CI: The Human in Brain-Computer Interfaces and
the Brain in Human-Computer Interaction._ Desney S. Tan and Anton Nijholt
(eds.), 2009, 241-259, http://dx.doi.org/10.1007/978-1-84996-272-8

## About the data
The dataset for this tutorial is provided by the fourth BCI competition.
Before you continue with this tutorial, please, go to http://www.bbci.de/competition/iv/#download
and fill in your name and email address.
You don't need to actually download the data, since it is installed in the virtual server already, but this way, the wonderful organizers of the competition get notified that their data is still being used and by whom.

[Description of the data](http://bbci.de/competition/iv/desc_1.html)

```python
%pylab inline
```

    Populating the interactive namespace from numpy and matplotlib

Executing the code cell below will load the data:

```python
import numpy as np
import scipy.io

m = scipy.io.loadmat('data/BCICIV_calib_ds1d.mat', struct_as_record=True)

# SciPy.io.loadmat does not deal well with Matlab structures, resulting in lots of
# extra dimensions in the arrays. This makes the code a bit more cluttered

sample_rate = m['nfo']['fs'][0][0][0][0]
EEG = m['cnt'].T
nchannels, nsamples = EEG.shape

channel_names = [s[0] for s in m['nfo']['clab'][0][0][0]]
event_onsets = m['mrk'][0][0][0]
event_codes = m['mrk'][0][0][1]
labels = np.zeros((1, nsamples), int)
labels[0, event_onsets] = event_codes

cl_lab = [s[0] for s in m['nfo']['classes'][0][0][0]]
cl1 = cl_lab[0]
cl2 = cl_lab[1]
nclasses = len(cl_lab)
nevents = len(event_onsets)
```

Now we have the data in the following python variables:

```python
# Print some information
print('Shape of EEG:', EEG.shape)
print('Sample rate:', sample_rate)
print('Number of channels:', nchannels)
print('Channel names:', channel_names)
print('Number of events:', len(event_onsets))
print('Event codes:', np.unique(event_codes))
print('Class labels:', cl_lab)
print('Number of classes:', nclasses)
```

    Shape of EEG: (59, 190473)
    Sample rate: 100
    Number of channels: 59
    Channel names: ['AF3', 'AF4', 'F5', 'F3', 'F1', 'Fz', 'F2', 'F4', 'F6', 'FC5', 'FC3', 'FC1', 'FCz', 'FC2', 'FC4', 'FC6', 'CFC7', 'CFC5', 'CFC3', 'CFC1', 'CFC2', 'CFC4', 'CFC6', 'CFC8', 'T7', 'C5', 'C3', 'C1', 'Cz', 'C2', 'C4', 'C6', 'T8', 'CCP7', 'CCP5', 'CCP3', 'CCP1', 'CCP2', 'CCP4', 'CCP6', 'CCP8', 'CP5', 'CP3', 'CP1', 'CPz', 'CP2', 'CP4', 'CP6', 'P5', 'P3', 'P1', 'Pz', 'P2', 'P4', 'P6', 'PO1', 'PO2', 'O1', 'O2']
    Number of events: 1
    Event codes: [-1  1]
    Class labels: ['left', 'right']
    Number of classes: 2

This is a large recording: 59 electrodes where used, spread across the entire scalp. The subject was given a cue and then imagined either right hand movement or the movement of his feet. As can be seen from the [Homunculus](http://en.wikipedia.org/wiki/Cortical_homunculus), foot movement is controlled at the center of the motor cortex (which makes it hard to distinguish left from right foot), while hand movement is controlled more lateral.

## Plotting the data

The code below cuts trials for the two classes and should look familiar if you've completed the previous tutorials. Trials are cut in the interval [0.5–2.5 s] after the onset of the cue.

```python
# Dictionary to store the trials in, each class gets an entry
trials = {}

# The time window (in samples) to extract for each trial, here 0.5 -- 2.5 seconds
win = np.arange(int(0.5*sample_rate), int(2.5*sample_rate))

# Length of the time window
nsamples = len(win)

# Loop over the classes (right, foot)
for cl, code in zip(cl_lab, np.unique(event_codes)):

    # Extract the onsets for the class
    cl_onsets = event_onsets[event_codes == code]

    # Allocate memory for the trials
    trials[cl] = np.zeros((nchannels, nsamples, len(cl_onsets)))

    # Extract each trial
    for i, onset in enumerate(cl_onsets):
        trials[cl][:,:,i] = EEG[:, win+onset]

# Some information about the dimensionality of the data (channels x time x trials)
print('Shape of trials[cl1]:', trials[cl1].shape)
print('Shape of trials[cl2]:', trials[cl2].shape)
```

    Shape of trials[cl1]: (59, 200, 100)
    Shape of trials[cl2]: (59, 200, 100)

Since the feature we're looking for (a decrease in $\mu$-activity) is a frequency feature, lets plot the PSD of the trials in a similar manner as with the SSVEP data. The code below defines a function that computes the PSD for each trial (we're going to need it again later on):

```python
from matplotlib import mlab

def psd(trials):
    '''
    Calculates for each trial the Power Spectral Density (PSD).

    Parameters
    ----------
    trials : 3d-array (channels x samples x trials)
        The EEG signal

    Returns
    -------
    trial_PSD : 3d-array (channels x PSD x trials)
        the PSD for each trial.
    freqs : list of floats
        Yhe frequencies for which the PSD was computed (useful for plotting later)
    '''

    ntrials = trials.shape[2]
    trials_PSD = np.zeros((nchannels, 101, ntrials))

    # Iterate over trials and channels
    for trial in range(ntrials):
        for ch in range(nchannels):
            # Calculate the PSD
            (PSD, freqs) = mlab.psd(trials[ch,:,trial], NFFT=int(nsamples), Fs=sample_rate)
            trials_PSD[ch, :, trial] = PSD.ravel()

    return trials_PSD, freqs
```

```python
# Apply the function
psd_r, freqs = psd(trials[cl1])
psd_f, freqs = psd(trials[cl2])
trials_PSD = {cl1: psd_r, cl2: psd_f}
```

The function below plots the PSDs that are calculated with the above function. Since plotting it for 118 channels will clutter the display, it takes the indices of the desired channels as input, as well as some metadata to decorate the plot.

```python
import matplotlib.pyplot as plt

def plot_psd(trials_PSD, freqs, chan_ind, chan_lab=None, maxy=None):
    '''
    Plots PSD data calculated with psd().

    Parameters
    ----------
    trials : 3d-array
        The PSD data, as returned by psd()
    freqs : list of floats
        The frequencies for which the PSD is defined, as returned by psd()
    chan_ind : list of integers
        The indices of the channels to plot
    chan_lab : list of strings
        (optional) List of names for each channel
    maxy : float
        (optional) Limit the y-axis to this value
    '''
    plt.figure(figsize=(12,5))

    nchans = len(chan_ind)

    # Maximum of 3 plots per row
    nrows = int(np.ceil(nchans / 3))
    ncols = min(3, nchans)

    # Enumerate over the channels
    for i,ch in enumerate(chan_ind):
        # Figure out which subplot to draw to
        plt.subplot(nrows,ncols,i+1)

        # Plot the PSD for each class
        for cl in trials.keys():
            plt.plot(freqs, np.mean(trials_PSD[cl][ch,:,:], axis=1), label=cl)

        # All plot decoration below...

        plt.xlim(1,30)

        if maxy != None:
            plt.ylim(0,maxy)

        plt.grid()

        plt.xlabel('Frequency (Hz)')

        if chan_lab == None:
            plt.title('Channel %d' % (ch+1))
        else:
            plt.title(chan_lab[i])

        plt.legend()

    plt.tight_layout()
```

Lets put the `plot_psd()` function to use and plot three channels:

 1. C3: Central, left
 2. Cz: Central, central
 3. C4: Central, right

```python
plot_psd(
    trials_PSD,
    freqs,
    [channel_names.index(ch) for ch in ['C3', 'Cz', 'C4']],
    chan_lab=['left', 'center', 'right'],
    maxy=500
)
```

A spike of mu activity can be seen on each channel for both classes. At the right hemisphere, the mu for the left hand movement is lower than for the right hand movement due to the ERD. At the left electrode, the mu for the right hand movement is reduced and at the central electrode the mu activity is about equal for both classes. This is in line with the theory that the left hand is controlled by the right hemisphere and the feet are controlled centrally.

## Classifying the data

We will use a machine learning algorithm to construct a model that can distinguish between the right hand and foot movement of this subject. In order to do this we need to:

 1. find a way to quantify the amount of mu activity present in a trial
 2. make a model that describes expected values of mu activity for each class
 3. finally test this model on some unseen data to see if it can predict the correct class label

We will follow a classic BCI design by Blankertz et al. [1] where they use the logarithm of the variance of the signal in a certain frequency band as a feature for the classifier.

[1] Blankertz, B., Dornhege, G., Krauledat, M., Müller, K.-R., & Curio, G. (2007). The non-invasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects. *NeuroImage*, 37(2), 539–550. doi:10.1016/j.neuroimage.2007.01.051

The script below designs a band pass filter using [`scipy.signal.irrfilter`](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.iirfilter.html) that will strip away frequencies outside the 8--15Hz window. The filter is applied to all trials:

```python
import scipy.signal

def bandpass(trials, lo, hi, sample_rate):
    '''
    Designs and applies a bandpass filter to the signal.

    Parameters
    ----------
    trials : 3d-array (channels x samples x trials)
        The EEGsignal
    lo : float
        Lower frequency bound (in Hz)
    hi : float
        Upper frequency bound (in Hz)
    sample_rate : float
        Sample rate of the signal (in Hz)

    Returns
    -------
    trials_filt : 3d-array (channels x samples x trials)
        The bandpassed signal
    '''

    # The iirfilter() function takes the filter order: higher numbers mean a sharper frequency cutoff,
    # but the resulting signal might be shifted in time, lower numbers mean a soft frequency cutoff,
    # but the resulting signal less distorted in time. It also takes the lower and upper frequency bounds
    # to pass, divided by the niquist frequency, which is the sample rate divided by 2:
    a, b = scipy.signal.iirfilter(6, [lo/(sample_rate/2.0), hi/(sample_rate/2.0)])

    # Applying the filter to each trial
    ntrials = trials.shape[2]
    trials_filt = np.zeros((nchannels, nsamples, ntrials))
    for i in range(ntrials):
        trials_filt[:,:,i] = scipy.signal.filtfilt(a, b, trials[:,:,i], axis=1)

    return trials_filt
```

```python
# Apply the function
trials_filt = {cl1: bandpass(trials[cl1], 8, 15, sample_rate),
               cl2: bandpass(trials[cl2], 8, 15, sample_rate)}
```

Plotting the PSD of the resulting `trials_filt` shows the suppression of frequencies outside the passband of the filter:

```python
psd_r, freqs = psd(trials_filt[cl1])
psd_f, freqs = psd(trials_filt[cl2])
trials_PSD = {cl1: psd_r, cl2: psd_f}

plot_psd(
    trials_PSD,
    freqs,
    [channel_names.index(ch) for ch in ['C3', 'Cz', 'C4']],
    chan_lab=['left', 'center', 'right'],
    maxy=300
)
```

As a feature for the classifier, we will use the logarithm of the variance of each channel. The function below calculates this:

```python
# Calculate the log(var) of the trials
def logvar(trials):
    '''
    Calculate the log-var of each channel.

    Parameters
    ----------
    trials : 3d-array (channels x samples x trials)
        The EEG signal.

    Returns
    -------
    logvar - 2d-array (channels x trials)
        For each channel the logvar of the signal
    '''
    return np.log(np.var(trials, axis=1))
```

```python
# Apply the function
trials_logvar = {cl1: logvar(trials_filt[cl1]),
                 cl2: logvar(trials_filt[cl2])}
```

Below is a function to visualize the logvar of each channel as a bar chart:

```python
def plot_logvar(trials):
    '''
    Plots the log-var of each channel/component.
    arguments:
        trials - Dictionary containing the trials (log-vars x trials) for 2 classes.
    '''
    plt.figure(figsize=(12,5))

    x0 = np.arange(nchannels)
    x1 = np.arange(nchannels) + 0.4

    y0 = np.mean(trials[cl1], axis=1)
    y1 = np.mean(trials[cl2], axis=1)

    plt.bar(x0, y0, width=0.5, color='b')
    plt.bar(x1, y1, width=0.4, color='r')

    plt.xlim(-0.5, nchannels+0.5)

    plt.gca().yaxis.grid(True)
    plt.title('log-var of each channel/component')
    plt.xlabel('channels/components')
    plt.ylabel('log-var')
    plt.legend(cl_lab)
```

```python
# Plot the log-vars
plot_logvar(trials_logvar)
```

We see that most channels show a small difference in the log-var of the signal between the two classes. The next step is to go from 118 channels to only a few channel mixtures. The CSP algorithm calculates mixtures of channels that are designed to maximize the difference in variation between two classes. These mixures are called spatial filters.

```python
from numpy import linalg

def cov(trials):
    ''' Calculate the covariance for each trial and return their average '''
    ntrials = trials.shape[2]
    covs = [ trials[:,:,i].dot(trials[:,:,i].T) / nsamples for i in range(ntrials) ]
    return np.mean(covs, axis=0)

def whitening(sigma):
    ''' Calculate a whitening matrix for covariance matrix sigma. '''
    U, l, _ = linalg.svd(sigma)
    return U.dot( np.diag(l ** -0.5) )

def csp(trials_r, trials_f):
    '''
    Calculate the CSP transformation matrix W.
    arguments:
        trials_r - Array (channels x samples x trials) containing right hand movement trials
        trials_f - Array (channels x samples x trials) containing foot movement trials
    returns:
        Mixing matrix W
    '''
    cov_r = cov(trials_r)
    cov_f = cov(trials_f)
    P = whitening(cov_r + cov_f)
    B, _, _ = linalg.svd( P.T.dot(cov_f).dot(P) )
    W = P.dot(B)
    return W

def apply_mix(W, trials):
    ''' Apply a mixing matrix to each trial (basically multiply W with the EEG signal matrix)'''
    ntrials = trials.shape[2]
    trials_csp = np.zeros((nchannels, nsamples, ntrials))
    for i in range(ntrials):
        trials_csp[:,:,i] = W.T.dot(trials[:,:,i])
    return trials_csp
```

```python
# Apply the functions
W = csp(trials_filt[cl1], trials_filt[cl2])
trials_csp = {cl1: apply_mix(W, trials_filt[cl1]),
              cl2: apply_mix(W, trials_filt[cl2])}
```

To see the result of the CSP algorithm, we plot the log-var like we did before:

```python
trials_logvar = {cl1: logvar(trials_csp[cl1]),
                 cl2: logvar(trials_csp[cl2])}
plot_logvar(trials_logvar)
```

Instead of 118 channels, we now have 118 mixtures of channels, called components. They are the result of 118 spatial filters applied to the data.

The first filters maximize the variation of the first class, while minimizing the variation of the second. The last filters maximize the variation of the second class, while minimizing the variation of the first.

This is also visible in a PSD plot. The code below plots the PSD for the first and last components as well as one in the middle:

```python
psd_r, freqs = psd(trials_csp[cl1])
psd_f, freqs = psd(trials_csp[cl2])
trials_PSD = {cl1: psd_r, cl2: psd_f}

plot_psd(trials_PSD, freqs, [0,28,-1], chan_lab=['first component', 'middle component', 'last component'], maxy=0.75 )
```

In order to see how well we can differentiate between the two classes, a scatter plot is a useful tool. Here both classes are plotted on a 2-dimensional plane: the x-axis is the first CSP component, the y-axis is the last.

```python
def plot_scatter(left, foot):
    plt.figure()
    plt.scatter(left[0,:], left[-1,:], color='b')
    plt.scatter(foot[0,:], foot[-1,:], color='r')
    plt.xlabel('Last component')
    plt.ylabel('First component')
    plt.legend(cl_lab)
```

```python
plot_scatter(trials_logvar[cl1], trials_logvar[cl2])
```

We will apply a linear classifier to this data. A linear classifier can be thought of as drawing a line in the above plot to separate the two classes. To determine the class for a new trial, we just check on which side of the line the trial would be if plotted as above.

The data is split into a train and a test set. The classifier will fit a model (in this case, a straight line) on the training set and use this model to make predictions about the test set (see on which side of the line each trial in the test set falls). Note that the CSP algorithm is part of the model, so for fairness sake it should be calculated using only the training data.

```python
# Percentage of trials to use for training (50-50 split here)
train_percentage = 0.5

# Calculate the number of trials for each class the above percentage boils down to
ntrain_r = int(trials_filt[cl1].shape[2] * train_percentage)
ntrain_f = int(trials_filt[cl2].shape[2] * train_percentage)
ntest_r = trials_filt[cl1].shape[2] - ntrain_r
ntest_f = trials_filt[cl2].shape[2] - ntrain_f

# Splitting the frequency filtered signal into a train and test set
train = {cl1: trials_filt[cl1][:,:,:ntrain_r],
         cl2: trials_filt[cl2][:,:,:ntrain_f]}

test = {cl1: trials_filt[cl1][:,:,ntrain_r:],
        cl2: trials_filt[cl2][:,:,ntrain_f:]}

# Train the CSP on the training set only
W = csp(train[cl1], train[cl2])

# Apply the CSP on both the training and test set
train[cl1] = apply_mix(W, train[cl1])
train[cl2] = apply_mix(W, train[cl2])
test[cl1] = apply_mix(W, test[cl1])
test[cl2] = apply_mix(W, test[cl2])

# Select only the first and last components for classification
comp = np.array([0,-1])
train[cl1] = train[cl1][comp,:,:]
train[cl2] = train[cl2][comp,:,:]
test[cl1] = test[cl1][comp,:,:]
test[cl2] = test[cl2][comp,:,:]

# Calculate the log-var
train[cl1] = logvar(train[cl1])
train[cl2] = logvar(train[cl2])
test[cl1] = logvar(test[cl1])
test[cl2] = logvar(test[cl2])
```

For a classifier the Linear Discriminant Analysis (LDA) algorithm will be used. It fits a gaussian distribution to each class, characterized by the mean and covariance, and determines an optimal separating plane to divide the two. This plane is defined as $r = W_0 \cdot X_0 + W_1 \cdot X_1 + \ldots + W_n \cdot X_n - b$, where $r$ is the classifier output, $W$ are called the feature weights, $X$ are the features of the trial, $n$ is the dimensionality of the data and $b$ is called the offset.

In our case we have 2 dimensional data, so the separating plane will be a line: $r = W_0 \cdot X_0 + W_1 \cdot X_1 - b$. To determine a class label for an unseen trial, we can calculate whether the result is positive or negative.

```python
def train_lda(class1, class2):
    '''
    Trains the LDA algorithm.
    arguments:
        class1 - An array (observations x features) for class 1
        class2 - An array (observations x features) for class 2
    returns:
        The projection matrix W
        The offset b
    '''
    nclasses = 2

    nclass1 = class1.shape[0]
    nclass2 = class2.shape[0]

    # Class priors: in this case, we have an equal number of training
    # examples for each class, so both priors are 0.5
    prior1 = nclass1 / float(nclass1 + nclass2)
    prior2 = nclass2 / float(nclass1 + nclass1)

    mean1 = np.mean(class1, axis=0)
    mean2 = np.mean(class2, axis=0)

    class1_centered = class1 - mean1
    class2_centered = class2 - mean2

    # Calculate the covariance between the features
    cov1 = class1_centered.T.dot(class1_centered) / (nclass1 - nclasses)
    cov2 = class2_centered.T.dot(class2_centered) / (nclass2 - nclasses)

    W = (mean2 - mean1).dot(np.linalg.pinv(prior1*cov1 + prior2*cov2))
    b = (prior1*mean1 + prior2*mean2).dot(W)

    return (W,b)

def apply_lda(test, W, b):
    '''
    Applies a previously trained LDA to new data.
    arguments:
        test - An array (features x trials) containing the data
        W    - The project matrix W as calculated by train_lda()
        b    - The offsets b as calculated by train_lda()
    returns:
        A list containing a classlabel for each trial
    '''
    ntrials = test.shape[1]

    prediction = []
    for i in range(ntrials):
        # The line below is a generalization for:
        # result = W[0] * test[0,i] + W[1] * test[1,i] - b
        result = W.dot(test[:,i]) - b
        if result <= 0:
            prediction.append(1)
        else:
            prediction.append(2)

    return np.array(prediction)
```

Training the LDA using the training data gives us $W$ and $b$:

```python
W,b = train_lda(train[cl1].T, train[cl2].T)

print('W:', W)
print('b:', b)
```

    W: [ 5.31347949 -5.52963938]
    b: 0.3802472099930121

It can be informative to recreate the scatter plot and overlay the decision boundary as determined by the LDA classifier. The decision boundary is the line for which the classifier output is exactly 0. The scatterplot used $X_0$ as $x$-axis and $X_1$ as $y$-axis. To find the function $y = f(x)$ describing the decision boundary, we set $r$ to 0 and solve for $y$ in the equation of the separating plane:

<div style="width:600px">
$$\begin{align}
W_0 \cdot X_0 + W_1 \cdot X_1 - b &= r &&\text{the original equation} \\\
W_0 \cdot x + W_1 \cdot y - b &= 0     &&\text{filling in $X_0=x$, $X_1=y$ and $r=0$} \\\
W_0 \cdot x + W_1 \cdot y &= b         &&\text{solving for $y$}\\\
W_1 \cdot y &= b - W_0 \cdot x \\\
\\\
y &= \frac{b - W_0 \cdot x}{W_1}
\end{align}$$
</div>

We first plot the decision boundary with the training data used to calculate it:

```python
# Scatterplot like before
plot_scatter(train[cl1], train[cl2])
title('Training data')

# Calculate decision boundary (x,y)
x = np.arange(-5, 1, 0.1)
y = (b - W[0]*x) / W[1]

# Plot the decision boundary
plt.plot(x,y, linestyle='--', linewidth=2, color='k')
plt.xlim(-5, 1)
plt.ylim(-2.2, 1)
```

The code below plots the boundary with the test data on which we will apply the classifier. You will see the classifier is going to make some mistakes.

```python
plot_scatter(test[cl1], test[cl2])
title('Test data')
plt.plot(x,y, linestyle='--', linewidth=2, color='k')
plt.xlim(-5, 1)
plt.ylim(-2.2, 1)
```

Now the LDA is constructed and fitted to the training data. We can now apply it to the test data. The results are presented as a confusion matrix:

<table>
    <tr><td></td><td colspan='2' style="font-weight:bold">True labels →</td></tr>
    <tr><td style="font-weight:bold">↓ Predicted labels</td><td>Right</td><td>Foot</td></tr>
    <tr><td>Right</td><td></td><td></td></tr>
    <tr><td>Foot</td><td></td><td></td></tr>
</table>

The number at the diagonal will be trials that were correctly classified, any trials incorrectly classified (either a false positive or false negative) will be in the corners.

```python
# Print confusion matrix
conf = np.array([
    [(apply_lda(test[cl1], W, b) == 1).sum(), (apply_lda(test[cl2], W, b) == 1).sum()],
    [(apply_lda(test[cl1], W, b) == 2).sum(), (apply_lda(test[cl2], W, b) == 2).sum()],
])

print('Confusion matrix:')
print(conf)
print()
print('Accuracy: %.3f' % (np.sum(np.diag(conf)) / float(np.sum(conf))))
```

    Confusion matrix:
    [[45  4]
     [ 5 46]]

    Accuracy: 0.910

The confusion matrix shows that 4 out of the 50 trials with foot movement were incorrectly classified as right hand movement and 5 out of the 50 trials with right hand movement were incorrectly classified as foot movement. In total, 91% of the trials were correctly classified, not a bad score!

If you want, you can continue with the next tutorial:
[4. Classifiying the P300](4.%20Classifying%20the%20P300.ipynb)


# Generative models - auto-encoders

### Author: Philippe Esling (esling@ircam.fr)

In this course we will cover
1. A brief introduction to [generative models](#generative)
2. A formal presentation of [Auto-Encoders](#ae) (AEs)
3. An explanation of how to [implement AEs](#implement)
4. An [application](#application) of AEs for modeling images
4. An practical exemple of [convolutional denoising AEs](#denoising) for image data **(exercise)**

## Generative models

### Supervised refresher

Until now, we have mostly discussed models that are developed for _supervised_ and _discriminative_ tasks. To formalize this problem, we have a set of data $\{\mathbf{x}_{i}, \mathbf{y}_{i}\}_{i\in[1,n]}$, where the $\mathbf{x}_{i}$ and $\mathbf{y}_{i}$ are linked. Therefore, we want to approximate this relation through
\begin{equation}
    \mathbf{\hat{y}} = \mathcal{F}_\mathbf{\theta}(\mathbf{x})
\end{equation}
where we train the parameters $\mathbf{\theta}$ so that $\mathbf{\hat{y}}\approx\mathbf{y}$. The existence of a label $\mathbf{y}$ ("correct answer") defines a _supervised_ problem

### Going unsupervised

In some cases, we might only have a set of data $\{\mathbf{x}_{i}\}_{i\in[1,n]}$, and still be interested in learning some underlying properties or structure of this set. In that case, the problem is _unsupervised_ as we have to learn without a given answer.

Here, we can turn to _generative_ models [[1](#reference1)], which allows to create new data instances based on the observation of existing examples. Although these models are more naturally defined in a _probabilistic way_, we will assume that we have some simple _code_ $\mathbf{z}$, which allows to control the properties of the generation, and need to learn
\begin{equation}
    \mathbf{\hat{x}} = \mathcal{F}_\mathbf{\theta}(\mathbf{z})
\end{equation}
where we still need to learn $\mathbf{\theta}$, so that $\mathbf{\hat{x}}$ have similar properties to that of the examples in $\{\mathbf{x}_{i}\}_{i\in[1,n]}$.

Now the problem to solve is how we can learn directly from a set of data.

## Auto-encoders

One way to understand a set of data is to try to _compress_, or _simplify_ the corresponding dataset. So the idea is to learn simultaneously how to _encode_ our unlabeled input $\{\mathbf{x}_{i}\}_{i\in[1,n]}$ and to _decode_ the corresponding representation. This idea give rise to the notion of **auto-encoder**.

### Architecture

The auto-encoder is an unsupervised architecture originally proposed to perform _dimensionality reduction_ [[3](#reference3)]. As its name indicates, we will try to train this model to learn an efficient _encoding_ $\mathbf{z}$ of unlabeled input data $\mathbf{x}$. The only way to learn efficient parameters is to also learn a _decoding_ function to _reconstruct_ $\mathbf{x}$ from $\mathbf{z}$.

As shown here, a first model $\mathcal{E}_\phi$ \textit{encodes} the input into a \textit{latent code} $\mathbf{z}$ in order to provide a low-dimensional representation of the data. A second model $\mathcal{D}_\theta$ designated as the \textit{decoder} aims to generate outputs from $\mathbf{z}$ that are as close to the original inputs as possible.

### Formal definition

The latent code $\mathbf{z}$ can be seen as a compressed abstract representation, and may be used as an intermediate space for analysis or generation. This helps to govern the distribution of the data through a simpler and higher-level representation, while enhancing the \textit{expressiveness} of the generative model.
The behaviour of an auto-encoder can be formalized as:
\begin{align}
    \mathbf{z} &= \mathcal{E}_\phi(\mathbf{x}) \\
    \mathbf{\hat{x}} &= \mathcal{D}_\theta(\mathbf{z})
\end{align}
with the \textit{encoder} $\mathcal{E}_\phi$ and \textit{decoder} $\mathcal{D}_\theta$ functions parameterized respectively by $\phi$ and $\theta$. As we can see this defines the reconstruction relationship
\begin{equation}
    \mathbf{\hat{x}} = \mathcal{D}_\theta(\mathcal{E}_\phi(\mathbf{x}))
\end{equation}

### Training

The training of an auto-encoder consists in finding the optimal functions of encoding $\mathcal{E}^*$ and decoding $\mathcal{D}^*$ by evaluating the \textit{reconstruction error} $\mathcal{L}$ between $\mathbf{x}$ and $\mathbf{\hat{x}}$, such that
\begin{equation}
    \mathcal{E}^*, \mathcal{D}^* = arg\,min_{ \phi, \theta}{\mathcal{L}}(\mathbf{x}, \mathcal{D}_\theta(\mathcal{E}_\phi(\mathbf{x})))
\end{equation}

As the latent space usually has a smaller dimensionality than the input, it acts as an incentive for the network to find the main attributes of variations in the dataset (and also explains its use for _dimensionality reduction_).

### Variants and discussion

There are several variants of auto-encoders, such as denoising auto-encoders or variational auto-encoders. Each address some downside of the basic AE model. For instance, the deterministic nature of the basic auto-encoder implies a point-wise mapping of the latent space, meaning that not all the latent positions can be leveraged to produce relevant reconstructions. Because of this reason, there is no way to ensure that the latent space could allow a robust generalization and that any random $\mathbf{z}$ would generate a meaningful output.

## Implementation

Here, we discuss how we can implement and train a simple auto-encoder network in _TensorFlow_. As discussed earlier, an AE is composed of two parts, an **encoder** and a **decoder**. The goal of the encoder is to "compress" the dataset, representing its principal features with a very small code, while the goal of the decoder is to learn how to reproduce the initial input from this code. Hence, we will first need to use some basic imports and definition to setup our problem

### Import TensorFlow and other libraries

```python
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import tensorflow as tf

from sklearn.metrics import accuracy_score, precision_score, recall_score
from sklearn.model_selection import train_test_split
from tensorflow.keras import layers, losses
from tensorflow.keras.datasets import fashion_mnist
from tensorflow.keras.models import Model
```

### Load the dataset

To start with a pragmatic and simple to understand example, we  will try to train the basic AE using the Fashon MNIST dataset. This dataset contains images of size 28x28 pixels, with different pieces of clothing. The following code allows to load (and eventually download) the dataset, by using the `tensorflow.keras.datasets` module. We also plot some randomly selected test examples

```python
# Load (and eventually download) the dataset
(x_train, _), (x_test, _) = fashion_mnist.load_data()
# Normalize the dataset in the [0, 1] range]
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
print('Number of training examples')
print (x_train.shape)
print('Number of testing examples')
print (x_test.shape)
plt.figure(figsize=(20, 4))
# Plot some randomly selected input
for i in range(10):
    ax = plt.subplot(1, 10, i + 1)
    i_val = np.random.randint(x_test.shape[0])
    plt.imshow(x_test[i_val])
    plt.gray()
    plt.title("Example %d"%(i_val))
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()
```

## Basic auto-encoder

We recall here that in order to define an autoencoder, we will need an `encoder`, which compresses the images into a small latent vector, and a `decoder`, that reconstructs the original image from this code. Here, we will start very basic and define the encoder and decoder as simple `Dense` layers. To define the model simply, we will use the [Keras API](https://www.tensorflow.org/guide/keras/custom_layers_and_models) defined in the `tensorflow.keras` module (Note that we pre-loaded the `layers` submodule

```python
class AE(Model):
    def __init__(self, encoding_dim):
        super(AE, self).__init__()
        self.latent_dim = encoding_dim
        self.encoder = tf.keras.Sequential([
          layers.Flatten(),
          layers.Dense(self.latent_dim, activation='relu'),
        ])
        self.decoder = tf.keras.Sequential([
          layers.Dense(784, activation='sigmoid'),
          layers.Reshape((28, 28))
        ])

    def call(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded
```

Here we can see that the model depends on a given `encoding_dim` variable, which defines the size of the latent code. Therefore, we can instantiate our model arbitrarliy with `64` dimensions

```python
latent_dim = 64
autoencoder = AE(latent_dim)
```

The only remaining part that we did not discuss yet is what type of _loss_ (defined as $\mathcal{L}$) we can use to train our model. First, we will simply rely on the _Mean Squared Error_ (MSE) loss, which is defined as
\begin{equation}
    \mathcal{L}_{MSE}(\hat{\mathbf{x}}, \mathbf{x}) = \mid \hat{\mathbf{x}}, \mathbf{x} \mid^{2}
\end{equation}

```python
autoencoder.compile(optimizer='adam', loss=losses.MeanSquaredError())
```

### Training the model

Train the model using `x_train` as both the input and the target. The `encoder` will learn to compress the dataset from 784 dimensions to the latent space, and the `decoder` will learn to reconstruct the original images.
.

```python
autoencoder.fit(x_train, x_train, epochs=10, shuffle=True, validation_data=(x_test, x_test))
```

    Epoch 1/10
    1875/1875 [==============================] - 2s 992us/step - loss: 0.0086 - val_loss: 0.0088
    Epoch 2/10
    1875/1875 [==============================] - 2s 966us/step - loss: 0.0086 - val_loss: 0.0088
    Epoch 3/10
    1875/1875 [==============================] - 2s 983us/step - loss: 0.0086 - val_loss: 0.0087
    Epoch 4/10
    1875/1875 [==============================] - 2s 993us/step - loss: 0.0086 - val_loss: 0.0087
    Epoch 5/10
    1875/1875 [==============================] - 2s 978us/step - loss: 0.0086 - val_loss: 0.0088
    Epoch 6/10
    1875/1875 [==============================] - 2s 1ms/step - loss: 0.0086 - val_loss: 0.0088
    Epoch 7/10
    1875/1875 [==============================] - 2s 1ms/step - loss: 0.0086 - val_loss: 0.0087
    Epoch 8/10
    1875/1875 [==============================] - 2s 1ms/step - loss: 0.0086 - val_loss: 0.0087
    Epoch 9/10
    1875/1875 [==============================] - 2s 1ms/step - loss: 0.0086 - val_loss: 0.0087
    Epoch 10/10
    1875/1875 [==============================] - 2s 1ms/step - loss: 0.0085 - val_loss: 0.0087

    <tensorflow.python.keras.callbacks.History at 0x15a4bf750>

Now that the model is trained, we can test it by encoding and decoding images from the test set.

```python
encoded_imgs = autoencoder.encoder(x_test).numpy()
decoded_imgs = autoencoder.decoder(encoded_imgs).numpy()
```

By plotting the images, we can see that the model is able to perform an adequate (yet somewhat blurry) reconstruction of the input images. The interesting point is that this reconstruction comes from a code of only `64` dimensions, whereas the original images have `784` dimensions.

```python
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
  # display original
  ax = plt.subplot(2, n, i + 1)
  plt.imshow(x_test[i])
  plt.title("original")
  plt.gray()
  ax.get_xaxis().set_visible(False)
  ax.get_yaxis().set_visible(False)

  # display reconstruction
  ax = plt.subplot(2, n, i + 1 + n)
  plt.imshow(decoded_imgs[i])
  plt.title("reconstructed")
  plt.gray()
  ax.get_xaxis().set_visible(False)
  ax.get_yaxis().set_visible(False)
plt.show()
```

### Improvements

Even though this very basic example seems to work, several improvements can be made over the original model. First, we can see that the overall framework does not depend on the exact nature of the `encoder` and `decoder`. Therefore, we can rewrite the original class to accept any type of architecture for these (see code below). Another aspect that has a major impact on the results is the type of `loss` used. The `MeanSquaredError` tend to favor blurry reconstructions, and could be replaced by a `Multinoulli` loss. You can experiment with these ideas below.

```python
class AE(Model):
  def __init__(self, encoder, decoder, encoding_dim):
    super(AE, self).__init__()
    self.latent_dim = encoding_dim
    self.encoder = encoder
    self.decoder = decoder

  def call(self, x):
    encoded = self.encoder(x)
    decoded = self.decoder(encoded)
    return decoded
```

## Exercise: Denoising AE

Imagine (for the sake of argument), that we choose an encoding dimension which is of same dimensionality as the input one. Then, one huge problem is that nothing prevents the AE from simply learning the _identity_ function (try to imagine why). An autoencoder can also be trained to remove noise from images. This type of _regularization_ prevents the model from learning this degenerate situation.

In this exercise, you will need to create your own denoising AE, by relying on a noisy version of the Fashion MNIST dataset (adding random Gaussian noise to each image). You will then train an autoencoder using the noisy image as input, and the original image as the target.

Let's reimport the dataset to omit the modifications made earlier.

```python
(x_train, _), (x_test, _) = fashion_mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
print(x_train.shape)
```

    (60000, 28, 28)

Here, we create two new train and test sets by adding random noise to the images

```python
noise_factor = 0.2
x_train_noisy = x_train + noise_factor * tf.random.normal(shape=x_train.shape)
x_test_noisy = x_test + noise_factor * tf.random.normal(shape=x_test.shape)
x_train_noisy = tf.clip_by_value(x_train_noisy, clip_value_min=0., clip_value_max=1.)
x_test_noisy = tf.clip_by_value(x_test_noisy, clip_value_min=0., clip_value_max=1.)
```

Plot the noisy images.

```python
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
    ax = plt.subplot(2, n, i + 1)
    plt.title("original")
    plt.imshow(tf.squeeze(x_test[i]))
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
    plt.gray()
    ax = plt.subplot(2, n, i + 1 + n)
    plt.title("original + noise")
    plt.imshow(tf.squeeze(x_test_noisy[i]))
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
    plt.gray()
plt.show()
```

### Define a convolutional autoencoder

In this example, you will train a convolutional autoencoder using  [Conv2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) layers in the `encoder`, and [Conv2DTranspose](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2DTranspose) layers in the `decoder`.

```python
class DenoisingAE(AE):
  def __init__(self):
    super(DenoisingAE, self).__init__()
    self.encoder = ...
    self.decoder = ...

  def call(self, x):
    encoded = ...
    decoded = ...
    return decoded

autoencoder = ...
```

Compile and fit your model with appropriate losses

```python
autoencoder.compile(...)
autoencoder.fit(...)
```

Let's take a look at a summary of the encoder and decoder. Notice how the images are downsampled and then upsampled back to the original input size.

```python
autoencoder.encoder.summary()
autoencoder.decoder.summary()
```

Plot both the noisy images and the denoised images produced by the autoencoder to check that your implementation is correct

```python
encoded_imgs = ...
decoded_imgs = ...
plt.figure(figsize=(20, 4))
for i in range(n):
    ...
plt.show()
```

### References

[1] Rezende, Danilo Jimenez, and Shakir Mohamed. "Variational inference with normalizing flows." _arXiv preprint arXiv:1505.05770_ (2015). [link](http://arxiv.org/pdf/1505.05770)

[2] Kingma, Diederik P., Tim Salimans, and Max Welling. "Improving Variational Inference with Inverse Autoregressive Flow." _arXiv preprint arXiv:1606.04934_ (2016). [link](https://arxiv.org/abs/1606.04934)

[3] Kingma, D. P., & Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. (2013). [link](https://arxiv.org/pdf/1312.6114)

[4] Rezende, D. J., Mohamed, S., & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082. (2014). [link](https://arxiv.org/pdf/1401.4082)

[5] Gulrajani, I., Kumar, K., Ahmed, F., Taiga, A. A., Visin, F., Vazquez, D., & Courville, A. (2016). Pixelvae: A latent variable model for natural images. arXiv preprint arXiv:1611.05013. [link](https://arxiv.org/pdf/1611.05013)

[6] Van den Oord, A., & Vinyals, O. (2017). Neural discrete representation learning. In NIPS 2017 (pp. 6306-6315). [link](http://papers.nips.cc/paper/7210-neural-discrete-representation-learning.pdf)


# PyTorch tutorial の内容について(4)

- date: 2020-0722
- author: 浅川伸一

[https://github.com/pytorch/tutorials/tree/master/beginner_source/nlp](https://github.com/pytorch/tutorials/tree/master/beginner_source/nlp) を見ると
PyTorch で 自然言語処理を行う場合のチュートリアルは以下とおりである

# Deep Learning for NLP with Pytorch

1. [pytorch_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/pytorch_tutorial.py):
	[PyTorch 入門 Introduction to PyTorch](https://pytorch.org/tutorials/beginner/nlp/pytorch_tutorial.html)

2. [deep_learning_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/deep_learning_tutorial.py):
	[PyTorch による深層学習 Deep Learning with PyTorch](https://pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html)

3. [word_embeddings_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/word_embeddings_tutorial.py):
	[単語埋め込み:語彙的意味の符号化 Word Embeddings: Encoding Lexical Semantics](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html)

4. [sequence_models_tutorial.py]((https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/sequence_models_tutorial.py):
	[系列モデルと LSTM Sequence Models and Long-Short Term Memory Networks](https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html)

5. [advanced_tutorial.py]((https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/advanced_tutorial.py):
	[動的意思決定と双方向 LSTM 条件付き確率場 Advanced: Making Dynamic Decisions and the Bi-LSTM CRF](https://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html)

以下では，このうちの 4 について解説している。

```python
%matplotlib inline
```

# 系列モデルと LSTM ネットワーク
ここまで，様々なフィードフォワードネットワークを見てきました。
フィードフォワードニューラルネットワークは，ネットワークが保持する(内部) 状態がありません。
フィードフォワードモデルは，系列処理に向いているモデルではないと言えます。
一方，系列モデルは 自然言語処理 (NLP) の中核をなすモデルです。
系列モデルとは，入力情報間に時間的依存関係を仮定したモデルです。
系列モデルの古典的な例としては，品詞タグ付けのための ``隠れマルコフモデル`` があります。
他の例としては ``条件付き確率場`` があります。

リカレント・ニューラルネットワークは，状態を保持することができるネットワークです。
例えば，ネットワークの出力は次の入力の一部として使用されます。
リカレントニューラルネットワークは，全系列を通して情報が伝播するようにすることもできます。
LSTM の場合 系列の各要素に対して，対応する *隠れ状態* $h_t$ があり，原則として，系列の任意の時点の情報を蓄えておくことができます。
隠れ状態を利用して，言語モデルの単語や品詞タグ付，その他の多くの予測することが可能です。

### PyTorch における LSTM

例題に入る前に いくつかのことに注意してください。
Pytorch の LSTM は、すべての入力が 3 次元テンソルであることを仮定しています。
この入力テンソルの軸の意味は重要です。
第 1 軸(次元)は系列そのものです。
第 2 軸(次元) はミニバッチ内の実体，
第 3 軸(次元) の軸は入力の要素を表します。
ミニバッチについてはここでは議論していません。
ですので ミニバッチは無視して，第 2 軸には常に 1つの次元があると仮定しましょう。
``牛が飛び跳ねた The cow jumped`` という文に対して系列モデルを実行したい場合，
入力は以下のようになります:

\begin{align}\begin{bmatrix}
   \overbrace{q_\text{The}}^\text{行ベクトル} \\
   q_\text{cow} \\
   q_\text{jumped}
   \end{bmatrix}
\end{align}

サイズ 1 の追加した第 2 次元があることを忘れないでください。

さらに 系列を一度に 1 つずつ進めることもできます。ですが その場合 第1軸 もサイズが 1 になります。

簡単な例を見てみましょう。

```python
# Author: Robert Guthrie
# modified by Shin Asakawa

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
```

    <torch._C.Generator at 0x7f8100644120>

```python
lstm = nn.LSTM(3, 3)  # 入力次元は 3 出力次元も 3 です
inputs = [torch.randn(1, 3) for _ in range(5)]  # 長さ 5 の系列を作成

# 隠れ状態の初期化
hidden = (torch.randn(1, 1, 3),
          torch.randn(1, 1, 3))
for i in inputs:
    # 全系列を通じて，一度に一つづづ，逐次，Step through the sequence one element at a time.
    # 1ステップ後，中間層は，中間層の状態と入力情報とが入力となる # after each step, hidden contains the hidden state.
    out, hidden = lstm(i.view(1, 1, -1), hidden)

# 代わりに 系列全体を一度に実行することもできます。
# LSTM が返す最初の値は  系列全体の 隠れ状態のすべてです。
# 2 番目の値は 最新の隠れ状態です
#（"out "と "hidden "の最後のスライスを比較してみてください。
# その理由は次のようになります:
# "out "は 系列内の全隠れ状態にアクセスできるようにします。
#  "hidden" は 系列を継続して誤差逆伝播することを可能にします。
# 後で LSTM に引数として渡すことで 2 次元を追加します。
# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument  to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (torch.randn(1, 1, 3), torch.randn(1, 1, 3))  # clean out hidden state
out, hidden = lstm(inputs, hidden)
print(out)
print(hidden)
```

    tensor([[[-0.0187,  0.1713, -0.2944]],

            [[-0.3521,  0.1026, -0.2971]],

            [[-0.3191,  0.0781, -0.1957]],

            [[-0.1634,  0.0941, -0.1637]],

            [[-0.3368,  0.0959, -0.0538]]], grad_fn=<StackBackward>)
    (tensor([[[-0.3368,  0.0959, -0.0538]]], grad_fn=<StackBackward>), tensor([[[-0.9825,  0.4715, -0.0633]]], grad_fn=<StackBackward>))

### 例: 品詞タグ付け LSTM

ここでは LSTM を使って品詞タグ付けを行います。
ビタビ (アルゴリズム) や 前向き-後ろ向き (アルゴリズム) などは使いません。
ですが，読者への (挑戦的な) 練習問題として，説明を読んだ後で，どうすれば ビタビ アルゴリズムが使えるかを考えてみてください。

モデルは以下のとおりです:
入力系列を語彙の系列 $w_1, \ldots, w_M$ とします。
また $T$ をタグ集合，$y_i$ を単語 $w_i$ に対応するタグとします。
単語 $w_i$ の品詞の予測値は $\hat{y}_i$ とします。

これは，予測モデルであり出力は $\hat{y}_1,\ldots,\hat{y}_M$ とします。ここで $\hat{y}_i \in T$ とします。

予測を行う際には 文の上を LSTM を入力します。
時刻 $i$ での中間層状態を$h_i$ とします。
また それぞれのタグに一意なインデックスを割り当てる(単語の埋め込みの項で word_to\_ixとしたように)。
すると $hat{y}_i$ の予測は以下のようになります:

\begin{align}\hat{y}_i = \text{argmax}_j \  (\log \text{Softmax}(Ah_i + b))_j\end{align}

すなわち，中間層状態のアフィン写像の 対数ソフトマックスから，このベクトル中の最大値を持つタグを予測タグとします。
このことは $A$ のターゲット空間の次元が $|T|$ であることを暗示していることに注意してください。

データの準備:

```python
def prepare_sequence(seq, to_ix):
    idxs = [to_ix[w] for w in seq]
    return torch.tensor(idxs, dtype=torch.long)

training_data = [
    ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
    # the は定冠詞:DET, dog は名詞:NN, ate は 動詞:v, the は定冠詞;DET, apple は名詞:NN
    ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
    # everybody は 名詞:NN, read は動詞:V, that は定冠詞:DET, book は名詞:NN
]
word_to_ix = {}
for sent, tags in training_data:
    for word in sent:
        if word not in word_to_ix:
            word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}

# 以下は例なので小さく 6 次元としていますが，通常は 32 とか 64 次元
# にしています。
EMBEDDING_DIM = 6
HIDDEN_DIM = 6
```

    {'The': 0, 'dog': 1, 'ate': 2, 'the': 3, 'apple': 4, 'Everybody': 5, 'read': 6, 'that': 7, 'book': 8}

モデルの作成:

```python
class LSTMTagger(nn.Module):

    def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
        super(LSTMTagger, self).__init__()
        self.hidden_dim = hidden_dim

        self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)

        # LSTM を用いた言語モデルでは，単語埋め込み表現を入力とし，
        # 中間層の状態と出力値を出力します
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)

        # 中間層から出力空間への写像関数
        self.hidden2tag = nn.Linear(hidden_dim, tagset_size)

    def forward(self, sentence):
        embeds = self.word_embeddings(sentence)
        lstm_out, _ = self.lstm(embeds.view(len(sentence), 1, -1))
        tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
        tag_scores = F.log_softmax(tag_space, dim=1)
        return tag_scores
```

モデルの訓練:

```python
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
# Here we don't need to train, so the code is wrapped in torch.no_grad()
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)
    print(tag_scores)

for epoch in range(300):  # again, normally you would NOT do 300 epochs, it is toy data
    for sentence, tags in training_data:
        # Step 1. Remember that Pytorch accumulates gradients.
        # We need to clear them out before each instance
        model.zero_grad()

        # Step 2. Get our inputs ready for the network, that is, turn them into
        # Tensors of word indices.
        sentence_in = prepare_sequence(sentence, word_to_ix)
        targets = prepare_sequence(tags, tag_to_ix)

        # Step 3. Run our forward pass.
        tag_scores = model(sentence_in)

        # Step 4. Compute the loss, gradients, and update the parameters by
        #  calling optimizer.step()
        loss = loss_function(tag_scores, targets)
        loss.backward()
        optimizer.step()

# See what the scores are after training
with torch.no_grad():
    inputs = prepare_sequence(training_data[0][0], word_to_ix)
    tag_scores = model(inputs)

    # The sentence is "the dog ate the apple".  i,j corresponds to score for tag j
    # for word i. The predicted tag is the maximum scoring tag.
    # Here, we can see the predicted sequence below is 0 1 2 0 1
    # since 0 is index of the maximum value of row 1,
    # 1 is the index of maximum value of row 2, etc.
    # Which is DET NOUN VERB DET NOUN, the correct sequence!
    print(tag_scores)
```

    tensor([[-1.1389, -1.2024, -0.9693],
            [-1.1065, -1.2200, -0.9834],
            [-1.1286, -1.2093, -0.9726],
            [-1.1190, -1.1960, -0.9916],
            [-1.0137, -1.2642, -1.0366]])
    tensor([[-0.0462, -4.0106, -3.6096],
            [-4.8205, -0.0286, -3.9045],
            [-3.7876, -4.1355, -0.0394],
            [-0.0185, -4.7874, -4.6013],
            [-5.7881, -0.0186, -4.1778]])

### 練習: 文字レベルの特徴による LSTM を用いた品詞タグ付けモデルの拡張

上記の例では 各単語には埋め込み表現があり，その表現を系列モデルへの入力としています。
単語の埋め込み表現を 単語の文字から派生した表現で拡張してみましょう。
接辞のような文字レベルの情報は品詞に大きな影響を与えるので これは大きな助けになると期待されます。
例えば 接尾辞 *-ly* を持つ単語は 英語ではほとんど常に副詞としてタグ付けされます。

そのために $c_w$ を単語 $w$ の文字レベル表現とします。
$x_w$ を先の例と同様の単語の埋め込み表現とします。
すると 系列モデルへの入力は $x_w$ と $c_w$ の結合になります。
したがって  $x_w$ の次元が 5 で  $c_w$ の次元が 3 の場合，LSTM は 8 次元の入力を受け入れることになります。

文字レベルの表現を得るには 単語の文字に対して LSTM を行い $c_w$ をこの LSTM の最終隠れ層とします。
ヒント:

* 新モデルには 2 つ LSTM が必要です。
  最初の例のモデルは 品詞タグのスコアを出力するものです。新しいモデルは 各単語の文字レベルの表現を出力します。
* 文字単位ので系列モデルを行うには 文字を埋め込む必要があります。
  文字の埋め込みは、文字 LSTM への入力となります。

```python

```


### Imports

```python
# Imports

import sys
'''
!{sys.executable} -m pip install --user -q cirq
!{sys.executable} -m pip install --user -q google-api-python-client==1.8.0
'''
import cirq
import math
import random
import numpy as np
import sympy
!{sys.executable} --version
print("\nCirq version " + cirq.__version__)
```

    Python 3.7.7

    Cirq version 0.7.0

# Quantum Computing Simulation using Google's cirq library for Python
### Dmitri Friedenberg

Quantum programming languages are a major developing topic in the field of quantum computing. As the availability of different quantum computing systems increases, the need rises for a clear syntax with which to program a series of operations on an arbitrary quantum computer.

# Review of literature and high-level quantum programming proposals

Discussion about quantum pseudoprogramming began immediately after Shor's algorithm and Grover's algorithm each took the spotlight in 1994 and 1996 respectively. In "Conventions for quantum pseudocode" \[1\] in 1996, E. Knill's often cited paper began the initiative to develop a common method for programming quantum computers. Although a physical implementation was nowhere in sight, the need for adopting expressions for constructing a quantum system was immediately clear. A number of quantum programming language proposals followed in the decade after Knill's paper, well documented by Peter Selinger's "A brief survey of quantum programming languages" \[2\].

### Imperative languages

Within a few years of Knill's paper, several proposals for *imperative* quantum programming languages emerged. The first proposed imperative language was **Quantum Computation Language (QCL)**, introduced in 1998 in B. Ömer's Master's thesis, "A Procedural Formalism for Quantum Computing" \[3\]. His thesis presented a fully developed imperative language, with examples such as the QCL implementation of Coppersmith's algorithm of a fast quantum discrete Fourier Transform (Table 2.1):

```java
    operator dft(qureg q) { // main operator
        const n=#q; // set n to length of input
        int i; int j; // declare loop counters
        for i=0 to n-1 {
            for j=0 to i-1 { // apply conditional phase gates
                CPhase(2*pi/2^(i-j+1),q[n-i-1] & q[n-j-1]);
            }
            Mix(q[n-i-1]); // qubit rotation
        }
    flip(q); // swap bit order of the output
    }
```

Generally, initial proposals for quantum programming languages were imperative, likely due to the intuition that the languages are designed to manipulate physical objects (qubits and gates), more easily expressed in an object-oriented language. The imperative languages that followed included **Q Language** (an extension of C++, from "Toward an architecture for quantum programming", S. Bettelli 2002), which tracked quantum memory using a class "Qreg" to create registers; and **quantum Guarded-Command Language (qGCL)**, modeled after Edsger Dijkstra's "Guarded Command Language", defined by P. Zuliani in his thesis, "Quantum Programming" (2001).

### Functional languages

As functional programming has risen back into the zeitgeist of the last decade or two among the software engineering community, quantum computation has seen its share of attempts at functional languages. Selinger defined the **Quantum Programming Language (QPL)** in another paper from 2004, "Towards a quantum programming language" \[4\]. His paper describes QPL as, "*functional*, in the sense that each (atomic or composite) statement operates by transforming a specific set of inputs to outputs." The language implements natural solutions for issues like the no-cloning property, where the syntax of the language prevents an implied duplication of a quantum state. Selinger's paper also defines **Quantum flow charts (QFC)**, an excellent tool for illustrating a quantum functional program:

### Quantum Computing Simulators

Going further high-level, we come to quantum circuit simulators. Quantum computing simulators not only allow for the simple construction of circuits in a programming language, but may run an input vector through a circuit to simulate the probabalistic results moment by moment. For instance, take the Quantum Forier Transform (QFT) - a simulator can simply compute the Fourier transform classically, and update the qubit states (note that it *could* also properly simulate the full circuit for QFT). This allows for advanced simulations to include features such as noise, error rates, run-times for individual operations, and so on.

# Cirq

Finally, we come to the simulator of focus in this notebook. Cirq was announced as an open source Python framework for Noisy Intermediate Scale Quantum (NISQ) computers on Google's AI Blog, on July 18, 2018 \[5\], fully available at https://github.com/quantumlib/Cirq. Cirq is actively used on Google's "Bristlecone" quantum processor, and has even recently been incorporated into TensorFlow Quantum (also by Google), "an open-source library for the rapid prototyping of quantum ML models" \[6\]. TensorFlow Quantum is a very recent development just announced on March 9, 2020.

### Notable features

#### Graphical ASCII printing of circuits

```python
# This example is modified from https://github.com/quantumlib/Cirq .

# Pick a qubit.
qubit = cirq.GridQubit(0, 0)

# Create a circuit
circuit = cirq.Circuit(
    cirq.X(qubit)**0.5,  # Square root of NOT.
    cirq.measure(qubit, key='m')  # Measurement.
)
print("Circuit:")
print(circuit)

# Simulate the circuit several times.
simulator = cirq.Simulator()
result = simulator.run(circuit, repetitions=20)
print("Ran circuit 20 times. Results:")
print(result)
```

    Circuit:
    (0, 0): ───X^0.5───M('m')───
    Ran circuit 20 times. Results:
    m=00101001001100001011

#### Moment-based circuits

(credit [cirq.readthedocs.io]())

By grouping operations into "moments", Cirq constructs operators in parallel and better simulation of scheduled operations on real hardware. This also permits inspection of the state of a circuit at each moment:

```python
qubits = cirq.LineQubit.range(3)
circuit = cirq.Circuit(
    cirq.IdentityGate(num_qubits = 3).on(*qubits),
    cirq.H(qubits[0]),
    cirq.H(qubits[2]),
    cirq.CNOT(qubits[0], qubits[1]),
    cirq.CNOT(qubits[1], qubits[2])
)
print(circuit, "\n")

simulator = cirq.Simulator()
for i, step in enumerate(simulator.simulate_moment_steps(circuit)):
    print('state at step %d: %s' % (i, step.dirac_notation(3)) + "\n")
```

    0: ───I───H───@───────
          │       │
    1: ───I───────X───@───
          │           │
    2: ───I───H───────X───

    state at step 0: |000⟩

    state at step 1: 0.5|000⟩ + 0.5|001⟩ + 0.5|100⟩ + 0.5|101⟩

    state at step 2: 0.5|000⟩ + 0.5|001⟩ + 0.5|110⟩ + 0.5|111⟩

    state at step 3: 0.5|000⟩ + 0.5|001⟩ + 0.5|110⟩ + 0.5|111⟩

### Easily extensible operations and classes

Cirq is designed to be extensible with abstract classes for gates/operations, simulators, etc. This allows for creation of operations such as unitary operations corresponding to a complex function. The catch is that Cirq allows such operations not to be reflective of the full circuit - that is, the operation does not have to be the sum of normally-defined gates, but can simply hide a classical operation. In the next section, we will see how Cirq can use this to abstract the hardest part of Shor's algorithm.

# Shor's algorithm with Cirq

### Review

A brief review of the procedure for Shor's algorithm \[7\] is as follows:

___

We begin by picking a number $N$ to be factored.

##### Classical component

1. Pick a random number $a < N$.
2. Compute $\gcd(a, N)$. If $\gcd(a, N) \neq 1$, then this common divisor is a nontrivial factor of $N$, so the algorithm is done (though for large numbers it is likely that the algorithm can be run again on those factors). Otherwise, we proceed.
3. Use the **quantum component** to determine $r$, the period of the function $f(x) = a^x \mod N$.
4. If any of the following conditions are true, return to step 1.
    - $r = 0$
    - No candidate $r$ was found
    - $r$ is odd
    - $a^{\frac{r}{2}} \equiv -1 \mod N$
    - $\gcd(a^{\frac{r}{2}}+1\text{ mod }N, N) = 1$
5. We can factor $N$ by $\gcd(a^{\frac{r}{2}}+1\text{ mod }N, N)$.

##### Quantum component

We receive $a, N$ with which we must solve the period $r$ for $f(x) = a^x \mod N$. We define the unitary operation $$U_f |x, 0^q\rangle = |x, f(x)\rangle, \ \ f(x) = a^x \text{ mod } N.$$

1. Let $n_0$ be the number of bits in N. Let $n$ be the unique integer satisfying $N^2 \leq 2^n \leq 2N^2$. Initialize an input and output qubit register, with $n_0$ and $n$ qubits respectively, to the state $|\psi_0\rangle = |0\ldots 0\rangle_n|0\ldots 0\rangle_{n_0}$.
2. Let $q=2^n$. Prepare a superposition $$|\psi_1\rangle = \frac{1}{\sqrt{q}}\sum_{x=0}^{q-1}|x\rangle|0\rangle$$ by applying $q$ Hadamard gates.
3. Apply $U_f$ so that $|\psi_2\rangle = U_f|\psi_1\rangle.$
4. Measure the output register of $|\psi_2\rangle$ (recall this register has size $n_0$), and discard the result of that measurement (the point is to force the input register into a particular superposition). This puts the input register into state $$|\psi_3\rangle = \frac{1}{\sqrt{m}}\sum_{k=0}^{m-1}|x_0 + kr\rangle.$$
5. Apply the Quantum Fourier Transform (QFT) to $|\psi_3\rangle$ to obtain $|\psi_4\rangle = QFT|\psi_3\rangle$.
    - *Note: Many sources describe incorrectly describe this as the "inverse QFT". This confusion is due to the fact that QFT is the quantum analog of the inverse discrete Fourier transform - but it should properly be referred to as just "QFT". Still, Cirq follows the trend of calling this the "inverse QFT".*
6. Measure $y$ from the input register of $|\psi_4\rangle$.
7. Determine the continued fraction representation of $y/q$. Test each convergent $j'/r'$ in order, where $j'/r'$ is reduced to lowest terms. If at any point $r' < N$ and $$|\frac{y}{q}-\frac{j'}{r'}|\leq\frac{1}{2q}, $$ then $r'$ is the candidate value for the period. Return $r'$ to the classical component's step 3.

___

With this in mind, let's first examine how Cirq can deal with the circuit for modular exponentiation in Shor's algorithm, satisfying the unitary operation

$$U_f |x, 0^q\rangle = |x, f(x)\rangle, \ \ f(x) = a^x \text{ mod } N$$

This is not at all a simple circuit, and minimizing the number of operations to achieve this unitary function for variables $a, N$ has been the subject of much recent research and contention. Shor's landmark paper did not specify how this unitary operation could be achieved, not even discussing the number of qubits needed for the operation. Given that this operation is seen as **the bottle-neck** for applying Shor's algorithm, the debate over optimizing such a circuit is warranted.

In “Fast Quantum Modular Exponentiation Architecture for Shor’s Factorization Algorithm” \[8\] (2013), Pavlidis and Gizopoulos present a circuit with a depth near $2000n^2$ requiring $9n+2$ qubits, where $n$ represents the number of bits of the classical number being factored, where "the total quantum cost of the proposed design is $1600n^3$. The implementation is complicated, and this notebook will not attempt to program that circuit into Cirq. Instead, we'll use a simple trick to ignore the physical circuit requirements entirely, and simply *pretend* we have such a physical device ready for use. The following code creates an ArithmeticOperation in Cirq which sweeps all the troublesome circuitry under the rug.

```python
# Code modified from https://github.com/quantumlib/Cirq/blob/master/examples/shor.py

from typing import Callable, List, Optional, Sequence, Union

class ModularExp(cirq.ArithmeticOperation):
    def __init__(self, target: Sequence[cirq.Qid],
                 exponent: Union[int, Sequence[cirq.Qid]], base: int,
                 modulus: int) -> None:
        if len(target) < modulus.bit_length():
            raise ValueError(f'Register with {len(target)} qubits is too small '
                             f'for modulus {modulus}')
        self.target = target
        self.exponent = exponent
        self.base = base
        self.modulus = modulus

    def registers(self) -> Sequence[Union[int, Sequence[cirq.Qid]]]:
        return self.target, self.exponent, self.base, self.modulus

    def with_registers(
            self,
            *new_registers: Union[int, Sequence['cirq.Qid']],
    ) -> cirq.ArithmeticOperation:
        if len(new_registers) != 4:
            raise ValueError(f'Expected 4 registers (target, exponent, base, '
                             f'modulus), but got {len(new_registers)}')
        target, exponent, base, modulus = new_registers
        return ModularExp(target, exponent, base, modulus)

    def apply(self, *register_values: int) -> int:
        assert len(register_values) == 4
        target, exponent, base, modulus = register_values
        if target >= modulus:
            return target
        return target ^ (base**exponent) % modulus

    def _circuit_diagram_info_(
            self,
            args: cirq.CircuitDiagramInfoArgs,
    ) -> cirq.CircuitDiagramInfo:
        assert args.known_qubits is not None
        wire_symbols: List[str] = []
        t, e = 0, 0
        for qubit in args.known_qubits:
            if qubit in self.target:
                if t == 0:
                    if isinstance(self.exponent, Sequence):
                        e_str = 'e'
                    else:
                        e_str = str(self.exponent)
                    wire_symbols.append(
                        f'ModularExp(t*{self.base}**{e_str} % {self.modulus})')
                else:
                    wire_symbols.append('t' + str(t))
                t += 1
            if isinstance(self.exponent, Sequence) and qubit in self.exponent:
                wire_symbols.append('e' + str(e))
                e += 1
        return cirq.CircuitDiagramInfo(wire_symbols=tuple(wire_symbols))
```

This may seem oddly simple for such a complicated circuit. The magic is in the method:

```
    def apply(self, *register_values: int) -> int:
        assert len(register_values) == 4
        target, exponent, base, modulus = register_values
        if target >= modulus:
            return target
        return (target * base**exponent) % modulus
```

Here, we see that this Cirq operation really just wraps `a**x % N`. Of course, this still means we get a classical run-time when factoring numbers, but that is to be expected.

### The Algorithm

```python
'''
Attempt to factor N with a single run of Shor's algorithm.
If this returns None, then it probablistically failed and
must be run again.
If this returns -1, then the number is prime.
'''

def shor_factor(N, seed=None, verbose=False):
    if type(N) is not int:
        raise TypeError("n must be an integer.")
    if N > (2 ** 30):
        raise ValueError("Number is too large. Try n <= 2^30.")
    if N < 1:
        raise ValueError("Number must be positive integer greater than 1.")

    if N % 2 == 0:
        if verbose:
            print(f"{N} has trivial factor of 2.")
        return 2, N // 2

    # using sympy.isprime is certainly 'cheating' - but Shor's algorithm
    # doesn't work on prime numbers, so we can save some wasted effort here.
    if sympy.isprime(N):
        if verbose:
            print(f"{N} is prime. Aborting.")
        return -1

    '''
    1. Pick a random number  𝑎<𝑁
    '''
    random.seed(seed)
    a = random.randint(2, N-1)
    if verbose:
        print(f"Chose random number a={a}.")

    '''
    2. Compute  gcd(𝑎,𝑁) . If  gcd(𝑎,𝑁)≠1 , then this common divisor
    is a nontrivial factor of  𝑁 , so the algorithm is done
    (though for large numbers it is likely that the algorithm
    can be run again on those factors). Otherwise, we proceed.
    '''
    gcd = math.gcd(N, a)
    if gcd != 1:
        if verbose:
            print(f"gcd({N}, {a}) is {gcd}, which is a trivial factor.")
        return gcd, N // gcd
    if verbose:
        print(f"a is relatively prime to N.")

    '''
    3. Use the quantum component to determine  𝑟 ,
    the period of the function  𝑓(𝑥)=𝑎^𝑥 mod𝑁 .
    '''
    if verbose:
        print(f"Finding order of `{a}**x % {N}`.")
    r = quantum_find_order(a, N, seed, verbose)
    if verbose:
        print(f"Quantum routine returned period r={r}.")

    '''
    4. If any of the following conditions are true, return to step 1.
    '''
    # No candidate 𝑟′ was found
    # 𝑟 is odd
    # 𝑎^(𝑟/2)≡−1mod𝑁
    # gcd(𝑎^(r/2),𝑁)=1
    if (r == 0 or
            r is None or
            r % 2 == 1):
        if verbose:
            print(f"The period r={r} failed on classical step 4. Try algorithm again.")
        return None

    c = (a**(r // 2)) % N
    d = math.gcd(c+1, N)
    if (c % N == N-1) or d == 1:
        if verbose:
            print(f"The period r={r} failed on classical step 4. Try algorithm again.")
        return None

    '''
    5. We can factor  𝑁  into  𝑔𝑐𝑑(𝑎^(𝑟/2)±1,𝑁) .
    '''
    if verbose:
        print("Algorithm succeeded. Returning factors.")
    return d, N // d

'''
The quantum component of Shor's algorithm. Returns 'r',
the candidate period.
'''
def quantum_find_order(a, N, seed=None, verbose=False):
    n_0 = N.bit_length()
    n = 2 * n_0 + 3

    '''
    1. Initialize an input and output qubit register,
    with  𝑛  and  𝑛0  qubits respectively, to
    the state  |𝜓0⟩=|0…0⟩𝑛|0…0⟩𝑛0 .
    '''
    input_qubits = cirq.LineQubit.range(n)
    output_qubits = cirq.LineQubit.range(n, n + n_0)
    order_circuit = cirq.Circuit()

    '''
    2. Let  𝑞=2^𝑛 . Prepare a superposition
    |𝜓1⟩=1𝑞⎯⎯√∑𝑥=0𝑞−1|𝑥⟩|0⟩

    by applying  𝑞  Hadamard gates.
    '''
    q = 2 ** n
    order_circuit.append(cirq.H.on_each(*input_qubits))

    '''
    3. Apply  𝑈𝑓  so that  |𝜓2⟩=𝑈𝑓|𝜓1⟩.
    '''
    order_circuit.append(
        ModularExp(output_qubits, input_qubits, a, N)
    )

    '''
    4. Measure the output register of  |𝜓2⟩
    (recall this register has size  𝑛 ),
    and discard the result of that measurement
    (the point is to force the input register into
    a particular superposition). This puts the input
    register into state |𝜓3⟩.
    '''
    order_circuit.append(cirq.measure(*output_qubits, key='output'))

    '''
    5. Apply the Quantum Fourier Transform (QFT)
    to  |𝜓3⟩  to obtain  |𝜓4⟩=𝑄𝐹𝑇|𝜓3⟩ .
    '''
    order_circuit.append(cirq.QFT(*input_qubits, inverse=True))

    '''
    6. Measure  𝑦  from the input register of  |𝜓4⟩
    '''
    order_circuit.append(cirq.measure(*input_qubits, key='input'))
    if verbose:
        print("Generating order-finding circuit:\n")
        print(order_circuit, "\n")
    simulator = cirq.Simulator(seed=seed)
    input_result = simulator.run(order_circuit).measurements['input'][0]
    y = int("".join(str(x) for x in input_result), 2)
    if verbose:
        print(f"Circuit returned value of input register, y={y}")

    '''
    7. Determine the continued fraction representation of  𝑦/𝑞 .
    Test each convergent  𝑗′/𝑟′  in order, where  𝑗′/𝑟′  is reduced
    to lowest terms. If at any point  𝑟′<𝑁  and |𝑦/𝑞−𝑗′/𝑟′|≤1/2𝑞,
    then  𝑟′  is the candidate value for the period.
    Return  𝑟′  to the classical component's step 3.
    '''
    def continued_fraction(num, denom):
        res = []
        quo, rem = divmod(num, denom)
        while rem != 0:
            res = res + [quo]
            quo, rem = divmod(denom, rem)
            denom = (denom-rem)//quo
        return res + [quo]

    def cf_to_frac(cf):
        num, denom = 1, 0
        for u in reversed(cf):
            num, denom = denom + (num * u), num
        return num, denom

    cf = continued_fraction(y, q)
    if verbose:
        print(f"Continued fraction for {y}/{q} is {cf}.")

    # test each convergent
    for i in range(len(cf)):
        j, r = cf_to_frac(cf[0:i+1])
        if math.fabs((y/q)-(j/r)) <= (1/(2*q)):
            if verbose:
                print(f"Using convergent j/r = {j}/{r}. Returning to classical routine.")
            if r == 1:
                return 0
            return r
    # all convergents failed
    if verbose:
        print("All convergents failed. Returning to classical routine.")
    return None
```

Now, we are ready to run the algorithm. Note that this is a *very expensive* algorithm on classical systems - so we use a rather low $N$ to test the algorithm, or else we wait an eternity to factor a number like 99.

```python
for i in range(5):
    res = shor_factor(21, seed=i+13, verbose=True)
    if res:
        print(res)
        break
    print("-"*30 + "\n")
```

    Chose random number a=10.
    a is relatively prime to N.
    Finding order of `10**x % 21`.
    Generating order-finding circuit:

    0: ────H───e0─────────────────────────QFT^-1────────M('input')───
               │                          │             │
    1: ────H───e1─────────────────────────#2────────────M────────────
               │                          │             │
    2: ────H───e2─────────────────────────#3────────────M────────────
               │                          │             │
    3: ────H───e3─────────────────────────#4────────────M────────────
               │                          │             │
    4: ────H───e4─────────────────────────#5────────────M────────────
               │                          │             │
    5: ────H───e5─────────────────────────#6────────────M────────────
               │                          │             │
    6: ────H───e6─────────────────────────#7────────────M────────────
               │                          │             │
    7: ────H───e7─────────────────────────#8────────────M────────────
               │                          │             │
    8: ────H───e8─────────────────────────#9────────────M────────────
               │                          │             │
    9: ────H───e9─────────────────────────#10───────────M────────────
               │                          │             │
    10: ───H───e10────────────────────────#11───────────M────────────
               │                          │             │
    11: ───H───e11────────────────────────#12───────────M────────────
               │                          │             │
    12: ───H───e12────────────────────────#13───────────M────────────
               │
    13: ───────ModularExp(t*10**e % 21)───M('output')────────────────
               │                          │
    14: ───────t1─────────────────────────M──────────────────────────
               │                          │
    15: ───────t2─────────────────────────M──────────────────────────
               │                          │
    16: ───────t3─────────────────────────M──────────────────────────
               │                          │
    17: ───────t4─────────────────────────M──────────────────────────

    Circuit returned value of input register, y=4096
    Continued fraction for 4096/8192 is [0, 2].
    Using convergent j/r = 1/2. Returning to classical routine.
    Quantum routine returned period r=2.
    The period r=2 failed on classical step 4. Try algorithm again.
    ------------------------------

    Chose random number a=5.
    a is relatively prime to N.
    Finding order of `5**x % 21`.
    Generating order-finding circuit:

    0: ────H───e0────────────────────────QFT^-1────────M('input')───
               │                         │             │
    1: ────H───e1────────────────────────#2────────────M────────────
               │                         │             │
    2: ────H───e2────────────────────────#3────────────M────────────
               │                         │             │
    3: ────H───e3────────────────────────#4────────────M────────────
               │                         │             │
    4: ────H───e4────────────────────────#5────────────M────────────
               │                         │             │
    5: ────H───e5────────────────────────#6────────────M────────────
               │                         │             │
    6: ────H───e6────────────────────────#7────────────M────────────
               │                         │             │
    7: ────H───e7────────────────────────#8────────────M────────────
               │                         │             │
    8: ────H───e8────────────────────────#9────────────M────────────
               │                         │             │
    9: ────H───e9────────────────────────#10───────────M────────────
               │                         │             │
    10: ───H───e10───────────────────────#11───────────M────────────
               │                         │             │
    11: ───H───e11───────────────────────#12───────────M────────────
               │                         │             │
    12: ───H───e12───────────────────────#13───────────M────────────
               │
    13: ───────ModularExp(t*5**e % 21)───M('output')────────────────
               │                         │
    14: ───────t1────────────────────────M──────────────────────────
               │                         │
    15: ───────t2────────────────────────M──────────────────────────
               │                         │
    16: ───────t3────────────────────────M──────────────────────────
               │                         │
    17: ───────t4────────────────────────M──────────────────────────

    Circuit returned value of input register, y=0
    Continued fraction for 0/8192 is [0].
    Using convergent j/r = 0/1. Returning to classical routine.
    Quantum routine returned period r=0.
    The period r=0 failed on classical step 4. Try algorithm again.
    ------------------------------

    Chose random number a=8.
    a is relatively prime to N.
    Finding order of `8**x % 21`.
    Generating order-finding circuit:

    0: ────H───e0────────────────────────QFT^-1────────M('input')───
               │                         │             │
    1: ────H───e1────────────────────────#2────────────M────────────
               │                         │             │
    2: ────H───e2────────────────────────#3────────────M────────────
               │                         │             │
    3: ────H───e3────────────────────────#4────────────M────────────
               │                         │             │
    4: ────H───e4────────────────────────#5────────────M────────────
               │                         │             │
    5: ────H───e5────────────────────────#6────────────M────────────
               │                         │             │
    6: ────H───e6────────────────────────#7────────────M────────────
               │                         │             │
    7: ────H───e7────────────────────────#8────────────M────────────
               │                         │             │
    8: ────H───e8────────────────────────#9────────────M────────────
               │                         │             │
    9: ────H───e9────────────────────────#10───────────M────────────
               │                         │             │
    10: ───H───e10───────────────────────#11───────────M────────────
               │                         │             │
    11: ───H───e11───────────────────────#12───────────M────────────
               │                         │             │
    12: ───H───e12───────────────────────#13───────────M────────────
               │
    13: ───────ModularExp(t*8**e % 21)───M('output')────────────────
               │                         │
    14: ───────t1────────────────────────M──────────────────────────
               │                         │
    15: ───────t2────────────────────────M──────────────────────────
               │                         │
    16: ───────t3────────────────────────M──────────────────────────
               │                         │
    17: ───────t4────────────────────────M──────────────────────────

    Circuit returned value of input register, y=4096
    Continued fraction for 4096/8192 is [0, 2].
    Using convergent j/r = 1/2. Returning to classical routine.
    Quantum routine returned period r=2.
    Algorithm succeeded. Returning factors.
    (3, 7)

# Sources:

\[1\]: E. Knill. Conventions for quantum pseudocode. Technical Report LAUR-96-2724, LANL, 1996. (https://pdfs.semanticscholar.org/60d1/e63ca31555ec7013c5eb9a8a63788398fd14.pdf)

\[2\]: Selinger, Peter. (2004). A Brief Survey of Quantum Programming Languages. 1-6. (https://www.mscs.dal.ca/~selinger/papers/flops04.pdf)

\[3\]: B. Ömer, A Procedural Formalism for Quantum Computing, Master thesis (computing science), Technical University of Vienna, 1998. (http://tph.tuwien.ac.at/~oemer/doc/qcldoc.pdf)

\[4\]: Selinger, Peter. “Towards a Quantum Programming Language.” Mathematical Structures in Computer Science, vol. 14, no. 4, Aug. 2004, pp. 527–86. Cambridge Core, doi:10.1017/S0960129504004256. (https://www.mscs.dal.ca/~selinger/papers/qpl.pdf)

\[5\]: "Announcing Cirq: An Open Source Framework for NISQ Algorithms." (https://ai.googleblog.com/2018/07/announcing-cirq-open-source-framework.html)

\[6\]: "Announcing TensorFlow Quantum: An Open Source Library for Quantum Machine Learning." (https://ai.googleblog.com/2020/03/announcing-tensorflow-quantum-open.html)

\[7\]: Shor, Peter W. “Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer.” SIAM Journal on Computing, vol. 26, no. 5, Oct. 1997, pp. 1484–509. arXiv.org, doi:10.1137/S0097539795293172. (https://arxiv.org/pdf/quant-ph/9508027.pdf)

\[7\]: Pavlidis, Archimedes, and Dimitris Gizopoulos. “Fast Quantum Modular Exponentiation Architecture for Shor’s Factorization Algorithm.” ArXiv:1207.0511 [Quant-Ph], Nov. 2013. arXiv.org, http://arxiv.org/abs/1207.0511. (https://arxiv.org/pdf/1207.0511.pdf)

\[8\]: Cirq on GitHub. (https://github.com/quantumlib/Cirq/blob/master/)