```python

```

Like in the previous notebook, we'll use example data for this notebook. The notebook assumes it's stored in a folder next to the notebook called `data`. You can download these data from Canvas (in which case you'll have to unzip it in the `data` folder), or the course GitHub page.

When you upload these data to CoLab, make sure that you're pointing the data in the right direction. If you've saved the data to your `Colab Notebooks/data` folder, that is the following location: `/content/drive/My Drive/Colab Notebooks/data/`. The code below tries to set it up so that you don't need to worry about it.

```python
# this will ask you to authenticate with Google
from google.colab import drive
drive.mount('/content/drive')

import os
os.chdir('/content/drive/My Drive/Colab Notebooks/')

## we'll also have to install the library to access nifti files:
!pip install nibabel
```

# Neuroimaging week 4: Preprocessing
This week's lab is about preprocessing of fMRI data. Preprocessing of (f)MRI data is quite a complex topic, involving techniques from signal-processing (high-pass filtering), linear algebra/statistics (prewhitening/autocorrelation correction), and optimization (image registration). Additionally, there is ongoing discussion about which preprocessing steps are necessary/sufficient and what preprocessing parameters are optimal. As always, this depends on your specific research question, the type and quality of your data, and the type of analysis.

In this week, we'll discuss a couple (not all!) of preprocessing steps that are common in univariate fMRI analyses. We won't discuss distortion-correction ("unwarping") and registration procedures; please use the videos, the book, and the lecture slides to understand these concepts.

### Contents
1. Introduction: the t-value (yet again)
2. Temporal filtering
3. Prewhitening/autocorrelation correction
4. Spatial filtering
5. Outlier censoring
6. Motion correction and motion filtering

### What you'll learn
Specifically, after this lab, you'll ...

- be able to explain the influence of preprocessing on the measured effects using the t-value formula
- understand (the advantage of) temporal filtering from both the time-domain and frequency-domain
- understand the necessity of prewhitening given the assumptions of the GLM
- understand the advantage of spatial filtering (smoothing)
- under how to handle outliers
- explain the necessity of motion correction and the advantage of motion regression
- know how to implement the concepts above in Python

**Estimated time needed to complete**: 8 hours

**No Deadline**

## 1. Introduction
As we said before, preprocessing is a topic that almost warrants its own course. Nonetheless, we'll try to show you (and let you practice with) some of the most common and important preprocessing operations. Additionally, we'll introduce the concept of the fast fourier transform, which allows us to analyze our signal in the frequency domain, which helps to understand several preprocessing steps, such as temporal filtering.

### 1.1. The (conceptual) t-value formula -- yet again
The previous two weeks, you have learned that, essentially, we want to find large effects (calculated as t-values) of our contrasts by optimizing various parts of the t-value formula. Conceptually, the t-value formula can be written as:

\begin{align}
t\mathrm{-value} = \frac{\mathrm{effect}}{\mathrm{uncertainty}} = \frac{\mathrm{effect}}{\sqrt{\mathrm{noise} \cdot \mathrm{design\ variance}}} = \frac{\mathbf{c}\hat{\beta}}{\sqrt{\hat{\sigma}^{2}\mathbf{c}(X'X')^{-1}\mathbf{c}'}}
\end{align}

Last week you've learned that by ensuring low design variance (through *high* predictor variance and *low* predictor correlations) leads to larger normalized effects (higher t-values). This week, we will discuss the other term of the denominator of this formula: the noise (also called the residual variance or unexplained variance), which is defined in the t-value formula as follows:

\begin{align}
\mathrm{noise} = \frac{SSE}{\mathrm{DF}} = \frac{\sum_{i=1}^{N}(y_{i} - \hat{y_{i}})^{2}}{N - P}
\end{align}

Through preprocessing, we aim to reduce the difference between our prediction ($\hat{y}$) and our true signal ($y$), thus reducing the noise-term of the formula and thereby optimizing our normalized effects.

### 1.2. The two approaches of preprocessing

Basically, there are *two* ways to preprocess your data:
1. Manipulating the signal ($y$) **directly** *before* fitting your GLM-model;
2. Including "noise predictors" in your design ($X$) when fitting your model;

Often, preprocessing steps can be done both by method 1 (manipulating the signal directly) and by method 2 (including noise predictors). For example, one of the videos showed that you could apply a high-pass filter by applying a "gaussian weighted running line smoother" (the method FSL employs) *directly* on the signal (method 1) **or** you could add "low-frequency (drift) predictors" to the design matrix (method 2; in the video they used a 'discrete cosine basis set'; the SPM method). In practice, both methods often yield very similar resuls. The most important thing to understand is that both methods are trying to accomplish the same goal: reduce the noise term of the model.

First, we will discuss how temporal and spatial filtering can *directly* filter the signal (method 1) to reduce error. Later in the tutorial, we will discuss including adding outlier-predictors and motion-predictors to the design to reduce noise (method 2).

## 2. Temporal filtering
In this section, we will discuss how temporal filtering of the voxel signals may greatly reduce the error term. Along the way, we will also explain how we can look at the representation of the signal in the frequency domain (using the fourier transform) to give us an idea about the nature of the noise components in our data.

### 2.1. A short primer on the frequency domain and the fourier transform
Thus far, we've always looked at our fMRI-signal as activity that varies across **time**. In other words, we're always looking at the signal in the *time domain*. However, there is also a way to look at a signal in the *frequency domain* (also called 'spectral domain') through transforming the signal using the *Fourier transform*.

Basically, the fourier transform calculates to which degree sine waves of different frequencies are present in your signal. If a sine wave of a certain frequency (let's say 2 hertz) is (relatively) strongly present in your signal, it will have a (relatively) high *power* in the frequency domain. Thus, looking at the frequency domain of a signal can tell you something about the frequencies of the (different) sources underlying your signal.

This may sound quite abstract, so let's look at some examples.

```python
# start with importing the python packages we'll need
import numpy as np
from numpy.linalg import lstsq
import nibabel as nib
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt
%matplotlib inline

def double_gamma(x, lag=6, a2=12, b1=0.9, b2=0.9, c=0.35, scale=True):

    a1 = lag
    d1 = a1 * b1
    d2 = a2 * b2
    hrf = np.array([(t/(d1))**a1 * np.exp(-(t-d1)/b1) - c*(t/(d2))**a2 * np.exp(-(t-d2)/b2) for t in x])

    if scale:
        hrf = (1 - hrf.min()) * (hrf - hrf.min()) / (hrf.max() - hrf.min()) + hrf.min()
    return hrf

def create_sine_wave(timepoints, frequency=1,amplitude=1, phase=0):
    return amplitude * np.sin(2*np.pi*frequency*timepoints + phase)
```

Sine waves are oscillating signals that have (for our purposes) two important characteristics: their *frequency* and their *amplitude*. Frequency reflects how fast a signal is oscillating (how many cycles it completes in a given time period) and the amplitude is the (absolute) height of the peaks and troughs of the signal. To illustrate this, we generate a couple of sine-waves (with a sampling rate of 500 Hz, i.e., 500 samples per second) with different amplitudes and frequencies, which we plot below:

```python
max_time = 5
sampling_rate = 500
timepoints = np.arange(0, max_time, 1.0 / sampling_rate)

amplitudes = np.arange(1, 4)
frequencies = np.arange(1, 4)
sines = []

fig, axes = plt.subplots(3, 3, sharex=True, sharey=True, figsize=(13, 8))
for i, amp in enumerate(amplitudes):

    for ii, freq in enumerate(frequencies):
        this_ax = axes[i, ii]

        if ii == 0:
            this_ax.set_ylabel('Activity (A.U.)')

        if i == 2:
            this_ax.set_xlabel('Time (seconds)')

        sine = create_sine_wave(timepoints, frequency=freq, amplitude=amp)
        sines.append((sine, amp, freq))
        this_ax.plot(timepoints, sine)
        this_ax.set_xlim(0, 5)
        this_ax.set_title('Sine with amp = %i and freq = %i' % (amp, freq))
        this_ax.set_ylim(-3.5, 3.5)

fig.tight_layout()
```

As you can see, the signals vary in their amplitude (from 1 to 3) and their frequency (from 1 - 3). Make sure you understand these characteristics! Now, we are going to use the fast fourier transform to plot the same signals in the *frequency domain*. We're not going to use a function to compute the FFT-transformation, but we're going to use a function that computes the "power spectrum density" directly (which makes life a little bit easier): the `periodogram` function from `scipy.signal`:

```python
from scipy.signal import periodogram
```

Now, the `periodogram` function takes two arguments, the signal and the sampling frequency (the sampling rate in Hz with which you recorded the signal), and returns both the reconstructed frequencies and their associated power values. An example:

```python
freqs, power = periodogram(some_signal, 1000)  # sampling_rate = 1000 Hz
```

We'll use the `periodogram` function to plot the 9 sine-waves (from the previous plot) again, but this time in the frequency domain:

```python
fig, axes = plt.subplots(3, 3, sharex=True, sharey=True, figsize=(13, 8))

for i, ax in enumerate(axes.flatten()):
    sine, amp, freq = sines[i]
    title = 'Sine with amp = %i and freq = %i' % (amp, freq)
    freq, power = periodogram(sine, sampling_rate)
    ax.plot(freq, power)
    ax.set_xlim(0, 4)
    ax.set_xticks(np.arange(5))
    ax.set_ylim(0, 25)

    if i > 5:
        ax.set_xlabel('Frequency (Hz)')

    if i % 3 == 0:
        ax.set_ylabel('Power')
    ax.set_title(title)

fig.tight_layout()
```

As you can see, the frequency domain correctly 'identifies' the amplitudes and frequencies from the signals. But the real 'power' from fourier transforms is that they can reconstruct a signal in *multiple underlying oscillatory sources*. Let's see how that works. We're going to load in a time-series recorded for 5 seconds of which we don't know the underlying oscillatory sources. First, we'll plot the signal in the time-domain:

```python
mystery_signal = np.load('data/mystery_signal.npy')
plt.figure(figsize=(15, 5))
plt.plot(np.arange(0, 5, 0.001), mystery_signal)
plt.title('Time domain', fontsize=25)
plt.xlim(0, 5)
plt.xlabel('Time (sec.)', fontsize=15)
plt.ylabel('Activity (A.U.)', fontsize=15)
plt.show()
```

ToDo

It's hard to see which frequencies (and corresponding amplitudes) are present in this 'mystery signal'. Get the frequencies and power of the signal using the `periodogram` function (you have to deduce the sampling rate of the signal yourself! It is *not* the variable `sampling_rate` from before). Set the x-limit of the x-axis to (0, 8) (`plt.xlim(0, 8)`). Also, give the plot appropriate labels for the axes.

```python
freqs, pows = periodogram(mystery_signal, fs=mystery_signal.size / 5)
plt.figure(figsize=(15, 5))
plt.plot(freqs, pows)
plt.xlim(0, 8)
plt.ylabel('Power')
plt.xlabel('Frequency (Hz)')
```

Now you know that you can use visualization of the signal in the frequency domain to help you understand from which underlying frequencies your signal is built up. Unfortunately, real fMRI data is not so 'clean' as the simulated sine waves we have used here, but the frequency representation of the fMRI signal can still tell us a lot about the nature and contributions of different (noise- and signal-related) sources!

### 2.2. Frequency characteristics of fMRI data
Now, we will load a (much noisier) voxel signal and the corresponding design-matrix (which has just one predictor apart from the intercept). The signal was measured with a TR of 2 seconds and contains 200 volumes (timepoints). The predictor reflects an experiment in which we showed 20 stimuli in intervals of 20 seconds (i.e., one stimulus every 20 seconds).

We'll plot both the signal ($y$) and the design-matrix ($X$; without intercept):

```python
with np.load('data/drift_data.npz') as data:
    X, sig = data['X'], data['sig']

plt.figure(figsize=(15, 8))
plt.subplot(2, 1, 1)
plt.plot(sig)
plt.xlim(0, 200)
plt.title('Signal in time domain', fontsize=20)
plt.ylabel('Activity (a.u.)', fontsize=15)

plt.subplot(2, 1, 2)
plt.plot(np.arange(sig.size), X[:, 1], c='tab:orange')
plt.title('Predictor in time domain', fontsize=20)
plt.xlabel('Time (TR)', fontsize=15)
plt.xlim(0, 200)
plt.ylabel('Activity (a.u.)', fontsize=15)
plt.ylim(-0.5, 1.5)
plt.tight_layout()

```

ToDo (2 points)

Run linear regression using the variable `X` (which already contains an intercept) to explain the variable `sig`. Calculate the model's MSE, and store this in a variable named `mse_no_filtering`. Then, in the next code-cell, plot the signal and the predicted signal ($\hat{y}$) in a single figure. Give the axes sensible labels and add a legend.

**Tip** (feel free to ignore): This tutorial, you'll be asked to compute t-values, R-squared, and MSE of several models quite some times. To make your life easier, you could (but certainly don't have to!) write a function that runs, for example, linear regression and returns the R-squared, given a design (X) and signal (y). For example, this function could look like:

```python
def compute_mse(X, y):
    # you implement the code here (run lstsq, calculate yhat, etc.)
    ...
    # and finally, after you've computed the model's MSE, return it
    return r_squared
```

If you're ambitious, you can even write a single function that calculates t-values, MSE, and R-squared. This could look something like this:

```python
def compute_all_statistics(X, y, cvec):
    # Implement everything you want to know and return it
    ...
    return t_value, MSE, r_squared # and whatever else you've computed!
```

Doing this will save you a lot of time and may prevent you from making unneccesary mistakes (like overwriting variables, typos, etc.). Lazy programmers are the best programmers!

(Note: writing these functions is *optional*!)

```python
b = np.linalg.lstsq(X, sig, rcond=None)[0]
yhat = X.dot(b)
mse_no_filtering = .....

assert(np.round(mse_no_filtering, 3) == 2.187)

```

```python
# Implement your y/yhat plot here

plt.figure(figsize=(15, 5))
plt.plot(sig)
plt.plot(yhat)
plt.xlim(0, 200)
plt.title("Signal and predicted signal", fontsize=20)
plt.xlabel("Time (TR)", fontsize=15)
plt.ylabel("Activation (A.U.)", fontsize=15)
plt.legend(['Signal', 'Predicted signal'], fontsize=15)
```

ToThink

In your plot above, you should see that the fit of your model is "off" due to some low frequency "drift". Name two potential causes of drift.

1. Slowly decreasing homogeneity of the magnetic field (e.g. due to subject movement)
2. Increasing thermal noise (i.e., the scanner warms up)

Note: just "subject movement" or respiration/cardiac signal is *NOT* correct.

### 2.3. High-pass filtering of fMRI data
From the previous ToDo, you probably noticed that the fit of the predictor to the model was not very good. The cause for this is the slow 'drift' - a low-frequency signal - that prevents the model from a good fit. Using a high-pass filter - meaning that you *remove* the low-frequency signals and thus *pass only the high frequencies* - can, for this reason, improve the model fit. But before we go on with actually high-pass filtering the signal, let's take a look at the frequency domain representation of our voxel signal:

```python
plt.figure(figsize=(17, 5))
TR = 2
sampling_frequency = 1 / TR  # our sampling rate is 0.5, because our TR is 2 sec!
freq, power = periodogram(sig, fs=0.5)
plt.plot(freq, power)
plt.xlim(0, freq.max())
plt.xlabel('Frequency (Hz)', fontsize=15)
plt.ylabel('Power (dB)', fontsize=15)
plt.axvline(x=0.01,color='r',ls='dashed', lw=2)

```

In the frequency-domain plot above, you can clearly see a low-frequency drift component at frequencies approximately below 0.01 Hz (i.e., left of the dashed red line).

ToThink

Apart from the low frequency drift component around 0.01 Hz, there is also a component visible at 0.05 Hz. What does this component represent? Please explain (concisely).

**This reflects our expected response to the stimuli! We presented a stimulus every 20 seconds (10 TRs), which represents a frequency of 0.05!**

Now, let's get rid of that pesky low frequency drift that messes up our model! There is no guideline on how to choose the cutoff of your high-pass filter, but most recommend to use a cutoff of 100 seconds (i.e., of 0.01 Hz). This means that any oscillation slower than 100 seconds (one cycle in 100 seconds) is removed from your signal.

Anyway, as you've seen in the videos, there are many different ways to high-pass your signal (e.g., frequency-based filtering methods vs. time-based filtering methods). Here, we demonstrate a time-based 'gaussian running line smoother', which is used in FSL. As you've seen in the videos, this high-pass filter is estimated by convolving a gaussian "kernel" with the signal (taking the element-wise product and summing the values) across time, which is schematically visualized in the image below:

One implementation of this filter is included in the scipy "ndimage" subpackage. Let's import it\*:

---
\***Note**: if you're going to restart your kernel during the lab for whatever reason, make sure to re-import this package to avoid `NameErrors` (i.e., the error that you get when you call a function that isn't imported).

```python
from scipy.ndimage import gaussian_filter
```

The `gaussian_filter` function takes two mandatory input: some kind of (n-dimensional) signal and a cutoff, "sigma", that refers to the width of the gaussian filter in standard deviations. "What? We decided to define our cutoff in seconds (or, equivalently, Hz), right?", you might think. For some reason neuroimaging packages seem to define cutoff for their temporal filters in **seconds** while more 'low-level' filter implementations (such as in scipy) define cutoffs (of gaussian filters) in **the width of the gaussial filter**, i.e., **sigma**. Fortunately, there is an easy way to (approximately) convert a cutoff in seconds to a cutoff in sigma, given a particular TR (in seconds):

\begin{align}
\sigma \approx \frac{\mathrm{cutoff}_{sec}}{2 \cdot \mathrm{TR}_{sec}}
\end{align}

ToDo

Suppose I acquire some fMRI data (200 volumes) with a sampling frequency of 0.25 Hz and I would like to apply a high-pass filter of 80 seconds. What sigma should I choose? Calculate sigma and store it in a variable named `sigma_todo`.

```python
sigma_todo = ......

np.testing.assert_equal(sigma_todo, 10)

```

Importantly, the gaussian filter does not return the filtered signal itself, but the estimated low-frequency component of the data. As such, to filter the signal, we have to subtract this low-frequency component from the original signal to get the filtered signal!

Below, we estimate the low-frequency component using the high-pass filter first and plot it together with the original signal, which shows that it accurately captures the low-frequency drift (upper plot). Then, we subtract the low-frequency component from the original signal to create the filtered signal, and plot it together with the original signal to highlight the effect of filtering (lower plot):

```python
TR = 2.0
sigma_hp = 100 / (2 * TR)
filt = gaussian_filter(sig, sigma_hp)

plt.figure(figsize=(17, 10))

plt.subplot(2, 1, 1)
plt.plot(sig, lw=2)
plt.plot(filt, lw=4)
plt.xlim(0, 200)
plt.legend(['Original signal', 'Low-freq component'], fontsize=20)
plt.title("Estimated low-frequency component using HP-filter", fontsize=25)
plt.ylabel("Activation (A.U.)", fontsize=20)

filt_sig = sig - filt

plt.subplot(2, 1, 2)
plt.plot(sig, lw=2)
plt.plot(filt_sig, lw=2, c='tab:green')
plt.xlim(0, 200)
plt.legend(['Original signal', 'Filtered signal'], fontsize=20)
plt.title("Effect of high-pass filtering", fontsize=25)
plt.xlabel("Time (TR)", fontsize=20)
plt.ylabel("Activation (A.U.)", fontsize=20)

plt.tight_layout()
plt.show()
```

The signal looks much better, i.e., it doesn't display much drift anymore. But let's check this by plotting the original and filtered signal in the frequency domain:

```python
plt.figure(figsize=(17, 5))
freq, power = periodogram(sig, fs=0.5)
plt.plot(freq, power, lw=2)

freq, power = periodogram(filt_sig, fs=0.5)
plt.plot(freq, power, lw=2)
plt.xlim(0, freq.max())
plt.ylabel('Power (dB)', fontsize=15)
plt.xlabel('Frequency (Hz)', fontsize=15)
plt.title("The effect of high-pass filtering in the frequency domain", fontsize=20)
plt.legend(["Original signal", "Filtered signal"], fontsize=15)
plt.show()
```

Sweet! It seems that the high-pass filtering worked as expected! But does it really improve our model fit?

ToDo

We've claimed several times that high-pass filtering improves model fit, but is that really the case in our case? To find out, fit the same design (variable `X`) on the filtered signal (variable `filt_sig`) using linear regression. Calculate MSE and store it in the variable `mse_with_filter`.

```python
# your code here, ending in

mse_with_filter = ....

assert(np.round(mse_with_filter, 3) == 0.971)
```

ToDo

So far, we've filtered only a single (simulated) voxel timeseries. Normally, you want to temporally filter *all* your voxels in your 4D fMRI data, of course. Below, we load in such a 4D fMRI file (`data_4d`), which has $50$ timepoints and ($40 \cdot 40 \cdot 19 = $) $30400$ voxels.

For this ToDo, you need to apply the high-pass filter (i.e., the `gaussian_filter` function; use sigma=25) on each and every voxel separately, which means that you need to loop through all voxels (which amounts to three nested for-loops across all three spatial dimensions). Below, we've already loaded in the data and have written the three nested for-loops. Now it's up to you to filter the signal in the inner-most loop and store it in the pre-allocated `data_4d_filt` variable (the loop may take a couple of seconds!).

There is a test-cell that you can use to test your implementation.

```python
data_4d = nib.load('data/unfiltered_data_ds.nii.gz').get_data()
print("Shape of the original 4D fMRI scan: %s" % (data_4d.shape,))

# Here, we pre-allocate a matrix of the same shape as data_4d, in which
# you need to store the filtered timeseries
data_4d_filt = np.zeros(data_4d.shape)

# Start loop across X-dimension
for i in range(data_4d.shape[0]):

    # Start loop across Y-dimension
    for ii in range(data_4d.shape[1]):

        # Start loop across Z-dimension
        for iii in range(data_4d.shape[2]):
            # Filter the timeseries for voxel at location X=i, Y=ii, Z=iii and store it
            # using an appropriate index in the pre-allocated variable data_4d_filt!

            # YOUR CODE BEGINS HERE
            filtered_signal = gaussian_filter(data_4d[i, ii, iii, :], sigma=25)
            data_4d_filt[i, ii, iii, :] = data_4d[i, ii, iii, :] - filtered_signal
            # YOUR CODE ENDS HERE
```

```python
np.testing.assert_array_almost_equal(data_4d_filt, np.load('data/answer_filt_4d.npy'))
```

### 3. Autocorrelation and prewhitening
As you (should) have seen in the previous ToDo, the model fit increases tremendously after high-pass filtering! This surely is the most important reason why you should apply a high-pass filter. But there is another important reason: high-pass filters reduce the signal's autocorrelation!

"Sure, but why should we care about autocorrelation?", you might think? Well this has to with the estimation of the standard error of our model, i.e., $\hat{\sigma}^{2}\mathbf{c}(X'X)^{-1}\mathbf{c}'$. As you've seen in the videos, the Gauss-Markov theorem states that in order for OLS to yield valid estimates (including estimates of the parameters' standard errors) *the errors (residuals) have a mean of 0, have 0 covariance (i.e., are uncorrelated), and have equal variance*.

Let's go through these three assumptions step by step. We'll use the previously filtered signal for this.

#### 3.1. Assumption of zero-mean of the residuals
First, let's check whether the mean of the residuals is zero:

```python
b = np.linalg.lstsq(X, filt_sig, rcond=None)[0]
y_hat = X.dot(b)
resids = filt_sig - y_hat
mean_resids = resids.mean()
print("Mean of residuals: %3.f" % mean_resids)
```

ToThink

What component of the design-matrix ($X$) ensures that the mean of the residuals is zero? Explain (concisely) why.

*The intercept! This predictor makes sure that any constant variance ('offset') is modeled and thus cannot occur in the residuals.*

#### 3.2. Equal variance of the residuals
Alright, sweet - the first assumption seems valid for our data. Now, the next two assumptions -- about equal variance of the residuals and no covariance between residuals -- are trickier to understand and deal with. In the book (and videos), these assumptions are summarized in a single mathemtical statement: the covariance-matrix of the residuals should be equal to the identity-matrix ($\mathbf{I}$) scaled by the noise-term ($\hat{\sigma}^{2}$). Or, put in a formula:

\begin{align}
\mathrm{cov}[\epsilon] = \hat{\sigma}^{2}\mathbf{I}
\end{align}

This sounds difficult, so let's break it down. First off all, the covariance matrix of the residuals is always a symmetric matrix of shape $N \times N$, in which the *diagonal represents the variances* and the *off-diagonal represents the covariances*. For example, at index $[i, i]$, the value represents the variance of the residual at timepoint $i$. At index $[i, j]$, the value represents the covariance between the residuals at timepoints $i$ and $j$.

In OLS, we assume that the covariance matrix of the residuals ($\mathrm{cov}[\epsilon]$) equals the
identity-matrix ($\mathbf{I}$) times the noise-term ($\hat{\sigma}^{2}$). The identity-matrix is simply a matrix with all zeros except for the diagonal, which contains ones. For example, the identity-matrix for a residual-array of length $8$ looks like:

```python
identity_mat = np.eye(8)  # makes an 'eye'dentity matrix
print(identity_mat)
```

Now, suppose we calculated that the noise-term of a model explaining this hypothetical signal of length $8$ equals 2.58 ($\hat{\sigma}^{2} = 2.58$). Then, OLS *assumes* the covariance matrix of the residuals equals the identity-matrix times the noise-term:

```python
noise_term = 2.58
assumed_cov_resid = noise_term * identity_mat
print(assumed_cov_resid)
```

In other words, this assumption about the covariance matrix of the residuals states that the *variance across residuals (the diagonal of the matrix) should be equal* and the *covariance between residuals (the off-diagonal values of the matrix) should be 0* (in the population).

Now, we won't explicitly calculate the covariance matrix of the residuals (which is usually estimated using techniques that fall beyond the scope of this course); however, we *do* want you to understand *conceptually* how fMRI data might invalidate the assumptions about the covariance matrix of the residuals and how fMRI analyses deal with this (i.e., using prewhitening, which is explained later).

So, let's check *visually* whether the assumption of equal variance of our residuals roughly holds for our (simulated) fMRI data. Now, when we consider this assumption in the context of our fMRI data, the assumption of "equal variance of the residuals" (also called homoskedasticity) means that we assume that the "error" in the model is equally big across our timeseries data. In other words, the mis-modelling (error) should be constant over time.

Let's check this for our data:

```python
plt.figure(figsize=(15, 5))
plt.plot(resids, marker='.')
plt.xlim(0, 200)
plt.xlabel("Time (TR)", fontsize=15)
plt.ylabel("Activation (A.U.)", fontsize=15)
plt.title("Residuals", fontsize=20)
plt.axhline(0, ls='--', c='black')
plt.show()
```

ToThink

What could cause unequal variance in the residuals of an fMRI signal, *given that autocorrelation (i.e. low-frequency components) are filtered out appropriately*? In other words, can you think of something that might cause larger (or smaller) errors across the duration of an fMRI run?

*One reason could be that the noise becomes larger (the signal becomes weaker) due to increasing inhomogeneities of the magnetic field caused by, for example, subject movement. Another cause could be that subjects stop paying attention or do other things that might increase the noise over time. Note: things that cause drift do not necessarily lead to unequal variance across time!*

#### 3.3. Zero covariance between residuals
The last assumption of zero covariance between residuals (corresponding to the assumption of all zeros on the off-diagonal elements of the covariance-matrix of the residuals) basically refers to the assumption that *there is no autocorrelation (correlation in time) in the residuals*. In other words, knowing the residual at timepoint $i$ does not tell you anything about the residual at timepoint $i+1$ (they are *independent*).

Take for example the residuals of our unfiltered signal from before (I emphasized the drift a little bit more below), which looked like:

```python
old_sig = sig + np.arange(-2, 2, 0.02)[::-1]
b = np.linalg.lstsq(X, old_sig, rcond=None)[0]
resids_new = old_sig - X.dot(b)

plt.figure(figsize=(15, 8))
plt.subplot(2, 1, 1)
plt.plot(resids_new, marker='.')
plt.axhline(0, ls='--', c='black')
plt.xlim(0, 200)
plt.xlabel("Time (TR)", fontsize=15)
plt.title('Residuals (containing unmodelled drift!)', fontsize=20)
plt.ylabel('Activity (a.u.)', fontsize=15)
plt.show()
```

In the above plot, reflecting the residuals of a signal in which the drift is obviously not modelled (and is thus contained in the residuals), there is strong autocorrelation: given the slow drift (decreasing values over time) we in fact *do know something about the residual at timepoint $i+1$ given the residual at timepoint $i$, namely that it is likely that the residual at timpoint $i+1$ is __lower__ than the residual at timepoint $i$*! As such, drift is a perfect example of something that (if not modelled) causes autocorrelation in the residuals (i.e. covariance between residuals)! In other words, autocorrelation (e.g. caused by drift) will cause the values of the covariance matrix of the residuals at the indices $[i, i+1]$ to be non-zero, violating the third assumption of Gauss-Markov's theorem!

ToDo

We stated that autocorrelation captures the information that you have of the residual at timepoint $i+1$ given that you know the residual at timepoint $i$. One way to estimate this ("lag 1") dependence is to calculate the covariance of the residuals with the residuals "shifted" by one "lag". In general, the autocorrelation for the
residuals $\epsilon$ with lag $\tau$ is calculated as:

\begin{align}
\mathrm{cov}[\epsilon_{i}, \epsilon_{i+\tau}] = \frac{1}{N-\tau-1}\sum_{i=1}^{N-\tau}(\epsilon_{i}\cdot\epsilon_{i+\tau})
\end{align}

Jeanette Mumford explains how to do this quite clearly in her [video on prewhitening](https://www.youtube.com/watch?v=4VSzZKO0k_w) (around minute 10). For this ToDo, calculate the "lag-1" covariance ($\tau = 1$) between the residuals (i.e., using the variable `resids_new`) and store this in a variable named `lag1_cov`.

```python
# your code here:

tau = ...
lag1_cov = ...
```

```python
# testing your answer
np.testing.assert_almost_equal(lag1_cov, np.load('data/lag1_cov.npy'))
```

### 3.4. Accounting for autocorrelation: prewhitening
So, in summary, if the covariance matrix of your residuals does not equal the identity-matrix scaled by the noise-term ($\mathrm{cov}[\epsilon] = \hat{\sigma}^{2}\mathbf{I}$), all statistics (beta-parameters, standard errors, t-values, p-values) from the GLM might be biased (usually inflated).

Unfortunately, even after high-pass filtering (which corrects for *most* but not *all* autocorrelation), the covariance matrix of the residuals of fMRI timeseries usually do no conform to the Markov-Gauss assumptions of equal variance and zero covariance. Fortunately, some methods have been developed by statisticians that transform the data such that the OLS assumptions hold again. One such technique is called *prewhitening*.

We won't discuss the mathematics of prewhitening, but you have to understand how it works conceptually.

Suppose you have a signal of 20 timepoints (an irrealistically low number, but just ignore that). Now, suppose you have estimated the covariance matrix of the residuals of this signal after modelling (how this covariance-mtarix is calculated is not important for now) - let call this matrix $\mathbf{V}$, which is an $N \times N$ matrix ($N$ referring to the number of timepoints of your signal). Now, suppose you take a look at it and you notice that it looks faaaaar from the identity-matrix ($\mathbf{I}$) that we need for OLS.

For example, you might see this:

```python
from scipy.linalg import toeplitz

N = 20

# Some magic to create a somewhat realistic covariance matrix
tmp = toeplitz(np.arange(N)).astype(float)
tmp[np.diag_indices_from(tmp)] += np.arange(0.1, 0.6, 0.025)[::-1]

# V represents the covariance matrix
V = 1 / tmp

plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.imshow(V, vmax=5, cmap='gray')
plt.axis('off')
plt.title("V (actual covariance matrix)", fontsize=15)
plt.subplot(1, 2, 2)
plt.imshow(np.eye(N), vmax=5, cmap='gray')
plt.title("Identity-matrix (assumed matrix)", fontsize=15)
plt.axis('off')
plt.tight_layout()
```

Well, shit. We have both unequal variance (different values on the diagonal) *and* non-zero covariance (some non-zero values on the off-diagonal). So, what to do now? Well, we can use the technique of prewhitening to make sure our observed covariance matrix ($\mathbf{V}$) will be "converted" to the identity matrix! Basically, this amounts to plugging in some extra terms to formula for ordinary least squares. As you might have seen in the book/videos, the *original* OLS solution (i.e., how OLS finds the beta-parameters is as follows):

\begin{align}
\hat{\beta} = (X'X)^{-1}X'y
\end{align}

Now, given that we've estimated our covariance matrix of the residuals, $\mathbf{V}$, we can rewrite the OLS solution such that it prewhitens the data (and thus the covariance matrix of the residuals will approximate $\hat{\sigma}^{2}\mathbf{I}$) as follows:

\begin{align}
\hat{\beta} = (X'V^{-1}X)^{-1}X'V^{-1}y
\end{align}

Then, accordingly, the standard-error of any contrast of the estimated beta-parameters becomes:

\begin{align}
SE_{\mathbf{c}\hat{\beta}} = \sqrt{\hat{\sigma}^{2} \cdot \mathbf{c}(X'V^{-1}X)^{-1}\mathbf{c}'}
\end{align}

This "modification" of OLS is also called "generalized least squares" (GLS) and is central to univariate fMRI analyses! You *don't* have to understand how this works mathematically; again, you should only understand *why* prewhitening makes sure that our data behaves according to the assumptions of the Gauss-Markov theorem.

(Fortunately for us, there is usually an option to 'turn on' prewhitening in existing software packages, so we don't have to do it ourselves. But it is important to actually turn it on whenever you want to meaningfully and in an unbiased way interpret your statistics in fMRI analyses!)

    ToDo (optional!)

If you want to practice your linear algebra/programming skills, you can (_optionally!_) do this ToDo: given the target signal (`some_sig`), design-matrix (`some_X`), and the (hypothetical) covariance-matrix of the residuals from before (the variable `V`), calculate the beta-parameters using the prewhitened version of OLS (i.e., 'generalized least squares'; the formula above). Also, calculate the t-value of the contrast `[0, 1]` given the appropriate (GLS) computation of the standard-error. Store your results in the variable `betas_gls` and `tval_gls`, respectively.

```python
# Implement your ToDo here!
some_sig = sig[:20]  # y
some_X = X[:20, :]   # X
c_vec = np.array([0, 1])  # the contrast you should use

betas_gls = ...
sigma_hat = ...
desvar = ...
tval_gls = ...

```

```python
np.testing.assert_array_almost_equal(betas_gls, np.array([1.845, 1.385]), decimal=3)
np.testing.assert_almost_equal(tval_gls, 1.139, decimal=3)
```

## 4. Spatial filtering (smoothing)
Alright, so in the previous two sections we've looked at how to filter our signal in the time-domain through high-pass filtering and prewhitening. Importantly, these operations are *performed on the timeseries of each voxel separately* (like you did in a previous ToDo)! Essentially, given our 4D fMRI data ($X \times Y \times Z \times T$), temporal filtering as we discussed here is only applied to the fourth dimension (the time-dimension, $T$).

In addition to temporal filtering, many people also apply a form of *spatial* filtering to their fMRI data, which is thus an operation applied to the *spatial* dimensions ($X$, $Y$, and $Z$) of our 4D data. The most common spatial filtering operation -- spatial smoothing -- is usually implemented using a "3D gaussian (low-pass) smoothing kernel". Sounds familiar? Well, it should, because it's essentially the same type of filter as we used for our temporal high-pass filter! Only this time, it's not a 1-dimensional gaussian ("kernel") that does the high-pass filtering, but it's a 3-dimensional gaussian that does *low-pass* filtering. Just like the temporal gaussian-weighted running line smoother is applied across time, we apply the 3D gaussian across space (i.e., the 3 spatial dimensions, $X$, $Y$, and $Z$). **Note that we don't advocate this at all in modern neuroimaging practice, but understanding the operation is important.**

The figure below schematically visualized the process\*:

(Note that we show the spatial data in 2D, simply because it's easier to visualize, but in reality this is always 3D!)

In fact, because both (high-pass) temporal filtering and (low-pass) spatial filtering in most fMRI applications depend on the same "gaussian filtering" principle, we can even use the same Python function: `gaussian_filter`! However, as we mentioned before, spatial smoothing in fMRI is used as a *low-pass filter*, which means that the (spatial!) frequencies *higher* than a certain cutoff are filtered out, while the (spatial!) frequencies *lower* than this cut-off are *passed*. Therefore, we don't need to subtract the output from the `gaussian_filter` function from the (spatial) data! (If we'd would that, than we're effectively high-pass filtering the data!)

---
\* 3D gaussian figure copied from [this website](https://blog.philippklaus.de/2012/10/creating-a-gaussian-window-in-3d-using-matlab).

### 4.1. Smoothing of fMRI data
Before we go on and demonstrate smoothing on fMRI data, we need to determine the sigma of our (3D) gaussian kernel that we'll use to smooth the data. Annoyingly, smoothing kernels in the fMRI literature (and software packages!) are usually not reported in terms of sigma, but as "full-width half maximum" (FWHM), which refers to the width of the gaussian at half the maximum height:

For example, you might read in papers something like "We smoothed our data with a gaussian kernel with a FWHM of 3 millimeter". Fortunately, we can easily convert FWHM values to sigmas, using:

\begin{align}
\sigma_{kernel} \approx \frac{\mathrm{FWHM}_{mm}}{2.355 \cdot \mathrm{voxel\ size}_{mm}}
\end{align}

So, for example, if I want to smooth at FWHM = 6 mm and my voxels are 3 mm in size (assuming equal size in the three spatial dimensions, which is common), my sigma becomes:

\begin{align}
\sigma_{kernel} \approx \frac{6}{2.355 \cdot 3} \approx 0.42
\end{align}

The entire conversion process is a bit annoying, but it's simply necessary due to conventions in the fMRI literature/software packages.

Anyway, having dealt with the conversion issue, let's look at an example. We'll use the 4D fMRI data from before (the `data_4d` variable) and we'll extract a single 3D volume which we'll smooth using the `gaussian_filter` function. This particular data has a voxel-size of $6\ mm^{3}$; given that we want to smooth at an FWHM of, let's say, 10 millimeter, we need a sigma of ($\frac{10}{2.355 \cdot 6} \approx $) $0.7$.

```python
vol = data_4d[:, :, :, 20] # We'll pick the 21st volume (Python is 0-indexed, remember?)

fwhm = 10
voxelsize = 6

sigma = fwhm / (2.355 * voxelsize)
smoothed_vol = gaussian_filter(vol, sigma=sigma)

# Let's plot both the unsmoothed and smoothed volume
plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.imshow(vol[:, :, 10], cmap='gray') # And we'll pick the 11th axial slice to visualize
plt.axis('off')
plt.title("Unsmoothed volume\n", fontsize=15)

plt.subplot(1, 2, 2)
plt.imshow(smoothed_vol[:, :, 10], cmap='gray')
plt.axis('off')
plt.title('Smoothed volume\n($\sigma = %.1f; FWHM = %s_{mm}$)' % (sigma, fwhm), fontsize=15)
plt.show()
```

ToDo

Now, in the above example we only smoothed a single volume, but in your analyses you would of course smooth all volumes in your fMRI run! (Just like with temporal filtering you need to filter the timeseries of all voxels separately.) In this ToDo, you need to loop through all (50) volumes from the `data_4d` variable and smooth them separately. Store the smoothed data in the already pre-allocated variable `data_4d_smoothed`. Use a sigma of 0.7 for the gaussian filter.

```python
# Implement your ToDo here

data_4d_smoothed = np.zeros(data_4d.shape)

for i in range(data_4d.shape[-1]):
    ....

```

```python
np.testing.assert_array_almost_equal(data_4d_smoothed, np.load('data/smoothed_data_todo.npy'))
```

ToThink

Since the `gaussian_filter` works for any $N$-dimensional array, one could argue that you don't have to loop through all volumes and apply a 3D filter, but you could equally well skip the loop and use a 4D filter straightaway. Explain (concisely) why this is a bad idea (for fMRI data).

*This is a bad idea because you have two opposing goals of spatial and temporal filtering: you want to high-pass the temporal dimension while low-pass the spatial dimensions. Applying a single 4D spatial (low-pass) filter will unintentionally also low-pass the temporal dimension.*

## 5. Dealing with outliers
As we've seen, high-pass filtering and spatial smoothing are operations that are applied on the signal directly (before model fitting) to reduce the noise term. Another way to reduce the noise term is to include *noise regressors* (also called 'nuisance variables/regressors') in the design matrix. As such, we can subdivide our design matrix into "predictors of interest" (which are included to model the task/stimuli) and "noise predictors". Or, to reformulate our linear regression equation:

\begin{align}
y = X_{interest}\beta_{interest}  + X_{noise}\beta_{noise} + \epsilon
\end{align}

Importantly, the difference between $X_{noise}$ and $\epsilon$ is that the $X_{noise}$ term refers to noise-related activity that you *are able to model* while the $\epsilon$ term refers to the noise that you *can't model* (this is often called the "irreducible noise/error" term).

### 5.1. Using noise-predictors for "despiking"
This technique of adding noise-predictors to the design-matrix is sometimes used to model 'gradient artifacts', which are also called 'spikes' (which you've heard about in one of the videos for this week). This technique is also sometimes called "despiking". These spikes reflect sudden large intensity increases in the signal across the entire brain that likely reflect scanner instabilities. One way to deal with these artifacts is to "censor" bad timepoints (containing the spike) in your signal using a noise-predictor.

But what defines a 'spike'/bad timepoint? One way is to average the timeseries across all voxels, generating one single 'global signal' (which makes sense because spikes usually affect measurements across the entire brain), and then apply a z-transform of this global signal, and subsequently identify spikes as any value above a certain threshold. For example, setting this threshold at 5 means that any activity value more than 5 standard deviations from the mean global signal intensity is defined as a spike.

Before explaining how to include these spikes in the design-matrix, let's take a look at the example signal that we're going to use for this section:

```python
with np.load('data/spike_data.npz') as spike_data:
    global_signal = spike_data['global_signal']
    spike_sig = spike_data['spike_sig']
    pred = spike_data['pred']

plt.figure(figsize=(15, 5))
plt.plot(global_signal)
plt.xlabel("Time (TR)", fontsize=20)
plt.ylabel("Activity (A.U.)", fontsize=20)
plt.title("Global signal (i.e., average across all voxels)", fontsize=25)
plt.xlim(0, 500)
plt.show()
```

As you can see from the plot, there are two apparent 'spikes' in the data (around $t=30$ and $t=300$)!

ToDo

Now, in order to identify spikes, we need to identify timepoints that have an activity-values that differ more than (let's say) 5 standard deviations from the mean activity value. As such, you need to 'z-score' the activity values: subtract the mean value from each individual value and divide each of the resulting 'demeaned' values by the standard deviation ($\mathrm{std}$) of the values. In other words, the z-transform of any signal $s$ with mean $\bar{s}$ is defined as:

\begin{align}
z(s) = \frac{(s - \bar{s})}{\mathrm{std}(s)}
\end{align}

Implement this z-score transform for the variable `global_signal` and store it in the variable `zscored_global_signal`. Then, identify any values above 5 in the `zscored_global_signal` and store these in a variable named `identified_spikes` which thus should contain a numpy array with boolean values in which the timepoints with spikes should be `True` and timepoints without spikes should be `False`.

(Hint: your `identified_spikes` variable should contain 2 spikes.)

```python
# Implement your ToDo here

```

```python
np.testing.assert_almost_equal(zscored_global_signal,
                               (global_signal - global_signal.mean()) / global_signal.std())

np.testing.assert_almost_equal(identified_spikes,
                               ((global_signal - global_signal.mean()) / global_signal.std() > 5))

```

Alright, if you've done the ToDo correctly, you should have found that there are two spikes in the data at timepoints $t = 30$ and $t = 308$. Now, to remove the influence of these spikes, we can include two regressors that model the influence of these timepoints in our design-matrix. These regressors simply contain all zeros except for at the timepoint of the spike, where it contains a 1. Let's create these regressors:

```python
spike_regressor_1 = np.zeros((500, 1))
spike_regressor_1[29] = 1  # remember, Python has 0-based indexing!
spike_regressor_2 = np.zeros((500, 1))
spike_regressor_2[307] = 1

plt.figure(figsize=(15, 5))
plt.plot(spike_regressor_1 + 0.01)  # The 0.01 is for visualization purposes only
plt.plot(spike_regressor_2)
plt.legend(['Spike regressor 1', 'Spike regressor 2'])
plt.xlabel("Time (A.U.)", fontsize=20)
plt.ylabel("Activity (A.U.)", fontsize=20)
plt.title("Spike regressors", fontsize=25)
plt.xlim(0, 500)
plt.show()
```

    ToThink

Why do you think we do not convolve the spike regressors with an HRF (or basis set)? Write your answer in the text-cell below.

*Because spikes are not "activity" related to neural activity! As such, we do not expect a BOLD-response and are thus not incorporating our "hypothesis" of a HRF-shaped response in our model.*

Now, let's plot a signal from a single voxel and the associated (hypothetical) stimulus-predictor.

```python
plt.figure(figsize=(15, 5))
plt.plot(spike_sig)
plt.plot(pred)
plt.xlabel("Time (A.U.)", fontsize=20)
plt.ylabel("Activity (A.U.)", fontsize=20)
plt.title("Signal + stimulus predictor", fontsize=25)
plt.xlim(0, 500)
plt.legend(['Signal', 'Stimulus-predictor'])
plt.show()
```

As you can see, the spikes that we found in the global signal are also (as expected) present in the signal of this particular voxel.

ToDo

Calculate the t-value of the stimulus-predictor-against-baseline contrast in the model with only the stimulus-predictor (don't forget to stack an intercept) and store it in the variable `tval_stimonly` (use `spike_sig` as your target, i.e., $y$). Then, add the two spike-predictors to the design-matrix (use `np.hstack`), which should have 4 columns afterwards (1 intercept, 1 stimulus predictor, 2 spike predictors). Then, calculate the t-value of the stimulus-predictor-against-baseline contrast for the extended model; store the t-value in a variable named `tval_with_spike_preds`.

```python
# Implement your ToDo here

```

```python
# test for part one
c = np.array([0, 1])
X_s = np.hstack((np.ones((pred.size, 1)), pred))
b_s = np.linalg.lstsq(X_s, spike_sig, rcond=None)[0]
sigmahat_s = np.sum((spike_sig - X_s.dot(b_s)) ** 2) / (X_s.shape[0] - X_s.shape[1])
desvar_s = c.dot(np.linalg.inv(X_s.T.dot(X_s))).dot(c.T)
answer_stimonly = c.dot(b_s) / np.sqrt(sigmahat_s * desvar_s)
np.testing.assert_almost_equal(answer_stimonly, tval_stimonly, decimal=3)
```

```python
# test for part two
c = np.array([0, 1, 0, 0])
X_sp = np.hstack((np.ones((pred.size, 1)), pred, spike_regressor_1, spike_regressor_2))
b_sp = np.linalg.lstsq(X_sp, spike_sig, rcond=None)[0]
sigmahat_sp = np.sum((spike_sig - X_sp.dot(b_sp)) ** 2) / (X_sp.shape[0] - X_sp.shape[1])
desvar_sp = c.dot(np.linalg.inv(X_sp.T.dot(X_sp))).dot(c.T)
answer_with_spike_preds = c.dot(b_sp) / np.sqrt(sigmahat_sp * desvar_sp)
np.testing.assert_almost_equal(answer_with_spike_preds, tval_with_spike_preds, decimal=3)
```

If you've done the ToDo correctly, you saw that the t-value from the model with the spike-predictors was much bigger! This is, of course, because the noise term got much smaller (and thus increasing the denominator of the t-value formula).

## 6. Motion preprocessing
This method of preprocessing through including noise predictors, like the spike-predictors in the previous section, is also often used in the context of motion filtering. Basically, in this process you'd like to remove all 'activity' in voxels that are correlated with motion. By including these motion predictors in our model, we make sure that variance in the signal caused my motion is accounted for and does *not* end up in our error term (i.e. the model's residuals/unexplained variance).

However, these 'motion predictors' do not magically appear, but are a result of a previous step in addressing motion in fMRI. Basically, "motion preprocessing" consists of two parts:

1. Motion correction (realignment);
2. Motion filtering (denoising)

In the first part, the functional image is - volume by volume - realigned such that all volumes are in the same orientation (i.e. the location of each voxel is constant over time). This realignment is done by translating (in three directions) and rotating (in three directions) each volume to match a 'reference volume' (usually the first or middle volume of each file).

Then, in a second step, we use those 'realignment parameters' (i.e. how much each subject moved over time) in our design matrix as 'noise predictors' that aim to improve model fit by explaining variance related to movement. Let's take a look at these motion parameters.

Let's take a closer look at both of these operations (motion correction/realignment and filtering/denoising).

### 6.1. Motion realignment
Arguably the largest source of noise/unwanted effects in fMRI data is subject motion. When subjects move in the scanner, the homogeneity of the magnetic field is decreased, voxels are shifted in space from timepoint to timepoint, and spurious (de)activity may occur (during modeling). As such, it is *very* important to deal with motion appropriately. By far the *most important thing* you can do to reduce the effect of motion is simply to try to make the subject move as little as possible!

But even if you have a subject that lies as still as possible in the scanner, some subject motion is unavoidable (due to e.g. breathing). As such, you always want to do motion realignment. This process works as follows:

1. Pick a "reference volume" within your fMRI run (e.g. the first or middle volume)
2. For each of the other volumes, try to rotate ("twist" around axes) and translate (move left/right/up/down/forward/backward) such that it matches the reference volume as well as possible

This process of translating and rotating is often called "rigid body transformation", which aims to "register" each volume to the reference volume using 6 parameters: $3$ (translation in $X, Y, Z$ directions) $+\ 3$ (rotation across $X, Y, Z$ axes).

Fortunately, you don't have to find these 6 parameters yourself for each volume; there are optimization algorithms that do the rotating/translating and matching to the reference volume for you (this type of registration algorithm that we use for motion realignment is similar to the algorithms used for spatial normalization from functional --> T1 --> standard space). These algorithms usually start out with random values for these six parameters, for which they calculate the "mismatch" (also called "cost"; for example in terms of the "correlation distance" between the realigned volume and the reference volume, $1 - \rho(vol, ref\_vol)$). Then, they adjust the parameters such that the "mismatch" decreases. This parameter adjustment is iterated until the "mismatch" is below some threshold.

To get a better intuition of this process, you're actually going to manually do motion realignment in the next ToDo!

ToDo

In this ToDo, you are going to try to manually register two brain images to a reference image using translation (we leave out rotation for the simplicity of the example). For this exercise, we are going to use a 2D brain image (instead of normal 3D volume), simply because it's easier to plot. As such, you can only tweak two parameters: translation in the up/down direction and translation in the left/right direction.

Below, we load in a very short fMRI run with 3 brain images (in 2D), `motion_data`. Then, we provide you with a function that mimicks a motion realigment algorithm, but which you have to tweak yourself manually. We set the reference volume to the middle image (`ref_image=2`). You start out with all the translation parameters set to 0. Run the cell below once to see the initial mismatch between the three volume.

After running the cell below, you should see an image with three brains: the red one represents the reference image (the middle image/second image); the blue one represents the first image and the green one represents the third image (in time). Now, to translate the first image (green) one voxel upwards and the third image (blue) one voxel downwards, set the `translate_up` parameter as follows:

```python
translate_up = [1, -1]
```

Equivalently, if you want to translate the first image (green) two voxels to the left and the second image (blue) three voxels to the right, set the `translate_left` parameter as follows:

```python
translate_left = [2, -3]
```

Try different values for these translation parameters until you you minimize the "mismatch" of the brain images relative to the reference image (i.e., until all three images overlay perfectly)!

```python
from scipy.stats import pearsonr

def translate_volumes(data, ref_image=2, translation_up=None, translation_left=None):

    ref_im = data[:, :, (ref_image - 1)]  # assuming data is 3D
    other_ims = np.dstack((data[:, :, :(ref_image -1)],
                           data[:, :, (ref_image):]))

    for i, up in enumerate(translation_up):
        other_ims[:, :, i] = np.roll(other_ims[:, :, i], -up, axis=0)

    for i, left in enumerate(translation_left):
        other_ims[:, :, i] = np.roll(other_ims[:, :, i], -left, axis=1)

    plt.imshow(ref_im, cmap='Reds')
    plt.axis('off')

    for i in range(other_ims.shape[-1]):

        if other_ims.shape[-1] > 2:
            plt.imshow(other_ims[:, :, i], cmap='Greens', alpha=0.5)
        else:
            if i == 1:
                plt.imshow(other_ims[:, :, i], cmap='Greens', alpha=0.5)
            else:
                plt.imshow(other_ims[:, :, i], cmap='Blues', alpha=0.5)
        plt.axis('off')
        corrdist = 1 - pearsonr(other_ims[:, :, i].ravel(),
                                ref_im.ravel())[0]
        print("Cost (1 - corr) image %i: %.3f" % (i + 1, corrdist))

motion_data = np.load('data/motion_data.npy')

# Tweak the parameters translate_up and translate_left to minimize the mismatches
translate_up = [0, 0]
translate_left = [0, 0]

translate_volumes(motion_data, ref_image=2, translation_up=translate_up, translation_left=translate_left)

translate_up = .....
translate_left = .....
```

```python
assert(translate_up == [-8, 9])
assert(translate_left == [-5, 3])
```

### 6.2 Motion filtering
Even after motion realignment, your data is still 'contaminated' by motion. This is because movement itself influences the measured activity. For example, suppose that you measure a single voxel in someone's brain; then, this person moves his/her hea†d 2 centimeters. Now, we can do motion realigment to make sure we measure the same voxel before and after the movement, but *this does not change the fact that this particular voxel was originally measured at two different locations*. It could be that after the movement, the voxel was actually a little bit closer to the headcoil, which results in a (slight) increase in signal compared to before the movement (this is also known as 'spin history effects').

Ideally, you want to account for these interactions between motion and the measured activity. One way to do this is through "motion filtering", of which one popular approach is to simply add the 6 realignment parameters (rotation and translation in 3 directions) to the design-matrix ($X$)! In other words, we treat the motion realignment parameters as "nuisance regressors" that are aimed to explain activity that is related to motion.

Alright, let's load some realigment parameters (6 in total) from an fMRI run of 200 volumes. We'll plot them below:

```python
""" This data has been motion-corrected using the FSL tool 'MCFLIRT', which outputs a file
ending in *.par that contains the 6 motion parameters (rotation/translation in 3 directions each).
We'll load in this file and plot these motion parameters. """

motion_params = np.loadtxt('data/mc/unfiltered_data_mcf.par')
rotation_params = motion_params[:, :3]
translation_params = motion_params[:, 3:]

plt.figure(figsize=(15, 7))
plt.subplot(2, 1, 1)
plt.title('Rotation', fontsize=20)
plt.plot(rotation_params)
plt.xlim(0, 199)
plt.legend(['x', 'y', 'z'])
plt.ylabel('Rotation in radians', fontsize=15)

plt.subplot(2, 1, 2)
plt.title('Translation', fontsize=20)
plt.plot(translation_params)
plt.legend(['x', 'y', 'z'])
plt.ylabel('Translation in mm', fontsize=15)
plt.xlim(0, 199)
plt.xlabel('Time (TR)', fontsize=15)
plt.tight_layout()
plt.show()
```

Looking at the plots, you could say this is pretty good data! Apart from some movement around volume 85 - 90, the participant didn't move a lot.

ToThink

Looking at the plots above, can you deduce which volume was used as a reference volume?

ToDo

For this ToDo, you have to compare two models and the resulting (normalized) effects (t-values): a model *without* the six motion parameters and a model *with* the six motion parameters.

We provide you with a design-matrix (`X`) with an intercept and a single stimulus-predictor and the signal of a single voxel (`sig`). Calculate the t-value for the contrast of the predictor-of-interest against baseline for both the original design-matrix (only intercept + predictor-of-interest) and the design-matrix extended with the six motion parameters (which thus has 8 predictors). To help you a little bit, we broke down this ToDo in multiple steps, each in a different cell.

```python
# First, we'll load the data
with np.load('data/data_last_todo.npz') as data_last_todo:
    X = data_last_todo['X']
    sig = data_last_todo['sig']

print("Shape of original X: %s" % (X.shape,))
print("Shape of signal: %s" % (sig.shape,))
```

Now, in the cell below, calculate the t-value corresponding to the contrast against baseline of our predictor-of-interest. Save this in a variable named `tvalue_simple_model`.

```python
# Calculate the t-value here

```

Then, in the cell below, make a new (extended) design-matrix by stacking the motion parameters (variable `motion_params`) to the original design-matrix (`X`).

```python
# Create here the extended design matrix with motion parameters (call it e.g. X_ext)

```

Lastly, calculate the t-value of the predictor-against-baseline for the extended model (i.e., the design-matrix with motion predictors). Store the t-value in a variable named `tvalue_extended_model`.

```python
# Calculate the t-value of the extended model here

```

```python
np.testing.assert_almost_equal(tvalue_simple_model, 12.399, decimal=3)
np.testing.assert_almost_equal(tvalue_extended_model, 10.213, decimal=3)

```

ToThink

If you did the above ToDo correctly, you should have found that the t-value of the extended model (with motion parameters) is actually **_smaller_** than the simple model (without motion parameters) ... What caused the t-value of *this predictor* to become smaller when the motion-parameters were included in your design-matrix?

*If your stimulus predictor is correlated to the model parameters, then the increased design-variance will actually lead to lower t-values! Note that it's not the increase in degrees of freedom that caused a smaller t-value, because the noise term is actually lower when the motion parameters are included!*

```python

```


```python
%%capture
## compile PyRoss for this notebook
import os
owd = os.getcwd()
os.chdir('../../')
%run setup.py install
os.chdir(owd)
```

```python
%matplotlib inline
import numpy as np
from matplotlib import pyplot as plt
import pyross
import time
import seaborn as sns
import pandas as pd
from matplotlib.pyplot import cm
```

In this notebook we consider a control protocol consisting of an initial lockdown, which is then partly released. For our numerical study we generate synthetic data using the stochastic SIIR model.

While we use the UK age structure and contact matrix, we used here simulated data.

**Summary:**

1. We load the age structure and contact matrix for Denmark. The contact matrix is generally given as
\begin{equation}
    C = C_{H} + C_{W} + C_{S} + C_{O},
\end{equation}
where the four terms denote the number of contacts at home, work, school, and all other remaining contacts.
2. We define the other model parameters of the SIIR model **(these are not fitted to any real data)**.
3. We define a "lockdown-protocol":
    Withing a certain time range, a lockdown is imposed (shcool closure). The contact matrix is reduced to
    \begin{equation}
        C = C_{H}
     \end{equation}

We want to see an impact if the school reopen, when do we see a change in the number of people infected ? Which age group is the most infected ?

## Get the contact Matrices for UK

```python
M=16  # number of age groups

# load age structure data
my_data = np.genfromtxt('../../data/age_structures/UK.csv', delimiter=',', skip_header=1)
aM, aF = my_data[:, 1], my_data[:, 2]

# set age groups
Ni=aM+aF;   Ni=Ni[0:M];  N=np.sum(Ni)
```

```python
df1= pd.DataFrame({'Female':aF,  'Age':['0-4','5-9','10-14','15-19','20-24','25-29','30-34','35-39','40-44','45-49','50-54','55-59','60-64','65-69','70-74','75-79','80-84','85-89','90-94','95-99','100+'], 'Sex':['F']*21})
df2 = pd.DataFrame({'Male':aM,  'Age':['0-4','5-9','10-14','15-19','20-24','25-29','30-34','35-39','40-44','45-49','50-54','55-59','60-64','65-69','70-74','75-79','80-84','85-89','90-94','95-99','100+'], 'Sex':['M']*21})
df3 = pd.concat([df1, df2], join='inner')
df3['number'] = np.concatenate((aF,aM))
```

C is the sum of contributions from contacts at home, workplace, schools and all other public spheres. Using superscripts $H$, $W$, $S$ and $O$ for each of these, we write the contact matrix as
$$
C_{ij} = C^H_{ij} + C^W_{ij} + C^S_{ij} + C^O_{ij}
$$

We read in these contact matrices from the data sets provided in the paper *Projecting social contact matrices in 152 countries using contact surveys and demographic data* by Prem et al, sum them to obtain the total contact matrix. We also read in the age distribution of UK obtained from the *Population pyramid* website.

```python
# Get individual contact matrices
CH, CW, CS, CO = pyross.contactMatrix.UK()

# By default, home, work, school, and others contribute to the contact matrix
C = CH + CW + CS + CO

# Illustrate the individual contact matrices:
fig,aCF =  plt.subplots(2,2);
aCF[0][0].pcolor(CH, cmap=plt.cm.get_cmap('GnBu', 10));
aCF[0][1].pcolor(CW, cmap=plt.cm.get_cmap('GnBu', 10));
aCF[1][0].pcolor(CS, cmap=plt.cm.get_cmap('GnBu', 10));
aCF[1][1].pcolor(CO, cmap=plt.cm.get_cmap('GnBu', 10));
```

## Covid19 data

```python
# Get the latest data from Johns Hopkins University
!git clone https://github.com/CSSEGISandData/COVID-19
```

    fatal: destination path 'COVID-19' already exists and is not an empty directory.

```python
cases = pd.read_csv('COVID-19/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv')
cases.head()
```

      Province/State
      Country/Region
      Lat
      Long
      1/22/20
      1/23/20
      1/24/20
      1/25/20
      1/26/20
      1/27/20
      ...
      5/24/20
      5/25/20
      5/26/20
      5/27/20
      5/28/20
      5/29/20
      5/30/20
      5/31/20
      6/1/20
      6/2/20

      0
      NaN
      Afghanistan
      33.0000
      65.0000
      0
      0
      0
      0
      0
      0
      ...
      10582
      11173
      11831
      12456
      13036
      13659
      14525
      15205
      15750
      16509

      1
      NaN
      Albania
      41.1533
      20.1683
      0
      0
      0
      0
      0
      0
      ...
      998
      1004
      1029
      1050
      1076
      1099
      1122
      1137
      1143
      1164

      2
      NaN
      Algeria
      28.0339
      1.6596
      0
      0
      0
      0
      0
      0
      ...
      8306
      8503
      8697
      8857
      8997
      9134
      9267
      9394
      9513
      9626

      3
      NaN
      Andorra
      42.5063
      1.5218
      0
      0
      0
      0
      0
      0
      ...
      762
      763
      763
      763
      763
      764
      764
      764
      765
      844

      4
      NaN
      Angola
      -11.2027
      17.8739
      0
      0
      0
      0
      0
      0
      ...
      69
      70
      70
      71
      74
      81
      84
      86
      86
      86

5 rows × 137 columns

```python
deaths = pd.read_csv('COVID-19/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_global.csv')
deaths.shape
```

    (266, 137)

```python
cases[cases['Country/Region']=='United Kingdom']
```

      Province/State
      Country/Region
      Lat
      Long
      1/22/20
      1/23/20
      1/24/20
      1/25/20
      1/26/20
      1/27/20
      ...
      5/24/20
      5/25/20
      5/26/20
      5/27/20
      5/28/20
      5/29/20
      5/30/20
      5/31/20
      6/1/20
      6/2/20

      217
      Bermuda
      United Kingdom
      32.3078
      -64.7505
      0
      0
      0
      0
      0
      0
      ...
      133
      133
      139
      139
      140
      140
      140
      140
      141
      141

      218
      Cayman Islands
      United Kingdom
      19.3133
      -81.2546
      0
      0
      0
      0
      0
      0
      ...
      129
      134
      137
      140
      140
      141
      141
      141
      150
      151

      219
      Channel Islands
      United Kingdom
      49.3723
      -2.3644
      0
      0
      0
      0
      0
      0
      ...
      558
      559
      559
      560
      560
      560
      560
      560
      560
      560

      220
      Gibraltar
      United Kingdom
      36.1408
      -5.3536
      0
      0
      0
      0
      0
      0
      ...
      154
      154
      154
      157
      158
      161
      169
      170
      170
      172

      221
      Isle of Man
      United Kingdom
      54.2361
      -4.5481
      0
      0
      0
      0
      0
      0
      ...
      336
      336
      336
      336
      336
      336
      336
      336
      336
      336

      222
      Montserrat
      United Kingdom
      16.7425
      -62.1874
      0
      0
      0
      0
      0
      0
      ...
      11
      11
      11
      11
      11
      11
      11
      11
      11
      11

      223
      NaN
      United Kingdom
      55.3781
      -3.4360
      0
      0
      0
      0
      0
      0
      ...
      259559
      261184
      265227
      267240
      269127
      271222
      272826
      274762
      276332
      277985

      248
      Anguilla
      United Kingdom
      18.2206
      -63.0686
      0
      0
      0
      0
      0
      0
      ...
      3
      3
      3
      3
      3
      3
      3
      3
      3
      3

      249
      British Virgin Islands
      United Kingdom
      18.4207
      -64.6400
      0
      0
      0
      0
      0
      0
      ...
      8
      8
      8
      8
      8
      8
      8
      8
      8
      8

      250
      Turks and Caicos Islands
      United Kingdom
      21.6940
      -71.7979
      0
      0
      0
      0
      0
      0
      ...
      12
      12
      12
      12
      12
      12
      12
      12
      12
      12

      257
      Falkland Islands (Malvinas)
      United Kingdom
      -51.7963
      -59.5236
      0
      0
      0
      0
      0
      0
      ...
      13
      13
      13
      13
      13
      13
      13
      13
      13
      13

11 rows × 137 columns

```python
cols = cases.columns.tolist()
case = cases.loc[223,][4:]
death = deaths.loc[223,][4:]
```

```python
plt.figure(figsize=(20,10))
sns.set(font_scale=3)  # crazy big
plt.legend(fontsize='x-large', title_fontsize='1000')
sns.set_style(style='white')
plt.legend(fontsize='x-large', title_fontsize='10000')
sns.scatterplot(np.arange(len(death)),death, label='death');
sns.scatterplot(x=np.arange(len(case)), y=case, label='case');
plt.ylabel('')
plt.title('')
plt.xticks(np.arange(0, 115,15), ('1/22', '2/6', '2/21', '3/16', '3/31', '4/15','4/30', '04/05'));
plt.savefig('UKcovid19.png', format='png', dpi=200)
```

## Deterministic SIR model for UK

Using this code : https://github.com/rajeshrinet/pyross/blob/master/examples/deterministic/ex03-age-structured-SIR-for-India.ipynb

Assume that the population has been partitioned into  $i=1,\ldots, M$ age groups and that we have available the $M\times M$ contact matrix $C_{ij}$.  We assume all initial cases are symptomatic, and remain so.

See SIR model : pyross/deterministic.pyx

```python
# Generate class with contact matrix for SIR model with UK contact structure
generator = pyross.contactMatrix.SIR(CH, CW, CS, CO)
```

The infection parameter $\beta$ is unknown, so we fit it to the case data till 25th March.

```python
## Parameters of the model (random)

beta  = 0.01546692 # infection rate assumed intrinsic to the pathogen
gIa   = 1./7             # recovery rate of asymptomatic infectives (7 days)
gIs   = 1./7             # recovery rate of symptomatic infectives
alpha = 0.               # fraction of asymptomatic infectives
fsa   = 1             # the self-isolation parameter

# initial conditions
Is_0 = np.zeros((M));  Is_0[0:15]=200
Ia_0 = np.zeros((M)) # no asymptomatic infectives
R_0  = np.zeros((M))
S_0  = Ni - (Ia_0 + Is_0 + R_0)
```

```python
# matrix for linearised dynamics
L0 = np.zeros((M, M))
L  = np.zeros((2*M, 2*M))

for i in range(M):
    for j in range(M):
        L0[i,j]=C[i,j]*Ni[i]/Ni[j]

L[0:M, 0:M]     =    alpha*beta/gIs*L0
L[0:M, M:2*M]   = fsa*alpha*beta/gIs*L0
L[M:2*M, 0:M]   =    ((1-alpha)*beta/gIs)*L0
L[M:2*M, M:2*M] = fsa*((1-alpha)*beta/gIs)*L0

r0 = np.max(np.linalg.eigvals(L))
print("The basic reproductive ratio for these parameters is", r0)
```

    The basic reproductive ratio for these parameters is (1.264513426078052+0j)

```python
# instantiate model
parameters = {'alpha':alpha,'beta':beta, 'gIa':gIa,'gIs':gIs,'fsa':fsa}
model = pyross.deterministic.SIR(parameters, M, Ni)
```

```python
# the contact structure is independent of time
def contactMatrix(t):
    return C
```

```python
# time_points to solve the ode (using odeint) Ti = 0 by default
Tf=350;  Nf=3500;  #Tf is the final day np.linspace(Ti, Tf, Nf)
```

```python
# run model
data=model.simulate(S_0, Ia_0, Is_0, contactMatrix, Tf, Nf)
```

```python
data['X'].shape # 48 because 16 groups * 4 equations
```

    (3500, 48)

```python
t = data['t']; IC  = np.zeros((Nf))
for i in range(M):
        IC += data['X'][:,2*M+i]
```

```python
index_max = np.argmax(IC)
```

```python
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 12})
plt.plot(t, IC, '-', lw=4, color='#A60628', label='forecast', alpha=0.8)
plt.axvline(x=index_max/10, ymin=0, ymax=175000, color='#A60628')
day, cases = np.array(np.arange(1,Tf)), np.array(case[0:Tf])
plt.plot(cases, 'o-', lw=4, color='#348ABD', ms=5, label='data', alpha=0.5)
plt.legend(fontsize=15, loc='upper left'); plt.grid()
plt.autoscale(enable=True, axis='x', tight=True)
plt.ylabel('Infected individuals');
plt.title('No measure');
plt.savefig('FullmatrixC.png', format='png', dpi=200)
```

```python
SC  = np.zeros((Nf))
for i in range(M):
        SC += data.get('X')[:,0*M+i]
        IC += data.get('X')[:,2*M+i]

fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 22})

plt.plot(t, SC*10**(-6), '-', lw=4, color='#348ABD', label='susceptible', alpha=0.8,)
plt.fill_between(t, 0, SC*10**(-6), color="#348ABD", alpha=0.3)

plt.plot(t, IC*10**(-6), '-', lw=4, color='#A60628', label='infected', alpha=0.8)
plt.fill_between(t, 0, IC*10**(-6), color="#A60628", alpha=0.3)

plt.plot(cases*10**(-6), 'ro-', lw=4, color='dimgrey', ms=16, label='data', alpha=0.5)

plt.legend(fontsize=26); plt.grid()
plt.autoscale(enable=True, axis='x', tight=True)
plt.ylabel('Individuals (millions)')
plt.xticks(np.arange(0, Tf, 90), ('22/01', '30/04'  ));
plt.savefig('C-SIRNomesure.png', format='png', dpi=200)
```

```python
# matrix for linearised dynamics
L0 = np.zeros((M, M))
L  = np.zeros((2*M, 2*M))
xind=[np.argsort(IC)[-1]]

rr = np.zeros((Tf))

for tt in range(Tf):
    Si = np.array((data['X'][tt*10,0:M])).flatten()
    for i in range(M):
        for j in range(M):
            L0[i,j]=C[i,j]*Si[i]/Ni[j]
    L[0:M, 0:M]     =    alpha*beta/gIs*L0
    L[0:M, M:2*M]   = fsa*alpha*beta/gIs*L0
    L[M:2*M, 0:M]   =    ((1-alpha)*beta/gIs)*L0
    L[M:2*M, M:2*M] = fsa*((1-alpha)*beta/gIs)*L0

    rr[tt] = np.real(np.max(np.linalg.eigvals(L)))

fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 22})

plt.plot(t[::10], rr, 'o', lw=4, color='#A60628', label='suscetible', alpha=0.8,)
plt.fill_between(t, 0, t*0+1, color="dimgrey", alpha=0.2); plt.ylabel('Basic reproductive ratio')
plt.ylim(np.min(rr)-.1, np.max(rr)+.1)
plt.xticks(np.arange(0, Tf, 90), ('22/01', '30/04'  ));
plt.savefig('C-R0Nomesure.png', format='png', dpi=200)
```

```python
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 22})

plt.bar(np.arange(16),data.get('X')[0,0:M]*10**(-6),   label='susceptible (initial)', alpha=0.8)
plt.bar(np.arange(16),data.get('X')[-1,0:M]*10**(-6),   label='susceptible (final)', alpha=0.8)

plt.xticks(np.arange(-0.4, 16.45, 3.95), ('0', '20', '40', '60', '80'));
plt.xlim(-0.45, 15.45); plt.ylabel('Individuals (millions)'); plt.xlabel('Age')
plt.legend(fontsize=22); plt.axis('tight')
plt.autoscale(enable=True, axis='x', tight=True)

plt.savefig('C-indsusNomesure.png', format='png', dpi=200)
```

### Mortality

We extract the number of susceptibles remaining in each age group, and the difference with the initial number of susceptibles is the total number that are infected. We multiply this with mortality data from China to obtain mortality estimates.

```python
MM = np.array((0,0,.0,1,1,1,1,1,1,3.5,3.5,3.5,3.5,6,6,14.2))
```

```python
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 22})

m1 = .01*MM*(data.get('X')[0,0:M]-data['X'][-1,0:M])
plt.bar(np.arange(16),m1*10**(-6),   label='susceptible (final)', alpha=0.8)

plt.axis('tight'); plt.xticks(np.arange(-0.4, 16.45, 3.95), ('0', '20', '40', '60', '80'));
plt.xlim(-0.45, 15.45); plt.ylabel('Mortality (millions)'); plt.xlabel('Age')

plt.autoscale(enable=True, axis='x', tight=True)

plt.savefig('C-mortalityNomesure.png', format='png', dpi=200)

```

## Non Pharmaceutical intervention

# School closure

Friday, March 20 in UK

###  Change the day to open again schools

```python
dayclosure = 58
dayopen1 = dayclosure+60
dayopen2 = dayclosure+80
dayopen3 = dayclosure+200

```

```python
model = pyross.deterministic.SIR(parameters, M, Ni)
```

```python
# the contact matrix is time-dependent
def contactMatrix1(t):
    if t<dayclosure:
        xx = C
    elif dayclosure<=t<dayopen1:
        xx = CH
    else:
        xx = C
    return xx

def contactMatrix2(t):
    if t<dayclosure:
        xx = C
    elif dayclosure<=t<dayopen2:
        xx = CH
    else:
        xx = C
    return xx

def contactMatrix3(t):
    if t<dayclosure:
        xx = C
    elif dayclosure<=t<dayopen3:
        xx = CH
    else:
        xx = C
    return xx

```

```python
# start simulation
Tf=1000;  Nf=3500
data=model.simulate(S_0, Ia_0, Is_0, contactMatrix1, Tf, Nf)
data2=model.simulate(S_0, Ia_0, Is_0, contactMatrix2, Tf, Nf)
data3=model.simulate(S_0, Ia_0, Is_0, contactMatrix3, Tf, Nf)
```

```python
t1 = data['t']; IC1  = np.zeros((Nf))
for i in range(M):
        IC1 += data['X'][:,2*M+i]

t2 = data2['t']; IC2 = np.zeros((Nf))
for i in range(M):
        IC2 += data2['X'][:,2*M+i]

t3 = data3['t']; IC3= np.zeros((Nf))
for i in range(M):
        IC3 += data3['X'][:,2*M+i]
```
index_max = np.argmax(IC1)

```python
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 6})
plt.plot(t1, IC1, '-', lw=3, color='#A60628', label='+10 days', alpha=0.8)
plt.fill_between(np.arange(dayclosure,dayopen1), 0, 1750000, color="#A60628", alpha=0.2)

plt.plot(t2, IC2, '-', lw=3, color='#063ba6', label='+30 days', alpha=0.8)
plt.fill_between(np.arange(dayopen1,dayopen2), 0, 1750000, color="#063ba6", alpha=0.2)

plt.plot(t3, IC3, '-', lw=3, color='#61a606', label='+150 days', alpha=0.8)
plt.fill_between(np.arange(dayopen2,dayopen3), 0, 1750000, color="#61a606", alpha=0.2)

plt.legend(fontsize=22); plt.axis('tight'); plt.grid()
plt.ylabel('Infected individuals');
plt.xlabel('Days')
plt.title('UK forecast with different school closure period ');
plt.savefig('UK-forecast-with-different-school-closure-period.png', format='png', dpi=200)
```

```python
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 6})
plt.plot(t1[1:1000], IC1[1:1000], '-', lw=3, color='#A60628', label='+10 days', alpha=0.8)
plt.fill_between(np.arange(dayclosure,dayopen1), 0, 1750000, color="#A60628", alpha=0.2)

plt.legend(fontsize=12); plt.axis('tight'); plt.grid()
plt.ylabel('Infected individuals');
plt.xlabel('Days')
plt.title('UK forecast with different school closure period ');
plt.savefig('UK-forecast-with-different-school-closure-period.png', format='png', dpi=200)
```

### Look at the numbers per age range

```python
S1= np.zeros((Nf, M))
S2 = np.zeros((Nf, M))
S3 = np.zeros((Nf, M))
I1 = np.zeros((Nf, M))
I2 = np.zeros((Nf, M))
I3 = np.zeros((Nf, M))

for i in range(M):
        S1[:,i] = data.get('X')[:,0*M+i]
        I1[:,i] = data.get('X')[:,2*M+i]
        S2[:,i] = data2.get('X')[:,0*M+i]
        I2[:,i] = data2.get('X')[:,2*M+i]
        S3[:,i] = data3.get('X')[:,0*M+i]
        I3[:,i] = data3.get('X')[:,2*M+i]
```

```python
fig = plt.figure(num=None, figsize=(7, 5), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 5})

color=['blue','red','green']
color1=['darkblue','darkred','green']
age = df3.Age.to_list()
plt.subplot(2,2,1)
for j,m in enumerate([1, 2 , 13]):
    plt.plot(t1, I1[:,m]*10**(-6), '-', lw=3, color=color[j], label='+10 days '+age[m]+' years old', alpha=0.8)
plt.legend(fontsize=10);
plt.tick_params(axis='both', which='major', labelsize=10)

plt.ylabel('Infected (millions)', fontsize=12)
plt.subplot(2,2,2)
for j,m in enumerate([1, 2 , 13]):
    plt.plot(t2, I2[:,m]*10**(-6), '-', lw=3, color=color1[j], label='+30 days '+age[m]+' years old', alpha=0.8)

plt.legend(fontsize=10);
plt.axis('tight')
plt.tick_params(axis='both', which='major', labelsize=10)
plt.savefig('UK-forecastI1.png', format='png', dpi=200);
```

```python
color=['blue','red']
color1=['darkblue','darkred']
age = df3.Age.to_list()
fig = plt.figure(num=None, figsize=(10, 8), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 22})

for j,m in enumerate([1, 2]):
    plt.plot(t1, I1[:,m]*10**(-6), '-', lw=4, color=color[j], label='+10 days '+age[m]+' years old', alpha=0.8)
    plt.plot(t2,I2[:,m]*10**(-6), '-', lw=4, color=color1[j], label='+30 days '+age[m]+' years old', alpha=0.8)

plt.ylabel('Infected(millions)')
plt.legend();
```

```python
S1d = pd.DataFrame(S1, columns=df3.Age.to_list()[0:16])
I1d = pd.DataFrame(I1, columns=df3.Age.to_list()[0:16])
S2d = pd.DataFrame(S2, columns=df3.Age.to_list()[0:16])
I2d = pd.DataFrame(I2, columns=df3.Age.to_list()[0:16])
S3d = pd.DataFrame(S3, columns=df3.Age.to_list()[0:16])
I3d = pd.DataFrame(I3, columns=df3.Age.to_list()[0:16])
```

After the closure day, the total number of infected per category is the same for all closure periods.

```python
list_eld = ['60-64','65-69','70-74','75-79']
list_young =  ['0-4','5-9','10-14', '15-19']
day = dayopen1
```

```python
fig = plt.figure(num=None, figsize=(23, 20), dpi=80, facecolor='w', edgecolor='k')
plt.rcParams.update({'font.size': 5})
plt.subplot(3,3,1)
plt.bar(list_eld,I1d.loc[:dayclosure, list_eld].mean(axis=0), label='before closure', alpha=0.8);
plt.legend(fontsize=15);
plt.subplot(3,3,2)
plt.bar(list_eld,I1d.loc[dayclosure:day, list_eld].mean(axis=0),label='during closure', alpha=0.8);
plt.legend(fontsize=15);
plt.subplot(3,3,3)
plt.bar(list_eld,I1d.loc[day:, list_eld].mean(axis=0),   label='after opening', alpha=0.8);
plt.legend(fontsize=15);
plt.axis('tight')

plt.title('Mean of infected people per day for different age with 10 more days of closure');

```


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)
```

Toggle cell visibility here.

```python
# Examples:
# Factored form: 1/(x**2*(x**2 + 1))
# Expanded form: 1/(x**4+x**2)

import sympy as sym
from IPython.display import Latex, display, Markdown, Javascript, clear_output
from ipywidgets import widgets, Layout # Interactivity module
```

## Decomposizione in fratti semplici

Quando si utilizza la trasformata di Laplace per l'analisi dei sistemi, la trasformata di Laplace del segnale di uscita si ottiene come prodotto della funzione di trasferimento del sistema per la trasformata di Laplace del segnale di ingresso. Il risultato di questa moltiplicazione solitamente è abbastanza complesso da interpretare. Per eseguire la trasformata inversa di Laplace, si esegue prima la decomposizione in fratti semplici. Questo esempio dimostra questa procedura.

---

### Come usare questo notebook?
Alterna tra l'opzione *Input da funzione* o *Input da coefficienti polinomiali*.

1. *Input da funzione*:
 * Esempio: per inserire la funzione $\frac{1}{x^2(x^2 + 1)}$ (formato fattorizzato) digitare 1/(x\*\*2\*(x\*\*2 + 1)); per inserire la stessa funzione nella forma espansa ($\frac{1}{x^4+x^2}$) digitare 1/(x\*\*4+x\*\*2).

2. *Input da coefficienti polinomiali*:
 * Usa i cursori per selezionare l'ordine del numeratore e del denominatore della funzione razionale di interesse.
 * Inserisci i coefficienti sia del numeratore che del denominatore nelle caselle di testo dedicate e fai clic su *Conferma*.

```python
## System selector buttons
style = {'description_width': 'initial'}
typeSelect = widgets.ToggleButtons(
    options=[('Input da funzione', 0), ('Input da coefficienti polinomiali', 1),],
    description='Select: ',style={'button_width':'230px'})

btnReset=widgets.Button(description="Reset")

# function
textbox=widgets.Text(description=('Inserisci la funzione:'),style=style)
btnConfirmFunc=widgets.Button(description="Conferma") # ex btnConfirm

# poly
btnConfirmPoly=widgets.Button(description="Conferma") # ex btn

display(typeSelect)

def on_button_clickedReset(ev):
    display(Javascript("Jupyter.notebook.execute_cells_below()"))

def on_button_clickedFunc(ev):
    eq = sym.sympify(textbox.value)

    if eq==sym.factor(eq):
        display(Markdown('La funzione $%s$ è scritta in forma fattorizzata. ' %sym.latex(eq) + 'La sua forma espansa è $%s$.' %sym.latex(sym.expand(eq))))

    else:
        display(Markdown('La funzione $%s$ è scritta in forma espansa. ' %sym.latex(eq) + 'La sua forma fattorizzata è $%s$.' %sym.latex(sym.factor(eq))))

    display(Markdown('Il risultato della decomposizione in fratti semplici è: $%s$' %sym.latex(sym.apart(eq)) + '.'))
    display(btnReset)

def transfer_function(num,denom):
    num = np.array(num, dtype=np.float64)
    denom = np.array(denom, dtype=np.float64)
    len_dif = len(denom) - len(num)
    if len_dif<0:
        temp = np.zeros(abs(len_dif))
        denom = np.concatenate((temp, denom))
        transferf = np.vstack((num, denom))
    elif len_dif>0:
        temp = np.zeros(len_dif)
        num = np.concatenate((temp, num))
        transferf = np.vstack((num, denom))
    return transferf

def f(orderNum, orderDenom):
    global text1, text2
    text1=[None]*(int(orderNum)+1)
    text2=[None]*(int(orderDenom)+1)
    display(Markdown('2. Inserisci i coefficienti del numeratore.'))
    for i in range(orderNum+1):
        text1[i]=widgets.Text(description=(r'a%i'%(orderNum-i)))
        display(text1[i])
    display(Markdown('3. Inserisci i coefficienti del denominatore.'))
    for j in range(orderDenom+1):
        text2[j]=widgets.Text(description=(r'b%i'%(orderDenom-j)))
        display(text2[j])
    global orderNum1, orderDenom1
    orderNum1=orderNum
    orderDenom1=orderDenom

def on_button_clickedPoly(btn):
    clear_output()
    global num,denom
    enacbaNum=""
    enacbaDenom=""
    num=[None]*(int(orderNum1)+1)
    denom=[None]*(int(orderDenom1)+1)
    for i in range(int(orderNum1)+1):
        if text1[i].value=='' or text1[i].value=='Please insert a coefficient':
            text1[i].value='Please insert a coefficient'
        else:
            try:
                num[i]=int(text1[i].value)
            except ValueError:
                if text1[i].value!='' or text1[i].value!='Please insert a coefficient':
                    num[i]=sym.var(text1[i].value)

    for i in range (len(num)-1,-1,-1):
        if i==0:
            enacbaNum=enacbaNum+str(num[len(num)-i-1])
        elif i==1:
            enacbaNum=enacbaNum+"+"+str(num[len(num)-i-1])+"*x+"
        elif i==int(len(num)-1):
            enacbaNum=enacbaNum+str(num[0])+"*x**"+str(len(num)-1)
        else:
            enacbaNum=enacbaNum+"+"+str(num[len(num)-i-1])+"*x**"+str(i)

    for j in range(int(orderDenom1)+1):
        if text2[j].value=='' or text2[j].value=='Please insert a coefficient':
            text2[j].value='Please insert a coefficient'
        else:
            try:
                denom[j]=int(text2[j].value)
            except ValueError:
                if text2[j].value!='' or text2[j].value!='Please insert a coefficient':
                    denom[j]=sym.var(text2[j].value)

    for i in range (len(denom)-1,-1,-1):
        if i==0:
            enacbaDenom=enacbaDenom+"+"+str(denom[len(denom)-i-1])
        elif i==1:
            enacbaDenom=enacbaDenom+"+"+str(denom[len(denom)-i-1])+"*x"
        elif i==int(len(denom)-1):
            enacbaDenom=enacbaDenom+str(denom[0])+"*x**"+str(len(denom)-1)
        else:
            enacbaDenom=enacbaDenom+"+"+str(denom[len(denom)-i-1])+"*x**"+str(i)

    funcSym=sym.sympify('('+enacbaNum+')/('+enacbaDenom+')')

    DenomSym=sym.sympify(enacbaDenom)
    NumSym=sym.sympify(enacbaNum)
    DenomSymFact=sym.factor(DenomSym);
    funcFactSym=NumSym/DenomSymFact;

    if DenomSym==sym.expand(enacbaDenom):
        if DenomSym==DenomSymFact:
            display(Markdown('La funzione di interesse è: $%s$. Il numeratore non può essere fattorizzato.' %sym.latex(funcSym)))
        else:
            display(Markdown('La funzione di interesse è: $%s$. Il numeratore non può essere fattorizzato. La funzione con il denominatore fattorizzato è: $%s$.' %(sym.latex(funcSym), sym.latex(funcFactSym))))

    if sym.apart(funcSym)==funcSym:
        display(Markdown('La decomposizione in fratti semplici non può essere eseguita.'))
    else:
        display(Markdown('Il risultato della decomposizione in fratti semplici è: $%s$' %sym.latex(sym.apart(funcSym)) + '.'))

    btnReset.on_click(on_button_clickedReset)
    display(btnReset)

def partial_frac(index):

    if index==0:
        x = sym.Symbol('x')
        display(widgets.HBox((textbox, btnConfirmFunc)))
        btnConfirmFunc.on_click(on_button_clickedFunc)
        btnReset.on_click(on_button_clickedReset)

    elif index==1:
        display(Markdown('1. Definisci l\'ordine del numeratore (orderNum) e del denominatore (orderDenom).'))
        widgets.interact(f, orderNum=widgets.IntSlider(min=0,max=10,step=1,value=0),
                 orderDenom=widgets.IntSlider(min=0,max=10,step=1,value=0));
        btnConfirmPoly.on_click(on_button_clickedPoly)
        display(btnConfirmPoly)

input_data=widgets.interactive_output(partial_frac,{'index':typeSelect})
display(input_data)
```

    ToggleButtons(description='Select: ', options=(('Input da funzione', 0), ('Input da coefficienti polinomiali',…

    Output()