```python
# This cell is added by sphinx-gallery
# It can be customized to whatever you like
%matplotlib inline
```

Plugins and Hybrid computation
==============================

.. meta::
    :property="og:description": This tutorial introduces the notion of hybrid
        computation by combining several PennyLane device backends to train an algorithm
        containing both photonic and qubit devices.
    :property="og:image": https://pennylane.ai/qml/_images/photon_redirection.png

.. related::

   tutorial_qubit_rotation Basic tutorial: qubit rotation
   tutorial_gaussian_transformation Gaussian transformation

*Author: PennyLane dev team. Last updated: 1 Feb 2021.*

This tutorial introduces the notion of hybrid computation by combining several PennyLane
plugins. We first introduce PennyLane's `Strawberry Fields plugin <https://pennylane-sf.readthedocs.io>`_
and use it to explore a non-Gaussian photonic circuit. We then combine this photonic circuit with a
qubit circuit — along with some classical processing — to create and optimize a fully hybrid computation.
Be sure to read through the introductory `qubit rotation <qubit_rotation>` and
`Gaussian transformation <gaussian_transformation>` tutorials before attempting this tutorial.

Note

To follow along with this tutorial on your own computer, you will require the
    `PennyLane-SF plugin <https://pennylane-sf.readthedocs.io>`_, in order to access the
    `Strawberry Fields <https://strawberryfields.readthedocs.io>`_ Fock backend using
    PennyLane. It can be installed via pip:

    .. code-block:: bash

        pip install pennylane-sf

A non-Gaussian circuit
----------------------

We first consider a photonic circuit which is similar in spirit to the
`qubit rotation <qubit_rotation>` circuit:

.. figure:: ../demonstrations/plugins_hybrid/photon_redirection.png
    :align: center
    :width: 30%
    :target: javascript:void(0);

Breaking this down, step-by-step:

1. **We start the computation with two qumode subsystems**. In PennyLane, we use the
   shorthand 'wires' to refer to quantum subsystems, whether they are qumodes, qubits, or
   any other kind of quantum register.

2. **Prepare the state** $\left|1,0\right\rangle$. That is, the first wire (wire 0) is prepared
   in a single-photon state, while the second
   wire (wire 1) is prepared in the vacuum state. The former state is non-Gaussian,
   necessitating the use of the ``'strawberryfields.fock'`` backend device.

3. **Both wires are then incident on a beamsplitter**, with free parameters $\theta$ and $\phi$.
   Here, we have the convention that the beamsplitter transmission amplitude is $t=\cos\theta$,
   and the reflection amplitude is
   $r=e^{i\phi}\sin\theta$. See :doc:`introduction/operations` for a full list of operation conventions.

4. **Finally, we measure the mean photon number** $\left\langle \hat{n}\right\rangle$ of the second wire, where

   .. math:: \hat{n} = \ad\a

   is the number operator, acting on the Fock basis number states, such that $\hat{n}\left|n\right\rangle = n\left|n\right\rangle$.

The aim of this tutorial is to optimize the beamsplitter parameters $(\theta, \phi)$ such
that the expected photon number of the second wire is **maximized**. Since the beamsplitter
is a passive optical element that preserves the total photon number, this to the output
state $\left|0,1\right\rangle$ — i.e., when the incident photon from the first wire has been
'redirected' to the second wire.

Exact calculation
~~~~~~~~~~~~~~~~~

To compare with later numerical results, we can first consider what happens analytically.
The initial state of the circuit is $\left|\psi_0\right\rangle=\left|1,0\right\rangle$, and the output state
of the system is of the form $\left|\psi\right\rangle = a\left|1, 0\right\rangle + b\left|0,1\right\rangle$, where
$|a|^2+|b|^2=1$. We may thus write the output state as a vector in this
computational basis, $\left|\psi\right\rangle = \begin{bmatrix}a & b\end{bmatrix}^T$.

The beamsplitter acts on this two-dimensional subspace as follows:

\begin{align}\left|\psi\right\rangle = B(\theta, \phi)\left|1, 0\right\rangle = \begin{bmatrix}
        \cos\theta & -e^{-i\phi}\sin\theta\\
        e^{i\phi}\sin\theta & \cos\theta
    \end{bmatrix}\begin{bmatrix} 1\\ 0\end{bmatrix} = \begin{bmatrix}
        \cos\theta\\
        e^{i\phi} \sin\theta
    \end{bmatrix}\end{align}

Furthermore, the mean photon number of the second wire is

\begin{align}\left\langle{\hat{n}_1}\right\rangle = \langle{\psi}\mid{\hat{n}_1}\mid{\psi}\rangle = |e^{i\phi} \sin\theta|^2
    \langle{0,1}\mid{\hat{n}_1}\mid{0,1}\rangle = \sin^2 \theta.\end{align}

Therefore, we can see that:

1. $0\leq \left\langle \hat{n}_1\right\rangle\leq 1$: the output of the quantum circuit is
   bound between 0 and 1;

2. $\frac{\partial}{\partial \phi} \left\langle \hat{n}_1\right\rangle=0$: the output of the
   quantum circuit is independent of the beamsplitter phase $\phi$;

3. The output of the quantum circuit above is maximised when $\theta=(2m+1)\pi/2$
   for $m\in\mathbb{Z}_0$.

Loading the plugin device
-------------------------

While PennyLane provides a basic qubit simulator (``'default.qubit'``) and a basic CV
Gaussian simulator (``'default.gaussian'``), the true power of PennyLane comes from its
`plugin ecosystem <https://pennylane.ai/plugins.html>`_, allowing quantum computations
to be run on a variety of quantum simulator and hardware devices.

For this circuit, we will be using the ``'strawberryfields.fock'`` device to construct
a QNode. This allows the underlying quantum computation to be performed using the
`Strawberry Fields <https://strawberryfields.readthedocs.io>`_ Fock backend.

As usual, we begin by importing PennyLane and the wrapped version of NumPy provided by PennyLane:

```python
import pennylane as qml
from pennylane import numpy as np
```

Next, we create a device to run the quantum node. This is easy in PennyLane; as soon as
the PennyLane-SF plugin is installed, the ``'strawberryfields.fock'`` device can be loaded
— no additional commands or library imports required.

```python
dev_fock = qml.device("strawberryfields.fock", wires=2, cutoff_dim=2)
```

Compared to the default devices provided with PennyLane, the ``'strawberryfields.fock'``
device requires the additional keyword argument:

* ``cutoff_dim``: the Fock space truncation used to perform the quantum simulation

Note

Devices provided by external plugins may require additional arguments and keyword arguments
    — consult the plugin documentation for more details.

Constructing the QNode
----------------------

Now that we have initialized the device, we can construct our quantum node. Like
the other tutorials, we use the :mod:`~.pennylane.qnode` decorator
to convert our quantum function (encoded by the circuit above) into a quantum node
running on Strawberry Fields.

```python
@qml.qnode(dev_fock, diff_method="parameter-shift")
def photon_redirection(params):
    qml.FockState(1, wires=0)
    qml.Beamsplitter(params[0], params[1], wires=[0, 1])
    return qml.expval(qml.NumberOperator(1))
```

The ``'strawberryfields.fock'`` device supports all CV objects provided by PennyLane;
see `CV operations <intro_ref_ops_cv>`.

Optimization
------------

Let's now use one of the built-in PennyLane optimizers in order to
carry out photon redirection. Since we wish to maximize the mean photon number of
the second wire, we can define our cost function to minimize the *negative* of the circuit output.

```python
def cost(params):
    return -photon_redirection(params)
```

To begin our optimization, let's choose the following small initial values of
$\theta$ and $\phi$:

```python
init_params = np.array([0.01, 0.01])
print(cost(init_params))
```

Here, we choose the values of $\theta$ and $\phi$ to be very close to zero;
this results in $B(\theta,\phi)\approx I$, and the output of the quantum
circuit will be very close to $\left|1, 0\right\rangle$ — i.e., the circuit leaves the photon in the first mode.

Why don't we choose $\theta=0$ and $\phi=0$?

At this point in the parameter space, $\left\langle \hat{n}_1\right\rangle = 0$, and
$\frac{d}{d\theta}\left\langle{\hat{n}_1}\right\rangle|_{\theta=0}=2\sin\theta\cos\theta|_{\theta=0}=0$.
Since the gradient is zero at those initial parameter values, the optimization
algorithm would never descend from the maximum.

This can also be verified directly using PennyLane:

```python
dphoton_redirection = qml.grad(photon_redirection, argnum=0)
print(dphoton_redirection([0.0, 0.0]))
```

Now, let's use the :class:`~.pennylane.GradientDescentOptimizer`, and update the circuit
parameters over 100 optimization steps.

```python
# initialise the optimizer
opt = qml.GradientDescentOptimizer(stepsize=0.4)

# set the number of steps
steps = 100
# set the initial parameter values
params = init_params

for i in range(steps):
    # update the circuit parameters
    params = opt.step(cost, params)

    if (i + 1) % 5 == 0:
        print("Cost after step {:5d}: {: .7f}".format(i + 1, cost(params)))

print("Optimized rotation angles: {}".format(params))
```

Comparing this to the `exact calculation <photon_redirection_calc>` above,
this is close to the optimum value of $\theta=\pi/2$, while the value of
$\phi$ has not changed — consistent with the fact that $\left\langle \hat{n}_1\right\rangle$
is independent of $\phi$.

Hybrid computation
------------------

To really highlight the capabilities of PennyLane, let's now combine the qubit-rotation QNode
from the `qubit rotation tutorial <qubit_rotation>` with the CV photon-redirection
QNode from above, as well as some classical processing, to produce a truly hybrid
computational model.

First, we define a computation consisting of three steps: two quantum nodes (the qubit rotation
and photon redirection circuits, running on the ``'default.qubit'`` and
``'strawberryfields.fock'`` devices, respectively), along with a classical function, that simply
returns the squared difference of its two inputs using NumPy:

```python
# create the devices
dev_qubit = qml.device("default.qubit", wires=1)
dev_fock = qml.device("strawberryfields.fock", wires=2, cutoff_dim=10)

@qml.qnode(dev_qubit)
def qubit_rotation(phi1, phi2):
    """Qubit rotation QNode"""
    qml.RX(phi1, wires=0)
    qml.RY(phi2, wires=0)
    return qml.expval(qml.PauliZ(0))

@qml.qnode(dev_fock, diff_method="parameter-shift")
def photon_redirection(params):
    """The photon redirection QNode"""
    qml.FockState(1, wires=0)
    qml.Beamsplitter(params[0], params[1], wires=[0, 1])
    return qml.expval(qml.NumberOperator(1))

def squared_difference(x, y):
    """Classical node to compute the squared
    difference between two inputs"""
    return np.abs(x - y) ** 2
```

Now, we can define an objective function associated with the optimization, linking together
our three subcomponents. Here, we wish to
perform the following hybrid quantum-classical optimization:

.. figure:: ../demonstrations/plugins_hybrid/hybrid_graph.png
    :align: center
    :width: 70%
    :target: javascript:void(0);

1. The qubit-rotation circuit will contain fixed rotation angles $\phi_1$ and $\phi_2$.

2. The photon-redirection circuit will contain two free parameters, the beamsplitter angles
   $\theta$ and $\phi$, which are to be optimized.

3. The outputs of both QNodes will then be fed into the classical node, returning the
   squared difference of the two quantum functions.

4. Finally, the optimizer will calculate the gradient of the entire computation with
   respect to the free parameters $\theta$ and $\phi$, and update their values.

In essence, we are optimizing the photon-redirection circuit to return the **same expectation value**
as the qubit-rotation circuit, even though they are two completely independent quantum systems.

We can translate this computational graph to the following function, which combines the three
nodes into a single hybrid computation. Below, we choose default values
$\phi_1=0.5$, $\phi_2=0.1$:

```python
def cost(params, phi1=0.5, phi2=0.1):
    """Returns the squared difference between
    the photon-redirection and qubit-rotation QNodes, for
    fixed values of the qubit rotation angles phi1 and phi2"""
    qubit_result = qubit_rotation(phi1, phi2)
    photon_result = photon_redirection(params)
    return squared_difference(qubit_result, photon_result)
```

Now, we use the built-in :class:`~.pennylane.GradientDescentOptimizer` to perform the optimization
for 100 steps. As before, we choose initial beamsplitter parameters of
$\theta=0.01$, $\phi=0.01$.

```python
# initialise the optimizer
opt = qml.GradientDescentOptimizer(stepsize=0.4)

# set the number of steps
steps = 100
# set the initial parameter values
params = np.array([0.01, 0.01])

for i in range(steps):
    # update the circuit parameters
    params = opt.step(cost, params)

    if (i + 1) % 5 == 0:
        print("Cost after step {:5d}: {: .7f}".format(i + 1, cost(params)))

print("Optimized rotation angles: {}".format(params))
```

Substituting this into the photon redirection QNode shows that it now produces
the same output as the qubit rotation QNode:

```python
result = [1.20671364, 0.01]
print(photon_redirection(result))
print(qubit_rotation(0.5, 0.1))
```

This is just a simple example of the kind of hybrid computation that can be carried
out in PennyLane. Quantum nodes (bound to different devices) and classical
functions can be combined in many different and interesting ways.


 Score de densité médicale

Kenza Elass et Rebecca Clain

```python
import pandas as pd
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn import preprocessing
sns.set_style('darkgrid')
pd.set_option('display.max_columns', None)
import ipywidgets as widgets
from __future__ import print_function
from ipywidgets import interact, interactive, fixed, interact_manual
from IPython.display import display
import warnings
warnings.filterwarnings("ignore")
```

```python
df = pd.read_csv('analyse_df.csv', sep = ';', low_memory = False)

df.columns = ['code commune établissement', 'ambulance', 'analyse médicale', 'autre', 'autre spécialiste',
              'chirurgien','dentiste', 'generaliste', 'hopital', 'infirmiers', 'organe', 'radiologiste',
              'réeducateur podologue', 'code géographique', 'libellé géographique','nombre ménage fiscaux',
              'nombre personne ménage fiscaux', 'médiane niveau de vie', 'part ménages fiscaux imposés',
              'taux de pauvreté ensemble', 'taux de pauvreté moins de 30 ans', 'taux de pauvreté 30 à 39 ans',
              'taux de pauvreté 40 à 49 ans', 'taux de pauvreté 50 à 59 ans', 'Taux de pauvreté 60 à 74 ans',
              'taux de pauvreté 75 ans ou plus','taux de pauvreté propriétaires','taux de pauvreté locataires',
              'part des revenus activités','dont part des salaires et traitement hors chomage',
              'dont part des indemnites de chomage', 'dont part des revenus des activites non salaries',
              'part des pensions retraites et rentes','part des revenus du patrimoine et autres revenus',
              'part ensemble prestations sociales','dont part prestations familiales', 'dont part des minima sociaux',
              'dont part des prestations logement','part des impots','1er décile niveau de vie',
              '9e décile niveau de vie','rapport interdecile 9e/1er','merge_']
```

```python
def missing_values_table(df) :

        mis_val = df.isnull().sum()
        mis_val_percent = 100 * df.isnull().sum() / len(df)
        mis_val_table = pd.concat([mis_val, mis_val_percent], axis=1)
        mis_val_table_ren_columns = mis_val_table.rename(
        columns = {0 : 'Missing Values', 1 : '% of Total Values'})
        mis_val_table_ren_columns = mis_val_table_ren_columns[
            mis_val_table_ren_columns.iloc[:,1] != 0].sort_values('% of Total Values', ascending = False).round(1)
        print ("Your selected dataframe has " + str(df.shape[1]) + " columns.\n"
            "There are " + str(mis_val_table_ren_columns.shape[0]) + " columns that have missing values.")

        return mis_val_table_ren_columns

missing_values_table(df)
```

    Your selected dataframe has 43 columns.
    There are 30 columns that have missing values.

      Missing Values
      % of Total Values

      taux de pauvreté 75 ans ou plus
      35540
      98.9

      taux de pauvreté moins de 30 ans
      35118
      97.8

      Taux de pauvreté 60 à 74 ans
      35068
      97.6

      taux de pauvreté 50 à 59 ans
      34623
      96.4

      taux de pauvreté 30 à 39 ans
      34480
      96.0

      taux de pauvreté 40 à 49 ans
      34047
      94.8

      taux de pauvreté propriétaires
      33769
      94.0

      taux de pauvreté locataires
      32722
      91.1

      taux de pauvreté ensemble
      31529
      87.8

      dont part des revenus des activites non salaries
      30665
      85.4

      dont part des minima sociaux
      30665
      85.4

      dont part des prestations logement
      30665
      85.4

      part des impots
      30665
      85.4

      1er décile niveau de vie
      30665
      85.4

      9e décile niveau de vie
      30665
      85.4

      dont part prestations familiales
      30665
      85.4

      part ensemble prestations sociales
      30665
      85.4

      part des revenus du patrimoine et autres revenus
      30665
      85.4

      part des pensions retraites et rentes
      30665
      85.4

      rapport interdecile 9e/1er
      30665
      85.4

      dont part des indemnites de chomage
      30665
      85.4

      dont part des salaires et traitement hors chomage
      30665
      85.4

      part des revenus activités
      30665
      85.4

      part ménages fiscaux imposés
      30661
      85.4

      code commune établissement
      15444
      43.0

      médiane niveau de vie
      3670
      10.2

      nombre personne ménage fiscaux
      3670
      10.2

      nombre ménage fiscaux
      3670
      10.2

      code géographique
      74
      0.2

      libellé géographique
      74
      0.2

# Construction d'un score de densité médicale

```python
df_dept = (df.dropna(subset = ['code commune établissement','nombre personne ménage fiscaux'])
             .assign(departement = lambda df : df['code commune établissement'].str[:2])
             .groupby('departement')
             [['nombre personne ménage fiscaux','ambulance', 'analyse médicale', 'autre',
             'autre spécialiste', 'chirurgien', 'dentiste', 'generaliste', 'hopital',
             'infirmiers', 'organe', 'radiologiste', 'réeducateur podologue']]
             .sum())
df_dept.head()
```

      nombre personne ménage fiscaux
      ambulance
      analyse médicale
      autre
      autre spécialiste
      chirurgien
      dentiste
      generaliste
      hopital
      infirmiers
      organe
      radiologiste
      réeducateur podologue

      departement

      01
      557566.0
      82.0
      39.0
      718.0
      203.0
      39.0
      368.0
      597.0
      138.0
      1070.0
      1.0
      38.0
      959.0

      02
      412914.0
      65.0
      36.0
      238.0
      263.0
      25.0
      261.0
      567.0
      119.0
      986.0
      5.0
      59.0
      562.0

      03
      300824.5
      59.0
      25.0
      307.0
      157.0
      35.0
      192.0
      392.0
      59.0
      774.0
      2.0
      30.0
      525.0

      04
      148467.5
      23.0
      18.0
      279.0
      118.0
      15.0
      134.0
      342.0
      48.0
      631.0
      0.0
      6.0
      419.0

      05
      127431.0
      33.0
      14.0
      235.0
      138.0
      21.0
      134.0
      379.0
      66.0
      483.0
      2.0
      25.0
      596.0

```python
soins = ['ambulance','analyse médicale','autre','autre spécialiste','chirurgien','dentiste',
         'generaliste','hopital','infirmiers','organe','radiologiste','réeducateur podologue']

for v in soins :
    df_dept[v + ' pour 100 000 habitants'] = 100000*df_dept[v]/df_dept['nombre personne ménage fiscaux']
```

Exemple de distribution des généralistes en France (estimation de la densité par méthode du noyau gaussien).

```python
plt.figure(figsize=(10,6))
sns.kdeplot(df_dept['generaliste pour 100 000 habitants'])
plt.axvline(df_dept['generaliste pour 100 000 habitants'].median(), color = 'red', linestyle = '--')
plt.show()
```

Nous appellerons densité médicale le nombre de médecins pour 100 000 habitants dans un département donné. Il existe une densité médicale pour chaque poste de soins (ambulance, dentiste, chirurgien, radiothérapiste, etc).

Pour simplifier, nous expliquerons la démarche en parlant uniquement de la densité de généraliste. La méthode reste la même pour les autres postes de soins.

Soit $x_{i}$ un vecteur ordonnant l'ensemble des densités de généralistes pour l'ensemble des départements $i$ (le rôle des poids $w_{i}$ sera présenté plus tard).

$x_{i} = [x_{min},...,x_{median},...,x_{max}] \:, w_{i} = [w_{min},...,w_{median},...,w_{max}]$

```python
df_dept['generaliste pour 100 000 habitants'].describe()
```

    count     97.000000
    mean     160.172290
    std       36.195403
    min      103.548581
    25%      135.185412
    50%      152.977729
    75%      182.096106
    max      297.415856
    Name: generaliste pour 100 000 habitants, dtype: float64

Nous avons donc : $x_{i} = [104,...,153,...,297] \:; w_{i} = [w_{min},...,w_{median},...,w_{max}]$

Nous fixons un poids égal à $w_{median} = 1$ pour la densité de généraliste médiane $x_{median}$.
Plus la densité de généraliste diminuera relativement à $x_{median}$, plus le poids $w$ diminuera.
A l'inverse, plus la densité de généraliste augmentera par rapport à $x_{median}$, plus le poids augmentera.

Pour cela, nous utilisons 2 boucles : la première pour remplir itérativement le vecteur les poids de $w_{median}$ jusqu'à $w_{min}$, la deuxième pour remplir itérativement le vecteur de poids de $w_{median}$ jusqu'à $w_{max}$.

```python
def RelativeWeight(variable) :

    x = np.sort(df_dept[variable].values)

    i = x.tolist().index(np.median(x))
    w = [0] * len(x)
    w[i] = 1

    i = x.tolist().index(np.median(x))
    while i >= 1 :
        w[i-1] = (w[i]*x[i-1])/x[i]
        i = i - 1

    j = x.tolist().index(np.median(x))
    while j <= len(w)-1 :
        try :
            w[j+1] = (w[j]*x[j+1])/x[j]
        except : IndexError
        pass
        j = j + 1

    d = pd.DataFrame(data = w, index = x, columns = ['Poids'])

    return d

df_dept['poids_generaliste'] = df_dept['generaliste pour 100 000 habitants'].map(
    RelativeWeight(variable = 'generaliste pour 100 000 habitants')["Poids"])

df_dept['poids_ambulance'] = df_dept['ambulance pour 100 000 habitants'].map(
    RelativeWeight(variable = 'ambulance pour 100 000 habitants')["Poids"])

df_dept['poids_chirurgien'] = df_dept['chirurgien pour 100 000 habitants'].map(
    RelativeWeight(variable = 'chirurgien pour 100 000 habitants')["Poids"])

df_dept['poids_dentiste'] = df_dept['dentiste pour 100 000 habitants'].map(
    RelativeWeight(variable = 'dentiste pour 100 000 habitants')["Poids"])

df_dept['poids_hopital'] = df_dept['hopital pour 100 000 habitants'].map(
    RelativeWeight(variable = 'hopital pour 100 000 habitants')["Poids"])

df_dept['poids_infirmiers'] = df_dept['infirmiers pour 100 000 habitants'].map(
    RelativeWeight(variable = 'infirmiers pour 100 000 habitants')["Poids"])

df_dept['poids_réeductateur_podologue'] = df_dept['réeducateur podologue pour 100 000 habitants'].map(
    RelativeWeight(variable = 'réeducateur podologue pour 100 000 habitants')["Poids"])

df_dept['poids_radiologiste'] = df_dept['radiologiste pour 100 000 habitants'].map(
    RelativeWeight(variable = 'radiologiste pour 100 000 habitants')["Poids"])

df_dept['poids_analyse_médicale'] = df_dept['analyse médicale pour 100 000 habitants'].map(
    RelativeWeight(variable = 'analyse médicale pour 100 000 habitants')["Poids"])

df_dept['poids_autre'] = df_dept['autre pour 100 000 habitants'].map(
    RelativeWeight(variable = 'autre pour 100 000 habitants')["Poids"])

df_dept['poids_autre_spécialiste'] = df_dept['autre spécialiste pour 100 000 habitants'].map(
    RelativeWeight(variable = 'autre spécialiste pour 100 000 habitants')["Poids"])

df_dept['poids_organe'] = df_dept['organe pour 100 000 habitants'].map(
    RelativeWeight(variable = 'organe pour 100 000 habitants')["Poids"].dropna())
```

Maintenant que les poids sont calculés, nous pouvons "ajuster" les densités, pour chaque département $i$, pour chaque poste de soins $j$.

\begin{equation}
x_{ij}^{adj} = w_{ij} \: x_{ij}
\end{equation}

```python
df_dept['ambulance_ajusté'] = df_dept['poids_ambulance'] * df_dept['ambulance pour 100 000 habitants']

df_dept['réeducateur_podologue_ajusté'] = df_dept['poids_réeductateur_podologue'] * df_dept['réeducateur podologue pour 100 000 habitants']

df_dept['analyse_médicale_ajusté'] = df_dept['poids_analyse_médicale'] * df_dept['analyse médicale pour 100 000 habitants']

df_dept['autre_ajusté'] = df_dept['poids_autre'] * df_dept['autre pour 100 000 habitants']

df_dept['autre_spécialiste_ajusté'] = df_dept['poids_autre_spécialiste'] * df_dept['autre spécialiste pour 100 000 habitants']

df_dept['chirurgien_ajusté'] = df_dept['poids_chirurgien'] * df_dept['chirurgien pour 100 000 habitants']

df_dept['dentiste_ajusté'] = df_dept['poids_dentiste'] * df_dept['dentiste pour 100 000 habitants']

df_dept['generaliste_ajusté'] = df_dept['poids_generaliste'] * df_dept['generaliste pour 100 000 habitants']

df_dept['hopital_ajusté'] = df_dept['poids_hopital'] * df_dept['hopital pour 100 000 habitants']

df_dept['infirmier_ajusté'] = df_dept['poids_infirmiers'] * df_dept['infirmiers pour 100 000 habitants']

df_dept['organe_ajusté'] = df_dept['poids_organe'] * df_dept['organe pour 100 000 habitants']

df_dept['radiologiste_ajusté'] = df_dept['poids_radiologiste'] * df_dept['radiologiste pour 100 000 habitants']
```

Nous avons ensuite décidé de sommer en colonne l'ensemble de ces nouvelles densités ajustées, de manière à obtenir comme une densité médicale globale ajustée, noté $y_{i}$

\begin{equation}
\displaystyle y_{i} = \sum_{j=1}^{m} x_{ij}^{adj} \\
\end{equation}

```python
f = []
for v in df_dept.columns :
    if v.endswith('ajusté') :
        f.append(v)
d = df_dept[f]
d['somme'] = d.sum(axis=1)
```

Enfin, nous appliquons une transformation min-max sur cette nouvelle variable, me permettant d'obtenir un score de densité médicale compris entre 0 et 1.

\begin{equation}
\displaystyle score_{i} = \frac{y_{i} - y_{min}}{y_{max} - y_{min}}
\end{equation}

```python
x = d['somme'].values.reshape(-1,1)
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
d['score'] = x_scaled
```

Visualisons la distribution de ce score (tend vers 1 si la densité médicale est relativement forte, et vers 0 si la densité médicale est relativement faible).

```python
plt.figure(figsize=(11,8))
plt.subplot(2, 1, 1)
plt.hist(d['score'], bins = 20)
plt.subplot(2, 1, 2)
sns.kdeplot(d['score'], shade = True)
plt.show()
```

# Tableau de bord

```python
df_dept = df_dept.rename(columns = {'ambulance pour 100 000 habitants' : 'Ambulance',
                                    'analyse médicale pour 100 000 habitants' : 'Analyse médicale',
                                    'autre pour 100 000 habitants' : 'Autre',
                                    'autre spécialiste pour 100 000 habitants' : 'Autre spécialiste',
                                    'chirurgien pour 100 000 habitants' : 'Chirurgien',
                                    'dentiste pour 100 000 habitants' : 'Dentiste',
                                    'generaliste pour 100 000 habitants' : 'Généraliste',
                                    'hopital pour 100 000 habitants' : 'Hôpital',
                                    'infirmiers pour 100 000 habitants' : 'Infirmiers',
                                    'organe pour 100 000 habitants' : 'Organe',
                                    'radiologiste pour 100 000 habitants' : 'Radiologiste',
                                    'réeducateur podologue pour 100 000 habitants' : 'Réeducateur podologue'})

df_dept['Département'] = df_dept.index.values
df_dept['Score'] = x_scaled

a = widgets.ToggleButtons(
                          options = ['Ambulance',
                                     'Analyse médicale',
                                     'Autre',
                                     'Autre spécialiste',
                                     'Chirurgien',
                                     'Dentiste',
                                     'Généraliste',
                                     'Hôpital',
                                     'Infirmiers',
                                     'Organe',
                                     'Radiologiste',
                                     'Réeducateur podologue'],
                          description = 'Spécialité :',
                          disabled = False,
                          button_style = ''
                         )
b = widgets.Select(
                   options = df_dept['Département'].values.tolist(),
                   value = '01',
                   description = 'Département :',
                   disabled = False
                  )

display(a,b)

button = widgets.Button(description= "Voir les résultats")
display(button)

output = widgets.Output()

@output.capture()
def on_button_clicked(z):

    print('Dans le département n°', b.value,', la densité en', a.value.lower(),
    'est de', int(np.round(df_dept.loc[df_dept['Département'] == b.value, a.value].item(),0)),
    'pour 100 000 habitants. \nLe score de densité médicale de ce département est égal à',
    np.round(df_dept.loc[df_dept['Département'] == b.value, "Score"].item(),2))

button.on_click(on_button_clicked)
display(output)
```

    ToggleButtons(description='Spécialité :', options=('Ambulance', 'Analyse médicale', 'Autre', 'Autre spécialist…

    Select(description='Département :', options=('01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11',…

    Button(description='Voir les résultats', style=ButtonStyle())

    Output()


# Statistics with Nilearn
This notebook is about the GLM/statistics-related functionality of the [nilearn](https://nilearn.github.io/) Python package for statistical analysis of fMRI data. While it is still in development, it promises to become a full-fledged Python-based alternative to existing (f)MRI analysis software packages such as FSL, SPM, and AFNI.

In this notebook, we'll showcase the most important features of the statistics-related functionality of the package in a step-by-step fashion. Notably, this notebook contains several exercises (which we call "ToDos"), which are meant to make this tutorial more interactive! Also, this tutorial is merely an introduction to (parts of) the Nilearn package. We strongly recommend checking out the excellent [user guide](https://nilearn.github.io/user_guide.html) and [example gallery](https://nilearn.github.io/auto_examples/index.html) on the Nilearn website if you want to delve deeper into the package's (more advanced) features.

While not strictly necessary, you'll get the most out of this tutorial if you are familiar with the Nilearn package. To familiarize yourself, you could go through our [Nilearn tutorial](https://github.com/lukassnoek/nilearn-tutorial). (For students doing either of the *Neuroimaging* courses at the University of Amsterdam: the `nilearn.ipynb` notebook should be in your home-folder already.) Also, this notebook contains a very short introduction to the [pandas](https://pandas.pydata.org/) package, but this can be skipped by those who are already familiar with it.

**Contents**
1. What is Nilearn?
2. Data formats
3. Creating design matrices
4. First-level models
5. Second-level models

**Estimated time needed to complete**: 1-3 hours (depending on your experience with Python)

**Credits**: if you end up using `nilearn` in your work, please cite the corresponding [article](https://www.frontiersin.org/articles/10.3389/fninf.2014.00014/full).

```python
# Install packages if necessary
try:
    import nilearn
except ImportError:
    !pip install nilearn

# We need to limit the amount of threads numpy can use, otherwise
# it tends to hog all the CPUs available when using Nilearn
import os
os.environ['MKL_NUM_THREADS'] = '1'
os.environ['OPENBLAS_NUM_THREADS'] = '1'
import numpy as np
```

    Warning: This notebook uses a lot of RAM (up to 8GB at some point), so make sure your computer/server can handle this!

## What is Nilearn?
Nilearn is one of the packages in the growing "nipy" ecosystem of Python packages for neuroimaging analysis (see also MNE, nilearn, nipype, nibabel, and dipy). One of the (recently added) features of the package is the ability to run statistical (mostly univariate) analyses of fMRI data. (This functionality was previously implemented in the "Nistats" package, which has been merged into Nilearn recently.) Importantly, Nilearn does not contain functionality to preprocess your (f)MRI data and assumes that your data has been preprocessed by another software package. Personally, we think it works very well in combination with preprocessed data using the [Fmriprep](https://fmriprep.readthedocs.io/) package.

    Note: Nilearn is a relatively new package and its API might change! If code in this notebook gives errors because it uses an old version of Nilearn, let us know.

Most statistical functionality in Nilearn is stored in the `glm` module (i.e., `nilearn.glm`), which itself contains submodules for first-level analyses (in `nilearn.glm.first_level`) and higher-level analyses (in `nilearn.glm.second_level`).

## Data formats
The two most important data formats that are used in the Nilearn package are nifti images and comma- or tab-separated values (CSV, TSV) text files.

### Nifti images
Like most packages in the *nipy* ecosystem, Nilearn assumes that your MRI data is stored in nifti images, and as such, many functions in Nilearn involving nifti images accept either strings pointing towards the path of a nifti file (or a list with multiple paths) or a `Nifti1Image` object from the `nibabel` package. Together, these two types of inputs (filenames pointing to nifti files and `Nifti1Images`) are often referred to a "niimgs" (or "niimg-like") by Nilearn — a term you'll see a lot in the documentation.

Let's actually download some data! In this tutorial, we use data from the [NARPS](https://www.narps.info/) project ([Botvinik-Nezer et al., 2019a](https://www.nature.com/articles/s41597-019-0113-7); [Botvinik-Nezer et al., 2019b](https://www.biorxiv.org/content/10.1101/843193v1)). This public dataset was analyzed by 70 different research groups to showcase the variety in analysis approaches and the way this affects the subsequent results. In the study, a "mixed gambles" experiment was used to investigate how potential monetary gains and losses are related to brain activity (which was based on the experiment by [Tom et al., 2007](https://science.sciencemag.org/content/315/5811/515)).

Each trial, participants were presented simulatenously with a potential monetary "gain" (e.g., +12) and a potential monetary "loss" (e.g., -6. For each trial, participants had to choose whether to accept this gamble (either "strongly accept", "weakly accept", "weakly reject", or "strongly reject"), knowing that at the end of the experiment, one trial ("gamble") would be picked randomly and (if chosen) would be "run" with a 50/50 chance of losing the original "loss" amount and winning the original "gain" amount. As such, the following information was recorded per trial: potential gain amount, potential loss amount, reaction time (of participant's response), and the participant's choice/decision.

    ToDo: Go to the NARPS website and read through the "data and analysis" section to get an idea of the experiment and scanning procedure.

The dataset is publicly available from [Openneuro.org](http://openneuro.org/), which also includes preprocessed data (using [Fmriprep](https://fmriprep.readthedocs.io/)) — perfect for our purposes! The cell below will download the data, which may take a while because it's 2.4 GB in size.

```python
# First, we need the `awscli` package to download the data; install if necessary
try:
    import awscli
except ImportError:
    print("Installing awscli package ...")
    !pip install awscli

# We'll save the data in your home folder
# Note: the os.path.join function concatenates paths using the delimiter specific
# to your operating system (\ for windows, / for Mac/Linux)
import os
save_dir = os.path.join(os.path.expanduser("~"), 'NARPS')
if not os.path.isdir(save_dir):
    print("Data will be saved in %s ..." % save_dir)
    print("Go get some coffee. This may take a while.\n")

    # Using the CLI interface of the `awscli` Python package, we'll download the data
    # Note that we only download the preprocessed data from a single run from a single subject
    # This may take about 10-60 minutes or so (depending on your download speed)
    !aws s3 sync --no-sign-request s3://openneuro.org/ds001734 {save_dir} --exclude "*" --include "*fmriprep/sub-00[1,3]/func/*run-01*space-MNI*.nii.gz"
    !aws s3 sync --no-sign-request s3://openneuro.org/ds001734 {save_dir} --exclude "*" --include "*fmriprep/sub-00[1,3]/func/*run-01*.tsv"
    !aws s3 sync --no-sign-request s3://openneuro.org/ds001734 {save_dir} --exclude "*" --include "*sub-00[1,3]/func/*run-01*events.tsv"
    print("\nDone!")
else:
    print("Data is already downloaded!")
```

Alright, let's check out the directory in which we saved the downloaded data:

```python
def list_files(startpath):
    """ Simple function to show directory tree.
    From: https://stackoverflow.com/questions/9727673/list-directory-tree-structure-in-python. """
    for root, dirs, files in os.walk(startpath):
        level = root.replace(startpath, '').count(os.sep)
        indent = ' ' * 4 * (level)
        print('{}{}/'.format(indent, os.path.basename(root)))
        subindent = ' ' * 4 * (level + 1)
        for f in sorted(files):
            print('{}{}'.format(subindent, f))

list_files(save_dir)
```

As you can see, this directory contains both subdirectories with "unprocessed" data (in `sub-001/func`) and preprocessed data (in `derivatives/fmriprep/sub-001/func`). Note that we excluded the raw (unprocessed) MRI data to save disk space (and time).

Now, let's check out an fMRI file. We'll load the preprocessed `sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_preproc.nii.gz` (which has been registered and resampled to standard MNI152 space already) using `nibabel`:

```python
import nibabel as nib
fmri_path = os.path.join(save_dir, 'derivatives', 'fmriprep', 'sub-001', 'func', 'sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_preproc.nii.gz')
if not os.path.isfile(fmri_path):
    raise ValueError("Data does not seem to be downloaded (correctly).\n"
                     "Try removing the data and re-downloading it!")
fmri = nib.load(fmri_path)
```

    ToDo: Do you remember how to inspect <tt>Nifti1Images</tt> from <tt>nibabel</tt>? Try to figure out this scan's TR and how many volumes (timepoints) this file has. Store the TR in a variable named <tt>tr_todo</tt> and the number of volumes in <tt>nvol_todo</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the ToDo above. '''
assert(tr_todo == 1)
assert(nvol_todo == 453)
print("Well done!")
```

### TSV files (or: a short introduction to Pandas)
*(If you're familiar with the Pandas package, you may skip this section.)*

The other type of file you'll probably encounter a lot when working with Nilearn is a TSV (tab-separated values) file. This plain-text file is like a spreadsheet with different "observations" in rows and different "attributes" in columns. For example, according to the Brain Imaging Data Structure ([BIDS](https://bids.neuroimaging.io/)), information about your experiment (like trial onsets, durations, conditions, etc.) should be stored in a TSV file ending in `_events.tsv`.

As expected, the data we downloaded also contains such an event-file, `sub-001_task-MGT_run-01_events.tsv`. (Note that these event-files are stored in the "unprocessed" data directory, not the "derivatives" directory).

These TSV files can be loaded into Python using the [pandas](https://pandas.pydata.org/) package. `pandas` is usually imported as follows:

```python
import pandas as pd
```

Then, TSV files (or any x-delimited files, like CSV files) can be loaded in using the `pd.read_csv` function. Let's do that for our events-file:

```python
path = os.path.join(save_dir, 'sub-001', 'func', 'sub-001_task-MGT_run-01_events.tsv')
events_df = pd.read_csv(path, sep='\t')
events_df  # putting events_df at the end of a cell (instead of printing it) will show a nicely formatted table
```

    ToDo: What do you think the argument <tt>sep='\t'</tt> does? Try removing the argument to see what happens.

As you can see, the `pd.read_csv` function returns a table-like object with 64 rows (corresponding to the experiment's events/trials) and 6 columns (corresponding to different attributes of the trials, like onset, duration, etc.). The `events_df` is a custom Pandas object called a `DataFrame`:

```python
print(type(events_df))
```

Pandas `DataFrames` are similar to dataframes in the R programming language. A full introduction to Pandas is beyond the scope of this tutorial (see [10 minutes to pandas](https://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html) for a nice introduction), but we'll highlight some useful Pandas-related functionality below.

One thing that is useful to know is that `DataFrames` have both row and column names. The row names are usually referred to as the `DataFrame`'s *index* (these are the elements left of the "onset" column, i.e., 0, 1, 2, ... 63). Row and column names can be of any type (strings, like the "onset" column, integers, like the row names in this `DataFrame`, etc.).

To select particular rows and/or columns, you can use the `.loc` and `.iloc` methods, where the `.loc` method selects based on the *name* of the row/column (like a dictionary) and the `.iloc` method selects based on the *position*(i.e., the "number") of the row/column (like a list).

The syntax of a `loc` and `iloc` based selection is as follows:

```
df.loc[row_name, col_name]
df.iloc[row_index, col_index]
```

Note that the square brackets (`[]`) are different than what you expect from a *method*, but the technicalities are beyond the scope of this tutorial.

Anyway, to for example select the `onset` column (and all rows), we can do the following:

```python
# the : tells loc to select *all* the rows
events_df.loc[:, 'onset']
```

And to select the first ten rows (and all columns), we can do the following:

```python
# Just like regular Python lists, iloc accepts slice syntax (like :10, or 5:12:2)
events_df.iloc[:10, :]
```

    ToThink In the example above, we used <tt>iloc</tt> to select the first ten rows of the <tt>DataFrame</tt>, but we could have also used <tt>loc</tt> for this purpose (try it out yourself by substituting <tt>loc</tt> for <tt>iloc</tt>). Why do you think this is the case? Is this always possible?

Note that you can also select multiple rows and/or columns at the same time by passing a list or tuple to the `loc` and `iloc` indexers:

```python
events_df.loc[:, ['onset', 'duration']]
```

```python
events_df.iloc[[0, 5], :]
```

    ToDo: Let's practice a little. Select from <tt>events_df</tt> all odd rows in a single statement and store it in a variable named <tt>odd_rows</tt>. For a refresher on slicing in Python, see here).

```python
''' Implement the ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the ToDo above. '''
import numpy as np
assert(odd_rows.shape == (32, 6))
np.testing.assert_array_equal(odd_rows.index, range(1, 64, 2))
print("Well done!")
```

    ToDo: Select the last 10 rows and the onset and duration columns in a single statement and store it in a variable named <tt>last10_onset_duration</tt>.

```python
''' Implement the ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
assert(last10_onset_duration.shape == (10, 2))
np.testing.assert_array_equal(last10_onset_duration.columns, ['onset', 'duration'])
np.testing.assert_array_equal(last10_onset_duration.index, range(54, 64))
print("Well done!")
```

Another useful approach for selecting subsets of observations (i.e., rows) in `DataFrames` is *boolean indexing*. For example, to create an index that selects all trials (rows) in which the participant indicated "strongly_accept" (i.e., accepting a trial's proposed bet), we can do:

```python
bool_idx = events_df.loc[:, 'participant_response'] == 'strongly_accept'
print(bool_idx)
```

and we can subsequently pass this index to `loc` (note that `iloc` doesn't work with boolean indices):

```python
strongly_accept_trials = events_df.loc[bool_idx, :]
strongly_accept_trials
```

    ToDo: Using boolean indexing, select all trials with a reaction time smaller than 1.5 seconds and store the result in a variable named <tt>rt_smaller_than_1p5</tt>.

```python
''' Implement the ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_almost_equal(rt_smaller_than_1p5.loc[:, 'RT'].mean(), 1.2196)
print('Well done!')
```

You can also use `loc` and `iloc` to create new columns. For example, let's add a new column, `"random_numbers"`, with some random numbers:

```python
# Note that you can access the shape (nr of rows, nr of cols) of a DataFrame using
# the .shape attribute (just like numpy arrays)!
rnd = np.random.randn(events_df.shape[0])
events_df.loc[:, 'random_numbers'] = rnd
events_df
```

To delete a row or column, you can use the `drop` method. Note that you should pass a particular `axis` to `drop`, either `axis=0` (should I drop rows?) or `axis=1` (should I drop columns?):

```python
events_df = events_df.drop('random_numbers', axis=1)
events_df
```

There are many, many more things you can do with Pandas, but for now, this should suffice for our purposes.

## Creating design matrices
Before we are going to fit any model on our fMRI data ($y$), we are going to focus on what we're going to put into the model: the design matrix ($\mathbf{X}$)! Note that Nilearn offers different ways to create design matrices (and fit univariate models in general), some offer a relatively "high-level" interface (such as `first_level_model_from_bids`) while others only offer the building blocks (such as different HRF models and various types of regression models) that you can use to build the models yourself.

These multiple interfaces for the same functionality is especially apparent when constructing design matrices using Nilearn. In this section, we'll start at the lowest level and work ourselves up to the more convenient (but less flexible) higher level interfaces.

That said, let's go back to design matrices. What do you need to construct them? Roughly, you need the following:
* A particular HRF model
* Information about event onset, duration, and (optionally) amplitude (weight);
* Information about fMRI scan timing;

Using these three components, we can construct HRF-informed regressors for our events on the time scale of our fMRI signal.

### Defining an HRF model
First, let's try to explicitly construct an HRF model using Nilearn. The `nilearn.glm.first_level.hemodynamic_models` module contains various HRF models. Let's start with the good old canonical "Glover" HRF:

```python
from nilearn.glm.first_level.hemodynamic_models import glover_hrf
```

This `glover_hrf` function takes a couple of arguments, most importantly: `tr` (temporal resolution of your fMRI scan in seconds) and `oversampling` (how much to temporally upsample the HRF relative to your `tr`; you can usually safely ignore the other two arguments, `time_length` and `onset`).

Suppose that we want to define our HRF on the scale of 0.01 seconds, and knowing that our fMRI scan has a TR of 1, we can do the following:

```python
canon_hrf = glover_hrf(tr=1, oversampling=100)
```

And let's plot it:

```python
import matplotlib.pyplot as plt
%matplotlib inline

# Define timepoints corresponding to HRF
t_hrf = np.linspace(0, 32, 32 * 100, endpoint=False)

plt.figure(figsize=(11, 4))
plt.plot(t_hrf, canon_hrf)
plt.grid()
plt.ylabel('Amplitude (A.U.)', fontsize=15)
plt.xlabel('Time (seconds)', fontsize=15)
plt.title('Canonical ("Glover"") HRF', fontsize=20)
plt.show()
```

    ToDo: Sometimes, people like to use a basis set of functions as a model for the HRF, such as the temporal and dispersion derivatives of the canonical HRF (in addition to the canonical HRF itself). The <tt>nilearn</tt> package also includes these temporal and dispersion derivatives: <tt>glover_time_derivative</tt> and <tt>glover_dispersion_derivative</tt>. Import these functions, construct these HRF derivatives, and plot all three together in the same figure (like the one above).

```python
''' Implement your ToDo here. (No test cell.) '''
# YOUR CODE HERE
raise NotImplementedError()
```

### Creating HRF-convolved regressors
Now, to create actual regressors, we need our event onsets and durations. At the lowest level interface, we only need the onsets, durations, and amplitudes (weights) of our events of a particular condition. For now, let's focus on the "strongly_accept" condition (note that you can define a "condition" using whatever feature from the experiment, such as the gains or losses, or even reaction time). There is no reason to focus only on these type of trials other than for the sake of the example:

```python
idx = events_df.loc[:, 'participant_response'] == 'strongly_accept'
sa_events = events_df.loc[idx, ['onset', 'duration']]
sa_events['amplitude'] = 1  # set the same amplitude for each event
sa_events
```

The next step is to create an event regressor (basically arrays with zeros, containing ones at the onset of events) and convolving this array with our HRF model. We can do this easily using the `compute_regressor` function from the `nilearn.glm.first_level.hemodynamic_models` module:

```python
from nilearn.glm.first_level.hemodynamic_models import compute_regressor
```

The most important arguments of this function are the following:
* `exp_condition`: an array of shape $3$ (onset, duration, amplitude) $\times N$ (events for a particular condition)
* `hrf_model`: a **string** indicating the HRF model ('spm', 'spm + derivative', 'spm + derivative + dispersion',
'glover', 'glover + derivative', 'glover + derivative + dispersion', 'fir')
* `frame_times`: an array of shape $T$ (timepoints of fMRI series) containing the acquisition time of each fMRI volume

First, let's define `exp_condition` by pulling out the array from `sa_events` (using the `.values` attribute of the `DataFrame`) and transposing it (otherwise it would be of the wrong shape, i.e., $N \times 3$).

```python
exp_condition = sa_events.values.T
print(exp_condition.shape)
```

Let's for simplicity just use a canonical (Glover) HRF:

```python
hrf_model = 'glover'
```

And we'll define the frame times relative to the middle slice (i.e., assuming for simplicity that each slice was recorded at TR / 2):

```python
n_vols = 453
tr = 1
frame_times = np.linspace(tr / 2, n_vols * tr + tr / 2, n_vols, endpoint=False)
```

Alright, now we have everything to compute the regressor using `compute_regressor`:

```python
# The function compute_regressor returns two things:
# the convolved regressor(s) and a (default) name(s)
reg, name = compute_regressor(
    exp_condition=exp_condition,
    hrf_model=hrf_model,
    frame_times=frame_times
)

plt.figure(figsize=(11, 4))
plt.plot(frame_times, reg)
plt.grid()
plt.ylabel('Amplitude (A.U.)', fontsize=15)
plt.xlabel('Time (seconds)', fontsize=15)
plt.title('HRF-convolved regressor', fontsize=20)
plt.show()
```

One of the analyses of NARPS project was to investigate the parametric effect of *gains* (i.e., the amount that could be gained for this trial), effectively asking: which voxels show activity that linearly relates to gains? For this analysis, you'd need both a simple "gamble-vs-baseline" regressor (like we computed before, but then for all trials instead of just the "strongly_accept" ones) and a "parametric modulation" regressor. This parametric modulation regressor uses the same onsets and durations as the "trial-vs-baseline" regressor, but *also* includes varying modulation values that indicate how much the effect of the gamble is modulated. Importantly, these modulation values should be mean-centered (or "demeaned"), i.e., its mean should be zero. You can mean-center any variable $x$ by subtracting the mean ($\bar{x}$) from all values:

\begin{align}
x_{\mathrm{demeaned}} = x - \bar{x}
\end{align}

    ToDo (optional): Create a parametric regressor that is modulated by the "gain" values. You can do this by making sure that the third row in the array that is passed as the <tt>exp_condition</tt> argument of <tt>compute_regressor</tt> reflects the mean-centered gain values. Store the resulting regressor in a variable named <tt>parametric_gain_reg</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_almost_equal(parametric_gain_reg.mean(), 0.0095463)
print("Well done!")
```

### Using the high-level `make_first_level_design_matrix` function
While the HRF-related functions and `compute_regressor` already take care of a lot of stuff for us, you might still think that constructing a complete design matrix this way is quite cumbersome. Fortunately, Nilearn has a more high-level function for constructing complete design matrices easily called `make_first_level_design_matrix`:

```python
from nilearn.glm.first_level.design_matrix import make_first_level_design_matrix
```

    Tip: in Jupyter notebooks, you can view any (Python) function's documentation, or "docstring", by running the function (without brackets) followed by a question mark, e.g., <tt>make_first_level_design_matrix?</tt>. Try this below. You can remove the pop-up by clicking the cross-symbol in the upper right-corner.

```python
make_first_level_design_matrix?
```

As you can see, this function needs (amongst other things) a `DataFrame` (events) containing the following columns: "onset", "duration", "trial_type", and (optionally) "modulation". Then, the function will basically apply the `compute_regressor` function across the different trial types as defined in the "trial_type" column.

For now, let's assume that we'd want to create a design matrix with a different (un-modulated) regressor for each of the participant responses ("weakly_accept", "strongly_accept", "weakly_reject", and "strongly_reject"). Below, we'll create the appropriate `DataFrame` for this purpose:

```python
# First, extract relevant columns
pr_events = events_df.loc[:, ['onset', 'duration', 'participant_response']]

# Then, rename the participant_response column to trial_type
# bonus: why do you think we have to specify axis=1 here?
pr_events = pr_events.rename({'participant_response': 'trial_type'}, axis=1)

# Lastly, create a "modulation" column with only ones (indicating no modulation)
pr_events.loc[:, 'modulation'] = 1  # this automatically fills all rows with 1
pr_events
```

Now, the different conditions in our design are defined by the unique values in our `trial_type` column:

```python
print(pr_events.loc[:, 'trial_type'].unique())
```

Ah darn, we also have 'NoResp' trials!

    ToDo Remove all 'NoResp' trials (i.e., rows) and store the new (filtered) <tt>DataFrame</tt> in a variable named <tt>pr_events_filt</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
assert('NoResp' not in pr_events_filt.loc[:, 'trial_type'].unique())
print("Well done!")
```

    ToDo Now you're ready to use <tt>make_first_level_design_matrix</tt>! Use the appropriate arguments (at least <tt>frame_times</tt>, <tt>events</tt>, and <tt>hrf_model</tt>) and make sure to use <tt>drift_model=None</tt>.
   This will prevent Nilearn from automatically adding a set of regressors to the design matrix which function as a high-pass filter (we'll take a look at this later). Store the output of the function in a new variable called <tt>dm</tt>, which should be a new pandas <tt>DataFrame</tt> of shape T (number of timepoints) x P (number of regressors, i.e., number of conditions + intercept). You can ignore the <tt>UserWarning</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the ToDo above. '''
assert(dm.shape == (453, 5))
np.testing.assert_array_equal(
    dm.columns,
    ['strongly_accept', 'strongly_reject', 'weakly_accept', 'weakly_reject', 'constant']
)
print("Well done!")
```

Alright, almost done with this section. One last thing we want to show you is the `plot_design_matrix` function, which you can use to, well, plot the design matrix:

```python
from nilearn.plotting import plot_design_matrix
dm = pd.read_csv('dm_todo.tsv', sep='\t')
plot_design_matrix(dm);
```

This plot shows you a "color-coded" version of your design matrix, with time on the y-axis and the different predictors on the x-axis. Note that the brighter the color (i.e., more yellow), the higher that regressor's value at that timepoint.

    ToDo Run <tt>make_first_level_design_matrix</tt> again, but add the following arguments: <tt>drift_model='cosine'</tt> and <tt>high_pass=0.01</tt> (i.e., in Hertz, corresponding to a cutoff of 100 seconds), which will add a set of cosine regressors to the design matrix, which function as a high-pass filter for our fMRI data. Store the design matrix in a new variable named <tt>dm_with_hp</tt>. Then, plot the design matrix again to see how it looks like!

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
assert(dm_with_hp.shape == (453, 14))
assert(all(f'drift_{i}' in dm_with_hp.columns for i in range(1, 10)))
print("Well done!")
```

## First-level models
In this section, we will talk about fitting first-level models and computing contrasts from their estimated parameters.

### Constructing and fitting first-level models
At last, we can start fitting first-level GLM models! Like with constructing design matrices, there are different ways to go about defining and fitting first-level models in Nilearn:
* Using the low-level `run_glm` function;
* Using the `FirstLevelModel` class;
* Using the `first_level_models_from_bids` function

For most purposes, we recommend using the `FirstLevelModel` interface, which strikes a nice balance between flexibility (unlike the very high-level `first_level_models_from_bids` function) and ease-of-use (unlike the very low-level `run_glm` function).

Let's start by importing the `FirstLevelModel` class (you may ignore the FutureWarning):

```python
# You may ignore the DeprecationWarning
from nilearn.glm.first_level import FirstLevelModel
```

Note that, unlike the previous functionality from Nilearn that we discussed, `FirstLevelModel` is not a *function*, but a custom *class* (often called a *type* in Python):

```python
print(type(FirstLevelModel))

# Contrast this with for example 'make_first_level_design_matrix'
print(type(make_first_level_design_matrix))
```

Using custom classes, you can instantiate ("create") objects that behave as specified in the class. Before we go on, let's digress a little and discuss briefly how custom classes and object work.

One way to think about classes and objects is to see classes as *building blueprints* and the objects constructed from them as the *actual buildings*. For example, neuroimaging software in Python often work with MRI data stored in objects from the `Nifti1Image` class (from the `nibabel` package). An object is usually *constructed* from a class as follows:

```
some_object = SomeClass(optional_arg1, optional_arg2, optional_arg3, etc.)
```

For example, to construct a `Nifti1Image` object, we need to pass it both data (as a `numpy` array) and an affine (also as a `numpy` array):

```python
import numpy as np

# Very small hypothetical brain image of 10x10x10
data = np.random.normal(0, 1, size=(10, 10, 10))
affine = np.eye(4)

# Here, we construct an object (nifti_obj) from a class (nib.Nifti1Image)
nifti_obj = nib.Nifti1Image(data, affine)
print(type(nifti_obj))
```

The class ("blueprint") contains instructions about the "properties" of the object ("building") and the "functionality" of the object. The "properties" of an object are, technically, called *attributes* and can be accessed as: `some_object.attribute_name`. For example, `Nifti1Image` objects have an attribute named `shape`:

```python
print(nifti_obj.shape)
```

The "functionality" of objects are, technically, called *methods*. You can see methods as functions that are bound to a particular object. For example, `Nifti1Image` objects hava a method called `get_fdata`, which extract the actual brain data:

```python
data = nifti_obj.get_fdata()
# 'data' is the underlying numpy array!
print(type(data))
```

Note that you should call methods in the same way as functions (i.e., using round brackets, (), which may or may not contain arguments). Like functions, method calls may (or may not) return one or more things.

    Tip: in Jupyter notebooks (and many other code editors), you can inspect an object's attributes and methods by typing the object's variable name followed by a dot and pressing <tt>tab</tt>, i.e., <tt>your_object.+TAB</tt>. Try it out below with the <tt>nifti_obj</tt> variable to inspect the different attributes/methods from the <tt>Nifti1Image</tt> class.

```python
# Try out the tip here

```

This topic of (custom) classes and objects is actually very important in Python. Technically, Python is an "object-oriented" programming language in which *everything is an object (from a particular class)*! But an in-depth explanation of object-oriented programming is beyond the scope of this tutorial. For now, knowing the basics about (custom) classes and objects suffices for our purposes.

Alright, back to the `FirstLevelModel` class.

    ToDo: Check out the arguments need for the initialization of a <tt>FirstLevelModel</tt> object below.

```python
FirstLevelModel?
```

As you can see, initializing a `FirstLevelModel` object takes a lot of arguments (`t_r`, `slice_time_ref`, `hrf_model`, `drift_model`, etc.). Fortunately, many arguments have sensible defaults. In what follows, we'll go through the most important arguments step by step, after which we'll (finally!) construct a `FirstLevelModel` object.

The `t_r` argument refers to the TR (temporal resolution, in seconds) of your fMRI scan.

```python
t_r = 1
```

The `slice_time_ref` argument refers to which slice you want resample your design matrix (this is used internally to define the `frame_times`), which should be between 0 (first slice) and 1 (last slice). For example, if you slice-time corrected your fMRI data to the first slice, you should set `slice_time_ref` to 0 (middle slice → 0.5, last slice → 1, etc.). If you did not apply any slice-time correction (as is the case for the NARPS data), we recommend setting it to 0.5. (Can you think of a reason why this is a sensible choice for non-slice-time-corrected data?)

```python
slice_time_ref = 0.5
```

While you can construct your design matrix using Nilearn functionality directly, you can actually also leave it up to the `FirstLevelModel` to do this for you! That's why you might have recognized some parameters that you've seen in functions from the previous section (such as `hrf_model`, `drift_model`, and `high_pass`). For now, we'll let the `FirstLevelModel` take care of constructing a design for us, so we'll define these parameters here:

```python
hrf_model = 'glover'
drift_model = 'cosine'
high_pass = 0.01  # cutoff: 0.01 Hz (i.e., 100 seconds)
```

The next important parameter is `mask_img` (note that we're skipping some parameters, as they represent some more advanced functionality/analyses). This (optional!) parameter refers to a nifti image (a path to a nifti file or a `Nifti1Image` object) with binary (0, 1) data indicting which voxels should be included (1) and which ones should be "masked" (0). Fortunately, Fmriprep also returns a brain-mask:

```python
mask_img = os.path.join(
    save_dir, 'derivatives', 'fmriprep', 'sub-001', 'func',
    'sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz'
)
print(mask_img)
```

Note that including a brain-mask may substantially speed up your analysis (as it reduces the number of voxels that will be analyzed).

We can also spatially smooth our fMRI data by setting the `smoothing_fwhm` parameter (in millimeters):

```python
smoothing_fwhm = 3.5
```

Then, the `standardize` and `signal_scaling` parameters refer to the method that will be used to mean-center the data. In this example, we'll leave it to the default values (`standardize=False` and `signal_scaling=0`), which will only mean-center the data in the time domain (i.e., substract the mean across time from each voxel).

The `noise_model` parameter refers to the particular noise model of the GLM. Do we assume independent and identically distributed (IID) noise (`noise_model='ols'`) or do we assume that our noise is temporally autocorrelated (`noise_model='ar1'`)? For almost all fMRI data, we should assume that the noise is at least somewhat autocorrelated, so we'll set `noise_model` to `'ar1'`. (Note that `'ar1'` is a bit slower than `'ols'`, so you may consider using `'ols'` when for example testing your code.)

```python
noise_model = 'ar1'
```

Then, the `n_jobs` parameter determines how many CPU cores the GLM analysis will use. For now, one core should suffice:

```python
n_jobs = 1
```

Lastly, you may want to set the `minimize_memory` argument to `False`. While doing this will increase the memory (RAM) necessary for the GLM analysis, it allows use to retrieve more outputs from the GLM after fitting (such as model fit, $R^2$).

```python
minimize_memory = False
```

Now we're *finally* ready to constuct our `FirstLevelModel` object!

```python
flm = FirstLevelModel(
    t_r=t_r,
    slice_time_ref=slice_time_ref,
    hrf_model=hrf_model,
    drift_model=drift_model,
    high_pass=high_pass,
    mask_img=mask_img,
    smoothing_fwhm=smoothing_fwhm,
    noise_model=noise_model,
    n_jobs=n_jobs,
    minimize_memory=minimize_memory,
    verbose=True  # this will print out some useful info later
)

""" Note that, here, we've defined all arguments beforehand,
but this is not necessary! We could have done something like:
flm = FirstLevelModel(
    t_r=1,
    slice_time_ref=0.5,
    hrf_model='glover',
    drift_model='cosine',
    high_pass=0.01,
    mask_img=os.path.join(etc., etc.),
    smoothing_fwhm=3.5,
    noise_model='ar1',
    n_jobs=1,
    minimize_memory=False,
    verbose=True
)
""";
```

Wait, what? Did it really just take a fraction of a second to fit the first-level model? Actually, no. We only *constructed* the first-level model, but we haven't fitted it yet! To do so, we can use the, guess what, `fit` method!

    ToDo: check out the arguments needed by the <tt>fit</tt> method.

```python
flm.fit?
```

There are actually two ways of using the `fit` method, depending on which parameters you use:
1. define `events` (+ optionally `confounds`)
2. define `design_matrices`

Approach 1 will construct a design matrix for you (by running `make_first_level_design_matrix` on your `events` and, optionally, concatenating this with your `confounds`) while approach 2 assumes that you have constructed a design matrix yourself.

Note, by the way, that the parameter names are all plural (run_img**s**, event**s**, confound**s**, design_matrice**s**). This is because you can either fit a model on a *single* fMRI run or fit a model on *multiple* fMRI runs with a single `fit` call. To fit multiple runs, just pass a list instead of a single object for each parameter (e.g., `design_matrices=[dm_run1, dm_run2]` instead of `design_matrices=dm`).

Regardless of the approach for constructing your design matrix, the `fit` method always needs (one or more) fMRI files (for the `run_img` parameter), which can either be a `Nifti1Image` object or string representing the path to a 4D nifti file. Furthermore, both the `events` and the `confounds` should be a Pandas `DataFrame`. For our `events`, let's use the `pr_events_filt` events, in which the participant responses represent our conditions:

```python
# This is also one possible solution to the ToDo earlier
events2use = pr_events.loc[pr_events.loc[:, 'trial_type'] != 'NoResp', :]
```

Almost always, you'd want to add some confounds to your design matrix, as confounds represent independent variables that are (often) not of interest to the researcher, but might be related to the signal (the depenent variable, $y$) anyway. Including these confounds in your design matrix will then explain some variance that might otherwise not be accounted for (which will "end up" in the model's noise term, $\hat{\sigma}^2$), increasing the model's power to detect effects related to the variables you're interested in. Note that including the "right" confounds in your model is in no way trivial (and opinions differ to what confounds are the "right" ones to include)!

That said, fortunately, Fmriprep also computes an extensive set of timepoint-by-timepoint confounds for each fMRI run, which are conveniently stored in TSV files. (You can really see that Nilearn was designed with Fmriprep's outputs in mind.) We'll load these confounds below:

```python
conf_path = os.path.join(save_dir, 'derivatives', 'fmriprep', 'sub-001', 'func', 'sub-001_task-MGT_run-01_bold_confounds.tsv')
confounds = pd.read_csv(conf_path, sep='\t')
confounds
```

    ToDo: Read the Fmriprep documentation about the different confounds (scroll down to the "Confounds" section). Note that the confound names in the documentation are slightly different that in our <tt>DataFrame</tt> because this dataset was preprocessed using an older version of Fmriprep.

As stated in Fmriprep's documentation, it's probably not a good idea to include *all* confounds in our model. Selecting a subset of confound parameter is, however, also not a trivial matter; which confounds are appropriate and explain to most variance in your data likely depends on the characteristics of your particular dataset (field strength, scan technique, the amount of motion in your data, etc.) and your intended analysis (univariate group-level GLM, functional connectivity, MVPA).

Here, we'll be relatively conservative and include only the 6 motion parameters (`X`, `Y`, `Z`, `RotX`, `RotY`, `RotZ`) and the non-steady-state outlier (`NonSteadyStateOutlier00`; i.e., a binary-coded regressor removing the influence of volumes at the beginning of a run that contain too much T1-contrast).

```python
confounds2use = confounds.loc[:, ['X', 'Y', 'Z', 'RotX', 'RotY', 'RotZ', 'NonSteadyStateOutlier00']]
confounds2use
```

In fact, we can reuse the `plot_design_matrix` function to visualize this "confound design matrix":

```python
ax = plot_design_matrix(confounds2use)
# Rescale the colors a little
ax.get_images()[0].set_clim(0, 0.1)
```

Alright, now we can *finally* fit our model (this may take 2-5 minutes or so, depending on how fast your computer/server is; go get some coffee)!

```python
flm.fit(run_imgs=fmri, events=events2use, confounds=confounds2use)
```

The `fit` function doesn't return the results of the GLM analysis, but stores it as attributes. This includes the constructed design matrix (or matrices, when there is more than one run):

```python
# the design_matrices_ attribute is a list with, in our case, only a single
# element
flm_dm = flm.design_matrices_[0]

# Let's plot it again
ax = plot_design_matrix(flm_dm)
ax.get_images()[0].set_clim(0, 0.2)
```

Another result from the GLM that we can inspect is model fit, expressed as $R^2$ (only when `minimize_memory=False`):

```python
# Again, it returns a list, and we'll take the first element,
# because we only have one run
r2_img = flm.r_square[0]
```

The `r2_img` variable is a 3D `Nifti1Image` object with voxelwise $R^2$ values. We can visualize this using the `plotting` module from Nilearn (e.g., using `plot_stat_map` with an arbitrary threshold of 0.2):

```python
from nilearn import plotting
plotting.plot_stat_map(r2_img, threshold=0.2);
```

Another thing we can inspect is the residual time series from the model fit, which we can retrieve using the `residuals` attribute (you can ignore the `FutureWarning`):

```python
resids = flm.residuals[0]
print(type(resids))
print(resids.shape)
```

    ToDo (optional): If you want to brush up your Nilearn skills, try computing the standard deviation across time for each voxel in the <tt>resids</tt> variable. Store the result (a <tt>Nifti1Image</tt>) in a new variable called <tt>r_std</tt>. Then, plot the result using <tt>plot_stat_map</tt> using a threshold of 3. You should clearly see the brain's veins and contours (effects of movement?) in the plot! Hint: perhaps the function <tt>nilearn.image.math_img</tt> is of use here ... (but this ToDo can be implemented in various ways)

```python
''' Implement the (optional) ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_almost_equal(r_std.get_fdata().mean(), 0.34867, decimal=5)
print("Well done!")
```

```python
# Let's remove the residuals variable, which uses a lot of memory
del flm.residuals, resids
```

### Computing contrasts from first-level models
The last thing we'll discuss in this section is computing contrasts and thresholding the resulting statistical images. Computing contrasts using `FirstLevelModel` objects is ridiculously easy. This is done using the `compute_contrast` method.

    ToDo: Check out the arguments needed for the <tt>compute_contrast</tt> method.

```python
flm.compute_contrast?
```

The `contrast_def` parameter represents the contrast you want compute and can be used in different ways. One way is to specify a contrast vector (a list or a numpy array). For example, if we'd like to evaluate the contrast "strongly_accept > baseline", we could specify the contrast vector as follows:

```python
con_vec = np.zeros(flm.design_matrices_[0].shape[1])
con_vec[0] = 1
print(con_vec)
```

Note that values of the contrast vector are assumed to relate to the regressors froms the design matrix in the same order (e.g., the first value from the contrast vector corresponds to the first regressor in your design matrix, i.e., the first column).

Then, we can compute the contrast as following (assuming a single contrast, i.e., `stat_type='t'`, and wanting $z$-values as output, i.e., `output_type='z_score'`):

```python
con_img = flm.compute_contrast(con_vec, stat_type='t', output_type='z_score')
```

How does that look like?

```python
plotting.plot_stat_map(con_img)
```

The same contrast can be evaluated by using a string describing a "formula". In the "strongly_accept > baseline" contrast, this is trivial:

```python
con_img2 = flm.compute_contrast('strongly_accept', stat_type='t', output_type='z_score')
# check that it is literally the same; if it's not, it will give an error
np.testing.assert_array_equal(con_img.get_fdata(), con_img2.get_fdata())
```

But we can also define more complicated contrasts, such as "(strongly_accept + weakly_accept) > strongly_reject" (note that this is theoretically a nonsense contrast and just used for the sake of the example):

```python
# Note: the same contrast can be defined using the contrast vector:
# [1, -2, 1, 0, 0, 0, 0, ..., 0]
contrast_def = 'strongly_accept + weakly_accept - 2*strongly_reject'
con_img = flm.compute_contrast(contrast_def, stat_type='t', output_type='z_score')
```

Note that $z$-values are not the only output type possible. The other options are:
* `'stat'`: either the $t$-value or $F$-value, depending on `stat_type`;
* `'p_value'`: the $p$-value associated with the $t$ of $F$-value;
* `'effect_size'`: the dot product of the contrast vector and the parameters ($c\hat{\beta}$);
* `'effect_variance'`: the variance of the dot product of the contrast vector and the parameters ($\mathrm{var}[c\hat{\beta}]$);
* `'all'`: all of the above

    ToDo: Using either the "formula" format or the contrast vector format, compute the $t$-values associated with the contrast "reject > accept" (regardless of strongly/weakly). Store the resulting image in a variable named <tt>reject_gt_accept</tt> (gt = greater than) and plot the image using <tt>plot_stat_map</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_almost_equal(reject_gt_accept.get_fdata().mean(), 0.04879, decimal=5)
print("Well done!")
```

So far, we plotted the resulting statistic maps without a threshold (or an arbitrary one), but in a proper analysis, you'd want to statistically threshold your image to reduce the risk of false positives. In Nilearn, this can be done using the function `threshold_stats_img`.

```python
from nilearn.glm.thresholding import threshold_stats_img
```

    ToDo: Checkout the arguments needed for the <tt>threshold_stats_img</tt> function.

```python
threshold_stats_img?
```

As you can see, the primary input to the `threshold_stats_img` function (i.e., `stat_img`) is assumed to consist of $z$-values.

Using `threshold_stats_img`, there are various ways of thresholding your statistical map. Your desired method can be indicated using the `height_control` parameter (either 'fpr', 'fdr', or 'bonferroni', where 'fpr' refers to simple p-value based thresholding without multiple comparison control). The `alpha` parameter represents the significance level that should be used. There is also an option to only include significant voxels that belong to a cluster larger than a certain amount of voxels using `cluster_threshold` (which can make plots a little more visually appealing), but realize that this is by no means a method that guarantees proper false positive control.

Importantly, the `threshold_stats_img` function optionally takes a mask (`mask_img`), which improves sensitivity a lot when using, for example, Bonferroni correction which depends on the number of tests. Also, note that the `threshold_stats_img` function returns two things: the thresholded map and the threshold that was actually used.

Anyway, let's focus on the "strongly_reject > baseline" contrast and let's threshold it using the `'bonferroni'` method with an `alpha` of 0.01 while included the previously defined mask:

```python
con_img = flm.compute_contrast('strongly_reject', stat_type='t', output_type='z_score')
con_img_thr, used_threshold = threshold_stats_img(con_img, mask_img=mask_img, height_control='bonferroni', alpha=0.05)
plotting.plot_stat_map(con_img_thr)
```

    ToDo: threshold the same image (<tt>con_img</tt>), but this time using 'fdr' correction, an alpha of 0.01 and a cluster threshold of 100. Store the result in a variable named <tt>con_img_thr_fdr</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_almost_equal(con_img_thr_fdr.get_fdata().mean(), 0.03169, decimal=5)
print("Well done!")
```

Before we go on to the next section, we want to show you one last nugget: the `make_glm_report` function (see [documentation](https://nistats.github.io/modules/generated/nistats.reporting.make_glm_report.html#nistats.reporting.make_glm_report)). This function takes an *already fitted* `FirstLevelModel` object, a contrast definition, and optionally several parameters related to computing and thresholding the statistic image(s) and returns a summary of the results (as an HTML file, which can be nicely rendered in Jupyter notebooks).

```python
from nilearn.reporting import make_glm_report
# This may take a couple of seconds
# You can use your trackpad or keyboard keys to navigate the cell below
# (make sure you are in "edit" mode, i.e., the cell border should be blue;
#  click the cell to enter "edit" mode)
make_glm_report(flm, 'strongly_accept - weakly_accept')
```

```python
# Let's clear up some memory for the last section
%reset -f
```

## Second-level models
The only major Nilearn functionality that we haven't discussed is *second-level models*. Unlike first-level models, there is basically only one way to fit second-level models: using the `SecondLevelModel` class.

```python
from nilearn.glm.second_level import SecondLevelModel
```

    ToDo: Check out the arguments needed for initialization of a <tt>SecondLevelModel</tt> object.

```python
SecondLevelModel?
```

The second-level model class allows you to run random-effects (but not mixed-effects!) models on multiple participants. Note that it assumes that all data have been registered/resampled to a common space (e.g., MNI, such as our data). It's interface is very similar to the `FirstLevelModel` class (i.e., it also has a `fit` and `compute_contrast` method). When constructing a `SecondLevelModel` object, the most important parameters are `mask_img`, an optional mask, and `smoothing_fwhm`, an optional smoothing kernel.

For our example, let's do a (horribly underpowered) group-level analysis of the two subjects whose data we downloaded. First of all, we need to fit both first-level models. We'll define a function below that does this for a single subject:

```python
import os
import numpy as np
import pandas as pd
import nibabel as nib
from glob import glob
from tqdm.notebook import tqdm
from nilearn import masking
from nilearn.glm.first_level import FirstLevelModel

def fit_firstlevel(sub, bids_dir, task='MGT', run='01', space='MNI152NLin2009cAsym',
                   conf_cols=None, **flm_kwargs):
    """ Example function of how you could implement a complete
    first-level analysis for a single subject. Note that this is
    just one way of implementing this; there may be (much more efficient)
    ways to do this.

    Parameters
    ----------
    sub : str
        Subject-identifier (e.g., 'sub-01')
    bids_dir : str
        Path to BIDS directory (root directory)
    task : str
        Name of task to analyse
    run : str
        Name of run to analyze
    space : str
        Name of space of the data
    conf_cols : list (or None)
        List of confound names to include; if None, only 6 motion params
        are included
    **flm_kwargs : kwargs
        Keyword arguments for the FirstLevelModel constructor

    Returns
    -------
    flm : FirstLevelModel
        Fitted FirstLevelModel object
    """

    # If conf_cols is not set, let's use a "standard" set of
    # motion parameters (translation and rotation in 3 dimensions)
    if conf_cols is None:
        # Note: in new versions of Fmriprep, these variables are named differently,
        # i.e., trans_x, trans_y, trans_z, rot_x, rot_y, rot_z
        conf_cols = ['X', 'Y', 'Z', 'RotX', 'RotY', 'RotZ']

    # We assume it's a BIDS formatted dataset with the Fmriprep outputs in
    # bids_dir/derivatives/fmriprep
    bids_func_dir = os.path.join(bids_dir, sub, 'func')
    fprep_func_dir = os.path.join(bids_dir, 'derivatives', 'fmriprep', sub, 'func')

    # Let's find the fMRI files, given a particular space (e.g., T1w)
    funcs = sorted(glob(os.path.join(fprep_func_dir, f'*space-{space}*_preproc*.nii.gz')))

    # In this loop, we'll find the events/confounds/masks associated with the funcs
    confs, events, masks = [], [], []
    for func in funcs:
        # First, find the associated mask
        # Note, this doesn't work for newer versions of Fmriprep, which uses
        # a slightly different naming convention for brainmasks (desc-brain_mask)
        mask_path = func.replace('preproc', 'brainmask')
        masks.append(mask_path)

        # Find the associated confounds file
        conf_path = func.replace(f'space-{space}_preproc.nii.gz', 'confounds.tsv')
        conf_df = pd.read_csv(conf_path, sep='\t').loc[:, conf_cols]
        confs.append(conf_df)

        # Find the associated events file
        event_path = os.path.join(bids_dir, sub, 'func', f'{sub}_task-{task}_run-{run}_events.tsv')
        event_df = pd.read_csv(event_path, sep='\t')

        # Exclude 'NoResp' trials (not strictly necessary)
        event_df = event_df.query("participant_response != 'NoResp'")

        # Set participant_response as the trial_type
        event_df = event_df.rename({'participant_response': 'trial_type'}, axis=1)
        events.append(event_df)

    # In case there are multiple masks, create an intersection;
    # if not, this function does nothing
    mask_img = masking.intersect_masks(masks, threshold=0.8)

    # Construct the first-level model!
    # We set the t_r to the first func we have, assuming
    # that the TR is the same for each run (if there are multiple runs)
    flm = FirstLevelModel(
        t_r=nib.load(func).header['pixdim'][4],
        slice_time_ref=0.5,
        mask_img=mask_img,
        **flm_kwargs
    )

    # Finally, fit the model and return the fitted model
    flm.fit(run_imgs=funcs, events=events, confounds=confs)
    return flm
```

Now we can create a loop across our two subjects, estimating a first-level model for each:

```python
bids_dir = os.path.join(os.path.expanduser('~'), 'NARPS')
flms = []

# This may take about 2-10 minutes, go get some coffee!
for sub in tqdm(('sub-001', 'sub-003')):
    flm = fit_firstlevel(sub, bids_dir, drift_model='cosine', high_pass=0.1)
    flms.append(flm)
```

Alright, so now we have two fitted `FirstLevelModel` objects (stored in `flms`). We will use these for our second-level model. Like `FirstLevelModel` objects, the `SecondLevelModel` object has a `fit` method, but it is a bit more complicated, because there are different ways to specify the input for second-level models (i.e., the `second_level_input` parameter):
1. a list of fitted `FirstLevelModel` objects (easy, but limited to within-subject/run analyses only);
2. a list of first-level contrast images (i.e., with $c\hat{\beta}$ values);
3. a pandas `DataFrame` with information about the lower-level inputs (useful when using first-level output from other packages, such as FSL, but for Nilearn-only analyses, this is not very useful)

The first method is quite efficient (in terms of code), because it assumes that you are only interested in an intercept-only group-level model (i.e., the average of a particular first-level contrast across participants), and you can leave out the second-level design matrix. We'll show this below:

```python
# We don't supply any mask for simplicity
slm_method1 = SecondLevelModel(smoothing_fwhm=5)
slm_method1.fit(flms)  # we don't have give it a design matrix!
```

The second option is to give the `SecondLevelModel` a list of first-level contrast images in combination with a second-level design matrix. First, let's compute a first-level contrast (say "strongly_accept") for both participants:

```python
fl_cons = []
for flm in flms:
    con = flm.compute_contrast('strongly_accept', stat_type='t', output_type='effect_size')
    fl_cons.append(con)
```

Now we only need to construct a second-level design matrix, which should be a pandas `DataFrame`. The columns in this `DataFrame` represent the different predictors for our second-level model. For simplicity, let's just define an intercept-only design matrix (exactly the same as we did using method 1):

```python
# Specifying the index is not stringly necessary, but it looks nice
slm_dm = pd.DataFrame([1, 1], columns=['intercept'], index=('sub-001', 'sub-003'))
slm_dm
```

Using the first-level contrast images (`fl_cons`) and the second-level design matrix (`slm_dm`), we can fit our second-level model:

```python
slm_method2 = SecondLevelModel(smoothing_fwhm=5)
slm_method2.fit(fl_cons, design_matrix=slm_dm)
```

Using this fitted second-level model for either method, we can also compute second-level contrasts. The way this is done is slightly different for each method. For method 1 (using fitted `FirstLevelModel` objects as inputs), the only option is to use the `first_level_contrast` parameter in the `compute_contrast` method. You can only use contrast definitions that were already present in the first-level model (because there is no second-level design matrix).

For example, if we'd want to compute the average (across participants) group-level contrast of "strongly_accept > baseline", we'd do:

```python
sr_average_method1 = slm_method1.compute_contrast(first_level_contrast='strongly_accept', output_type='z_score')
```

For method 2 (using first-level contrast images + a second-level design matrix), we cannot use the `first_level_contrast` parameter; instead, we should use the `second_level_contrast` parameter, which applies to contrasts based on the second-level design matrix. Because we also defined an intercept-only model for method 2, the only contrast we *can* evaluate is the intercept contrast:

```python
sr_average_method2 = slm_method2.compute_contrast(second_level_contrast='intercept', output_type='z_score')
```

Note that the the results from both methods are virtually the same; the only difference is the way we used the `SecondLevelModel` interface.

    ToDo Alright then, one last ToDo! Suppose that we want to construct a group-level model that investigates the difference between the first-level "strongly_accept" contrast images from sub-001 and sub-003 (a nonsensical analysis for just two subjects, but just ignore that for now). Can you construct an appropriate second-level design matrix for this analysis? Store this design matrix (a <tt>DataFrame</tt>) in a variable called <tt>slm_dm_todo</tt>.

```python
''' Implement your ToDo here. '''
# YOUR CODE HERE
raise NotImplementedError()
```

```python
''' Tests the above ToDo. '''
np.testing.assert_array_equal(slm_dm_todo.sum(axis=0).values, [1, 1])
np.testing.assert_array_equal(slm_dm_todo.sum(axis=1).values, [1, 1])
print("Well done!")
```

## Concluding remarks
Hopefully this tutorial gave you an idea how to use Nilearn to run statistical models! We recommend you check out the ample excellent tutorials and examples on the Nilearn [website](https://nilearn.github.io/) or to check out the codebase on [Github](https://github.com/nilearn/nilearn). As said before, Nilearn is a relatively novel project and its codebase may still change, so keep an eye out for new releases and extended functionality!

That said, we hope that this tutorial helps you to get started with your analyses using Nilearn.

Happy hacking!


```python
%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)

from IPython.display import Image
```

Chapter 9 On-policy Prediction with Approximation
=========

approximate value function: parameterized function $\hat{v}(s, w) \approx v_\pi(s)$

+ applicable to partially observable problems.

### 9.1 Value-function Approximation

$s \to u$: $s$ is the state updated and $u$ is the update target that $s$'s estimated value is shifted toward.

We use machine learning methods and pass to them the $s \to g$ of each update as a training example. Then we interperet the approximate function they produce as an estimated value function.

not all function approximation methods are equally well suited for use in reinforcement learning:
+ learn efficiently from incrementally acquired data: many traditional methods assume a static training set over which multiple passes are made.
+ are able to handle nonstationary target functions.

### 9.2 The Prediction Objective (VE)

which states we care most about: a state distribution $\mu(s) \geq 0$, $\sum_s \mu(s) = 1$.
+ Often $\mu(s)$ is chosen to be the fraction of time spent in $s$.

objective function, the Mean Squared Value Error, denoted $\overline{VE}$:

\begin{equation}
    \overline{VE}(w) \doteq \sum_{s \in \delta} \mu(s) \left [ v_\pi (s) - \hat{v}(s, w) \right ]^2
\end{equation}

where $v_\pi(s)$ is the true value and $\hat{v}(s, w)$ is the approximate value.

Note that best $\overline{VE}$ is no guarantee of our ultimate purpose: to find a better policy.
+ global optimum.
+ local optimum.
+ don't convergence, or diverge.

### 9.3 Stochastic-gradient and Semi-gradient Methods

SGD: well suited to online reinforcement learning.

\begin{align}
    w_{t+1} &\doteq w_t - \frac1{2} \alpha \nabla \left [ v_\pi(S_t) - \hat{v}(S_t, w_t) \right ]^2 \\
    &= w_t + \alpha \left [ \color{blue}{v_\pi(S_t)} - \hat{v}(S_t, w_t) \right ] \nabla \hat{v}(S_t, w_t) \\
    &\approx w_t + \alpha \left [ \color{blue}{U_t} - \hat{v}(S_t, w_t) \right ] \nabla \hat{v}(S_t, w_t) \\
\end{align}

$S_t \to U_t$, is not the true value $v_\pi(S_t)$, but some, possibly random, approximation to it. (前面各种方法累计的value）:
+ If $U_t$ is an unbiased estimate, $w_t$ is guaranteed to converge to a local optimum.
+ Otherwise, like boostrappig target or DP target => semi-gradient methods. (might do not converge as robustly as gradient methods)
  - significantly faster learning.
  - enable learning to be continual and online.

state aggregation: states are grouped together, with one estimated value for each group.

### 9.4 Linear Methods

For every state $s$, there is a real-valued feature vector $x(s) \doteq (x_1(s), x_2(s), \dots, x_d(s))^T$:

\begin{equation}
    \hat{v}(s, w) \doteq w^T x(s) \doteq \sum_{i=1}^d w_i x_i(s)
\end{equation}

### 9.5 Feature Construction for Linear Methods

Choosing features appropriate to the task is an important way of adding prior domain knowledge to reinforcement learing systems.

+ Polynomials
+ Fourier Basis: low dimension, easy to select, global properities
+ Coarse Coding
+ Tile Coding: convolution kernel?
+ Radial Basis Functions

### 9.6 Selecting Step-Size Parameters Manually

A good rule of thumb for setting the step-size parameter of linear SGD methods is then $\alpha \doteq (\gamma \mathbf{E}[x^T x])^{-1}$

### 9.7 Nonlinear Function Approximation: Artificial Neural Networks

+ANN, CNN

### 9.8 Least-Squares TD

$w_{TD} = A^{-1} b$: data efficient, while expensive computation

### 9.9 Memory-based Function Approximation

nearest neighbor method

### 9.10 Kernel-based Function Approximation

RBF function

### 9.11 Looking Deeper at On-policy Learning: Interest and Emphasis

more interested in some states than others:
+ interest $I_t$: the degree to which we are interested in accurately valuing the state at time $t$.
+ emphaisis $M_t$:

\begin{align}
    w_{t+n} & \doteq w_{t+n-1} + \alpha M_t \left [ G_{t:t+n} - \hat{v}(S_t, w_{t+n-1} \right ] \nabla \hat{v}(S_t, w_{t+n-1}) \\
    M_t & = I_t + \gamma^n M_{t-n}, \qquad 0 \leq t < T
\end{align}

```python

```