## Introduction

This notebook will go over a how to use propensity_scores in mr_uplift packages.The main change is that builds a random forest of the form $t = f(x)$ for the propensity model. It them predicts and calculates the inverse probability as the observation weight:  1/f(x). Note that the propensity model is a multi-classification model and supports more than two treatments. This weight is then fed into both the ERUPT calculations and loss functions of the uplift model.

```python
import numpy as np
import pytest

from mr_uplift.dataset.data_simulation import get_observational_uplift_data_2
from mr_uplift.mr_uplift import MRUplift, get_t_data
from mr_uplift.keras_model_functionality import prepare_data_optimized_loss
import sys
import pandas as pd

```

### Data Generating Process

Below is the data generating process of the data. This is similar to the previous data function except that the treatment assignment is a function of the explanatory variable $x_1$. This should simulate a more observational scenario.

\begin{equation}
x_1  \sim runif(0,1)
\end{equation}

\begin{equation}
x_2 \sim runif(0,1)
\end{equation}

\begin{equation}
t \sim rbinom(.2+I(x_1>0)*.6 )
\end{equation}

\begin{equation}
noise \sim rnorm(0,1)
\end{equation}

\begin{equation}
y = x_1*t + noise
\end{equation}

Below the data is generated and two models are built. The first is an uplift model using MSE loss and does not weight observations based on explanatory variables. The second is an uplift model that uses the optimized loss with observations weighted by the propensity model.

```python
y, x, t = get_observational_uplift_data_2(10000)

y = y[:,0].reshape(-1,1)
uplift_model = MRUplift()
param_grid = dict(num_nodes=[8,64], dropout=[.1, .5], activation=[
                          'relu'], num_layers=[1], epochs=[25], batch_size=[512])

uplift_model.fit(x, y, t.reshape(-1,1),
                 param_grid = param_grid, n_jobs = 1)

uplift_model_propensity = MRUplift()
param_grid_propensity = dict(num_nodes=[8, 64], dropout=[.1, .5], activation=[
                          'relu'], num_layers=[1], epochs=[25], batch_size=[512],
                 alpha = [.9999,.99], copy_several_times = [1])

uplift_model_propensity.fit(x, y, t.reshape(-1,1),
                 param_grid = param_grid_propensity, n_jobs = 1,
                optimized_loss = True, use_propensity = True)

```

```python
#calculate oos predictions

y_test, x_test, t_test = get_observational_uplift_data_2(10000)

pred_uplift = uplift_model.predict_optimal_treatments(x = x_test)
pred_uplift_propensity = uplift_model_propensity.predict_optimal_treatments(x = x_test,
                use_propensity_score_cutoff = True)
correct_tmt = (x_test[:,0]>0).reshape(-1,1)*1
```

FInally the cells below compare the estimated optimal decisions with correct ones for both models. Note that the propensity mdoel correctly identifies optimal treatment for 58% of observations while the standard model only identifies 66% of observations correctly

```python
print('Uplift Model Correct Decisions')
print((pred_uplift == correct_tmt).mean())

print('Uplift Model using Propensity Correct Decisions')
print((pred_uplift_propensity == correct_tmt).mean())
```

    Uplift Model Correct Decisions
    0.5754
    Uplift Model using Propensity Correct Decisions
    0.9886

Finally I calculate the estimated expected responses under propoposed treatments vs actual for both models.

Without the propensity model the estimate ERUPT is ~.018 which is higher than the actual response under proposed treatments of .06.

With the propensity model the ERUPT metric is much closer to actual treatments.

```python
#Estimated uplift vs actual
print(uplift_model.get_erupt_curves())

print('Actual Estimate of Gains for Uplift Model')
print((pred_uplift*x_test[:,0].reshape(-1,1)).mean())
```

    Using Test Data Set
    (       mean       std response_var_names weights assignment treatment
    0  0.018898  0.001978              var_0       1      model        -1
    0  0.008591  0.002204              var_0       1     random        -1
    0 -0.001738  0.001734              var_0      -1        ate       0.0
    0  0.150436  0.004449              var_0      -1        ate       1.0,    num_observations  tmt weights  percent_tmt
    0              6433  0.0       1        0.919
    1               567  1.0       1        0.081)
    Actual Estimate of Gains for Uplift Model
    0.01450183238033326

```python
#Estimated uplift with propensity vs actual
print(uplift_model_propensity.get_erupt_curves())

print('Actual Estimate of Gains for Uplift Model with propensity scores')
print((pred_uplift_propensity*x_test[:,0].reshape(-1,1)).mean())
```

    Using Test Data Set
    (       mean       std response_var_names weights assignment treatment
    0  0.127240  0.002598              var_0       1      model        -1
    0  0.001792  0.004851              var_0       1     random        -1
    0 -0.001134  0.002104              var_0      -1        ate       0.0
    0  0.010204  0.006303              var_0      -1        ate       1.0,    num_observations  tmt weights  percent_tmt
    0              3658  1.0       1     0.522571
    1              3342  0.0       1     0.477429)
    Actual Estimate of Gains for Uplift Model with propensity scores
    0.1262029577145106

```python
print('Theoretical Best')
print((correct_tmt*x_test[:,0].reshape(-1,1)).mean())
```

    Theoretical Best
    0.12626648857998107


- [Forward Problem](#Forward-Problem)
    - [Verify Assumption 1](#Verify-Assumption-1)
    - [Verify Lemma 1](#Verify-Lemma-1)
    - [Verify Theorem 3.1](#Verify-Theorem-3.1)
- [Inverse Problem](#Inverse-Problem)
    - [Verify Assumption 2](#Verify-Assumption-2)
    - [Verify Theorem 4.2](#Verify-Theorem-4.2)

```python
import numpy as np
import numpy.polynomial.hermite_e as H
from scipy.stats import norm
from math import factorial
from scipy.integrate import odeint
from scipy.stats import gaussian_kde as kde
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets

from matplotlib import pyplot as plt
%matplotlib inline
```

```python
####### Plot Formatting ######
plt.rc('lines', linewidth = 2)
plt.rc('xtick', labelsize = 14)
plt.rc('ytick', labelsize = 14)
plt.rc('legend',fontsize=14)
plt.rcParams["font.family"] = "serif"
plt.rcParams['axes.labelsize'] = 20
plt.rcParams['axes.titlesize'] = 12
plt.rcParams['lines.markersize'] = 8
plt.rcParams['figure.figsize'] = (8.0, 6.0)
```

# Example from Xiu2002

$$ \frac{dy(t)}{dt} = -\lambda y, \ \  y(0)=1 $$
$$ y(t) = e^{-\lambda t} $$
$$QoI = y(0.5)$$
$\Lambda = (-\infty, \infty)$, $\mathcal{D}=(0,\infty)$

```python
def Phi(n):
    '''Define H_n'''
    coeffs = [0]*(n+1)
    coeffs[n] = 1
    return coeffs

def inner2_herm(n):
    return np.sqrt(2*np.pi)*factorial(n)

def product3_herm(i,j,l):
    #compute \Phi_i*\Phi_j*\Phi_l
    return lambda x: H.hermeval(x, H.hermemul(H.hermemul(Phi(i),Phi(j)),Phi(l)))

def inner3_herm(i,j,l):
    '''
    compute <\Phi_i\Phi_j\Phi_l>
    Set up Gauss-Hermite quadrature, weighting function is exp^{-x^2}
    '''

    x, w=H.hermegauss(20)
    inner=sum([product3_herm(i,j,l)(x[idx]) * w[idx] for idx in range(20)])

    return inner
```

```python
def ode_system_herm(y, t, P):
    '''P indicates highest order of Polynomial we use'''
    dydt = np.zeros(P+1)
    for l in range(len(dydt)):
        dydt[l] = -(sum(sum(inner3_herm(i,j,l)*ki_herm[i]*y[j] for j in range(P+1)) for i in range(P+1)))/inner2_herm(l)
    return dydt
```

```python
P=5
ki_herm = [0,1]+[0]*(P-1)
sol_herm = odeint(ode_system_herm, [1.0]+[0.0]*P, np.linspace(0,1,101), args=(P,))
```

```python
def a(i):
    return sol_herm[:,i][50]
coef = np.array([a(0), a(1), a(2), a(3), a(4), a(5)])   #fixed

def Q(i,x):
    return H.hermeval(x,coef[:(i+1)])

def Qexact(x):
    return np.exp(-x*0.5)
```

```python
#### Use plot to show the difference between the exact and approximate map #####
fig = plt.figure()

def plot_Qn(n):
    fig.clear()
    x = np.linspace(-3,3,100)
    y = Qexact(x)
    yn = Q(n, x)
    plt.plot(x,y,linestyle='-.',linewidth=4,label="$Q(\lambda)$")
    plt.plot(x,yn,label='Q_'+str(n)+'$(\lambda)$')
    plt.xlabel('$\Lambda$')
    plt.legend();

interact(plot_Qn,
         n = widgets.IntSlider(value=1,min=1,max=5,step=1));
```

    <Figure size 576x432 with 0 Axes>

    interactive(children=(IntSlider(value=1, description='n', max=5, min=1), Output()), _dom_classes=('widget-inte…

## Forward Problem

$\lambda\sim N(0,1)$, QOI is the value at $t=0.5$ ($y(0.5)$). $Q_n$ defines the Polynomial Chaos expansion with degree $n$.

$$
Q(\lambda)=y(0.5)=\sum\limits_{i=0}^{\infty} y_i(0.5)\Phi_i
$$
$$
Q_n(\lambda)=\sum\limits_{i=0}^n y_i(0.5)\Phi_i
$$

Verify Result of Lemma 2:

$Q_n(\lambda)\to Q(\lambda)$ in $L^p(\Lambda)$, if Assumptions 1 holds and $D_c\subset\mathcal{D}$ being compact, then
\begin{equation}
	\pi_{\mathcal{D}}^{Q_n}(q) \to \pi_{\mathcal{D}}^{Q}(q) \text{ almost in} {L^r(D_c)}
\end{equation}

Verify Result of Theorem 3.1:

$Q_n(\lambda)\to Q(\lambda)$ in $L^p(\Lambda)$, if Assumptions 1 holds, $\{\pi_{\mathcal{D}}^{Q_n}\}$ are uniformly integrable in $L^p(\mathcal{D})$, $\mathcal{D}$ is compact, then
\begin{equation}
        \pi_{\mathcal{D}}^{Q_n}(Q_n(\lambda)) \to \pi_{\mathcal{D}}^{Q}(Q(\lambda)) \text{ in } L^p(\Lambda)
\end{equation}

### Verify Assumption 1

```python
def assumption1(n, J):
    np.random.seed(123456)
    initial_sample = np.random.normal(size=J)
    pfprior_sample_n = Q(n, initial_sample)
    pfprior_dens_n = kde(pfprior_sample_n)
    x = np.linspace(-1, 4.5, 1000)
    return np.round(np.max(np.abs(np.gradient(pfprior_dens_n(x), x))), 2), np.round(np.max(pfprior_dens_n(x)), 2)

size_J = [int(1E3), int(1E4), int(1E5)]
degree_n = [1, 2, 3, 4, 5]
Bound_matrix, Lip_Bound_matrix = np.zeros((3, 5)), np.zeros((3, 5))
for i in range(3):
    for j in range(5):
        n, J = degree_n[j], size_J[i]
        Lip_Bound_matrix[i, j] = assumption1(n, J)[0]
        Bound_matrix[i, j] = assumption1(n, J)[1]
```

```python
print('Bound under certain n and J values')
print(Bound_matrix)
```

    Bound under certain n and J values
    [[0.71 0.84 0.89 0.87 0.86]
     [0.71 0.87 0.9  0.87 0.86]
     [0.7  0.92 0.94 0.9  0.9 ]]

```python
print('Lipschitz bound under certain n and J values')
print(Lip_Bound_matrix)
```

    Lipschitz bound under certain n and J values
    [[0.88 2.09 1.76 1.6  1.62]
     [0.84 4.07 2.49 2.14 2.21]
     [0.81 7.01 2.74 2.22 2.32]]

```python
#### Use plot to show the difference between the exact pushforward and approximate pushforward #####
fig=plt.figure()

def plot_pushforward(n,J):
    np.random.seed(123456)
    initial_sample = np.random.normal(size = J)

    pfprior_sample = Qexact(initial_sample)
    pfprior_dens = kde(pfprior_sample)

    pfprior_sample_n = Q(n,initial_sample)
    pfprior_dens_n = kde(pfprior_sample_n)

    fig.clear()
    x = np.linspace(-1,4.5,1000)
    y = pfprior_dens(x)
    yn = pfprior_dens_n(x)
    plt.plot(x,y,color='r', linestyle='-.', linewidth=4, label="$\pi_{\mathcal{D}}^Q$")
    plt.plot(x,yn,linewidth=2,label="$\pi_{\mathcal{D}}^{Q_{n}}$")
    plt.title('Lipschitz const. = %4.2f and Bound = %2.2f' %(np.max(np.abs(np.gradient(pfprior_dens_n(x), x))),
                                                                             np.max(pfprior_dens_n(x))))
    plt.xlabel("$\mathcal{D}$")
    plt.legend();

interact(plot_pushforward,
         n = widgets.IntSlider(value=1,min=1,max=5,step=1),
         J = widgets.IntSlider(value=int(1E3),min=int(1E3),max=int(1E5),step=int(1E3)))
```

    <Figure size 576x432 with 0 Axes>

    interactive(children=(IntSlider(value=1, description='n', max=5, min=1), IntSlider(value=1000, description='J'…

    <function __main__.plot_pushforward(n, J)>

### Verify Lemma 1
**Print out Monte Carlo Approximation of $	\|\pi_{\mathcal{D}}^Q(q)-\pi_{\mathcal{D}}^{Q_n}(q)\|_{L^r(\mathcal{D_c})} $ where $r>0$ and $D_c=[1,4]$**

```python
#Build $\pi_D^Q$ and $\pi_D^{Q,n}$, use 10,000 samples
N_kde = int(1E4)
N_mc = int(1E4)
np.random.seed(123456)
initial_sample = np.random.normal(size = N_kde)
pfprior_sample = Qexact(initial_sample)
pfprior_dens = kde(pfprior_sample)

def pfprior_dens_n(n,x):
    pfprior_sample_n = Q(n,initial_sample)
    pdf = kde(pfprior_sample_n)
    return pdf(x)
```

```python
error_r_D = np.zeros((5,5))
np.random.seed(123456)
qsample = np.random.uniform(1,4,N_mc)
for i in range(5):
    for j in range(5):
        error_r_D[i,j] = (np.mean((np.abs(pfprior_dens(qsample) - pfprior_dens_n(j+1,qsample)))**(i+1)))**(1/(i+1))
```

```python
np.set_printoptions(linewidth=110)
print('L^r error on data space for Forward Problem',end='\n\n')
print(error_r_D)
```

    L^r error on data space for Forward Problem

    [[0.05194112 0.01350692 0.0035189  0.00081095 0.00017277]
     [0.0765559  0.02297857 0.00564043 0.0014609  0.00028286]
     [0.09346265 0.03183619 0.00723372 0.00205836 0.00037162]
     [0.10484368 0.03904701 0.00835824 0.00255138 0.00043737]
     [0.11299713 0.04469492 0.00917825 0.00294367 0.00048629]]

```python
fig = plt.figure()
plt.xlim([0,6])
marker = ['-D', '-o', '-v', '-s', '-.']
for i in range(5):
    plt.semilogy([1,2,3,4,5],error_r_D[i,:],marker[i],label='r = ' + np.str(i+1))
plt.xlabel('Order of PCE (n)')
plt.ylabel('$L^r$'+' Error in Push-Forward on '+'$\mathcal{D}$')
plt.legend();
```

### Verify Theorem 3.1
**Print out Monte Carlo Approximation of $	\|\pi_{\mathcal{D}}^Q(Q(\lambda))-\pi_{\mathcal{D}}^{Q_n}(Q_n(\lambda))\|_{L^2(\Lambda)} $**

```python
np.random.seed(123456)
lamsample = np.random.normal(size = N_mc)

error_2 = np.zeros(5)
for i in range(5):
    error_2[i] = (np.mean((np.abs(pfprior_dens(Qexact(lamsample))\
                                  - pfprior_dens_n(i+1,Q(i+1,lamsample))))**2))**(1/2)
```

```python
np.set_printoptions(linewidth=110)
print('L^2 error on parameter space for Forward Problem',end='\n\n')
print(error_2)
```

    L^2 error on parameter space for Forward Problem

    [0.1962035  0.10691977 0.02296583 0.00508628 0.00096028]

```python
fig = plt.figure()
plt.xlim([0,6])
plt.semilogy([1,2,3,4,5],error_2,'-s')#,label='$L^2(\Lambda)$ error')
plt.xlabel('Order of PCE (n)')
plt.ylabel('$L^2$'+' Error in Push-Forward on '+'$\Lambda$');
```

## Inverse Problem

Initial guess is $\lambda\sim N(0,1^2)$.

Observation is $\pi_{\mathcal{D}}\sim N(3,0.5^2)$

Verify Result of Theorem 4.2:

$Q_n(\lambda)\to Q(\lambda)$ in $L^p(\Lambda)$, $\pi_{\Lambda}^{init}\in L^p(\mathcal{D})$. If Assumptions 1, 2 hold, $\{\pi_{\mathcal{D}}^{Q_n}\}$ are uniformly integrable in $L^p(\mathcal{D})$, then
\begin{equation}
        \pi_{\Lambda}^{u,n}(\lambda) \to \pi_{\Lambda}^{u}(\lambda) \text{ in } L^p(\Lambda)
\end{equation}

```python
def pdf_obs(x):
    return norm.pdf(x, loc=3, scale=0.5)
```

```python
#### Use plot to show the difference between the pushforward of the init and the observed #####
fig = plt.figure()
xx = np.linspace(-1,8,1000)
y = pdf_obs(xx)
y_pf = pfprior_dens(xx)
plt.plot(xx,y,label="$\pi_{\mathcal{D}}^{obs}$")
plt.plot(xx,y_pf, label="$\pi_{\mathcal{D}}^{Q(init)}$")
plt.xlabel("$\mathcal{D}$")
plt.legend();
```

### Verify Assumption 2

```python
def Meanr(n):
    pfprior_sample_n = Q(n,initial_sample)
    if n==0:
        r = pdf_obs(pfprior_sample)/pfprior_dens(pfprior_sample)
    else:
        r = pdf_obs(pfprior_sample_n)/pfprior_dens_n(n,pfprior_sample_n)
    return np.mean(r)

def pdf_update(n,x):
    if n==0:
        r = pdf_obs(pfprior_sample)/pfprior_dens(pfprior_sample)
        pdf = kde(initial_sample,weights=r)
    else:
        pfprior_sample_n = Q(n,initial_sample)
        r = pdf_obs(pfprior_sample_n)/pfprior_dens_n(n,pfprior_sample_n)
        pdf = kde(initial_sample,weights=r)
    return pdf(x)

Expect_r = np.zeros(6)
for i in range(6):
    Expect_r[i] = Meanr(i)
```

```python
print('Expected ratio for verifying Assumption 2')
print(Expect_r[1:])
```

    Expected ratio for verifying Assumption 2
    [0.58714658 0.97697376 0.98109416 0.98064304 0.98038583]

```python
#### Use plot to show the difference between the initial, updated, approximate updated #####
fig=plt.figure()
def plot_update(n):
    fig.clear()
    xx = np.linspace(-5,5,100)
    plt.plot(xx, norm.pdf(xx, loc=0, scale=1), label="Initial Density")
    plt.plot(xx, pdf_update(0,xx), label="$\pi_{\Lambda}^u$")
    plt.plot(xx, pdf_update(n,xx), label="$\pi_{\Lambda}^{u,n}$, n="+str(n))
    plt.legend()
    plt.xlabel("$\Lambda$")
    plt.title('$\mathbb{E}(r) =$ %3.2f' %(Expect_r[n]));

interact(plot_update,
         n = widgets.IntSlider(value=int(1),min=int(1),max=int(5),step=1))
```

    <Figure size 576x432 with 0 Axes>

    interactive(children=(IntSlider(value=1, description='n', max=5, min=1), Output()), _dom_classes=('widget-inte…

    <function __main__.plot_update(n)>

```python
#### Use plot to show the difference between the observed and the pushforward of the approximate updated pdf #####
def update_pushforward(n,x):
    pfprior_sample_n = Q(n,initial_sample)
    r = pdf_obs(pfprior_sample_n)/pfprior_dens_n(n,pfprior_sample_n)
    pdf = kde(pfprior_sample_n,weights=r)
    return pdf(x)

fig = plt.figure()
xx = np.linspace(0,6,100)
y = pdf_obs(xx)
plt.plot(xx,y,label="$\pi_{\mathcal{D}}^{obs}$")
for i in range(1,6,1):
    y_pf = update_pushforward(i,xx)
    plt.plot(xx,y_pf, label="n="+str(i))
plt.xlabel("$\mathcal{D}$")
plt.legend();
```

### Verify Theorem 4.2

Print out Monte Carlo Approximation of $\|\pi_{\Lambda}^{u,n}(\lambda)-\pi_{\Lambda}^u(\lambda)\|_{L^2(\Lambda)} $

```python
np.random.seed(123456)
lamsample = np.random.normal(size = N_mc)

error_update = np.zeros(5)
for i in range(5):
    error_update[i] = (np.mean((np.abs(pdf_update(0,lamsample) - pdf_update(i+1,lamsample)))**2))**(1/2)
```

```python
np.set_printoptions(linewidth=110)
print('L^2 Error for Inverse Problem',end='\n\n')
print(error_update)
```

    L^2 Error for Inverse Problem

    [0.11973975 0.03226845 0.00655868 0.00194303 0.00036956]

```python
fig = plt.figure()
plt.xlim([0,6])
plt.semilogy([1,2,3,4,5],error_update,'-s')#,label='$L^2(\Lambda)$ error')
plt.xlabel('Order of PCE (n)')
plt.ylabel('$L^2$'+' Error in Update');
```


# QuTiP Example: Stinespring Dilation of Quantum Channels

[Christopher Granade](http://www.cgranade.com/)

University of Sydney
$\newcommand{\ket}[1]{\left|#1\right\rangle}$
$\newcommand{\bra}[1]{\left\langle#1\right|}$
$\newcommand{\cnot}{{\scriptstyle \rm CNOT}}$
$\newcommand{\TT}{\operatorname{T}}$
$\newcommand{\Tr}{\operatorname{Tr}}$
$\newcommand{\Uni}{\operatorname{U}}$
$\newcommand{\Chan}{\operatorname{C}}$
$\newcommand{\Hil}{\mathcal{H}}$

## Introduction

In this notebook, we will demonstrate the ``to_stinespring`` function, which converts a ``Qobj`` representing the quantum channel $\Phi$ to a pair of partial isometries that describe the action of $\Phi$.

In introducing the Stinespring dilation, it is helpful to first adopt some notation. Let $\Hil_X$, $\Hil_Y$ and $\Hil_Z$ be finite-dimensional Hilbert spaces. Let $\Uni(\Hil_X, \Hil_Y)$ be the set of partial isometries from $\Hil_X$ to $\Hil_Y$, and $\TT(\Hil_X, \Hil_Y)$ be the set of maps (that is, superoperators) from operators on $\Hil_X$ to those acting on $\Hil_Y$. Finally, let $\Chan(\Hil_X, \Hil_Y) \subset \TT(\Hil_X, \Hil_Y)$ be the set of *channels*; that is, those maps which are completely positive and trace-preserving.

For a quantum map $\Phi \in \TT(\Hil_X, \Hil_Y)$, then, the Stinespring dilation of $\Phi$ is a pair of partial isometries $A, B \in \Uni(\Hil_X, \Hil_Y \otimes \Hil_Z)$ for some space $\Hil_Z$ such that
\begin{equation}
    \Phi(X) = \Tr_Z (A X B^\dagger).
    \label{eq:stinespring-action}
\end{equation}
If $\Phi$ is completely positive, then $A = B$. We do not insist on complete positivity, however, as it is common to consider the Stinespring dilation for the *difference* between two quantum channels, $\Phi - \Phi'$. By construction, such a map is not completely positive, nor is it trace-preserving.

Informally, the ancillary space $\Hil_Z$ serves the same role as the Kraus index. If $\Phi$ is a channel with Kraus operators $\{K_i : i \in \{0, \dots, k - 1\}\}$, then $A = B = \sum_i K_i \otimes \ket{i}$, such that summation over $i$ is performed by the partial trace in \eqref{eq:stinespring-action}.

The Stinespring representation is useful for [evaluating norms](http://theoryofcomputing.org/articles/v005a011/) and for building system/environment models of a quantum channel.

### Further Resources

- Watrous. [CS 766: Theory of Quantum Information (lecture notes)](https://cs.uwaterloo.ca/~watrous/CS766/) (2013).
-  Wood, Biamonte and Cory. *Tensor networks and graphical calculus for open quantum systems*. [arXiv:1111.6950](https://scirate.com/arxiv/1111.6950) (2011).
- Watrous. *Semidefinite Programs for Completely Bounded Norms*. [Theory of Computing **5** 217-238](http://theoryofcomputing.org/articles/v005a011/), [arXiv:0901.4709](https://scirate.com/arxiv/0901.4709) (2009).

## Preamble

### Features

We enable a few features such that this notebook runs in both Python 2 and 3.

```python
from __future__ import division, print_function
```

### Imports

```python
import numpy as np
import qutip as qt

from qutip.ipynbtools import version_table
```

### Plotting Support

```python
%matplotlib inline
```

### Settings

```python
qt.settings.colorblind_safe = True
```

##  Dilations of CPTP Maps

We'll start by considering the case in which $\Phi(X) = U X U^\dagger$ for a random unitary operator $U \in \Uni(\Hil_X, \Hil_X)$.

```python
phi = qt.to_super(qt.rand_unitary_haar(4))
```

```python
qt.visualization.hinton(phi)
```

By construction, $\Phi$ has one Kraus operator--- the random unitary itself. Because of numerical precision in finding eigenvalues of the Choi matrix, a decomposition will result in many operators with very small norms.

```python
for K in qt.to_kraus(phi):
    print(K.norm())
```

    1.44965724751e-08
    4.0
    2.17301265733e-08
    1.52222408748e-08
    1.5017315825e-08
    1.76642230882e-08
    1.37895174864e-08
    1.40385925527e-08
    1.07348372039e-08
    9.50757741411e-09
    6.71512203218e-09
    1.14139195174e-08
    1.01210695111e-08
    6.00690608844e-09
    3.38230626595e-09
    4.01926307771e-10

Because $\Phi$ only has one Kraus operator, the Stinespring dilation will use a 1-dimensional ancillary space.

```python
A, B = qt.to_stinespring(phi)
print(A.dims)
```

    [[4, 1], [4]]

If we ask for a random channel of Choi rank 2 (that is, with 2 Kraus operators), then the corresponding space will be two-dimensional instead.

```python
A, B = qt.to_stinespring(qt.rand_super_bcsz(4, rank=2))
print(A.dims)
```

    [[4, 2], [4]]

Moreover, since we have demanded that $A$ and $B$ are the Stinespring pair for a channel, we can easily verify that $A = B$.

```python
A - B
```

Quantum object: dims = [[4, 2], [4]], shape = (8, 4), type = other\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

## Differences of Random Channels

Next, let's consider two random qubit channels from the [BCSZ distribution]([``qutip.visualization.hinton``](http://qutip.org/docs/3.0.1/apidoc/functions.html#qutip.random_objects.rand_super_bcsz).

```python
phi1 = qt.rand_super_bcsz(2)
phi2 = qt.rand_super_bcsz(2)
```

Looking at the Pauli-basis Hinton diagram for the difference between these channels gives some insight into how this map behaves.

```python
qt.visualization.hinton(phi1 - phi2)
```

In particular, note that $\Delta\Phi := \Phi_1 - \Phi_2$ annilates the traceful parts of its input, leaving an operator with negative eigenvalues.

```python
d_phi = qt.to_super(phi1 - phi2)
rho_in = qt.rand_dm(2)
rho = qt.vector_to_operator(d_phi * qt.operator_to_vector(rho_in))
print("Trace: {}\nEigenvalues:{}".format(rho.tr(), rho.eigenenergies()))
rho
```

    Trace: -3.885780586188048e-16
    Eigenvalues:[-0.39781299  0.39781299]

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}-0.334 & (-0.211+0.051j)\\(-0.211-0.051j) & 0.334\\\end{array}\right)\end{equation*}

Because $\Delta \Phi$ is not completely positive, we cannot obtain a Kraus decomposition where the left and right operators are the same:

```python
Ks = qt.to_kraus(d_phi)
```

```python
rho - sum(K * rho_in * K.dag() for K in Ks)
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}-0.993 & (-0.141+0.001j)\\(-0.141-0.001j) & -0.327\\\end{array}\right)\end{equation*}

On the other hand, if we allow the left and right Kraus operators to be different, or consider a Stinespring dilation, we get the right answer:

```python
A, B = qt.to_stinespring(d_phi)
rho - (A * rho_in * B.dag()).ptrace((0,))
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0\\0.0 & 0.0\\\end{array}\right)\end{equation*}

## Constructing System-Environment Models

For a channel $\Phi \in \Chan(\Hil_X, \Hil_X)$, once we have found the Stinespring dilation $A$, we then represent $\Phi$ as a *unitary* on the full space, $U \in \Uni(\Hil_X \otimes \Hil_Z)$, interpreting $\Hil_Z$ as a preparation of an environment. Concretely, we want to find $U$ such that for a preparation $\ket{\psi}$ of the environment,
\begin{equation}
    \Phi(X) = \Tr_Z(U \rho \otimes \ket{\psi}\bra{\psi} U^\dagger).
\end{equation}
By convention, we'll choose a basis for $\Hil_Z$ such that $\ket{\psi} = \ket{0}$. Then, if we let $V = A \otimes \bra{0}$,
\begin{equation}
    \Tr_Z(V \rho \otimes \ket{0}\bra{0} V^\dagger) =
    \Tr_Z(A \rho \otimes \bra{0}\ket{0}\bra{0}\ket{0} A) =
    \Tr_Z(A \rho A^\dagger)
\end{equation}
as desired.

```python
A, B = qt.to_stinespring(qt.rand_super_bcsz(3))
V = qt.tensor(A, qt.basis(A.dims[0][-1], 0).dag())
# This adds an annoying left index, so let's drop it now.
del V.dims[0][-1]
V
```

Quantum object: dims = [[3, 9], [3, 9]], shape = (27, 27), type = oper, isherm = False\begin{equation*}\left(\begin{array}{*{11}c}-0.135 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\-0.208 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.171 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\-0.266 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.233 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \vdots & \vdots\\(-0.057-0.048j) & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\(-0.098-0.087j) & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\(0.135+0.086j) & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\(0.035+0.025j) & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\(-0.060+0.028j) & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

```python
rho = qt.rand_dm_ginibre(3)
(A * rho * A.dag()).ptrace((0,)) - (V * qt.tensor(rho, qt.ket2dm(qt.basis(A.dims[0][-1], 0))) * V.dag()).ptrace((0,))
```

Quantum object: dims = [[3], [3]], shape = (3, 3), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

It thus remains to extend $V$ to act non-trivially on the full space. This is done by noting that the singular values of $V$ are each one or zero, convienently partitioning $\Hil_X \otimes \Hil_Z$ into the null space of $V$ and its complement.

```python
Vu, Vs, Vv = np.linalg.svd(V.data.todense())
U = qt.Qobj(Vu, dims=V.dims) * qt.Qobj(Vv, dims=V.dims)
U * U.dag()
```

Quantum object: dims = [[3, 9], [3, 9]], shape = (27, 27), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}1.000 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 1.000 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 1.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 1.000 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \vdots & \vdots\\0.0 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 1.000 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 1.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 1.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 1.000 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0 & \cdots & 0.0 & 0.0 & 0.0 & 0.0 & 1.000\\\end{array}\right)\end{equation*}

We finish by verifying that preparing an environment state, evolving under the system/environment unitary $U$, then partial tracing over the environment produces the same map as the original channel.

```python
(A * rho * A.dag()).ptrace((0,)) - (U * qt.tensor(rho, qt.ket2dm(qt.basis(A.dims[0][-1], 0))) * U.dag()).ptrace((0,))
```

Quantum object: dims = [[3], [3]], shape = (3, 3), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

## Epilouge

```python
version_table()
```

Software	Version

QuTiP	4.3.0.dev0+6e5b1d43

Numpy	1.13.1

SciPy	0.19.1

matplotlib	2.0.2

Cython	0.25.2

Number of CPUs	2

BLAS Info	INTEL MKL

IPython	6.1.0

Python	3.6.2 |Anaconda custom (x86_64)| (default, Jul 20 2017, 13:14:59)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]

OS	posix [darwin]

Thu Jul 20 22:58:26 2017 MDT


# Variational Autoencoder

Cette classe d'algorithmes fait appel à des notions de probabilités et d'inférence bayésienne avancées. Un effort a été fait pour simplifier ces notions et favoriser l'aspect pratique. Je conseille néanmoins la lecture de ce [cours](https://www.ceremade.dauphine.fr/~xian/mr081.pdf) très complet pour se familiariser avec le vocabulaire bayésien et pour les anglophones le chapitre "Probability and Information Theory" du [Deep Learning Book](http://www.deeplearningbook.org/contents/prob.html) (qui est trouvable en ligne gratuitement et est en quelque sorte la Bible du Deep Learning).

Pour toute question, me contacter ou ouvrir une issue à laquelle je répondrai dans les plus brefs délais.

## Chargement des données MNIST

```python
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("./mnist/", one_hot=True)

# Load data
x_train = mnist.train.images
y_train = mnist.train.labels
x_test = mnist.test.images
y_test = mnist.test.labels

print("x_train: ", x_train.shape)
print("y_train: ", y_train.shape)
print("x_test: ", x_test.shape)
print("y_test: ", y_test.shape)
```

    WARNING: Logging before flag parsing goes to stderr.
    W1108 17:07:27.812216 4537341376 deprecation.py:323] From <ipython-input-1-9c48f15a73a7>:2: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
    W1108 17:07:27.813454 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please write your own downloading logic.
    W1108 17:07:27.814388 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please use tf.data to implement this functionality.

    Extracting ./mnist/train-images-idx3-ubyte.gz

    W1108 17:07:28.026752 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please use tf.data to implement this functionality.
    W1108 17:07:28.028398 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please use tf.one_hot on tensors.
    W1108 17:07:28.069115 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
    Instructions for updating:
    Please use alternatives such as official/mnist/dataset.py from tensorflow/models.

    Extracting ./mnist/train-labels-idx1-ubyte.gz
    Extracting ./mnist/t10k-images-idx3-ubyte.gz
    Extracting ./mnist/t10k-labels-idx1-ubyte.gz
    x_train:  (55000, 784)
    y_train:  (55000, 10)
    x_test:  (10000, 784)
    y_test:  (10000, 10)

* Visualisation des données

```python
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline

def plot_mnist(data, classes):

    for i in range(10):
        idxs = (classes == i)

        # get 10 images for class i
        images = data[idxs][0:10]

        for j in range(5):
            plt.subplot(5, 10, i + j*10 + 1)
            plt.imshow(images[j].reshape(28, 28), cmap='gray')
            # print a title only once for each class
            if j == 0:
                plt.title(i)
            plt.axis('off')
    plt.show()

classes = np.argmax(y_train, 1)
plot_mnist(x_train, classes)
```

## Application de l'algorithme

On utilise Tensorflow pour construire et entraîner notre variational autoencoder.
* D'abord, on construit le graphe de notre réseau de neurones. Celui-ci aura l'architecture suivante :

Entrée de 784 cellules (nombre de pixels d'une image)

Encoder : Trois couches cachées composées respectivement 512, 256 et 128 neurones

Vecteur latent : Par contraste avec un autoencoder classique, le vecteur latent est ici composé de deux vecteurs de deux neurones. En effet, nous avons ici un double objectif : encoder correctement les données dans une dimension plus faible et générer de nouveaux exemples à partir de valeurs aléatoires (ou bruit). Le choix ici est d'utiliser une distribution Gaussienne pour générer des images réalistes par rapport aux données d'entrées. La dimension de l'espace latent sera donc composée d'un vecteur d'écart type et d'un vecteur de moyennes.

Decoder : Deux couches cachées composées respectivement 512 et 256 neurones

Sortie de 784 cellules (nombre de pixels d'une image)

__Fonctionnement :__

Rappel sur les structures neuronales : Le réseau est composé de plusieurs couches de neurones. Les neurones de chaque couche sont connectés à chaque neurones de la couche suivante (resp. précédente) par des poids (notés w). Le rôle d'un neurone est d'appliquer une fonction non-linéaire (ici l'opérateur tanh et sigmoid pour la dernière couche du décodeur) à son entrée, qui est une somme des sorties des neurones de la couche précédente pondérée par chaque poids w à laquelle on ajoute une constante propre à chaque neurone appellée biais.

Bien que portant le nom d'autoencoder, les VAE difèrent nettement des autoencoders classiques (e-g. les Denoising autoencoders) car ils projettent maintenant les données dans un espace latent __continu__ : le vecteur latent ne modélise plus seulement les données d'entrées mais aussi les variations possibles entre celles-ci (dans le cas présent les angles de vues des images, les formes intrinsèques de classes modélisées, les contrastes) en préservant les interractions statistiques entre les variables.

Plus concrètement, voici à quoi ressemble l'espace latent appris par un autoencoder classique sur les données de MNIST :

Les différents clusters (nombres de 0 à 9) sont bien distinct, tout vecteur latent ne tombant pas dans un des cluster générera une sortie irréaliste. A contrario, l'espace latent continu d'un VAE capturera / démêlera les facteurs de variations entre les clusters :

(Source : [Medium](https://towardsdatascience.com/intuitively-understanding-variational-autoencoders-1bfe67eb5daf))

En termes probabilistes, nous cherchons à modéliser la distribution de probabilité des données d'entrées par rapport à des variables cachées z :

$$P(X) = \int P(X \vert z) P(z) dz$$

Malheureusement, ce calcul n'est pas réalisable (nous ne connaissons aucune des intégrandes car il serait trop coûteux de sommer toutes les valeurs possibles de z). C'est pourquoi $P(z)$ sera inférée à l'aide de la distribution postérieure $P(z \vert X)$, la probabilité que le vecteur latent $z$ ait généré les données d'entrée $X$, par une méthode bayésienne appelée [inférence variationnelle](https://arxiv.org/abs/1601.00670).

Le principe est le suivant :

- Nous approximons $P(z \vert X)$ à l'aide d'une distribution plus simple (ici Gaussienne) $Q_\phi(\mathbf{z}\vert\mathbf{X})$ telle que : $% <![CDATA[
\begin{aligned}
\mathbf{z} &\sim Q_\phi(\mathbf{z}\vert\mathbf{X}^{(i)}) = \mathcal{N}(\mathbf{z}; \boldsymbol{\mu}^{(i)}, \boldsymbol{\sigma}^{2(i)}\boldsymbol{I})
\end{aligned} %]]>$

- Pour rendre $Q_\phi(\mathbf{z}\vert\mathbf{X})$ la plus proche possible de $P_\theta(\mathbf{z}\vert\mathbf{X})$, nous minimison la [KL-Divergence](https://fr.wikipedia.org/wiki/Divergence_de_Kullback-Leibler) entre ces deux distributions définie par :

$$% <![CDATA[
\begin{aligned}
& D_\text{KL}( q_\phi(\mathbf{z}\vert\mathbf{X}) \| p_\theta(\mathbf{z}\vert\mathbf{X}) ) & \\
&=\int q_\phi(\mathbf{z} \vert \mathbf{X}) \log\frac{q_\phi(\mathbf{z} \vert \mathbf{X})}{p_\theta(\mathbf{z} \vert \mathbf{X})} d\mathbf{z} &
\end{aligned} %]]>$$

En appliquant le théorême de Bayes et en réarrangeant cette équation, nous pouvons réécrire :

$$\log P_\theta(\mathbf{X}) - D_\text{KL}( Q_\phi(\mathbf{z}\vert\mathbf{X}) \| P_\theta(\mathbf{z}\vert\mathbf{X}) ) = \mathbb{E}_{\mathbf{z}\sim Q_\phi(\mathbf{z}\vert\mathbf{X})}\log p_\theta(\mathbf{X}\vert\mathbf{z}) - D_\text{KL}(Q_\phi(\mathbf{z}\vert\mathbf{X}) \| p_\theta(\mathbf{z}))$$

Le premier terme à gauche est appelé log-vraisemblance, maximiser ce terme par rapport à ses paramètres (ici vecteurs de moyennes et écarts-types) revient à maximiser la probabilité de générer des données proches des données d'entrée.
Le deuxième terme à gauche peut être considéré comme un terme d'erreur assurant que la distribution plus simple $Q$ produise des vecteurs latents générant des $X$ proche des données d'entrée.

Malheureusement, nous ne pouvons pas maximiser l'expression de gauche car nous ne connaissons ni la vraisemblance ni la distribution postérieure $D_\text{KL}( P_\phi(\mathbf{z}\vert\mathbf{X}))$. C'est pourquoi nous utiliserons l'expression de droite comme fonction perte à minimiser par notre réseau de neurones :

$$% <![CDATA[
\begin{aligned}
L_\text{VAE}(\theta, \phi)
&= - \mathbb{E}_{\mathbf{z} \sim Q_\phi(\mathbf{z}\vert\mathbf{X})} \log p_\theta(\mathbf{X}\vert\mathbf{z}) + D_\text{KL}( Q_\phi(\mathbf{z}\vert\mathbf{X}) \| P_\theta(\mathbf{z}))
\end{aligned} %]]>$$

Nous verrons dans la partie "Fonction de perte à minimiser" comment exprimer cette fonction sous hypothèse gaussienne et la minimiser à l'aide d'une descente de gradient stochastique.

### Paramètres du modèle

```python
# Nombre de cellules en entrée (nombre de pixels d'une image):
number_inputs = 28 * 28

# Nombre de neurones sur la première couche de l'encodeur et dernière couche du décodeur
number_neurons_hidden_1 = 512

# Nombre de neurones sur la deuxième couche de l'encodeur et deuxième couche du décodeur
number_neurons_hidden_2 = 256

# Nombre de neurones sur la troisième couche de l'encodeur et première couche du décodeur
number_neurons_hidden_3 = 128

# Dimension latente
latent_dim = 2

# Taux d'apprentissage de la descente de gradient
learning_rate = 0.00007

# Nombre d'itérations à l'entraînement
number_training_epochs = 30

# Taille du batch : Nombre d'images à coonsidérer pour chaque itération
train_batch_size = 128

import tensorflow as tf

def glorot_init(shape):
    return tf.random_normal(shape=shape, stddev=1. / tf.sqrt(shape[0] / 2.))

# Création des poids et biais à l'aide de variables Tensorflow
weights = {
'encoder_weights_1': tf.Variable(glorot_init([number_inputs, number_neurons_hidden_1])),
'encoder_weights_2': tf.Variable(glorot_init([number_neurons_hidden_1, number_neurons_hidden_2])),
'encoder_weights_3': tf.Variable(glorot_init([number_neurons_hidden_2, number_neurons_hidden_3])),
'z_mean': tf.Variable(glorot_init([number_neurons_hidden_3, latent_dim])),
'z_std': tf.Variable(glorot_init([number_neurons_hidden_3, latent_dim])),
'decoder_weights_1': tf.Variable(glorot_init([latent_dim, number_neurons_hidden_1])),
'decoder_weights_2': tf.Variable(glorot_init([number_neurons_hidden_1, number_neurons_hidden_2])),
'decoder_weights_3': tf.Variable(glorot_init([number_neurons_hidden_2, number_neurons_hidden_3])),
'decoder_weights_out': tf.Variable(glorot_init([number_neurons_hidden_3, number_inputs]))
}

biases = {
'encoder_biases_1': tf.Variable(glorot_init([number_neurons_hidden_1])),
'encoder_biases_2': tf.Variable(glorot_init([number_neurons_hidden_2])),
'encoder_biases_3': tf.Variable(glorot_init([number_neurons_hidden_3])),
'z_mean': tf.Variable(glorot_init([latent_dim])),
'z_std': tf.Variable(glorot_init([latent_dim])),
'decoder_biases_1': tf.Variable(glorot_init([number_neurons_hidden_1])),
'decoder_biases_2': tf.Variable(glorot_init([number_neurons_hidden_2])),
'decoder_biases_3': tf.Variable(glorot_init([number_neurons_hidden_3])),
'decoder_biases_out': tf.Variable(glorot_init([number_inputs]))
}
```

### Création de l'autoencoder par couches

* Encoder

```python
def create_encoder(encoder_inputs):
    encoder_layer_1 = tf.nn.tanh(tf.add(tf.matmul(encoder_inputs, weights['encoder_weights_1']),
                                        biases['encoder_biases_1']))

    encoder_layer_2 = tf.nn.tanh(tf.add(tf.matmul(encoder_layer_1, weights['encoder_weights_2']),
                                        biases['encoder_biases_2']))

    encoder_layer_3 = tf.nn.tanh(tf.add(tf.matmul(encoder_layer_2, weights['encoder_weights_3']),
                                        biases['encoder_biases_3']))

    mu = tf.matmul(encoder_layer_3, weights['z_mean']) + biases['z_mean']
    log_sigma = tf.matmul(encoder_layer_3, weights['z_std']) + biases['z_std']

    return mu, log_sigma
```

* Decoder

```python
def create_decoder(decoder_inputs):
    decoder_layer_1 = tf.nn.tanh(tf.add(tf.matmul(decoder_inputs, weights['decoder_weights_1']),
                                   biases['decoder_biases_1']))

    decoder_layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(decoder_layer_1, weights['decoder_weights_2']),
                                   biases['decoder_biases_2']))

    decoder_layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(decoder_layer_2, weights['decoder_weights_3']),
                                   biases['decoder_biases_3']))

    decoder_layer_out = tf.nn.sigmoid(tf.add(tf.matmul(decoder_layer_3, weights['decoder_weights_out']),
                                   biases['decoder_biases_out']))

    return decoder_layer_out
```

* Autoencoder complet

```python
vae_inputs = tf.placeholder("float", [None, number_inputs])

mu, log_sigma = create_encoder(vae_inputs)

eps = tf.random_normal(tf.shape(log_sigma), dtype=tf.float32, mean=0., stddev=1.0,
                       name='epsilon')

z = mu + tf.exp(log_sigma / 2) * eps

decoder_model = create_decoder(z)

y_prediction = decoder_model

y_true = vae_inputs
```

### Fonction de perte à minimiser

La fonction de perte utilisée pour les variational autoencoder est appellée ELBO (Evidence Lower Bound) car elle représente la borne inférieure de l'évidence $p(x)$. Elle se distingue de celle des autoencoders classiques dans la mesure où elle tient compte de deux termes d'erreur :

* L'erreur de reconstruction : elle mesure l'écart entre l'image d'entrée et celle générée en sortie de décodeur
* La KL-Divergence entre entre la distribution postérieure variationnelle la probabilité marginale des vecteurs latents

Nous avons vu que cette fonction s'exprimait :

$$% <![CDATA[
\begin{aligned}
L_\text{VAE}(\theta)
&= - \mathbb{E}_{\mathbf{z} \sim Q_\phi(\mathbf{z}\vert\mathbf{X})} \log p_\theta(\mathbf{X}\vert\mathbf{z}) + D_\text{KL}( Q_\phi(\mathbf{z}\vert\mathbf{X}) \| P_\theta(\mathbf{z}))
\end{aligned} %]]>$$

* L'erreur de reconstruction (premier terme) se minimise à l'aide de l'entropie croisée binaire, plus joliement appelée [Binary crossentropy](https://fr.wikipedia.org/wiki/Entropie_crois%C3%A9e):

$$% <![CDATA[
\begin{aligned}
L_\text{reconstruction}(\theta)
&= - \mathbb{E}_{\mathbf{z} \sim Q_\phi(\mathbf{z}\vert\mathbf{X})} \log p_\theta(\mathbf{X}\vert\mathbf{z}) & \\
&= - \sum_{i=1}^n {X^{(i)}\log(X_\text{decoder}^{(i)}) + (1 - X^{(i)})\log(1 - X_\text{decoder}^{(i)})}
\end{aligned} %]]>$$

Où $X^{(i)}$ est la i-ème image réelle (son vecteur de pixels) et $X_\text{decoder}^{(i)}$ sa reconstruction en sortie de décoder. Plusieurs types de fonctions de pertes existent et peuvent être performantes selon le besoin.

* La KL-Divergence, de son côté est plus difficile à obtenir car elle nécessite le calcul d'intégrales sur des fonctions gaussiennes. Or, une solution de forme fermée existe est décrite dans le papier original de [Kingma et al.](https://arxiv.org/pdf/1312.6114.pdf). Elle s'exprime ainsi :

$${-\mathcal{D}}_{KL}(Q_\phi(z|X) || P_\theta(z)) = -\frac{1} 2 \sum{(1 + log(\sigma^2) - \mu^2 - \sigma^2)}$$

(Démonstration dans la partie "Annexe")

Pour simplifier l'entraînement, nous considérons que l'encoder ne calcule pas $\sigma$ mais $log(\sigma)$, c'est pourquoi nous appliquons l'opérateur exponentiel pour obtenir la valeur de $\sigma$.

* Nous désignons ensuite la fonction de perte comme la moyenne de ces deux fonctions :

```python
def reconstruction_loss(x_reconstructed, x_true):
    # Perte de reconstruction, le terme 1e-10 dans la fonction tf.log permet d'éviter de calculer le log de 0.
    encode_decode_loss = x_true * tf.log(1e-10 + x_reconstructed) \
                         + (1 - x_true) * tf.log(1e-10 + 1 - x_reconstructed)
    encode_decode_loss = -tf.reduce_sum(encode_decode_loss, 1)

    return encode_decode_loss

def kullback_leibler_divergence(mu, log_sigma):    # KL Divergence loss

    kl_div_loss = 1 + log_sigma - tf.square(mu) - tf.exp(log_sigma)
    kl_div_loss = -0.5 * tf.reduce_sum(kl_div_loss, 1)
    return -0.5 * tf.reduce_sum(1 + 2 * log_sigma - mu**2 -
                                        tf.exp(2 * log_sigma), 1)

vae_loss = tf.reduce_mean(reconstruction_loss(y_prediction, y_true) + kullback_leibler_divergence(mu, log_sigma))

#loss_function = vae_loss(decoder_model, vae_inputs)
```

* Puis, nous utiliserons l'algorithme [RMSProp](https://towardsdatascience.com/understanding-rmsprop-faster-neural-network-learning-62e116fcf29a) pour minimiser cette fonction.

```python
optimizer = tf.train.RMSPropOptimizer(learning_rate=learning_rate)
train = optimizer.minimize(vae_loss)
```

    W1108 17:07:29.751254 4537341376 deprecation.py:323] From /anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/math_grad.py:1205: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
    Instructions for updating:
    Use tf.where in 2.0, which has the same broadcast rule as np.where
    W1108 17:07:29.883757 4537341376 deprecation.py:506] From /anaconda3/lib/python3.7/site-packages/tensorflow/python/training/rmsprop.py:119: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
    Instructions for updating:
    Call initializer instance with the dtype argument instead of passing it to the constructor

* Sauvegarde du modèle pour réutilisation future

```python
import os

model_dir = 'Models_directory'
autoencoder_file = 'VAE.ckpt'
autoencoder_model = os.path.join(model_dir, autoencoder_file)
saver = tf.train.Saver()
```

### Entraînement

On lance maintenant la boucle d'entraînement afin d'initialiser les valeurs de chaque variable tensorflow (ou tenseurs) définies précédemment et de lancer l'entraînement.

L'ensemble des tenseurs et opérations définis plus haut forme un Graph, l'étape d'initialisation consiste tout simplement à assigner des valeurs réelles (les données ainsi que les poids / biais aléatoirement initialisés) aux valeurs symboliques (tenseurs) et lancer les opérations sur celles-ci.

Ici, les opérations sont :

* Création des mini-batch de données par échantillonnage pour chaque itération
* Pour chaque epoch : tentative de reconstruction de l'image par encodage et décodagee, calcul de l'erreur de reconstruction / KL-Divergence et de son [gradient](https://fr.wikipedia.org/wiki/R%C3%A9tropropagation_du_gradient) par rapport aux poids et biais (forward pass) puis mise à jour des poids et biais en soustrayant une partie de ce gradient à chaque poids / biais (backward pass).

```python
var_initializer = tf.global_variables_initializer()

with tf.Session() as session:
    session.run(var_initializer)
    number_batches = int(x_train.shape[0]/train_batch_size)
    # Training cycle
    for epoch in range(number_training_epochs):
        # Iterations over all transaction batches
        for i in range(number_batches):
            # Random selection of transactions in train subset to build the mini-batch
            index_batch = np.random.choice(x_train.shape[0], train_batch_size)
            batch_transactions = x_train[index_batch]
            # Run of the model, which performs:
            #       - loss calculation
            #       - optimization of weights and biases values, by using gradient descent
            optimization, loss = session.run(
                [train, vae_loss], feed_dict={vae_inputs: batch_transactions})

        print("Epoch:", '%02d' % (epoch + 1), " -  loss = ", "{:.4f}".format(loss))

    model_path = saver.save(session, autoencoder_model)
    print("Modèle sauvegardé dans le fichier : {}".format(model_path))

    print("Fin de l'entraînement, bon autoencodage ! \n")
```

    Epoch: 01  -  loss =  232.9648
    Epoch: 02  -  loss =  202.4954
    Epoch: 03  -  loss =  196.2549
    Epoch: 04  -  loss =  193.1108
    Epoch: 05  -  loss =  184.7171
    Epoch: 06  -  loss =  187.2911
    Epoch: 07  -  loss =  184.0617
    Epoch: 08  -  loss =  176.4549
    Epoch: 09  -  loss =  181.6848
    Epoch: 10  -  loss =  179.1732
    Epoch: 11  -  loss =  174.1978
    Epoch: 12  -  loss =  173.6547
    Epoch: 13  -  loss =  172.5066
    Epoch: 14  -  loss =  168.4610
    Epoch: 15  -  loss =  171.9714
    Epoch: 16  -  loss =  178.2183
    Epoch: 17  -  loss =  174.7566
    Epoch: 18  -  loss =  176.5271
    Epoch: 19  -  loss =  164.7077
    Epoch: 20  -  loss =  163.3408
    Epoch: 21  -  loss =  164.3710
    Epoch: 22  -  loss =  168.9598
    Epoch: 23  -  loss =  167.7850
    Epoch: 24  -  loss =  162.5034
    Epoch: 25  -  loss =  168.4838
    Epoch: 26  -  loss =  168.8366
    Epoch: 27  -  loss =  165.8277
    Epoch: 28  -  loss =  161.1838
    Epoch: 29  -  loss =  152.7943
    Epoch: 30  -  loss =  158.8061
    Modèle sauvegardé dans le fichier : Models_directory/VAE.ckpt
    Fin de l'entraînement, bon autoencodage !

A tester : Visualisation de l'espace latent (sortie de la dernière couche de l'encoder) et des images encodées. Ré-entrainement de l'autoencoder avec un espace latent de deux ou trois dimensions et visualisation par label afin de comparer les résultat avec une ACP.

```python
# Testing
# Autoencoder model obtained during training phase is retrieved
autoencoder_model = os.path.join(model_dir, autoencoder_file)
saver = tf.train.Saver()

with tf.Session() as sess:
    saver.restore(sess, autoencoder_model)

    noise_input = tf.placeholder(tf.float32, shape=[None, latent_dim])

    decoder = create_decoder(noise_input)
    # Building a manifold of generated digits
    n = 20
    x_axis = np.linspace(-3, 3, n)
    y_axis = np.linspace(-3, 3, n)

    canvas = np.empty((28 * n, 28 * n))
    for i, yi in enumerate(x_axis):
        for j, xi in enumerate(y_axis):
            z_mu = np.array([[xi, yi]] * train_batch_size)
            x_mean = sess.run(decoder, feed_dict={noise_input: z_mu})
            canvas[(n - i - 1) * 28:(n - i) * 28, j * 28:(j + 1) * 28] = \
            x_mean[0].reshape(28, 28)

    plt.figure(figsize=(8, 10))
    Xi, Yi = np.meshgrid(x_axis, y_axis)
    plt.imshow(canvas, origin="upper", cmap="gray")
    plt.show()
```

A tester : Comparer les résultat avec une ACP. Entraîner l'algorithme sur une autre source de données

## Annexe

Nous modélisons ces distributions à l'aide d'une loi Normale, soit :
$$q(z|x)=\mathcal{N}(z;\mu,\Sigma)$$
$$p(z)=\mathcal{N}(0,I)$$

Où :

- $I$ désigne la matrice identité (matrice nulle avec des 1 sur la diagonale)

- $\mu$ désigne le vecteur de moyenne en sortie d'encoder

- $\Sigma$ désigne le vecteur d'écart-type en sortie d'encoder

Nous pouvons calculer la KL-Divergence ainsi :

$$\begin{align}
D_\text{KL}( Q_\phi(\mathbf{z}\vert\mathbf{X}) \| P_\theta(\mathbf{z}))
&=
\int \left[\log( p(x)) - log( q(x)) \right] p(x) dx\\
&=
\frac{1}{2}\left[\log\frac{|\Sigma_2|}{|\Sigma_1|} - n + \text{tr} \{ \Sigma_2^{-1}\Sigma_1 \} + (\mu_2 - \mu_1)^T \Sigma_2^{-1}(\mu_2 - \mu_1)\right]\\
&= \frac{1}{2}\left[\log\frac{|I|}{|\Sigma|} - n + \text{tr} \{ I^{-1}\Sigma \} + (\vec{0} - \mu)^T I^{-1}(\vec{0} - \mu)\right]\\
&= \frac{1}{2}\left[-\log{|\Sigma|} - n + \text{tr} \{ \Sigma \} + \mu^T \mu\right]\\
&= \frac{1}{2}\left[-\log\prod_i\sigma_i^2 - n + \sum_i\sigma_i^2 + \sum_i\mu^2_i\right]\\
&= \frac{1}{2}\left[-\sum_i\log\sigma_i^2 - n + \sum_i\sigma_i^2 + \sum_i\mu^2_i\right]\\
&= \frac{1}{2}\left[-\sum_i\left(\log\sigma_i^2 + 1\right) + \sigma_i^2 + \mu^2_i\right]\\
\end{align}$$

```python

```


# Домашнее задание по рекомендательным системам

В данном домашнем задании вам предлагается реализовать User-based рекомендательную систему. Так же требуется реализовать несколько вспомогательных функций, шаблоны которых вы можете найти в `utils.py`.

Требования к выполнению задания:
- Реализация функции из `utils.py` засчитывается, только если пройдены все соответствующие тесты из `test.py`. Запуск тестов: pytest test.py. Для тестов вам потребуются библиотеки `numpy`, `scipy`, `pytest` и `hypothesis`.
- Плагиат запрещен. Если будет замечено, что часть задания списана, то 0 баллов ставится как списывающему, так и давшему списать.
- Если пользуетесь кодом из открытых источников, то указывайте ссылки, откуда взяли решение. Иначе такой код может быть воспринят как плагиат.
- При выполнении задания нельзя использовать библиотеку `scipy` и функцию `numpy.linalg.norm`

При запуске тестов могут появиться предупреждения: PearsonRConstantInputWarning и PearsonRNearConstantInputWarning. На них можно не обращать внимания.

Возможный максимум баллов за задание: 10 баллов

Дедлайн: ???

Штраф: ??? - будет ли в курсе штраф?

Для ускорения проверки, напишите здесь получившееся количество баллов: ...

## 1. Метрика сходства
1.1. Реализация метрик (2 балла)

Первое, с чем необходимо разобраться, при реализации User-based подхода, это с метрикой, с помощью которой будет решаться, насколько похожи пользователи. Вам предлагается реализовать 2 метрики: на основе евклидовой метрики и коэффициент корреляции Пирсона. Шаблоны для обоих функций можете найти в `utils.py`. Не забудьте проверить реализацию на тестах.

Евклидова метрика:
\begin{equation}
d(p,q)=\sqrt{(p_1-q_1)^2+(p_2-q_2)^2+\dots+(p_n-q_n)^2} = \sqrt{\sum_{k=1}^n (p_k-q_k)^2}
\end{equation}

В этом случае $d(p, q) \in [0, \infty)$, при этом если $d(p, q) \to 0$, то $sim(p, q) \to 1$. С учетом этого конечная формула будет выглядеть следующим образом:
\begin{equation}
sim(p, q) = \frac{1}{1 + d(p, q)}
\end{equation}
Так же в этой формуле не будет проблем с делением на 0.

Коэффициент корреляции Пирсона:
\begin{equation}
r_{xy} = \frac {\sum_{i=1}^{m} \left( x_i-\bar{x} \right)\left( y_i-\bar{y} \right)}{\sqrt{\sum_{i=1}^{m} \left( x_i-\bar{x} \right)^2 \sum_{i=1}^{m} \left( y_i-\bar{y} \right)^2}}
\end{equation}

1.2. (1 балл)

Рассмотрим пользователей $u$ и $v$. Им соотвествуют векторы $x_u$ и $x_v$, где $x_u[i] = r_{ui}$ и $x_v[i] = r_{vi}$. Из лекции известно, что похожесть между векторами $x_u$ и $x_v$ вычисляются только для тех индексов i, для которых существует и $r_{ui}$, и $r_{vi}$. То есть верно следуюющее:
\begin{equation}
sim(u, v) = sim(x_uI_{uv}, x_vI_{uv}),
\end{equation}
где $I_{uv} = [i | \exists r_{ui} \& \exists r_{vi}]$. При этом если $I_{uv} = \emptyset$, то $sim(u, v) \to -\infty$.

Реализуйте два новых метода, которые переиспользуют написанные вами `euclidean_distance` и `pearson_distance`, добавляющие условия на $x_u$ и $x_v$. Считается, что $x_u[i] = 0$, если $\nexists r_{ui}$. То же верно для $x_v$.

При реализации заданий можно как написать новые функции, так и использовать декораторы.

```python
import warnings
warnings.filterwarnings("ignore")

import numpy as np
from utils import euclidean_similarity, pearson_similarity
```

## 2. User-based method
2.1. (3 балла)

Реализовать User-based подход, реализовав методы класса `UserBasedRecommendation`, основанного на использовании `NearestNeighbors`. В качестве метрики может для нахождения похожих пользователей может быть использована как евклидова метрика, так и коэффициент корреляции Пирсона.

Не забывайте, что `NearestNeighbors` ищет минимум расстояния между элементами, поэтому логично в качестве метрики при инициализации `NearestNeighbors` использовать обратную метрике схожести. То есть такую, что когда $sim(u, v) \to 1$, то $d(u, v) \to 0$. Например: $d(u, v) = 1 - sim(u, v)$

```python
from sklearn.neighbors import NearestNeighbors
from typing import Optional

from collections import Counter

class UserBasedRecommendation:
    def __init__(self, metric: str = 'euclidean', n_recommendations: int = 5, alpha: float = 0.8):
        """
        Args:
            metric: name of metric: ['euclidean', 'pearson']
            n_recommendations: number of recommendations. Also can be specified in self.make_recommendation
            alpha: similarity threshold: if sim(u, v) > alpha then u and v are similar
        """
        self.metric = metric
        self.n_recommendations = n_recommendations
        self.alpha = alpha

    def fit(self, X: np.array):
        """
        Args:
            X: matrix N x M where X[u, i] = r_{ui} if r_{ui} exists else X[u, i] = 0
        """
        self.X = X
        if self.metric == 'euclidean':
            self.used_metric = lambda x, y : 1 - euclidean_similarity(x, y)
        if self.metric == 'pearson':
            self.used_metric = lambda x, y : (1 - pearson_similarity(x, y)) / 2
        self.model = NearestNeighbors(metric=self.used_metric, radius=1-self.alpha)
        self.model.fit(self.X)

        return self

    def __find_closest_users(self, user_id: int, n_closest_users: int = None):

        if n_closest_users is None:
            n_closest_users = self.X.shape[0]

        distances, similar_users = self.model.radius_neighbors([self.X[user_id]], return_distance=True)
        similar_users = similar_users[0][distances[0].argsort()]
        return similar_users[similar_users != user_id][:n_closest_users]

    def make_recommendation(self, user_id: int, n_closest_users: int = None, n_recommendations: Optional[int] = None):
        """
        Args:
            user_id: user id to whom you want to recommend
            n_recommendations: number of recommendations
        """
        if n_recommendations is None:
            n_recommendations = self.n_recommendations
        ind = self.__find_closest_users(user_id=user_id, n_closest_users=n_closest_users)
        X_users = self.X[ind]
        popularity = (X_users > 0).mean(axis=0).argsort()

        return popularity[::-1][:n_recommendations]
```

2.2. (1 балла)

Приведите пример, для которого использование разных метрик будет давать разные рекомендации. Объясните свой пример.

```python
X = np.array([[0, 0.1, 0.3],
             [0, 2, 1.5],
             [0, 0.6, 0.5],
             [0, 0.5, 0],
             [0, 1.6, 1.05]])
```

```python
model = UserBasedRecommendation(alpha=0.1).fit(X)
model.make_recommendation(user_id=2, n_recommendations=1, n_closest_users=2)
```

    array([1], dtype=int64)

```python
model = UserBasedRecommendation(alpha=0, metric='pearson').fit(X)
model.make_recommendation(user_id=2, n_recommendations=1, n_closest_users=2)
```

    array([2], dtype=int64)

Объяснение: Для ясности пронумеруем товары с 0. Видно, что третьему пользователю так же как и второму больше понравился товар 1 и чуть меньше понравился товар 2, тогда как первому пользователю оба эти товары не сильно понравились, причем товар 1 ему понравился меньше, чем товар 2, то есть наблюдается обратная завизимость. Но все эти факторы никак не учитываются при использовании евклидовой метрики, поэтому в первом случае алгоритм посчитал, что первый и третий пользователь похожи только по тому, что они ставят оценки примерно в одном и том отрезке: {0, 1}, что является довольно странным предположением. А алгоритм, использующий коэффициент корреляции Пирсона учитывает эти факторы, поэтому находит третьему пользователю соседа со схожими интересами в виде второго пользователя. Отсюда получаем разные рекомендации.

## 3. Оценка качества
3.1. (1 балл)

Реализуйте Average Precision at k и Mean Average Precision at k. Шаблоны можете найти в `utils.py`.
\begin{align*}
AP@K = \frac{1}{m}\sum_{k=1}^K P(k)*rel(k), \\
MAP@K = \frac{1}{|U|}\sum_{u=1}^{|U|}(AP@K)_u
\end{align*}
где $P(k)$ - Precision at k, $rel(k) = 1$, если рекомендация релевантна, иначе $rel(k) = 0$.

## 4. Применение модели
4.1. (2 балла)

Выгрузите датасет `ratings_small.csv`: https://www.kaggle.com/rounakbanik/the-movies-dataset#ratings_small.csv

```python
import pandas as pd
from sklearn.model_selection import train_test_split
```

```python
data = pd.read_csv('ratings_small.csv', index_col=False)
data.shape
```

    (100004, 4)

```python
data.userId.min(), data.userId.max(), len(data.userId.unique())
```

    (1, 671, 671)

```python
data.movieId.min(), data.movieId.max(), len(data.movieId.unique())
```

    (1, 163949, 9066)

Для простоты работы с данными, измените нумерацию пользователей и фильмов так, чтобы нумерация начиналась с 0 и шла непрерывно.

```python
data = data.sort_values(by=['userId', 'movieId'])
data.userId = data.userId - 1
maple = {k:i for i, k in enumerate(sorted(data.movieId.unique()))}
data.movieId = data.movieId.map(maple)
data.head()
```

      userId
      movieId
      rating
      timestamp

      0
      0
      30
      2.5
      1260759144

      1
      0
      833
      3.0
      1260759179

      2
      0
      859
      3.0
      1260759182

      3
      0
      906
      2.0
      1260759185

      4
      0
      931
      4.0
      1260759205

```python
data.userId.min(), data.userId.max(), len(data.userId.unique())
```

    (0, 670, 671)

```python
data.movieId.min(), data.movieId.max(), len(data.movieId.unique())
```

    (0, 9065, 9066)

Удалим для наиболее активных пользователей 5 оценок

```python
active_users = data.userId.value_counts()[:10].index
test_data = pd.DataFrame([], columns=data.columns)
for user_id in active_users:
    _, test = train_test_split(data[data.userId == user_id], test_size=5, random_state=42)
    test_data = test_data.append(test, ignore_index=True)
    data = data[~((data.userId == user_id) & (data.movieId.isin(test.movieId.values)))]
data.shape, test_data.shape
```

    ((99954, 4), (50, 4))

```python
test_data.head()
```

      userId
      movieId
      rating
      timestamp

      0
      546
      3340
      4.0
      1028129718

      1
      546
      772
      4.0
      1039880724

      2
      546
      8781
      4.5
      1468681977

      3
      546
      5835
      4.0
      1242992741

      4
      546
      1369
      3.5
      1342849917

```python
test_actual = np.array([test_data.movieId[test_data.userId == user] for user in test_data.userId.unique()])
```

```python
test_data.userId.unique()
```

    array([546, 563, 623, 14, 72, 451, 467, 379, 310, 29], dtype=object)

```python
active_users
```

    Int64Index([546, 563, 623, 14, 72, 451, 467, 379, 310, 29], dtype='int64')

Преобразуем данные в таблицу `X`, с которой может работать `UserBasedRecommendation`, где $X_{ui} = r_{ui}$, если пользователь $u$ поставил оценку фильму $i$, и $X_{ui} = 0$, если пользователь $u$ не проставил оценку фильму $i$.

Вам может пригодиться `csr_matrix`.

```python
from scipy.sparse import csr_matrix
```

```python
data_matrix = csr_matrix((data.rating, (data.userId, data.movieId)))
data_matrix[0, 30]
```

    2.5

Для пользователей, у которых были удалены фильмы, найдите топ 100 фильмов, который должен посмотреть каждый из этих пользователей, используя `UserBasedRecommendation`. Не забудьте подобрать параметр alpha.

```python
model = UserBasedRecommendation(alpha=0.1).fit(data_matrix.toarray())
predicted = np.empty([0,100])
for user_id in active_users:
    prediction = model.make_recommendation(user_id=user_id, n_recommendations=100)
    predicted = np.append(predicted, [prediction], axis=0)
```

Используя метрику `MAP@5`, `MAP@10` и `MAP@100`, определите, насколько эффективна user-based рекомендательная система для данной задачи.

```python
from utils import mapk
```

```python
for k in (5, 10, 100):
    print(round(mapk(test_actual, predicted, k),3))
```

    0.024
    0.024
    0.026

Как можно улучшить работу модели?

Ответ: Учет рейтинга фильмов у схожих пользователей; ансамбль с другими моделями или той же моделью на других параметрах.


# Multi-Tissue NODDI: Including S0 responses in regular MC models

It is well known that different tissues in the brain have different T1, T2 times and proton densities. This causes the contrast that can be seen in any usual b0 image, as the bone below.

```python
from dmipy.data import saved_data
scheme_hcp, data_hcp = saved_data.wu_minn_hcp_coronal_slice()

import matplotlib.pyplot as plt
import matplotlib.patches as patches
%matplotlib inline

fig, ax = plt.subplots(1)
ax.imshow(data_hcp[:, 0, :, 0].T, origin=True)
ax.set_axis_off()
ax.set_title('HCP coronal slice B0 with ROI');
```

Multi-tissue models, such as Multi-Tissue CSD *(Jeurissen et al. 2014)*, include an estimate of the S0 value of each compartment in the volume fraction estimation. In Dmipy, go further and allow any MC-model (regular MC, MC-SM or MC-SH) to have S0 tissue responses associated with the input models. Generally, this can be described as

\begin{align}
    \textbf{p}^*(\textbf{x})=&\text{argmin}_{\textbf{p}}\int \left[S(\textbf{x},\textbf{A})-\hat{S}^{\text{MT}}(\textbf{A},\textbf{p})\right]^2d\textbf{A},\\
    &\text{with}\,\,\hat{S}^{\text{MT}}(\textbf{A},\textbf{p})=\sum_i^NS_{0,i}f_iC_i(\textbf{A},\textbf{p}_i).
\end{align}

In this example, we demonstrate Dmipy's generalized multi-tissue modeling by implemented *Multi-Tissue NODDI*. Looking at the equation, it is clear that standard multi-compartment modeling is the same as assuming the $S_0$ of each compartment is the same and equal to the $S_0$ of the measured signal. Including the good estimate of $S_{0,i}$ will correct volume fractions in areas where compartments partial volume with different $S_0$ responses.

We will show that correcting for the differences in NODDI between CSD and white matter $S_0$ responses makes signficant different in estimated volume fractions.

# Using Dmipy to set up the Multi-Tissue NODDI-Watson model

### Recovering an estimate of the WM and CSF S0 tissue responses
First we get an estimate of the $S_0$ responses of the different tissue types, so that we can include it in the NODDI model.

```python
from dmipy.tissue_response.three_tissue_response import three_tissue_response_dhollander16
[S0_wm, S0_gm, S0_csf], _, _ = three_tissue_response_dhollander16(
    scheme_hcp, data_hcp, wm_algorithm='tournier07',
    wm_N_candidate_voxels=150, gm_perc=0.2, csf_perc=0.4)

print 'white matter S0 response: {}'.format(S0_wm)
print 'CSF S0 response: {}'.format(S0_csf)
```

    white matter S0 response: 3184.17626953
    CSF S0 response: 10908.4316406

Notice that the $S_0$ response of the CSF is three times higher than that of white matter.

### Setting up Multi-Tissue NODDI

We set up the NODDI-Watson model as usual first.

```python
from dmipy.signal_models import cylinder_models, gaussian_models
from dmipy.distributions.distribute_models import SD1WatsonDistributed
ball = gaussian_models.G1Ball()
stick = cylinder_models.C1Stick()
zeppelin = gaussian_models.G2Zeppelin()
watson_dispersed_bundle = SD1WatsonDistributed(models=[stick, zeppelin])
watson_dispersed_bundle.set_tortuous_parameter('G2Zeppelin_1_lambda_perp','C1Stick_1_lambda_par','partial_volume_0')
watson_dispersed_bundle.set_equal_parameter('G2Zeppelin_1_lambda_par', 'C1Stick_1_lambda_par')
watson_dispersed_bundle.set_fixed_parameter('G2Zeppelin_1_lambda_par', 1.7e-9)
```

But, when instantiating the MC-model we can now also give the `S0_tissue_responses` that we previously estimated.

```python
from dmipy.core.modeling_framework import MultiCompartmentModel
NODDI_mod = MultiCompartmentModel(
    models=[ball, watson_dispersed_bundle],
    S0_tissue_responses=[S0_csf, S0_wm])  # notice one value for each model.
```

The last thing is to fix the diffusivity of the Ball compartment to static values.

```python
NODDI_mod.set_fixed_parameter('G1Ball_1_lambda_iso', 3e-9)
```

We can visualize the model as follows, but nothing on the composition changed, only the $S_0$ values associated with the top-level models.

```python
from IPython.display import Image
NODDI_mod.visualize_model_setup(view=False, cleanup=False)
Image('Model Setup.png')
```

# Fitting Multi-Tissue NODDI to Human Connectome Project data

Fitting is then the same as usual. But notice there is a second optimization happening for the multi-tissue fractions. This is because correcting for the tissue-specific $S_0$ values is independent from the estimation of the non-linear parameters of the original NODDI model.

```python
NODDI_fit_hcp = NODDI_mod.fit(
    scheme_hcp, data_hcp, mask=data_hcp[..., 0]>0)
```

    Using parallel processing with 8 workers.
    Setup brute2fine optimizer in 6.95517396927 seconds
    Fitting of 8181 voxels complete in 1132.75483418 seconds.
    Average of 0.138461659232 seconds per voxel.
    Starting secondary multi-tissue optimization.
    Multi-tissue fitting of 8181 voxels complete in 28.0474832058 seconds.

## Compare volume fractions with and without including S0 responses

Once fitted, the fitted model has access to both the original and the multi-tissue fractions. Here, we can easily compare them and illustrate where the differences at the interfaces are significant.

```python
fractions_csf = NODDI_fit_hcp.fitted_parameters['partial_volume_0']
fractions_wm = NODDI_fit_hcp.fitted_parameters['partial_volume_1']
mt_fractions_csf = NODDI_fit_hcp.fitted_multi_tissue_fractions_normalized['partial_volume_0']
mt_fractions_wm = NODDI_fit_hcp.fitted_multi_tissue_fractions_normalized['partial_volume_1']

fig, axs = plt.subplots(2, 3, figsize=[15, 8])
axs = axs.ravel()
opts = {'origin': True, 'interpolation': 'nearest'}

im = axs[0].imshow(fractions_csf.squeeze().T, vmin=0, vmax=1, **opts)
fig.colorbar(im, ax=axs[0], shrink=0.7)
axs[0].set_title('f_csf no S0')
im = axs[1].imshow(mt_fractions_csf.squeeze().T,  vmin=0, vmax=1, **opts)
fig.colorbar(im, ax=axs[1], shrink=0.7)
axs[1].set_title('f_csf with S0')
im = axs[2].imshow((mt_fractions_csf - fractions_csf).squeeze().T, **opts)
fig.colorbar(im, ax=axs[2], shrink=0.7)
axs[2].set_title('(f_csf with S0) - (f_csf no S0)')

im = axs[3].imshow(fractions_wm.squeeze().T,vmin=0, vmax=1, **opts)
fig.colorbar(im, ax=axs[3], shrink=0.7)
axs[3].set_title('f_wm no S0')
im = axs[4].imshow(mt_fractions_wm.squeeze().T, vmin=0, vmax=1, **opts)
fig.colorbar(im, ax=axs[4], shrink=0.7)
axs[4].set_title('f_wm with S0')
im = axs[5].imshow((mt_fractions_wm - fractions_wm).squeeze().T, **opts)
fig.colorbar(im, ax=axs[5], shrink=0.7)
axs[5].set_title('(f_wm with S0) - (f_wm no S0)')

for ax in axs:
    ax.set_axis_off()
```

## Discussion of results

As you can see, including the estimated $S_0$ responses for CSF and WM for the corresponding models makes a significant difference in the estimated volume fractions. In fact, by including an estimate of the S0 response, we are in earnest estimated a **volume** fraction giving a certain signal contribution, rather than just the **signal** fraction contribution.

As the CSF $S_0$ response is about 3x higher than that of WM, that means that the same volume of CSF tissue produces 3x more signal to the $b_0$ than the same volume of WM. By (approximately) correcting for this bias, we can see that at the interfaces between CSF and WM, the actual volume fraction can be more than 0.3 higher than estimated without the $S_0$ response. In practice, this means that the white matter bundles such as the corpus callosum grow a little more into the CSF volume, making them bigger.

In conclusion, any MC-model that models tissues with different $S_0$ responses should include an estimate of the values -- if possible. Otherwise, the estimated volume fractions will **always** be biased in the direction of the difference of the tissues' $S_0$ responses.

## References
- Zhang, Hui, et al. "NODDI: practical in vivo neurite orientation dispersion and density imaging of the human brain." Neuroimage 61.4 (2012): 1000-1016.
- Jeurissen, Ben, et al. "Multi-tissue constrained spherical deconvolution for improved analysis of multi-shell diffusion MRI data." NeuroImage 103 (2014): 411-426.