# Exercise 4a

Group Members: Luis Pazos Clemens, Robert Freund, Eugen Dizer

Deadline: 12.01.2021, 16:00.

```python
#Load standard libraries
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

import pandas as pd
import os
```

## 2 Red Cards Study

Download the dataset from https://osf.io/gvm2z/ (`1. Crowdsourcing Dataset July 01, 2014 Incl.Ref Country.zip` contains the dataset, `README.txt` a detailed description of the data and `2._Crowdstorming_Pictures_Skin_Color_Ratings.zip` the images of the players). Feel free to look at `Crowdsourcing Analytics - Final Manuscript.pdf` for a more detailed description of the experiment, its features, and the different methods participants applied to tackle the question.

### 2.1 Loading and Cleaning the Data

The first step consists of loading the `.csv` file and preparing the data for the experiment. One participant of the official experiment provided a nice jupyter notebook demonstrating how the python library
`pandas` can be utilized to achieve this: http://nbviewer.ipython.org/github/mathewzilla/redcard/blob/master/Crowdstorming_visualisation.ipynb. You should get inspiration from this example, but still choose your own data preparation steps.

```python
# Import original data and get it in displayable form.
file = os.path.join('data','CrowdstormingDataJuly1st.csv')
df = pd.read_csv(file)
```

By default the data is in a rather counter-intuitive format: referee-player dyads.

A referee-player dyad describes the interactions between a particular ref and one player. This means that each row in the dataset is of a unique player-ref combination, listing all of the games by a given player with a particular referee at any point in his career. Let's look at the first few rows of the dataset as an example:

```python
# Display the first 10 rows of the dataset. Only 13 columns for space reasons
df.iloc[:10,:13]
```

      playerShort
      player
      club
      leagueCountry
      birthday
      height
      weight
      position
      games
      victories
      ties
      defeats
      goals

      0
      lucas-wilchez
      Lucas Wilchez
      Real Zaragoza
      Spain
      31.08.1983
      177.0
      72.0
      Attacking Midfielder
      1
      0
      0
      1
      0

      1
      john-utaka
      John Utaka
      Montpellier HSC
      France
      08.01.1982
      179.0
      82.0
      Right Winger
      1
      0
      0
      1
      0

      2
      abdon-prats
      Abdón Prats
      RCD Mallorca
      Spain
      17.12.1992
      181.0
      79.0
      NaN
      1
      0
      1
      0
      0

      3
      pablo-mari
      Pablo Marí
      RCD Mallorca
      Spain
      31.08.1993
      191.0
      87.0
      Center Back
      1
      1
      0
      0
      0

      4
      ruben-pena
      Rubén Peña
      Real Valladolid
      Spain
      18.07.1991
      172.0
      70.0
      Right Midfielder
      1
      1
      0
      0
      0

      5
      aaron-hughes
      Aaron Hughes
      Fulham FC
      England
      08.11.1979
      182.0
      71.0
      Center Back
      1
      0
      0
      1
      0

      6
      aleksandar-kolarov
      Aleksandar Kolarov
      Manchester City
      England
      10.11.1985
      187.0
      80.0
      Left Fullback
      1
      1
      0
      0
      0

      7
      alexander-tettey
      Alexander Tettey
      Norwich City
      England
      04.04.1986
      180.0
      68.0
      Defensive Midfielder
      1
      0
      0
      1
      0

      8
      anders-lindegaard
      Anders Lindegaard
      Manchester United
      England
      13.04.1984
      193.0
      80.0
      Goalkeeper
      1
      0
      1
      0
      0

      9
      andreas-beck
      Andreas Beck
      1899 Hoffenheim
      Germany
      13.03.1987
      180.0
      70.0
      Right Fullback
      1
      1
      0
      0
      0

```python
# Display the other columns too
df.iloc[:10,13:28]
```

      yellowCards
      yellowReds
      redCards
      photoID
      rater1
      rater2
      refNum
      refCountry
      Alpha_3
      meanIAT
      nIAT
      seIAT
      meanExp
      nExp
      seExp

      0
      0
      0
      0
      95212.jpg
      0.25
      0.50
      1
      1
      GRC
      0.326391
      712.0
      0.000564
      0.396000
      750.0
      0.002696

      1
      1
      0
      0
      1663.jpg
      0.75
      0.75
      2
      2
      ZMB
      0.203375
      40.0
      0.010875
      -0.204082
      49.0
      0.061504

      2
      1
      0
      0
      NaN
      NaN
      NaN
      3
      3
      ESP
      0.369894
      1785.0
      0.000229
      0.588297
      1897.0
      0.001002

      3
      0
      0
      0
      NaN
      NaN
      NaN
      3
      3
      ESP
      0.369894
      1785.0
      0.000229
      0.588297
      1897.0
      0.001002

      4
      0
      0
      0
      NaN
      NaN
      NaN
      3
      3
      ESP
      0.369894
      1785.0
      0.000229
      0.588297
      1897.0
      0.001002

      5
      0
      0
      0
      3868.jpg
      0.25
      0.00
      4
      4
      LUX
      0.325185
      127.0
      0.003297
      0.538462
      130.0
      0.013752

      6
      0
      0
      0
      47704.jpg
      0.00
      0.25
      4
      4
      LUX
      0.325185
      127.0
      0.003297
      0.538462
      130.0
      0.013752

      7
      0
      0
      0
      22356.jpg
      1.00
      1.00
      4
      4
      LUX
      0.325185
      127.0
      0.003297
      0.538462
      130.0
      0.013752

      8
      0
      0
      0
      16528.jpg
      0.25
      0.25
      4
      4
      LUX
      0.325185
      127.0
      0.003297
      0.538462
      130.0
      0.013752

      9
      0
      0
      0
      36499.jpg
      0.00
      0.00
      4
      4
      LUX
      0.325185
      127.0
      0.003297
      0.538462
      130.0
      0.013752

We already see some strange things in the data. Some refs officiated very few games. The two raters disagree about player skintone quite often. For some players there isn't a photo, so their skin tone couldn't be rated. Most dyads don't feature cards at all. In general, it's difficult to get an intuition for what the population looks like from inspecting small samples of the data. This is particularly difficult in the dyad format, as each player or ref's cards are spread un-evenly across the dataset.

#### Data Preparation Steps

The following questions may guide
you:

- What do the feature names (e.g. `column games`) stand for?
  - There is a list (`README.md`) that explains all the names.
- Which irrelevant features might be dropped?
  - We dropped the name of the players but kept the short code to distinguish the players.
  - Also the players characteristics like weight, height, etc. are not important.
  - The club and league is not important.
- What relevant features might be missing, but can be computed? E.g., you can obtain the age of a player (which might be relevant) from his birthday, or create entirely new features by non-linear combinations of existing ones.
  - We combined yellowReds and redCards into total number of red cards ('reds')
  - We don't think that there are more relevant features.
- Are there missing data values (e.g. missing skin color ratings), and how should they be dealt with (see https://en.wikipedia.org/wiki/Missing_data)?
  - Yes, there are missing data values. We just drop them from our data set and don't use them for training.
- How good are the skin color ratings? Do the raters agree?
  - The ratings disagree quite often but are not too different. That's why we just take the mean of the two ratings (reported as the new 'skin' column). A simple chart shows this disagreement below in the code.
- Should referees with very few appearances be excluded from the dataset?
  - No, they should not. We take every referee into account.
- Should features be normalized and/or centralized?
  - For each unique player, we calculate an average probability of getting a red card from all his encounters with different referees.

Categorical features (e.g. `league`) should be transformed to a one-hot  ncoding (see https://en.wikipedia.org/wiki/One-hot). In case of league, you can also repreat the experiment independently for the different leagues to check if there are differences between countries. Provide a detailed description and justification of your data preparation.

```python
### Cleaning the data:

df2 = df.dropna(subset=['photoID'])  #drop all dyads without photo

df2 = df2.assign(skin=df.loc[:, ["rater1", "rater2"]].mean(axis=1))   #create skin feature (mean of the 2 raters)

df2 = df2.assign(reds=df.loc[:, ["yellowReds", "redCards"]].sum(axis=1))   # create new column summing both types of red cards

### simple chart to show disagreement in skin colour evaluation:
df3 = df2.groupby(["playerShort",'rater1', 'rater2'], as_index=False).sum()
rate_diff = abs(df3['rater1'] - df3['rater2']).value_counts()
print('Raters Disgreement (absolute difference between the 2 skin rates | percentaage of players): \n', rate_diff/len(df3['rater1'])*100 )
###

DATA = df2.groupby(["playerShort",'skin'], as_index=False).sum()  #group dyads by player; now number of rows = number of players

DATA["reds/N"] = DATA["reds"]/DATA["games"]   # create new column with the label corresponding to each player

DATA = DATA.loc[:,["skin","reds/N","refNum"]]   #display final DATA; just skin (1D feature) and red cards/games played (labels)
DATA

#DATA = DATA.loc[:,["playerShort","games","reds","skin","reds/N"]]   #we could show other columns too
#DATA = DATA.groupby("skin", as_index=False).mean()
```

    Raters Disgreement (absolute difference between the 2 skin rates | percentaage of players):
     0.00    76.088328
    0.25    23.785489
    0.50     0.126183
    dtype: float64

      skin
      reds/N
      refNum

      0
      0.125
      0.000000
      267701

      1
      0.125
      0.002976
      164589

      2
      0.250
      0.000000
      161486

      3
      0.000
      0.003846
      173524

      4
      0.250
      0.048387
      59603

      ...
      ...
      ...
      ...

      1580
      0.250
      0.005102
      220384

      1581
      0.000
      0.000000
      79202

      1582
      0.875
      0.020253
      140649

      1583
      0.000
      0.042553
      34125

      1584
      0.000
      0.000000
      46935

1585 rows × 3 columns

### 2.2 Model Creation

#### Linear Regression

Linear regression determines a weighted sum of the features $\hat{Y}_i = X_i \hat{\beta} + \hat{b}$, where optimal weights
and intercept minimize the squared error:

\begin{equation}
\hat{\beta}, \, \hat{b} = \text{arg min}_{\beta, b} \sum_i (X_i \beta + b - Y_i^*)^2 \,.
\end{equation}

Differentiating with respect to $\beta$ and $b$ and finding the minimum yields:
\begin{equation}
\hat{\beta} = \frac{\sum_i X_i Y_i^* - N \bar{X}\bar{Y}^*}{\sum_i X_i^2 - N \bar{X}^2} \,, \,
\hat{b} = \bar{Y}^* - \hat{\beta} \bar{X} \,,
\end{equation}
where $N$ is the total number of players and $\bar{X}$, $\bar{Y}^*$ are the averages of $X_i$ and $Y_i^*$.

```python
class LinearRegression:
    def __init__(self):
        self.beta = None
        self.b = None

    def train(self, data, labels):
        '''
        data: the feature matrix (N, D)
        labels: the corresponding ground-truth responses (N)
        '''
        N, D = data.shape # N: number of players, D: feature dimension
        labels = labels.reshape((N, 1))

        # Average feature and label vector
        X = np.mean(data, axis=0)
        Y = np.mean(labels, axis=0)

        self.beta = ( np.mean(data * labels, axis=0) - X * Y ) / ( np.mean(np.sum(data**2, axis=1), axis=0) - np.dot(X, X) )
        self.b = Y - np.dot(self.beta, X)

        return self.beta, self.b

    def predict(self, data):
        # Compute Y = beta * X + b
        return np.matmul(data, self.beta) + self.b
```

```python
# Short visualization of how the Linear Regression works

data = np.atleast_2d(DATA[["skin","refNum"]])
labels = np.atleast_2d(DATA["reds/N"])

a = LinearRegression()
a.train(data, labels)
predictions = a.predict(data)

print('Predictions: ', predictions)

N = np.linspace(0, 1584, 1585)
plt.plot(N, labels.reshape((1585)), color='green', label='Data')
plt.plot(N, predictions.reshape((1585)), color='red', label='Prediction')
plt.xlim(0, 200)
plt.ylim(0, 0.08)
plt.legend()
plt.show()
```

#### Regression Forest

```python
# Base classes
class Node:
    pass

class Tree:
    def __init__(self):
        self.root = Node()

    def find_leaf(self, x):
        node = self.root
        while hasattr(node, "feature"):
            j = node.feature
            if x[j] <= node.threshold:
                node = node.left
            else:
                node = node.right
        return node
```

```python
class RegressionTree(Tree):
    def __init__(self):
        super(RegressionTree, self).__init__()

    def train(self, data, labels, n_min=10):
        '''
        data: the feature matrix
        labels: the corresponding ground-truth responses
        n_min: termination criterion (don't split if a node contains fewer instances)
        '''
        N, D = data.shape
        D_try = int(np.sqrt(D)) # how many features to consider for each split decision

        # initialize the root node
        self.root.data = data
        self.root.labels = labels

        #put root in stack
        stack = [self.root]
        while len(stack):
            node = stack.pop()
            n = node.data.shape[0] # number of instances in present node
            if n >= n_min:
                # Call 'make_decision_split_node()' with 'D_try' randomly selected
                # feature indices. This turns 'node' into a split node
                # and returns the two children, which must be placed on the 'stack'.
                perm = np.random.permutation(D)   # permute D indices
                left, right = make_decision_split_node(node, perm[:D_try]) #select :D_try of permuted indices
                                                       #for 'make_decision_split_node()'
                # put children in stack
                stack.append(left)
                stack.append(right)
            else:
                # Call 'make_decision_leaf_node()' to turn 'node' into a leaf node.
                make_decision_leaf_node(node)

    def predict(self, x):
        leaf = self.find_leaf(x)
        # compute p(y | x)
        return leaf.response
```

```python
def make_decision_split_node(node, feature_indices):
    '''
    node: the node to be split
    feature_indices: a numpy array of length 'D_try', containing the feature
                     indices to be considered in the present split
    '''
    n, D = node.data.shape

    # find best feature j (among 'feature_indices') and best threshold t for the split
    e_min = 1e100
    j_min, t_min = 0, 0
    for j in feature_indices:
        # remove duplicate features
        dj = np.sort(np.unique(node.data[:,j]))
        # compute candidate thresholds in the middle between consecutive feature values
        tj = 0.5 * (dj[1:] + dj[:-1])
        # each candidate threshold we need to compute squared errors of the resulting children node
        for t in tj:
            left_indices = node.data[:,j] <= t
            ll = node.labels[left_indices]
            el = np.sum(np.square(ll - np.mean(ll)))
            lr = node.labels[node.data[:,j] > t]
            er = np.sum(np.square(lr - np.mean(lr)))
            # choose the the best threshold that minimizes sum of squared errors
            if el + er < e_min:
                e_min = el + er
                j_min = j
                t_min = t

    # create children
    left = Node()
    right = Node()

    # initialize 'left' and 'right' with the data subsets and labels
    # according to the optimal split found above
    left.data = node.data[node.data[:,j_min] <= t_min, :]
    left.labels = node.labels[node.data[:,j_min] <= t_min]
    right.data = node.data[node.data[:,j_min] > t_min, :]
    right.labels = node.labels[node.data[:,j_min] > t_min]

    # turn the current 'node' into a split node
    # (store children and split condition)
    node.left = left
    node.right = right
    node.feature = j_min
    node.threshold = t_min

    # return the children (to be placed on the stack)
    return left, right
```

```python
def make_decision_leaf_node(node):
    '''
    node: the node to become a leaf
    '''
    node.N = node.labels.shape[0]
    node.response = np.mean(node.labels)
```

```python
class RegressionForest():
    def __init__(self, n_trees=10):
        # create ensemble
        self.trees = [RegressionTree() for i in range(n_trees)]

    def train(self, data, labels, n_min=10):
        for tree in self.trees:
            # train each tree, using a bootstrap sample of the data
            bootstrap = np.random.choice(len(data), size=len(data))
            tree.train(data[bootstrap], labels[bootstrap], n_min)

    def predict(self, x):
        # compute the ensemble prediction
        return np.mean([tree.predict(x) for tree in self.trees], axis=0)
```

```python
# Short visualization of how the Regression Forest works

data = np.atleast_2d(DATA[["skin","refNum"]])
labels = np.atleast_2d(DATA["reds/N"]).reshape((len(data)))

b = RegressionForest()
b.train(data, labels)

predictions = np.zeros(len(data[:,0]))

for i in range(len(data[:,0])):
    predictions[i] = b.predict(data[i,:])

print('Predictions: ', predictions)

N = np.linspace(0, 1584, 1585)
plt.plot(N,labels.reshape((1585)), color='green', label='Data')
plt.plot(N,predictions.reshape((1585)), color='red', label='Prediction')
plt.xlim(0, 200)
plt.ylim(0, 0.08)
plt.legend()
plt.show()
```

We see that the Regression Forest is much more flexible and fits the data better.

#### Cross-Validation

```python
def cross_validate(n_folds, data, labels):
    N, D = data.shape
    linear_regression = LinearRegression()
    regression_forest = RegressionForest()

    results = []
    error_rates_linear = []; error_rates_forest = []

    perm = range(N)
    X_folds = np.array_split(data[perm], n_folds)
    y_folds = np.array_split(labels[perm], n_folds)

    for n in range(n_folds):
        # construct train and test set from folds
        X_train_f = np.concatenate([X_folds[i] for i in range(n_folds) if i != n])
        X_test_f = X_folds[n]
        y_train_f = np.concatenate([y_folds[i] for i in range(n_folds) if i != n])
        y_test_f = y_folds[n]

        # run linear regression
        beta, b = linear_regression.train(X_train_f, y_train_f)
        results_linear = linear_regression.predict(X_test_f)
        error_rates_linear.append(np.sum(np.square(results_linear - y_test_f)))

        # run regression forest
        regression_forest.train(X_train_f, y_train_f)
        results_forest = np.zeros(len(X_test_f[:,0]))
        for i in range(len(X_test_f[:,0])):
            results_forest[i] = regression_forest.predict(X_test_f[i,:])
        error_rates_forest.append(np.sum(np.square((results_forest - y_test_f)[np.isfinite(results_forest)]))) # filter out nan

    results.append(np.array(
            [[np.mean(error_rates_linear), np.std(error_rates_linear)],
              [np.mean(error_rates_forest), np.std(error_rates_forest)]]
        ))

    return results[0]
```

```python
# Perform cross-validation for 10 folds

errors = cross_validate(10, data, labels)
```

    C:\Users\Eugen\anaconda3\lib\site-packages\numpy\core\fromnumeric.py:3334: RuntimeWarning: Mean of empty slice.
      return _methods._mean(a, axis=axis, dtype=dtype,
    C:\Users\Eugen\anaconda3\lib\site-packages\numpy\core\_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
      ret = ret.dtype.type(ret / rcount)

```python
print('Linear Regression Error: {:2.2%} +- {:2.2%}.'.format(errors[0][0], errors[0][1]))
print('Regression Forest Error: {:2.2%} +- {:2.2%}.'.format(errors[1][0], errors[1][1]))
```

    Linear Regression Error: 1.74% +- 0.55%.
    Regression Forest Error: 2.17% +- 0.64%.

```python
# Dependence on the n-folds

errors_linear = []
errors_forest = []
stds_linear = []
stds_forest = []

for i in range(2,11):
    errors = cross_validate(i, data, labels)
    errors_linear.append(errors[0][0])
    errors_forest.append(errors[1][0])
    stds_linear.append(errors[0][1])
    stds_forest.append(errors[1][1])
```

```python
n = np.linspace(2, 10, 9)
plt.errorbar(n, errors_linear, yerr=stds_linear, label='Linear Regression', capsize=2)
plt.errorbar(n, errors_forest, yerr=stds_forest, label='Regression Forest', capsize=2)
plt.xlabel('n-folds')
plt.ylabel('Cross-Validation Mean Squared Error')
plt.legend()
plt.show()
```

We can follow that the error of the Regression Forest is slightly higher than that of the Linear Regression. It seems like the Regression Forest fits the training data very well but doesn't generalize so well for unseen data.

### 2.3 Answering the research question

```python
# Produce 19 different test sets with shuffled skin color variable

def permutation_test(n_folds, data, labels, n_permutations):
    # produce some array to fit the data in
    performation_data = np.full((n_permutations, len(data), len(data[0])), data)
    skin_var = data[:,0] # skin has to be at first position in data

    for i in range(n_permutations):
        # shuffle skin data and then write it into the data set
        np.random.shuffle(skin_var)
        performation_data[i,:,0] = skin_var

    # calculate the cross validation error for each data set
    performation_test_error = np.zeros((n_permutations, 2, 2))
    for i in range(n_permutations):
        performation_test_error[i] = cross_validate(n_folds, performation_data[i], labels)

    return performation_test_error
```

```python
# Get the error for all permutations seperately

error_with_perm = permutation_test(4, data, labels, 19)
```

```python
error_no_perm = cross_validate(4, data, labels)
```

We produced N_permutations data sets with differently shuffeled skin variable among the players. Then we calculate the error by cross-validation. If the error of the shuffeled datasets is always larger than the original dataset, we can state thatthere is a bias. The following graph should show wether the error is above or below the original datset for the several shuffeled datasets.

```python
n = np.arange(19) #number of permutations

plt.errorbar(n, error_with_perm[:,0,0], yerr=error_with_perm[:,0,1], label="linear regression", capsize=3)
plt.plot(n, np.full(len(n),error_no_perm[0,0]), "g", label="linear regression without permutation")
plt.plot(n, np.full(len(n),error_no_perm[0,0])+np.full(len(n),error_no_perm[0,1]), ":", color="g")
plt.plot(n, np.full(len(n),error_no_perm[0,0])-np.full(len(n),error_no_perm[0,1]), ":", color="g")

plt.ylabel("error")
plt.xlabel("number of permutations")
plt.xlim(0,18)
plt.ylim(0.034,0.055)
plt.legend(bbox_to_anchor=(1, 0.5))
plt.show()
```

```python
plt.errorbar(n, error_with_perm[:,1,0], yerr=error_with_perm[:,1,1], label="regression forest", capsize=3)
plt.plot(n, np.full(len(n),error_no_perm[1,0]), "r", label="regression forest without permutation")
plt.plot(n, np.full(len(n),error_no_perm[1,0])+np.full(len(n),error_no_perm[1,1]), ":", color="r")
plt.plot(n, np.full(len(n),error_no_perm[1,0])-np.full(len(n),error_no_perm[1,1]), ":", color="r")

plt.ylabel("error")
plt.xlabel("number of permutations")
plt.xlim(0,18)
plt.ylim(0.04,0.069)
plt.legend(bbox_to_anchor=(1, 0.5))
plt.show()
```

Since the absolute error of the Regression Forest with permutations is not significantly larger than the absolute error of the Regression Forest without permutations (we have the same behavior for the Linear Regression model), this gives rise to no bias!
We thus think that there is no correspondence between the skin color of a player and the number of red cards.

```python

```


```python
# %load /Users/facai/Study/book_notes/preconfig.py
%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
#sns.set(font='SimHei', font_scale=2.5)
#plt.rcParams['axes.grid'] = False

import numpy as np

import pandas as pd
#pd.options.display.max_rows = 20

#import sklearn

#import itertools

#import logging
#logger = logging.getLogger()

#from IPython.display import SVG
from IPython.display import Image
```

Chapter 3: Finite Markov Decision Processes
==========

MDP(Markov Decision Processes): actions influence not just immediate rewards, but also subsequential situations.

### 3.1 The Agent-Environment Interface

```python
Image('./res/fig3_1.png')
```

*finite* MDP: the sets of states, actions and rewards all have a finite number of elements.

\begin{align}
    & p(s', r \mid s, a) \doteq \operatorname{Pr} \{ S_t = s', R_t = r \mid S_{t-1} = s, A_{t-1} = a \} \\
    & \displaystyle \sum_{s' \in \mathcal{S}} \sum_{r \in \mathcal{R}} p(s', r \mid s, a) = 1 \quad \text{, for all $s \in \mathcal{S}$, $a \in \mathcal{A}(s)$}
\end{align}

+ state-transition probabilities:

\begin{align*}
    p(s' \mid s, a) & \doteq \operatorname{Pr} \{ S_t = s' \mid S_{t-1} = s, A_{t-1} = a \} \\
                    & = \sum_{r \in \mathcal{R}} p(s', r \mid s, a)
\end{align*}

+ expected rewards for state-action paris:

\begin{align*}
    r(s, a) & \doteq \mathbb{E} \left [ R_t \mid S_{t-1} = s, A_{t-1} = a \right ] \\
            & = \sum_{r \in \mathcal{R}} \left ( r \sum_{s' \in \mathcal{S}} p(s', r \mid s, a) \right )
\end{align*}

+ expected rewards for state-action-next-state triples:

\begin{align*}
    r(s, a, s') & \doteq \mathbb{E} \left [ R_t \mid S_{t-1} = s, A_{t-1} = a, S_t = s' \right ] \\
            & = \sum_{r \in \mathcal{R}} r \frac{p(s', r \mid s, a)}{p(s' \mid s, a)}
\end{align*}

agent-environment boundary represents the limit of the agent's *absolute control*, not of its knowledge.

```python
# Transition Graph
Image('./res/ex3_3.png')
```

##### Exercise 3.4

$r(S_t, a) \; \pi(a \mid S_t)$

### 3.2 Goals and Rewards

goal: to maximize the total amount of reward it receives.

In particular, the reward signal is not the place to impart to the agent prior knowledge about *how* to achieve what it to do.

### 3.3 Returns and Episodes

+ episodic tasks: $G_t \doteq R_{t+1} + R_{t+2} + R_{t+3} + \cdots + R_T$, $\quad G_t$ is *expected return*.
+ continuing tasks: $G_t \doteq R_{t+1} + \gamma R_{t+2} + \gamma^2 R_{t+3} + \cdots = \displaystyle \sum_{k=0}^\infty \gamma^k R_{t+k+1} = R_{t+1} + \gamma G_{t+1}$
  - $\gamma$ is called *discount rate*, and $0 \leq \gamma \leq 1$
  - $G_T = 0$ often makes it easy to compute returns from reward sequences.

##### exercise 3.8

0 for escaping from the maze and -1 at all other times

##### exercise 3.10

$G_1 = \frac{7}{1 - r} = 70$

$G_0 = R_1 + 0.9 G_1 = 65$

##### exercise 3.11

\begin{align}
    G_t &= \sum_{k=0}^\infty r^k \\
        &= 1 + r \sum_{k=0}^\infty r^k \\
        &= 1 + r G_t \\
        &= \frac1{1 - r}
\end{align}

### 3.4 Unified Notation for Episodic and Continuing Tasks

$G_t \doteq \sum_{k=t+1}^T \gamma^{k-t-1} R_k$, including the possibility that $T = \infty$ or $\gamma = 1$.

### 3.5 Policies and Value Functions

+ value functions: estimate *how good* it is for the agent to be in a given state.
+ policy $\pi$: a mapping from states to probabilities of selecting each possible action.
+ The *value* of a state $s$ under a policy $\pi$, denoted $v_\pi(s)$, is the expected return when starting in $s$ and following $\pi$ thereafter:
  $v_\pi(s) \doteq \mathbb{E}_\pi [ G_t \mid S_t = s ]$
  - $v_\pi$: state-value function for policy $\pi$
+ $q_\pi$: action-value function for policy $\pi$, $q_\pi(s, a) \doteq \mathbb{E}_\pi [ G_t \mid S_t = s, A_t = a]$

Bellman equation for $v_\pi$:

\begin{equation}
    v_\pi(s) \doteq \displaystyle \sum_a \pi(a \mid s) \sum_{s', r} p(s', r \mid s, a) \left [ r + \gamma v_\pi(s') \right ] \quad \text{, for all $s \in \mathcal{S}$}
\end{equation}

The value functions $v_\pi$ and $q_\pi$ can be estimated from experience, say Monte Carlo methods (average).

```python
# Example 3.5
from scipy.signal import convolve2d

reward_matrix = np.zeros((5, 5))
# kernel
kernel = np.array([[0, 1, 0],
                   [1, 0, 1],
                   [0, 1, 0]])

iteration_nums = 100

for _ in range(iteration_nums):
    reward = convolve2d(reward_matrix, kernel, mode='same', boundary='fill', fillvalue=-1)
    reward /= 4.0
    # A -> A'
    reward[0, 1] = 10 + reward[-1, 1]
    # B -> B'
    reward[0, -2] = 5 + reward[2, -2]
    reward_matrix = reward

pd.DataFrame(reward_matrix)
```

      0
      1
      2
      3
      4

      0
      2.253150
      9.592586
      4.524364
      6.244556
      1.271214

      1
      1.420013
      3.970976
      3.260312
      2.897431
      0.840299

      2
      0.455929
      1.610993
      1.648482
      1.244556
      0.192553

      3
      -0.207290
      0.368589
      0.478065
      0.239763
      -0.314645

      4
      -0.653676
      -0.407414
      -0.344568
      -0.448925
      -0.690892

##### exercise 3.12

(2.3 + 0.4 - 0.4 + 0.7) / 4 = 0.75

##### exercise 3.13

##### exercise 3.14

$\sum_{k=0}^\infty \gamma^k C = \frac{C}{1 - r}$ = constant offset

### 3.6 Optimal Policies and Optimal Value Functions

optimal policy:

\begin{align}
    v_\ast(s) & \doteq \displaystyle \max_\pi v_\pi(s) \quad \text{ for all } s \in \mathcal{S} \\
              & = \max_a \sum_{s', r} p(s', r \mid s, a) \left [ r + \gamma v_\ast(s') \right ] \\
\end{align}

Any policy that is *greedy* with respect to the optimal evaluation function $v_\ast$ is an optimal policy.

optimal action-value function:

\begin{align}
    q_\ast(s, a) & \doteq \max_\pi q_\pi(s, a) \quad \text{ for all $s \in \mathcal{S}$ and $a \in \mathcal{A}(s)$} \\
                 & = \mathbb{E} [ R_{t+1} + \gamma v_\ast(S_{t+1}) \mid S_t = s, A_t = a ] \\
                 & = \sum_{s', r} p(s', r \mid s, a) \left [ r + \gamma \max_{a'} q_\ast (s', a') \right ] \\
\end{align}

Explicitly solving the Bellman optimality equation relies on at least three assumptions that are rarely true in practice:

1. we accurately know the dynamics of the environment;
2. we have enough computational resources;
3. the Markov property.

### 3.7 Optimality and Approximation

extreme computational cost, memory => approximations

put more effort into learning to make good decisions for frequently encountered states, at the expense of less effort for infrequently encountered states.

```python

```


# Load data

```
from google.colab import drive
drive.mount('/content/drive')
import sys
sys.path.append('BScThesis/')
```

    Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code

    Enter your authorization code:
    ··········
    Mounted at /content/drive

```
!pip3 install torch torchvision
```

    Collecting torch
      Downloading https://files.pythonhosted.org/packages/7e/60/66415660aa46b23b5e1b72bc762e816736ce8d7260213e22365af51e8f9c/torch-1.0.0-cp36-cp36m-manylinux1_x86_64.whl (591.8MB)
        100% |████████████████████████████████| 591.8MB 26kB/s
    tcmalloc: large alloc 1073750016 bytes == 0x62826000 @  0x7fda7e9502a4 0x591a07 0x5b5d56 0x502e9a 0x506859 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x504c28 0x502540 0x502f3d 0x507641
    Collecting torchvision
      Downloading https://files.pythonhosted.org/packages/ca/0d/f00b2885711e08bd71242ebe7b96561e6f6d01fdb4b9dcf4d37e2e13c5e1/torchvision-0.2.1-py2.py3-none-any.whl (54kB)
        100% |████████████████████████████████| 61kB 21.1MB/s
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.14.6)
    Collecting pillow>=4.1.1 (from torchvision)
      Downloading https://files.pythonhosted.org/packages/85/5e/e91792f198bbc5a0d7d3055ad552bc4062942d27eaf75c3e2783cf64eae5/Pillow-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (2.0MB)
        100% |████████████████████████████████| 2.0MB 6.6MB/s
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.11.0)
    Installing collected packages: torch, pillow, torchvision
      Found existing installation: Pillow 4.0.0
        Uninstalling Pillow-4.0.0:
          Successfully uninstalled Pillow-4.0.0
    Successfully installed pillow-5.4.1 torch-1.0.0 torchvision-0.2.1

```
!git clone https://github.com/wiatrak2/BScThesis
```

    Cloning into 'BScThesis'...
    remote: Enumerating objects: 14, done.
    remote: Counting objects: 100% (14/14), done.
    remote: Compressing objects: 100% (11/11), done.
    remote: Total 364 (delta 5), reused 9 (delta 3), pack-reused 350
    Receiving objects: 100% (364/364), 56.72 MiB | 10.07 MiB/s, done.
    Resolving deltas: 100% (209/209), done.

```
from importlib import reload
```

```
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
```

```
import torchvision
import torchvision.transforms as transforms
```

```
import matplotlib.pyplot as plt
import pickle
import sympy
from collections import defaultdict, namedtuple
import copy
```

```
from mnist_m import Mnist_M
from random_dataset import RandomDataset
import datasets_merge
import models
import svhn_models
import trainer
import domain_trainer
from test_model import test_model
import plots
import test
import plotly_plots
```

```
import backward_functions
```

```
import plotly
from plotly import tools
from plotly.offline import init_notebook_mode, iplot
import plotly.graph_objs as go
```

```
def convertMnist(img):
  img = img.squeeze().numpy()
  return torch.from_numpy(np.tile(img, (3,1,1)))
```

```
batch_size=128
num_workers=1
```

```
mnist_transform=transforms.Compose([
                           transforms.ToTensor(),
                           transforms.Normalize((0.1307, 0.1307, 0.1307), (0.3081, 0.3081, 0.3081)),
                           transforms.Lambda(convertMnist)
                       ])

mnist_train_set = torchvision.datasets.MNIST(root='./data', train=True, download=True,
                                     transform=mnist_transform)
mnist_train_loader = torch.utils.data.DataLoader(mnist_train_set, batch_size=batch_size,
                                         shuffle=True, num_workers=num_workers)

mnist_test_set = torchvision.datasets.MNIST(root='./data', train=False, download=True,
                                     transform=mnist_transform)
mnist_test_loader = torch.utils.data.DataLoader(mnist_test_set, batch_size=batch_size,
                                         shuffle=False, num_workers=num_workers)

```

    Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
    Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
    Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
    Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
    Processing...
    Done!

```
mnist_m_transform = transforms.Compose([
                           transforms.Normalize((0.1307, 0.1307, 0.1307), (0.3081, 0.3081, 0.3081)),
                       ])
mnist_m_train_set = Mnist_M('drive/My Drive/BScThesis/keras_mnistm.pkl',
                        transform=mnist_m_transform)
mnist_m_test_set = Mnist_M('drive/My Drive/BScThesis/keras_mnistm.pkl',train=False,
                     transform=mnist_m_transform)
mnist_m_train_loader = torch.utils.data.DataLoader(mnist_m_train_set,
                                                   batch_size=batch_size,
                                         shuffle=True, num_workers=num_workers)
mnist_m_test_loader = torch.utils.data.DataLoader(mnist_m_test_set,
                                                  batch_size=batch_size,
                                         shuffle=False, num_workers=num_workers)
```

```
merge_train_set = datasets_merge.Merge_Datasets([mnist_train_set, mnist_m_train_set], get_labels=False)
merge_test_set = datasets_merge.Merge_Datasets([mnist_test_set, mnist_m_test_set], get_labels=False)
merge_train_loader = torch.utils.data.DataLoader(merge_train_set,
                                                   batch_size=batch_size,
                                         shuffle=True, num_workers=num_workers)
merge_test_loader = torch.utils.data.DataLoader(merge_test_set,
                                                   batch_size=batch_size,
                                         shuffle=True, num_workers=num_workers)
```

```
log_interval = 100
```

```
use_cuda = True
use_cuda = use_cuda and torch.cuda.is_available()
device = torch.device("cuda" if use_cuda else "cpu")
```

```
lr = 0.01
momentum=0.9
```

```
Models = namedtuple('Models', ['model_f', 'model_c', 'model_d'])
Optims = namedtuple('Optims', ['optim_f', 'optim_c', 'optim_d'])
Criterions = namedtuple('Criterions', ['criterion', 'criterion_domain'])
Loaders = namedtuple('Loaders', ['train_loader', 'train_loader_domain',
                                'source_test_loader', 'target_test_loader',
                                'merged_test_loader'])
```

```
loaders = Loaders(mnist_train_loader, mnist_m_train_loader,
                 mnist_test_loader, mnist_m_test_loader, merge_test_loader)
```

```
criterion = F.nll_loss
criterion_domain = F.nll_loss
```

```
criterions = Criterions(criterion, criterion_domain)
```

```
epochs=10
```

# Testing functions

```
def test_grad_function(grad_function, epochs=4):
  model_f = models.MnistFeatureExtractor().to(device)
  model_c = models.MnistClassPredictor().to(device)
  model_d = models.MnistDomain(grad_func = grad_function).to(device)

  optim_f = optim.SGD(model_f.parameters(), lr=lr, momentum=momentum, nesterov=True)
  optim_c = optim.SGD(model_c.parameters(), lr=lr, momentum=momentum, nesterov=True)
  optim_d = optim.SGD(model_d.parameters(), lr=lr, momentum=momentum, nesterov=True)

  train_models = Models(model_f, model_c, model_d)
  optims = Optims(optim_f, optim_c, optim_d)

  model_trainer = trainer.Trainer(train_models, optims, criterions, device,
                                train_domain=True, tune_lr=True)

  test_history = defaultdict(lambda:[])
  model_trainer.train(4, loaders, test_history=test_history)
```

```
grad_log = []
def grlog(x, lambd):
  return backward_functions.GRLog(lambd, grad_log)(x)
```

```
grad_log = []
def grinv(x, lambd):
  return backward_functions.GRInv(-1, lambd, grad_log)(x)
```

```
test_grad_function(grlog)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.302571, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.632978, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.591048, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.404582, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.264745, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 1.2978, Accuracy: 6271/10000 (63%)
    Source Domain Test set: Average loss: 0.1211, Accuracy: 9614/10000 (96%)
    Domains predictor:  Accuracy: 14577/20000 (73%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.295873, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.283074, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.267564, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.297445, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.162265, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 1.1145, Accuracy: 7147/10000 (71%)
    Source Domain Test set: Average loss: 0.0864, Accuracy: 9736/10000 (97%)
    Domains predictor:  Accuracy: 13037/20000 (65%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.189036, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.161942, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.078210, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.164317, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.218790, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 0.9194, Accuracy: 7466/10000 (75%)
    Source Domain Test set: Average loss: 0.0728, Accuracy: 9761/10000 (98%)
    Domains predictor:  Accuracy: 13804/20000 (69%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.184848, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.241879, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.165163, lr: 0.00184 lambd: 0.99962

```
test_grad_function(grinv)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.318720, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.810656, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.549511, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.457427, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.222553, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 1.7708, Accuracy: 6613/10000 (66%)
    Source Domain Test set: Average loss: 0.1464, Accuracy: 9558/10000 (96%)
    Domains predictor:  Accuracy: 10350/20000 (52%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.371020, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.264524, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.335705, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.398240, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.152408, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 1.3885, Accuracy: 6716/10000 (67%)
    Source Domain Test set: Average loss: 0.1081, Accuracy: 9662/10000 (97%)
    Domains predictor:  Accuracy: 13639/20000 (68%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.273087, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.390106, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.130011, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.231334, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.475967, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 1.0248, Accuracy: 7316/10000 (73%)
    Source Domain Test set: Average loss: 0.0907, Accuracy: 9712/10000 (97%)
    Domains predictor:  Accuracy: 12666/20000 (63%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.357871, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.313296, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.105079, lr: 0.00184 lambd: 0.99962
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.172043, lr: 0.00177 lambd: 0.99978
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.175342, lr: 0.00170 lambd: 0.99987

    Target Domain Test set: Average loss: 0.9377, Accuracy: 7331/10000 (73%)
    Source Domain Test set: Average loss: 0.0753, Accuracy: 9746/10000 (97%)
    Domains predictor:  Accuracy: 14138/20000 (71%)

```
grad_log = []
def grinv2(x, lambd):
  return backward_functions.GRInv(1., lambd, grad_log)(x)
```

```
test_grad_function(grinv2)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.327919, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.745220, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.533236, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.456913, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.288192, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 2.3159, Accuracy: 3049/10000 (30%)
    Source Domain Test set: Average loss: 0.0995, Accuracy: 9698/10000 (97%)
    Domains predictor:  Accuracy: 19797/20000 (99%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.233389, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.372094, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.303700, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.326173, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.291125, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 2.3855, Accuracy: 2562/10000 (26%)
    Source Domain Test set: Average loss: 0.0701, Accuracy: 9779/10000 (98%)
    Domains predictor:  Accuracy: 19952/20000 (100%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.210821, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.246365, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.287860, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.165999, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.073122, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 2.6827, Accuracy: 2394/10000 (24%)
    Source Domain Test set: Average loss: 0.0575, Accuracy: 9818/10000 (98%)
    Domains predictor:  Accuracy: 19988/20000 (100%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.164869, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.075003, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.145272, lr: 0.00184 lambd: 0.99962
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.138908, lr: 0.00177 lambd: 0.99978
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.106489, lr: 0.00170 lambd: 0.99987

    Target Domain Test set: Average loss: 2.9522, Accuracy: 2196/10000 (22%)
    Source Domain Test set: Average loss: 0.0562, Accuracy: 9821/10000 (98%)
    Domains predictor:  Accuracy: 19993/20000 (100%)

```
grad_log = []
def grandmult(x, lambd):
  return backward_functions.GRandMult(1, lambd, grad_log)(x)
```

```
test_grad_function(grandmult)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.356206, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.634399, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.509887, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.173120, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.351211, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 1.9298, Accuracy: 3438/10000 (34%)
    Source Domain Test set: Average loss: 0.1113, Accuracy: 9651/10000 (97%)
    Domains predictor:  Accuracy: 19679/20000 (98%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.223685, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.170962, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.182587, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.096928, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.161787, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 2.1723, Accuracy: 2546/10000 (25%)
    Source Domain Test set: Average loss: 0.0742, Accuracy: 9770/10000 (98%)
    Domains predictor:  Accuracy: 19886/20000 (99%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.254556, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.233278, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.178423, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.196384, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.183332, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 2.0319, Accuracy: 2830/10000 (28%)
    Source Domain Test set: Average loss: 0.0538, Accuracy: 9836/10000 (98%)
    Domains predictor:  Accuracy: 19911/20000 (100%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.116799, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.159713, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.168556, lr: 0.00184 lambd: 0.99962
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.187594, lr: 0.00177 lambd: 0.99978
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.108288, lr: 0.00170 lambd: 0.99987

    Target Domain Test set: Average loss: 2.2315, Accuracy: 2308/10000 (23%)
    Source Domain Test set: Average loss: 0.0509, Accuracy: 9838/10000 (98%)
    Domains predictor:  Accuracy: 19960/20000 (100%)

```
grad_log = []
def grandmult(x, lambd):
  return backward_functions.GRandMult(1e-1, lambd, grad_log)(x)
```

```
test_grad_function(grandmult)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.312164, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.551612, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.499983, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.353211, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.277616, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 1.5431, Accuracy: 4493/10000 (45%)
    Source Domain Test set: Average loss: 0.1045, Accuracy: 9670/10000 (97%)
    Domains predictor:  Accuracy: 19372/20000 (97%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.343850, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.236943, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.216584, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.133211, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.202780, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 1.4694, Accuracy: 4692/10000 (47%)
    Source Domain Test set: Average loss: 0.0734, Accuracy: 9756/10000 (98%)
    Domains predictor:  Accuracy: 19642/20000 (98%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.237773, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.107190, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.184812, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.199738, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.153193, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 1.5249, Accuracy: 4696/10000 (47%)
    Source Domain Test set: Average loss: 0.0575, Accuracy: 9820/10000 (98%)
    Domains predictor:  Accuracy: 19755/20000 (99%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.079153, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.149514, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.103344, lr: 0.00184 lambd: 0.99962
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.132674, lr: 0.00177 lambd: 0.99978
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.108807, lr: 0.00170 lambd: 0.99987

    Target Domain Test set: Average loss: 1.5512, Accuracy: 4521/10000 (45%)
    Source Domain Test set: Average loss: 0.0498, Accuracy: 9843/10000 (98%)
    Domains predictor:  Accuracy: 19818/20000 (99%)

```
grad_log = []
def grand(x, lambd):
  return backward_functions.GRand(1e-3, lambd, grad_log)(x)
```

```
test_grad_function(grand)
```

    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f1853807358>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process
    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f185046d3c8>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process
    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f1853738c50>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process
    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f1853807358>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process
    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f185046d3c8>>

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.317810, lr: 0.01000 lambd: 0.00000

    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process
    Exception ignored in: <bound method _DataLoaderIter.__del__ of <torch.utils.data.dataloader._DataLoaderIter object at 0x7f1853738c50>>
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 717, in __del__
        self._shutdown_workers()
      File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 713, in _shutdown_workers
        w.join()
      File "/usr/lib/python3.6/multiprocessing/process.py", line 122, in join
        assert self._parent_pid == os.getpid(), 'can only join a child process'
    AssertionError: can only join a child process

    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.712805, lr: 0.00726 lambd: 0.26052
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.296397, lr: 0.00580 lambd: 0.48792
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.255301, lr: 0.00488 lambd: 0.66404
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.488634, lr: 0.00425 lambd: 0.78820

    Target Domain Test set: Average loss: 1.4624, Accuracy: 5065/10000 (51%)
    Source Domain Test set: Average loss: 0.1266, Accuracy: 9597/10000 (96%)
    Domains predictor:  Accuracy: 19248/20000 (96%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.316768, lr: 0.00391 lambd: 0.84828
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.188924, lr: 0.00351 lambd: 0.90812
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.195166, lr: 0.00320 lambd: 0.94505
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.286140, lr: 0.00295 lambd: 0.96740
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.112674, lr: 0.00273 lambd: 0.98074

    Target Domain Test set: Average loss: 1.5594, Accuracy: 4544/10000 (45%)
    Source Domain Test set: Average loss: 0.1062, Accuracy: 9645/10000 (96%)
    Domains predictor:  Accuracy: 18857/20000 (94%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.336886, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.251837, lr: 0.00245 lambd: 0.99213
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.221007, lr: 0.00231 lambd: 0.99537
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.228762, lr: 0.00218 lambd: 0.99728
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.464130, lr: 0.00208 lambd: 0.99841

    Target Domain Test set: Average loss: 1.6057, Accuracy: 4844/10000 (48%)
    Source Domain Test set: Average loss: 0.0963, Accuracy: 9720/10000 (97%)
    Domains predictor:  Accuracy: 18671/20000 (93%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.212811, lr: 0.00201 lambd: 0.99889
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.214758, lr: 0.00192 lambd: 0.99935
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.165383, lr: 0.00184 lambd: 0.99962
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.199856, lr: 0.00177 lambd: 0.99978
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.125342, lr: 0.00170 lambd: 0.99987

    Target Domain Test set: Average loss: 1.5942, Accuracy: 4788/10000 (48%)
    Source Domain Test set: Average loss: 0.0893, Accuracy: 9712/10000 (97%)
    Domains predictor:  Accuracy: 18776/20000 (94%)

# Gradient Inverse Layer test

```
def grinv(x, lambd):
  return backward_functions.GRInv(-1, lambd)(x)
```

```
model_f = models.MnistFeatureExtractor().to(device)
model_c = models.MnistClassPredictor().to(device)
model_d = models.MnistDomain(grad_func=grinv).to(device)
optim_f = optim.SGD(model_f.parameters(), lr=lr, momentum=momentum, nesterov=True)
optim_c = optim.SGD(model_c.parameters(), lr=lr, momentum=momentum, nesterov=True)
optim_d = optim.SGD(model_d.parameters(), lr=lr, momentum=momentum, nesterov=True)

train_models = Models(model_f, model_c, model_d)
optims = Optims(optim_f, optim_c, optim_d)
```

```
model_trainer = trainer.Trainer(train_models, optims, criterions, device,
                                tune_lr=True)
```

```
test_history = defaultdict(lambda:[])
model_trainer.train(epochs, loaders, test_history=test_history)
```

    Train Epoch: 					1 [0/60000 (0%)]	Loss: 2.312509, lr: 0.01000 lambd: 0.00000
    Train Epoch: 					1 [12800/60000 (21%)]	Loss: 0.470715, lr: 0.00865 lambd: 0.10626
    Train Epoch: 					1 [25600/60000 (43%)]	Loss: 0.401973, lr: 0.00766 lambd: 0.21015
    Train Epoch: 					1 [38400/60000 (64%)]	Loss: 0.208745, lr: 0.00690 lambd: 0.30951
    Train Epoch: 					1 [51200/60000 (85%)]	Loss: 0.331025, lr: 0.00630 lambd: 0.40253

    Target Domain Test set: Average loss: 1.2261, Accuracy: 6299/10000 (63%)
    Source Domain Test set: Average loss: 0.1081, Accuracy: 9680/10000 (97%)
    Domains predictor:  Accuracy: 14677/20000 (73%)

    Train Epoch: 					2 [0/60000 (0%)]	Loss: 0.371136, lr: 0.00595 lambd: 0.46212
    Train Epoch: 					2 [12800/60000 (21%)]	Loss: 0.307323, lr: 0.00551 lambd: 0.54178
    Train Epoch: 					2 [25600/60000 (43%)]	Loss: 0.395909, lr: 0.00514 lambd: 0.61276
    Train Epoch: 					2 [38400/60000 (64%)]	Loss: 0.271887, lr: 0.00483 lambd: 0.67507
    Train Epoch: 					2 [51200/60000 (85%)]	Loss: 0.285515, lr: 0.00455 lambd: 0.72904

    Target Domain Test set: Average loss: 1.2675, Accuracy: 7115/10000 (71%)
    Source Domain Test set: Average loss: 0.0991, Accuracy: 9695/10000 (97%)
    Domains predictor:  Accuracy: 13576/20000 (68%)

    Train Epoch: 					3 [0/60000 (0%)]	Loss: 0.158411, lr: 0.00439 lambd: 0.76159
    Train Epoch: 					3 [12800/60000 (21%)]	Loss: 0.225826, lr: 0.00417 lambd: 0.80288
    Train Epoch: 					3 [25600/60000 (43%)]	Loss: 0.284502, lr: 0.00397 lambd: 0.83768
    Train Epoch: 					3 [38400/60000 (64%)]	Loss: 0.182873, lr: 0.00379 lambd: 0.86678
    Train Epoch: 					3 [51200/60000 (85%)]	Loss: 0.272523, lr: 0.00364 lambd: 0.89098

    Target Domain Test set: Average loss: 0.9923, Accuracy: 7328/10000 (73%)
    Source Domain Test set: Average loss: 0.0817, Accuracy: 9736/10000 (97%)
    Domains predictor:  Accuracy: 12820/20000 (64%)

    Train Epoch: 					4 [0/60000 (0%)]	Loss: 0.145459, lr: 0.00354 lambd: 0.90515
    Train Epoch: 					4 [12800/60000 (21%)]	Loss: 0.103794, lr: 0.00340 lambd: 0.92267
    Train Epoch: 					4 [25600/60000 (43%)]	Loss: 0.321543, lr: 0.00328 lambd: 0.93706
    Train Epoch: 					4 [38400/60000 (64%)]	Loss: 0.179386, lr: 0.00316 lambd: 0.94884
    Train Epoch: 					4 [51200/60000 (85%)]	Loss: 0.218501, lr: 0.00306 lambd: 0.95846

    Target Domain Test set: Average loss: 1.0416, Accuracy: 7215/10000 (72%)
    Source Domain Test set: Average loss: 0.0614, Accuracy: 9807/10000 (98%)
    Domains predictor:  Accuracy: 13779/20000 (69%)

    Train Epoch: 					5 [0/60000 (0%)]	Loss: 0.169267, lr: 0.00299 lambd: 0.96403
    Train Epoch: 					5 [12800/60000 (21%)]	Loss: 0.177790, lr: 0.00290 lambd: 0.97084
    Train Epoch: 					5 [25600/60000 (43%)]	Loss: 0.220318, lr: 0.00281 lambd: 0.97637
    Train Epoch: 					5 [38400/60000 (64%)]	Loss: 0.137977, lr: 0.00273 lambd: 0.98087
    Train Epoch: 					5 [51200/60000 (85%)]	Loss: 0.232367, lr: 0.00266 lambd: 0.98452

    Target Domain Test set: Average loss: 0.9560, Accuracy: 7393/10000 (74%)
    Source Domain Test set: Average loss: 0.0687, Accuracy: 9778/10000 (98%)
    Domains predictor:  Accuracy: 14327/20000 (72%)

    Train Epoch: 					6 [0/60000 (0%)]	Loss: 0.200763, lr: 0.00261 lambd: 0.98661
    Train Epoch: 					6 [12800/60000 (21%)]	Loss: 0.173522, lr: 0.00254 lambd: 0.98917
    Train Epoch: 					6 [25600/60000 (43%)]	Loss: 0.108857, lr: 0.00248 lambd: 0.99124
    Train Epoch: 					6 [38400/60000 (64%)]	Loss: 0.126563, lr: 0.00242 lambd: 0.99292
    Train Epoch: 					6 [51200/60000 (85%)]	Loss: 0.141906, lr: 0.00236 lambd: 0.99428

    Target Domain Test set: Average loss: 0.8650, Accuracy: 7519/10000 (75%)
    Source Domain Test set: Average loss: 0.0599, Accuracy: 9809/10000 (98%)
    Domains predictor:  Accuracy: 12617/20000 (63%)

    Train Epoch: 					7 [0/60000 (0%)]	Loss: 0.182191, lr: 0.00232 lambd: 0.99505
    Train Epoch: 					7 [12800/60000 (21%)]	Loss: 0.201400, lr: 0.00227 lambd: 0.99600
    Train Epoch: 					7 [25600/60000 (43%)]	Loss: 0.115925, lr: 0.00222 lambd: 0.99677
    Train Epoch: 					7 [38400/60000 (64%)]	Loss: 0.166766, lr: 0.00218 lambd: 0.99739
    Train Epoch: 					7 [51200/60000 (85%)]	Loss: 0.189622, lr: 0.00213 lambd: 0.99789

    Target Domain Test set: Average loss: 0.8678, Accuracy: 7332/10000 (73%)
    Source Domain Test set: Average loss: 0.0636, Accuracy: 9796/10000 (98%)
    Domains predictor:  Accuracy: 13513/20000 (68%)

    Train Epoch: 					8 [0/60000 (0%)]	Loss: 0.107886, lr: 0.00210 lambd: 0.99818
    Train Epoch: 					8 [12800/60000 (21%)]	Loss: 0.134360, lr: 0.00206 lambd: 0.99853
    Train Epoch: 					8 [25600/60000 (43%)]	Loss: 0.202102, lr: 0.00202 lambd: 0.99881
    Train Epoch: 					8 [38400/60000 (64%)]	Loss: 0.237934, lr: 0.00198 lambd: 0.99904
    Train Epoch: 					8 [51200/60000 (85%)]	Loss: 0.116310, lr: 0.00195 lambd: 0.99922

    Target Domain Test set: Average loss: 0.8703, Accuracy: 7438/10000 (74%)
    Source Domain Test set: Average loss: 0.0566, Accuracy: 9810/10000 (98%)
    Domains predictor:  Accuracy: 13463/20000 (67%)

    Train Epoch: 					9 [0/60000 (0%)]	Loss: 0.117980, lr: 0.00192 lambd: 0.99933
    Train Epoch: 					9 [12800/60000 (21%)]	Loss: 0.236234, lr: 0.00189 lambd: 0.99946
    Train Epoch: 					9 [25600/60000 (43%)]	Loss: 0.235230, lr: 0.00186 lambd: 0.99956
    Train Epoch: 					9 [38400/60000 (64%)]	Loss: 0.136297, lr: 0.00183 lambd: 0.99965
    Train Epoch: 					9 [51200/60000 (85%)]	Loss: 0.181172, lr: 0.00180 lambd: 0.99971

    Target Domain Test set: Average loss: 0.7648, Accuracy: 7665/10000 (77%)
    Source Domain Test set: Average loss: 0.0642, Accuracy: 9814/10000 (98%)
    Domains predictor:  Accuracy: 13079/20000 (65%)

    Train Epoch: 					10 [0/60000 (0%)]	Loss: 0.093681, lr: 0.00178 lambd: 0.99975
    Train Epoch: 					10 [12800/60000 (21%)]	Loss: 0.141431, lr: 0.00175 lambd: 0.99980
    Train Epoch: 					10 [25600/60000 (43%)]	Loss: 0.117273, lr: 0.00172 lambd: 0.99984
    Train Epoch: 					10 [38400/60000 (64%)]	Loss: 0.232385, lr: 0.00170 lambd: 0.99987
    Train Epoch: 					10 [51200/60000 (85%)]	Loss: 0.148965, lr: 0.00167 lambd: 0.99989

    Target Domain Test set: Average loss: 0.7481, Accuracy: 7762/10000 (78%)
    Source Domain Test set: Average loss: 0.0525, Accuracy: 9838/10000 (98%)
    Domains predictor:  Accuracy: 13521/20000 (68%)

```

```


---

---

 Quantum teleportation

“A trick that quantum magicians use to produce phenomena that cannot be imitated by classical magicians.” - A. Peres

To get a feel for how Strawberry Fields works, let's walk through an example of quantum teleportation.

## Background info
Quantum teleportation is the reliable transfer of an unknown quantum state across spatially separated qubits or qumodes, through the use of a classical transmission channel and quantum entanglement. Considered a fundamental quantum information protocol, it has applications ranging from quantum communication to enabling distributed information processing in quantum computation.

In general, all quantum teleportation circuits work on the same basic principle. Two distant observers, Alice and Bob, share a maximally entangled quantum state (in discrete variables, any one of the four Bell states; or in CV, a maximally entangled state for a fixed energy), and have access to a classical communication channel. Alice, in possession of an unknown state which she wishes to transport to Bob, makes a joint measurement of the unknown state and her half of the entangled state, by projecting onto the Bell basis. By transmitting the results of her measurement to Bob, Bob is then able to transform his half of the entangled state to an accurate replica of the original unknown state, by performing a conditional phase flip (for qubits) or displacement (for qumodes).

## The teleportation circuit

$$\newcommand{ket}[1]{\left|#1\right\rangle}$$

* Here, qumodes $q_1$ and $q_2$ are initially prepared as highly squeezed vacuum states in momentum and position space respectively. We do this by highly squeezing these two qumodes in the phase space:

\begin{align}
&\ket{0}_x \approx S(2)\ket{0}\\
&\ket{0}_p \approx S(-2) \ket{0}
\end{align}

* They are then maximally entangled by a 50-50 beamsplitter. This is a beamsplitter with parameters $\theta=\pi/4$, $\phi=0$.

* These two qumodes are now spatially separated, with $\ket{q_1}$ held by Alice, and $\ket{q_2}$ held by Bob, with the two connected via the classical communication channels $c_0$ and $c_1$.

* To teleport her unknown state $\ket{\psi}$ to Bob, Alice now performs a projective measurement of her entire system onto the maximally entangled basis states. This is done by entangling $\ket{\psi}$ and $\ket{q_1}$ via another 50-50 beamsplitter, before performing two homodyne measurements, in the $x$ and $p$ quadratures respectively. The two resulting measurements are referred to as $m_1$ and $m_2$.

* The results of these measurements are then transmitted to Bob, who performs both a position displacement (conditional on the $x$ measurement) using a $X$ gate, and a momentum displacement (conditional on the $p$ measurement) using a $Z$ gate, to recover exactly the transmitted state $\ket{\psi}$. In particular, Bob has to make the following displacements:

\begin{align}
Z\left(-\sqrt{2}m_2\right)X\left(\sqrt{2}m_1\right)\ket{q_2}
\end{align}

## Importing Strawberry Fields

We import the Strawberry Fields, as well as all quantum operations, using the following commands:

```python
import strawberryfields as sf
from strawberryfields.ops import *
```

The first import statement imports Strawberry Fields as sf, allowing us to access the engine and backends. The second import statement imports all available CV gates into the global namespace.

In addition, we will also need some utilities from the `utils` subpackage, a module containing Strawberry Fields utilities and extensions. Finally, we import $\pi$ and the square root from NumPy so that we can pass angle parameters to gates such as beamsplitters, and perform some custom classical processing.

```python
from strawberryfields.utils import scale
from numpy import pi, sqrt
```

**Available CV gates**

Check out our documentation to see the available CV gates, states, measurements, and other operations available in Strawberry Fields.

## Engine initialization

We can now initialise our chosen backend, engine, and quantum register, using the
`sf.Engine()` function, which has the following syntax:

    eng, q = sf.Engine(num_subsystems)

where

* `num_subsystems` (*int*) is the number of modes we want to initialise in our quantum register

returning

* `eng` is the resulting engine; we use this engine object to manipulate our engine/circuit.
* `q` (*tuple*) contains the **qumodes** in our quantum circuit. They are accessed by indexing; for example `q[0]` for the first qumode, `q[1]` for the second qumode, etc.

**Exercise**

Initialise an engine below, using the correct number of modes for quantum teleportation as per the circuit above.

```python
eng, q = sf.Engine(3)
```

## Circuit construction

Now that we have created the engine and qumodes, we can start to construct our circuit. In Strawberry Fields, circuit construction *always* occurs within a `with eng:` context. Inside this context, operations are applied using the syntax

```python3
with eng:
    Operation(arguments) | (sequence of qumodes)
```

For example, to apply prepare a squeezed state, followed by a beamsplitter, we can write

```python3
with eng:
    Squeezed(0.5) | q[0]
    BSgate(pi/4, 0) | (q[0], q[1])
```

Some operations in Strawberry Fields apply to one mode, and some apply to two. Be sure to check the documentation to double check the operation arguments, and to see how many qumodes they act on.

**Exercise**

The cell below corresponding to the teleportation circuit is partially complete. Using the above, and referring to the docs, fill in the cell below to finish constructing the teleportation circuit.

```python
with eng:
    # prepare the initial states
    Coherent(1+0.5j) | q[0]  # This is the state we will teleport
    Sgate(-2) | q[1]
    Sgate(2) | q[2]

    # apply the gates
    BSgate(pi/4, 0) | (q[1], q[2])  # a 50-50 beamsplitter
    BSgate(pi/4, 0) | (q[0], q[1])  # a 50-50 beamsplitter

    # perform the homodyne measurements
    MeasureX | q[0]
    MeasureP | q[1]

    # displacement gates conditioned on the measurements
    Xgate(scale(q[0], sqrt(2))) | q[2]
    Zgate(scale(q[1], sqrt(2))) | q[2]
```

A couple of things to note:

* **Operations must be applied in temporal order.** Different operation orderings can result in the same quantum circuit, providing the operations do not apply sequentially to the same mode.

* **Gates are standard Python objects, and can be treated as such.** In this case, since both beamsplitters use the same parameters, a single instance can be created and stored under variable BS.

* **The results of measured modes are passed to gates simply by passing the measured mode as an argument.** In order to perform additional classical processing to the measured mode, we can use the basic classical processing functions available in `strawberryfields.utils`; here we used the `scale()` function.

## Running the engine

Once the circuit is constructed, you can run the engine via the `eng.run()` method:

```python3
    state = eng.run(backend, return_state=True, modes=None, *args, **kwargs)
```

The eng.run method accepts the arguments

* `backend`: a string representing the Strawberry Fields backend we wish to use; we have the choice of two Fock basis backends, the NumPy based (`'fock'`) and Tensorflow (`'tf'`), and one Gaussian representation backend (`'gaussian'`).

    This is required the first time running the engine, but optional for subsequent runs - if not provided, the previously used backend will continue to be used.

and returns

* `state`: an object representing the quantum state after the circuit simulation.

    Depending on backend used, the state returned might be a BaseFockState, which represents the state using the Fock/number basis, or might be a BaseGaussianState, which represents the state using Gaussian representation, as a vector of means and a covariance matrix. Many methods are provided for state manipulation, see Quantum states API for more details.

For more details on the technical differences between the backends, see the [Backend API](https://strawberryfields.readthedocs.io/en/stable/code/backend.html) in the documentation.

### How do we choose a backend?

* **Fock backends** are backends which represent the quantum state and operations via the Fock basis.
These can represent all possible continuous-variable states and operations, but also introduce numerical error due to truncation of the Fock space, and can consume more memory.

* The **gaussian backend**, due to its ability to represent states and operations as gaussian objects/transforms in the phase space, can be less computationally intensive then the Fock backends.
However, it cannot represent non-gaussian operations and states (such as the cubic phase gate, fock measurements, and fock states, amongst others).

In the Fock backends, due to the infinite Hilbert space, we must specify a **cutoff dimension**. For example, choosing `cutoff_dim=5`, a state $|\psi\rangle$ has approximation

$$|{\psi}\rangle = \sum_{n=0}^\infty c_n|{n}\rangle \approx\sum_{n=0}^{5-1} c_n|{n}\rangle$$

in our truncated Fock basis - i.e. all information of quantum amplitudes of Fock states $|n\rangle$, with $n\geq 5$, is discarded.

**Warning**

To avoid significant numerical error when working with the Fock backend, we generally need to ensure that all initial states and gates we apply result in negligible amplitude in the Fock basis for Fock states $|{n}\rangle, ~~n\geq \texttt{cutoff_dim}$.

For example, to prepare a squeezed vacuum state in the $x$ quadrature with `cutoff_dim=10`, a squeezing factor of $r=1$ provides an acceptable approximation, since $\left|\left\langle{n}\middle|{\xi}\right\rangle\right|^2<0.02$ for $n\geq 10$.

**Scaling**

As we are performing a classical simualation of a quantum system, we still need to consider how the simulation scales. Using the Fock backends, memory usage and computational time will scale like $\sim D^N$, where $N$ is the number of modes and $D$ the cutoff dimension.

Ideally, we would like to choose a cutoff large enough that minimises truncation error, but small enough to reduce classical computational resources!

**Exercise**

Run the engine using the Gaussian or the Fock backend.

```python
state = eng.run('fock', cutoff_dim=15)
```

## Results and visualization
Once the engine has been run, we can extract results of measurements and the quantum state from the circuit. Any measurements performed on a mode are stored in the mode attribute `val`:

```python
q[0].val
```

    -1.0993109931099312

```python
q[1].val
```

    -0.038300383003829097

The returned `state` object stores the resulting state of the circuit after the engine run, and also contains some convenient functions for state manipulation.

**Exercise**

Print the state object below to see some info about the state.

```python
print(state)
```

    <FockState: num_modes=3, cutoff=15, pure=False, hbar=2>

The actual state representation can also be returned; in the Fock backends, the state vector in the truncated Fock basis is available via `state.ket()`, while the density matrix is given by `state.dm()`. In the Gaussian backend, the vector of means and covariance matrix are accessible via `state.means()` and `state.cov()` respectively.

### Fock backend example
To return the density matrix representing the Fock state, we can use the method state.dm(). In this case, the density matrix has dimension

```python
state.dm().shape
```

    (15, 15, 15, 15, 15, 15)

Here, we use the convention that every pair of consecutive dimensions corresponds to a subsystem; i.e.,

$$\rho_{\underbrace{ij}_{q[0]}\underbrace{kl}_{q[1]}\underbrace{mn}_{q[2]}}$$

Thus we can calculate the reduced density matrix for mode `q[2]`, $\rho_2$:

```python
import numpy as np
rho2 = np.einsum('kkllij->ij', state.dm())
rho2.shape
```

**Note**

The Fock state also provides the method `reduced_dm()` for extracting the reduced density matrix automatically.

```python
state.reduced_dm([2])
```

    array([[  5.20017225e-01 -1.73472348e-17j,
              2.62001634e-01 -8.69150141e-02j,
              2.80055494e-01 +2.94986447e-02j,
              1.67123169e-01 -7.90550672e-02j,
              1.22974690e-01 +4.66482169e-03j,
              8.70370392e-02 -5.99165249e-02j,
              6.12599236e-02 -7.23598638e-04j,
              5.03755666e-02 -4.24049729e-02j,
              4.56986981e-02 +4.90277701e-03j,
              3.27242806e-02 -1.95025911e-02j,
              3.94015196e-02 +1.75404805e-02j,
              1.02001102e-02 +1.03391172e-02j,
              8.68933695e-03 +1.84366657e-02j,
             -1.62624603e-02 +1.79535624e-03j,
              2.02890892e-03 -1.94914994e-02j],
           [  2.62001634e-01 +8.69150141e-02j,
              1.46531831e-01 -3.90312782e-18j,
              1.36170724e-01 +6.16704962e-02j,
              9.74152648e-02 -1.18977678e-02j,
              6.11789861e-02 +2.29041217e-02j,
              5.38664697e-02 -1.56406395e-02j,
              3.09856883e-02 +9.87433272e-03j,
              3.24683660e-02 -1.29453002e-02j,
              2.22050503e-02 +1.01082009e-02j,
              1.97471977e-02 -4.35654686e-03j,
              1.69200768e-02 +1.54229856e-02j,
              3.41108128e-03 +6.91401776e-03j,
              1.29649055e-03 +1.07413180e-02j,
             -8.49363133e-03 -1.81352782e-03j,
              4.28001476e-03 -9.48134370e-03j],
           [  2.80055494e-01 -2.94986447e-02j,
              1.36170724e-01 -6.16704962e-02j,
              1.52497352e-01 +4.33680869e-18j,
              8.55197524e-02 -5.20554158e-02j,
              6.64926885e-02 -4.46365547e-03j,
              4.34749920e-02 -3.72053573e-02j,
              3.29505104e-02 -3.86474216e-03j,
              2.47243059e-02 -2.56948345e-02j,
              2.48891693e-02 +4.80752896e-05j,
              1.65173655e-02 -1.23594554e-02j,
              2.22147111e-02 +7.21133132e-03j,
              6.07977332e-03 +4.98952155e-03j,
              5.72548958e-03 +9.43616022e-03j,
             -8.65631093e-03 +1.88939879e-03j,
             -1.30105816e-05 -1.06122476e-02j],
           [  1.67123169e-01 +7.90550672e-02j,
              9.74152648e-02 +1.18977678e-02j,
              8.55197524e-02 +5.20554158e-02j,
              6.57283175e-02 +1.82145965e-17j,
              3.88124491e-02 +2.01942775e-02j,
              3.70807154e-02 -6.02426302e-03j,
              1.97977225e-02 +9.08042475e-03j,
              2.26362739e-02 -5.96982079e-03j,
              1.39413111e-02 +8.52295091e-03j,
              1.34817921e-02 -1.29286990e-03j,
              9.99628993e-03 +1.16271350e-02j,
              1.70631874e-03 +4.87344709e-03j,
             -1.02348873e-05 +7.24616404e-03j,
             -5.49936768e-03 -1.89529159e-03j,
              3.61522540e-03 -5.95573660e-03j],
           [  1.22974690e-01 -4.66482169e-03j,
              6.11789861e-02 -2.29041217e-02j,
              6.64926885e-02 +4.46365547e-03j,
              3.88124491e-02 -2.01942775e-02j,
              2.91231408e-02 -8.67361738e-18j,
              2.00452071e-02 -1.49499439e-02j,
              1.44803754e-02 -7.20651010e-04j,
              1.15325180e-02 -1.04799056e-02j,
              1.08508979e-02 +7.49477487e-04j,
              7.56375358e-03 -4.90556446e-03j,
              9.47509547e-03 +3.79455523e-03j,
              2.50488919e-03 +2.35351442e-03j,
              2.22025775e-03 +4.28199092e-03j,
             -3.82967314e-03 +5.70451977e-04j,
              3.04951581e-04 -4.62758823e-03j],
           [  8.70370392e-02 +5.99165249e-02j,
              5.38664697e-02 +1.56406395e-02j,
              4.34749920e-02 +3.72053573e-02j,
              3.70807154e-02 +6.02426302e-03j,
              2.00452071e-02 +1.49499439e-02j,
              2.14712813e-02 -1.42572586e-17j,
              1.03366535e-02 +6.93727377e-03j,
              1.33174411e-02 -1.29317715e-03j,
              7.08384615e-03 +6.08601067e-03j,
              7.72426717e-03 +5.06285910e-04j,
              4.57374650e-03 +7.47566316e-03j,
              5.15951020e-04 +2.90575242e-03j,
             -6.69913922e-04 +4.08699475e-03j,
             -2.92876434e-03 -1.57327023e-03j,
              2.58540115e-03 -3.02858665e-03j],
           [  6.12599236e-02 +7.23598638e-04j,
              3.09856883e-02 -9.87433272e-03j,
              3.29505104e-02 +3.86474216e-03j,
              1.97977225e-02 -9.08042475e-03j,
              1.44803754e-02 +7.20651010e-04j,
              1.03366535e-02 -6.93727377e-03j,
              7.21764906e-03 -3.68628739e-18j,
              5.99343135e-03 -4.92536320e-03j,
              5.37665096e-03 +6.41154263e-04j,
              3.88217713e-03 -2.25194078e-03j,
              4.61723515e-03 +2.12115931e-03j,
              1.18722337e-03 +1.23217902e-03j,
              9.97981117e-04 +2.18399751e-03j,
             -1.91827529e-03 +1.88870459e-04j,
              2.66135083e-04 -2.29334643e-03j],
           [  5.03755666e-02 +4.24049729e-02j,
              3.24683660e-02 +1.29453002e-02j,
              2.47243059e-02 +2.56948345e-02j,
              2.26362739e-02 +5.96982079e-03j,
              1.15325180e-02 +1.04799056e-02j,
              1.33174411e-02 +1.29317715e-03j,
              5.99343135e-03 +4.92536320e-03j,
              8.33795348e-03 -4.01154804e-18j,
              4.02716599e-03 +4.20146125e-03j,
              4.76044043e-03 +7.79239875e-04j,
              2.38659455e-03 +4.91220653e-03j,
              1.45007401e-04 +1.83335136e-03j,
             -6.61662767e-04 +2.49458771e-03j,
             -1.72179429e-03 -1.15220626e-03j,
              1.78598688e-03 -1.72274966e-03j],
           [  4.56986981e-02 -4.90277701e-03j,
              2.22050503e-02 -1.01082009e-02j,
              2.48891693e-02 -4.80752896e-05j,
              1.39413111e-02 -8.52295091e-03j,
              1.08508979e-02 -7.49477487e-04j,
              7.08384615e-03 -6.08601067e-03j,
              5.37665096e-03 -6.41154263e-04j,
              4.02716599e-03 -4.20146125e-03j,
              4.06218896e-03 +1.08420217e-18j,
              2.69191115e-03 -2.02240007e-03j,
              3.62794754e-03 +1.16996174e-03j,
              9.93855816e-04 +8.12425648e-04j,
              9.37434806e-04 +1.53827546e-03j,
             -1.41220521e-03 +3.11098655e-04j,
             -5.46900937e-06 -1.73202615e-03j],
           [  3.27242806e-02 +1.95025911e-02j,
              1.97471977e-02 +4.35654686e-03j,
              1.65173655e-02 +1.23594554e-02j,
              1.34817921e-02 +1.29286990e-03j,
              7.56375358e-03 +4.90556446e-03j,
              7.72426717e-03 -5.06285910e-04j,
              3.88217713e-03 +2.25194078e-03j,
              4.76044043e-03 -7.79239875e-04j,
              2.69191115e-03 +2.02240007e-03j,
              2.79073371e-03 +3.02831219e-17j,
              1.82167344e-03 +2.58151320e-03j,
              2.54129455e-04 +1.03317491e-03j,
             -1.44630691e-04 +1.48608771e-03j,
             -1.09071659e-03 -4.96922717e-04j,
              8.58681799e-04 -1.15049327e-03j],
           [  3.94015196e-02 -1.75404805e-02j,
              1.69200768e-02 -1.54229856e-02j,
              2.22147111e-02 -7.21133132e-03j,
              9.99628993e-03 -1.16271350e-02j,
              9.47509547e-03 -3.79455523e-03j,
              4.57374650e-03 -7.47566316e-03j,
              4.61723515e-03 -2.12115931e-03j,
              2.38659455e-03 -4.91220653e-03j,
              3.62794754e-03 -1.16996174e-03j,
              1.82167344e-03 -2.58151320e-03j,
              3.57708958e-03 +2.88397778e-17j,
              1.12160309e-03 +4.39335630e-04j,
              1.28026731e-03 +1.10384324e-03j,
             -1.17164242e-03 +6.84575654e-04j,
             -5.03729795e-04 -1.54530022e-03j],
           [  1.02001102e-02 -1.03391172e-02j,
              3.41108128e-03 -6.91401776e-03j,
              6.07977332e-03 -4.98952155e-03j,
              1.70631874e-03 -4.87344709e-03j,
              2.50488919e-03 -2.35351442e-03j,
              5.15951020e-04 -2.90575242e-03j,
              1.18722337e-03 -1.23217902e-03j,
              1.45007401e-04 -1.83335136e-03j,
              9.93855816e-04 -8.12425648e-04j,
              2.54129455e-04 -1.03317491e-03j,
              1.12160309e-03 -4.39335630e-04j,
              4.05639627e-04 +3.08997619e-18j,
              5.37003447e-04 +1.88870568e-04j,
             -2.83291557e-04 +3.58550268e-04j,
             -3.47738101e-04 -4.22664017e-04j],
           [  8.68933695e-03 -1.84366657e-02j,
              1.29649055e-03 -1.07413180e-02j,
              5.72548958e-03 -9.43616022e-03j,
             -1.02348873e-05 -7.24616404e-03j,
              2.22025775e-03 -4.28199092e-03j,
             -6.69913922e-04 -4.08699475e-03j,
              9.97981117e-04 -2.18399751e-03j,
             -6.61662767e-04 -2.49458771e-03j,
              9.37434806e-04 -1.53827546e-03j,
             -1.44630691e-04 -1.48608771e-03j,
              1.28026731e-03 -1.10384324e-03j,
              5.37003447e-04 -1.88870568e-04j,
              7.98848958e-04 -8.78203760e-18j,
             -2.08088520e-04 +6.06568369e-04j,
             -6.57148203e-04 -3.97630139e-04j],
           [ -1.62624603e-02 -1.79535624e-03j,
             -8.49363133e-03 +1.81352782e-03j,
             -8.65631093e-03 -1.88939879e-03j,
             -5.49936768e-03 +1.89529159e-03j,
             -3.82967314e-03 -5.70451977e-04j,
             -2.92876434e-03 +1.57327023e-03j,
             -1.91827529e-03 -1.88870459e-04j,
             -1.72179429e-03 +1.15220626e-03j,
             -1.41220521e-03 -3.11098655e-04j,
             -1.09071659e-03 +4.96922717e-04j,
             -1.17164242e-03 -6.84575654e-04j,
             -2.83291557e-04 -3.58550268e-04j,
             -2.08088520e-04 -6.06568369e-04j,
              5.14773180e-04 -1.31188463e-17j,
             -1.30744199e-04 +6.02551428e-04j],
           [  2.02890892e-03 +1.94914994e-02j,
              4.28001476e-03 +9.48134370e-03j,
             -1.30105816e-05 +1.06122476e-02j,
              3.61522540e-03 +5.95573660e-03j,
              3.04951581e-04 +4.62758823e-03j,
              2.58540115e-03 +3.02858665e-03j,
              2.66135083e-04 +2.29334643e-03j,
              1.78598688e-03 +1.72274966e-03j,
             -5.46900937e-06 +1.73202615e-03j,
              8.58681799e-04 +1.15049327e-03j,
             -5.03729795e-04 +1.54530022e-03j,
             -3.47738101e-04 +4.22664017e-04j,
             -6.57148203e-04 +3.97630139e-04j,
             -1.30744199e-04 -6.02551428e-04j,
              7.38504422e-04 +1.40946282e-18j]])

The diagonal values of the reduced density matrix contain the marginal Fock state probabilities $\left|\left\langle{i}\mid{\rho_2}\right\rangle\right|^2,~~ 0\leq i< cutoff$:

```python
np.real_if_close(np.diag(state.reduced_dm([2])))
```

    array([  3.30124234e-01,   3.66661916e-01,   2.02037477e-01,
             6.96202751e-02,   1.39445869e-02,   1.68472510e-03,
             3.31591938e-03,   5.00648663e-03,   1.49447948e-03,
             1.08845772e-04,   1.21982477e-03,   5.89777516e-04,
             1.06575755e-03,   1.65544901e-04,   5.62370962e-05])

This can also be returned via the state method `all_fock_probs`.

### Plotting the Wigner function

**Exercise**

Using the `state.wigner` function, plot the wigner function of the input state and the resulting teleported state.

```python
import matplotlib.pyplot as plt
```

```python
x = np.arange(-5, 5, 0.1)
p = np.arange(-5, 5, 0.1)
```

```python
W = state.wigner(2, x, p)
```

```python
W.shape
```

    (100, 100)

```python
X, P = np.meshgrid(x, p)
plt.contourf(X, P, W)
```

**Exercise**

Repeat this notebook, but using the `'gaussian'` backend instead. How does the teleportation fidelity compare to the fock backend?

```python

```