# Initialization

```python
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

import numpy as np
import scqubits as scq
```

# Transmon qubit

The transmon qubit and the Cooper pair box are described by the Hamiltonian
\begin{equation}
H_\text{CPB}=4E_\text{C}(\hat{n}-n_g)^2+\frac{1}{2}E_\text{J}\sum_n(|n\rangle\langle n+1|+\text{h.c.}),
\end{equation}
expressed in the charge basis. Here,  $E_C$ is the charging energy, $E_J$ the Josephson energy, and $n_g$ the offset charge. Internal representation of the Hamiltonian proceeds via the charge basis with charge-number cutoff specified by `ncut`, which must be chosen sufficiently large for convergence.

An instance of the transmon qubit is initialized as follows:

```python
tmon = scq.Transmon(
    EJ=30.02,
    EC=1.2,
    ng=0.3,
    ncut=31
)
```

Or, alternatively, we can use the graphical user interface (if the `ipywidgets` package is installed)

```python
tmon = scq.Transmon.create()
```

    HBox(children=(VBox(children=(HBox(children=(Label(value='EJ'), FloatText(value=30.0)), layout=Layout(justify_…

    Output()

## Calculating and plotting energy levels and eigenfunctions

The energy eigenvalues of the transmon Hamiltonian for the given set of model parameters are obtained by calling the `eigenvals()` method. The optional parameter `evals_count` specifies the sought number of eigenenergies.

```python
tmon.eigenvals(evals_count=12)
```

    array([-21.82674091,  -6.15959855,   7.94124586,  20.84080224,
            28.09898516,  45.79505843,  46.25965954,  78.30745776,
            78.31056405, 120.95203447, 120.95204064, 173.4572327 ])

To plot eigenenergies as a function of one of the qubit parameters (`EJ`, `EC`, or `ng`), we generate an array of values for the desired parameter and call the method `plot_evals_vs_paramvals`:

```python
ng_list = np.linspace(-2, 2, 220)
tmon.plot_evals_vs_paramvals('ng', ng_list, evals_count=6, subtract_ground=False);
```

    HBox(children=(FloatProgress(value=0.0, description='Spectral data', max=220.0, style=ProgressStyle(descriptio…

```python
tmon.plot_n_wavefunction(esys=None, which=0, mode='real');
```

```python
tmon.plot_wavefunction(esys=None, which=0, mode='real');
```

```python
tmon.plot_phi_wavefunction(which=[0, 1, 4, 5], mode='abs_sqr');
```

## Calculating and visualizing matrix elements

Matrix elements can be calculated by referring to the `Transmon` operator methods in string form. For instance, `.n_operator` yields the charge operator:

```python
tmon.matrixelement_table('n_operator', evals_count=3)
```

    array([[ 0.2999574 , -0.90286728, -0.00126536],
           [-0.90286728,  0.30195231, -1.21191324],
           [-0.00126536, -1.21191324,  0.26241099]])

Visualizing matrix elements is accomplished by calling the `.plot_matrixelements` method:

```python
tmon.plot_matrixelements('n_operator', evals_count=10);
```

```python
tmon.plot_matrixelements('cos_phi_operator', evals_count=10, show3d=False, show_numbers=True);
```

```python
fig, ax = tmon.plot_matelem_vs_paramvals('n_operator', 'ng', ng_list, select_elems=4, filename='./data/test');
```

    HBox(children=(FloatProgress(value=0.0, description='Spectral data', max=220.0, style=ProgressStyle(descriptio…

    HBox(children=(FloatProgress(value=0.0, max=220.0), HTML(value='')))

## Tunable Transmon

An important modification of the transmon qubit is the replacement of the Josephson junction by a SQUID loop of two Josephson junctions. A flux threaded through this loop can then be used to change the effective Josephson energy of the circuit and thus make the transmon tunable. The resulting Hamiltonian is
\begin{equation}
H_\text{CPB}=4E_\text{C}(\hat{n}-n_g)^2+\frac{1}{2}E_\text{J,eff}(\Phi_\text{ext})\sum_n(|n\rangle\langle n+1|+\text{h.c.}),
\end{equation}
expressed in the charge basis. Here,  parameters are as above except for the effective Josephson energy $E_\text{J,eff}(\Phi_\text{ext}) = E_{\text{J,max}} \sqrt{\cos^2(\pi\Phi_\text{ext}/\Phi_0)+ d^2 \sin^2 (\pi\Phi_\text{ext}/\Phi_0)}$, where $E_\text{J,max} = E_\text{J1} + E_\text{J2}$ is the maximum Josephson energy, and $d=(E_\text{J1}-E_\text{J2})/(E_\text{J1}+E_\text{J2})$ is the relative junction asymmetry.

An instance of a tunable transmon qubit is obtained like this:

```python
tune_tmon = scq.TunableTransmon(
    EJmax=50.0,
    EC=0.5,
    d=0.01,
    flux=0.0,
    ng=0.0,
    ncut=30
)
```

Alternatively, the `.create` method can again be used if `ipywidgets` is available:

```python
tune_tmon = scq.TunableTransmon.create()
```

    HBox(children=(VBox(children=(HBox(children=(Label(value='EJmax'), FloatText(value=30.0)), layout=Layout(justi…

    Output()

```python
flux_list = np.linspace(-1.1, 1.1, 220)
tune_tmon.plot_evals_vs_paramvals('flux', flux_list, subtract_ground=True);
```

    HBox(children=(FloatProgress(value=0.0, description='Spectral data', max=220.0, style=ProgressStyle(descriptio…

```python

```


# Home Assignment -- 1

In this part of the homework you are asked implement some "interesting" functions
usinig Numpy. You are advised to **avoid using python loops**.

At the same time you are **encouraged** to google anything and read relevant
documentation, e.g. for [Numpy](https://docs.scipy.org/doc/numpy/reference/)
[matlpotlib](https://matplotlib.org/) and [python](https://docs.python.org/3/).

Please, write your solutions within the designated blocks:
```python
...
### BEGIN Solution

# >>> your solution here <<<

### END Solution
...
```

# Part 1 (28 + 7 pt.): Numpy and (some) Matplotlib

Let's import numpy, fix the randomness, and then import matplotlib.

```python
import numpy as np
np.random.seed(42)

%matplotlib inline
import matplotlib.pyplot as plt
```

## Task 1 (1 pt.)

Write a function, that returns an $n \times m$ integer matrix with
a specified value on the border and another specified value inside.

You might want to study numpy's indexing and [array creation](https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html).

```python
def make_bordered_matrix(n_rows, n_cols, fill_value=1, border_value=0):
    """Constructs a bordered (n x m) matrix.

    Parameters
    ----------
    n_rows : int
        Nonnegative integer, spcifying the number of rows.
    n_cols : int
        Nonnegative integer, spcifying the number of columns.
    fill_value : float
        The value to fill the matrix with.
    border_value : float
        The value to set the border of the matrix to.

    Returns
    -------
    arr : array, 2 dimensional, shape = (n_rows, n_cols)
        The final bordered matrix.
    """
    ### BEGIN Solution
    out = np.ones( (n_rows,n_cols) )*fill_value
    out[:,0]=border_value
    out[:,n_cols-1]=border_value
    out[0,1:-1]=border_value
    out[n_rows-1,1:-1]=border_value
    ### END Solution

    return out

```

## Task 2 (1 pt.)

Implement a routine, which creates a binary checkerboard matrix with the specified
shape.

Example 4x4:
```python
[[0, 1, 0, 1],
 [1, 0, 1, 0],
 [0, 1, 0, 1],
 [1, 0, 1, 0]]
```

```python
def make_binary_checkerboard_matrix(n_rows, n_cols):
    """Construct an n x m matrix with checkerboard pattern of 0 and 1.

    Parameters
    ----------
    n_rows : int
        Nonnegative integer, spcifying the number of rows.
    n_cols : int
        Nonnegative integer, spcifying the number of columns.

    Returns
    -------
    arr : array, 2 dimensional, shape = (n_rows, n_cols)
        The final checkerboard matrix.
    """
    ### BEGIN Solution
    out = np.zeros( (n_rows, n_cols) )
    ii, jj = np.indices( (n_rows, n_cols) )
    out[(ii-jj) % 2 == 1] = 1
    ### END Solution

    return out

```

## Task 3 (1 pt.)

Write a function that shifts and scales a given array `arr` so that its values
lie within $[0, 1]$.

```python
def normalize(arr):
    """Shift and scale the array to fit it within [0, 1].

    Parameters
    ----------
    arr : array, multidimensional
        The array to be normalized.

    Returns
    -------
    out : array
        The normalized array.
    """
    ### BEGIN Solution
    minVal = np.min(arr)
    out = (arr - minVal)/(np.max(arr)-minVal)
    ### END Solution

    return out

```

## Task 4 (1 pt.)

Write a function, which takes a matrix `arr` and centers each of its rows
by the mean within that row. Check [this](https://docs.scipy.org/doc/numpy/reference/routines.statistics.html)
out for documentation.

```python
def center(arr):
    """Center each row of a matrix by the mean value in that row.

    Parameters
    ----------
    arr : array, shape = (n_rows, n_cols)
        The matrix, the rows of which to center.

    Returns
    ----------
    out : array, shape = (n_rows, n_cols)
        The final row-centered matrix.
    """
    assert arr.ndim == 2
    n_rows, n_cols = arr.shape

    ### BEGIN Solution
    out= arr -np.mean(arr, axis=1)[:, np.newaxis]
    ### END Solution

    return out

```

## Task 5 (2 pt.)

Implement the following fancy function:
\begin{equation}
    f(x) = \sigma\bigl(
                \max\{x + 5, 0\} + \max\{5 - x, 0\}
                + \max\{\min\{\cos(2 x \pi), \tfrac12\}, -\tfrac14\}
            \bigr)
         \,,
\end{equation}
where $\sigma(x) = (1+e^{-x})^{-1}$ is the sigmoid function.

```python
def sigmoid(x):
    return 1/(1+np.exp(-x))

def fancy_function(x):
    """Compute some fancy function.

    Parameters
    ----------
    X : array, 1 dimendional, shape=(n_samples,)
        The array argument values.

    Returns
    -------
    y : array, 1 dimendional, shape=(n_samples,)
        The values of the fancy function.
    """
    ### BEGIN Solution
    out = sigmoid(np.maximum(x+5,0)+np.maximum(5-x,0)+np.maximum(np.minimum(np.cos(2*np.pi*x),0.5),-0.25) )
    ### END Solution

    return out
```

Plot the function

```python
x = np.linspace(-12.5, 12.5, num=1001)
fig, ax = plt.subplots(figsize=(5, 5))
ax.plot(x, fancy_function(x))
plt.show()
```

## Task 6 (2 pt.)

Write a function that sorts the rows of a given matrix by the values of a
specified column in **descending order**.

```python
def sort_by_column_desc(arr, col):
    """Sort the rows of a matrix in descending order of values in a specified column.

    Parameters
    ----------
    arr : array, 2 dimensional
        The array to sort.

    col : int
        The index of the column to sort the rows by.

    Returns
    ----------
    out : array, 2 dimensional
        The sorted array.

    """
    ### BEGIN Solution
    print(arr[:,col])
    out = arr[np.argsort(arr[:,col])[::-1],:]
    ### END Solution

    return out

```

## Task 7 (4 pt.)

Write a function that takes two real valued matrices $X$ and $Y$ with different
number of rows and equal number of columns and outputs the nearest neighbours'
indices. Specifically, each row $i$ in the output matrix indexes the rows of $Y$
in order **from the closest to the farthest** from the $i$-th row of $X$.

**Remeber** to avoid explicit loops!

You might like to have a look at *np.linalg.norm* and *np.argsort*.
Example:
```python
>>> X = [[0, 1], [3, 1]]
>>> Y = [[1, 1], [2, 2], [-20, -20]]
>>> index_of_nearest(X, Y, norm=2.0)
array([[0, 1, 2],
       [1, 0, 2]])
```

```python
def nearest_neighbours(X, Y, norm=2.0):
    """Index the rows of `Y` in increasing order of distance from each row in `X`.

    Parameters
    ----------
    X : array, 2 dimensional, shape = (n_samples, n_dim)
        The matrix, for the rows of which to find the closest row in `Y`.

    Y : array, 2 dimensional, shape = (n_reference, n_dim)
        The matrix, among the rows of which to seek the closest one for `X`.

    norm : float, positive
        The order of the l^p norm to use for computing the pairwise distances.

    Returns
    -------
    index : 2 dimensional
        The matrix of indices of the rows of `Y` for each vector in `X`
        in increasing order of distnace with respect to the l^p norm.

    """
    assert X.ndim == Y.ndim and X.ndim == 2

    ### BEGIN Solution
    out = X-Y.reshape((Y.shape[0],Y.shape[1],-1))
    out=(np.argsort(np.linalg.norm(out,axis=2,ord=norm),axis=0)).T
    ### END Solution

    return out

#X=np.array([[0,1],[3,1]])
#Y=np.array([[1,1],[2,2],[-20,-20]])
#nearest_neighbours(X, Y, norm=np.inf) #some tests
```

## Task 8 (4+1 pt.)

Plot the level sets of the $l^p$ norm
\begin{equation}
    \|z\|_p = \biggl(\sum_i \lvert x_i\rvert^p\biggr)^\tfrac1{p}
        \,.
\end{equation}
and make the contour of the unit ball in $l^p$ norm stand out. Draw plots
for $p \in \{0, \tfrac1{25}, \tfrac12, 1, 1.5, 2, 7, \infty\}$.

Study plotting examples on [this](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.contourf.html#matplotlib.pyplot.contourf) and [this](http://scikit-learn.org/stable/auto_examples/svm/plot_oneclass.html) pages (**especially the last one**) and have a look at these functions: *np.meshgrid*, *np.linspace* in numpy's documentation. You
will get extra points if you use *np.linalg.norm* (which is more numerically stable).

Try to produce a plot that **looks like the one below**:

```python

p_values = [0., 0.04, 0.5, 1, 1.5, 2, 7, np.inf]
xx, yy = np.meshgrid(np.linspace(-3, 3, num=101),
                     np.linspace(-3, 3, num=101))

fig, axes = plt.subplots(ncols=(len(p_values) + 1)// 2,
                         nrows=2, figsize=(14, 7))

for p, ax in zip(p_values, axes.flat):
    ### BEGIN Solution
    Z=np.zeros((2,xx.shape[0],xx.shape[1]))
    Z[0,:,:]=xx
    Z[1,:,:]=yy
    Z=np.linalg.norm(Z,axis=0,ord=p)

    ax.contourf(xx,yy, Z)
    contLine = ax.contour(xx,yy, Z, np.array([1]),colors="red")

    if(not p==np.inf):
        contLine.collections[0].set_label(r"$\lbrace x:  \Vert x \Vert_{"+str(p)+r"} = 1\rbrace$")
    else:
        contLine.collections[0].set_label(r"$\lbrace x:  \Vert x \Vert_\infty = 1\rbrace$")
    ax.legend()
    ### END Solution

plt.show()

```

## Task 9 (4 pt.)

Write a function, that accepts an array of real values, a set of splitting points
and returns the sample mean of all values that end up within **each bin**, defined
by two consecutive splitting points.

Example for $\mathtt{splits} = [-1, 0, 1]$ the bins are
\begin{equation}
    \mathtt{bins} = \bigl[[-\infty, -1), [-1, 0), [0, +1), [+1, +\infty)\bigr]\,.
\end{equation}

Look up functions *np.digitize* and *np.bincount*, and **pay close attention** to what they
return and how to correctly use them.

```python
def means_in_bins(bin_splits, arr):
    """Compute the sample mean of the values within bins.

    Parametrs
    ---------
    bin_splits : array, 1 dimension, shape=(n_splits,)
        1d array of values in ascending order.

    arr : arraylike, 1 dimension
        1d array of values to be put in bins.

    Returns
    -------
    means : arrat, 1 dimensional, shape = (n_splits + 1,)
        The sample mean within each bin defined by `bin_splits`.

    Details
    -------
    The flat array `bin_splits` of size `n_splits` defines the following bins:
    (-inf, B[0]), ..., [B[i-1], B[i]), ..., [B[n_splits], +inf), where B denotes
    `bin_splits`. The vlaues in `arr` are assigned to each bin and then averages
    within each bin are computed.
    """
    bins = np.r_[-np.inf, bin_splits, +np.inf]

    ### BEGIN Solution
    arrDigitized = np.digitize(arr,bins)

    print((arrDigitized == np.arange(1,len(bins))[:,np.newaxis])*2)
    out = np.sum( np.multiply(np.ones((len(bins)-1,len(arr)))*arr,(arrDigitized == np.arange(1,len(bins))[:,np.newaxis])*1) ,axis=1) * 1/(np.bincount(arrDigitized)[1:])
    ### END Solution

    return out

#means_in_bins(np.array([-1,0,1]),np.array([-2,-1,-0.5,-0.2,-0.2,0.2,0.2,0.4,5.4,2]))
#some tests
```

## Task 10 (2 pt.)

Write a function that takes a multidimentional real valued array with
**missing values** (represented by *NaN*) and returns its copy with the
missing values filled with the sample mean of all nonmissing values. If
the input array **contains nothing but missing values**, then fill it
with zero.

```python
def fill_missing(arr):
    """Fill the missing values in an array with mean of nonmissing values.

    Parameters
    ---------
    arr : arraylike, multidimensional
        The real-valued array with missing values to be filled.

    Returns
    ----------
    out : arraylike, multidimensional
        A copy of the input array with filled missing values.
    """

    ### BEGIN Solution
    out=0 #init
    nans = np.isnan(arr)
    if(np.prod(nans)):
        out= np.zeros(arr.shape)
    else:
        out = np.copy(arr) # out is not the same as arr
        out[nans]=np.mean(out[~nans])
    ### END Solution

    return out

#tests
#print(fill_missing(np.array([[1,2,np.nan],[0,np.nan,np.nan],[3,1,np.nan]])))
#print(fill_missing(np.array([[np.nan,np.nan,np.nan],[np.nan,np.nan,np.nan],[np.nan,np.nan,np.nan]])))

```

## Task 11 (2 pt.)

[pandas](https://pandas.pydata.org/pandas-docs/stable/) is great, no doubt about it,
but in some cases you might need a bit simpler solution. In this task you are asked
to load some text data with **missing** values using [Numpy's API](https://docs.scipy.org/doc/numpy/reference/routines.io.html) and apply `fill_missing`
to it. You will have to **tweak some parameters** the data loader call.

P.S.: You might want to look up what **io.StringIO** does in Python.
P.P.S: Generally pandas's data loaders are by far more versatile than numpy's,
take, for example, [pd.read_html](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_html.html), [pd.read_excel](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_excel.html), [pd.read_json](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_json.html) and
others.

```python
from io import StringIO
from io import BytesIO #for some reason genfromtxt returned ERROR: CAn't convert bytes to str when using StringIO

data = b"""version 8.0.0815
    Some irrelevant garbage before the data block
    @first line of data block is a comment
    1.||65.21|4|5.87||||9|5.4||
    @this is another pertinent comment
    |3.1|4|||8.7|45|1||9|-3.654|15
    """

### BEGIN Solution
stream = BytesIO(data)

print(fill_missing(np.genfromtxt(stream,delimiter="|",comments="@",filling_values=np.nan,skip_header=2)))

### END Solution
```

    [[  1.          12.33042857  65.21         4.           5.87        12.33042857
       12.33042857  12.33042857   9.           5.4         12.33042857
       12.33042857]
     [ 12.33042857   3.1          4.          12.33042857  12.33042857   8.7
       45.           1.          12.33042857   9.          -3.654       15.        ]]

## Task 12 (4 pt.)

Write a function that solves the following optimization problem for any
$X \in \mathbb{R}^{N\times d}$, $y\in \mathbb{R}^{N}$ and $C > 0$:
\begin{equation}
    \begin{aligned}
        & \underset{\beta, \beta_0}{\text{minimize}}
          & & \tfrac12 \|X \beta + \mathbf{1}\beta_0 - y\|^2 + \tfrac{C}2\|\beta\|^2
              \,,
    \end{aligned}
\end{equation}
where $\|\cdot\|^2$ is the squared $l^2$ norm, and $\mathbf{1}$ is a column vector of ones.
You will have to derive the explicit solution and implement it using numpy.

### Solution

1. Write out the FOC for $\beta_0$ and solve it for $\beta_0$.

Using chain rule one can obtain $$
\nabla_\beta F = X^T(X\beta+\mathbf{1}\beta_0-y)+C\beta
$$
and $$\partial_{\beta_0} F = \sum_{i=1}^N(x_{i\cdot}\beta+\beta_0-y_i)=N\beta_0+\mathbf{1}^T(X\beta -y).$$

From first-order condition,
$$
\beta_0= \frac{1}{N}\mathbf{1}^T(y-X\beta)
$$

2. Plug $\beta_0$ into the objective and write out the new optimization problem. Observe how $X$ and $y$ are augmented.
$$
F= \tfrac12 \|X \beta + \frac{1}{N}\mathbf{1}\mathbf{1}^T(y-X\beta) - y\|^2 + \tfrac{C}2\|\beta\|^2.
$$

3. Get the FOC for $\beta$ for the new problem and solve it.

First-order condition implies
$\nabla F = (X-\frac{1}{N}\mathbf{1}\mathbf{1}^TX)^T(X\beta+\frac{1}{N}\mathbf{1}\mathbf{1}^T(y-X\beta)-y)+C\beta =0$, so in order to find $\beta$ we need to solve the corresponding linear system
$$
((X-\frac{1}{N}\mathbf{1}\mathbf{1}^TX)^T(X-\frac{1}{N}\mathbf{1}\mathbf{1}^TX) +CI)\beta = (X-\frac{1}{N}\mathbf{1}\mathbf{1}^TX)^T(y- \frac{1}{N}\mathbf{1}\mathbf{1}^Ty),
$$
where $I$ is an identity matrix of valid size.

4. Recover $\beta_0$ from the known solution for $\beta$.

It can be done using

$$
\beta_0= \frac{1}{N}\mathbf{1}^T(y-X\beta).
$$

Implement the function.

```python
def ridge_leastsq(X, y, alpha=1.0):
    """Find the Ridge regression solution for the given dataset (X, y).

    Parameters
    ----------
    X : array, 2 dimensional, shape = (n_samples, n_features)
        The input feature matrix.

    y : array, 1 dimensional, shape = (n_samples,)
        The target values for the regression task.

    alpha : float
        The value of the ridge regularization coefficient.

    Returns
    -------
    intercept : float
        The optimal intercept term.
    beta : array, shape = (n_features,)
        The optimal ridge regression coefficients.
    """
    assert X.ndim == 2

    n_samples, n_features = X.shape
    assert y.ndim == 1 and len(y) == n_samples

    ### BEGIN Solution
    X_cent = X - np.ones(X.shape)* np.mean(X, axis=0)
    print(X_cent.T.dot((y-np.mean(y)*np.ones(y.shape)).reshape((y.shape[0],1))))
    #Note: of course, we should never invert the matrix, however np.linalg.solve performs regularization by default
    #in order to avoid it, I write np.linalg.inv instead of np.linalg.solve
    beta = np.linalg.inv(X_cent.T.dot(X_cent)+ alpha*np.eye(X.shape[1])).dot(X_cent.T.dot((y-np.mean(y)*np.ones(y.shape)).reshape(y.shape[0],1)))
    beta_0= np.mean( y - X.dot(beta))
    ### END Solution

    return beta_0, beta

```

```python
ts = np.linspace(0,3,50)
y = np.sin(ts)+np.cos(5*ts) + np.random.uniform(-0.2,0.2,50)
print(y.shape)
f, ax = plt.subplots(figsize=(12,4))

ax.scatter(ts,y)
```

```python
X = np.array([np.power(ts,k) for k in np.arange(0,10)]).T

print()
beta_0, beta = ridge_leastsq(X, y, alpha=1e-7)

f, ax = plt.subplots(figsize=(12,4))

ax.scatter(ts,y)
ax.plot(ts, X.dot(beta)+beta_0)

```

## Task 13 (6 pt., optional)

In this exercise you are asked to remember and use the
[Singular Value Decomposition](https://en.wikipedia.org/wiki/Singular-value_decomposition)
to solve the previous task not for just one value of $C$,
but for an arbitrary flat array of positive regularization
coefficients.

**Hint** Carefully study the capabilities of [np.einsum](https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html)
and the parameters of [np.svd](https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.svd.html)
Then figure out a way to use just one call to SVD instead of multiple
matrix inversions.

**Remember**: avoid using python loops!

```python
def ridge_path(X, y, alpha=(1.0,)):
    """Find the Ridge regression solution for the given dataset (X, y).

    Parameters
    ----------
    X : array, 2 dimensional, shape = (n_samples, n_features)
        The input feature matrix.

    y : array, 1 dimensional, shape = (n_samples,)
        The target values for the regression task.

    alpha : arraylike, 1 dimensional, shape = (n_coefficients,)
        The value of the ridge regularization coefficient.

    Returns
    -------
    intercept : array, 1 dimensional, shape = (n_coefficients,)
        The optimal intercept term for each value in `C`.
    beta : array, shape = (n_features, n_coefficients)
        The optimal ridge regression coefficients for each value in `C`.
    """
    assert X.ndim == 2

    n_samples, n_features = X.shape
    assert y.ndim == 1 and len(y) == n_samples

    alpha = np.asarray(alpha, dtype=float).ravel()
    assert np.all(alpha > 0)

    ### BEGIN Solution

    ### END Solution

    return beta_0, beta
```


# pill 5 - A gentle introduction to supervised learning

**DEPENDENCIES:** `ipywidgets` dependency.

## 1. Disectioning machine learning methods

In order to be able to learn, any algorithm has to define at least three components:

+ **The model class/hypothesis space** defines the family of mathematical models that will be used. The target decision boundary will be approximated from one element of this space. For example, we can consider the class of linear models. In this case our decision boundary will be a line if the problem is defined in ${\bf R}^2$ and the model class is the space of all posible lines in ${\bf R}^2$.

    Model classes define the geometric properties of the decision function. There are different taxonomies but the most well-known are the *families* of **linear** and **non-linear** models. These families usually depend on some parameters. And the solution to a learning problem is the selection of a particular set of parameters, i.e. the selection of an instance model from the model class space. The model class space is also called **hypothesis space**.

    The selection of the best model will depend on our problem and what we want to obtain from the problem. The primary goal in learning is usually achieving the minimum error/maximum performance. But according to what else we want from the algorithm we will find different algorithms. Other common desirable properties are interpretability, behavior in front of missing data, fast training, etc.

+ **The problem model** formalizes and encodes the desired properties of the solution. In many cases this formalization takes the form of an optimization problem. In it most basic instantiation, the problem model can be the **minimization of an error function**. The error function measures the difference between our model and the target one. Informally speaking, in a classification problem it measures how "irritated" we are when our model misses the right label of a training sample. For example, in classification the ideal error function is the **0-1 loss**. This function takes value $1$ when we incorrectly classify a training sample and zero otherwise. In this case, it can be intrepreted that one is only irritated by "one unit of irritation" when one sample is misclassified.

    Problem model can also be used to impose other constraints on our solution, such as finding a smooth approximation, small complexity model, sparse solution, etc.

+ **The learning algorithm** is an optimization/search method or algorithm that given a model class fits it to the training data according to the error function. According to the nature of our problem there are many different algorithms. In general, we are talking about finding the minimum error approximation or maximum probable model. In those cases, if the problem is convex/quasi-convex we will typically use first or second order methods (i.e. gradient descent, coordinate descent, Newton's method, Interior Point methods, etc). Other searching techniques such as genetic algorithms or monte-carlo techniques can be used if we do not have access to the derivatives of the objective function.

```python
%matplotlib inline
#from sklearn import
import numpy as np
#Create some data
X = np.concatenate([1.25*np.random.randn(40,2),5+1.5*np.random.randn(40,2)])
y = np.concatenate([np.ones((40,1)),-np.ones((40,1))])

#Visualize
import matplotlib.pyplot as plt
plt.scatter(X[0:40,0],X[0:40,1],color='r')
plt.scatter(X[40:,0],X[40:,1],color='b')
```

```python
#from sklearn import
import numpy as np
#Create some data
X = np.concatenate([1.25*np.random.randn(40,2),5+1.5*np.random.randn(40,2)])
y = np.concatenate([np.ones((40,1)),-np.ones((40,1))])

#Visualize
import matplotlib.pyplot as plt
from ipywidgets import interact

def human_learning_algorithm(X,y):

    plt.scatter(X[0:40,0],X[0:40,1],color='r')
    plt.scatter(X[40:,0],X[40:,1],color='b')
    #plt.scatter(X[0:40,0],X[0:40,1],color='r')
    #plt.scatter(X[40:,0],X[40:,1],color='b')
    delta = 0.025
    xx = np.arange(-5.0, 10.0, delta)
    yy = np.arange(-5.0, 10.0, delta)
    XX, YY = np.meshgrid(xx, yy)
    Xf = XX.flatten()
    Yf = YY.flatten()
    sz=XX.shape
    data = np.concatenate([Xf[:,np.newaxis],Yf[:,np.newaxis]],axis=1);

    def hml_display(w0,w1,offset):
        w=np.array([w0,w1])
        w.shape=(2,1)
        #Evaluate the model for a given weight
        Z = data.dot(w)+offset
        Z.shape=sz
        plt.scatter(X[0:40,0],X[0:40,1],color='r')
        plt.scatter(X[40:,0],X[40:,1],color='b')
        plt.imshow(Z, interpolation='bilinear', origin='lower', extent=(-5,10,-5,10),alpha=0.3, vmin=-15, vmax=15)
        plt.contour(XX,YY,Z,[0])
        fig = plt.gcf()
        fig.set_size_inches(9,9)

    #Ipython 2.0
    interact(hml_display, w0=(-10.,10.), w1=(-10.,10.), offset=(-20.,40.));
    #Ipython 1.1 back compatibility
    #w0=-10. #-10.,10.
    #w1=-10. #-10.,10.
    #offset=-20. #-20.,40.
    #hml_display(w0,w1,offset)

human_learning_algorithm(X,y)

```

**QUESTION:** Describe the process you used for fitting the classifier

**Some notes on the learning process**

The main goal of any learning process is to achive the maximum predictive power (*accuracy*). This is minimize the error. However, there are three other important properties we usually desire our models to have:

 **Simplicity** - how much fiddling do we need for the method to work? Can I modify it to handle the particularities of my problem?

 **Speed** - How long does it take to train a reliable model? (training time) Can I use it in embedded and real time applications? (testing time), How long do I have to wait for processing my 1YB (yottabyte - 1e24 Bytes) dataset?

 **Interpretability** - Why did it make this predictions?

It happens that accuracy trades off with all the rest of the desirable properties.

In general, we can represent the problem of supervised machine learning as in the following scheme:

## 2. First steps into a model: Linear regression

Consider a two dimensional problem such as the following.

```python
import numpy as np
np.random.seed(2)
x = 4+np.random.normal(size=(20,1))
y = 0.5 + 1.2*x + 0.5*np.random.normal(size=(20,1))

%matplotlib inline
import matplotlib.pyplot as plt
plt.scatter(x,y)
```

In every problem we have to decide the **model class**, the **cost function**, and the **learning/searching algorithm**.

In this problem we could decide to use as **model class** the set of *linear models*, i.e. $$\mathcal{H}(w_0,w_1) = \hat{y} = w_1 x + w_0,$$
where $w_1$ and $w_0$ are the parameters of the model.

Our **cost function** could be the *mean least squares* function, i.e. $$\mathcal{L}(y_i,\hat{y}_i) = \frac{1}{N}\sum_{i=0}^N (y_i - \hat{y}_i)^2$$, where $\hat{y}$ is the predicted value.

Using these two elements we can model the problem of learning as:

$$
\begin{align}
\underset{w_0,w_1}{\text{minimize}}&& \frac{1}{N}\sum_{i=0}^N (y_i - (w_1x_i + w_0))^2.
\end{align}
$$

Our last component is the way of solving this problem. Fortunately for us we know about *optimization* and *numerical linear algebra*. Thus we can proceed in different directions.

Let us start with using a very simple and intuitive idea:

Start with a random point, ${\bf w}^0$.

Change the parameters in such a way that we reduce the cost function

$$\mathcal{L}({\bf w}^{t})\leq \mathcal{L}({\bf w}^{t-1})$$

Repeat until we can not reduce the cost function

Let us refine the process

Start with a random point, ${\bf w}^0$.

Select a descent direction $\Delta {\bf w}$ and a step size $\eta$ (this last can be selected by line search).

Update the parameters:

$$ {\bf w}^{t} = {\bf w}^{t-1} + \eta \Delta {\bf w}$$

Repeat 2-3 until the stoping criterion is met (This can be a certain number of iterations or a desired error tolerance).

The most well known technique in this kind of techniques is **Steepest descent** or **Gradient descent**, where $\Delta {\bf w} = -\nabla_{\bf w} \mathcal{L}$.

In order to solve the problem we have to compute the gradient of the loss function. We can easily derive the expression considering matrix calculus. Let us express the linear model as the inner product of the parameters and the data point. For the sake of simplicity consider an extended data example as

$$\tilde{\bf x} = \left(\begin{matrix} 1\\ {\bf x} \end{matrix}\right)$$

thus

$$\hat{y} = \sum_{i=1}^d w_i x^{(i)} + w_0 = \sum_{i=0}^d w_i \tilde{x}^{(i)} = \tilde{\bf x}^T {\bf w}.$$

Moreover, given a data set with $N$ data points, where each one is stored as a column vector, we can define the design matrix as

$${\bf X} = (x_1, \dots x_N).$$

then, the output of our model for the whole data set is a prediction vector defined as

$$\hat{\bf y} = \tilde{\bf X}^T {\bf w}.$$

The loss function can also be expressed as a matrix product,

$$\mathcal{L}(y_i,\hat{y}_i) = \frac{1}{N}\sum_{i=0}^N (y_i - \hat{y}_i)^2 = \frac{1}{N} ({\bf y}-\hat{\bf y})^T({\bf y}-\hat{\bf y})$$

Thus our problem can be rewriten as

$$
\begin{align}
\underset{{\bf w}}{\text{minimize}}&& \frac{1}{N}({\bf y}-\tilde{\bf X}^T {\bf w})^T({\bf y}-\tilde{\bf X}^T {\bf w}).
\end{align}
$$

We can differentiate this expression with respect the vector ${\bf w}$, considering the following rule

$$\partial (A B) = \partial(A) B= + (A \partial (B))^T.$$

then

$$\nabla_{\bf w} \mathcal{L} = - \frac{2}{N} \tilde{\bf X}({\bf y}-\tilde{\bf X}^T {\bf w}).$$

 **EXERCISE/QUIZ:** Code a basic gradient descent for the problem. What is the value optained for $\eta=0.01$ and 50 iterations if we initialize the weights to $(0,0)$? (Write down your solution, it will be asked for in the quiz later).

```python
#your code here
def mean_least_squares(y,y_hat, N):
    return np.sum((y-y_hat)**2)/N
def lin_reg(learning_rate = 0.01):
    n_iter = 50
    N = x.shape[0]
    x_augmented = np.c_[np.ones((x.shape[0],1)),x].T
    w = np.zeros((2,1))
    objective=np.zeros((n_iter,1))

    for i in range(n_iter):
        y_hat = x_augmented.T@w
        L = mean_least_squares(y,y_hat, N)
        dL_dw = -(2/N)*(x_augmented@(y-(x_augmented.T@w)))
        dw = -dL_dw
        w += dw*learning_rate
        objective[i] = L
    return w, objective
w, objective = lin_reg()
w

```

    array([[0.28500563],
           [1.23328861]])

```python
# Run this code to compute the value for the quiz

assert w.shape==(2,1), 'The shape of the answer must be (2,1)'

w_opt = np.array([[0.5],[1.2]])
d = np.sqrt(np.dot((w_opt-w).T,(w_opt-w)))
print('The distance between w* and the obtained w is: '+ str(d))
```

    The distance between w* and the obtained w is: [[0.21755622]]

```python
import matplotlib.pyplot as plt
plt.plot(objective)
print(objective[-1])
```

 **EXERCISE/QUIZ:** Let us speed up the algorithm using a larger step. Solve the problem using the same code and $\eta=0.1$  and 100 iterations (starting point still the zero vector). What is the resulting value of the optimal parameters?

```python
#Your code
w, objective = lin_reg(learning_rate = 0.1)
w, objective[-1]
```

    (array([[-3.35020533e+17],
            [-1.36801971e+18]]), array([6.20225911e+36]))

```python
# Run this code to compute the value for the quiz

assert w.shape==(2,1), 'The shape of the answer must be (2,1)'

w_opt = np.array([[0.5],[1.2]])
d = np.sqrt(np.dot((w_opt-w).T,(w_opt-w)))
print('The distance between w* and the obtained w is: '+ str(d))
```

    The distance between w* and the obtained w is: [[1.40844478e+18]]

```python
import matplotlib.pyplot as plt
plt.plot(objective)
```

 **EXERCISE/QUIZ:** Repeat the former exercise but compute the value of the objective function at each iteration and plot it. This is called "convergence plot".

## Closed form solution for  least squares linear regression

Fortunately for us, this problem has a close form solution. As you may recall from optimization the condition for a point being an extremum is

$$\nabla_{\bf w} \mathcal{L} = 0$$

In this problem, we can write

$$ - \frac{2}{N} \tilde{\bf X}({\bf y}-\tilde{\bf X}^T {\bf w}) = 0 $$

which, with simple manipulation gives us the following answer:

$${\bf w} = (\tilde{\bf X}\tilde{\bf X}^T)^{-1}\tilde{\bf X}{\bf y}$$

 **EXERCISE/QUIZ:** Repeat the former exercise using the analytic solution and write down the value for the quiz.

```python
#your code here

import numpy as np

x_augmented = np.c_[np.ones((x.shape[0],1)),x].T

#USE STANDARD LINEAR ALGEBRA HERE (np.linalg.pinv for the pseudo inverse)
w = (np.linalg.pinv(x_augmented@x_augmented.T)@x_augmented)@y
w
```

    array([[0.06909552],
           [1.28616382]])

```python
# Run this code to compute the value for the quiz

assert w.shape==(2,1), 'The shape of the answer must be (2,1)'

w_opt = np.array([[0.5],[1.2]])
d = np.sqrt(np.dot((w_opt-w).T,(w_opt-w)))
print('The distance between w* and the obtained w is: '+ str(d))
```

    The distance between w* and the obtained w is: [[0.43943472]]

## A comparison between both solvers

Iterative methods	Analytic solution

Need to choose the learning rate $\eta$.

May be slow.

Each iteration is usually cheap.

Can be used with large data sizes.

Looks nicer

"No iterations are required"

Large computational cost. In the simplest case where we need matrix inversion, a naive method would need $O(n^3)$

Pill5_A gentle introduction to supervised machine learning/pill5 - A gentle introduction to supervised machine learning_student.ipynb


# Tutrial 5: Fitting 1D/2D Cross-Correlation Statistc

## Parameters fit within error to probablity function for complete spatial randomness

### Introduction

Complete spatial Randomness descripes a spatial point process where points measured are distributed by a random mechanism or process. This is the basis for measuring whether or not a seen distribution or density of points in a given spatial range was a direct result of a homogenous spatial poisson process. This test allows statasticians to determine whether a the average distance between two particles in a distribution is significantly short or long.

In this tutorial we will fit with python package pyplot in matplotlib the following PCF statistics derived in tutorial 4 with the probability function for Complete Spatial Randomness which is given by the following equation(s).

### 1D version

$$\begin{align} P_{r} = \frac{1}{\beta!}*\lambda^N*r^\beta*e^{-\lambda*r} \end{align}$$

### 2D Version

$$\begin{align} P_{(x,y)} = \frac{1}{\beta!}*\lambda^N*(\sqrt{x^2+y^2})^\beta*e^{-\lambda*\sqrt{x^2+y^2}} \end{align}$$

### The fitpcf() function can be called from the pygempick.modeling package which is defined below:

```python

def fitpcf(data, N, p0, p1):

    data = pd.DataFrame(data)
    data = data.fillna(0)

    #determine guess filtering parameters
    pcfp1 = np.array([N,p0,p1])
    x = data[2].values
    y = data[0].values
    dy = data[1].values

    popt1, pcov1 = opt.curve_fit(spa.pcf , x, y,  p0 = pcfp1)
    popt1 = np.around(popt1, decimals=2)

    plt.figure()
    plt.title('Probability of Gold Particle Colocolization on TTR micrographs' )
    #CSR of CD1 Micgrgrap set
    plt.plot(x,y,'xr') #keypoints of CD1 micrographs
    plt.plot(np.arange(0,210,1), spa.pcf(np.arange(0,210,1), popt1[0], popt1[1], popt1[2]),
                       'g-', label='V30M CSR, N = {} +/- {}, L = {} +/- {}'.format(popt1[0],
                                                 np.around(np.sqrt(pcov1[0,0]), decimals=3),
                                                 popt1[1], np.around(np.sqrt(pcov1[1,1]), decimals=3)))
    plt.errorbar(x, y, yerr=dy, fmt='og')
    plt.legend(loc='best')
    plt.grid(True)
    plt.xlabel('Radius (r)')
    #Probability Nth point at distance r
    plt.ylabel('P(r)')

    plt.show()

    return popt1, np.around(np.sqrt(pcov1), decimals=3)

```

```python
import pylab as plt
import pygempick.modeling as mod
import pygempick.spatialstats as spa
import numpy as np
import pandas as pd

```

```python
data1 = pd.read_csv('/home/joseph/Documents/pygempick/supdocs/pcf-dr5-error3.csv',\
                    header=None, skiprows=1)
```

```python
print('Here is the fitted 1D pair-wise probability distribution of immunogold particles on ATTR micrographs.')
popt, pcov = mod.fitpcf(data1, 10., 1., 1.)

print('Here is the 2D representation of the fitted funtion...')
plt.figure()
x = np.arange(-41,41,.25)
y = np.arange(-41,41,.25)

X,Y = plt.meshgrid(x, y) # grid of point

Z = spa.pcf2d(X,Y, popt[0], popt[1], popt[2])

im = plt.imshow(Z,cmap=plt.cm.coolwarm, extent=(-41, 41, 41, -41)) # drawing the function

#plt.clabel(cset, inline=True, fmt='%1.1f', fontsize=10)
plt.colorbar(im) # adding the colobar on the right
plt.title(r'$P_r= \frac{1}{\beta!}*\lambda^{N}*(\sqrt{x^2 + y^2})^\beta*e^{-\lambda*\sqrt{x^2 + y^2}}$')

plt.show()
```

```python

```


# Lab 1: Floating Point Numbers, Optimization & Gradient Descent

**Tomas Beuzen, January 2021**

In this lab, we'll work on solidifying concepts learned in Lecture 1 (floating point numbers) and Lecture 2 (optimization & gradient descent). This is the "vegetables" lab of the course - you're probably eager to get to neural networks, but we need to practice these fundamentals first! This lab contains quite a few pretty difficult optional exercises for those looking to go the extra mile - only do them if you have time! Good luck!

## Table of Contents

Instructions

Imports

Exercise 1: Floating Point Numbers

Exercise 2: logpdf

Exercise 3: Gradient Descent Learning Rate

Exercise 4: Logistic Regression from Scratch

(Optional) Exercise 5: log(1 + exp(z))

(Optional) Exercise 6: softmax logistic regression and log-sum-exp

Submit to Canvas and GitHub

## Instructions

rubric={mechanics:3}

**Link to your GitHub repository:**

You will receive marks for correctly submitting this assignment. To submit this assignment you should:

1. Push your assignment to your GitHub repository!
2. Provide a link to your repository in the space provided above.
2. Upload a HTML render of your assignment to Canvas. The last cell of this notebook will help you do that.
3. Be sure to follow the [General Lab Instructions](https://ubc-mds.github.io/resources_pages/general_lab_instructions/). You can view a description of the different rubrics used for grading in MDS [here](https://github.com/UBC-MDS/public/tree/master/rubric).

Here's a break down of the required and optional exercises in this lab:

|         | Number of Exercises | Points |
|:-------:|:-------------------:|:------:|
| Required| 20 | 40 |
| Optional| 4  | 4 |

## Imports

```python
import numpy as np
import pandas as pd
import scipy.stats
from scipy.optimize import minimize
import plotly.graph_objects as go
from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix, log_loss
from utils.floating_point import *
from canvasutils.submit import submit, convert_notebook
```

## Exercise 1: Floating Point Numbers

In this exercise you're going to explore floating point behaviour. You'll be provided with a code snippet which you should run and then you need to explain the result. Below are three example snippets and answers:

**Sample Code Snippet 1**

```python
0.3 - 0.2 - 0.1
```

**Sample Answer**: The result is not zero because 0.3, 0.2, and 0.1 are not represented exactly as floating point numbers.

**Sample Code Snippet 2**

```python
0.5 - 0.25 - 0.125 - 0.125
```

**Sample Answer**: The result is zero because 0.5 ($2^{-1}$), 0.25 ($2^{-2}$), and 0.125 ($2^{-3}$) are powers of 2 and there they _are_ represented exactly as floating point numbers. There is no roundoff error present.

**Sample Code Snippet 3**

```python
0.4 - 0.2 - 0.2
```

**Sample Answer**: The result is correct (zero) despite the fact that 0.4 and 0.2 are not represented exactly as floating point numbers. This is a case of good luck: while 0.4 and 0.2 are not represented exactly, the roundoff errors happened to cancel out during the subtractions.

**Hint:** You can determine if a number can be exactly represented using a function I made called `float_rep()` from `utils.floating_point` that we imported at the top of the notebook. For example:

```python
float_rep(0.25)
```

```python
float_rep(0.2)
```

```python
float_rep(0.4)
```

### 1.1
rubric={reasoning:1}

```python
30 - 20 - 10
```

Your solution goes here.

### 1.2
rubric={reasoning:1}

```python
30.0 - 20.0 - 10.0
```

Your solution goes here.

### 1.3
rubric={reasoning:1}

```python
(10.0 ** 100 + 1) == 10.0 ** 100
```

Your solution goes here.

### 1.4
rubric={reasoning:1}

```python
(10.0 ** 100000 + 1) - 10.0 ** 100000
```

Your solution goes here.

### 1.5
rubric={reasoning:1}

```python
np.exp(1000) - np.exp(1000)
```

Your solution goes here.

### 1.6
rubric={reasoning:1}

```python
1 / np.exp(1000) == np.exp(-1000)
```

Your solution goes here.

### 1.7
rubric={reasoning:1}

```python
np.exp(1000) == np.exp(10000)
```

Your solution goes here.

### 1.8
rubric={reasoning:1}

```python
x = np.ones(100000)
x[0] = 1e20

y = np.ones(100000)
y[-1] = 1e20

sum(x) == sum(y)
```

Your solution goes here.

## Exercise 2: logpdf

### 2.1
rubric={reasoning:2}

In most scientific computing libraries there are functions to compute the probability density function (pdf) of a probability distribution, and separate functions to compute the log of the pdf. For example, looking at the [R normal distribution documentation](https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Normal.html), you will see that you can set `log=TRUE` or `log=FALSE`. Likewise in Python we have `scipy.stats.norm.pdf` and `scipy.stats.norm.logpdf`.

You might wonder why we bother creating these extra functions, when we can just take the log ourselves. For example, why do we need this:

```python
scipy.stats.norm.logpdf(1)  # is this function useless?
```

When we can do this:

```python
np.log(scipy.stats.norm.pdf(1))  # seems to do the same thing as above?
```

In the space below, tell me why do we need these "extra" functions for computing the log of a pdf?

>Hint: try using a larger number in the functions above and see what happens and if you can explain the result...

Your solution goes here.

### (Optional) 2.2
rubric={accuracy:1}

Write a numerically safe function `log_gaussian_pdf` that computes the log of the standard Gaussian PDF (zero mean, unit variance). Your function should produce the same result as `scipy.stats.norm.logpdf` when evaluated at $x=100$.

> Hint: try working out the log of the [Gaussian PDF](https://en.wikipedia.org/wiki/Normal_distribution) by hand and coding that up directly. Sometimes re-arranging a formula directly can help avoid overflow errors! (Yay for math + CS + programming)!

```python
def log_gaussian_pdf(x):
    pass  # Your code here

assert log_gaussian_pdf(100) == scipy.stats.norm.logpdf(100), "Result not the same as scipy.stats.norm.logpdf"
print("Passed!")
```

Your solution goes here.

## Exercise 3: Gradient Descent Learning Rate

Below are the same functions we saw in Lecture 2 for calculating the MSE and gradient of the MSE for a linear regression model $\hat{y}=\boldsymbol{w^T}\boldsymbol{x}$, and a function for doing gradient descent. Note that the gradient descent function returns both the final optimized `w` and a list of the loss at each iteration (you'll need this "loss history" for a leter question).

```python
def mse(w, X, y):
    """Mean squared error.

    Parameters
    ----------
    w : ndarray
        Weight vector.
    X : ndarray
        Feature data.
    y : ndarray
        Response data.

    Returns
    -------
    ndarray
        Mean squared error.
    """
    return np.mean((X @ w - y) ** 2)

def mse_grad(w, X, y):
    """Gradient of mean squared error.

    Parameters
    ----------
    w : ndarray
        Weight vector.
    X : ndarray
        Feature data.
    y : ndarray
        Response data.

    Returns
    -------
    ndarray
        Gradient of MSE loss w.r.t parameters w.
    """
    return X.T @ (X @ w) - X.T @ y

def gradient_descent(f, f_grad, w, X, y, alpha, ϵ=2e-4, max_iterations=500, verbose=False):
    """Gradient descent.

    Parameters
    ----------
    f : ndarray
        Function to minimize.
    f_grad : ndarray
        Gradient of function w.r.t weights.
    w : ndarray
        Weight vector.
    X : ndarray
        Feature data.
    y : ndarray
        Response data.
    alpha : float
        Learning rate.
    ϵ : float
        Stop GD if step-size smaller than this.
    max_iterations : int
        Maximum iterations of GD to do.
    verbose : bool
        Whether to print progress.

    Returns
    -------
    ndarray
        Mean squared error.
    """

    iterations = 1         # init. iterations
    dw = np.array(2 * ϵ)   # init. dw
    losses = [f(w, X, y)]  # init. loss history
    while abs(dw).sum() > ϵ and iterations <= max_iterations:
        g = f_grad(w, X, y)  # calculate current gradient
        dw = alpha * g       # change in w
        w = w - dw           # adjust w based on gradient * learning rate
        iterations += 1      # increase iteration
        losses.append(f(w, X, y))
        if verbose and iterations % 50 == 0: print(f"Loss: {f(w, X, y):.4f}")
    if verbose: print(f"Terminated after {iterations - 1} iterations!")
    return w, losses
```

Let's create a simple regression dataset to test our gradient descent function. Our model is going to be a simple linear regression model, $\hat{y_i}=wx_i$, with just one parameter to fit, $w$, and no intercept.

```python
X, y = make_regression(100, 1, random_state=2020)
```

Let's fit our simple linear model $\hat{y_i}=wx_i$:

```python
np.random.seed(123)  # fix the random seed for reproducibility
w0 = np.random.randint(0, 20, 1)  # random intial weight
w, losses = gradient_descent(mse, mse_grad, w0, X, y, alpha=0.001, verbose=True)
print(f"Initial w = {w0[0]:.2f}")
print(f"Optimum w = {w[0]:.2f}")
```

To confirm everything is working as expected, let's compare our result to scikit learn's `LinearRegression`:

```python
lr = LinearRegression(fit_intercept=False).fit(X, y)
print(f"w = {lr.coef_[0]:.2f}")
```

Looks good!

> You could of course fit a more complex model if you wanted to. For example, if you wanted to fit a model with 3 parameters, we would just re-create our regression dataset to have 3 features and off we go:

```python
X, y = make_regression(100, 3, random_state=2020)
w0 = np.random.randint(0, 20, 3)  # random intial weight
w, losses = gradient_descent(mse, mse_grad, w0, X, y, alpha=0.001, verbose=True)
for n, _ in enumerate(w):
    print(f"w{n} = {_:.2f}")
lr = LinearRegression(fit_intercept=False).fit(X, y)
print("sklearn coefs:", lr.coef_)
```

### 3.1
rubric={viz:4}

As we saw in Lecture 2, in the case of linear regression, the MSE loss is [convex](https://en.wikipedia.org/wiki/Convex_function), that basically means that there is a "global minimum" that gradient descent will always be able to find (if we do enough iterations and the learning rate is not too high that the gradients explode). This is what the "loss surface" looks like - i.e., the loss function for different values of $w$:

>If the below doesn't run, you may need the plotly extension. Run this from your base environment: `jupyter labextension install jupyterlab-plotly@4.14.2`.

```python
# Don't worry too much about this plotting code, it's just for illustration purposes
weights = np.arange(0, 41, 1)
loss = [mse(np.array([_]), X, y) for _ in weights]
w, losses = gradient_descent(mse, mse_grad, w0, X, y, alpha=0.001, verbose=False)
fig = go.Figure()
fig.add_trace(go.Scatter(x=weights, y=loss, name="Loss"))
fig.add_trace(go.Scatter(x=w0, y=losses[:1], mode="markers", marker=dict(size=14, line=dict(width=1, color="DarkSlateGrey")), name="Initial w"))
fig.add_trace(go.Scatter(x=w, y=losses[-1:], mode="markers", marker=dict(size=14, line=dict(width=1, color="DarkSlateGrey")), name="Optimal w"))
fig.update_layout(width=600, height=400, margin=dict(t=30)); fig.update_xaxes(title="w", dtick=2); fig.update_yaxes(title="MSE loss")
```

See how it's convex? Remember with gradient descent we are iteratively taking steps "down hill" so it makes sense that with each iteration of gradient descent (i.e., each "step" down hill), our loss *should* go down and move us towards that "global minimum" (assuming our learning rate is not too high).

Using the gradient descent code and data we defined at the beginning of the exercise, I want you to show that loss decreases with iterations by making a plot of `loss` vs `iterations` for the following five learning rates: 0.00001, 0.00005, 0.0001, 0.001, 0.022. Your plot should look something like the below and have 5 lines on it (**you don't have to use plotly though, use any plotting library you like!**):

**Hint:** I've coded up our function above so that it returns the losses. For example, to get the losses per iteration for `α=0.00001`, run: `w, losses = gradient_descent(mse, mse_grad, w0, X, y, alpha=0.00001)` and you can then plot `losses`. It doesn't matter what `w0` you use for this exercise.

```python
# Your solution here.
```

### 3.2
rubric={reasoning:2}

If all went well, you probably saw in the plot above that as learning rate increases, the number of iterations to convergence typically decreases - makes sense, a higher learning rate means we are taking bigger steps towards the minimum. **However** the final learning rate of 0.022 actually converged slower than a rate of 0.001. Just in case your plot didn't show this, we can confirm by running our code again:

```python
gradient_descent(mse, mse_grad, w0, X, y, alpha=0.001, verbose=True);
```

```python
gradient_descent(mse, mse_grad, w0, X, y, alpha=0.022, verbose=True);
```

Why does a learning rate of 0.022 converge slower here?

Your solution goes here.

### 3.3
rubric={reasoning:2}

Based on the data and simple linear regression model we are using in this exercise, answer the following:

1. Does the number of iterations it takes gradient descent to converge depend on the initial weight (`w0`) we specify?
2. Does gradient descent always converge to the same optimum `w` regardless of the initial weight (`w0`) value?

Your solution goes here.

### 3.4
rubric={reasoning:2}

In the next few lectures we'll get to neural networks which are typically non-linear models. This makes them harder to optimize, especially with gradient descent! Non-linearities in our model lead to loss surfaces that are no longer convex. They are bumpy and have what we call "local minima" which we can get stuck in (there are also things called "saddle points" where we can get stuck but we'll talk about those in a later lecture). Gradient descent is particularly prone to getting stuck in local minimum. We'll explore what that means in this exercise.

Imagine we now want to fit a non-linear model, $\hat{y_i} = wcos(wx_i)$, to our dataset (rather than our previous linear model $\hat{y_i}=wx_i$). I've coded up that function (`nonlin`), the mse loss function (`mse_nonlin`), and the gradient of the loss function  (`mse_nonlin_grad`) below, along with a plotting function (I'm omitting docstrings now to be concise). Run the cell.

```python
X, y = make_regression(100, 1, random_state=2020)
X = X.flatten()

def nonlin(w, x):
    return w * np.cos(w * x)

def mse_nonlinear(w, x, y):
    """Mean squared error."""
    return np.mean((nonlin(w, x) - y) ** 2)

def mse_nonlinear_grad(w, x, y):
    """Gradient of mean squared error."""
    t = np.cos(w * x) - w * x * np.sin(w * x)
    return np.mean((nonlin(w, x) - y) * t)

def plot_loss_vs_weight(w0, alpha=0.01):
    """Plotting function using plotly - don't worry too much about the code here!"""
    if abs(w0) > 10: raise ValueError("Please specify w0 between -10 and 10")
    wf = gradient_descent(mse_nonlinear, mse_nonlinear_grad, w0, X, y, alpha=alpha, ϵ=2e-5, verbose=False)[0];
    weights = np.arange(-10, 10.1, 0.1)
    loss = [mse_nonlinear(_, X, y) for _ in weights]
    fig = go.Figure()
    fig.add_trace(go.Scatter(x=weights, y=loss, name="Loss"))
    fig.add_trace(go.Scatter(x=[w0], y=[mse_nonlinear(w0, X, y)], mode="markers", marker=dict(size=14, line=dict(width=1, color="DarkSlateGrey")), name="Initial W"))
    fig.add_trace(go.Scatter(x=[wf], y=[mse_nonlinear(wf, X, y)], mode="markers", marker=dict(size=14, line=dict(width=1, color="DarkSlateGrey")), name="Final W"))
    fig.update_layout(width=600, height=500)
    fig.update_xaxes(title="w", dtick=2)
    fig.update_yaxes(title="loss")
    return fig
```

Let's go ahead and plot this loss surface for different weights. Our plotting function also takes in an initial weight, `w0` and uses gradient descent to find the optimal weight and plots those too. Run the cell below to see what I mean:

```python
w0 = 9
plot_loss_vs_weight(w0, alpha=0.01)
```

See how our non-linear model results in this non-linear loss surface? It's very different to our nice convex one from earlier! Play around with the inital weight `w0` and answer the following:

1. Does the number of iterations it takes gradient descent to converge depend on the initial weight (`w0`)?
2. Does gradient descent always converge to the same optimum `w` regardless of the initial weight (`w0`) value?

Your solution goes here.

### 3.5
rubric={reasoning:1}

In practice, you don't know the loss surface in advance (I've been able to plot it here because our model only has a single parameter and our dataset is very simple). In fact, when we get to neural networks, we'll be dealing with tens, hundreds, thousands, or even millions of parameters resulting in an incredibly complex loss surface that is impossible to visualize (imagine that plot above but in hundreds of dimensions 🤯 ). Without knowing what the loss surface looks like you won't know whether you've optimized your model based on the global minimum or if you're stuck in a local minimum. What is one method you could use to try and avoid getting stuck in a local minima?

Your solution goes here.

## Exercise 4: Logistic Regression from Scratch

In this exercise, I'm going to get you to implement your very own Logistic Regression model from scratch - just like sklearn's `LogisticRegression` class. This is going to be so cool - I hope after this exercise you have a much better understanding of how sklearn models work, you could even start building your own ML library, the world is your oyster!

To train and test our model, we're going to try predicting the presence of [Pulsar Stars](https://en.wikipedia.org/wiki/Pulsar), using a dataset available on the UCI Machine Learning Repository [here](https://archive.ics.uci.edu/ml/datasets/HTRU2). Understanding the dataset is not that important to the question but I'll briefly explain it below.

Pulsar stars are a rare type of (dead) star that produce regular pulses of radiation in two symmetrical beams, like a lighthouse, as shown in the gif below. They can rotate up to hunderds of times per second!

Source: [www.ligo.org](https://www.ligo.org/science/Publication-S6VSR24KnownPulsar/Images/lightnew.gif)

Pulsars are hard to find, yet they are extremely useful for a variety of astrophysical purposes. As pulsars spin, their radiation beams sweep across the sky and can be detected from Earth (if they cross our line of sight), producing a detectable pattern of radiation in our measurment devices. Scientists typically average this signal over many rotations of the pulsar and then describe the pattern using various statical measures like the mean, standard deviation, skewness, etc. These are the features we have access to in this exercise. We have a training set of 4 features and 9273 observations, and we wish to classify which observations are pulsar stars (i.e., we have a simple binary classification problem). You can read more about pulsar stars and this dataset [here](https://archive.ics.uci.edu/ml/datasets/HTRU2).

The code below reads in the data:

```python
df = pd.read_csv("data/pulsar_data.csv")
df.head()
```

### 4.1
rubric={accuracy:1}

First, use sklearn's `train_test_split()` to split data into training and testing sets using the `test_size` and `random_state` given below.

```python
test_size = 0.2
random_state = 2021

# Your solution goes here
```

### 4.2
rubric={accuracy:2}

Now, standardise the feature data in the training and test sets using sklearn's `StandardScaler()`. As we saw in Lecture 2, standardising often helps with optimization.

```python
# Your solution goes here
```

### 4.3
rubric={accuracy:6}

Okay, time to build our very own logistic regression class! The cell below defines the logistic loss function and its gradient which we saw previously in [Lecture 2](https://pages.github.ubc.ca/MDS-2020-21/DSCI_572_sup-learn-2_students/lectures/lecture2_gradient-descent.html#lecture-exercise-optimizing-logistic-regression) (you can also find their derivations in [Appendix B](https://pages.github.ubc.ca/MDS-2020-21/DSCI_572_sup-learn-2_students/lectures/appendixB_logistic-loss.html)). Run the cell - you'll need these functions.

```python
def sigmoid(w, X):
    """Sigmoid function."""
    return 1 / (1 + np.exp(-X @ w))

def logistic_loss(w, X, y):
    """Logistic loss."""
    return -(y * np.log(sigmoid(w, X)) + (1 - y) * np.log(1 - sigmoid(w, X))).mean()

def logistic_loss_grad(w, X, y):
    """Gradient of logistic loss."""
    return (X.T @ (sigmoid(w, X) - y) / len(X))
```

Now, complete the code below to create the class `MyLogisticRegression` - this is our custom logistic regression class with its very own `fit`, `predict`, `predict_proba`, and `score` methods. Here is what you should do:

1. Use `scipy.optimize.minimize` in the `fit` method to solve the optimization problem (we did this in [Lecture 2 here](https://pages.github.ubc.ca/MDS-2020-21/DSCI_572_sup-learn-2_students/lectures/lecture2_gradient-descent.html#lecture-exercise-optimizing-logistic-regression)). I suggest initializing your weights to all zeros in the `fit` method before calling `minimize()` and passing in the gradient (`logistic_loss_grad`) to `minimize()` using the `jac` argument, just like we did in Lecture 2.
2. `predict_proba` should just return the predicted probabilities from your model (i.e., the output of the `sigmoid()` function). I've done this for you.
3. `predict` should convert predicted probabilities to 0's and 1's depending on the `threshold`.
4. `score` should return the accuracy of the model, i.e., the accuracy when comparing the output of `predict` to the true class values `y`.
5. I've provided some `assert` statements to help you see if you're on the right track.

This is so cool, I love doing things from scratch and opening up the "black-box", I'm getting all giddy writing this question. If you can do this question, I have full faith that you have a good understanding of logistic regression, and of optimization.

```python
class MyLogisticRegression:

    def __init__(self, method="L-BFGS-B"):
        self.method = method  # optimization method to use in scipy.optimize.minimize

    def fit(self, X, y):
        """Fit the model according to the given training data."""
        pass  # Your solution goes here

    def predict_proba(self, X):
        """Probability estimates for samples in X."""
        return sigmoid(self.w, X)

    def predict(self, X, threshold=0.5):
        """Predict class labels (0 or 1) for samples in X."""
        pass  # Your solution goes here

    def score(self, X, y, threshold=0.5):
        """Accuracy of model on the given test data and labels."""
        pass  # Your solution goes here

```

```python
model = MyLogisticRegression()
model.fit(X_train, y_train)
assert model.w.size == 4, "Wrong number of parameters"
assert (model.predict_proba(X_test) <= 1).all(), "Predicted probabilities exceed 1"
assert set(model.predict(X_test)) == {0, 1}, "Model predicts value other than 0 and 1"
assert np.isclose(model.score(X_train, y_train), 0.93, atol=0.05), "Model score on training data seems to be off..."
assert np.isclose(model.score(X_test, y_test), 0.92, atol=0.05), "Model score on training data seems to be off..."
```

> Do you get a `divide by zero encountered in log` when running the code above and fitting your model? AWESOME! This is caused by underflow, overflow and floating point rounding errors. If you're interested, read on, if you're not, ignore me. The error occurs when we try to take the log of the sigmoid function in the `logistic_loss()` function: `np.log(sigmoid(w, X))` or `np.log(1 - sigmoid(w, X))`. The term `sigmoid(w, X)` may be 0 or 1 if `np.exp(-X @ w)` in the `sigmoid()` function overflows or underflows respectively, and that means we'd be trying to take the log of 0 which gives us the error. If you're interested in learning more, try out the optional exercises 5 and 6 to come up with clever ways of avoiding this error - the tricks you'll implement are used in real Python computing packages! In fact, if you replace the `logistic_loss()` function I defined earlier with the sklearn implementation, you'll avoid this error because sklearn uses the tricks in optional exercises 5 and 6 to avoid underflow/overflow errors:

```python
from sklearn.metrics import log_loss
def logistic_loss(w, X, y):
    y_hat = sigmoid(w, X)
    return log_loss(y, y_hat)
```

>SOOOO COOOOOOOL!!!!!! I love it!

### 4.4
rubric={accuracy:3}

Report the training accuracy and test accuracy of your model along with a confusion matrix for the test data.

```python
# Your solution goes here.
```

### 4.5
rubric={accuracy:2, reasoning:2}

Compare the fitting speed, fitted coefficients and test accuracy of your logistic regression implementation called `MyLogisticRegression` to the sklearn one (`from sklearn.linear_model import LogisticRegression`). For a fair comparison, use the following hyperparameters with scikit-learn's `LogisticRegression`:

- `C=1e8` to (mostly) disable regularization for sklearn, since your implementation doesn't use regularization.
- `fit_intercept=False` since your code above doesn't fit an intercept term.

Is there a significant difference in the fit speed, coefficients or test accuracy between the two models?

```python
# Your solution goes here.
```

### (Optional) 4.6
rubric={reasoning:1}

This question is trickier, I'm getting a little carried away but like I said, I love doing things from scratch so let's throw in some regularization - yewwwwwww!. By default, sklearn's `LogisticRegression` implements regularization - which is typically a good idea. Modify our custom class `MyLogisticRegression` to include L2 regularization. You actually don't need to change the `MyLogisticRegression` class at all for this, we just need to change the loss function and gradient of the loss function!

Here's the log loss function we're already familiar with:

$$f(w)=-\frac{1}{n}\sum_{i=1}^ny_i\log\left(\frac{1}{1 + \exp(-w^Tx_i)}\right) + (1 - y_i)\log\left(1 - \frac{1}{1 + \exp(-w^Tx_i)}\right)$$

And here it is with L2 regularization:

$$f(w)=-\frac{1}{n}\sum_{i=1}^ny_i\log\left(\frac{1}{1 + \exp(-w^Tx_i)}\right) + (1 - y_i)\log\left(1 - \frac{1}{1 + \exp(-w^Tx_i)}\right) + \frac{\lambda}{2n}\sum_{j=1}^dw_j^2$$

Where $d$ is the number of parameters in the model (4 in this case, one for each feature), and $\lambda$ is the regularization parameter. Re-define the functions `logistic_loss()` and `logistic_loss_grad()` to include L2 regularization (you'll need to work out the formular for the gradient of the loss yourself, it's not very difficult). Your model should then give similar results to sklearn's `LogisticRegression()` with the parameter `C=1` - I've included a cell for you to test this out.

```python
def logistic_loss(w, X, y, λ=1):
    pass # your answer goes here

def logistic_loss_grad(w, X, y, λ=1):
    pass # your answer goes here

```

```python
# Test that model coefficients are similar to sklearn
my_model = MyLogisticRegression()
my_model.fit(X_train, y_train)
sk_model = LogisticRegression(C=1, fit_intercept=False)
sk_model.fit(X_train, y_train)

print(f"MyLogisticRegression coefs: {my_model.w}")
print(f"sklearn LogisticRegression coefs: {sk_model.coef_[0]}")
```

## (Optional) Exercise 5: `log(1 + exp(z))`

rubric={accuracy:1}

[In Lecture 1 we discussed computing](https://pages.github.ubc.ca/MDS-2020-21/DSCI_572_sup-learn-2_students/lectures/lecture1_floating-point.html#writing-functions) $\log(1+\exp(z))$ and we wrote a better version of the function for when $z\gg1$ which avoids overflow errors:

```python
def log_1_plus_exp(z):
    return np.log(1 + np.exp(z))

def log_1_plus_exp_safe(z):
    if z > 100:
        return z
    else:
        return np.log(1 + np.exp(z))
```

You might have thought this was not all that important at the time, but as we just saw in Exercise 4.3, even fairly simple equations like the log loss function are at the mercy of underflow and overflow errors. Now let's consider the case of $z\ll -1$, i.e., when $z$ is a large negative number. In that case, we get zero with both of the above implementations:

```python
print(log_1_plus_exp(-100))
```

```python
print(log_1_plus_exp_safe(-100))
```

Your tasks:

1. Investigate why this is happening. Is the problem that $\exp(-100)$ itself underflows?
2. Write a function `log_1_plus_exp_safer` that works well when $z\gg 1$ and $z\ll -1$.
3. For what range of values of $z$ does your `log_1_plus_exp_safer` function give reasonable results?

Your solution goes here.

## (Optional) Exercise 6: softmax logistic regression and log-sum-exp
rubric={reasoning:1}

In the "multinomial" (aka softmax) approach to multi-class logistic regression, your loss ends up having one term for each class, so you get something of the form $\log \sum_{k=1}^c \exp(z_k)$, where $c$ is the number of classes, and $z_k$ is a quantity involving the weights and training data (we'll talk more about softmax later in the course, you don't really need to be familiar with it to answer this question - it's all math and floating point here). For any choice of a constant $a$, we can rewrite this as follows:

$$\begin{align}\log \displaystyle\sum_{k=1}^c \exp(z_k) &= \log \displaystyle\sum_{k=1}^c \exp(z_k-a+a)\\ &= \log \left( \displaystyle\sum_{k=1}^c \exp(z_k-a)\exp(a) \right) \\ &= \log \left( \exp(a)\displaystyle\sum_{k=1}^c \exp(z_k-a)\right) \\ &= \log \left( \exp(a) \right) + \log\displaystyle\sum_{k=1}^c \exp(z_k-a) \\ &= a+ \log \displaystyle\sum_{k=1}^c \exp(z_k-a)\end{align}$$

Note: you only need to look at the first and last expression - the middle parts are only there to convince you they are indeed equivalent.

1. Explain why this final expression might be more numerically stable and why $a=\max \{z_1,z_2,\ldots,z_c\}$ is a sensible choice.
2. If $a=\max \{z_1,z_2,\ldots,z_c\}$, this trick seems to rely on the fact that overflow is more of a danger than underflow, because we may now compute $exp$ of some very large negative values if $a$ is large. Explain why overflow is more of a problem than underflow in this calculation.
3. Write a python function `log_sum_exp` that takes in an array `z` and computes the _original_ expression. Write another function `log_sum_exp_stable` that computes the final (more stable) expression using $a=\max \{z_1,z_2,\ldots,z_c\}$. Discuss the behaviour of the two functions. Also, compare your implementation to In fact, [`scipy.special.logsumexp`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.logsumexp.html#scipy.special.logsumexp) for one or two cases.
4. Earlier, we used the approximation $\log(1+\exp(z))\approx z$ when $z\gg 1$. Is that just a specific case of what we have here? It seems that what we did earlier was an approximation, whereas what we did here is mathematically exact. Do you think there is any significance to this distinction, in practice?

>FYI: you see this trick in real implementations of ML algorithms all the time!

Your solution goes here.

## Submit to Canvas and GitHub

When you are ready to submit your assignment do the following:
1. Run all cells in your notebook to make sure there are no errors by doing `Kernel -> Restart Kernel and Run All Cells...`
2. Save your notebook.
3. Convert your notebook to `.html` format using the `convert_notebook()` function below or by `File -> Export Notebook As... -> Export Notebook to HTML`
4. Run the code `submit()` below to go through an interactive submission process to Canvas.
5. Finally, push all your work to GitHub (including the rendered html file).

```python
# convert_notebook("lab1.ipynb", "html")  # save your notebook, then uncomment and run when you want to convert to html
```

```python
# submit(course_code=59090)  # uncomment and run when ready to submit to Canvas
```


# The Efficient Frontier of Optimal Portfolio Transactions

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) showed that for each value of risk aversion there is a unique optimal execution strategy. The optimal strategy is obtained by minimizing the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

where $E(x)$ is the **Expected Shortfall**, $V(x)$ is the **Variance of the Shortfall**, and $\lambda$ corresponds to the trader’s risk aversion. The expected shortfall and variance of the optimal trading strategy are given by:

In this notebook, we will learn how to visualize and interpret these equations.

# The Expected Shortfall

As we saw in the previous notebook, even if we use the same trading list, we are not guaranteed to always get the same implementation shortfall due to the random fluctuations in the stock price. This is why we had to reframe the problem of finding the optimal strategy in terms of the average implementation shortfall and the variance of the implementation shortfall. We call the average implementation shortfall, the expected shortfall $E(x)$, and the variance of the implementation shortfall $V(x)$. So, whenever we talk about the expected shortfall we are really talking about the average implementation shortfall. Therefore, we can think of the expected shortfall as follows. Given a single trading list, the expected shortfall will be the value of the average implementation shortfall if we were to implement this trade list in the stock market many times.

To see this, in the code below we implement the same trade list on 50,000 trading simulations. We call each trading simulation an episode. Each episode will consist of different random fluctuations in stock price. For each episode we will compute the corresponding implemented shortfall. After all the 50,000 trading simulations have been carried out we calculate the average implementation shortfall and the variance of the implemented shortfalls. We can then compare these values with the values given by the equations for $E(x)$ and $V(x)$ from the Almgren and Chriss model.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the liquidation time
l_time = 60

# Set the number of trades
n_trades = 60

# Set trader's risk aversion
t_risk = 1e-6

# Set the number of episodes to run the simulation
episodes = 100

utils.get_av_std(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, trs = episodes)

# Get the AC Optimal strategy for the given parameters
ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

```python
episodes = 100

ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

AC Optimal Strategy

  Number of Days to Sell All the Shares:	     60	        Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                    4.1	       Expected Shortfall:	                $477,712.60

  Utility:	                               $704,723.22	   Standard Deviation of Shortfall:	   $476,456.32

# Extreme Trading Strategies

Because some investors may be willing to take more risk than others, when looking for the optimal strategy we have to consider a wide range of risk values, ranging from those traders that want to take zero risk to those who want to take as much risk as possible. Let's take a look at these two extreme cases. We will define the **Minimum Variance** strategy as that one followed by a trader that wants to take zero risk and the **Minimum Impact** strategy at that one followed by a trader that wants to take as much risk as possible. Let's take a look at the values of $E(x)$ and $V(x)$ for these extreme trading strategies. The `utils.get_min_param()` uses the above equations for $E(x)$ and $V(x)$, along with the parameters from the trading environment to calculate the expected shortfall and standard deviation (the square root of the variance) for these strategies. We'll start by looking at the Minimum Impact strategy.

```python
import utils

# Get the minimum impact and minimum variance strategies
minimum_impact, minimum_variance = utils.get_min_param()
```

### Minimum Impact Strategy

This trading strategy will be taken by trader that has no regard for risk. In the Almgren and Chriss model this will correspond to having the trader's risk aversion set to $\lambda = 0$. In this case the trader will sell the shares at a constant rate over a long period of time. By doing so, he will minimize market impact, but will be at risk of losing a lot of money due to the large variance. Hence, this strategy will yield the lowest possible expected shortfall and the highest possible variance, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of \$197,000 dollars but has a very big standard deviation of over 3 million dollars.

```python
minimum_impact
```

AC Optimal Strategy for Minimum Impact

  Number of Days to Sell All the Shares:	     250	       Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                1,284,394.9	   Expected Shortfall:	                $197,000.00

  Utility:	                               $197,000.00	   Standard Deviation of Shortfall:	  $3,453,707.55

### Minimum Variance Strategy

This trading strategy will be taken by trader that wants to take zero risk, regardless of transaction costs. In the Almgren and Chriss model this will correspond to having a variance of $V(x) = 0$. In this case, the trader would prefer to sell the all his shares immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. This strategy will yield the smallest possible variance, $V(x) = 0$, and the highest possible expected shortfall, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of over 2.5 million dollars but has a standard deviation equal of zero.

```python
minimum_variance
```

AC Optimal Strategy for Minimum Variance

  Number of Days to Sell All the Shares:	       1	         Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                     0.2	        Expected Shortfall:	               $2,562,500.00

  Utility:	                               $2,562,500.00	   Standard Deviation of Shortfall:	      $0.00

# The Efficient Frontier

The goal of Almgren and Chriss was to find the optimal strategies that lie between these two extremes. In their paper, they showed how to compute the trade list that minimizes the expected shortfall for a wide range of risk values. In their model, Almgren and Chriss used the parameter $\lambda$ to measure a trader's risk-aversion. The value of $\lambda$ tells us how much a trader is willing to penalize the variance of the shortfall, $V(X)$, relative to expected shortfall, $E(X)$. They showed that for each value of $\lambda$ there is a uniquely determined optimal execution strategy. We define the **Efficient Frontier** to be the set of all these optimal trading strategies. That is, the efficient frontier is the set that contains the optimal trading strategy for each value of $\lambda$.

The efficient frontier is often visualized by plotting $(x,y)$ pairs for a wide range of $\lambda$ values, where the $x$-coordinate is given by the equation of the expected shortfall, $E(X)$, and the $y$-coordinate is given by the equation of the variance of the shortfall, $V(X)$. Therefore, for a given a set of parameters, the curve defined by the efficient frontier represents the set of optimal trading strategies that give the lowest expected shortfall for a defined level of risk.

In the code below, we plot the efficient frontier for $\lambda$ values in the range $(10^{-7}, 10^{-4})$, using the default parameters in our trading environment. Each point of the frontier represents a distinct strategy for optimally liquidating the same number of stocks. A risk-averse trader, who wishes to sell quickly to reduce exposure to stock price volatility, despite the trading costs incurred in doing so, will likely choose a value of $\lambda = 10^{-4}$. On the other hand, a trader
who likes risk, who wishes to postpones selling, will likely choose a value of $\lambda = 10^{-7}$. In the code, you can choose a particular value of $\lambda$ to see the expected shortfall and level of variance corresponding to that particular value of trader's risk aversion.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Plot the efficient frontier for the default values. The plot points out the expected shortfall and variance of the
# optimal strategy for the given the trader's risk aversion. Valid range for the trader's risk aversion (1e-7, 1e-4).
utils.plot_efficient_frontier(tr_risk = 1e-6)
```

```python

```


# The Efficient Frontier of Optimal Portfolio Transactions

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) showed that for each value of risk aversion there is a unique optimal execution strategy. The optimal strategy is obtained by minimizing the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

where $E(x)$ is the **Expected Shortfall**, $V(x)$ is the **Variance of the Shortfall**, and $\lambda$ corresponds to the trader’s risk aversion. The expected shortfall and variance of the optimal trading strategy are given by:

In this notebook, we will learn how to visualize and interpret these equations.

# The Expected Shortfall

As we saw in the previous notebook, even if we use the same trading list, we are not guaranteed to always get the same implementation shortfall due to the random fluctuations in the stock price. This is why we had to reframe the problem of finding the optimal strategy in terms of the average implementation shortfall and the variance of the implementation shortfall. We call the average implementation shortfall, the expected shortfall $E(x)$, and the variance of the implementation shortfall $V(x)$. So, whenever we talk about the expected shortfall we are really talking about the average implementation shortfall. Therefore, we can think of the expected shortfall as follows. Given a single trading list, the expected shortfall will be the value of the average implementation shortfall if we were to implement this trade list in the stock market many times.

To see this, in the code below we implement the same trade list on 50,000 trading simulations. We call each trading simulation an episode. Each episode will consist of different random fluctuations in stock price. For each episode we will compute the corresponding implemented shortfall. After all the 50,000 trading simulations have been carried out we calculate the average implementation shortfall and the variance of the implemented shortfalls. We can then compare these values with the values given by the equations for $E(x)$ and $V(x)$ from the Almgren and Chriss model.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the liquidation time
l_time = 60

# Set the number of trades
n_trades = 60

# Set trader's risk aversion
t_risk = 1e-6

# Set the number of episodes to run the simulation
episodes = 100

utils.get_av_std(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, trs = episodes)

# Get the AC Optimal strategy for the given parameters
ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

# Extreme Trading Strategies

Because some investors may be willing to take more risk than others, when looking for the optimal strategy we have to consider a wide range of risk values, ranging from those traders that want to take zero risk to those who want to take as much risk as possible. Let's take a look at these two extreme cases. We will define the **Minimum Variance** strategy as that one followed by a trader that wants to take zero risk and the **Minimum Impact** strategy at that one followed by a trader that wants to take as much risk as possible. Let's take a look at the values of $E(x)$ and $V(x)$ for these extreme trading strategies. The `utils.get_min_param()` uses the above equations for $E(x)$ and $V(x)$, along with the parameters from the trading environment to calculate the expected shortfall and standard deviation (the square root of the variance) for these strategies. We'll start by looking at the Minimum Impact strategy.

```python
import utils

# Get the minimum impact and minimum variance strategies
minimum_impact, minimum_variance = utils.get_min_param()
```

### Minimum Impact Strategy

This trading strategy will be taken by trader that has no regard for risk. In the Almgren and Chriss model this will correspond to having the trader's risk aversion set to $\lambda = 0$. In this case the trader will sell the shares at a constant rate over a long period of time. By doing so, he will minimize market impact, but will be at risk of losing a lot of money due to the large variance. Hence, this strategy will yield the lowest possible expected shortfall and the highest possible variance, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of \$197,000 dollars but has a very big standard deviation of over 3 million dollars.

```python
minimum_impact
```

AC Optimal Strategy for Minimum Impact

  Number of Days to Sell All the Shares:	     250	       Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                1,284,394.9	   Expected Shortfall:	                $197,000.00

  Utility:	                               $197,000.00	   Standard Deviation of Shortfall:	  $3,453,707.55

### Minimum Variance Strategy

This trading strategy will be taken by trader that wants to take zero risk, regardless of transaction costs. In the Almgren and Chriss model this will correspond to having a variance of $V(x) = 0$. In this case, the trader would prefer to sell the all his shares immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. This strategy will yield the smallest possible variance, $V(x) = 0$, and the highest possible expected shortfall, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of over 2.5 million dollars but has a standard deviation equal of zero.

```python
minimum_variance
```

AC Optimal Strategy for Minimum Variance

  Number of Days to Sell All the Shares:	       1	         Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                     0.2	        Expected Shortfall:	               $2,562,500.00

  Utility:	                               $2,562,500.00	   Standard Deviation of Shortfall:	      $0.00

# The Efficient Frontier

The goal of Almgren and Chriss was to find the optimal strategies that lie between these two extremes. In their paper, they showed how to compute the trade list that minimizes the expected shortfall for a wide range of risk values. In their model, Almgren and Chriss used the parameter $\lambda$ to measure a trader's risk-aversion. The value of $\lambda$ tells us how much a trader is willing to penalize the variance of the shortfall, $V(X)$, relative to expected shortfall, $E(X)$. They showed that for each value of $\lambda$ there is a uniquely determined optimal execution strategy. We define the **Efficient Frontier** to be the set of all these optimal trading strategies. That is, the efficient frontier is the set that contains the optimal trading strategy for each value of $\lambda$.

The efficient frontier is often visualized by plotting $(x,y)$ pairs for a wide range of $\lambda$ values, where the $x$-coordinate is given by the equation of the expected shortfall, $E(X)$, and the $y$-coordinate is given by the equation of the variance of the shortfall, $V(X)$. Therefore, for a given a set of parameters, the curve defined by the efficient frontier represents the set of optimal trading strategies that give the lowest expected shortfall for a defined level of risk.

In the code below, we plot the efficient frontier for $\lambda$ values in the range $(10^{-7}, 10^{-4})$, using the default parameters in our trading environment. Each point of the frontier represents a distinct strategy for optimally liquidating the same number of stocks. A risk-averse trader, who wishes to sell quickly to reduce exposure to stock price volatility, despite the trading costs incurred in doing so, will likely choose a value of $\lambda = 10^{-4}$. On the other hand, a trader
who likes risk, who wishes to postpones selling, will likely choose a value of $\lambda = 10^{-7}$. In the code, you can choose a particular value of $\lambda$ to see the expected shortfall and level of variance corresponding to that particular value of trader's risk aversion.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Plot the efficient frontier for the default values. The plot points out the expected shortfall and variance of the
# optimal strategy for the given the trader's risk aversion. Valid range for the trader's risk aversion (1e-7, 1e-4).
utils.plot_efficient_frontier(tr_risk = 1e-5)
```

```python

```